Dependency Management

With most software projects these days, including open source components is almost a given. There is no reason to reinvent the wheel and some components are so customized that it would take months to mimic the behavior. There are many ways to integrate these components into an application. One of my mantras when working on projects is that I just want to be able to checkout the project from a repository and build it; there shouldn’t be several steps and I shouldn’t have to worry about something outside of my control breaking the build.

Years ago when I worked on a particular project, there was a several step process to just get the source code which integrated open source components. This was extremely fragile as the references to the open source components was for an external repository that could go away at any time. When I became in charge of the project, I changed things so that we only relied on repositories that were under the control of the company (we had shared components).

In iOS development, there are now 2 (or 3) main systems for managing these external dependencies. The first is CocoaPods which is very popular, but relies on the external repositories to always be there and requires modifications to how the project is built. The newer entrant into this arena is called Carthage. Carthage gives me more control on the dependencies. The system makes it easy to store the dependencies in my repo and easily update them. In particular, I use the following command to update

carthage update --platform iOS --no-build

Basically I just let Carthage update the components and I have my project setup to do all the builds. When I checkout the project, it has everything in it and I just build. I think that this setup, at least currently, strikes the best balance to handling dependencies.

The other day I was reminded of this problem when a developer was describing dependencies on one of my projects; the developer basically said that he used a dependency management system that didn’t store the components with the source code. I’ve been writing software for awhile now and while open source makes it easier to get things done, many developers don’t consider the entire build process or risks involved in not having control over all the components.

Native vs Web App for IoT Devices

Recently I was chatting with a friend about a new WiFi router. I hadn’t heard of it and he sent me a link to it. The first thing I noticed about it was that the configuration was done via an iOS or Android app. As an iOS developer, I know that a native app is going to generally provide a better user experience than a web app. However, as a consumer, I shy away from devices that only have a native app interface. If the app stops working, isn’t updated quickly when an OS gets updated, or the company stops supporting the app, I’d be out of luck. In addition, I like being able to configure devices using my desktop machine and most devices don’t have a Mac app for configuration.

The native apps are great, but they have to be secondary to a web interface for any IoT device. I mentioned this to my friend and he understood right away my point. I look at the serial to Ethernet gateway I have that I bought used 3.5 years ago and is likely not made any more and am glad that it has a web interface. Granted it is a very specialized device on my network, but the web interface is the only reason that I’m still able to use it. If it were a device that I wanted to look at more often, like a router that I needed to control various aspects of it, the lack of a web interface makes the device a no go in my opinion.

I wish that more companies would implement web interfaces first for their IoT devices and have native apps as secondary interfaces. I’m not saying that all apps should be web apps; in fact, I believe that native apps provide a better user experience. I am saying that web apps should always be a backup option in case the native app isn’t available or doesn’t work.

When is zip not zip?

Like most experienced iOS developers, I use an automated build system. A colleague of mine and I have spent portions of the last 2 years building up our system so what we do looks like magic to others! As part of this system, we’ve written tools and put together scripts to package our application as an .ipa (iPhone application). An .ipa file is simply a zip file with the extension changed.

Well, it isn’t that simple. It appears that how the zip is created is just as important as the structure of the package. There are various flavors of zip, libraries that do zip, and other tools that zip. In one of our tools, we were using a zip library. It appears that Apple made a change in iOS 9.0.2 or 9.1 that caused applications created by our tool to not install on devices. However, the problem was only present if the app was installed over the air or through iTunes; installed through Xcode’s Devices window succeeded. After an arduous day of debugging trying to determine the failure point (provisioning is usually to blame for failures and they can be super frustrating), I switched our tool to use the command line zip (/usr/bin/zip) and amazingly the problem went away.

It would appear that iTunes, iOS itself, and Xcode use slightly different methods for unzipping and installing applications. Since Apple’s xcrun command for packaging (PackageApplication) uses /usr/bin/zip, I think it is a safe bet. It is invoked using something like:

/usr/bin/xcrun -sdk iphoneos PackageApplication -v MyApp.app -o MyApp.ipa" --sign "iPhone Distribution: Scott Gruby"

On a side note, it also appears that there is an error in the PackageApplication script found at:

/Applications/Xcode.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/usr/bin/PackageApplication

that has:

"--resource-rules=$destApp/ResourceRules.plist");

In Mac OS X 10.10 and higher, this line is no longer valid, so if you use this command, you need to modify the script.

The best, underutilized and poorly implemented accessibility feature

[Update – October 21, 2015: It looks like the issues with News have been fixed with the iOS 9.1 update. Yeah!]

iOS 7 brought a feature called dynamic type which moves away from developers specifying exact point sizes for text and instead uses a number of descriptions for fonts.

From UIFontDescriptor.h:

    // Font text styles, semantic descriptions of the intended use for a font returned by +[UIFont preferredFontForTextStyle:]
    UIKIT_EXTERN NSString *const UIFontTextStyleTitle1 NS_AVAILABLE_IOS(9_0);
    UIKIT_EXTERN NSString *const UIFontTextStyleTitle2 NS_AVAILABLE_IOS(9_0);
    UIKIT_EXTERN NSString *const UIFontTextStyleTitle3 NS_AVAILABLE_IOS(9_0);
    UIKIT_EXTERN NSString *const UIFontTextStyleHeadline NS_AVAILABLE_IOS(7_0);
    UIKIT_EXTERN NSString *const UIFontTextStyleSubheadline NS_AVAILABLE_IOS(7_0);
    UIKIT_EXTERN NSString *const UIFontTextStyleBody NS_AVAILABLE_IOS(7_0);
    UIKIT_EXTERN NSString *const UIFontTextStyleCallout NS_AVAILABLE_IOS(9_0);
    UIKIT_EXTERN NSString *const UIFontTextStyleFootnote NS_AVAILABLE_IOS(7_0);
    UIKIT_EXTERN NSString *const UIFontTextStyleCaption1 NS_AVAILABLE_IOS(7_0);
    UIKIT_EXTERN NSString *const UIFontTextStyleCaption2 NS_AVAILABLE_IOS(7_0);

When developers use these instead of say Helvetica Neue 12, a user can change the font size in Settings->Display & Brightness->Text Size.

Text Size

This is generally thought of as an accessibility feature as it helps people who have trouble seeing. However, for people like me who can see well with glasses, larger type is just more comfortable to read. Implementing this is quite easy, but requires a few extra steps like listening for changes to the fonts and making sure that table rows resize to accommodate the text. These steps aren’t rocket science and don’t take much effort, but many developers are constrained by what their designers give them and many designers are still used to specifying exact fonts as well as spacing. This needs to change as it is hurting those that want to increase the font size and also makes it harder to adapt to different screen sizes.

I’ve implemented dynamic type in a few of the apps I’ve done and it worked out well; the extra effort was worth it in my opinion. Some developers just don’t care and other developers including Apple make an attempt, but fall short.

Here are images from the Apple News app. The first image is the standard text size; the second is the largest text size (largest before going into Accessibility and moving it to super large).

News - Standard Size News - Large Text

(I never knew there were images with the posts because they aren’t seen with the large text.) You can see that the text resizes along with the cells, but the title collides with the first part of the article. That’s pretty sloppy.

The next example is in Calendar. This one is worse than the first because the row is a fixed height and it looks like each row of text is also a fixed height so that when a larger font is used, it looks awful.

Calendar - Regular Font Size Calendar - Large Font Size

For a company that pays so much attention to accessibility, these examples show that individual teams making the apps aren’t doing enough to look at their apps. Maybe all the engineers have great eyes and can see the text, but this does need to get fixed. (Filed as Apple Radar 23196322.)

Outside of Apple, developers need to pay more attention to this; accessibility is hard and I’ll be the first to admit that I don’t do enough on accessibility. Handling dynamic type is an easy first step in making apps more accessible and easier for everyone to use.

Determination or insanity?

For the last week I’ve been tracking down a bug in my current project. I’ve spent just about every spare minute including the weekend trying to figure out why my code works in a test app, but not the main app. I rewrote a huge piece of some communications code to no avail. I tried threading the communications, putting in proper locks, changed how I processed the bytes, etc. Basically when I was receiving large amounts of data from a Bluetooth device, some data was lost in the transmission and the checksum was incorrect; for small amounts of data, this problem didn’t show up. At this moment in time, I believe I’m the only person in the world working with this combination of hardware and software which made using Stackoverflow useless.

A co-worker of mine kept sending me ideas to try as I was completely out of ideas. His latest idea was to put the files from my test app into the main app to make sure the project settings weren’t the problem. That didn’t solve the problem, so I started ripping out pieces of the code and came across some code we had put in for a demo.

All of the communications code that I was working on was using the External Accessory protocol over Bluetooth. The demo code we had in there was using Multipeer Connectivity which uses WiFi and Bluetooth to discover peers. Once I removed that code, all of my communication issues went away! Now I should file a Radar issue with Apple on issue, but I can’t reproduce it outside of the particular hardware I’m using, networking library I’ve written, and a few other pieces so a bug report would be quite lacking.

I must have spent 40+ hours on this one issue which has caused me to lose sleep and be a bit on edge all week. As failure was not an option and I had no place to turn, I had to solve this. I’m sure that others would have given up as 40+ hours is far too long to spend on a bug. For me, however, I actually spent 4 months full time chasing down a bug in a network router for a cellular base station; like this one, I did eventually discover the fix.

Lesson learned…be careful of using different communication libraries in an app.

Swift Lessons

As I wrote in my last post, I’ve been learning Swift. This has been interesting. Chris Lattner and his team have been working on Swift for years and have done something that is amazing. Not only have I learned a bit about programming in Swift, I’ve learned a few things about using Swift.

  • Swift isn’t ready for prime time. I’ve seen references to this from Apple.
  • There are lots of bugs in the compiler; SourceKit crashes all the time and auto complete dies.
  • The syntax, while flexible, is a bit confusing at times. For instance, I have a framework in my app called Vera. In some places I access classes of that framework using the Vera namespace, i.e. Vera.Device.Category.Audio to get an enum value and in other places, I use an array of Device. This is confusing and inconsistent.
  • In some places, parameters have names and others they don’t. While you can specifically require the parameter names, sometimes you don’t have to do that and sometimes they’re required.
  • While Swift is extremely flexible, it is confusing as anything once you start getting into wacky things go haywire. Take a look at this code with AnyObject and Unmanaged.
  • There are bugs in the optimization such that turning on optimizations breaks certain code (like the one referenced in the last bullet). I can’t remember the last time that turning on optimizations broke code that I didn’t discover until I tried to deploy an app.

I’m sure that things will get better, but at the moment more work needs to be done. I’ll keep poking with Swift and enhancing my app.

Learning Swift

When Apple announced Swift at WWDC, I knew that if I was going to keep up with the youngsters, I had to learn Swift. Unfortunately it has been a long time since I learned a new language; I learned Objective-C in 2000, I think. While I am capable of learning, finding the time to learn it and use it are getting harder and harder. However, I know that I didn’t have a choice.

I started reading Apple’s books on Swift and made it part way through when I got distracted. I’ve restarted a few times, but still haven’t finished. So I decided to take my automation app and completely rewrite it in Swift. While I could have re-used my existing code, I decided to do my implementation completely in Swift using a few open source libraries (there may have been better libraries to use, but the ones I picked seemed reasonable for now). While I’m still learning Swift, I thought it was a decent attempt to learn it. I’ve put my rewrite on GitHub for all to mock.

I’m going to keep learning and I think that reading the books will make a lot more sense now that I have a project under my belt.

Giving up on Xcode Server

Several weeks ago, I started having problems with my media center where recorded shows started stuttering in playback. There were a huge number of variables that could have caused it as lots of things changed, like an update to EyeTV, update to my HDHomeRun, Xcode update, OS X Server update, etc. While I’m still trying to identify the cause, I think I’ve narrowed down the problem to Xcode and Xcode Server.

Why am I blaming Xcode? Well, I went into OS X Server and looked to disable it. It wasn’t turned on and didn’t have Xcode selected. Since I hadn’t turned it off and had selected a version of Xcode, I knew something went crazy. I selected Xcode again, restarted the Xcode Server and then turned it off. I noticed that even after I did this, there were still log entries related to Xcode Server. Huh? If I disabled it, why was it still doing stuff? I found a blog entry which identified a way to completely reset Xcode Server.

Before I had completely disabled Xcode Server, I had noticed periodic log entries about the server doing stuff which wasn’t a complete surprise as it had to check source code and do other things. However, the status showed a huge number of little spikes in CPU usage. After I disabled Xcode Server, the spikes stopped. When it wasn’t doing builds, it really shouldn’t have done much to warrant the CPU spikes.

I can’t say that I’m going to miss fighting Xcode server; I installed Jenkins and got it configured how I wanted my jobs configured. Jenkins, itself, was easy to setup and do the builds; the tricky part was getting my release notes to TestFlight and to get the archives named the way I wanted. Even though Jenkins is written in Java, I’m not seeing spikes in CPU usage; in fact, when idle, I’m seeing a decrease in CPU usage.

Fun with Xcode Server – Round 2

Earlier this year, I wrote about Xcode Server as well as some issues I had with it. My Xcode Server has been running well for months spitting out builds for testing as well as ones that goto the AppStore.

Unfortunately, Apple changed Xcode Server as part of Xcode 6/Server 3.2.1 (the Xcode Server part of OS X Server primarily is found in Xcode itself as far as I can tell; feel free to correct me if I’m wrong). There were a number of things I had to do in order to get my server building again due to a change in the location of the provisioning profiles and where the key/certificate pair are stored for the profile. These steps are long and not guaranteed to work, but give them a try if you’re feeling lucky.

  1. Switch to root user
    sudo su -
    
  2. Switch to the Xcode Server keychain directory
    cd /Library/Developer/XcodeServer/Keychains
    
  3. Copy the Portal keychain to your Desktop
    cp Portal.keychain /Users/<youruser>/Desktop/
    
  4. Make a backup copy of the keychain.
    cp Portal.keychain Portal.keychain.bu
    
  5. Change the owner of the keychain to your user.
    chown <youruser> /Users/<youruser>/Desktop/Portal.keychain
    
  6. Record the password of the Portal keychain
    cat /Library/Developer/XcodeServer/SharedSecrets/PortalKeychainSharedSecret
    
  7. Leave this terminal window open.

  8. Open Keychain Access as your user.

  9. Add the keychain you just copied to Keychain Access – /Users//Desktop/Portal.keychain

  10. Open a new terminal window

  11. Unlock the Portal keychain

    security unlock-keychain -p \
    <secret you copied from the PortalKeychainSharedSecret File> /Users/<youruser>/Desktop/Portal.keychain
    
  12. Copy the key and certificate used for the provisioning profile to the Portal Keychain in Keychain Access.

  13. Lock the Portal Keychain

  14. Go back to the Terminal window where you are the root user. You’ll still be in the Keychains directory for Xcode Server.

  15. Copy the Portal Keychain back to the directory.

    cp /Users/<youruser>/Desktop/Portal.keychain .
    
  16. Change the owner of the keychain
    chown _xcsbuildd Portal.keychain
    
  17. Switch to the Provisioning Profiles directory
    cd /Library/Developer/XcodeServer/ProvisioningProfiles
    
  18. Copy your provisioning profile.
    cp /Users/<youruser>/Desktop/MyProfile.mobileprovision
    
  19. Edit your bot in Xcode and reset git credentials. I always let the bot create a new SSH key pair and add that to either GitHub or Bitbucket depending on the project.

  20. Cross your fingers and build.

It appears to me that Xcode Server has gotten worse where more functionality has moved to Xcode and less is in the web interface. This means I can’t change the git branch I use for building and I can’t edit my bots using the web interface. I think that there is a lot of potential with Xcode Server, but there is a long way to go in terms of usability. The server must be able to do more than run tests and must be able to let the small development shop do builds that can’t be distributed via TestFlight and uploaded to the app store.

Automation is key in any development project as the manual process is for the birds. Many developers don’t need the power of Jenkins, but could easily use Xcode Server if it worked a little better. Doing builds locally and then uploading for testers is cumbersome and a process that should be avoided. I’d be more than happy to talk to the Xcode Server folks about build practices. At work, I have our Jenkins based build system crank out 13 builds at a time and upload to TestFlight with minimal effort; we just push code to a build branch in git and out pops the builds. We can do several builds a day if need be without blinking an eye. For my personal projects, I was doing a build nightly for someone I was working with and without the automation, he wouldn’t have been able to see the progress as quickly as I would have spent 5-10 minutes a day doing builds which really adds up.

Tweaking a web server with Nginx

A few weeks ago, I started playing with nginx as a replacement for Apache. The exact reasons I decided to do this are a bit unclear, but I think it had something to do with me playing with node.js and MongoDB. The initial setup was pretty smooth and the configuration files are much easier to understand than Apache for my sites. However, I also decided to offload the PHP processing to php-fpm and that’s where the problems began. My VPS is pretty lightweight and doesn’t have a lot of RAM. My blog was running out of memory all the time and php-fpm stopped responding.

After a lot of tweaking to reduce memory usage, I managed to stabilize the system and get much faster response times for my blog; it doesn’t get a lot of traffic, but I think I’ve done a decent job with performance. If I ever get hit with a massive load, I’m sure my server will keel over.

Is there a real reason to use nginx vs Apache? I’m sure some people will say yes, but for me the jury is still out. Things are going smoothly, but was it worth the trouble? I have no idea.