Tag Archives: Programming

Tweaking a web server with Nginx

A few weeks ago, I started playing with nginx as a replacement for Apache. The exact reasons I decided to do this are a bit unclear, but I think it had something to do with me playing with node.js and MongoDB. The initial setup was pretty smooth and the configuration files are much easier to understand than Apache for my sites. However, I also decided to offload the PHP processing to php-fpm and that’s where the problems began. My VPS is pretty lightweight and doesn’t have a lot of RAM. My blog was running out of memory all the time and php-fpm stopped responding.

After a lot of tweaking to reduce memory usage, I managed to stabilize the system and get much faster response times for my blog; it doesn’t get a lot of traffic, but I think I’ve done a decent job with performance. If I ever get hit with a massive load, I’m sure my server will keel over.

Is there a real reason to use nginx vs Apache? I’m sure some people will say yes, but for me the jury is still out. Things are going smoothly, but was it worth the trouble? I have no idea.

Thoughts on Storyboards

When Apple came out with Storyboards in Xcode, I thought that they would be quite useful in making it easier to see how an app was laid out. However, after attempting one project with them, I determined that they had limited utility as they wouldn't scale and wouldn't work well for multi person teams. I left storyboards for a year and didn't revisit them.

I started another project about a year after my first experience and decided to give it another go. This project was a small project and I was the only developer, so it was well suited for storyboards. The project grew and grew. Storyboards worked OK, but the layout in Interface Builder was almost too hard to manage. I finally decided to see about splitting up the storyboards into multiple storyboards.

The app was a tab bar based app, so the logical breakdown was one storyboard per tab. It only took about 15 minutes for me to do this and undo a poor initial decision. However, there were still one or two links between storyboards that were easy to fix in code. Some peopl have come up with ways to link storyboards like RBStoryboardLink which looks interesting, but I haven't explored it.

Multiple storyboards seems scary at first, but it is worth doing to keep difference parts of an app more contained and easier to visualize.

I haven't used XIBs in about a year and I'm not sure there is a good reason to use them as storyboards are easy to work with and make it simple to use custom UITableViewCells and UICollectionViewCells.

Code Signing Woes on Xcode Server

Following up on my post from yesterday, I’ve discovered that Xcode Server does something very strange in the build process. After the build, it creates a .ipa file that is suitable to install on an iOS device and a .xcarchive file that contains the .dSYM file as well as the .app file which is the .ipa file before it has been packaged up. The packaging process is basically putting the .app file in a Payload directory, zipping it up and changing the .zip extension to .ipa. Sounds simple enough and unzipping the .ipa should yield a file identical to the .app. Unfortunately this is not the case. If you run the following command:

codesign --display --entitlements - AppName.app

on the .app in the xcarchive and on the .app that is extracted from the .ipa file, you will get different results.

From the .ipa, I get:

<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">

from the .app, I get:

??qq?<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">

It appears that the code signing entitlement, aps-environment is missing from the .ipa Xcode Server built. I can’t figure out why this is the case, but it does show that I’m not crazy and my setup works. The .ipa I create myself and upload to TestFlight works because my simple packaging script doesn’t re-codesign the .app.

So I have 2 choices when uploading to the app store:
1. Do what I described yesterday and that is resign the app on my desktop machine

  1. Take the .app file out of the xcarchive, zip it up and upload that.

Xcode Server does mangle the product and I’m not sure that is intentional. I should file a bug with Apple to see if they can fix it in the future.

I’d love to be proven wrong on this whole setup. If you have more information, please let me know.

Using Xcode Server for Builds

At last year’s WWDC, Apple introduced Xcode Server for doing continuous integration. Lots of people have tried to use it with mixed results. Recently after dealing with our build server for work (we use Jenkins), I decided to give Xcode server a try for one of my projects.

I was hesitant at first to use OS X Server (which includes Xcode Server) as I’ve had poor results with it in the past. It intrigued me often that I decided to install server and see where it went. OS X server install was a breeze and so was setting up Xcode Server.

I created a “Bot” from Xcode on my development machine and saw that it failed when the server tried to build it. No problem, I searched the Internet and all the results said to add the repository from the server instead of when creating the bot. Perfect, I did that and then the build succeeded. Yeah! I also added my Team to Xcode server so it could pull down provisioning profiles and create a certificate to build.

I checked the log and found 2 problems. First, I have a run script build phase that puts the version number in the Settings.bundle that failed and second, it was being signed using the wrong mobile provisioning profile.

I decided to tackle the second problem first. Through my searches, I found that I needed to put the mobile provisioning profile (an AdHoc one) in /Library/Server/Xcode/Data/ProvisioningProfiles. No problem. Next I also found that I needed to put the certificate and private key for the profile in the System keychain as the Xcode Server runs under _teamserver which doesn’t have its own keychain. I did that as well and found the problem didn’t go away. I started thinking about it and found that sometimes Xcode ignores the Automatic Distribution Code Signing Setting. So, I removed my Team and deleted all the other mobile provisioning profiles. Bingo, now my app was building and being signed correctly.

Next step was to figure out my build phase. After dumping the environment variables, I found that in order to put files in the .app after the build, I had to set my script phase to be:

shortversionnumber=`/usr/libexec/PlistBuddy -c "print CFBundleShortVersionString" ${BUILT_PRODUCTS_DIR}/${INFOPLIST_PATH}`
bundleversionnumber=`/usr/libexec/PlistBuddy -c "print CFBundleVersion" ${BUILT_PRODUCTS_DIR}/${INFOPLIST_PATH}`
shortversionnumber=`echo $shortversionnumber '('$bundleversionnumber')'`

echo ${shortversionnumber}


echo $dest_file

/usr/libexec/PlistBuddy -c "delete :PreferenceSpecifiers" "$dest_file"
/usr/libexec/PlistBuddy -c "add :PreferenceSpecifiers array" "$dest_file"
/usr/libexec/PlistBuddy -c "add :PreferenceSpecifiers:0:Title string About" "$dest_file"
/usr/libexec/PlistBuddy -c "add :PreferenceSpecifiers:0:Type string PSGroupSpecifier" "$dest_file"

/usr/libexec/PlistBuddy -c "add :PreferenceSpecifiers:1:Key string VersionKey" "$dest_file"
/usr/libexec/PlistBuddy -c "add :PreferenceSpecifiers:1:DefaultValue string ${shortversionnumber}" "$dest_file"
/usr/libexec/PlistBuddy -c "add :PreferenceSpecifiers:1:Title string Version" "$dest_file"
/usr/libexec/PlistBuddy -c "add :PreferenceSpecifiers:1:Type string PSTitleValueSpecifier" "$dest_file"

/usr/libexec/PlistBuddy -c "add :PreferenceSpecifiers:2:File string Acknowledgements" "$dest_file"
/usr/libexec/PlistBuddy -c "add :PreferenceSpecifiers:2:Title string Acknowledgements" "$dest_file"
/usr/libexec/PlistBuddy -c "add :PreferenceSpecifiers:2:Type string PSChildPaneSpecifier" "$dest_file"

The key was the ${BUILT_PRODUCTS} variable.

So I was almost done with the build server. I setup 1 bot for the master branch that builds when code changes and another on the build branch that also builds when code changes; however, I only push to build when I’m ready for a build. The build branch also had to upload to TestFlight, so I researched that and came up with very convoluted solutions.

I created a second scheme for the TestFlight upload and set my build branch bot to use that scheme. Then I added a Post Action to the Archive phase that looks like this (make sure you select Provide Build Settings From “Your Target”):

rm -rf ${IPA}
/usr/bin/xcrun -sdk iphoneos PackageApplication -v ${ARCHIVE_PRODUCTS_PATH}${INSTALL_PATH}/${WRAPPER_NAME} -o ${IPA}

RELEASE_NOTES=`cat ${SRCROOT}/CurrentRelease.txt`
echo "*** Uploading ${IPA} to TestFlight ***"
/usr/bin/curl "http://testflightapp.com/api/builds.json" \
-F file=@"${IPA}" \
-F api_token="${API_TOKEN}" \
-F team_token="${TEAM_TOKEN}" \
-F distribution_lists="${DISTRIBUTION_LISTS}" \
-F notify=True \
-F notes="${RELEASE_NOTES}"

echo "TestFlight upload finished!"

I thought I was all done; builds worked and were uploaded to TestFlight. The moment of truth came when it was time to upload to the AppStore. I have push notifications enabled in this app and it is properly provisioning. After the upload, I got email saying that the ape-environment key wasn’t in the entitlements. Very strange, I checked and rechecked the build I got out of Xcode Server and it was correct. I tried a number of things and in the end sort of gave up. I took the archive that the build server created, put it on my main machine, imported into Xcode, exported for AdHoc distribution, resigned it using the same profile it was already signed with and submitted it. For some strange reason, this worked. I can live with that as I don’t submit to the app store all the time.

Things to note

  • You do NOT have to create an AppStore distribution profile. You can submit the AdHoc build to the AppStore without problems. This is key as it lets you test the exact version going to the AppStore without rebuilding or resigning.
  • You have to manually move your mobile provisioning profiles for AdHoc builds to your build server.
  • You have to manually add your private key and certificate to the System Keychain on your build machine.
  • Be careful about your build scripts as the paths are different on Xcode Server.
  • You have to add your repositories from Xcode Server and not from the bot you create.
  • Read the logs carefully when there is a problem.
  • Apple appears to have created Xcode Server for testing and not necessarily for release builds; I hope some of this changes in the future. I’m not a huge fan of unit tests, so I may just be abusing the server by doing what I’m doing.

Overall, I’m pretty plea with having my own continuous integration server. It will catch issues where I have something different from Debug builds to Release builds. In addition, it will upload to TestFlight along with the release notes saving me time. I am a little disappointed about having to resign for the app store, but it isn’t a huge deal. I could probably modify my script to do that during the archive, but I really don’t like having to hard code in the ID of a mobile provisioning profile. I could probably check the profile into source control and reference that, but I’m OK with this solution. As I already want to move the archives to my main machine, the extra signing process isn’t a big deal.

I know that there is a lot here, but hopefully it helps someone else in the future (that person may be me when I forget what I’ve done!).

Flaws in RESTful response codes

Anyone that uses or develops APIs is familiar with REST. It is widely used and by some developers considered superior to other ways of doing APIs. Since REST was developed alongside the HTTP 1.1 spec, HTTP status codes are generally used for responses to REST calls.

To me, it seems that using the standard HTTP response codes (a nice list is given here) is overloading a standard server response with the response of a query. While the spec says that for client errors (400 series errors), the response should contain the reason why in the response. I’ve looked at a lot of APIs and the best ones at handling errors return the error response inside of a successful 200 HTTP response. For instance, if you are creating an API for dealing with support tickets and you ask for a ticket ID that doesn’t exist or you don’t have permission to access, returning a 400 series error generally tells the client that something bad has happened when, in fact, the error is kind of minor. Embedding the error in the response seems like a better way to go.

The problem, however, with embedding the error in the response is that the API developer has to come up with his own error codes and convey that to those that use the APIs. Many developers that implement APIs treat anything but a 200 series response as a complete failure and give up. So, the API developer has to make sure that the client implementors properly handle the responses. In general, I find it easier to have one set of parsing code and deal with the errors inside of a successful response.

My argument, of course, is not very compelling as it is simply to swap one way of handling responses and errors with another. Last night, however, I came up with a potential argument that leads credence to what I’m saying. If there is a proxy server or a load balancer in front of the server, it could potentially return an error code which the client may not expect. Let’s take the case where our support ticket server is behind a proxy server and the client requests a particular ticket. If the proxy server or load balancer is misconfigured and we make a request, the proxy server could return a 400 series error, even a 404 error. The client could interpret this as “the ticket doesn’t exist”. This may not be the case; the ticket could exist, but the proxy returned an error. A better way of handling this would be to use 200 series status codes when the request succeeds and embed the API error code inside of the response. If the proxy server returned a 400 series error, the client could always treat it as an error and wouldn’t have to figure out if the support ticket really existed or not.

In addition, the HTTP status codes are not granular enough for many APIs. This would require the API developer to have a secondary set of error codes that would be returned with the 400 series error codes anyway. If that is the case, why not just create one set of error codes and always return 200 series success codes unless something major has happened?

I know that some of this may be confusing (it is crystal clear in my head), but it is worth some thought to consider the best way to handle response codes in APIs. Many people have opinions on this and may think that I have no idea what I’m talking about and HTTP response codes are perfectly fine for RESTful APIs.

Wrong definition of a pioneer?

The other day I was looking at a website of a contracting company that specializes in mobile development. The site said that they were pioneers in the field. The site then had as its next line that the company was started in 2009. I normally think of pioneers as people who initially settle an area or who are first in a particular field. Unless my history of mobile is a little off, 2009 is a bit late to the party. If we just consider the “modern” era of mobile development that began with the iPhone and Android, they are still late as the iPhone was released in 2007 and the SDK in 2008.

However, saying that mobile development began in 2007/2008 is doing a disservice to all the predecessors to iOS. PalmOS came out in 1996, the Newton came out in 1993, Windows CE came out in 1996 and the Psion was about the same time.

I have no idea what meaning of pioneer this company had in mind, but they are certainly not pioneers in mobile applications. I’ve been writing mobile applications since 1994 when I first got a Newton (the Apple Personal Interactive Electronics group which made the Newton licensed my NotifyMail program and sent me a Newton MessagePad 110 in exchange for the license) and I don’t consider myself a pioneer. I know a number of people that wrote mobile applications prior to this.

If the company considers themselves pioneers, what do they call their developers that have 1-2 years of experience? Experts?

The wrong way to use asserts

One of the ways programmers use to help ensure that inputs to methods are correct is to use asserts. Basically these will test conditions and stop execution if a test fails. By default, Xcode suppresses these asserts in release builds so that the code will simply continue.

This past weekend, I was helping out on a project and found that the app stopped because of an assert indicating that a condition was not satisfied. As I looked into it closer, the programmer had put in asserts to test the results of what came back from the server. The author then didn’t do runtime checks to validate the data, he simply assumed that if the assert passed, that the data was valid and continued. The problem is that, in this case, the server returned data that the code wasn’t expecting, so in a release build, the code would have continued and then crashed because the data wasn’t in the correct format.

While I’m not a huge fan of asserts, they do have their place, just not here. The code should have done runtime checks and gracefully handle bad server data.

I suspect that the author hadn’t done enough client-server communication to know that servers return bad data more often than you’d expect. I’ve worked on so many projects where a simple server change causes clients to have hiccups, that I’ve gotten pretty good at defensively programming against this.

If you choose to use asserts, make sure that you also use runtime checks and gracefully handle conditions that you test for in the asserts as the asserts go away in release builds.