I periodically have to interview people and I find that I struggle to come up with ways to adequately determine a candidate’s technical knowledge. While they can present their resume and what they’ve done in the past, it is hard to tell how they really think. I don’t ask questions that I couldn’t answer myself as I don’t think that is fair. Many people ask basic computer science questions that I’d probably get wrong as I don’t have a computer science background; so I tend to ask questions to see how a candidate would handle a situation.
So far I’ve had mixed results in weeding out the good from the bad. One skill that I think is of utmost important for a developer is debugging. You might be saying that all developers do that, but some are far better at it than others. Debugging is one of my best skills and I’ve had a ton of experience at it inheriting other people’s projects. With that in mind, I came up with one technical test that could actually tell me if a candidate could debug and that skill goes a long way in development.
Now you ask, what is the question? The other day as I was poking through crash logs, a few stood out at me. Take a look at the following logs and see if you can spot why the apps crashed:
Thread 0 name: Dispatch queue: com.apple.main-thread Thread 0: 0 libsystem_kernel.dylib 0x35cab054 semaphore_wait_trap + 8 1 libdispatch.dylib 0x342961c0 _dispatch_semaphore_wait_slow + 184 2 libdispatch.dylib 0x342961f4 dispatch_semaphore_wait$VARIANT$mp + 32 3 libxpc.dylib 0x3200e89a xpc_connection_send_message_with_reply_sync + 206 4 SystemConfiguration 0x374f5be6 _reach_server_target_status + 938 5 SystemConfiguration 0x374f6d56 __SCNetworkReachabilityServer_targetStatus + 14 6 SystemConfiguration 0x374dfaee __SCNetworkReachabilityGetFlags + 198 7 SystemConfiguration 0x374e0f7a SCNetworkReachabilityGetFlags + 190 8 MSNBC 0x000cb9ec 0x1000 + 829932 9 MSNBC 0x0006f998 0x1000 + 453016 10 MSNBC 0x0006abfa 0x1000 + 433146 11 MSNBC 0x00014d54 0x1000 + 81236 12 MSNBC 0x0006ab6e 0x1000 + 433006
Last Exception Backtrace: 0 CoreFoundation 0x3567188f __exceptionPreprocess + 163 1 libobjc.A.dylib 0x37a18259 objc_exception_throw + 33 2 CoreFoundation 0x356713b3 __NSFastEnumerationMutationHandler + 163 3 EmSea 0x001f9c2b 0xe2000 + 1145899 4 EmSea 0x00199bed 0xe2000 + 752621 5 EmSea 0x00223453 0xe2000 + 1315923 6 EmSea 0x002239c9 0xe2000 + 1317321 7 Foundation 0x351b6c29 __65-[NSURLConnectionInternal _withConnectionAndDelegate:onlyActive:]_block_invoke_0 + 17 8 Foundation 0x3510e6d9 -[NSURLConnectionInternalConnection invokeForDelegate:] + 29 9 Foundation 0x3510e6a3 -[NSURLConnectionInternal _withConnectionAndDelegate:onlyActive:] + 199 10 Foundation 0x3510e5c5 -[NSURLConnectionInternal _withActiveConnectionAndDelegate:] + 61
On the first one, the hint is that the app failed to launch in time. I’m going to use these crash logs as a test to see if a candidate can tell me what went wrong. Neither app is mine and while I can’t see the code, I can immediately spot the problems.
In the first crash log, the app calls SCNetworkReachabilityGetFlags on the main thread. This call is a blocking call which means it won’t return until it is done; this call can take a significant amount of time to return and should never be called on the main thread, especially at app startup.
In the second crash log, it isn’t as obvious. What the developer has done is something like:
for (NSString *string in someMutableArray)
{
if ([string isEqualToString:@"Yuck"])
{
[someMutableArray removeObject:string];
}
}
The array is getting modified while it is being enumerated. The fix is pretty simple.
NSMutableArray *deleteArray = [NSMutableArray array];
for (NSString *string in someMutableArray)
{
if ([string isEqualToString:@"Yuck"])
{
[deleteArray addObject:string];
}
}
[someMutableArray removeObjectsInArray:deleteArray];
Are you good at reading crash logs?
I knew both answers, but only because I’ve done each once.
Hey, that’s cheating!
Seems like a great exercise for an interview. I would suggest mixing in a crash with a dangling weak reference (delegate, etc.). Seems to be the one of the most common crashes in iOS apps.
Sounds like a good idea; along the lines of a delegate, something about retain cycles could also be interesting.
Hmmm, I would probably agree with you about doing such tests on other platforms and development languages, but for Objective C? I’ve been a programmer for nearly 20 years, mainly with dotNet/C# from the very first version, and I don’t know if it’s because we’re spoilt by Microsoft but crash logs are MUCH more readable and give you much better information about the problem. I just haven’t managed to get good at working out what’s going on with iOS crash logs. Sure, some crash logs are obvious, especially when you spot one of your own methods in there, but others are just plain cryptic. Maybe I’ve not been messing with ObjC enough (been doing it for just over 18 months now), but some crash reports still drive me a bit nuts.
Here’s an example that’s driving me nuts (and actually brought me to your blog whilst Google’ing for possible causes):
Exception Type: SIGSEGV
Exception Codes: SEGV_ACCERR at 0xb2224cc5
Crashed Thread: 0
Thread 0 Crashed:
0 libobjc.A.dylib 0x393a65b0 objc_msgSend + 16
1 Foundation 0x31e3b6fd __65-[NSURLConnectionInternal _withConnectionAndDelegate:onlyActive:]_block_invoke_0 + 17
2 Foundation 0x31d7b1f9 -[NSURLConnectionInternal _withConnectionAndDelegate:onlyActive:] + 201
3 Foundation 0x31d7b115 -[NSURLConnectionInternal _withActiveConnectionAndDelegate:] + 61
4 CFNetwork 0x311dd45f ___delegate_didFinishLoading_block_invoke_0 + 27
5 CFNetwork 0x311dcb43 ___withDelegateAsync_block_invoke_0 + 55
6 CFNetwork 0x31204fcb ___performAsync_block_invoke_068 + 19
7 CoreFoundation 0x3144674d _CFArrayApplyFunction 177
8 CFNetwork 0x3120542b RunloopBlockContext::perform() + 75
9 CFNetwork 0x3116903d MultiplexerSource::perform() + 189
10 CoreFoundation 0x314d5683 15
11 CoreFoundation 0x314d4f7f 363
12 CoreFoundation 0x314d3cb7 647
13 CoreFoundation 0x31446ebd _CFRunLoopRunSpecific 357
14 CoreFoundation 0x31446d49 _CFRunLoopRunInMode 105
15 GraphicsServices 0x350122eb _GSEventRunModal 75
16 UIKit 0x3335c301 _UIApplicationMain 1121
17 MyApp 0x0019a77b main (main.m:34)
None of the other threads show anything that could remotely tell us where the problem could have occurred, and it’s only the additional info that a user sent in that has actually given us a clue as to where the problem could be (and we’re still stuck!).
Anyway, out of interest, when you test someone with such crash logs, do you allow them to Google some of the internal classes/calls? For example, the Apple docs for SCNetworkReachabilityGetFlags don’t seem to mention that it’s a blocking call, so unless the developer has come across this, it may not be as obvious as it was to you.
Your crash log seems to indicate that it’s crashing when NSURLConnection is calling the delegate or executing a block (depending on if you’re using the async variety with a completion handler). I’d check where you call NSURLConnection and carefully examine the code that is called when it completes.
As for allowing people to Google for responses, that would be too easy and wouldn’t tell me about their experience. Any developer that has done networking code, has come across Apple’s reachability sample code and read the read me knows that the readability calls block.
Of course, you probably have a different opinion on interview techniques.
Hi Scott,
That’s the bizarre thing (about my crash report), as NSURLConnection is not called at the point when the user says they experience this problem, and certainly not on the main thread. Also, if we can’t replicate this on any of our local devices, it shows that it’s not a bad call (ie a call to a delegate that doesn’t exist, or doesn’t implement a given method), and something that varies based on certain conditions. Which brings me on to 3 points why I don’t think that testing people based on their reading of crash reports is a good idea:
1) I have seen similar problems where certain conditions cause weird crashes. Eg, what’s this all about? -[UIWebView cut:]: unrecognized selector sent to instance – I’m convinced that memory management under ARC does some VERY weird things with classes (where an instance is still “alive”, but for some reason SOME of its properties and/or methods just seem to “disappear”), but I’ve not been able to nail it down yet
2) Even Apple support staff can’t explain some crash reports. I’ve got at least 2 tickets open with Apple where they too couldn’t decypher crash reports
3) I too have done plenty on networking (our app relies heavily on networking), and have tried the Apple reachability class, but instead opted to roll my own code precisely because the reachability class didn’t fully do what I was hoping for (and certainly not in a nice way, either). But if you were interviewing *me*, you certainly wouldn’t find that out by testing me with the above crash report. My point is that, even if you’re explicitly looking for people with skills in networking/connectivity, you may still let the right candidate slip through your net.
Don’t get me wrong, Scott. I commend you for trying to find good ways to weed out the right candidates when interviewing – It’s a tough job to interview and find the right people from a steady stream of “possibles”. And having read a few other posts on your blog, I generally agree fully with your thinking. Also, your test is a LOT better than “written exams”. But I just don’t think ObjC (or at least iOS) crash reports are a good way to test candidates. Heck, even Apple staff would fail! 🙂 So far, the best way that I have found to test people (admittedly for dotNet and SQL Server) is to get them to fix some broken code/projects, or optimise some procedures. This shows how good they are at fault finding and tweaking with actual code, how well they can get around Xcode and other tools, and whether they think the way that you want them to.
With regards to stopping people from Google’ing, I think this is a bad idea too. In my experience, those candidates that are great with theory (ie they know syntax and theory from the top of their head without having to look it up anywhere) make the WORST developers. It depends on what they need to look up. As I said earlier, I wouldn’t have automatically known what SCNetworkReachabilityGetFlags is, and I’ve even looked at the Reachability class. And I know from personal experience that Stack Overflow has proved FAR more useful over time than any Microsoft, Apple or any other official documentation. 🙂
Sorry, it’s just my 2p’s worth…