So I decided to make a short quiz. Just a few 1 sentence answers required. Some were multiple choice. Some were read these 3-5 lines of code and tell me what gets printed (really simple, straight forward lines -- if you could read the language you should be able to answer them in about 10 seconds each). Nobody was passing my quiz, the CEO told me to simplify it. This happened a few times. Eventually I only asked people to write code to print the first 100 digits of the Fibbonacci sequence. I wrote out a description of how to calculate it (in english) as well as gave the first 10 numbers as an example. I had about a 10% pass rate and was told I was doing a poor job interviewing because my tests were too difficult and I had to use a 3rd party testing service instead of my quiz.
The truth is that finding a "good" programmer is REALLY hard. A good programmer to me is someone who:
1) Is willing to follow established company guidelines on source control and code structure even if it doesn't agree with what they've done in the past (although I'm happy to discuss changing them if the logic is sound).
2) Someone who is an abstract problem solver. Stackoverflow is a great resource to how to do things, but not necessarily the why or when you would use them. A good programmer would know when to use certain in-memory data structures as opposed to writing to disk -- when to write it to disk, etc. More importantly, when to optimize code and when not to (and where to optimize -- low hanging fruit). Personally, I don't care if you don't remember the syntax of writing a bubble sort, but I'll be extremely impressed if you can tell me when a bubble sort might not be as good as insertion sort or quick sort, etc.
3) Dependable -- can pull the hours when a big project needs them, etc. Frankly, I don't care if they work from home or from a strip club as long as they can excel in whatever environment works for them.
4) Leadership qualities -- the best programmers aren't just programmers, but can understand what a company is trying to do on a high level and accept a lot of that risk/reward themselves in what they do by doing whatever is necessary to give that company the competitive edge.
That brings me to what a lot of company's do wrong. Your boss thought your interview was bad, but honestly, if a programmer can't figure out how to write the first 100 digits of the Fibonacci sequence, I'm not going to be really thrilled. It's really hard to find good talent -- and that's why the good talent can demand six figure incomes.
If you find someone good, treat them like an adult. Good programmers are a rare breed. I like to work late at night listening to music -- so I'd appreciate some time away from the office to do that. A company would respect good talent and give them a lot of flexibility (Time off, remote options, working at the coffee shop down the street, etc.) Yes, I understand some meetings are mandatory and I'm happy to be there for them, if you're happy to let me do my think if I bring you results. I don't think a lot of company's understand this about programmers.
Honestly, if you are trying to find a "good programmer," you might have to phone screen 100+ people. Think about that for a moment -- that's a huge time commitment just for phone screens if each runs 10 minutes. That's over 15 hours and no one has even walked into the door for the bigger interview. My phone interview test consisted of "categories" of questions with each category having two questions. If you could at least get one question right, I'd move to the next harder category. Most of the candidates failed at the first category with questions like (I shit you not), "Tell me ONE difference between Linux and Windows." The *hardest* category would have questions like, "Which data structure would you use to retrieve a value in O(1) time -- a hash or an array?" If they made it to that level, they were probably coming in for an in-person interview at that point.
