Click here to monitor SSC

Phil Factor's Phrenetic Phoughts

Simple-Talk columnist
The wilder shores of Transact SQL    Phil on Twitter   Phil on SQL Server Central  Phil on BOS

Technical Interviews, and tests, have got to stop!

Published Thursday, January 22, 2009 6:40 PM

‘Technical Interviews’ have got to stop. They are a disgrace to the IT profession.  Two MVPs who I asked the question ‘Have you ever passed a technical interview’ have admitted ‘Never’.  I’d like more successful developers to confess their inability to remember much more than their name under the pressure of a technical interview.  The most extreme geeks all have brains that blue-screen with a temporary aphasia under  stress.

I have a new proposal to make. Instead of employing developers after asking them a series of trivia based from trawling obscure facts in MSDN, you should see how good they are at Table Tennis, Table footie, or guitar. I might include pool/Billiards too. 

I intend to justify this apparently ludicrous assertion

There are many theories of what makes a good developer. In the 1960s, when the shortage of people with IT skills first became apparent, people were selected on their mathematical skills. This proved to be disastrous, and IT departments soon filled with strange geeky folk devoid of social skills, or any understanding of real business applications. This puzzled the recruiters who consulted the psychologists. The Psychologists pondered over this and, in the fashion of the time, came up with a set of problem-solving tests based on Hans Eysenck’s intelligence tests.  In Britain, these seemed so obviously appropriate that they even appeared on adverts ‘If you can solve this puzzle in five minutes, call xxxxx’.  Subsequent exercises in validation showed that they were no more successful than chance in successfully predicting a good IT programmer. In fact, it just told you that the person was good at puzzles; it also suggested that they didn’t get out enough in the evenings. The psychologists then took the more sensible approach of testing real successful IT programmers, and seeing what their skill-set consisted of. It turned out that good programmers were very articulate.  Otherwise, they had no obvious dominant skill in common.

Selecting programmers who were good at language skills just didn’t look right. It had no ‘Face Validation’, and it didn’t impress the brash new profession of ‘HR’. They wanted tests like they had at school. Where there is a demand, there are always people prepared to supply it, and soon a lucrative market in Technical tests developed.  The validation of any psychometric test of ability is a highly technical subject which I won’t bore you with.  Please just remember that, because these many tests of technical competence have never been scientifically validated, they are no more effective in selecting the good candidates than testing to see if the applicant can sing ‘Somewhere over the rainbow’ in a high voice.  The classic Microsoft interview questions, such as ‘Why are manhole covers round?’ shouldn’t be used until you can prove that successful programmers believe (wrongly, it turns out) that it is to stop the covers dropping into the hole, more frequently than do the duffers.

Besides fluency with spoken language, there is another essential component, a personality trait that is necessary for a good developer. This is a cussed stubbornness that won’t let go of a problem until he or she has solved it.  It is that will to go through pain, boredom, hunger, and lack of sleep in pursuit of doing something that is almost impossible…. Such as playing guitar properly, solving advanced physics and mathematical problems, playing badminton or table Tennis well. 

The best programmers I’ve ever met are astonishingly alike in this respect. If they can’t get something in their lives to work as well as they want, it turns into a man-machine struggle of epic proportions. If I give someone a programming puzzle to solve in an interview, the good programmers don’t necessarily know the answer; the good programmers refuse to finish the interview until they’ve got the answer right. It suddenly rears up in their minds as being even more important than the interview. The whole objective of getting a job is temporarily put to one side in pursuit of a solution.

The same personality, if she, or he, finds himself unable to play table-tennis properly, or play ‘tiptoe through the tulips’ on the ukulele, or beat the regulars at Pool, will bloody practice until she, or he, does or renders himself or herself incapacitated in the attempt.

If you think that this is an absurd theory, you will be surprised to hear that the standard American test of mathematics, which is prefaced by a rather long-winded form for name, address etc. , has an almost perfect correlation between the Mathematics score and the ability to fill in the preliminary form accurately. If the applicant has the mental stamina and obstinacy to apply to fill in the preliminary form you can be certain that he or she will be good at Maths.  That same cussedness turns out a good IT programmer, or a good table-tennis player, or any skill that required dogged determination.

So, if I ever have to interview you for a technical development job, have no fear of being taxed with knowledge of obscure parameters to DBCC functions you would never dream of using, or obscure behavior of Transaction rollback in linked servers; No, beyond establishing a sound ability to understand questions and solve general programming problems, I shall be there to challenge you to a game of table football, croquet,  pool,  or any other skill  that can only be achieved by bone-headed  stubbornness and practice beyond the patience of an ordinary mortal.

Comments

 

marmot4 said:

I completely agree.  And only partially because I'm one of those guys who has the blue screening brain problem...

I think you could add two video games to your list as well, Lemmings and World of Goo.  Both have a tendency to bring out that almost obsessive compulsive behavior.
January 22, 2009 1:08 PM
 

Jack the DBA said:

Hey, I hope my next job interview is with you.  I can play table-tennis all day.

I agree to a point, but you do need to ask some technical questions.  I have a pretty basic answer for most technical questions, "I look it up in (BOL, google, etc...)".
January 22, 2009 2:32 PM
 

Phil Factor said:

Yes, I agree with Jack the DBA. My views haven't changed from when I wrote my 'Database Mole' article http://www.simple-talk.com/opinion/opinion-pieces/the-database-mole/ in 1985. You need to check out how they solve problems, but you also need to check out their attitude to solving problems and that they possess a  bloody-minded persistence in solving problems
January 22, 2009 4:45 PM
 

bradmcgehee said:

I agree with Phil. Technical interviews don't really separate the Exceptional DBA from the DBA who memorizes arcane facts and syntax. People skills, leadership skills, a philosophy of life-long learning, general problem solving skills, honesty, persistence, and other "soft" skills are just as important, if not more important, and also are much harder to identify in an interview. This is very much like technical certification. Just because a DBA can get certified doesn't mean he or she is a good DBA. There's a lot more to it.
January 23, 2009 1:10 AM
 

Andrew Gothard said:

Well, the validity of the technical interview in the selection process relies heavily on the interviewer actually knowing the correct answers to the questions in the first place.  This is not always the case
January 23, 2009 5:33 AM
 

mjswart said:

January 23, 2009 9:29 AM
 

efreedom said:

I certainly agree that dogged determination would be an important factor.

Two concerns though....

    1- Confusing competitive with tenacity
    2- Interest in solving the particular problem

I'll play table tennis with you, but I won't won't be upset if I lose and think that we have to play until I win.  It's a fun game, I like to win, but I have no desire to spend many hours trying to improve my game just so I can best any particular person.

There are many problems that I will dig and dig until I get an answer. However, I am quite capable of walking away from an incomplete puzzle because I have no interest in puzzles.

The things that drive me nuts are people who will not quit until they win, even when it's a game of chance. They are stubborn and honestly, no fun to play with because it's never over until they win.

Bob



January 25, 2009 12:43 AM
 

StarNamer said:

Similar arguments apply to a lot of technical certifications. Just because someone has passed an exam often says nothing about whether they can apply the knowledge in the real world, it just means they know how to answer exam questions! These days a major skill in many roles is framing a good search query on Google, but no-one ever asks for that at interview...
January 26, 2009 8:44 AM
 

Rowland said:

I'll take the opposing view and say tech interviews aren't all bad in seperating the wannabe's from the real DBAs. We seem to have a rather large crop of folks who claim to have 20 years of experience in just 2 years of verifiable documented work history.

While I agree that they're not always the best way to understand what level someone is at tech interviews can cut through the clutter pretty quickly. I recently interviewed a CompSci pHD who couldn't tell me what a constraint is! Another who couldn't even give one common data type and so on.
January 27, 2009 7:17 AM
 

timothyawiseman@gmail.com said:

There are definitely problems with technical interviews, but I have not yet seen anyone propose a better alternative.  

Now, I think it would be unwise to hire someone based solely on a a technical interview, but I do think they are a valuable part of a "whole person evaluation".  If someone wants to be a senior SQL developer, then I think it is quite reasonable to expect them to know how to do a join.  I have seriously had interviewees tell me they were senior, experienced SQL developers and then not have any clue how to do a join at all.  

Obviously, someone can, to borrow a quote from this thread, memorize arcane facts and pass the technical questions in the interview and be a horrible employee (or vice versa for that matter).  But it does help establish a minimum baseline, and that minimum baseline can be important, especially with dozens of resumes and 6-7 scheduled interviews to fill one position.
February 27, 2009 11:56 PM
 

Matt Birchall said:

Technical interviews have their place but should not be the sole criterion for selection. I agree that some stuff can be safely remembered in documentation rather than cluttering up your mind. On the other hand, an ability to set down ideas on paper is also useful.
These tests are, however, no good at elucidating judgement (aka wisdom) which is an essential facet of a candidate.
I've just been ruled out of a role by one of these tests and wish that I had had an opportunity to demonstrate the competencies that I do have (judgement, tenacity, ability to use the tools etc) whereas I was only able to show that I couldn't pass the test. I've passed other tests, and also set up interviews that included technical questions, and can see their place in the selection process but they shouldn't be a crucial hurdle for the candidate.
P.S. Datawarehouse analyst/developer still available for hire!
March 5, 2009 2:04 AM
 

puzsol said:

I worked with a guy who had his masters in Computer Science - so he obviously knew how to pass exams - and most likely technical interviews - especially the sort that look like an exam paper (ask some obscure fact, get the answer, or complete this statement etc)... but he would often come to me with the simplest questions... he had all this learning but no-one had taught him to think, or how to apply all the knowledge to the problem domain... that is the harder thing to test in an interview, because it comes more from character than text books.  I'm with Phil here, I would much rather work with someone who was willing to tackle a problem (even one out of their depth), than someone who just does what they are told and nothing more.  

Though I will add this observation... sometimes that tendency can be a bad thing... ever wonder why the industry keeps re-inventing iteslf?  Its because that same mentality (of: I can/will do it myself) that makes a good programmer, also hinders us from using/adopting things that other people have done... ie spend two weeks re-creating a control that you can buy for less than $500, or create a new language when Cobol works just fine... Perhaps that's why the ones that are not totally obsessed with programming into the small hours of the morning end up being more cost-effective?

PS, I'm up for a game, table-tennis, billiards, etc any time!
March 19, 2009 5:51 PM
You need to sign in to comment on this blog
<January 2009>
SuMoTuWeThFrSa
28293031123
45678910
11121314151617
18192021222324
25262728293031
1234567
How to Kill a Company in One Step or Save it in Three
 The majority of companies that suffer a major data loss subsequently go out of business. Wesley David... Read more...

Migrating from OCS 2007 R2 to Lync: Part 4
 Having migrated the rest of our users and legacy resources across and started getting ready to... Read more...

Automated Script-generation with Powershell and SMO
 In the first of a series of articles on automating the process of building, modifying and copying SQL... Read more...

Seth Godin: Big in the IT Business
 Seth Godin has transformed our understanding of marketing in IT. He invented the concept of 'permission... Read more...

Using SQL Test Database Unit Testing with TeamCity Continuous Integration
 With database applications, the process of test and integration can be frustratingly slow because so... Read more...