How to check or assess debugging skills of a person?

Question

What kind of skills determine a person that is capable of debugging code with ease? Some time ago my friend carried out an interview with a relatively good programmer. The programmer got hired. He could write good code, understood the frameworks and design patterns. The thing he was missing was - debugging skills. He could not debug at all and finding problems with his or someone else's code was huge pain for him.

Since then we are thinking about how we can assess or estimate a person's debugging skills.

So the first question is: what skills determine if a person can effectively debug software?

And the second: how to test those skills during the interview?

Answer 1

If the first thing the person wants to do is look at the code and step through it with a debugger that person is not a great troubleshooter.

If you don't already have a plan of action and you dive into the debugger blind you are basically Easter Egging. This is true for ANY kind of troubleshooting.

In an interview situation a person that asks how the system operates and asks about history of the system would be somebody that might be a good troubleshooter. A person that thinks system first and mechanics second could be a good troubleshooter.

This is true of any complex system.

Answer 2

I would argue that the best measure of a good software developer in a particular language or framework is the ability to critically analyze complex problems and to have good debugging skills in the language or framework. They must be able to demonstrate low level debugging as well as proficiency in high level debugging with common debugging tools.

This means creating a scenario for them that demonstrates high aptitude of debugging tools in their chosen IDE. You should look for things like:

• Running sandboxed application or server in debug mode or building application with symbols for debugging

• Making available and demonstrating remote debugging ports or debugging of non-sandboxed application that was built with symbols (if applicable to language)

• Strategic use of breakpoints

• Custom properties of breakpoints, conditional expressions on breakpoints (if applicable to language)

• Use of expressions or variable watches for monitoring variable values or references

• Use of ad-hoc variable value or reference or pointer manipulation in real time

• Demonstrate ability to step into, over and out of application flow

• Critical evaluation of call stack

• Debugging multi-threaded applications and comprehending this.

Other debugging strategies without tools should be demonstrated as well, such as analyzing logs and source code, as well as ability to perform some low level debugging without the use of an IDE as well.

Answer 3

I'd say distill a bug that you had in your system to something that can be discussed in the context of an interview. Fire up the debugger and let him at it.

Answer 4

Ask him questions like this:

1. How do you tackle a problem?

2. What is one of the complex project that you did and how did you achieve it?

3. Which debugging tools did you use?

4. Do you have any preference for certain tools?

5. Give an example of your own scenario and ask him how he will tackle it?

6. How would you rate your ability to get into someone else code?

You can address your concerns by asking questions. There is always a risk he may or may not be good in certain skills. But if he is a good learner, that will help a lot.

Answer 5

If you want to see if a programmer can debug, give them code to fix. It's the same approach if you want to see if they can write code. Give them a problem and have them write code.

Now, I'm confused about this programmer who has no problems writing code but fails misserably when asked to debug. Does this person regurgitate code examples or just sticks to areas he has experience like reading and writing to a database? Unless they get the code right the first time, they can't fix it?

Maybe the person just doesn't like debugging and makes no effort? I'm no good at this so stop asking me to do it - learned helplessness.

Working on an existing code base requires looking through code, documention and possibly making some notes and desgins of your own.

I know we think of debugging as fixing production code that has failed, but I need to debug code while I'm writing it. Either this person isn't a very good programmer or they just prefer to write new code. Don't we all.

Answer 6

In the same way you would determine someone's coding ability, ask them questions about debugging.

Ask them "how" they would track down a bug in a given situation.

Take it a step further and sit them down in front of a computer and watch them debug an issue.

Answer 7

I've often given candidates hypothetical situations... for example, a production system has stopped responding. What do you do? They might answer "check the logs" and I say "the logs show nothing abnormal, except that there is nothing written in them since the problem started happening". And so it continues, until I'm satisfied that I've assessed the candidates ability to problem solve.

Answer 8

Usually people with good aptitude are also the one with good debugging skills.

During the interview, (depending on their seniority) you can give them an assignment of the like of puzzle such as some algorithm or so. That's the simple way.

If you can, you can print a code out of some work ask the person if something is wrong here and if so how to fix it.

I don't quite prefer to ask obfuscated interview questions which tends to focus on people's ability to read and fix syntax.

Answer 9

During an interview, ask them to tell you about a bug that they fixed in the past and the steps that they used in debugging it.

Make them tell you about the what they have done in their last job, homework assignment, etc. And what they went through in finding the problem.

Answer 10

I will share a experience together with a recruits perspective about test of a candidate skills in debugging. I onced got on a interview that had three stages. The second stage was a "practical case". I didn't know more at that moment. While there I was informed there are a system that stopped working and they don't know. Some bugs is lying behind.

It was arranged as a remote desktop to a old testing environment. Probably to a unplugged or isolated environment. The project was a few webforms with some ASP.NET controls and related Code-file code. The codefile referred to a kind of business layer for which i just have a dll, no source code and method descriptions. The Webforms did the CRUD functions you can expect. Also a small search function. The business layer, in turn, talked to Views and SP in a sql server.

They broked some parts at different levels. I was given a paper with symptoms. "Can't search" "The 'region' field disappeared after last update" and such. Such as you can receive from your users.

I don't remember all details but at least a table field got renamed, which lead to a broken SP, which was used by the search function. That means no error in VS and no BL source code to trace field names. A SELECT parameter against Sqlcommand was misspelled and caused a webform to malfunction. Also a field was omitted which was the missing field in GridView (Autogeneratecolumns). A ASP.NET Button was referred to something who must meant to be a duplicated, enhanced, method and "forgot" to point button to new method.

Also such minor thing using title in a html tag that don't allow it. Also opposite ALT tag was omitted in a control that required it. There was also some errors with uncorrect closed html tags but which did not malfunction. Not sure if all those was a pure playhouse-project-error or perhaps same project for different recruitments. I never asked. The level of difficulty should of course match the need of the recruit.

Such test should probably be screened (not followed) to see, after interview, how debugging was done. For myself at that stage, I found the test a little ridiculous, but that's would also be the big point. If it was or wasn't, should worth a lot have the candidate in right place.

*I think that test was proved the candidates/my skills to *
* Analyze a foreign system
* Use a minimal of information to find errors and bug
* Under time stress and without someone help you, code assumed corrections
* Different levels of knowledge;
** sql db and stored procedures,
** use of dll in project,
** asp.net technique,
** layered architecture
** problem-oriented aspect

But also the more obvious things like handle the developer environment, find and understand Db Server Management tool. Surely there are candidates that looks really nice on paper but, in practice, could stuck on such tasks.

Answer 11

I pick an actual problem I encountered that's relevant to the position and I present it to the candidate much as it was presented to me. Of course I offer them some general background, and a small amount of relevant documentation like a code snippet or a schematic diagram.

I tell them their job is to solve the problem and I offer to answer any technical questions they have and tell them the outcome of any experiments they want to perform. If they say, "I'd put a scope probe here," then I will sketch them a trace of what they might find. If they want to insert a printf in a loop I will tell them that it never comes out (!) or first prints "7" and then repeatedly "5". If they get so far off in the weeds that I can't give meaningful answers I will admit that we're on the wrong track and back up to somewhere else. If they become stuck I'll ask leading questions or give hints until we can move on.

What I want to see is orderly thought processes, determination to get to the solution, well considered questions and experiments, and ideally a successful identification of the problem. Sometimes I choose problems that are too complex for someone to fully debug in a one hour interview and at the end I give them the real answer. At that point I am looking for a reaction that shows that they were engaged with the problem and experience that "aha" moment and gratification at getting to the cause. The best candidates will spontaneously ask followup questions at that point, trying to link their mental map of the problem with what was really going on.

Answer 12

Sit them down at a computer with some simple binary (with debug) symbols that segfaults with null pointer reference or such + source code + gdb and see if they can find the cause of the crash?

Answer 13

If you have your candidates do a preliminary code tests, ask them to modify the code during the interview to either solve a bug or add a new feature or better yet both. If you make the code test specifications fairly vague, that would make it easier to create test cases with "bugs" in then.

Answer 14

Finding "the bug" in a little code snippet is a very artificial situation. I suppose it might be helpful in the same way that puzzles and brain teasers are.

A more comprehensive approach would ask behavioral questions about how the candidate has performed debugging in the past citing specific incidents and then following-up with questions.

Someone who is good at trouble shooting will be able to talk about more than just the debug facilities in the IDE. What about... the bug-reporting tools, end-user interaction, reproducing the bug, logfile analysis, verification?

There is MUCH MORE to debugging than tracing through a block of code and any evaluation of someone's skill in debugging should refect that.

Answer 15

Give someone some awesome code your company runs in production. Ask them to introduce a subtle bug. Ask them why they picked that one. Ask them how they would go about finding and fixing it.

Bonus point if they find a bug in the original code.

Double bonus point if they can fix the bug in the original code.

Answer 16

I tend to ask people to describe to me the most difficult bug they ever had to track down and fix and what they did to find it and fix it. I also know that if the most difficult bug was something that you would expect only a beginner to find difficult, then likely they are not good troubleshooters (unless this is an interview for entry level). If it is something genuinely difficult and they describe their thought process as they tried to track it down, then I can get a feel for what their skill level is. What has alaways amazed me is the sheer number of people who get a "deer in the headlights" look and can't think of a single example of something they did that was complicated. Well sorry someone who leaves the hard problems for someone else to fix is not someone I'm interested in for anything except a straight-out-of-school, very junior level job.

Answer 17

I would ask a couple of technology agnostic questions like the following:

• Take me through all steps you take to identify root cause and fix a bug (defect)
• How would you use the Call Stack while debugging a multi threading app

This works really well specially in phone interviews as you only need the person to give you a convincing answer that shows how he really things while troubleshooting a problem.