Should I give the answer to a failed interview coding exercise?

Question

We had a senior level interview candidate fail a nuance of the FizzBuzz question¹².
I mean, really, utterly, completely, failed the question - not even close.
I even coached him through to thinking about using a loop and that 3 and 5 were really worth considering as special cases.

He blew it.

Just for QA purposes, I gave the same exact question to three teammates; gave them 5 minutes; and then came back to collect their pseudo-code. All of them nailed it and hadn't seen the question before. Two asked what the trick was...

On a different logic exercise, the candidate showed some understanding of some of the features available within the language he chose to use (C#). So it's not as if he had never written a line of code. But his logic still stunk.

My question is whether or not I should have given him the answer to the logic questions.

He knew he blew them, and acknowledged it later in the interview.
On the other hand, he never asked for the answer or what I was expecting to see.

I know coding exercises can be used to set candidates up for failure (again, see second link from above). And I really tried to help him home in on answering the core of the question. But this was a senior level candidate and Fizz-Buzz is, frankly, ridiculously easy even after accounting for interview jitters.

I felt like I should have shown him a way of solving the problem so that he could at least learn from the experience. But again, he didn't ask.

What's the right way to handle that situation?

¹_{Okay, that's not the link to the actual FizzBuzz question, but it is a good P.SE discussion around FizzBuzz and links to the various aspects of it.}

² _{To help clarify, this is the nuance of Fizz-Buzz I asked and it's from Project Euler's first problem. Substitute printing Fizz | Buzz for summing the numbers and you have the same fundamental question.

If we list all the natural numbers below 10 that are multiples of 3 or 5, we get 3, 5, 6 and 9. The sum of these multiples is 23. Write a function that finds the sum of all the multiples of 3 or 5 below 1000.}

³_{This question attracted more attention than I had expected, and I appreciate all of the replies. Some of the later answers have really gotten to the core of my question, so I'll allow the community to review and up-vote before assigning "the" answer.}

⁴_{I selected "the" answer based upon community votes at that point in time. And I think Yannis' answer is appropriate for interviews with newer devs. I think the collective response focusing on the lack of asking for the answer is spot-on as well.}

score 15 · Accepted Answer · answered Nov 14 '12 at 05:04

Most of my interviews were with students looking for an internship position, and more often than not they screwed up simple (?) exercises. I wanted an easy and friendly way to communicate their errors, and what I came up with was quite simple: I solved the exercises myself, and showed them my solutions after they finished with theirs.

I had quite a few extremely interesting and revealing discussions with candidates that started by comparing our different approaches to solving the same problem. After a while I even anticipated some of the errors, just by checking which school the candidate was attending (some "professors" are... morons). And, well, in the few instances that a candidate couldn't come up with any solution, I had already given them one for next time.

score 15 · Answer 2 · edited Apr 12 '17 at 07:31

Giving the Answer

I was going to say that if the candidate isn't interested enough to ask, I wouldn't waste my breath, but @Yannis_Rizos answer is much better.

Interviews are pretty fast paced. I know I often look things up for days after an interview.

People Who Can't Code FizzBuzz Classic

I imagine a big sticking point is being aware of the % operator. You would hope that someone could think to compare (myInt / 3) == (myDouble / 3.0) but maybe with the stress of an interview... Still, FizzBuzz Classic forces a brute-force approach, putting it in the category of the easiest algorithm problems to solve. As a hint, have you tried asking people to just code "Fizz" for half-credit and maybe add "Buzz" later as an enhancement?

I think the ultimate answer to your question is that it's just really hard to find good candidates.

Interview Questions in General

I've often found it easier and more productive to ask candidates to describe the last programming project that they were excited about. I've had 100% success with this question, meaning that people who spoke animatedly about a programming project and could answer technical questions about it were great hires and those who couldn't, weren't. This has the nice side effects of putting the candidate at ease, and encouraging open-ended discussion. With this question, the candidate will, in effect, tell you what they are most suited for.

Maybe at a think tank algorithm questions are necessary as well, but I've abandoned them in favor of the "favorite project" question.

Sum (Son) of FizzBuzz

Your interview question is not equivalent to FizzBuzz:

If we list all the natural numbers below 10 that are multiples of 3 or 5, we get 3, 5, 6 and 9. The sum of these multiples is 23. Write a function that finds the sum of all the multiples of 3 or 5 below 1000.

Where FizzBuzz Classic forces you to go through n iterations (to print out every number or Fizz/Buzz), your problem can be done in n/5 + n/3 + n/15 iterations, or even no iterations - a direct fixed-point calculation is possible. The following program compares these three methods:

public static void main(String[] args) {
  long n = Long.valueOf(args[0]).longValue();
  long sum = 0;
  long ms = System.currentTimeMillis();
  // Brute force method  Performance: cn
  for (long i = 1; i <= n; i++) {
    if ((i % 3) == 0) { sum += i;
    } else if ((i % 5) == 0) { sum += i; }
  }
  System.out.print("Brute force sum:    " + sum);
  System.out.println(" time: " + (System.currentTimeMillis() - ms));
  ms = System.currentTimeMillis();

  // Second solution: iterate through only numbers we are
  // interested in.  Performance: c * (n/3 + n/5 + n/15)
  // We counted multiples of 15 twice, so subtract one of them
  sum = countSum(n, 3) + countSum(n, 5) - countSum(n, 15);
  System.out.print("Only multiples sum: " + sum);
  System.out.println(" time: " + (System.currentTimeMillis() - ms));
  ms = System.currentTimeMillis();

  // Third solution: Use high school algebra.  Performance: c
  sum = sumSeries(n, 3) + sumSeries(n, 5) - sumSeries(n, 15);
  System.out.print("Sum of series:      " + sum);
  System.out.println(" time: " + (System.currentTimeMillis() - ms));
}

// Iteravely sum all multiples of skip
private static long countSum(long n, long skip) {
  long skipTotal = skip;
  long grandTotal = 0;
  while (skipTotal <= n) {
    grandTotal += skipTotal; skipTotal += skip;
  }
  return grandTotal;
}

// Thanks to @Caleb for pointing this out!  High school algebra
// tells us that the sum of a series is: (n * (a1 + an)) / 2
// where a1 is the first term and an is the nth term.  E.g. The
// sum of a series of 3 is: (n/3 * (3 + n - (n % 3))) / 2
// Since we are thinking about performance here, we'll shift
// right one instead of dividing by 2 for style points.  ;-D
private static long sumSeries(long n, long skip) {
  return (n/skip * (skip + n - (n % skip))) >> 1;
}

Output (sum of FizzBuzz < 1000):

$JDK_HOME/bin/java FizzBuzzNot 999
Brute force sum:    233168 time: 0
Only multiples sum: 233168 time: 0
Sum of series:      233168 time: 0

With a larger n for performance comparison:

$JDK_HOME/bin/java FizzBuzzNot 1000000000
Brute force sum:    233333334166666668 time: 4744
Only multiples sum: 233333334166666668 time: 818
Sum of series:      233333334166666668 time: 0

Note to those who down-voted this as off-topic

The point of presenting a solution to this question is to show that while the brute force solution to Sum of FizzBuzz is similar to FizzBuzz Classic, better solutions to the Sum problem are available, making it a fundamentally different problem. Sum of FizzBuzz is extremely tricky if you don't remember the proper formula for the sum of a series, or don't realize that it applies when stepping by 3 or 5.

If you re-derive the formula for the sum of a series by splitting the series in half, reversing one half, and pairing them up, you get (n + 1)(n / 2) which can take you down a really messy path as far as integer division and truncated remainders are concerned. The (n(a1 + an)) / 2 version of this formula is absolutely critical for a simple answer for all values of n.

score 8 · Answer 3 · answered Nov 14 '12 at 13:50

I felt like I should have shown him a way of solving the problem so that he could at least learn from the experience. But again, he didn't ask. What's the right way to handle that situation?

I don't care what level the interview is for, nor even really if its a "FizzBuzz"-level question or an advanced question. If you ask a candidate to solve a question, and they can't, but don't even bother asking you for the right answer then they aren't worth your time. How in the world could you be so intellectually lazy?!?

And even if you totally stink as a programmer and are just trying to buffalo your way into a job, why wouldn't you at least be so pragmatic as to get the right answer now, so that you know it for the next interview?

So no, you shouldn't "give" the answer, but you should expect to have the candidate insist on hearing the right answer after they fail. If they don't, its a huge red flag in my book.

If someone blew FizzBuzz in a junior level Dev interview because they couldn't remember the modulus operator and just couldn't bring themselves to move on without it, but they then got all passionate about re-doing it once you explained the right answer, or at least talking through the right code with you, then that's almost as good as answering it correctly.

score 4 · Answer 4 · answered Nov 14 '12 at 05:05

I typed out fizzbuzz while on the phone with the interviewer during a pre-screen. That's something that, even if everyone hasn't heard of, you should be able to cobble together after a semester of course work but definitely after acquiring "senior" status.

There's really no recovery from not being able to do that. It's one of those necessary nuisances that you need to get out of the way just in case.

I would say that it makes sense to deliver as a pre-screen so you're not wasting everyone's time with bringing them on site for an interview.

Caleb · Answer 5 · 2012-11-14T23:59:10.140

I even coached him through to thinking about using a loop and that 3 and 5 were really worth considering as special cases.

It'd be interesting to know what you think the "correct" answer to your FizzBuzz-ish question is. From where I sit, a good one (in C) written to the letter of your question is:

int f(void) {
    // sum the multiples of 3 and of 5 and subtract multiples of 15 so we don't count them twice
    return ((1000/3)/2)*(999+3) + ((1000/5)/2)*(995+5) - ((1000/15)/2)*(990+15);
}

A better one might be:

Why the heck would you write a program to do that when you can calculate it directly?

The point is that there's more than one way to skin a cat, and the fact that the candidate in question didn't immediately start writing for loops and mod operators doesn't mean that he's stupid. If you want to know what the candidate knows, discuss the problem -- find out what he's thinking. If he's stuck or confused, find out where and why. He might lead you to an approach that you never considered.

My question is whether or not I should have given him the answer to the logic questions.

As the interviewer, it's not your place to teach the candidate a lesson. If they really don't know how to write code, there's absolutely no need to embarrass them by dwelling on how much they don't know. If they're interested enough to ask, then by all means feel free to share. Otherwise, finish up the interview, thank them for their time, and move on to the next candidate.

score 2 · Answer 6 · edited Apr 12 '17 at 07:31

To answer your question, no I would not give the answer. If the person wants to be a better software engineer, they will find the answer. If you give them answer you are robbing them of this opportunity.

The more relevant question is when can you call yourself a senior developer? This varies between organizations and countries. For example, a company I worked with considered Software Engineers with 5 years of experience as seniors. There was no emphasis on the quality of the experience, just the length.

Until we come up with a standard that will categorize all Software Engineers regardless of their language we are left with the individual deciding the level of their skills. And we will continue hearing of "Senior Engineers" failing the most rudimentary skills test.

A couple weeks ago the question was asked "When should you call yourself a Senior Developer". I also wrote a blog post on the topic.

score 2 · Answer 7 · answered Nov 14 '12 at 21:27

Your "problem" is that you're an empathetic human, so it's hard to watch someone struggle with a problem that you know the answer to (this is also a problem if you run usability studies). From a duties-as-an-interviewer perspective, you are not in any way obliged to teach the interviewee how to program or problem-solve, or the solution to a problem that you ask.

When you coach an interviewee, it's not so that they can get the right answer. It's so that you can see if they really can't solve the problem or if they just got hung up on one or two mistakes or misunderstandings.

So if you want to give an interviewee the solution after the fact, you're doing it purely for yourself. In general, I'd rather use that time to allow the interviewee attempt another problem. But a "senior" programmer who can't answer FizzBuzz is probably off the list. If you decide to give solutions make sure you don't fool yourself into thinking the interview went better than it did (if you find yourself thinking, "he could not solve the problem, but when I explained it he understood it well", then you're on a dangerous path).

And yes, I have been an interviewee who was in over his head and couldn't even make the first attempt at solving an interview problem. That simply meant that I needed to learn a lot more in order to pursue a job in that field.

score 2 · Answer 8 · answered Nov 15 '12 at 02:23

Getting the right answer is not the important part of this test. What you're measuring is someone's approach to problem solving, how they engage with the question, anything creative or interesting they come up with along the way; that kind of stuff. Even if they get the wrong answer they may still be viable by these criteria.

OK, not knowing the mod operator is inexcusable, and by the metrics I've given this candidate appears to still be a write-off, but I don't believe that just giving the right answer to this candidate is going to be of any benefit.

This comes down to your own personal opinion from here. Do you want to give interview feedback with a view to helping the candidate do better in future interviews (and so that future interviewers don't have to suffer what you've just gone through)? If so, couch your feedback in the terms I've just outlined above: tell them that it's not just about the right answer but how they work to arrive at the answer is a critical factor.

Stephen C · Answer 9 · 2012-11-15T06:34:31.247

I felt like I should have shown him a way of solving the problem so that he could at least learn from the experience. But again, he didn't ask. What's the right way to handle that situation?

The way I see it, there is no situation to handle. Assuming that you have rejected his application, the candidate's (apparent) lack of interest is not something you need to concern yourself about.

Indeed, even if he did ask, you don't owe him an explanation. And if you talked to your HR folks about it, they might advise you that further discussions with the candidate is inadvisable for legal reasons.

It is also worth noting that the FizzBuzz problem has different "best" answers depending on how you ask it. If you ask for the "simplest" solution and the "most efficient" solution, the best answers are radically different. And this can colour your judgement unfairly ... if you were unclear in the way that you asked the question. (On the other hand, a good / experienced candidate would have the foresight to clarify that before starting to code ...)