Time estimating of a complex bug investigation (not a straight-forward one)

Question

(It is not a duplicate: Bug investigation is much more non-deterministic than a defined development task where things to be done are specified. Investigation is about narrowing a huge search space, which is different from building software. We cannot compare Memory Corruption bugs, C/C++ Undefined Behaviors, and Multi-Thread data races to scope-limited development tasks).

It's not the first time that my superiors are asking for an estimation when dealing about bug investigation. Just like it was a delimited development task.

I tend to prefer a scale of complexity. Something like: Easy, Not That Easy, Hazy, Difficult, and Really Difficult. And giving a time estimate only for easy investigations.

For simple investigations, I do agree that it is feasible. But I have no success in explaining them that a complex investigation is a cycling process:

1. Reason about the facts, the code, the trouble
2. Make hypothesis
3. Find ways to measure the hypothesis by facts (traces, logs, debug...)
4. If facts are clear and hypothesis is false, restart at 1.

It's a rather scientific/rationalist process indeed.

What other arguments could I tell, so as to explain the uncertainty in the very process of bug investigation?

Answer 1

Depending on the severity of the issue and the urgency of fixing it, see if they'll agree to a "time-boxed" effort. Say, two days investigating/trying to re-create the problem. If you don't have it by then, you'll report on what you've eliminated as the problem and see if they want you to keep digging.

For minor issues, they'll likely agree to a short time-box as long as work continues on other issues or features. For major ones, especially those with major visibility, the time-box will let them get regular reports on your effort.

Answer 2

This isn't practically useful to you, but I wouldn't sink to the level of explaining to non-technical coworkers why certain technical tasks are harder than others. The point of hiring specialists for programming tasks who are better at technical things and worse at business or management things than their employers is that thy are better at them. To a large part, understanding why a defect arises equates to being able to fix it, and to a lesser degree to being able to avoid similar errors in the future.

Shortly and brutally, if your superiors could understand why some debugging tasks are so hard, they would be able to perform those tasks themselves - which they aren't, since they hired you. It's hard for anyone to accept that there are distinctions that they cannot appreciate, so there is little point explaining to a manager that you are having difficulties that you can't explain to him. The best thing you can do is to rise to his level of the view and tell him, "Look, I know this sounds lame, but I cannot tell you very well how long this will take no matter what the incentive". Once you've said this in the cases where it's true, and they have seen that it is actually true (because your effort did vary unpredictably), maybe they'll begin to believe you when you say that you dobn't know something. But I wouldn't hold my breath. Saying "I don't know" to something in a competetive environment is very hard to do, and many professionals avoid doing it at all costs; and so they tend to believe that it isn't true when someone else is saying it either. Strange, but (in my previous experience) all too true.

Answer 3

Combine two approaches: the "fixed time estimate" and the "I don't know" approach without literally saying I don't know. Let's face it, if you knew exactly what is causing the bug you wouldn't have written it that way in the first place (yes, I know we don't always fix our own code, but isn't cleaning up someone else's mess even worse?).

Get into the habit of responding to bug time quote requests with, "I'll take a look at it and let you know how long it will take by ." People want to know that you care enough by demonstrating you're taking steps to solving the problem and that you're prioritizing their task appropriately. Making promises (and they are promises in their mind) you can't keep is going to decrease their level of trust in you.

Don't forget to reorganize your other commitments. In simple terms, I can't fix your new found bug and deliver a new feature at the same time. Which one do you want me to do first? Hopefully, they'll get the bug out of the way, but you never know.

They don't want a technical explanation. When a non-technical users asks, "Why is this going to take so long?" they don't necessarily want a technical answer and they really don't want details. By taking time to investigate the error and then giving a time estimate afterwards, you can also summarize the solution. There may be multiple areas of the app that need to be altered (The technical debt has now come due.), patching other technology, etc.

Answer 4

I suggest giving a range as an estimate.

For example, for easy issues it might be "between 15 minutes and three hours", for hard issues it might be "between two hours and a week".

This should get your point across. If they ask why your best and wost case estimate differ so wildly, you can start explaining the things you mentioned in your question (cyclic process, etc) and even give examples for the best and worst case.

Answer 5

Looking for a solution to a problem can be a lot like looking for anything else that's missing. Take car keys for example. If mine aren't on the shelf where they belong, I check the other side of the shelf & a couple of coat pockets and usually find them in minutes.

When my wife lost her keys a while back, we had to wait for a snow drift to melt first. That took a couple of months.

I think you nailed it with "easy", "hazy", and "hard". A bug is easy to find when you can quickly narrow your search space down to something reasonable. It's hard when you can't.

All your superiors really need to know is that this is a priority, you're working methodically / you have a search plan, and you're not out of ideas yet.

Answer 6

Time estimation for completion of tasks in software development is a dark art indeed. I think your arguments are good, but you forgot one critical one:

However much time you think it's going to take, multiply that by two.

I've found this to work quite well. Things will always take longer than you think it will, especially in software development.