How do you tackle really bizarre errors that keep you puzzled for more than 10 hours?

Question

You know them, those errors that make NO sense. Where it seems like a gremlin just jumped deep inside your chips and messed up something. Do you take a walk, write stuff, call an uncle?

Answer 1

Quit. No, not your job! Just get up and go home. You're done for the day or the weekend. 19 times out of 20 when you come back to the problem next, the solution will present itself within an hour.

Answer 2

Before ten hours go by, I would get some help.

1. Describe the problem to someone else, anyone else, even your rubber duck.
2. Ask someone else to take a look at the code, or step through it with them.
3. Isolate it. Delete a bunch of stuff, then bring it back bit by bit until the problem reappears.
4. Get some sleep!

Answer 3

One word, timebox, set a limited amount of time to work on something, and if it isn't solved, move on to something else and come back to it the next day with a fresh perspective.

That and another set of eyes, is always worth more than any time you can waste staring at something.

I would never spend more than 45 mins to an hour trying to solve something in one sitting, it violates the law of diminishing returns.

Answer 4

For those really horrible problems my strategy usually goes as follows.

• Experiment and google. Keep trying to solve the problem. Most of the time this solves the problem in an hour or less.

• So that hasn't worked. Take a break. Have a coffee, talk about something unrelated to a colleague. Push the problem out of your mind. When you look at the problem 5 or 10 minutes later you are looking at it from a slightly different perspective. Most of the time this works.

• In this case it hasn't. So spend another 10 - 30 minutes looking at it. Then call in a colleague. But before you do, make some notes; you want to demonstrate the problem, reproduce it, then list of the things you have tried, and most importantly prove that you have tried them. So do a dry run first. Set some book marks in the code, close any superfluous open documents etc. This way you may either solve the problem yourself, or when you do demonstrate the problem you won't be wasting their time.

• Ask your colleague to make you prove all your assumptions. is that setter actually being invoke? Is that method really returning what you claim it is? You think that object isn't null - show them it isn't null.

• Most of the time, either demonstrating the problem will make you realise that you haven't tried all the possibilities or your colleague will see your mistake.

• If that doesn't work its time to get serious. Document exactly what you are trying to do, what you have tried, and why it didn't work. Email this to all your colleagues. Post it on SO. At this point the document should be a perfect SO question.

• While you wait for responses, google google google. Try every permutation of the question you have. Open up a bunch of tabs. You probably aren't going to get an answer by this point, but you're looking for ideas, possibilities, different ways of approaching the problem.

• Do something else, if you've spent 5 hours on a problem its time to leave it for another day. Maybe you will get a useful response. Maybe when you attack the problem the next day it will be obvious.

• If none of that works, its time to look for a different solution. Maybe you can use a different method, a different technology. Maybe you should consider abandoning the feature for now. Are you billing the client by the hour? Are you working for a company on an internal app? You need to escalate this to the owner and tell them "look, I've spent x hours on this and made no progress, is the cost benefit worth it?". You don't want to go to your boss and tell them you spent 16 hours on a problem only for them to turn around and say, it nots that important, skip it for this release. you need to find that out earlier.

• And if that doesn't work? Well your only options are to keep hammering away at the problem or seek out industry expertise. Ask technology experts on twitter. Email your technology provider.

Answer 5

Explain the problem to someone else.

By explaining the problem to someone else, you have to clarify it: this often lets you see the solution.

(One of the UK professional computer magazines once proposed selling life size cardboard cut-outs of a senior programmer specifically for this purpose.)

I find sleeping on a problem (sometimes for a couple of days) can also help.

Answer 6

I have a three step plan:

1. Get a coffee or other tasty beverage.
2. Work on something else for the rest of the day.
3. "Phone a friend" and doodle on the whiteboard.

Each stage is an escalation if the previous step failed. There's almost always something else productive I can work on at stage 2.

Answer 7

Sleep over it

Otherwise, call someone nearby and ask him to take a quick look over the code.

Often errors which would take you a long time to find (since its your code) are found very easily by others

Answer 8

You could see if getting up, pacing around, and thinking about the problem helps you find a solution. Whether or not you're actually standing or pacing, try getting away from the computer while you're thinking.

Answer 9

I generally do one of the three:

1. Take a walk/bike ride...some that gets you away from the computer.
2. Play with my dog or cat
3. If you have a hobby, work on that for a while.

Any of the three do a good job of distracting oneself from the situation at hand. I find the distractions let my subconscious brain chew on something for a while. After an hour or so of this, bam, there's the solution :-).

Answer 10

Build a test Harness to target that exact Defect and Isolate it

Just keep eliminating good code.. while replicating the defect. Until you target the exact piece of code casing the error. Then trace the code.

Recommended reading : The Pragmatic Programmer Specifically Chapter 10 : Tracer bullets

Answer 11

Edit:

The short answer:

Q: How do you tackle really bizarre errors that keep you puzzled for more than 10 hours?

A: Make sure they never happen: understand your design, know your code, learn how to use your debugger.

---

Explanation:

"Where it seems like a gremlin just jumped deep inside your chips and messed up something"

This should never happen. If it's your code, you should have a very good idea of what is causing the error before you attempt to fix it.

Futhermore, when you write your code you should already know where and why it's likely to fail.

Having said that - asking a peer, posting on SO, retracing and rolling back your steps and taking a break - all the suggestions mentioned above will help.

The other thing, is you must know your tools - your debugging toolkit. Logging messages at suspect points in your code, examining your call stack carefully, using conditional breakpoints and watches, etc etc. Debugging skills are not extras - they are part of programming.

Answer 12

All of these suggestions are great. However, I use a technique quite often that I didn't see mentioned. Make lists to organize your thoughts about the problem. If I have a particularly sticky problem I usually write out multiple lists such as: Facts, Assumptions, Questions, Symptoms, etc. I find that oftentimes in the process of organizing things in this way I discover assumptions I didn't realize I had (that often turn out to be wrong), questions I didn't realize need to be asked, other permutations I can check, etc.

Answer 13

I had a similar problem, an apparent memory corruption in Objective-C, which I struggled with for many hours. But then me and my colleagues just took a walk for lunch, and I explained the problem (and one particular bit having to do with deserialization of an object in its init method), and basically explained the whole problem to myself.

(techy details: basically, I initialized and returned an object onto something else than self, so there were two allocs, but just one object returned. Memory shifted and went crazy, crashes, and the debugger didn't really know what to do with it either).

Answer 14

Take a bath.

Any Rodney McKay fans?

Seriously, though, if there's one commonality among all these answers, it's to take a break and do something else.

I like to think of it as relegating the problem to your subconscious. Even if we are otherwise unaware, our minds (seem to) continue to work on the problem, even when we're doing something else, such as taking a bath.

Answer 15

Step through it step by step, down into assembly. Who calls what, break-point on memory access. That usually catches the bug real fast.

If not, take a walk.

Answer 16

A combination of all of these :

• Get away from it for a while so it can sit on your back burner. Sleep, rest, eat, take a walk, whatever.

• Examine the problem more, what else does it do wrong, what other symptoms can you find?

• Research the problem, see what you can find. Remember to try different keywords

• Try something different. A work around. A different debugging technique. A validator. A different computer.

• Talk to someone. Even if they are not able to help, or not even a programmer, sometimes talking will trigger the idea lightbulb

• Restart! If appropriate, try restarting your computer, the server, etc. If nothing else, you can use the time to think.

• Ask StackOverflow! We are here to help

Answer 17

I really didn't like the most up-voted answer, because even though that sometime works, some times you just need to figure it out that same day, so what I'd recommend, in this order, is:

1. Confirm that it's not only happening to you. This can save you a lot of time. Maybe you uninstalled a required component, or made a change in your environment, and an exception is being swallowed somewhere in your code. If it's happening only to you, I would use an environment comparison tool. I recently read about a software called Envy, which allows you to do just that, although it's not freeware, it costs 10 USD.

2. Happening to everybody? Fine, now do a View History on the code and verify for recent changes that might have caused the error, either directly or indirectly.

3. There aren't any recent changes? If it's a very specific error (an exception), 'stackoverflow it'. Now that doesn't sound better than 'google it' but I feel good to say I first search stackoverflow for programming research than google. If it's a really known issue, it's very likely that you'll find a solution here. If not, then post a question on the related stackexchange site. You might get a very quick answer, or even if you don't, your question will be out there while you do more research. That's a benefit.

4. If you didn't find an answer online or it isn't a general error, then walk through the code step by step, checking if the obtained results of each step make sense for the result you expect. Go start to finish on each method, and bottom to top on a tiered solution. (i.e. if you're troubleshooting performance, start with the code that retrieves the records. doesn't make sense to start in the UI if you can determine quickly if the first step is the problem).

5. If after going through the code a couple of times you still haven't found what's wrong, call-in someone to talk about it. As someone already mentioned, talking about it out loud can light the bulb. Plus pair-programming is really useful.

6. At this point, if it's feasible, walk away either for some time or for the day. I read a very true tweet yesterday that said "I went to bed thinking 'how the fck' and woke up thinking 'but of course'". So true.

7. If you still don't have an answer, I'd dare say you could try refactoring into smaller tasks/methods/functions. Henry Ford said something like 'There isn't a task so complex that cannot be accomplished by breaking it into smaller tasks'. At this point, if the solution is too complex and you haven't figured out either by yourself or with the help of someone else, refactor the code into smaller tasks. Even if you don't end up commiting it, it can help you find the reason.

8. Add instrumentation to your code.

9. Tweet about it??

Answer 18

You need to step back. My motto is 'if the problem is too hard then you are solving the wrong problem'. What are your assumptions? don't trust anything.

The corollary to that is 'the weirder the problem, the weirder the solution'. The computer's strength is its logic so you can't win on logic. You have a brain and have to out-think it.

In modern times there are so many other things interacting on a system - firewalls, AV, Antispyware, automatic updates happening every night - you have to deal with moving targets.

Answer 19

Google it. Stackoverflow it. Post it on forums. Basically if you can't solve it alone, get people to help.

Answer 20

1. Write down the problem.
2. Think hard.
3. Implement the solution.