How to deal with misconceptions about "premature optimization is the root of all evil"?

Question

I've encountered many people who are dogmatically against anything which can be considered "optimization" in the general English-language sense of the word, and they very often quote verbatim the (partial) quote "premature optimization is the root of all evil" as a justification for their stance, implying that they interpret whatever I'm talking about to be "premature optimization". However, these views are sometimes so ridiculously entrenched that they dismiss pretty much any kind of algorithmic or data-structure deviations from the purest "naive" implementation... or at least any deviations from the way they've done things before. How can one approach people like this in a way to make them "open their ears" again after they shut down from hearing about "performance" or "optimization"? How do I discuss a design/implementation topic which has an impact on performance without having people instantly think: "This guy wants to spend two weeks on ten lines of code?"

Now, the stance of whether "all optimization is premature and therefore evil" or not has already been covered here as well as in other corners of the Web, and it has already been discussed how to recognize when optimization is premature and therefore evil, but unfortunately there are still people in the real world who are not quite as open to challenges to their faith in Anti-Optimization.

Previous attempts

A few times, I've tried supplying the complete quote from Donald Knuth in order to explain that "premature optimization is bad" ↛ "all optimization is bad":

We should forget about small efficiencies, say about 97% of the time: premature optimization is the root of all evil. Yet we should not pass up our opportunities in that critical 3%.

However, when supplying the entire quote, these people sometimes actually become more convinced that what I'm doing is Premature Optimization™ and dig in and refuse to listen. It's almost as if the word "optimization" scares them: On a couple of occasions, I was able to propose actual performance-improving code changes without them being vetoed by simply avoiding the use of the word "optimiz(e|ation)" (and "performance" as well -- that word is scary too) and instead using some expression like "alternative architecture" or "improved implementation". For this reason, it really seems like this truly is dogmatism and not them in fact evaluating what I say critically and then dismissing it as not necessary and/or too costly.

Answer 1

It seems you are looking for shortcuts not to try out the "purest naive implementation" first, and directly implement a "more sophisticated solution because you know beforehand that the naive implementation will not do it". Unfortunately, this will seldom work — when you do not have hard facts or technical arguments to prove that the naive implementation is or will be too slow, you are most probably wrong, and what you are doing is premature optimization. And trying to argue with Knuth is the opposite of having a hard fact.

In my experience, you will either have to bite the bullet and try the "naive implementation" first (and will probably be astonished how often this is fast enough), or you will at least make a rough estimation about the running time, like:

"The naive implementation will be O(n³), and n is bigger than 100.000; that will run some days, while the not-so-naive implementation will run in O(n), which will take only a few minutes".

Only with such arguments at hand you can be sure your optimization is not premature.

There is IMHO only one exception from this: when the faster solution is also the simpler and cleaner one, then you should use the faster solution right from the start. The standard example is the one of using a dictionary instead of a list to avoid unnecessary loop code for lookups, or the usage of a good SQL query which gives you exactly the one result record you need, instead of a big resultset which has to be filtered afterwards in code. If you have such a case, do not argue about performance - the performance might be an additional, but most probably irrelevant benefit, and when you mention it, people might be tempted to use Knuth against you. Argue about readability, shorter code, cleaner code, maintainability - no need to "mask" anything here, but because those (and only those) are the correct arguments here.

To my experience, the latter case is rare - the more typically case is one can first implement a simple, naive solution which is better understandable and less error prone than a more complicated, but probably faster one.

And of course, you should know the requirements and the use case well enough to know what performance is acceptable, and when things become "too slow" in the eyes of your users. In an ideal world, you would get a formal performance spec by your customer, but in real world projects, required performance is often a grey area, something your users will only tell you when they note the program behaves "too slow" in production. And often, that is the only working way of finding out when something is too slow — the user feedback, and then you do not need to cite Knuth to convince your teammates that their "naive implementation" was not sufficient.

Answer 2

Ask yourself this:

• Is the software NOT meeting performance specification?
• Does the software HAVE a performance issue?

These are reasons to optimize. So, if people are opposed, just show them the specification and go back to them and explain we need to optimize because we are not meeting spec. Other than that, one would have a hard time convincing others that optimization is necessary.

I think the main point of the quote is, if you don't have a problem, don't perform needless optimization as time and energy could be spent elsewhere. From a business prospective, this makes perfect sense.

Secondary, for those who fear optimization, always back up performance findings with metrics. How much faster is the code? How much did the performance improve over previous? If one spent two weeks only to improve code by 2% over previous version, if I were your boss I would not be happy. Those two weeks could have been spent implementing a new feature that could attract more customers and make more money.

Finally, most software does not have to be highly optimized. Only in a few specialized industries is speed really important. For example. a system to track missiles versus a web site. Shaving milliseconds off matters in the first case. For software like websites, establish SLAs for all major features and functionality. It's important to baseline performance so over time as new functionality and features are added their impact to performance can be determined and potentially optimizations can be made. Most of the time one can use pre-existing libraries and frameworks to good effect and optimization should only be needed in rare cases.

Answer 3

How to work with people who stonewall a discussion the minute it has to do with performance?

Begin with shared principles that build on the strategic direction of your group.

My Principles:

My personal principles on writing code are to first aim for correctness in my program, then to profile it and determine if it needs optimization. I profile my code myself because other programmers are potential consumers of my code - and they will not use it if it is slow - thus for my code, speed is a feature.

If your consumers are customers, your customers will tell you if you need faster code.

However, there are known, demonstrably better choices in code that one can make. I would rather get it right in in my first draft for several reasons:

1. If I get it right the first time, then I can forget about the implementation (taking advantage of information hiding), and I don't clutter up my code with TODOs.
2. Others (particularly those who only learn on the job) see it done the right way, and they learn from and use the same style of code going forward. Conversely, if they see me do it the wrong way, they'll do it the wrong way too.

Assuming the need for optimization is correct

Assuming this is a truly important part of your code that needs optimization, you could tell the parable of Schlemiel the Painter, or emphasize the remainder of the quote:

"Programmers waste enormous amounts of time thinking about, or worrying about, the speed of noncritical parts of their programs, and these attempts at efficiency actually have a strong negative impact when debugging and maintenance are considered. We should forget about small efficiencies, say about 97% of the time: premature optimization is the root of all evil. Yet we should not pass up our opportunities in that critical 3%." - Donald Knuth

Weigh the costs of additional complexity

Sometimes there's a real cost in terms of the maintainability of the added complexity. In this case, you might keep the secondary implementation in a different function or subclass, and apply the same unittests to it so that there is no question that it is correct. Later, if you profile your code and find the naïve implementation to be a bottleneck, you can switch in your optimized code and demonstrably improve your program.

Leadership

Sometimes the problem is ego - some people would rather use suboptimal or buggy code than have someone else be more right than they are. You probably want to avoid working with these people.

Leadership, especially when you do not have positional authority over people, is about making reasonable suggestions and guiding others to a consensus. If you can't guide your team to a meeting of the minds, perhaps the matter is not worth pressing. There's probably bigger fish to fry.

Answer 4

The way forward is to forget about the actual quote and the various interpretations - it it dogmatism either way to focus so much on a specific quote by a guru. Who says Knuth is always right anyway?

Instead focus on the project at the hand, the piece of software you are developing along with the colleagues you disagree with. What is the requirements for acceptable performance for this piece of software? Is it slower than this? Then optimize.

You don't have to call it "optimization", you can call it "fixing a bug", since it is by definition a bug if the implementation fails to conform to the requirements.

More generally, there are two possibilities regarding optimizations:

1. The optimized code is also shorter, simpler to understand and easier to maintain.

2. The optimized code is more complex to understand, takes a longer time to write and test, or would be more complex to change in the future if requirements change in unexpected ways.

If the case is (1) you don't even have to argue about optimization. But if (2) is the case, then you are engaging in a trade-off decision. This is actually a business level decision, not purely a technical decision. You have to weigh the cost of the optimization against the benefit. In order for there to even be a benefit, the inefficiency has to be problem in the first place, either as bad user experience or significantly increased cost of hardware or other resources.

Answer 5

In my experience, if you get this kind of opposition to optimization regularly, people are not really complaining about optimization. They are complaining about what you are sacrificing in the name of optimization. This is usually readability, maintainability, or timeliness. If your code is delivered in the same amount of time, and just as easy to understand, people couldn't care less if you're using more efficient data structures and algorithms. My suggestion in this case is to work on making your code more elegant and maintainable.

If you're getting this kind of opposition in regards to other people's code, it's usually because you're suggesting a significant amount of rework. In those cases you really do need some actual measurements to show it's worth the effort, or perhaps try to get involved earlier in the design phase, before code is written. In other words, you need to prove it's in the 3%. If we rewrote all the code that wasn't exactly how we liked it, we'd never get anything accomplished.

Answer 6

I think the full quote in context is instructive. I'll copy from a post I made on Reddit on the topic:

There is no doubt that the grail of efficiency leads to abuse. Programmers waste enormous amounts of time thinking about, or worrying about, the speed of noncritical parts of their programs, and these attempts at efficiency actually have a strong negative impact when debugging and maintenance are considered. We should forget about small efficiencies, say about 97% of the time: premature optimization is the root of all evil.

Yet we should not pass up our opportunities in that critical 3%. A good programmer will not be lulled into complacency by such reasoning, he will be wise to look carefully at the critical code; but only after that code has been identified.

-- Donald Knuth, Structured Programming with go to Statements, ACM Computing Surveys, Vol 6, No. 4, Dec. 1974, p.268

The point, and implication, is that there are more important things to worry about than putting your attention to optimization too early. Certainly, you should carefully consider your data structures and algorithms (this is in the 3%) but you shouldn't worry about whether subtraction is faster than modulo (this being in the 97%) until it becomes clear that low-level optimization is necessary.

The former is not necessarily optimization in the sense that your colleagues are thinking, but it is optimization in the sense that poorly-chosen algorithms and data structures are suboptimal.

Answer 7

This seems like a communication problem and not a programming problem. Try to understand why people feel the way they do and try to crystallize why you think your way would be better. When you've done that, don't start a confrontational argument where your goal is to tell others why they're wrong and you're right. Explain your thoughts and feelings and just let people react to that. If you can't reach any sort of consensus and you feel like this is a really critical issue, then you probably have some serious issues in the team overall.

More focused on actual programming, don't waste time on long arguments over something you just have a gut feeling is "faster". If you see someone writing a method that is called once per request in a web app and it has O(n^2) time complexity when you KNOW it's really a O(log(n)) time problem, then sure, if it's such a no brainer, go ahead.

Be aware though, as humans, we programmers are really bad (and I mean AWFUL) at guessing which parts of our applications that will bottleneck. Eric Lippert writes about this interesting subject in this blog post. Always favor maintainability. Any performance issues that eventually are found can then easily (well, relatively) be fixed when you have more info.

Answer 8

There is indeed lots of misunderstandings about this quote, so it's best to step back and look at what the actual issue is. The issue isn't so much that you should never "optimize". It's that "optimize" is never a task you should set out to do. You should never wake up in the morning and say to yourself "hey, I should optimize the code today!".

This leads to wasted effort. Just looking at code and saying "I can make it faster!" leads to lots of effort making something faster that was fast enough in the first place. You might find pride in telling yourself that you made a bit of code four times faster, but if that code was a calculation that happened on a button press, and it took 10 msecs in the first place before displaying to a human user, no one's going to give a damn.

That is the "premature" in "premature optimization". When is it not "premature"? When customers tell you "this is too damn slow, fix it!" That's when you dig in the code and try to make it faster.

This doesn't mean that you should turn off your brain. It doesn't mean that you should keep 10,000 customer records in a singly linked list. You should always understand the performance impacts of what you do in mind and act accordingly. But the idea is that you are not spending extra time deliberately trying to make it faster. You are simply choosing the more performant choice out of otherwise equal choices.

Answer 9

You can either do things the wrong way, or do things the right way.

Often, things are done the wrong way and code is refactored so that it's done the right way. If you're going to write new code, and you know that you can do things the right way without a major penalty, I'd just err on the side of doing it the right way. (Note that, after performance testing, etc, some things might need to change - but that's okay. Alternatively, a completely naive implementation is rarely, if ever, right.)

It's not necessarily premature optimization if you a) know that it will help in the future or b) know that the suboptimal path will lead to problems down the road. It's like a chess game, really.

I think that people will tend to want to do things right, rather than do them wrong. Use this when you discuss alternative strategies with your peers.