Optimizing Code for Speed/How to best optimize?

Automated tests

The first thing you need to do before optimizing is make sure that your code has many automated tests with a good test coverage. Optimizing the code (or any other kind of modification) may break something, and the automated tests will likely and hopefully catch that.

Refactoring

The next thing you need to do is to make sure your code is modular. Duplicate code and other properties that make a code non-modular can prevent a clear analysis of the code's bottlenecks by profiling it.

Profiling

Both before and after changing code to "optimize" it, it is important to profile your code using a good software profiler. There are many profiler tools. Profiling will help show which code does not take a lot of time to run and therefore it would be best not to invest a lot of optimization effort in it. One should start by optimizing the most time-consuming tasks first.

It takes some expertise to know how to properly analyze the results given by the profiler, and seeing what needs to be optimized. For example, some IO-intensive routines may appear to consume a lot of time, while in fact that time cannot be effectively eliminated because the amount of IO is minimal (while still being relatively time consuming). I feel that discussing this aspect of profiling further is orthogonal to the mission of this article, so I'm not going to cover it here.

Code review

Finally, it is a good idea that someone will review the code and give general remarks on speed bottlenecks found there. To quote Eric Raymond from "The Cathedral and the Bazaar" "Given enough eyeballs, all bugs become shallow".

Other ways to optimize

Someone who commented on the first draft of this essay, said I was not giving enough focus about profiling and said that "profiling is the only good way to optimize.". While I agree that one should profile before optimizing, I strongly disagree that the only way to optimize is by profiling first.

For once, it's possible that one will perform accidental optimizations by trying to achieve a different aim. Sometimes I increased Freecell Solver's performance by changing the code in order to overcome running out of stack on Windows NT. Moreover, sometimes a programmer can realize that changing an aspect of the program in a certain way will increase the performance without profiling the code.

Finally, sometimes profiling the code (at least with the limited tools that one has) does not tell one much. On one of my projects after profiling, the most offending function takes about 15% of the time, with most of the other top candidates taking a few percents. So what should I profile there?

So while profiling is important, it is not the only good way to optimize and sometimes one needs to take different approaches.