Optimizing Code for Speed/Optimizing by Reducing Feature-Set

"There ain't no such thing as a free lunch". The more features your program has, the more code is required to implement them, which in turn consumes more memory, and slows things down. Furthermore, it requires more conditionals and often even loops, which also require extra processing. Thus, by removing features from one's application, one can have faster code.

For example, in "How Microsoft Lost the API War", Joel Spolsky tells that Microsoft has gone to great lengths to maintain bug-to-bug (binary) backward compatibility for their Windows-line of operating system. As a result, buggy software that relied on API quirks that got changed, often could still work in later versions of Windows, because Microsoft made sure that Windows supplied them with a similar environment. As a result of maintaining all of this backward compatibility, there was more code in Microsoft's products, which may have contributed to the common and notorious perception that they became slower with each release.

Optimizing by Introducing Bugs

Furthermore, we can say that often bug fixes (for example boundary checks, sanity checks, etc.) will have a negative influence on the program's performance, and as a result not fixing these bugs or introducing new bugs is also a possible way to optimize (but not a very recommended one).

Optimizing by Compromising on Security

I can go further than that, and claim that one can often optimize one's code by compromising on its security. The less sanity checks, input checks, etc. a code has - the faster it will run. For example, a code that does not have a lot of checking for the failure (and graceful handling) of dynamic memory allocation calls (in languages such as C without managed programming) such as malloc(), will run faster. (Until and if it runs out of memory and then crashes.)

I read somewhere that programmers complained that checking and handling errors in their programs made them have to write about 4 times more code, which makes sense. All this code slows the program down when everything runs as expected.

Note that "Here be dragons". It's probably never a good idea to compromise on security in order to increase speed. But I've mentioned it here for completeness sake.