Introduction to Software Engineering/Tools/Build Tools

Build automation is the act of scripting or automating a wide variety of tasks that software developers do in their day-to-day activities including things like:

• compiling computer source code into binary code
• packaging binary code
• running tests
• deployment to production systems
• creating documentation and/or release notes

History

Historically, developers used build automation to call compilers and linkers from inside a build script versus attempting to make the compiler calls from the command line. It is simple to use the command line to pass a single source module to a compiler and then to a linker to create the final deployable object. However, when attempting to compile and link many source code modules, in a particular order, using the command line process is not a reasonable solution. The make scripting language offered a better alternative. It allowed a build script to be written to call in a series, the needed compile and link steps to build a software application. GNU Make also offered additional features such as "makedepend" which allowed some source code dependency management as well as incremental build processing. This was the beginning of Build Automation. Its primary focus was on automating the calls to the compilers and linkers. As the build process grew more complex, developers began adding pre and post actions around the calls to the compilers such as a check-out from version control to the copying of deployable objects to a test location. The term "build automation" now includes managing the pre and post compile and link activities as well as the compile and link activities.

New breed of solutions

In recent years, build management solutions have provided even more relief when it comes to automating the build process. Both commercial and open source solutions are available to perform more automated build and workflow processing. Some solutions focus on automating the pre and post steps around the calling of the build scripts, while others go beyond the pre and post build script processing and drive down into streamlining the actual compile and linker calls without much manual scripting. These tools are particularly useful for continuous integration builds where frequent calls to the compile process are required and incremental build processing is needed.

Advanced build automation

Advanced build automation offers remote agent processing for distributed builds and/or distributed processing. The term "distributed builds" means that the actual calls to the compiler and linkers can be served out to multiple locations for improving the speed of the build. This term is often confused with "distributed processing". Distributed processing means that each step in a process or workflow can be sent to a different machine for execution. For example, a post step to the build may require the execution of multiple test scripts on multiple machines. Distributed processing can send the different test scripts to different machines. Distributed processing is not distributed builds. Distributed processing cannot take a make, ant or maven script, break it up and send it to different machines for compiling and linking. The distributed build process must have the machine intelligence to understand the source code dependencies in order to send the different compile and link steps to different machines. A build automation solution must be able to manage these dependencies in order to perform distributed builds. Some build tools can discover these relationships programmatically (Rational ClearMake distributed, Electric Cloud ElectricAccelerator), while others depend on user-configured dependencies (Platform LSF lsmake) Build automation that can sort out source code dependency relationships can also be configured to run the compile and link activities in a parallelized mode. This means that the compiler and linkers can be called in multi-threaded mode using a machine that is configured with more than one core.

Not all build automation tools can perform distributed builds. Most only provide distributed processing support. In addition, most solutions that do support distributed builds can only handle C or C++. Build automation solutions that support distributed processing are often make based and many do not support Maven or Ant.

An example of a distributed build solution is Xoreax's IncrediBuild for the Microsoft Visual Studio platform or the open-source CMake. These may require particular configurations of a product environment so that it can run successfully on a distributed platform—library locations, environment variables, and so forth.

Advantages

• Improve product quality
• Accelerate the compile and link processing
• Eliminate redundant tasks
• Minimize "bad builds"
• Eliminate dependencies on key personnel
• Have history of builds and releases in order to investigate issues
• Save time and money - because of the reasons listed above.

Types

• On-Demand automation such as a user running a script at the command line
• Scheduled automation such as a continuous integration server running a nightly build
• Triggered automation such as a continuous integration server running a build on every commit to a version control system.

Makefile

One specific form of build automation is the automatic generation of Makefiles. This is accomplished by tools like

• GNU Automake
• CMake
• imake
• qmake
• nmake
• wmake
• Apache Ant
• Apache Maven
• OpenMake Meister

Requirements of a build system

Basic requirements:

1. Frequent or overnight builds to catch problems early.
2. Support for Source Code Dependency Management
3. Incremental build processing
4. Reporting that traces source to binary matching
5. Build acceleration
6. Extraction and reporting on build compile and link usage

Optional requirements:

1. Generate release notes and other documentation such as help pages
2. Build status reporting
3. Test pass or fail reporting
4. Summary of the features added/modified/deleted with each new build

References

Notes

• Mike Clark: Pragmatic Project Automation, The Pragmatic Programmers ISBN 0-9745140-3-9