- Measure first
- Do more in parallel
- Work around the inefficiencies of C++ compilation
- Use faster tools
- Do less disk I/O
Use those cores!
When running unit tests, use the option in the driver to run multiple tests in parallel. ctest supports a '-j' argument for this as well. An important thing to remember before enabling this is that your tests need to be set up so that they can't interfere with one another. This means not trying to use the same resources (files, settings keys, I/O ports, web service accounts etc.) at the same time. Some tests might be easier to isolate than others in which case you can split your test suite into subsets and only run some of the subsets in parallel. ctest has a facility for assigning labels to tests using.
CTest then has a set of command-line arguments that can be used to run only tests with labels matching a certain pattern, or exclude tests with labels matching a certain pattern. This can then be used to run only a subset of tests which are known not to interfere with one another concurrently.
Working around C++ compilation inefficiency
Consider this very simple list view app. There are only 15 lines of actual code in the example but the preprocessed output, which can be produced by passing the -E flag to gcc, is just under 43,000 lines of actual code (as determined by sloccount) or just under 60,000 lines when C++11 mode is enabled (using the '-std=c++0x' flag).
In a language with a proper package/module system (eg. C#, Go or many other languages), processing an import only involves reading some metadata from the already-compiled module rather than re-parsing everything. A proper module system for C++ is in the works but is still some way off. In the meantime, there are
With the small example above, creating a precompiled header which includes just the QStringList header
The steps to enable precompiled headers will depend on the build system you are using. With qmake, this is relatively simple. CMake lacks a simple built-in command for this but there are samples online that we used as a basis.
A downside of precompiled headers is that you are effectively automatically #including an extra header with every file that you build, so a file may compile in a build with precompiled headers but fail to build in one without if the file is missing necessary #includes that are supplied by the precompiled header when enabled. If you're running a CI system is therefore useful to have at least one regular build that is not using precompiled headers.
More efficient build tools
Part of the reason for a gradual creep in built times as a project grows is due to scaling issues with build tools. The amount of time taken for a do-nothing build (ie. running 'make' when everything is up to date) grows noticeably with cmake + make as the total number of targets to build increases. Fortunately for us, engineers on Google Chrome ran into this problem harder and long before we did so they have produced some helpful replacements for the standard tools:
- The Ninja build system is designed to be faster, especially for incremental builds where little changed. Recent versions of CMake have built-in support for generating Ninja build files (use 'cmake -G Ninja' to generate Ninja build files). The difference in build speed for incremental builds where little changed is decent on Mac and Linux but very noticeable on Windows compared to nmake. Prior to Ninja, Qt developers also created jom as a faster alternative to make.
- On Linux, the Gold linker is faster than the traditional ld linker and can often be used as a drop-in replacement.
Reducing total disk I/O
Use faster hardware
- Adding more memory will reduce the likelihood of the build system swapping.
- A good SSD drive will speed up disk I/O, especially for operations which do a lot of random I/O.
- If you have a lot of memory spare you can create a RAMDisk and do the build on that.
Reducing debug info size
- All compilers (MSVC, gcc, clang) have switches to control the amount of debug info that is generated. With gcc/clang these are controlled by the -gXYZ switches.
- Recent versions of clang have a -gline-tables-only option which considerably reduces the amount of debug info that is generated.
Generating fewer binaries for tests
- Each binary will add a number of additional targets to the build system
- Each binary requires a linking step - which can be memory and I/O intensive.
- Each binary generated requires reading/writing additional data to disk. The cost of this depends on how large the generated binary is and how many files need to be processed to assemble the final binary.
We changed the test builds to produce one test binary per source directory instead of one per test class. This was done by replacing the QTEST_MAIN() macro with a substitute which instead declares a '$TESTCLASS_main()' function and registered it in a global map from test class to init function on startup. All of the test classes are then compiled and linked together with a small stub library which declares the 'int main()' function that reads the name of the test to run from the command-line and calls the corresponding '$TESTCLASS_main()' function, forwarding the other command-line arguments to it. This allows multiple Qt test cases to be linked into a single binary which improves build times in several ways:
- The number of linking operations during builds was considerably reduced.
- The total amount of binary data generated on disk was reduced as code that was previously statically linked into the test binary for each test class is now only linked into a single test binary for each group of tests.
- The total number of make steps and targets for the whole project was reduced.
Generating smaller binaries
Note that by doing this you are deferring some of the linking work from build time to runtime and consequently startup will slow down as the number of dynamically loaded libraries increases.
I hope these notes are useful - please let me know if you have other recommendations in the comments. In the meantime, here are a few notes for existing projects which I found useful background reading:
- Notes on accelerating Chromium builds on Windows, Linux and Mac - this doesn't involve Qt but the advice is still quite relevant.
- Notes on improving Firefox's build system.
- An explanation of how a language designed with build performance in mind differs from C++