Thank you so much for investing time and effort into this area! Improvements to build times are always always welcome.
I seem to recall we collect opt-in Telemetry on things like build times. If so, have we noticed any changes in those graphs? -Mike On Mon, 14 Dec 2020 at 11:11, Botond Ballo <bba...@mozilla.com> wrote: > Hey Simon, > > Thanks a lot for working on this! It's hard to notice the effect of > individual changes in this area, but in aggregate they can add up to a > significant build speed improvement. > > Botond > > On Mon, Dec 14, 2020 at 6:23 AM Simon Giesecke <sgiese...@mozilla.com> > wrote: > > > > tl;dr Build times on all platforms, in particular for incremental > builds, > > have decreased in the last weeks by landing several cleanups to C++ > include > > dependencies that reduce the aggregated number of included files by about > > 30%. > > > > Hi, > > > > Did you notice a reduction in build times lately? This might not be a > > coincidence. In this mail, I want to provide some details on the > > improvements made. I want to thank everyone who contributed to this > through > > up-front discussions and reviews. > > > > Recently I landed a number of patches on Bug 1676346 [1] that in various > > ways clean up C++ include directives in our codebase. Some landed ca. 3 > > weeks ago, some landed last week. Overall, these reduce the aggregated > > number of included non-system header files in a full build by about 30% > on > > Linux. I don't have numbers for other platforms, but they should benefit > as > > well. On my machine, this reduced the time for a clobber build by about > > 10%. While that might go unnoticed, incremental builds are often sped up > > even a lot more, since the number of translation units that need to be > > rebuilt decreases. E.g. the number of translation units that include > > dom/base/Document.h reduced by ca. 52%, resulting in a build time > reduction > > of 48% on my machine after modifying that. Your mileage may vary. > > > > While this might not spare you from buying new hardware, it will make > > builds faster regardless of the hardware you are running on, and > hopefully > > increase productivity. If you want to share your experiences with me, > > please get in touch! > > > > You might be curious what I did to achieve that, or how you can > contribute > > to reducing build times yourself. It's a combination of things, most > > importantly three things: > > 1. Remove unused include directives > > 2. Split some headers > > 3. Use forward declarations more > > 4. Hide definitions to allow using forward declarations even more > > > > About 1: I found there are several include directives that are not needed > > at all. They could simply be removed. However, the majority of cases > were a > > bit more complex, because of a lot of missing include directives. When > > removing an include for X.h from a header file Y.h that doesn't need it, > > another file that included Y.h might need X.h. Or, Y.h itself might need > > something from a header indirectly included from X.h. Or similar cases. > > This meant quite a lot of more include directives for more basic things > > needed to be added to ensure the build doesn't break. > > > > About 2: Some headers have a lot of dependencies, but only relatively few > > users of that header need them. One example was IPCMessageUtils.h, which > is > > included by all files generated by IPDL, which also contained a lot of > > specializations of the ParamTraits template that are needed only by few > > files. Apart from some very basic specializations, these were moved to > the > > new EnumSerializers.h and IPCMessageUtilsSpecializations.h as well as > some > > existing headers, so that the remaining IPCMessageUtils.h header has only > > much more limited dependencies. > > > > About 3: Our coding style favors forward declarations over inclusion of > the > > full definition of types where possible. I replaced the inclusion of > header > > files containing full definitions of types by forward declarations at a > > number of places where this is sufficient, e.g. because they are only > used > > in function signatures. It's worth noting that there were also a number > of > > cases where a forward declaration was present, but actually the full > > definition is required, e.g. when a type is used as a base class or as > the > > value type of a data member, or an inline function body dereferences > into a > > member. > > > > About 4: As mentioned in the last point, inline function bodies often > > require the inclusion of additional headers because they dereference into > > types of which otherwise only forward declarations were necessary. > Similar > > considerations apply to private (nested) classes. Some of those were > moved > > to the corresponding implementation files to hide these dependencies, and > > reduce the number of necessary includes in the header files. > > > > I was using some tools to support this, notably ClangBuildAnalyzer [2] > and > > include-what-you-use [3]. ClangBuildAnalyzer helped to detect headers > that > > are expensive to parse throughout the build, and direct efforts to reduce > > those specifically. But there remains a long tail of similar things that > > can and should be fixed. include-what-you-use helps to identify the > headers > > from which a file uses declarations to avoid depending on indirectly > > included files which might disappear from the include chain by unrelated > > changes. These tools are not readymade for everyday use, but I will try > to > > provide some ideas for using them effectively later on. > > > > Best wishes > > Simon > > > > [1] https://bugzilla.mozilla.org/show_bug.cgi?id=1676346 > > [2] https://github.com/aras-p/ClangBuildAnalyzer > > [3] https://github.com/include-what-you-use/include-what-you-use > > _______________________________________________ > > dev-platform mailing list > > dev-platform@lists.mozilla.org > > https://lists.mozilla.org/listinfo/dev-platform > _______________________________________________ > dev-platform mailing list > dev-platform@lists.mozilla.org > https://lists.mozilla.org/listinfo/dev-platform > _______________________________________________ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
