On Fri, Aug 02, 2013 at 04:32:08PM -0700, Gregory Szorc wrote: > On 8/2/13 3:38 PM, Ehsan Akhgari wrote: > >Hmm. I'm not sure if the number of source files is directly correlated > >to build times, but yeah there's clearly a trend here! > > I concede a lines of code count would be a better indicator. I'm lazy. > > >># Header dependency hell > >I have been playing with an idea in my head about this. What if we had > >a list of the most popular headers in our tree, and we looked through > >them and tried to cut down the number of #includes in the headers? That > >should help create more isolated sub-graphs and hopefully help with > >breaking the most severe dependency chains. > > > >Writing a tool to spit out this information should be fairly easy. > > I'll try to get a tool in the tree for people to run. > https://bugzilla.mozilla.org/show_bug.cgi?id=901132 > > >># Increased reliance on C++ language features > >But I'm not convinced at all about the C++11 features contributing to > >this. I cannot think of any reason at all why that should be the case > >for the things that we've started to use. Do you have any evidence to > >implicate some of those features? > > No. Just my general distrust of new/young vs mature software. > > >># Clobbers are more frequent and more annoying > >This should be relatively "easy" to address (compared to the other > >things that we can do, of course). I assert that every time we touch > >the CLOBBER file, it's because the build system could not figure out the > >dependencies properly. Fortunately we can easily log the CLOBBER file > >and go back in time and find all of the patches that included CLOBBER > >modifications and debug the build dependency issues. Has there been any > >effort to address these issues by looking at the testcases that we have > >in form of patches? > > To some degree, yes. > https://bugzilla.mozilla.org/show_bug.cgi?id=890744 is a good > example. Vacation schedules didn't align for quick action. There may > also be a pymake bug or two involved. > > Also, you could say people have been touching CLOBBER prematurely. I > know there are a few cases where CLOBBER was touched in hopes it > fixed a problem, didn't, and the commit history was left with a > changeset that changed CLOBBER. > > >># Slowness Summary > >Every time that we don't utilize 100% of our cores during the build > >process, that's an unnecessary slowdown. That consistently wastes a lot > >of time during every build, and it also means that we can't address this > >by getting more powerful machines. :( > > Right. If you plot CPU usage vs time, we can make the build faster > by "filling out the box" and using 100% of all cores or by > decreasing the total number of required CPU cycles to build. We have > chosen to focus mostly on the former because optimizing build > actions can be a lot of work. We've got lucky in some cases (e.g. > WebIDLs in bug 861587). I fear compiling C++ will be much harder. > I'm hoping PCH and fixing dependency hell are "medium-hanging" > fruits. > > I also have measurements that show we peak out at certain > concurrency levels. The trend in CPUs is towards more cores, not > higher clock speed. So focusing on effective core usage will > continue to be important. Derecursifying the build will allow us to > use more cores because make won't be starved during directory > traversal. Remember, concurrent make only works within the same > directory or for directories under PARALLEL_DIRS. Different > top-level directories during tier traversal (e.g. dom and xpcom) are > executed sequentially. > > >># Building faster > >> > >>One of our Q3 goals is to replace the "export tier" with something more > >>efficient. More on tiers at [1]. This should make builds faster, > >>especially on pymake. Just earlier this week we made WebIDL and XPIDL > >>code generation concurrent. Before, they executed serially, failing to > >>utilize multiple CPU cores. Next steps are XPIDL code gen, installing > >>headers, and preprocessing. This is all tracked in bug 892644. > > > >Out of curiosity, why was the export tier the fist target for this? I > >may lack context here, but the slowest tier that we have is the platform > >libs tier. Wouldn't focusing on that have given us the biggest possible > >bang for the buck? > > Making platform libs faster will without a doubt have the biggest > impact. We chose to start with export first for a few reasons. > > First, it's simple. We had to start somewhere. platform/libs is a > magnitude more complex. We are making major refactorings in export > already and we felt it best to prove out concepts with export rather > that going for the hardest problem first. > > Second, export is mostly standalone targets. We would like to "port" > the build backend bottom up instead of top down so we can make the > dependencies right from the beginning. If we started with > platform/lib, we'd have to hack something together now and revamp it > with proper dependencies later. > > Third, export is horribly inefficient. pymake spends an absurd > amount of time traversing directories, parsing make files and doing > very little for each directory in the export tier. Platform, by > contrast, tends to have longer-running jobs so cores aren't starved > as often. When you look at a graph of resource usage for the > different tiers, this is obvious. By removing export, we'll be > removing a traversal from the tiers. This could decrease no-op build > times by as much as 30%! > > Fourth, removing export actually makes libs more efficient! > Specifically, moving XPIDL code gen out of export and libs and into > the "precompile" phase/tier makes libs faster! I was surprised when > I first saw this. Essentially, the XPIDL codegen tasks during libs > block their dependent C++ compile targets from running. So by moving > XPIDL out of libs, C++ compilation starts immediately without having > to wait on XPIDL. > > Fifth, it allows simultaneous development on the build system. For > example, I'm currently spending a lot of time refactoring export > while joey and mshal are moving pieces related to libs to moz.build, > paving the road for work there. We largely don't get in each others' > way.
Sixth, we're currently starting top level builds with the removal of everything make export does in dist/, and as a result we're rebuilding more things than we should because some files in dist/ end up newer. The removal of make export will also fix this, which also will lead in faster builds. Mike _______________________________________________ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
