Hi Thomas, > Am 26.11.2014 um 17:35 schrieb Michael Shal: > > If the code for the library isn't changing, it's the build system's > > responsibility to ensure that nothing is done. One of the problems is that > > the build system we use (make) is so broken that we have to clobber > > frequently. > > That's not true. We use CLOBBER because the build scripts are broken, > not make or the concepts behind make. > > I once worked on a software with similar requirements, tons of > auto-generated code, and make-based build system. The Makefiles were > badly written and didn't track dependencies correctly. Consequently we > ran into exactly the same problems as with Gecko: we sometimes had to > cleanup dependencies files manually and often rebuilt too many files. > > Once we fixed the Makefiles, building got fast and we never again had to > fix any dependencies by hand.
The issues I am talking about are indeed specific to make and its concepts, not to our particular use of Makefiles. Although things can certainly be better or worse depending on how the Makefiles are written, there are upper limits to the performance and limitations to the correctness of builds that are inherent with the way make works. For more details see [1], or for some blog posts specifically on clobber builds, see [2][3][4]. [1] http://gittup.org/tup/build_system_rules_and_algorithms.pdf [2] http://gittup.org/blog/2014/03/6-clobber-builds-part-1---missing-dependencies [3] http://gittup.org/blog/2014/05/7-clobber-builds-part-2---fixing-missing-dependencies [4] http://gittup.org/blog/2014/06/8-clobber-builds-part-3---other-clobber-causes (I haven't finished part 4 of the clobber series). > Make is fine and we're not the only project of this size that uses make. > The Linux kernel also does and achieves way better results here. The > problem is in our build scripts. I agree the Linux kernel results are better, but for me it is still insufficient, where a no-op build time of the linux tree is 7.7s. In comparison, with tup a no-op build time is 0.002s [5]. I'm not saying tup is perfect, or the only way forward. But to say that we just need to spend a little more time writing better Makefiles is a myth that needs to be dispelled. Although there are certainly improvements that can be had with that approach, there is no end to it. You will always have slow incremental build times for a project of this size. You will always have cases where you need to clobber and start over from scratch. And you will always try to find clever hacks and work-arounds to avoid these issues (like cd'ing into a subdirectory and running make there. Why is it that you can figure this out faster than a machine?). That's why I outlined these rules clearly in the paper, as well as the algorithms necessary to accomplish them. Anyway, this is probably getting off-topic for dev-platform. If you still believe that we just need to add some extra goop to our Makefiles and we'd all be happy, I'd love to meet up with you in Portland and hear why. -Mike [5] The comparison isn't entirely valid, since with make I'm only building linux, but with tup I'm also building alsa, binutils, busybox, libfuse, gcc, mplayer, ssh, uClibc, and others. _______________________________________________ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
