Salikh, This is very interesting. How much time it takes to build the whole DRLVM from scratch using your script compared to 5 minutes using the old one?
Thanks! On 8/1/07, Salikh Zakirov <[EMAIL PROTECTED]> wrote: > Hi, > > Recently I've been trying out some massive code reorganizations in DRLVM > code in order to improve its modularity (especially concerning > interpreter vs. JIT issue), and that involved mass moving of source > files between components and frequent recompilations, as I am trying > to use resulting compiler and linker errors as a guidance on what else > needs to be done. > > Unfortunately, this kind of exercise proves the current ant-based build > system highly inconvenient, as it has following problems: > > * It neither cleans up stale object files, nor uses object file lists, > so in case .cpp file was moved, the subsequent rebuild will link the > stale object file. > > The obvious workaround is to do a 'build.sh clean', but this is very > time consuming (~5 min on my machine) > > * Even in case of simple changes to several .cpp files, the compilation > process takes unacceptably long time. In the extreme, the 'build.sh' > command with no changes at all takes about 51 seconds on my machine. > > * Ant does not support parallel builds (and my machine is dual-core) > > In order to make the workflow more effcient (so as not to spend too much > time on recompilation), I've created a build system using GNU Make from > scratch. It uses a little bit of makefile generation on-the-fly, but > it is relatively simple: > (a) dependencies are tracked using 'makedepend' > (b) individual file compilation rules are generated from source lists by > a very simple shell script, which generates output like the following: > > $ gen-sources-make *.cpp > OBJECTS := > DEPENDS := > > OBJECTS := $(OBJ)/a.o > $(OBJ)/a.o: a.c > makedepend $(CPPFLAGS) $< > a.d > $(CC) -c -o $@ $< > > ... (repeat the above for each source file) > > The end result is included into the makefile. This is needed for two > reasons: (1) flatten object file directory (i.e. source files from a > deep tree will have a flat list of corresponding object files) > (2) to get the list of object files explicitly, in order to prevent > erroneous linking of stale object files. > > The resulting build system has the following advantages over current > ant-based system: > > + No-recompilation run is fast: 3 sec (with hot cache) vs. 51 sec with > current system > + Stale object files are not linked, even without cleaning > + Parallel make is supported > + The output is nice and clean, without noise > + The configuration files (e.g. vm/em/Makefile) are shorter, and the whole > build process is far easier to understand (to me anyway) > > However, it also has some disadvantages, compared to the ant system > * It only covers building of the native targets (libraries and > executables), so testing and class/jar building is not covered. > If this system is accepted, it will probably be called from ant. > * It does not support multiple file compilation with a single compiler > invocation, and executes compiler once for each source file, which > may be a real performance problem for some platforms > (e.g. Windows/Intel compiler). While some hacks to alleviate this > issue exist for initial compilation, I am not aware of any general > Make-based solution. > > The result is an attached patch. I do not file a JIRA with patch because > it is obviously not (yet) suitable for inclusion. > I would like to hear the comments on the approach and general view on > whether DRLVM needs yet another one build system. > > build/Makefile | 58 +++++++++ > build/gen-sources-make | 55 +++++++++ > build/makerules.inc | 263 > ++++++++++++++++++++++++++++++++++++++++++ > vm/em/Makefile | 25 ++++ > vm/gc_cc/Makefile | 21 ++++ > vm/gc_gen/Makefile | 19 +++ > vm/interpreter/Makefile | 26 ++++ > vm/jitrino/Makefile | 47 ++++++++ > vm/port/Makefile | 28 +++++ > vm/port/src/encoder/Makefile | 11 ++ > vm/thread/Makefile | 30 +++++ > vm/thread/jthread/Makefile | 21 ++++ > vm/vmcore/Makefile | 62 ++++++++++ > vm/vmi/Makefile | 22 ++++ > 14 files changed, 688 insertions(+), 0 deletions(-) > > In addition to disadvantages mentioned above, it is not complete, and > misses following items, though these can be implemented easily > provided there is enough interest (as the system already does everything > I personally need). > > * Configuration is currently written for Linux/ia32/gcc/debug only. > > * Deployment of built targets is currently rather crude (using cp -u) > > * I've just noticed that some changes happened after the I started doing this, > so the Makefiles with source lists will need some update too. > > Comments are welcome. > > -- With best regards, Alexei, ESSD, Intel
