Hi All,
I spent a little bit of time investigating how to reduce the build time of
gem5. I find that even when I don't edit a single file, it takes scons
around 1 min to tell me the build is up to date. I have identified several
build optimizations. On my system I was able to reduce an empty build (run
scons without any file edits) from 67 seconds to 24 seconds.
I build gem5 on an NFS filesystem. My general use case is to clone gem5
onto my mounted user directory (which creates an obvious IO bottleneck). In
addition, I am able to run experiments out of a local /tmp directory.
I ran experiments in both /tmp and the mounted network directory using the
time command in linux. I found the user and system time out of both the
local and mounted directories to be be the same. The total time (which
includes IO) was around 2 seconds slower than usr+system out of /tmp and
much slower from my mounted drive.
For brevity, I report the experiments ran out of my mounted directory with
usr/system distinguished from total time. The time on /tmp can be
extrapolated by adding ~2 seconds to the usr+system times. I did not use
the parallel scons option since no files are actually being compiled.
A table summarizing the data that I collected is below. Further down, I
explain each data point.
usr+system (s) total (s) 1. normal ruby build 45.2 66.9 2. set NO_HTML
to True 54.3 44.9 3. 1 makeEnv call in src/SConscript 41.5 29.4 4.
decider/--implicit-cache 19.9 23.6
*1. normal ruby build:*
This is just an out the box build of gem5. All data points including this
one use the Ruby memory system.
*cmd: time scons --default=X86 build/test/gem5.debug RUBY=True
PROTOCOL=MI_example NO_HTML=False*
*2. set NO_HTML to True:
* Normal ruby build with NO_HTML set to True.
*cmd: **time scons --default=X86 build/test/gem5.debug RUBY=True
PROTOCOL=MI_example NO_HTML=True*
*
question: Should NO_HTML default to true instead of False?*
*3. 1 makeEnv call in src/SConscript**:*
At the bottom of src/SConscript, there are 4 calls to the makeEnv function
(for debug, opt, fast, and profiling). In this experiment, I commented out
all of the calls to makeEnv except the one for debug. The bottleneck in
makeEnv is line 919: targets = new_env.Program(progname, main_objs +
static_objs). It is also worth noting that it takes ~10s to reach the first
makeEnv call. This 10 seconds seems to be distributed throughout the
src/SConscript file.
*cmd: time scons --default=X86 build/test/gem5.debug RUBY=True
PROTOCOL=MI_example NO_HTML=True*
* *commented 3 makenv calls at bottom of src/Sconscript (lines
960-974)*
*
question: Is it necessary to call makeEnv for all 4 binaries when a single
compile only generates a single binary?*
*4. decider/--implicit-cache**:
*a. By default, scons seems to run a checksum on every file to determine if
it has changed. Scons can be configured to only do a checksum if the time
stamp has changed via the decider function.
b. An --implicit-cache option can be passed through the command line to
reduce the number of dependency checks done on header files (at least this
is my understanding). I believe there is a way to make this the default
behavior as well.
I found that out of /tmp, the -implicit-cache option made a big difference
by itself and the decider function didn't do much. On my mounted directory,
neither of these optimizations did much alone, but together they made a big
difference.
*cmd: time scons --implicit-cache --default=X86 build/test/gem5.debug
RUBY=True PROTOCOL=MI_example NO_HTML=True
** *commented 3 makenv calls at bottom of src/Sconscript (lines
960-974)
** *add line main.Decider('MD5-timestamp') to top-level SConstruct
file (line 183)
**question: Should the decider be configured to default to MD5-timestamp?
Should the implicit-cache option be the default behavior?*
*5. ???*
*question: Can anything else be done to speed up the gem5 build?*
Thanks,
Marc
_______________________________________________
gem5-dev mailing list
[email protected]
http://m5sim.org/mailman/listinfo/gem5-dev
