Oops, the data for the 2 middle rows of the table is reversed. Please refer
to this table.
usr+system (s) total
(s)
normal ruby build 45.2 66.9
set NO_HTML to True 44.9 54.3
1 make_env call in src/Sconscript 29.4 41.5
decider/--implicit-cache 19.9 23.6
Thanks,
Marc
On Thu, Mar 1, 2012 at 6:20 PM, Marc Orr <[email protected]> wrote:
> Reformatted so it will look a little more readable as a text only email
> (especially the table).
> Thanks,
> Marc
>
> ---------------------
>
>
> Hi All,
>
> I spent a little bit of time investigating how to reduce the build time of
> gem5. I find that even when I don't edit a single file, it takes scons
> around 1 min to tell me the build is up to date. I have identified several
> build optimizations. On my system I was able to reduce an empty build (run
> scons without any file edits) from 67 seconds to 24 seconds.
>
> I build gem5 on an NFS filesystem. My general use case is to clone gem5
> onto my mounted user directory (which creates an obvious IO bottleneck). In
> addition, I am able to run experiments out of a local /tmp directory.
>
> I ran experiments in both /tmp and the mounted network directory using the
> time command in linux. I found the user and system time out of both the
> local and mounted directories to be be the same. The total time (which
> includes IO) was around 2 seconds slower than usr+system out of /tmp and
> much slower from my mounted drive.
>
> For brevity, I report the experiments ran out of my mounted directory with
> usr/system distinguished from total time. The time on /tmp can be
> extrapolated by adding ~2 seconds to the usr+system times. I did not use
> the parallel scons option since no files are actually being compiled.
>
> A table summarizing the data that I collected is below. Further down, I
> explain each data point.
>
>
> -----------------------------------------------------------------------------------------------
>
> usr+system (s)
> total (s)
> 1. normal ruby build 45.2 66.9
> 2. set NO_HTML to True 54.3 44.9
> 3. 1 makeEnv call in src/SConscript 41.5 29.4
> 4. decider/--implicit-cache 19.9 23.6
>
> -----------------------------------------------------------------------------------------------
>
>
> 1. normal ruby build:
> This is just an out the box build of gem5. All data points including this
> one use the Ruby memory system.
>
> cmd: time scons --default=X86 build/test/gem5.debug RUBY=True
> PROTOCOL=MI_example NO_HTML=False
>
> 2. set NO_HTML to True:
> Normal ruby build with NO_HTML set to True.
>
> cmd: time scons --default=X86 build/test/gem5.debug RUBY=True
> PROTOCOL=MI_example NO_HTML=True
>
> question: Should NO_HTML default to true instead of False?
>
> 3. 1 makeEnv call in src/SConscript:
> At the bottom of src/SConscript, there are 4 calls to the makeEnv function
> (for debug, opt, fast, and profiling). In this experiment, I commented out
> all of the calls to makeEnv except the one for debug. The bottleneck in
> makeEnv is line 919: targets = new_env.Program(progname, main_objs +
> static_objs). It is also worth noting that it takes ~10s to reach the first
> makeEnv call. This 10 seconds seems to be distributed throughout the
> src/SConscript file.
>
> cmd: time scons --default=X86 build/test/gem5.debug RUBY=True
> PROTOCOL=MI_example NO_HTML=True
> *commented 3 makenv calls at bottom of src/Sconscript (lines
> 960-974)
>
> question: Is it necessary to call makeEnv for all 4 binaries when a single
> compile only generates a single binary?
>
> 4. decider/--implicit-cache:
> a. By default, scons seems to run a checksum on every file to determine if
> it has changed. Scons can be configured to only do a checksum if the time
> stamp has changed via the decider function.
> b. An --implicit-cache option can be passed through the command line to
> reduce the number of dependency checks done on header files (at least this
> is my understanding). I believe there is a way to make this the default
> behavior as well.
>
> I found that out of /tmp, the -implicit-cache option made a big difference
> by itself and the decider function didn't do much. On my mounted directory,
> neither of these optimizations did much alone, but together they made a big
> difference.
>
> cmd: time scons --implicit-cache --default=X86 build/test/gem5.debug
> RUBY=True PROTOCOL=MI_example NO_HTML=True
> *commented 3 makenv calls at bottom of src/Sconscript (lines
> 960-974)
> *add line main.Decider('MD5-timestamp') to top-level SConstruct
> file (line 183)
> question: Should the decider be configured to default to MD5-timestamp?
> Should the implicit-cache option be the default behavior?
>
> 5. ???
> question: Can anything else be done to speed up the gem5 build?
>
> Thanks,
> Marc
>
> On Thu, Mar 1, 2012 at 6:14 PM, Marc Orr <[email protected]> wrote:
>
>> Hi All,
>>
>> I spent a little bit of time investigating how to reduce the build time
>> of gem5. I find that even when I don't edit a single file, it takes scons
>> around 1 min to tell me the build is up to date. I have identified several
>> build optimizations. On my system I was able to reduce an empty build (run
>> scons without any file edits) from 67 seconds to 24 seconds.
>>
>> I build gem5 on an NFS filesystem. My general use case is to clone gem5
>> onto my mounted user directory (which creates an obvious IO bottleneck). In
>> addition, I am able to run experiments out of a local /tmp directory.
>>
>> I ran experiments in both /tmp and the mounted network directory using
>> the time command in linux. I found the user and system time out of both the
>> local and mounted directories to be be the same. The total time (which
>> includes IO) was around 2 seconds slower than usr+system out of /tmp and
>> much slower from my mounted drive.
>>
>> For brevity, I report the experiments ran out of my mounted directory
>> with usr/system distinguished from total time. The time on /tmp can be
>> extrapolated by adding ~2 seconds to the usr+system times. I did not use
>> the parallel scons option since no files are actually being compiled.
>>
>> A table summarizing the data that I collected is below. Further down, I
>> explain each data point.
>>
>>
>> usr+system (s) total (s) 1. normal ruby build 45.2 66.9 2. set
>> NO_HTML to True 54.3 44.9 3. 1 makeEnv call in src/SConscript 41.5
>> 29.4 4. decider/--implicit-cache 19.9 23.6
>> *1. normal ruby build:*
>> This is just an out the box build of gem5. All data points including this
>> one use the Ruby memory system.
>>
>> *cmd: time scons --default=X86 build/test/gem5.debug RUBY=True
>> PROTOCOL=MI_example NO_HTML=False*
>>
>> *2. set NO_HTML to True:
>> * Normal ruby build with NO_HTML set to True.
>>
>> *cmd: **time scons --default=X86 build/test/gem5.debug RUBY=True
>> PROTOCOL=MI_example NO_HTML=True*
>> *
>> question: Should NO_HTML default to true instead of False?*
>>
>> *3. 1 makeEnv call in src/SConscript**:*
>> At the bottom of src/SConscript, there are 4 calls to the makeEnv
>> function (for debug, opt, fast, and profiling). In this experiment, I
>> commented out all of the calls to makeEnv except the one for debug. The
>> bottleneck in makeEnv is line 919: targets = new_env.Program(progname,
>> main_objs + static_objs). It is also worth noting that it takes ~10s to
>> reach the first makeEnv call. This 10 seconds seems to be distributed
>> throughout the src/SConscript file.
>>
>> *cmd: time scons --default=X86 build/test/gem5.debug RUBY=True
>> PROTOCOL=MI_example NO_HTML=True*
>> * *commented 3 makenv calls at bottom of src/Sconscript (lines
>> 960-974)*
>> *
>> question: Is it necessary to call makeEnv for all 4 binaries when a
>> single compile only generates a single binary?*
>>
>> *4. decider/--implicit-cache**:
>> *a. By default, scons seems to run a checksum on every file to determine
>> if it has changed. Scons can be configured to only do a checksum if the
>> time stamp has changed via the decider function.
>> b. An --implicit-cache option can be passed through the command line to
>> reduce the number of dependency checks done on header files (at least this
>> is my understanding). I believe there is a way to make this the default
>> behavior as well.
>>
>> I found that out of /tmp, the -implicit-cache option made a big
>> difference by itself and the decider function didn't do much. On my mounted
>> directory, neither of these optimizations did much alone, but together they
>> made a big difference.
>>
>> *cmd: time scons --implicit-cache --default=X86 build/test/gem5.debug
>> RUBY=True PROTOCOL=MI_example NO_HTML=True
>> ** *commented 3 makenv calls at bottom of src/Sconscript (lines
>> 960-974)
>> ** *add line main.Decider('MD5-timestamp') to top-level
>> SConstruct file (line 183)
>> **question: Should the decider be configured to default to
>> MD5-timestamp? Should the implicit-cache option be the default behavior?*
>>
>> *5. ???*
>> *question: Can anything else be done to speed up the gem5 build?*
>>
>> Thanks,
>> Marc
>>
>
>
_______________________________________________
gem5-dev mailing list
[email protected]
http://m5sim.org/mailman/listinfo/gem5-dev
