Re: [U-Boot] [PATCH] Reduce build times
Hi Daniel, Wolfgang, On Thursday 03 November 2011 08:55 PM, Daniel Schwierzeck wrote: > Hi Wolfgang, [snip ..] > > Conclusion: > - complete build time reduced from 1m11s to 20s > - incremental rebuild time reduced from 20s to 3s > - cc-option calls reduced from 3024 to 8 > - execve calls reduced from 16502 to 3329 Results for omap4 sdp build: Build machine: Intel Core i5 2.5 GHz, 3M cache, 4GB DDR3 Un-modified HEAD: real0m21.463s user0m31.278s sys 0m9.281s With only Wolfgang's patch: real0m11.226s user0m23.937s sys 0m4.200s With only Daniel's patch: real0m10.842s user0m21.725s sys 0m2.532s With both patches: real0m8.306s user0m21.201s sys 0m2.408s Looks like both patches are helping. Thanks!! br, Aneesh > > Best regards, > Daniel > ___ > U-Boot mailing list > U-Boot@lists.denx.de > http://lists.denx.de/mailman/listinfo/u-boot ___ U-Boot mailing list U-Boot@lists.denx.de http://lists.denx.de/mailman/listinfo/u-boot
Re: [U-Boot] [PATCH] Reduce build times
Dear Daniel Schwierzeck, In message you wrote: > > Conclusion: > - complete build time reduced from 1m11s to 20s > - incremental rebuild time reduced from 20s to 3s > - cc-option calls reduced from 3024 to 8 > - execve calls reduced from 16502 to 3329 That's really cool. Can we please add another two or three of such optimizations? :-) Best regards, Wolfgang Denk -- DENX Software Engineering GmbH, MD: Wolfgang Denk & Detlev Zundel HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany Phone: (+49)-8142-66989-10 Fax: (+49)-8142-66989-80 Email: w...@denx.de CONSUMER NOTICE: Because of the "Uncertainty Principle," It Is Impossible for the Consumer to Find Out at the Same Time Both Precisely Where This Product Is and How Fast It Is Moving. ___ U-Boot mailing list U-Boot@lists.denx.de http://lists.denx.de/mailman/listinfo/u-boot
Re: [U-Boot] [PATCH] Reduce build times
Hi Wolfgang, On Wed, Nov 2, 2011 at 11:48 PM, Wolfgang Denk wrote: > Dear Daniel Schwierzeck, > > In message > you > wrote: >> >> On Wed, Nov 2, 2011 at 7:54 AM, Wolfgang Denk wrote: >> > U-Boot Makefiles contain a number of tests for compiler features etc. >> > which so far are executed again and again. =C2=A0On some architectures >> > (especially ARM) this results in a large number of calls to gcc. >> > >> > This patch makes sure to run such tests only once, thus largely >> > reducing the number of "execve" system calls. >> > >> >> maybe you want to try this experimental patch. >> http://patchwork.ozlabs.org/patch/123313/ >> >> It significantly reduces the count of gcc calls by caching the results. >> This also improves compilation times. > > Do you suggest this in addition or instead of the patch I posted? > > Can you provide some measurements of build times and/or execve system > calls? I ran some additonal tests with interesting results. Board: ARM, Tegra2, seaboard Toolchain: Sourcery G++ Lite 2011.03-41 for ARM GNU/Linux Workstation: Core 2 Duo E6600 @2,4 Ghz, 4 GB, x86_64 I patched the cc-option macro to count all calls like this: cc-option = $(shell if $(CC) $(CFLAGS) $(1) -S -o /dev/null -xc /dev/null \ - > /dev/null 2>&1; then echo "$(1)"; else echo "$(2)"; fi ;) + > /dev/null 2>&1; then echo "$(1)"; echo "$1" >> $(OBJTREE)/cc-option; else echo "$(2)"; fi ;) I ran the steps below for following source trees: - unmodified HEAD - only your patch - only my patch - both patches combined Steps: Complete build: -> git clean -xdf -> CROSS_COMPILE=/opt/codesourcery/arm-2011.03/bin/arm-none-linux-gnueabi- make seaboard_config -> time CROSS_COMPILE=/opt/codesourcery/arm-2011.03/bin/arm-none-linux-gnueabi- CACHE_CC_OPTIONS=y make -s -> cat cc-option | wc -l Incremental rebuild: -> time CROSS_COMPILE=/opt/codesourcery/arm-2011.03/bin/arm-none-linux-gnueabi- CACHE_CC_OPTIONS=y make -s Complete build with strace: -> git clean -xdf -> CROSS_COMPILE=/opt/codesourcery/arm-2011.03/bin/arm-none-linux-gnueabi- make seaboard_config -> CROSS_COMPILE=/opt/codesourcery/arm-2011.03/bin/arm-none-linux-gnueabi- CACHE_CC_OPTIONS=y strace -f -e trace=execve -o strace.out make -s -> grep execve strace.out | wc -l Results: unmodified HEAD: real1m11.540s user2m7.170s sys 0m19.840s cc-option calls 3024 real0m20.176s user0m39.260s sys 0m6.480s execve calls 16502 only your patch: real0m32.371s user0m47.440s sys 0m7.900s cc-option calls 864 real0m9.606s user0m16.890s sys 0m2.940s execve calls 5906 only my patch: real0m28.187s user0m56.030s sys 0m7.820s cc-option calls 20 real0m5.013s user0m13.300s sys 0m2.200s execve calls 7415 both patches combined: real0m19.777s user0m28.010s sys 0m4.100s cc-option calls 8 real0m2.902s user0m6.400s sys 0m1.070s execve calls 3329 Conclusion: - complete build time reduced from 1m11s to 20s - incremental rebuild time reduced from 20s to 3s - cc-option calls reduced from 3024 to 8 - execve calls reduced from 16502 to 3329 Best regards, Daniel ___ U-Boot mailing list U-Boot@lists.denx.de http://lists.denx.de/mailman/listinfo/u-boot
Re: [U-Boot] [PATCH] Reduce build times
On Wednesday 02 November 2011 02:54:02 Wolfgang Denk wrote: > U-Boot Makefiles contain a number of tests for compiler features etc. > which so far are executed again and again. On some architectures > (especially ARM) this results in a large number of calls to gcc. seems to shave ~10% off for Blackfin boards Acked-by: Mike Frysinger > Note: There is further potential for build time reductions by > performing similar optimizations for a number of $(shell ...) > constructs in the Makefiles, but I have no good ways to test these at > the moment so this is left as exercise for the respective > architecture maintainers (mostly blackfin and coldfire, AFAICT) -- wd Blackfin does two $(shell), one of which i already cache. the other, i should be able to send a patch for. -mike signature.asc Description: This is a digitally signed message part. ___ U-Boot mailing list U-Boot@lists.denx.de http://lists.denx.de/mailman/listinfo/u-boot
Re: [U-Boot] [PATCH] Reduce build times
HI Wolfgang, 2011/11/2 Wolfgang Denk : > U-Boot Makefiles contain a number of tests for compiler features etc. > which so far are executed again and again. On some architectures > (especially ARM) this results in a large number of calls to gcc. board before after reduction adp-ag101 7259 70592.7% Tested-by: Macpaul Lin Thanks -- Best regards, Macpaul Lin ___ U-Boot mailing list U-Boot@lists.denx.de http://lists.denx.de/mailman/listinfo/u-boot
Re: [U-Boot] [PATCH] Reduce build times
Hi Wolfgang, On 02.11.2011 23:48, Wolfgang Denk wrote: > Dear Daniel Schwierzeck, > > In > message > you wrote: >> >> On Wed, Nov 2, 2011 at 7:54 AM, Wolfgang Denk wrote: >>> U-Boot Makefiles contain a number of tests for compiler features etc. >>> which so far are executed again and again. =C2=A0On some architectures >>> (especially ARM) this results in a large number of calls to gcc. >>> >>> This patch makes sure to run such tests only once, thus largely >>> reducing the number of "execve" system calls. >>> >> >> maybe you want to try this experimental patch. >> http://patchwork.ozlabs.org/patch/123313/ >> >> It significantly reduces the count of gcc calls by caching the results. >> This also improves compilation times. > > Do you suggest this in addition or instead of the patch I posted? as an additional but separate patch to further reduce the execution time of MAKEALL. > > Can you provide some measurements of build times and/or execve system > calls? I have attached the results of some MAKEALL runs in the patch mail (I cc-ed you). Best regards, Daniel ___ U-Boot mailing list U-Boot@lists.denx.de http://lists.denx.de/mailman/listinfo/u-boot
Re: [U-Boot] [PATCH] Reduce build times
Dear Daniel Schwierzeck, In message you wrote: > > On Wed, Nov 2, 2011 at 7:54 AM, Wolfgang Denk wrote: > > U-Boot Makefiles contain a number of tests for compiler features etc. > > which so far are executed again and again. =C2=A0On some architectures > > (especially ARM) this results in a large number of calls to gcc. > > > > This patch makes sure to run such tests only once, thus largely > > reducing the number of "execve" system calls. > > > > maybe you want to try this experimental patch. > http://patchwork.ozlabs.org/patch/123313/ > > It significantly reduces the count of gcc calls by caching the results. > This also improves compilation times. Do you suggest this in addition or instead of the patch I posted? Can you provide some measurements of build times and/or execve system calls? Best regards, Wolfgang Denk -- DENX Software Engineering GmbH, MD: Wolfgang Denk & Detlev Zundel HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany Phone: (+49)-8142-66989-10 Fax: (+49)-8142-66989-80 Email: w...@denx.de I can type faster than I can move a mouse, so I find menu-driven drawing packages time consuming and frustrating. - W. R. Stevens ___ U-Boot mailing list U-Boot@lists.denx.de http://lists.denx.de/mailman/listinfo/u-boot
Re: [U-Boot] [PATCH] Reduce build times
Hi Wolfgang, On Wed, Nov 2, 2011 at 7:54 AM, Wolfgang Denk wrote: > U-Boot Makefiles contain a number of tests for compiler features etc. > which so far are executed again and again. On some architectures > (especially ARM) this results in a large number of calls to gcc. > > This patch makes sure to run such tests only once, thus largely > reducing the number of "execve" system calls. > maybe you want to try this experimental patch. http://patchwork.ozlabs.org/patch/123313/ It significantly reduces the count of gcc calls by caching the results. This also improves compilation times. Best regards, Daniel ___ U-Boot mailing list U-Boot@lists.denx.de http://lists.denx.de/mailman/listinfo/u-boot
Re: [U-Boot] [PATCH] Reduce build times
On Wed, Nov 2, 2011 at 8:37 AM, Tom Rini wrote: > On Wed, Nov 2, 2011 at 7:49 AM, Premi, Sanjeev wrote: >>> -Original Message- >>> From: u-boot-boun...@lists.denx.de >>> [mailto:u-boot-boun...@lists.denx.de] On Behalf Of Wolfgang Denk >>> Sent: Wednesday, November 02, 2011 12:24 PM >>> To: u-boot@lists.denx.de >>> Cc: Graeme Russ; Kumar Gala; Albert Aribaud; Andy Fleming >>> Subject: [U-Boot] [PATCH] Reduce build times >>> >>> U-Boot Makefiles contain a number of tests for compiler features etc. >>> which so far are executed again and again. On some architectures >>> (especially ARM) this results in a large number of calls to gcc. >>> >>> This patch makes sure to run such tests only once, thus largely >>> reducing the number of "execve" system calls. >>> >>> Example: number of "execve" system calls for building the "P2020DS" >>> (Power Architecture) and "qong" (ARM) boards, measured as: >>> -> strace -f -e trace=execve -o /tmp/foo ./MAKEALL >>> -> grep execve /tmp/foo | wc -l >>> >>> Before: After: Reduction: >>> == >>> P2020DS 20555 15205 -26% >>> qong 31692 14490 -54% >>> >>> As a result, built times are significantly reduced, typically by >>> 30...50%. >>> >>> Signed-off-by: Wolfgang Denk Tested-by: Simon Glass >>> Cc: Andy Fleming >>> Cc: Kumar Gala >>> Cc: Albert Aribaud >>> cc: Graeme Russ >>> cc: Mike Frysinger >>> --- >> >> Results for OMAP3EVM. >> (Tried 5 times just to be sure as I see >50% reduction.) >> >> Before After >> -- -- >> real 109.03 49.78 >> user 71.43 29.06 >> sys 26.83 7.66 > > Over here omap3_evm wall-clock time on make -j12 goes from 27sec to 10sec. > > -- > Tom > ___ > U-Boot mailing list > U-Boot@lists.denx.de > http://lists.denx.de/mailman/listinfo/u-boot > For Tegra2 Seaboard (armv7) and -j15 or so: before and after times: full build (clobber, config) 17.177s -> 7.060s incremental build 7.432s -> 2.267s Thank you! Regards, Simon ___ U-Boot mailing list U-Boot@lists.denx.de http://lists.denx.de/mailman/listinfo/u-boot
Re: [U-Boot] [PATCH] Reduce build times
On Wed, Nov 2, 2011 at 7:49 AM, Premi, Sanjeev wrote: >> -Original Message- >> From: u-boot-boun...@lists.denx.de >> [mailto:u-boot-boun...@lists.denx.de] On Behalf Of Wolfgang Denk >> Sent: Wednesday, November 02, 2011 12:24 PM >> To: u-boot@lists.denx.de >> Cc: Graeme Russ; Kumar Gala; Albert Aribaud; Andy Fleming >> Subject: [U-Boot] [PATCH] Reduce build times >> >> U-Boot Makefiles contain a number of tests for compiler features etc. >> which so far are executed again and again. On some architectures >> (especially ARM) this results in a large number of calls to gcc. >> >> This patch makes sure to run such tests only once, thus largely >> reducing the number of "execve" system calls. >> >> Example: number of "execve" system calls for building the "P2020DS" >> (Power Architecture) and "qong" (ARM) boards, measured as: >> -> strace -f -e trace=execve -o /tmp/foo ./MAKEALL >> -> grep execve /tmp/foo | wc -l >> >> Before: After: Reduction: >> == >> P2020DS 20555 15205 -26% >> qong 31692 14490 -54% >> >> As a result, built times are significantly reduced, typically by >> 30...50%. >> >> Signed-off-by: Wolfgang Denk >> Cc: Andy Fleming >> Cc: Kumar Gala >> Cc: Albert Aribaud >> cc: Graeme Russ >> cc: Mike Frysinger >> --- > > Results for OMAP3EVM. > (Tried 5 times just to be sure as I see >50% reduction.) > > Before After > -- -- > real 109.03 49.78 > user 71.43 29.06 > sys 26.83 7.66 Over here omap3_evm wall-clock time on make -j12 goes from 27sec to 10sec. -- Tom ___ U-Boot mailing list U-Boot@lists.denx.de http://lists.denx.de/mailman/listinfo/u-boot
Re: [U-Boot] [PATCH] Reduce build times
> -Original Message- > From: u-boot-boun...@lists.denx.de > [mailto:u-boot-boun...@lists.denx.de] On Behalf Of Wolfgang Denk > Sent: Wednesday, November 02, 2011 12:24 PM > To: u-boot@lists.denx.de > Cc: Graeme Russ; Kumar Gala; Albert Aribaud; Andy Fleming > Subject: [U-Boot] [PATCH] Reduce build times > > U-Boot Makefiles contain a number of tests for compiler features etc. > which so far are executed again and again. On some architectures > (especially ARM) this results in a large number of calls to gcc. > > This patch makes sure to run such tests only once, thus largely > reducing the number of "execve" system calls. > > Example: number of "execve" system calls for building the "P2020DS" > (Power Architecture) and "qong" (ARM) boards, measured as: > -> strace -f -e trace=execve -o /tmp/foo ./MAKEALL > -> grep execve /tmp/foo | wc -l > > Before: After: Reduction: > == > P2020DS 20555 15205 -26% > qong 31692 14490 -54% > > As a result, built times are significantly reduced, typically by > 30...50%. > > Signed-off-by: Wolfgang Denk > Cc: Andy Fleming > Cc: Kumar Gala > Cc: Albert Aribaud > cc: Graeme Russ > cc: Mike Frysinger > --- Results for OMAP3EVM. (Tried 5 times just to be sure as I see >50% reduction.) Before After -- -- real109.03 49.78 user 71.43 29.06 sys 26.83 7.66 Compiled u-boot works fine on the board as well. Tested-by: Sanjeev Premi [snip]...[snip] ___ U-Boot mailing list U-Boot@lists.denx.de http://lists.denx.de/mailman/listinfo/u-boot
Re: [U-Boot] [PATCH] Reduce build times
Am 02.11.2011 07:54, schrieb Wolfgang Denk: > U-Boot Makefiles contain a number of tests for compiler features etc. > which so far are executed again and again. On some architectures > (especially ARM) this results in a large number of calls to gcc. > > This patch makes sure to run such tests only once, thus largely > reducing the number of "execve" system calls. > > Example: number of "execve" system calls for building the "P2020DS" > (Power Architecture) and "qong" (ARM) boards, measured as: > -> strace -f -e trace=execve -o /tmp/foo ./MAKEALL > -> grep execve /tmp/foo | wc -l > > Before: After: Reduction: > == > P2020DS 20555 15205 -26% > qong 31692 14490 -54% > > As a result, built times are significantly reduced, typically by > 30...50%. > > Signed-off-by: Wolfgang Denk > Cc: Andy Fleming > Cc: Kumar Gala > Cc: Albert Aribaud > cc: Graeme Russ > cc: Mike Frysinger > --- Nice. Some additional numbers: zmx25: make --- real1m47.546s 0m57.213s -53% user1m39.698s 0m54.831s sys 0m24.798s 0m9.509s zmx25: make -j2 --- real0m56.791s 0m32.187s -57% user1m38.478s 0m55.571s sys 0m24.522s 0m9.513s Tested-by: Matthias Weisser Matthias ___ U-Boot mailing list U-Boot@lists.denx.de http://lists.denx.de/mailman/listinfo/u-boot
Re: [U-Boot] [PATCH] Reduce build times
Hi Wolfgang, On 02/11/11 17:54, Wolfgang Denk wrote: > U-Boot Makefiles contain a number of tests for compiler features etc. > which so far are executed again and again. On some architectures > (especially ARM) this results in a large number of calls to gcc. > > This patch makes sure to run such tests only once, thus largely > reducing the number of "execve" system calls. > > Example: number of "execve" system calls for building the "P2020DS" > (Power Architecture) and "qong" (ARM) boards, measured as: > -> strace -f -e trace=execve -o /tmp/foo ./MAKEALL > -> grep execve /tmp/foo | wc -l > > Before: After: Reduction: > == > P2020DS 20555 15205 -26% > qong 31692 14490 -54% > > As a result, built times are significantly reduced, typically by > 30...50%. > > Signed-off-by: Wolfgang Denk > Cc: Andy Fleming > Cc: Kumar Gala > Cc: Albert Aribaud > cc: Graeme Russ > cc: Mike Frysinger > --- Tested on x86, does what is written on the box ;) Tested-by: Graeme Russ Regards, Graeme ___ U-Boot mailing list U-Boot@lists.denx.de http://lists.denx.de/mailman/listinfo/u-boot
