On Mon, Nov 21, 2011 at 1:01 PM, Mike Frysinger <vap...@gentoo.org> wrote: > On Monday 21 November 2011 01:28:50 Andy Fleming wrote: >> On Sun, Nov 20, 2011 at 11:15 PM, Simon Glass wrote: >> > On Sun, Nov 20, 2011 at 1:23 PM, Mike Frysinger wrote: >> >> On Thursday 03 November 2011 03:28:29 Andy Fleming wrote: >> >> >> >> you don't need those semicolons. also, this is not as good as it should >> >> be: if you're running 10 jobs in parallel, you fork 10, and then you >> >> wait for all of them to finish before forking another set of 10. >> >> >> >> what you could do is something like: >> >> JOB_IDX=0 >> >> JOB_IDX_FIRST=0 >> >> BUILD_NBUILDS=1 >> >> >> >> ... foreach target ... ; do >> >> build_target & >> >> pids[$(( JOB_IDX++ ))]=$! >> >> if [ $(( JOB_IDX - JOB_IDX_FIRST )) -ge ${BUILD_NBUILDS} >> >> ] ; then wait ${pids[$(( JOB_IDX_FIRST++ ))]} >> >> fi >> >> done >> >> wait >> >> >> >> this isn't perfect as it assumes the first job launched will always >> >> finish first, but it's a lot closer than the current code. >> > >> > Since all the jobs are launched at the same time this is not really a >> > valid assumption is it? IMO on this point it's good enough as it is >> > for now... >> >> Mike's idea led me to a slightly crazier one that (I think) works. >> Basically, I have each build_targets invocation create a file named >> LOG/build_N when it is done, where N is the value of TOTAL_CNT when it >> is invoked. Then I keep track of the oldest uncompleted build #. When >> we hit our build limit, the script scans for LOG/build_N, for N from >> oldest to TOTAL_CNT, and updates the count of outstanding builds, as >> appropriate (and moves the index of the oldest). If the scanning >> completes without freeing up room for another build, it waits for a >> second, and then re-scans. > > sounds like it would work, but i'd want to see how much code it takes to > actually implement this > >> I tried this on my 24-core system with BUILD_NBUILDS set to 8 and >> BUILD_NCPUS set to 4, and pegged the system at ~24-30 the whole time. >> >> I will note that it only shaved about a minute off the 15 minutes it >> took to build all 85xx. > > compare to what ? your original patch, or my tweaked version ?
Compared to my original. However, I think that may be due to my configuration. 8 builds with -j4 makes was 14, but 16 builds with -j2 resulted in just over 10 minutes. I'm doing more testing. Andy _______________________________________________ U-Boot mailing list U-Boot@lists.denx.de http://lists.denx.de/mailman/listinfo/u-boot