On Sun, Nov 20, 2011 at 11:15 PM, Simon Glass <s...@chromium.org> wrote: > Hi Mike, > > On Sun, Nov 20, 2011 at 1:23 PM, Mike Frysinger <vap...@gentoo.org> wrote: >> On Thursday 03 November 2011 03:28:29 Andy Fleming wrote: >> >> you don't need those semicolons. also, this is not as good as it should be: >> if you're running 10 jobs in parallel, you fork 10, and then you wait for all >> of them to finish before forking another set of 10. >> >> what you could do is something like: >> JOB_IDX=0 >> JOB_IDX_FIRST=0 >> BUILD_NBUILDS=1 >> >> ... foreach target ... ; do >> build_target & >> pids[$(( JOB_IDX++ ))]=$! >> if [ $(( JOB_IDX - JOB_IDX_FIRST )) -ge ${BUILD_NBUILDS} ] ; >> then >> wait ${pids[$(( JOB_IDX_FIRST++ ))]} >> fi >> done >> wait >> >> this isn't perfect as it assumes the first job launched will always finish >> first, >> but it's a lot closer than the current code. > > Since all the jobs are launched at the same time this is not really a > valid assumption is it? IMO on this point it's good enough as it is > for now...
Mike's idea led me to a slightly crazier one that (I think) works. Basically, I have each build_targets invocation create a file named LOG/build_N when it is done, where N is the value of TOTAL_CNT when it is invoked. Then I keep track of the oldest uncompleted build #. When we hit our build limit, the script scans for LOG/build_N, for N from oldest to TOTAL_CNT, and updates the count of outstanding builds, as appropriate (and moves the index of the oldest). If the scanning completes without freeing up room for another build, it waits for a second, and then re-scans. I tried this on my 24-core system with BUILD_NBUILDS set to 8 and BUILD_NCPUS set to 4, and pegged the system at ~24-30 the whole time. I will note that it only shaved about a minute off the 15 minutes it took to build all 85xx. I'm still debugging a bit (I want to try 16 builds/2 cores, and clean up a bit), but I figured I'd ask what people thought of this approach. My next patch will also address Mike's comments. If I'm feeling inspired, I might address some of your suggestions (I think I can use the build-completion files to coalesce some of the debug output). Andy _______________________________________________ U-Boot mailing list U-Boot@lists.denx.de http://lists.denx.de/mailman/listinfo/u-boot