This is awesome. It works for me (as you said, plus or minus a few processes). Thank you.
Would it be possible to use this for the '--slowstart X' feature, that starts one job at a time, waits X seconds, and looks at the load before starting another? On Wed, Mar 6, 2013 at 8:23 PM, Ole Tange <[email protected]> wrote: > I have implemented an instant --load calculator. With this it should > be possible to tell GNU Parallel to spawn a process for each CPU that > is currently sitting idle. > > On a 64 core machine this: > > seq 10000 | parallel --load 20% burnP6 & > sleep 2 > seq 10000 | parallel --load 100% burnMMX & > sleep 4 > seq 10000 | parallel --load 100% burnBX & > > will start 14 burnP6, 52 burnMMX, and 0 burnBX. If the first parallel > is killed, the last remaining parallels will spawn around 14 processes > it total. > > It should also count processes that are waiting for local disks as > non-idle. So this: > > ls *iso | parallel --load 1 -j0 -v cat > > will only spawn 3 cat and not one for every iso file. > > The biggest problem I have seen so far is that it is not very > accurate. So it may shoot a little over or under the target (2-4). But > it seems way better than the previous --load. > > http://alpha.gnu.org/gnu/parallel/parallel-20130307.tar.bz2 > > Please give it a spin and provide feedback. > > > /Ole > >
