On 20 Feb 2004, john moser <[EMAIL PROTECTED]> wrote: > try reading it again, from the beginning. > > Let me try it like this. > > DISTRIBUTED COMPUTING NETWORK: 3700 NODES > NODE X: 1.5 Ghz Processor > TIME TO PROCESS 5 PARALLEL PREPROCESSINGS: 45 SECONDS > NODES USED: 5 > TIME TO PROCESS 10 PARALLEL PREPROCESSINGS: 130 SECONDS > NODES USED: 10 > TIME TO PROCESS 3700 PARALLEL PREPROCESSINGS: 49 DAYS, 6 HOURS, 15 MINUTES, 27 > SECONDS > NODES USED: about 1-2 at a time, as the preprocessings slowly finish on those > last 3 days and get sent out at random times. > > You can NOT do as many preprocesses in parallel as you have nodes sometimes. > To MAXIMIZE efficienty, you need to specify -j$NUMBEROFNODES and LOCK the > number of parallel preprocessing operations to a lower number. Then, WHILE > one node is compiling a complex source file, you can preprocess AND send out > another job, possibly BEFORE that one finishes. > > Simple enough? The idea is to get the job OFF the box ASAP so it can come > back FINISHED ASAP. > > Now, THNK this time, before you incur my wrath again.
Very funny. Remember kids, THNK first! > > On 19 Feb 2004, john moser <[EMAIL PROTECTED]> wrote: > > > distcc needs a way to limit how many preprocessing jobs it can run at once. > > It may be advantageous to have, say, 150,000 make jobs (if you have a 100000 > > node computing network, for example; let's say HP decides it wants to wait > > 2 minutes to compile a new operating system and all its tools). Running > > 150,000 parallel preprocessings will take hours. After maybe 80% of that time, > > a few jobs will trickle out to the compiling nodes. > > > > Instead, one could limit how many preprocessings can occur. The distcc would > > sleep until there's a free local preprocessing slot, and then run that > > preprocessing, then ship out to a free node. In this way, the actual > > efficiency will more effectively approach the theoretical efficiency. > > > > Think about it. Need you wait 10 minutes with 50 jobs before shoving them out > > the network? Are you always going to have enough processing power to get close > > to theorectical values? What's the best way to get off the box > > ASAP? > > That's what the -j level is for. > -- Martin __ distcc mailing list http://distcc.samba.org/ To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/distcc