Fergus Henderson wrote:
Have you tried using pump mode? That helps a lot.
Also, if the issue is that distcc isn't running enough jobs on each
host, try using the "host/JOBS" format in DISTCC_HOSTS,
e.g. DISTCC_HOSTS="foo.riken.jp/20,cpp,lzo
<http://foo.riken.jp/20,cpp,lzo> bar.riken.jp/40,cpp,lzo
<http://bar.riken.jp/40,cpp,lzo>".
This would allow up to 20 concurrent compiles on host foo.riken.jp
<http://foo.riken.jp> and 40 concurrent compiles on host bar.riken.jp
<http://bar.riken.jp>.
(The ",cpp,lzo" part is required for pump mode.)
Thanks a lot! I am trying this right away.
I'll post my speedup results when I'm done.
At some point, depending on the size and structure of the software that
you're trying to compile,
and the way your Makefiles are arranged, you may run into limitations in
the parallelizability of your build graph.
For example, if your software has only three source files, then distcc
won't be able to speed up compilation by more than a factor of three at
very best.
Similarly, if your software has 30 files, but they are split into 10
directories of three files each, and your Makefile(s) sequentialize the
compilation of each directory, then you still won't get more than a
factor of three speed up at very best.
We use scons. I am not so happy with it (like when it spends 30s before
giving me back control of the command line when theres is _nothing_ to
compile because everything is already done).
However, it manages the dependencies far better than any Makefile I
could have written by hand.
Regards,
Francois.
On Mon, Nov 23, 2009 at 8:19 PM, Francois Berenger <beren...@riken.jp
<mailto:beren...@riken.jp>> wrote:
Hello,
Is there a way to use a pull scheduler instead of a push scheduler
inside distcc?
I have many compiling hosts and when they join a compiling job, they are
not enough loaded, at least to my taste. Meaning: they could compile
more things and I could wait less for them to finish.
I think the problem is that the scheduler is pushing: the distcc
server sends jobs to distcc compiling hosts.
Instead, I would like this behavior: distcc compiling hosts ask and
take jobs as long as they are not completely loaded with compilation
jobs.
The distcc server should distribute jobs to whom asks for.
This should scale better.
If there is no way now to do so but you can give me some hint on how to
do so (where to look, what to modify, etc.), it might be helpful.
Best regards,
Francois.
__
distcc mailing list http://distcc.samba.org/
To unsubscribe or change options:
https://lists.samba.org/mailman/listinfo/distcc
--
Fergus Henderson <fer...@google.com <mailto:fer...@google.com>>
__
distcc mailing list http://distcc.samba.org/
To unsubscribe or change options:
https://lists.samba.org/mailman/listinfo/distcc
