nono.. we where running using distcc across our cluster (you could
see the compile jobs running on all the different nodes.)
Though I would not be surprised in the least if I configured
something wrong.. Plus our Mac OS NFS Servers have been... flaky.. at
best. So there might just be an issue in network file system
performance.
It appears that my experience with distcc is wrong =) Thats good to
know.. next time I have to re-build a cluster I will make sure to
give it another go and see if I can make it perform better =)
On Apr 11, 2006, at 1:51 PM, Donnie Berkholz wrote:
Brady Catherman wrote:
One last question for you all.. Why is distcc so popular? We used
it on
our 134 node cluster and it actually made compiling much slower than
just running it on one of the nodes. The network overhead killed the
performance gain. The only way we found that it helped was writing
the
makefile itself to take advantage of parallelism. Is this uncommon
for
most people?
Autotools (properly used) create parallelizable Makefiles, so
that's not
much of an issue. Clearly if you're just exporting the same job to
another node instead of parallelizing across multiple nodes, you will
see a performance loss.
distcc is particularly useful when not all nodes are attempting to
locally upgrade/install something at the same time, so they team up
on a
parallelized compilation for a single node. There's no effective
gain by
using distcc on a large cluster if you're just compiling
everything on
every node -- you should be using it in parallel to build binary
packages once, then installing across all nodes.
distcc is also useful when you've got a mixture of slow and fast
nodes,
for obvious reasons.
Thanks,
Donnie
--
[email protected] mailing list
