Brady Catherman wrote: > One last question for you all.. Why is distcc so popular? We used it on > our 134 node cluster and it actually made compiling much slower than > just running it on one of the nodes. The network overhead killed the > performance gain. The only way we found that it helped was writing the > makefile itself to take advantage of parallelism. Is this uncommon for > most people?
Autotools (properly used) create parallelizable Makefiles, so that's not much of an issue. Clearly if you're just exporting the same job to another node instead of parallelizing across multiple nodes, you will see a performance loss. distcc is particularly useful when not all nodes are attempting to locally upgrade/install something at the same time, so they team up on a parallelized compilation for a single node. There's no effective gain by using distcc on a large cluster if you're just compiling everything on every node -- you should be using it in parallel to build binary packages once, then installing across all nodes. distcc is also useful when you've got a mixture of slow and fast nodes, for obvious reasons. Thanks, Donnie
signature.asc
Description: OpenPGP digital signature
