Re: Parallel make

2015-04-29 Thread SF Markus Elfring
> GNU make has no built-in capability to use multiple machines: How are the chances to integrate additional job submission systems? > conceptually it may be a straightforward extension but the effort needed > to communicate between multiple systems over a network, send and receive > results reli

Re: Parallel make

2015-04-29 Thread Edward Welbourne
> We ran into similar things in Linux due to NFS, autofs, and NFS via > apparmor not scaling well when 100 compilers are trying to search for > header files through a long number of sourcedirs. I suppose this can be mitigated by using #include "path/to/file.h" in source, for paths relative to a sm

Re: Parallel make

2015-04-29 Thread Edward Welbourne
> Luckily, if you are building C or C++ code someone has already done all > the necessary work for you. I recommend you investigate the distcc > package: https://code.google.com/p/distcc/ Likewise the icecream project's icecc: https://en.opensuse.org/Icecream https://github.com/icecc/icecream I'

Re: Parallel make

2015-04-29 Thread Kruppa, Jason
Last time I looked at distcc, it still ran the c++ preprocessor on the main machine (to eliminate the risk of different machines having different configurations), and then sent the preprocessed files to the various machines at its disposal. This, in my experience, still does a huge portion of t

Re: Parallel make

2015-04-29 Thread Tim Murphy
Without wanting to turn this into a commercial/advert you might want to consider trying the Electric Cloud Huddle beta since it works with multiple machines in a convenient way and deals with the problems of getting correct parallel builds. It is also free for now at least. Sorry for that :) On 29

Re: Parallel make

2015-04-29 Thread Paul Smith
On Wed, 2015-04-29 at 13:50 -0600, Ryan P. Steele wrote: > The multithreaded version of make (-j#) is wonderful, and we have made > great use of it. Because we're dealing with some very large code, > however, it would be great to be able to parallelize compilation over > multiple machines. I ca

Parallel make

2015-04-29 Thread Ryan P. Steele
Hi, folks - I encountered a situation that I can't seem to resolve through the "make" documentation... The multithreaded version of make (-j#) is wonderful, and we have made great use of it. Because we're dealing with some very large code, however, it would be great to be able to paralleliz