Oh, I should probably have included a link:
https://github.com/icecc/icecream
Short story is SuSE made an improved version of distcc. It's been around
a while, but I just recently stumbled over it and decided to give it a
go. Results were good enough, and adding support simple enough, so I
decided to just do it.
It's a wrapper around the compiler executable that sends compile jobs
into a cluster of machines. You run a scheduler on one machine and a
daemon on all participating slave machines. It supports the distribution
of the actual compiler along with the preprocessed source files and
executes the compilation in a chroot. This effectively guarantees that
you get the same output as you would on your local computer. The setup
is pretty simple. The usecase as I see it is repeated compilation on
your local machine in a development situation. The developers in the
same physical location would run the daemon on their own computers and
share unused cpu cycles with each other. The compile jobs run with nice
by default so shouldn't interfere much.
Things to note. The default nice level in icecc is 5, while the jdk9
build seems to get 10 by default. This means you probably want to bump
icecc to something larger than 10 so that the remote jobs don't get
priority over your local jobs. To reap the benefits you need to bump the
number of jobs to match the cluster rather than your own machine. I have
had to run with JOBS=150 to fully saturate all the machines I have
managed to install the daemon on so far in the office.
/Erik
On 2015-11-14 09:02, David Holmes wrote:
I'll bite - what are icecc and icecream? :)
David
On 14/11/2015 5:55 AM, Erik Joelsson wrote:
We should enable builds using icecc/icecream distributed native
compilation. The setup is rather simple and the required changes to
configure are no more complex than how we currently handle ccache.
I have done some experiments and it seems to be working quite well. I
can easily get build times of less than 2 minutes for "make
exploded-image" using JOBS=100 to fully saturate a couple of extra hosts
in the cluster. (I need to combine this with --enable-javac-server to
get the java compilation to not be a bottleneck.)
This is likely not that interesting to automated, already distributed
build systems, but for a developers day to day work, being attached to
an icecream cluster could greatly reduce local compile times.
Icecream supports both linux and macosx but I have only tested this on
linux myself. The implementation in configure will support both gcc and
clang.
To use the feature, given that you have icecc installed and setup to
join a cluster, make sure icecc is on the path and simply configure with
--enable-icecc and it should all work automatically. Configure will
generate the necessary compiler build environments that get sent to the
build slaves and setup the needed make variables.
When enabling icecc, precompiled header is disabled. It is possible to
get precompiled header to work with icecc, but the performance boost
gets completely negated (and more) by the extra file transfer times of
the precompiled header file, at least in my experience.
Bug: https://bugs.openjdk.java.net/browse/JDK-8142983
Webrev: http://cr.openjdk.java.net/~erikj/8142983/webrev.01/
/Erik
