On Mon, Nov 15, 2010 at 9:34 PM, Marvin Humphrey <[email protected]> wrote: > Greets, > > My laptop has two cores, but the Lucy build process is single threaded and > doesn't take advantage of the second processor. > > I hacked up the patch below for trunk/perl/buildlib/Lucy/Build.pm to try to > speed things up. It forks off a max of 4 child processes which compile up to > 10 C files each. Here are before-and-after results for "time ./Build code":
Sorry, I am a perl dummy, so forgive me if i interpreted your statements/code wrong :) why is it 10 C files each? wouldn't this require some wasted fork() overhead respawning many children? couldnt you instead just split all the .c files into 4 pieces, and only have 4 children up front? p.s. we do a similar thing in Lucene-java with running tests, and a problem can be balancing the workload with the children. one heuristic about how long a test will take to run, or file will take to compile, is its length in bytes. it might be useful to sort the list of files by their size in bytes, and use mod to divide them up.
