I wasted a few hours of time writing a wrapper for make. The program would execute "make -n" plus any other command line options. The program executed each command make printed out in parallel (each in a different thread) If the command was successful (return 0) then it was deleted. Otherwise, it was re-executed the next round. In this way I hoped to have as many of the commands in parallel as possible, but if dependencies aren't satisfed, gcc will just return non-zero, and try again the next round. The program works for simple make files, it even built distcc, but not so good for the complex ones (make -n can fail when make doesn't!). It also turned out to be slower than just using make, distcc compiles in 4 seconds, and it takes 8 with this program :(
I can try again with a recursive wrapper, and execute subdirectories in order (but also parallel) At this point I don't think it's very easy to improve performance, since the slowest computer working on the largest file is on average less than a second!
On 1/6/06, Victor Norman <[EMAIL PROTECTED]> wrote:
oops: I mean that I'm trying to get the tree uploaded to sourceforge.net, but am having problem with cvs connectivity, due to our firewall, I think.
Vic
Victor Norman <[EMAIL PROTECTED]> wrote:Ring in the New Year with Photo Calendars. Add photos, events, holidays, whatever.__Sean, et al.,
Sean D'Epagnier < [EMAIL PROTECTED]> wrote:On 1/5/06, Victor Norman <[EMAIL PROTECTED]> wrote:Sean, et al.,
< ... snip .... >
So it takes 44 minutes to compile on one computer, and 56 minutes to compile 6 times using 6 computers? Can you test the difference without your wrapper but still using distcc? Ideally we will get this down to 44 minutes and 44 minutes :-P
[Victor:] No, it takes 44 minutes to compile our software branch using -j20 when there is no one else using the compilation farm, which consists of about 60 cpus of various strengths. It takes about 48 minutes when 2 people compile with -j20, sharing the farm. About 49 minutes with 3 people compiling with -j20; 51 minutes for 4 simultaneous compiles; 56 minutes for 5 compiles, 56.5 minutes for 6 compiles.
o ideally, it would be very nice to be able to know how long different files take to compile, and what order to do them in to produce the very best use of the compilation farm. But that is probably an NP-complete problem, and I needed to bite off something I could chew.
I agree, but maybe we can set things up now so it won't be hard to do that later?
Can we download and test it? I would like to see if it really speeds things up, it's obvious it adds useful features.
-sean
[Victor:] I'm trying to get the tree uploaded to cvs, but am having problems because of our firewall here at work. I can send the tree to you if you want it.
Vic
Yahoo! Photos
distcc mailing list http://distcc.samba.org/
To unsubscribe or change options:
https://lists.samba.org/mailman/listinfo/distcc
Yahoo! DSL Something to write home about. Just $16.99/mo. or less
__
distcc mailing list http://distcc.samba.org/
To unsubscribe or change options:
https://lists.samba.org/mailman/listinfo/distcc
__ distcc mailing list http://distcc.samba.org/ To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/distcc
