On Mon, Apr 13, 2009 at 5:22 AM, James M. Lawrence <[email protected]> wrote: > Subtitle: My sales pitch to end the Drake project. > > Drake and its engine, CompTree, have been slimmed down since their > beginnings. No more IPC forking for CompTree, no more utility libs, > no more duplication of CompTree inside Rake -- just the meat. > > CompTree has been running well and its API is minimal and stable. No > bugs of which I am aware. I am confident about using it as a > dependency. > > Drake had just one bug, the result of prerequisites being > context-sensitive (this behavior did not even cross my mind). The > rubinius folks use Drake, last I heard. > > Some remarks about the pull request: > * Just one git commit -- all noise has been removed. > * No effective changes to the default single-threaded mode. > * Rather small, code-wise (especially when refactoring subtracted) > * Passes all unit tests in single-threaded mode. > * Passes all unit tests in parallel mode. > * Adds parallel-mode-specific tests. > * Full documentation: adds a note in rakefile.rdoc with a link to the > new parallel.rdoc. Update command_line_usage.rdoc. > * The changes to rake.rb are minor and straightforward: > - Add -j option. > - Add a cache for accumulating tasks with a dry run. > - Add hooks to rake/parallel.rb > - Add a lock to prevent invoke inside invoke (can't have cake and > eat it too--either hand off control to the parallelizer or don't). > * Add rake/parallel.rb: 79 lines of code, 100% coverage. The > interface to comp_tree lies here and only here. > * lazy 'require': rake/parallel.rb and comp_tree are included only > when threads > 1. > * Requires comp_tree gem. I am willing to embed a copy of CompTree > inside Rake if that is preferred instead. > > Some remarks about CompTree: > * 257 lines of code, 100% coverage. > * Has a grand total of 5 methods in the API; Rake uses 3 of them. > * Unlikely to change in any significant way; similar to a math library. > * The core algorithm not changed since its inception, and its > implementation has only undergone cosmetic changes. > * Tests have pounded on a wide variety of graph sizes and shapes with > small to large numbers of threads. No deadlocks, race conditions, > or other thread problems have been discovered. > * No longer sets Thread.abort_on_exception (though the user is free to > set it). > > Some remarks about auto-parallelizing: > * It gives Rake a little more prestige. More wealth and fame for Jim. > * Ever greater numbers of CPU cores are becoming available. Rake has > to jump on the train eventually; might as well be now. > > http://github.com/quix/rake/tree/mainline-merge-candidate
Awesome! I've made Phusion Passenger's Rakefile thread-safe (I was using chdir before) in order to test your fork. Strangely the results are not as spectacular as I've hoped: OS X with Intel Core 2 Duo, Ruby 1.8: -j 1: 1 minute 4 seconds -j 2: 56 seconds -j 4: 57 seconds I thought that it might have something to do with Ruby 1.8's green threading, although I'm pretty sure that 1.8 is able to preempt system(). So I tried Ruby 1.9: -j 1: 1 minute 2 seconds -j 4: 56 seconds I'm not sure whether this is a problem in your Rake fork or in my Rakefile. It would be nice if there's a way to output the parallelized dependency tree so that I can see whether I did something wrong. On the bright side, compilation succeeded so this probably means that your code is mostly bug-free. Regards, Hongli Lai -- Phusion | The Computer Science Company Web: http://www.phusion.nl/ E-mail: [email protected] Chamber of commerce no: 08173483 (The Netherlands) _______________________________________________ Rake-devel mailing list [email protected] http://rubyforge.org/mailman/listinfo/rake-devel
