On 1/8/10, Dan Fabulich <d...@fabulich.com> wrote: > Kristian Rosenvold wrote: > >> I will re-add your stuff, and I will also set it up to use my output >> demultiplexer that causes output to appear in "normal" order. > > Does the demultiplexer do anything in weave mode when threads=1? Does it > make the projects appear to unweave (as far as the log is concerned)?
Clever question. The answer is yes. But it would be possible to run without it for threads=1 (and reveal weave mode to whoever's looking). > >> Which brings me to another high-level concern here; >> >> All I *really* want/need is to run test in the exact manner you are >> describing. In reality I'm not sure if the later-phase concurrencies >> (war, install etc) provide any real value, and some may even have >> negative contributions in some case (I/O trashing due to concurrency >> comes to mind). I run my builds on ramdisk in linux, so I don't suffer, >> but those poor windows users with their crappy IO and virus controls are >> not as lucky. > > What if we just added a surefire:force-test mojo that would ignore > -DskipTests and -Dmaven.test.skip? Then you could do this: > > mvn install surefire:force-test -DskipTests > > The "mvn install" part could build at Project Granularity without much > loss; then the surefire:force-test mojo could execute after each module > but in parallel with the rest of the build. > > To put it another way, given three projects Z Y X where Z depends on Y > which depends on X, at Project Granularity the builds are just run > linearly: > > XCTPI YCTPI ZCTPI > > (Compile Test Package Install = CTPI) > > In Weave mode we do it like this: > XCTPI > YCTPI > ZCTPI > > ... but that's dangerous, because Z's Compile may depend on X's Package. > But if we did this: > > XCPIT > YCPIT > ZCPIT > > The tests are the long part anyway, so this should have considerably > improved concurrency without adding project risk. > > -Dan > > --------------------------------------------------------------------- > To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org > For additional commands, e-mail: d...@maven.apache.org > I concentrated on doing measurements yesterday. I'm working on a write-up of some pretty strong evidence (with lot's of interesting numbers!) that backs up even more radically disconnecting test running from the lifecycle execution. I think I can "prove" that the part that makes my "weave" mode significantly faster than Project Granularity is mostly in the non-concurrent parts of the build. My favourite testproject has a 1-1-1-2-2-1-2 module concurrency width. The data I have indicates that the "huge" part of the difference between the two algorithms is in the "1"-module wide part of scheduling. In the "2" wide scenarios it is not unlikely to think that both perform much the same (when both are running with 2 threads). It's not hard to see when you think about it; you're running on a single thread, just waiting for the tests to finish. Tapping your fingers. Weave mode consistently manages to pick up more executable work - so even though there is a maximal module concurrency-width of 2, weave mode manages to utilize threads > 2. But I'm rigging up a test-scenario running with parallel test execution as well before I publish the data. Now that Surefire2.5-snapshot also supports parallel execution of junit, I can try both Project Granularity AND weave mode with concurrent testing. There's little point in optimizing maven core to death if we only end up competing with multithreaded plugins. There's always other time-consuming plugins too, often some heavy aggregators at the end. I can't see any of our algorithms really helping with those. Kristian --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org For additional commands, e-mail: dev-h...@maven.apache.org
