Thanks for the quantitative measurement! Out of curiosity, can you also try running your MbUnit v2.4 tests using Gallio (you don't actually need to upgrade to MbUnit v3 for this)? I imagine most of the new overhead is in the infrastructure rather than the framework itself.
We do take this issue very seriously and have several initiatives planned for performance optimization, particularly in the v3.1 timeframe. Here are a few sources of the increased overhead: 1. There's process-level isolation in place which means setup/teardown time for the test run is higher and there is greater latency in communications remoting costs between the test runner and the test host process. We intend to remedy this issue by simplifying the remote communication protocol and introducing more asynchronous message processing to reduce the overall latency. 2. There's a fair amount of startup time overhead currently due to the plugin loading process. We intend to remedy this issue by introducing a new plug-in model with more cached metadata and lazy initialization. 3. Memory usage is somewhat higher. Partly this is because we're loading more code than we need to right now in the way of plugins. Another cause has to do with the fact that we keep more AppDomains alive right now because of the design of the test runner lifecycle. We intend to remedy this issue by redesigning the test runner lifecycle. In addition to these measures listed above, we are planning to introduce support for running tests concurrently, when possible. This feature will be based on a concept of test isolation mode. eg. For each test or fixture we might declare a certain level of isolation required from other tests. Jeff. On Wed, Nov 19, 2008 at 11:08 AM, James Kovacs <[EMAIL PROTECTED]>wrote: > > On a current project, I upgraded from MbUnit 2.4 to Gallio 3.0.5. Test > execution times skyrocketed. The test suite is a reasonable size at > 694 tests - mostly unit with some integration. I tried running the > test suite under Gallio compiled for MbUnit2.4 and MbUnit3. The tests > are run with NCover2 integration using the Gallio NAnt task. > > MbUnit3 under NCover2 > Run#1 272.553s > Run#2 278.308s > > MbUnit2.4 under NCover2 > Run#1 165.769s > Run#2 145.258s > > The only code difference is correcting some minor syntactic > differences between MbUnit2.4 and MbUnit3, notably the > ApartmentStateAttribute on two test classes. Unfortunately I can't > share the code since it is a client project. I'm wondering if anyone > else has encountered this when upgrading to MbUnit3? I'm using a > fairly standard tool stack - .NET 3.5, Rhino Mocks, Oracle, Castle > Windsor, ... I just can't quite understand why MbUnit3 performance is > so much worse than MbUnit2.4. On the build server, which has a lot > less horsepower than my dev machine, it's taking around 8 minutes for > the build with MbUnit3 whereas it was around 4 minutes with MbUnit2.4. > Because of this, I've had to revert to Gallio running MbUnit2.4. Any > thoughts or suggestions would be appreciated. > > Thanks, > James > > > --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "MbUnit.User" group. To post to this group, send email to [email protected] To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/MbUnitUser?hl=en -~----------~----~----~----~------~----~------~--~---
