Hi, marc fleury wrote: > so I not only second that but I think we should introduce "speed" tests that > make sure that not only the functionality but also the performance is better > or at least not degrading across commits.
Yes, this would be a good idea. > I don't know if there is a way to automate that in Chris's test but if there > was that would be absolutely kick ass. When a test is done we would know > immediately if the performance is going down because of a problem. I don't think that JUnit can handle speed measurements the way we want, as it only does pass/fail. But it could be possible to write such a system. We should probably have a baseline speed test not depending on JBoss that never changes and use that to correct for differences in speed of the machine used for testing. It would also be nice if we had some database with the historic results of individual tests run with the automated daily test. This way, if someone finds out that some test stopped running, he can look up the database and find out which day it happened. It we only have to look at committed changes for a single day when something fails, it will be a lot easier to fix. (D*nm that the automated tests no longer includes detailed test failure information. Maybe there is room for it again, now that the failure count is no longer so high?) But to make developers use the test suite it should also be simple, straightforward and _fast_ to run. Currently, on my 800MhZ Duron system, the test suite takes about 40 minutes to run. This is so long that it may scare some developers from running the suite. Most of the time it runs stress tests that take a long time. Maybe it would be an idea to have another test target that only includes basic functionality tests, and that can run in a few minutes? Chances are higher that developers will run tests if it only takes a few minutes, and basic functionality testing is better than no testing at all. Best Regards, Ole Husgaard. _______________________________________________ Jboss-development mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/jboss-development
