-1 It is a good idea but I think the time is wrong. We have introduced JDBC test slowness into 2.1 and 2.2 - see OPENJPA-1876 [1] - which is partly responsible for the long build time. The slowness should be identified and got rid off.
Only after that we should determine which tests are slow by nature and separate them. Otherwise we risk hiding a problem instead of resolving it. As for OPENJPA-1876 I spent some hours on it and am still spending when the time permits but without amazing results. I have recently determined that DELETE statements are suspiciously slow on Derby but did not have yet time to compare it with older branches and determine whether it is the cause. Maybe someone will have better luck. Cheers, Milosz [1] https://issues.apache.org/jira/browse/OPENJPA-1876 > +1 Sounds good. I like the idea of having a faster test suite as the > default. > > -Jeremy > > On Thu, Jan 6, 2011 at 9:39 AM, Michael Dick <michael.d.d...@gmail.com>wrote: > > > Hi Mark, > > > > Comments inline. > > > > On Thu, Jan 6, 2011 at 8:15 AM, Mark Struberg <strub...@yahoo.de> wrote: > > > > > Might have been a temporal problem locally since this special test > > performs > > > rather quick in our hudson: > > > > > > > > > > > https://hudson.apache.org/hudson/job/OpenJPA-trunk/370/org.apache.openjpa$openjpa-persistence-jdbc/testReport/ > > > > > > > Those tests don't take long on our CI server either, but I've seen the same > > problem with different tests. I've always assumed that there's GC or some > > other environmental issue that makes them take longer in Maven than they do > > independently. > > > > > > > The locking tests of corse are really slow in hudson too. > > > > > > > I'll confess that I usually skip the locking tests before committing (I > > don't think I'm alone here). One of the continuous integrations servers > > will > > catch locking bugs on the rare occasions that they crop up. > > > > > > > Might be worth moving all the expensive test into an own > > 'integration-test' > > > profile? > > > Otherwise developers will just take the shortcut and don't run the whole > > > build prior to a checkin because it takes so long ;) > > > > > > > I agree that's a real problem. We'll need to make sure that Hudson and any > > other CI servers hit all the tests. Part of the problem here is that the > > Hudson build errors have a poor signal to noise ratio. A large number of > > Hudson problems don't seem to be repeatable for me. > > > > But I digress. A build that takes over 1 hour is an impediment to new > > committers, and I'd be fine with moving the locking and other timing tests > > to an integration-test profile as long as Hudson executes them. > > > > How does this sound to others on the list? > > > > -mike > > > > > > > > > LieGrue, > > > strub > > > > > > --- On Thu, 1/6/11, Mark Struberg <strub...@yahoo.de> wrote: > > > > > > > From: Mark Struberg <strub...@yahoo.de> > > > > Subject: Tests take forever > > > > To: dev@openjpa.apache.org > > > > Date: Thursday, January 6, 2011, 1:56 PM > > > > Hi! > > > > > > > > Anyone has an idea why some of our tests take almost > > > > forever when they run inside the build, but are very cheap > > > > if run standalone? > > > > > > > > My build currently takes ~ 1h 15minutes on a 4x3GHz and > > > > also not much longer on my notebook. This indicates that > > > > there is something 'foul' with them. Either they wait > > > > unnecessarily long or they do not scale (even when built > > > > with mvn -T4) > > > > > > > > An example: > > > > > > > > I got the following results for > > > > org.apache.openjpa.lib.conf.TestEquivalentConfiguration: > > > > testOldStylePersistenceUnitConfiguration > > > > 6.541 > > > > testNewStylePersistenceUnitConfiguration > > > > 206 > > > > > > > > testMixedStylePersistenceUnitConfiguration > > > > 223 > > > > > > > > testConflictStylePersistenceUnitConfiguration > > > > 18 > > > > testNewStyleSystemPropertyConfiguration > > > > 214 > > > > testOldStyleSystemPropertyConfiguration > > > > 223 > > > > > > > > testMixedStyleSystemPropertyConfiguration > > > > 256 > > > > > > > > testConflictStyleSystemPropertyConfiguration > > > > 16 > > > > testOldStyleRuntimePropertyConfiguration > > > > 249 > > > > testNewStyleRuntimePropertyConfiguration > > > > 231 > > > > > > > > testMixedStyleRuntimePropertyConfiguration > > > > 24 > > > > > > > > testConflictStyleRuntimePropertyConfiguration > > > > 18 > > > > > > > > makes 844 seconds in summary. > > > > > > > > But the same test passes in 4.034 seconds when I run this > > > > test in Idea (all 12 tests pass). > > > > > > > > I used the following VM settings: > > > > -Dopenjpa.Log=DefaultLevel=INFO > > > > -Dopenjpa.DynamicEnhancementAgent=false > > > > -Dopenjpa.ConnectionDriverName=org.apache.derby.jdbc.EmbeddedDriver > > > > > > > > > -Dopenjpa.ConnectionURL=jdbc:derby:target/database/openjpa-derby-database;create=true > > > > -Dopenjpa.ConnectionUserName= -Dopenjpa.ConnectionPassword= > > > > > > > > Is there anything I miss? > > > > > > > > Looks like we should pretty easily be able to slash down > > > > our build to 20 minutes or so... > > > > > > > > > > > > LieGrue, > > > > strub > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >
