Hi all, Added https://issues.apache.org/jira/browse/HBASE-4454 to track just adding the failsafe plugin to the regular build process. This is a stopgap till we do the module split (which is going to be painful), to help speed up people's builds.
-Jesse Yates On Tue, Sep 20, 2011 at 7:22 AM, Doug Meil <doug.m...@explorysmedical.com>wrote: > > Hi there- > > https://issues.apache.org/jira/browse/HBASE-4448 > > I attached a prototype of HBaseTestingUtilityFactory to this ticket. I'd > like to get some feedback on the general idea (emphasis on "general idea") > and then if the consensus is that it makes sense I will continue and then > we can do a more formal review on the reviews thing. > > Doug > > > > > On 9/19/11 11:55 PM, "Stack" <st...@duboce.net> wrote: > > >On Mon, Sep 19, 2011 at 1:06 PM, Jesse Yates <jesse.k.ya...@gmail.com> > >wrote > >>> ... will spent (at least on my laptop) about 10.7 seconds setting up > >>>the > >>> cluster, and 7.3 seconds tearing down. Assuming that we aren't > >>>running in > >>> a separate JVM each test invocation, sharing the same instance of > >>> HBaseTestingUtility would be an enormous benefit. > >>> > > > >Seems like it would be worth investigating makeing this run > > > > > >>> 1) Re-use JVMs between test runs (if we are already doing this, then > >>>no > >>> action) > >>> > >> > >> This is already built into maven when we run tests normally. This leads > >>to > >> some interesting situations for messing around with class loading/global > >> variables; if you don't reset the environment, it can cause other > >> (essentially non-broken) tests to fail. Unless we are running things in > >> forked mode . > >> > >> Forked mode could also cut down on the straight unit test time, but in > >>my > >> experience this has lead to difficulty in figuring out what crashed. For > >> some reason maven doesn't realize which tests failed, just that they > >>did, in > >> its final output (though it still stdouts "FAILURE"). > >> > > > >Yeah, we run forked mode. > > > > <configuration> > > > ><forkedProcessTimeoutInSeconds>900</forkedProcessTimeoutInSeconds> > > <argLine>-enableassertions -Xmx1400m</argLine> > > <redirectTestOutputToFile>true</redirectTestOutputToFile> > > </configuration> > > > >I think the thinking was fork it because so much is going on and yeah, > >for sure there'll be cleanup afterward if all goes awry. > > > >Lohit suggested running tests in parallel. I tried it but things > >didn't seem to run faster in my playing around. See HBASE-4207. > > > > > >>> 2) Re-use the same HBaseTestingUtility instance for all possible > >>>tests. > >>> > >>> I think #2 isn't "that bad" if we had a factory method to get/return > >>> HBaseTestingUtility instance and internally this factory could do a > >>> ref-count and auto-detect of non-activity, then it could shutdown the > >>> cluster behind the scenes. > >>> > >>> > >> +1 on this idea. > >> We should also look into adding more/less region servers whenever people > >> request more. We could just leave it up for the entire span of the > >> integration tests and then bring it down afterwards (to make it easier > >>on > >> ourselves. > >> > > > >We do some of this now. TestFromClientSide and TestAdmin are an > >aggregation of a bunch of tests that used to each spin up own > >clusters. > > > >We should do more of this. I think what happens is that a bunch of > >our tests do one test only and to run the test, they spin up and down > >a cluster. Could put a few together -- those that make sense. > > > >St.Ack > >
