I'm not too familiar with the Maven Failsafe plugin, but I've been
reviewing the timings of some 'client' unit tests and where the unit test
framework spends it's time...
Anything that uses HBaseTestingUtility and does something like...
protected void setUp() throws Exception {
TEST_UTIL.startMiniCluster(1);
TEST_UTIL.createTable(TABLENAME,
HConstants.CATALOG_FAMILY);
}
protected void tearDown() throws IOException {
TEST_UTIL.shutdownMiniCluster();
}
... will spent (at least on my laptop) about 10.7 seconds setting up the
cluster, and 7.3 seconds tearing down. Assuming that we aren't running in
a separate JVM each test invocation, sharing the same instance of
HBaseTestingUtility would be an enormous benefit.
In terms of startup costs... It's all right here...
this.hbaseCluster = new MiniHBaseCluster(c, numMasters, numSlaves);
// Don't leave here till we've done a successful scan of the .META.
HTable t = new HTable(c, HConstants.META_TABLE_NAME);
ResultScanner s = t.getScanner(new Scan());
while (s.next() != null) {
continue;
}
... MiniHBaseCluster starts up via the init method in the constructor,
and the following code sits and spins until it's ready.
As I see it the required actions are:
1) Re-use JVMs between test runs (if we are already doing this, then no
action)
2) Re-use the same HBaseTestingUtility instance for all possible tests.
I think #2 isn't "that bad" if we had a factory method to get/return
HBaseTestingUtility instance and internally this factory could do a
ref-count and auto-detect of non-activity, then it could shutdown the
cluster behind the scenes.
On 9/19/11 2:31 PM, "Akash Ashok" <[email protected]> wrote:
>Hi,
>
>FAIL SAFE PLUGIN:
>FailSafe Plugin sounds like a great idea. My understanding is that we
>startup and kill the hdfs, zookeeper and HBase cluster instances for every
>test suite ( Please correct me if I am wrong here ) . .
>
>With FailSafe plugin we would be able to achieve the following things:
>1. Spawn hdfs, zookeeper and HBase cluster once, run the tests and shut
>them
>down at the end. I guess this is what pre and post integration tests are
>meant for ( Considering we aren't testing failure scenarios )
>2. Also have to make sure the tests are runnable alone and not only in
>bulk.
>
>Considering that we wouldn't have to run tests for 4 hrs ( 2 w/0 patch
>and 2
>with patch ). I would be glad to work on this transition. Please count me
>in.
>
>BARNEY :
>As for a 'barney', maybe something through twitter where when someone
>breaks
>the build, the previous person posts a pic of barney to the receiver?
>
>I was wondering why not have a VIRTUAL BARNEY as a Hudson Barney Plugin :)
>Once a build breaks because of some1 the next time the same person checks
>in
>it would warn saying this person had already broken the build last time
>and
>he had also broken the build so many times in the past so BEWARE :P
>
>Apologies since I am increasing non-productive work unrelated to HBase.
>But
>I feel it would be kinda nice :)
>
>Cheers,
>Akash A
