[
https://issues.apache.org/jira/browse/PHOENIX-674?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13980435#comment-13980435
]
Samarth Jain commented on PHOENIX-674:
--------------------------------------
I used the conservative setting of 0.5C (which is 3 on my dev box) because
using a higher setting like 1C (6) was causing OOM exceptions. I think 3 would
be a reasonable enough default setting (I didn't see a lot of extra gain going
up to 4).
I think what Gabriel is suggesting is to have the default setting of number of
forks and let users override that number by providing a -D option. I have that
change ready.
Tests inside phoenix-pig module are failing for me though in the forked mode.
And they are passing just fine for phoenix-core which also uses
HBaseTestingUtility and mini cluster.
Stacktrace:
java.io.IOException: Shutting down
at
org.apache.hadoop.hbase.util.JVMClusterUtil.startup(JVMClusterUtil.java:190)
at
org.apache.hadoop.hbase.LocalHBaseCluster.startup(LocalHBaseCluster.java:425)
at
org.apache.hadoop.hbase.MiniHBaseCluster.init(MiniHBaseCluster.java:224)
at
org.apache.hadoop.hbase.MiniHBaseCluster.<init>(MiniHBaseCluster.java:93)
at
org.apache.hadoop.hbase.HBaseTestingUtility.startMiniHBaseCluster(HBaseTestingUtility.java:889)
at
org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:853)
at
org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:770)
at
org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:741)
at
org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:710)
at
org.apache.phoenix.pig.PhoenixHBaseStorerIT.setUpBeforeClass(PhoenixHBaseStorerIT.java:65)
Digging in more here to see why this is happening.
> Make unit test suite run faster
> -------------------------------
>
> Key: PHOENIX-674
> URL: https://issues.apache.org/jira/browse/PHOENIX-674
> Project: Phoenix
> Issue Type: Task
> Reporter: James Taylor
> Assignee: Samarth Jain
> Attachments: fasttests_v1.patch
>
>
> The main reason for the slowness is the time it takes to spin up and tear
> down a mini cluster for each test suite. We do this to guarantee that there's
> no left over state from previous tests.
> A few things we can do to improve things:
> * consolidate test suites where it makes sense, if there are just a few
> strangler tests that can put combined with other test suites.
> * spin up a single mini cluster and before a test suite runs, disable and
> drop any hbase tables that exist.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
