[
https://issues.apache.org/jira/browse/HBASE-4737?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13146689#comment-13146689
]
nkeywal commented on HBASE-4737:
--------------------------------
@commiters: are you ok with the patch? The pom has been already commited. The
important one is 4737_all.patch because it impacts many test files, so it will
have conflict if we wait too much. It does not modify the executed code itself,
it just adds categories. The shell script can wait if necessary.
Note that I will have to write another patch for the small tests, but I believe
it's simpler if done in two steps vs. one.
thank you :-)
> Split the tests in small/medium/large; allow small tests to be ran in
> parallel within a single JVM
> --------------------------------------------------------------------------------------------------
>
> Key: HBASE-4737
> URL: https://issues.apache.org/jira/browse/HBASE-4737
> Project: HBase
> Issue Type: Improvement
> Components: test
> Affects Versions: 0.94.0
> Reporter: nkeywal
> Assignee: nkeywal
> Priority: Minor
> Attachments: 20111103_4737_pom.dummy.patch, 20111103_4737_pom.patch,
> 20111103_4737_pom.patch, 20111103_4737_pom.patch, 20111103_4737_pom.patch,
> 20111103_4737_pom.v2.patch, 20111103_4737_pom.v2.patch, 4737_all.patch,
> hbasetests.sh, hbasetests.sh
>
>
> 1) Split the tests in 3 categories
> - small: no cluster, less than 15s, can be run in parallel with other tests
> in a JVM
> - medium: <45s, no flaky, useful to detect bugs immediatly
> - large: remaining
>
> 2) Allow to run a subset: developpers should need to run only small and
> medium before submitting a patch
> - will need a surefire patch, see
> http://jira.codehaus.org/browse/SUREFIRE-329
> Small is the default. All other tests will have to be marked Medium or Large
> with a JUnit category.
> Proposed split:
> Small:122 classes, 479 methods, ~3 minutes when no //)
> Medium: 78 classes, 373 methods, ~23 minutes
> Large: 34 classes, 221 methods, ~60 minutes
> I will have to extract the methods that are today in large or medium but
> could be in small (typically io.hfile.TestHFileBlock#testBlockHeapSize), it
> will be done in a second step (and another JIRA).
> MEDIUM LIST (name; number of methods, time)
> org.apache.hadoop.hbase.avro.TestAvroServer 3 31.468
> org.apache.hadoop.hbase.catalog.TestCatalogTracker 8 4.174
> org.apache.hadoop.hbase.catalog.TestMetaReaderEditorNoCluster 1 3.888
> org.apache.hadoop.hbase.catalog.TestMetaReaderEditor 5 30.157
> org.apache.hadoop.hbase.client.replication.TestReplicationAdmin 1
> 0.762
> org.apache.hadoop.hbase.client.TestHCM 3 21.961
> org.apache.hadoop.hbase.client.TestHTablePool 18 26.274
> org.apache.hadoop.hbase.client.TestHTableUtil 2 16.997
> org.apache.hadoop.hbase.client.TestMetaMigrationRemovingHTD 3 24.629
> org.apache.hadoop.hbase.client.TestMetaScanner 1 16.365
> org.apache.hadoop.hbase.client.TestMultiParallel 10 34.077
> org.apache.hadoop.hbase.client.TestTimestampsFilter 3 27.547
> org.apache.hadoop.hbase.coprocessor.TestAggregateProtocol 44 16.834
> org.apache.hadoop.hbase.coprocessor.TestClassLoading 5 31.346
> org.apache.hadoop.hbase.coprocessor.TestCoprocessorEndpoint 2 32.736
> org.apache.hadoop.hbase.coprocessor.TestMasterCoprocessorExceptionWithAbort
> 1 13.874
> org.apache.hadoop.hbase.coprocessor.TestMasterCoprocessorExceptionWithRemove
> 1 16.923
> org.apache.hadoop.hbase.coprocessor.TestMasterObserver 3 29.97
> org.apache.hadoop.hbase.coprocessor.TestRegionObserverBypass 2 14.976
> org.apache.hadoop.hbase.coprocessor.TestRegionObserverInterface 5
> 33.353
> org.apache.hadoop.hbase.coprocessor.TestRegionServerCoprocessorExceptionWithAbort
> 1 16.596
> org.apache.hadoop.hbase.coprocessor.TestRegionServerCoprocessorExceptionWithRemove
> 1 18.183
> org.apache.hadoop.hbase.coprocessor.TestWALObserver 3 19.373
> org.apache.hadoop.hbase.filter.TestColumnRangeFilter 1 19.045
> org.apache.hadoop.hbase.io.hfile.slab.TestSingleSizeCache 5 24.294
> org.apache.hadoop.hbase.io.hfile.slab.TestSlabCache 7 19.818
> org.apache.hadoop.hbase.io.hfile.TestHFileBlock 7 25.226
> org.apache.hadoop.hbase.io.hfile.TestLruBlockCache 7 0.343
> org.apache.hadoop.hbase.mapreduce.TestImportTsv 8 40.391
> org.apache.hadoop.hbase.master.TestActiveMasterManager 2 0.724
> org.apache.hadoop.hbase.master.TestHMasterRPCException 1 1.17
> org.apache.hadoop.hbase.master.TestLogsCleaner 1 2.953
> org.apache.hadoop.hbase.master.TestMaster 1 18.918
> org.apache.hadoop.hbase.master.TestOpenedRegionHandler 2 20.57
> org.apache.hadoop.hbase.master.TestSplitLogManager 10 13.979
> org.apache.hadoop.hbase.master.TestZKBasedOpenCloseRegion 3 21.675
> org.apache.hadoop.hbase.regionserver.handler.TestOpenRegionHandler 3
> 0.887
> org.apache.hadoop.hbase.regionserver.TestBlocksRead 4 1.42
> org.apache.hadoop.hbase.regionserver.TestCompoundBloomFilter 3 22.694
> org.apache.hadoop.hbase.regionserver.TestFSErrorsExposed 3 29.764
> org.apache.hadoop.hbase.regionserver.TestHRegion 57 28.552
> org.apache.hadoop.hbase.regionserver.TestMasterAddressManager 1 0.525
> org.apache.hadoop.hbase.regionserver.TestMultiColumnScanner 6 19.568
> org.apache.hadoop.hbase.regionserver.TestRpcMetrics 1 2.028
> org.apache.hadoop.hbase.regionserver.TestSeekOptimizations 6 3.031
> org.apache.hadoop.hbase.regionserver.TestServerCustomProtocol 6 20.087
> org.apache.hadoop.hbase.regionserver.TestSplitLogWorker 5 2.062
> org.apache.hadoop.hbase.regionserver.TestStoreFileBlockCacheSummary 1
> 16.88
> org.apache.hadoop.hbase.regionserver.wal.TestLogRollAbort 1 22.545
> org.apache.hadoop.hbase.regionserver.wal.TestWALReplay 5 29.678
> org.apache.hadoop.hbase.replication.regionserver.TestReplicationSink 5
> 23.19
> org.apache.hadoop.hbase.replication.regionserver.TestReplicationSourceManager
> 1 0.918
> org.apache.hadoop.hbase.replication.TestReplicationPeer 1 25.518
> org.apache.hadoop.hbase.rest.client.TestRemoteAdmin 1 16.141
> org.apache.hadoop.hbase.rest.client.TestRemoteTable 3 17.827
> org.apache.hadoop.hbase.rest.TestGzipFilter 3 17.404
> org.apache.hadoop.hbase.rest.TestMultiRowResource 3 17.314
> org.apache.hadoop.hbase.rest.TestRowResource 11 17.73
> org.apache.hadoop.hbase.rest.TestScannerResource 5 26.868
> org.apache.hadoop.hbase.rest.TestScannersWithFilters 10 20.774
> org.apache.hadoop.hbase.rest.TestSchemaResource 2 22.441
> org.apache.hadoop.hbase.rest.TestStatusResource 2 14.884
> org.apache.hadoop.hbase.rest.TestTableResource 8 25.658
> org.apache.hadoop.hbase.rest.TestTransform 1 15.966
> org.apache.hadoop.hbase.rest.TestVersionResource 7 19.124
> org.apache.hadoop.hbase.TestGlobalMemStoreSize 1 18.154
> org.apache.hadoop.hbase.TestInfoServers 2 14.9
> org.apache.hadoop.hbase.TestMultiVersions 3 27.945
> org.apache.hadoop.hbase.thrift.TestThriftServer 1 20.072
> org.apache.hadoop.hbase.util.TestDefaultEnvironmentEdge 1 0.055
> org.apache.hadoop.hbase.util.TestEnvironmentEdgeManager 2 0.215
> org.apache.hadoop.hbase.util.TestFSUtils 2 19.988
> org.apache.hadoop.hbase.util.TestHBaseFsck 6 41.133
> org.apache.hadoop.hbase.util.TestIncrementingEnvironmentEdge 1 0.054
> org.apache.hadoop.hbase.util.TestMergeTable 1 27.125
> org.apache.hadoop.hbase.util.TestRegionSplitter 4 28.845
> org.apache.hadoop.hbase.zookeeper.TestZKTable 1 0.529
> org.apache.hadoop.hbase.zookeeper.TestZooKeeperNodeTracker 2 0.569
> LARGE
> org.apache.hadoop.hbase.catalog.TestCatalogTrackerOnCluster 1 84.463
> org.apache.hadoop.hbase.client.TestAdmin 33 369.833
> org.apache.hadoop.hbase.client.TestFromClientSide 49 154.41
> org.apache.hadoop.hbase.client.TestMultipleTimestamps 8 66.497
> org.apache.hadoop.hbase.client.TestScannerTimeout 4 47.309
> org.apache.hadoop.hbase.client.TestShell 1 69.65
> org.apache.hadoop.hbase.mapred.TestTableInputFormat 6 24.39
> org.apache.hadoop.hbase.mapred.TestTableMapReduce 1 63.846
> org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat 8 152.609
> org.apache.hadoop.hbase.mapreduce.TestLoadIncrementalHFilesSplitRecovery
> 3 74.628
> org.apache.hadoop.hbase.mapreduce.TestLoadIncrementalHFiles 6 35.25
> org.apache.hadoop.hbase.mapreduce.TestTableInputFormatScan 11 516.619
> org.apache.hadoop.hbase.mapreduce.TestTableMapReduce 1 92.144
> org.apache.hadoop.hbase.mapreduce.TestTimeRangeMapRed 1 42.761
> org.apache.hadoop.hbase.master.TestDistributedLogSplitting 4 112.606
> org.apache.hadoop.hbase.master.TestMasterFailover 4 76.658
> org.apache.hadoop.hbase.master.TestMasterRestartAfterDisablingTable 1
> 20.86
> org.apache.hadoop.hbase.master.TestMasterTransitions 3 22.043
> org.apache.hadoop.hbase.master.TestRestartCluster 2 36.325
> org.apache.hadoop.hbase.master.TestRollingRestart 1 54.738
> org.apache.hadoop.hbase.regionserver.TestEndToEndSplitTransaction 1
> 40.935
> org.apache.hadoop.hbase.regionserver.TestHRegionServerBulkLoad 1
> 47.073
> org.apache.hadoop.hbase.regionserver.TestSplitTransactionOnCluster 4
> 43.359
> org.apache.hadoop.hbase.regionserver.wal.TestHLogSplit 28 230.379
> org.apache.hadoop.hbase.regionserver.wal.TestHLog 9 59.889
> org.apache.hadoop.hbase.regionserver.wal.TestLogRolling 3 310.537
> org.apache.hadoop.hbase.replication.TestMasterReplication 2 75.346
> org.apache.hadoop.hbase.replication.TestMultiSlaveReplication 1 47.873
> org.apache.hadoop.hbase.replication.TestReplication 7 163.158
> org.apache.hadoop.hbase.TestFullLogReconstruction 1 57.742
> org.apache.hadoop.hbase.TestHBaseTestingUtility 7 73.654
> org.apache.hadoop.hbase.TestRegionRebalancing 1 58.25
> org.apache.hadoop.hbase.TestZooKeeper 7 86.267
> org.apache.hadoop.hbase.util.TestMergeTool 1 257.223
> SMALL
> Everyting else
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
