[ https://issues.apache.org/jira/browse/HBASE-5708?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Phabricator updated HBASE-5708: ------------------------------- Attachment: D2601.3.patch mbautin updated the revision "[jira] [HBASE-5708] [89-fb] Improving robustness of map-reduce-related and other unit tests". Reviewers: Kannan, Karthik, Liyin, JIRA, khemani A few more improvements: - Fixing a bug in handling of MSG_REGION_OPEN retries in case root region is unavailable. Previously we would put the event into the queue but handle it right away anyway, which resulted in missing znode exceptions down the road. - Fixing how we wait until ZK session expires after we force it to close in HBaseTestingUtility. The polling delay should be constant and should not depend on ZK session timeout. - Increasing TTL in TestScannerSelectionUsingTTL to 10 seconds. With the previous value of two seconds some of the "newer" HFiles had time to expire by the time they were read under high load conditions (multiple tests running on a machine). - Adding extra delay tolerance to a few more wait loops in TestSplitLogManager. This also came up under high system load. - Increasing the number of client retries for tests from 10 to 100. This came up in some unit test failures. Unfortunately, there are still unit test failures, especially if every test is run 5 times. Those are not easy to fix, but we will gradually isolate and fix them. As it turns out, quite a few bugs could be found by simply running unit tests multiple times. However, this patch does reduce the number of various test problems from 64 to 28 (counting test method failures, test method errors, and timeouts for the whole test class) when every test class is run 5 times. I think this is a good step towards having a stable test suite. REVISION DETAIL https://reviews.facebook.net/D2601 AFFECTED FILES src/main/java/org/apache/hadoop/hbase/HConstants.java src/main/java/org/apache/hadoop/hbase/master/OldLogsCleaner.java src/main/java/org/apache/hadoop/hbase/master/RegionManager.java src/main/java/org/apache/hadoop/hbase/master/SplitLogManager.java src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java src/test/java/org/apache/hadoop/hbase/HBaseTestCase.java src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java src/test/java/org/apache/hadoop/hbase/TestFullLogReconstruction.java src/test/java/org/apache/hadoop/hbase/TestZooKeeper.java src/test/java/org/apache/hadoop/hbase/client/TestFromClientSide.java src/test/java/org/apache/hadoop/hbase/filter/TestColumnPrefixFilter.java src/test/java/org/apache/hadoop/hbase/io/TestHalfStoreFileReader.java src/test/java/org/apache/hadoop/hbase/io/hfile/TestCacheOnWrite.java src/test/java/org/apache/hadoop/hbase/io/hfile/TestFixedFileTrailer.java src/test/java/org/apache/hadoop/hbase/io/hfile/TestHFile.java src/test/java/org/apache/hadoop/hbase/io/hfile/TestHFileBlock.java src/test/java/org/apache/hadoop/hbase/io/hfile/TestHFileBlockIndex.java src/test/java/org/apache/hadoop/hbase/io/hfile/TestHFileDataBlockEncoder.java src/test/java/org/apache/hadoop/hbase/io/hfile/TestHFilePerformance.java src/test/java/org/apache/hadoop/hbase/io/hfile/TestHFileSeek.java src/test/java/org/apache/hadoop/hbase/io/hfile/TestHFileWriterV2.java src/test/java/org/apache/hadoop/hbase/io/hfile/TestReseekTo.java src/test/java/org/apache/hadoop/hbase/io/hfile/TestScannerSelectionUsingTTL.java src/test/java/org/apache/hadoop/hbase/mapred/TestLegacyTableMapReduce.java src/test/java/org/apache/hadoop/hbase/mapreduce/TestHFileOutputFormat.java src/test/java/org/apache/hadoop/hbase/mapreduce/TestLoadIncrementalHFiles.java src/test/java/org/apache/hadoop/hbase/mapreduce/TestTableMapReduce.java src/test/java/org/apache/hadoop/hbase/master/TestSplitLogManager.java src/test/java/org/apache/hadoop/hbase/regionserver/TestBlocksRead.java src/test/java/org/apache/hadoop/hbase/regionserver/TestColumnSeeking.java src/test/java/org/apache/hadoop/hbase/regionserver/TestCompactSelection.java src/test/java/org/apache/hadoop/hbase/regionserver/TestCompoundBloomFilter.java src/test/java/org/apache/hadoop/hbase/regionserver/TestFSErrorsExposed.java src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegion.java src/test/java/org/apache/hadoop/hbase/regionserver/TestMultiColumnScanner.java src/test/java/org/apache/hadoop/hbase/regionserver/TestScannerResets.java src/test/java/org/apache/hadoop/hbase/regionserver/TestStore.java src/test/java/org/apache/hadoop/hbase/regionserver/TestStoreFile.java src/test/java/org/apache/hadoop/hbase/regionserver/wal/TestHLog.java src/test/java/org/apache/hadoop/hbase/regionserver/wal/TestHLogMethods.java src/test/java/org/apache/hadoop/hbase/replication/regionserver/TestReplicationSourceManager.java src/test/java/org/apache/hadoop/hbase/util/TestMiniClusterLoadSequential.java src/test/resources/hbase-site.xml > [89-fb] Make MiniMapRedCluster directory a subdirectory of target/test > ---------------------------------------------------------------------- > > Key: HBASE-5708 > URL: https://issues.apache.org/jira/browse/HBASE-5708 > Project: HBase > Issue Type: Bug > Reporter: Mikhail Bautin > Priority: Minor > Attachments: D2601.1.patch, D2601.2.patch, D2601.3.patch > > > Some map-reduce-based tests are failing when executed concurrently in 89-fb > because mini-map-reduce cluster uses /tmp/hadoop-<username> for temporary > data. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira