[
https://issues.apache.org/jira/browse/HBASE-5708?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Phabricator updated HBASE-5708:
-------------------------------
Attachment: D2601.3.patch
mbautin updated the revision "[jira] [HBASE-5708] [89-fb] Improving robustness
of map-reduce-related and other unit tests".
Reviewers: Kannan, Karthik, Liyin, JIRA, khemani
A few more improvements:
- Fixing a bug in handling of MSG_REGION_OPEN retries in case root region is
unavailable. Previously we would put the event into the queue but handle it
right away anyway, which resulted in missing znode exceptions down the road.
- Fixing how we wait until ZK session expires after we force it to close in
HBaseTestingUtility. The polling delay should be constant and should not depend
on ZK session timeout.
- Increasing TTL in TestScannerSelectionUsingTTL to 10 seconds. With the
previous value of two seconds some of the "newer" HFiles had time to expire by
the time they were read under high load conditions (multiple tests running on a
machine).
- Adding extra delay tolerance to a few more wait loops in
TestSplitLogManager. This also came up under high system load.
- Increasing the number of client retries for tests from 10 to 100. This came
up in some unit test failures.
Unfortunately, there are still unit test failures, especially if every test
is run 5 times. Those are not easy to fix, but we will gradually isolate and
fix them. As it turns out, quite a few bugs could be found by simply running
unit tests multiple times.
However, this patch does reduce the number of various test problems from 64
to 28 (counting test method failures, test method errors, and timeouts for the
whole test class) when every test class is run 5 times. I think this is a good
step towards having a stable test suite.
REVISION DETAIL
https://reviews.facebook.net/D2601
AFFECTED FILES
src/main/java/org/apache/hadoop/hbase/HConstants.java
src/main/java/org/apache/hadoop/hbase/master/OldLogsCleaner.java
src/main/java/org/apache/hadoop/hbase/master/RegionManager.java
src/main/java/org/apache/hadoop/hbase/master/SplitLogManager.java
src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
src/test/java/org/apache/hadoop/hbase/HBaseTestCase.java
src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java
src/test/java/org/apache/hadoop/hbase/TestFullLogReconstruction.java
src/test/java/org/apache/hadoop/hbase/TestZooKeeper.java
src/test/java/org/apache/hadoop/hbase/client/TestFromClientSide.java
src/test/java/org/apache/hadoop/hbase/filter/TestColumnPrefixFilter.java
src/test/java/org/apache/hadoop/hbase/io/TestHalfStoreFileReader.java
src/test/java/org/apache/hadoop/hbase/io/hfile/TestCacheOnWrite.java
src/test/java/org/apache/hadoop/hbase/io/hfile/TestFixedFileTrailer.java
src/test/java/org/apache/hadoop/hbase/io/hfile/TestHFile.java
src/test/java/org/apache/hadoop/hbase/io/hfile/TestHFileBlock.java
src/test/java/org/apache/hadoop/hbase/io/hfile/TestHFileBlockIndex.java
src/test/java/org/apache/hadoop/hbase/io/hfile/TestHFileDataBlockEncoder.java
src/test/java/org/apache/hadoop/hbase/io/hfile/TestHFilePerformance.java
src/test/java/org/apache/hadoop/hbase/io/hfile/TestHFileSeek.java
src/test/java/org/apache/hadoop/hbase/io/hfile/TestHFileWriterV2.java
src/test/java/org/apache/hadoop/hbase/io/hfile/TestReseekTo.java
src/test/java/org/apache/hadoop/hbase/io/hfile/TestScannerSelectionUsingTTL.java
src/test/java/org/apache/hadoop/hbase/mapred/TestLegacyTableMapReduce.java
src/test/java/org/apache/hadoop/hbase/mapreduce/TestHFileOutputFormat.java
src/test/java/org/apache/hadoop/hbase/mapreduce/TestLoadIncrementalHFiles.java
src/test/java/org/apache/hadoop/hbase/mapreduce/TestTableMapReduce.java
src/test/java/org/apache/hadoop/hbase/master/TestSplitLogManager.java
src/test/java/org/apache/hadoop/hbase/regionserver/TestBlocksRead.java
src/test/java/org/apache/hadoop/hbase/regionserver/TestColumnSeeking.java
src/test/java/org/apache/hadoop/hbase/regionserver/TestCompactSelection.java
src/test/java/org/apache/hadoop/hbase/regionserver/TestCompoundBloomFilter.java
src/test/java/org/apache/hadoop/hbase/regionserver/TestFSErrorsExposed.java
src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegion.java
src/test/java/org/apache/hadoop/hbase/regionserver/TestMultiColumnScanner.java
src/test/java/org/apache/hadoop/hbase/regionserver/TestScannerResets.java
src/test/java/org/apache/hadoop/hbase/regionserver/TestStore.java
src/test/java/org/apache/hadoop/hbase/regionserver/TestStoreFile.java
src/test/java/org/apache/hadoop/hbase/regionserver/wal/TestHLog.java
src/test/java/org/apache/hadoop/hbase/regionserver/wal/TestHLogMethods.java
src/test/java/org/apache/hadoop/hbase/replication/regionserver/TestReplicationSourceManager.java
src/test/java/org/apache/hadoop/hbase/util/TestMiniClusterLoadSequential.java
src/test/resources/hbase-site.xml
> [89-fb] Make MiniMapRedCluster directory a subdirectory of target/test
> ----------------------------------------------------------------------
>
> Key: HBASE-5708
> URL: https://issues.apache.org/jira/browse/HBASE-5708
> Project: HBase
> Issue Type: Bug
> Reporter: Mikhail Bautin
> Priority: Minor
> Attachments: D2601.1.patch, D2601.2.patch, D2601.3.patch
>
>
> Some map-reduce-based tests are failing when executed concurrently in 89-fb
> because mini-map-reduce cluster uses /tmp/hadoop-<username> for temporary
> data.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
