[GitHub] [hbase] huaxiangsun commented on a change in pull request #1719: HBASE-24370 Avoid aggressive MergeRegion and GCMultipleMergedRegionsP…
huaxiangsun commented on a change in pull request #1719:
URL: https://github.com/apache/hbase/pull/1719#discussion_r426901856
##
File path:
hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestMetaFixer.java
##
@@ -308,6 +314,75 @@ public void testOverlapWithSmallMergeCount() throws
Exception {
}
}
+ /**
+ * This test covers the case that one of merged parent regions is a merged
child region that
+ * has not been GCed but there is no reference files anymore. In this case,
it will kick off
+ * a GC procedure, but no merge will happen.
+ */
+ @Test
+ public void testMergeWithMergedChildRegion() throws Exception {
+TableName tn = TableName.valueOf(this.name.getMethodName());
+Table t = TEST_UTIL.createMultiRegionTable(tn, HConstants.CATALOG_FAMILY);
+List ris =
MetaTableAccessor.getTableRegions(TEST_UTIL.getConnection(), tn);
+assertTrue(ris.size() > 5);
+HMaster services = TEST_UTIL.getHBaseCluster().getMaster();
+CatalogJanitor cj = services.getCatalogJanitor();
+cj.scan();
+CatalogJanitor.Report report = cj.getLastReport();
+assertTrue(report.isEmpty());
+RegionInfo overlapRegion = makeOverlap(services, ris.get(1), ris.get(2));
+Threads.sleep(1);
Review comment:
Checked the code, it seems the sleep is not needed at all, as procedure
has been waited to complete.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] huaxiangsun commented on a change in pull request #1719: HBASE-24370 Avoid aggressive MergeRegion and GCMultipleMergedRegionsP…
huaxiangsun commented on a change in pull request #1719:
URL: https://github.com/apache/hbase/pull/1719#discussion_r426893099
##
File path:
hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestMetaFixer.java
##
@@ -308,6 +314,75 @@ public void testOverlapWithSmallMergeCount() throws
Exception {
}
}
+ /**
+ * This test covers the case that one of merged parent regions is a merged
child region that
+ * has not been GCed but there is no reference files anymore. In this case,
it will kick off
+ * a GC procedure, but no merge will happen.
+ */
+ @Test
+ public void testMergeWithMergedChildRegion() throws Exception {
+TableName tn = TableName.valueOf(this.name.getMethodName());
+Table t = TEST_UTIL.createMultiRegionTable(tn, HConstants.CATALOG_FAMILY);
+List ris =
MetaTableAccessor.getTableRegions(TEST_UTIL.getConnection(), tn);
+assertTrue(ris.size() > 5);
+HMaster services = TEST_UTIL.getHBaseCluster().getMaster();
+CatalogJanitor cj = services.getCatalogJanitor();
+cj.scan();
+CatalogJanitor.Report report = cj.getLastReport();
+assertTrue(report.isEmpty());
+RegionInfo overlapRegion = makeOverlap(services, ris.get(1), ris.get(2));
+Threads.sleep(1);
+
+cj.scan();
+report = cj.getLastReport();
+assertEquals(2, report.getOverlaps().size());
+
+// Mark it as a merged child region.
+RegionInfo fakedParentRegion = RegionInfoBuilder.newBuilder(tn).
+ setStartKey(overlapRegion.getStartKey()).
+ build();
+
+Table meta = MetaTableAccessor.getMetaHTable(TEST_UTIL.getConnection());
+long time = HConstants.LATEST_TIMESTAMP;
Review comment:
Do not think so, here is the error.
{code}
[ERROR] org.apache.maven.surefire.booter.SurefireBooterForkException:
ExecutionException The forked VM terminated without properly saying goodbye. VM
crash or System.exit called?
[ERROR] Command was /bin/sh -c cd
/home/jenkins/jenkins-slave/workspace/Base-PreCommit-GitHub-PR_PR-1719/yetus-jdk8-hadoop3-check/src/hbase-server
&& /usr/lib/jvm/jdk8u232-b09/jre/bin/java -enableassertions
-Dhbase.build.id=2020-05-18T05:49:26Z -Xmx2200m
-Djava.security.egd=file:/dev/./urandom -Djava.net.preferIPv4Stack=true
-Djava.awt.headless=true -Djdk.net.URLClassPath.disableClassPathURLCheck=true
-Dorg.apache.hbase.thirdparty.io.netty.leakDetection.level=advanced
-Dio.netty.eventLoopThreads=3 -jar
/home/jenkins/jenkins-slave/workspace/Base-PreCommit-GitHub-PR_PR-1719/yetus-jdk8-hadoop3-check/src/hbase-server/target/surefire/surefirebooter7252273393886972447.jar
/home/jenkins/jenkins-slave/workspace/Base-PreCommit-GitHub-PR_PR-1719/yetus-jdk8-hadoop3-check/src/hbase-server/target/surefire
2020-05-18T05-50-20_960-jvmRun4 surefire6893180967496752951tmp
surefire_2101684540561928323544tmp
[ERROR] Process Exit Code: 0
[ERROR] at
org.apache.maven.plugin.surefire.booterclient.ForkStarter.awaitResultsDone(ForkStarter.java:513)
[ERROR] at
org.apache.maven.plugin.surefire.booterclient.ForkStarter.runSuitesForkPerTestSet(ForkStarter.java:460)
[ERROR] at
org.apache.maven.plugin.surefire.booterclient.ForkStarter.run(ForkStarter.java:301)
[ERROR] at
org.apache.maven.plugin.surefire.booterclient.ForkStarter.run(ForkStarter.java:249)
[ERROR] at
org.apache.maven.plugin.surefire.AbstractSurefireMojo.executeProvider(AbstractSurefireMojo.java:1217)
[ERROR] at
org.apache.maven.plugin.surefire.AbstractSurefireMojo.executeAfterPreconditionsChecked(AbstractSurefireMojo.java:1063)
[ERROR] at
org.apache.maven.plugin.surefire.AbstractSurefireMojo.execute(AbstractSurefireMojo.java:889)
[ERROR] at
org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:137)
[ERROR] at
org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:210)
[ERROR] at
org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:156)
[ERROR] at
org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:148)
[ERROR] at
org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:117)
[ERROR] at
org.apache.maven.lifecycle.internal.builder.multithreaded.MultiThreadedBuilder$1.call(MultiThreadedBuilder.java:190)
[ERROR] at
org.apache.maven.lifecycle.internal.builder.multithreaded.MultiThreadedBuilder$1.call(MultiThreadedBuilder.java:186)
[ERROR] at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[ERROR] at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
[ERROR] at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[ERROR] at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[ERROR] at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[ERROR] at java.lang.Thread
[GitHub] [hbase] huaxiangsun commented on a change in pull request #1719: HBASE-24370 Avoid aggressive MergeRegion and GCMultipleMergedRegionsP…
huaxiangsun commented on a change in pull request #1719:
URL: https://github.com/apache/hbase/pull/1719#discussion_r426371166
##
File path:
hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestAsyncRegionAdminApi2.java
##
@@ -161,20 +163,39 @@ public void testMergeRegions() throws Exception {
.getTableHRegionLocations(metaTable, tableName).get();
RegionInfo regionA;
RegionInfo regionB;
+RegionInfo regionC;
+RegionInfo mergedChildRegion = null;
// merge with full name
assertEquals(3, regionLocations.size());
regionA = regionLocations.get(0).getRegion();
regionB = regionLocations.get(1).getRegion();
+regionC = regionLocations.get(2).getRegion();
admin.mergeRegions(regionA.getRegionName(), regionB.getRegionName(),
false).get();
regionLocations = AsyncMetaTableAccessor
.getTableHRegionLocations(metaTable, tableName).get();
+
assertEquals(2, regionLocations.size());
+for (HRegionLocation rl : regionLocations) {
+ if (regionC.compareTo(rl.getRegion()) != 0) {
+mergedChildRegion = rl.getRegion();
+break;
+ }
+}
+
+assertTrue(mergedChildRegion != null);
Review comment:
Thanks @HorizonNet, let me address the comments and push another update.
For some reason, the second push did not trigger unittest runs.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
