[jira] [Commented] (HBASE-7143) TestMetaMigrationRemovingHTD fails when used with Hadoop 0.23/2.x
[ https://issues.apache.org/jira/browse/HBASE-7143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13494841#comment-13494841 ] Hadoop QA commented on HBASE-7143: -- {color:red}-1 overall{color}. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12553012/HBASE-7143-0.94.patch against trunk revision . {color:green}+1 @author{color}. The patch does not contain any @author tags. {color:green}+1 tests included{color}. The patch appears to include 3 new or modified tests. {color:red}-1 patch{color}. The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/3305//console This message is automatically generated. TestMetaMigrationRemovingHTD fails when used with Hadoop 0.23/2.x - Key: HBASE-7143 URL: https://issues.apache.org/jira/browse/HBASE-7143 Project: HBase Issue Type: Bug Components: test Affects Versions: 0.94.2 Reporter: Andrey Klochkov Fix For: 0.94.3 Attachments: 7143-trunk-v2.txt, HBASE-7143-0.94.patch, HBASE-7143-0.94.patch, HBASE-7143-trunk.patch, HBASE-7143-trunk.patch TestMetaMigrationRemovingHTD fails when build is done with -Dhadoop.profile=23 option. The reason is the changes of defaults in -mkdir CLI call. In 0.23/2.x, it doesn't create parent directories by default anymore. The patch will be submitted shortly. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7143) TestMetaMigrationRemovingHTD fails when used with Hadoop 0.23/2.x
[ https://issues.apache.org/jira/browse/HBASE-7143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13494842#comment-13494842 ] Andrey Klochkov commented on HBASE-7143: the failures reported by Hadoop QA do not seem relevant to the changes TestMetaMigrationRemovingHTD fails when used with Hadoop 0.23/2.x - Key: HBASE-7143 URL: https://issues.apache.org/jira/browse/HBASE-7143 Project: HBase Issue Type: Bug Components: test Affects Versions: 0.94.2 Reporter: Andrey Klochkov Fix For: 0.94.3 Attachments: 7143-trunk-v2.txt, HBASE-7143-0.94.patch, HBASE-7143-0.94.patch, HBASE-7143-trunk.patch, HBASE-7143-trunk.patch TestMetaMigrationRemovingHTD fails when build is done with -Dhadoop.profile=23 option. The reason is the changes of defaults in -mkdir CLI call. In 0.23/2.x, it doesn't create parent directories by default anymore. The patch will be submitted shortly. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7103) Need to fail split if SPLIT znode is deleted even before the split is completed.
[ https://issues.apache.org/jira/browse/HBASE-7103?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramkrishna.s.vasudevan updated HBASE-7103: -- Attachment: HBASE-7103_0.94.patch Need to fail split if SPLIT znode is deleted even before the split is completed. Key: HBASE-7103 URL: https://issues.apache.org/jira/browse/HBASE-7103 Project: HBase Issue Type: Bug Reporter: ramkrishna.s.vasudevan Assignee: ramkrishna.s.vasudevan Fix For: 0.94.3, 0.96.0 Attachments: 7103-6088-revert.txt, HBASE-7103_0.94.patch, HBASE-7103_testcase.patch This came up after the following mail in dev list 'infinite loop of RS_ZK_REGION_SPLIT on .94.2'. The following is the reason for the problem The following steps happen - Initially the parent region P1 starts splitting. - The split is going on normally. - Another split starts at the same time for the same region P1. (Not sure why this started). - Rollback happens seeing an already existing node. - This node gets deleted in rollback and nodeDeleted Event starts. - In nodeDeleted event the RIT for the region P1 gets deleted. - Because of this there is no region in RIT. - Now the first split gets over. Here the problem is we try to transit the node to SPLITTING to SPLIT. But the node even does not exist. But we don take any action on this. We think it is successful. - Because of this SplitRegionHandler never gets invoked. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7103) Need to fail split if SPLIT znode is deleted even before the split is completed.
[ https://issues.apache.org/jira/browse/HBASE-7103?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13494843#comment-13494843 ] ramkrishna.s.vasudevan commented on HBASE-7103: --- This patch makes the transition to SPLITTING after creating the node once the first journal entry is added. What do we get out of this is - If any parallel split request comes the second one will fail because the znode creation will fail saying node already exists. So there is no impact due to rollback as nothing is added in the journal for the second split and so no deletion of the znode happens. - Now if while transitioning to SPLITTING if it fails, then it will lead to rollback that will delete the znode. Anyway that is not going to impact anything on the RIT in master side as only after the transition is done the RIT will be populated first time. If nothing is there then no impact. Pls provide your comments on this, i can prepare for trunk too if this is fine. :) Need to fail split if SPLIT znode is deleted even before the split is completed. Key: HBASE-7103 URL: https://issues.apache.org/jira/browse/HBASE-7103 Project: HBase Issue Type: Bug Reporter: ramkrishna.s.vasudevan Assignee: ramkrishna.s.vasudevan Fix For: 0.94.3, 0.96.0 Attachments: 7103-6088-revert.txt, HBASE-7103_0.94.patch, HBASE-7103_testcase.patch This came up after the following mail in dev list 'infinite loop of RS_ZK_REGION_SPLIT on .94.2'. The following is the reason for the problem The following steps happen - Initially the parent region P1 starts splitting. - The split is going on normally. - Another split starts at the same time for the same region P1. (Not sure why this started). - Rollback happens seeing an already existing node. - This node gets deleted in rollback and nodeDeleted Event starts. - In nodeDeleted event the RIT for the region P1 gets deleted. - Because of this there is no region in RIT. - Now the first split gets over. Here the problem is we try to transit the node to SPLITTING to SPLIT. But the node even does not exist. But we don take any action on this. We think it is successful. - Because of this SplitRegionHandler never gets invoked. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7103) Need to fail split if SPLIT znode is deleted even before the split is completed.
[ https://issues.apache.org/jira/browse/HBASE-7103?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13494845#comment-13494845 ] Hadoop QA commented on HBASE-7103: -- {color:red}-1 overall{color}. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12553013/HBASE-7103_0.94.patch against trunk revision . {color:green}+1 @author{color}. The patch does not contain any @author tags. {color:green}+1 tests included{color}. The patch appears to include 3 new or modified tests. {color:red}-1 patch{color}. The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/3306//console This message is automatically generated. Need to fail split if SPLIT znode is deleted even before the split is completed. Key: HBASE-7103 URL: https://issues.apache.org/jira/browse/HBASE-7103 Project: HBase Issue Type: Bug Reporter: ramkrishna.s.vasudevan Assignee: ramkrishna.s.vasudevan Fix For: 0.94.3, 0.96.0 Attachments: 7103-6088-revert.txt, HBASE-7103_0.94.patch, HBASE-7103_testcase.patch This came up after the following mail in dev list 'infinite loop of RS_ZK_REGION_SPLIT on .94.2'. The following is the reason for the problem The following steps happen - Initially the parent region P1 starts splitting. - The split is going on normally. - Another split starts at the same time for the same region P1. (Not sure why this started). - Rollback happens seeing an already existing node. - This node gets deleted in rollback and nodeDeleted Event starts. - In nodeDeleted event the RIT for the region P1 gets deleted. - Because of this there is no region in RIT. - Now the first split gets over. Here the problem is we try to transit the node to SPLITTING to SPLIT. But the node even does not exist. But we don take any action on this. We think it is successful. - Because of this SplitRegionHandler never gets invoked. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7142) TestSplitLogManager#testDeadWorker may fail because of hard limit on the TimeoutMonitor's timeout period
[ https://issues.apache.org/jira/browse/HBASE-7142?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13494846#comment-13494846 ] Himanshu Vashishtha commented on HBASE-7142: Yeah, only 0.94. The timeout in 0.96 is already set to 3 sec. TestSplitLogManager#testDeadWorker may fail because of hard limit on the TimeoutMonitor's timeout period Key: HBASE-7142 URL: https://issues.apache.org/jira/browse/HBASE-7142 Project: HBase Issue Type: Test Components: test Affects Versions: 0.94.2 Reporter: Himanshu Vashishtha Priority: Minor Fix For: 0.94.3 Attachments: HBASE-7142.patch The timeout in testDeadWorker is set to 1 sec, it is the same as the TimeoutMonitor thread timeout. In some case, this may fail: {code} java.lang.AssertionError at org.junit.Assert.fail(Assert.java:92) at org.junit.Assert.assertTrue(Assert.java:43) at org.junit.Assert.assertTrue(Assert.java:54) at org.apache.hadoop.hbase.master.TestSplitLogManager.waitForCounter(TestSplitLogManager.java:147) at org.apache.hadoop.hbase.master.TestSplitLogManager.waitForCounter(TestSplitLogManager.java:127) at org.apache.hadoop.hbase.master.TestSplitLogManager.testDeadWorker(TestSplitLogManager.java:433) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) {code} Fix is to increase the timeout for this test. Its not needed in trunk as the timeout is 3 seconds. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6694) Test scanner batching in export job feature HBASE-6372 AND report on improvement HBASE-6372 adds
[ https://issues.apache.org/jira/browse/HBASE-6694?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13494853#comment-13494853 ] Alexander Alten-Lorenz commented on HBASE-6694: --- Ping you guys for ;) Test scanner batching in export job feature HBASE-6372 AND report on improvement HBASE-6372 adds Key: HBASE-6694 URL: https://issues.apache.org/jira/browse/HBASE-6694 Project: HBase Issue Type: Task Reporter: stack Assignee: Alexander Alten-Lorenz Attachments: HBASE-6694.patch From tail of HBASE-6372, Jon had raised issue that test added did not actually test the feature. This issue is about adding a test of HBASE-6372. We should also have numbers for the improvement that HBASE-6372 brings. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7143) TestMetaMigrationRemovingHTD fails when used with Hadoop 0.23/2.x
[ https://issues.apache.org/jira/browse/HBASE-7143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13494884#comment-13494884 ] Ted Yu commented on HBASE-7143: --- Integrated to 0.94 and trunk. Thanks for the patch, Andrey. Thanks for the review, Lars. TestMetaMigrationRemovingHTD fails when used with Hadoop 0.23/2.x - Key: HBASE-7143 URL: https://issues.apache.org/jira/browse/HBASE-7143 Project: HBase Issue Type: Bug Components: test Affects Versions: 0.94.2 Reporter: Andrey Klochkov Fix For: 0.94.3 Attachments: 7143-trunk-v2.txt, HBASE-7143-0.94.patch, HBASE-7143-0.94.patch, HBASE-7143-trunk.patch, HBASE-7143-trunk.patch TestMetaMigrationRemovingHTD fails when build is done with -Dhadoop.profile=23 option. The reason is the changes of defaults in -mkdir CLI call. In 0.23/2.x, it doesn't create parent directories by default anymore. The patch will be submitted shortly. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HBASE-7143) TestMetaMigrationRemovingHTD fails when used with Hadoop 0.23/2.x
[ https://issues.apache.org/jira/browse/HBASE-7143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu reassigned HBASE-7143: - Assignee: Andrey Klochkov TestMetaMigrationRemovingHTD fails when used with Hadoop 0.23/2.x - Key: HBASE-7143 URL: https://issues.apache.org/jira/browse/HBASE-7143 Project: HBase Issue Type: Bug Components: test Affects Versions: 0.94.2 Reporter: Andrey Klochkov Assignee: Andrey Klochkov Fix For: 0.94.3, 0.96.0 Attachments: 7143-trunk-v2.txt, HBASE-7143-0.94.patch, HBASE-7143-0.94.patch, HBASE-7143-trunk.patch, HBASE-7143-trunk.patch TestMetaMigrationRemovingHTD fails when build is done with -Dhadoop.profile=23 option. The reason is the changes of defaults in -mkdir CLI call. In 0.23/2.x, it doesn't create parent directories by default anymore. The patch will be submitted shortly. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7143) TestMetaMigrationRemovingHTD fails when used with Hadoop 0.23/2.x
[ https://issues.apache.org/jira/browse/HBASE-7143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-7143: -- Fix Version/s: 0.96.0 Hadoop Flags: Reviewed TestMetaMigrationRemovingHTD fails when used with Hadoop 0.23/2.x - Key: HBASE-7143 URL: https://issues.apache.org/jira/browse/HBASE-7143 Project: HBase Issue Type: Bug Components: test Affects Versions: 0.94.2 Reporter: Andrey Klochkov Assignee: Andrey Klochkov Fix For: 0.94.3, 0.96.0 Attachments: 7143-trunk-v2.txt, HBASE-7143-0.94.patch, HBASE-7143-0.94.patch, HBASE-7143-trunk.patch, HBASE-7143-trunk.patch TestMetaMigrationRemovingHTD fails when build is done with -Dhadoop.profile=23 option. The reason is the changes of defaults in -mkdir CLI call. In 0.23/2.x, it doesn't create parent directories by default anymore. The patch will be submitted shortly. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HBASE-7056) TestMetaMigrationConvertingToPB failed in HBase-TRUNK-on-Hadoop-2.0.0 two times consecutively
[ https://issues.apache.org/jira/browse/HBASE-7056?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu resolved HBASE-7056. --- Resolution: Duplicate Duplicate with HBASE-7143 TestMetaMigrationConvertingToPB failed in HBase-TRUNK-on-Hadoop-2.0.0 two times consecutively - Key: HBASE-7056 URL: https://issues.apache.org/jira/browse/HBASE-7056 Project: HBase Issue Type: Bug Reporter: Ted Yu Priority: Critical Fix For: 0.96.0 Building against hadoop 2.0.2-alpha, TestMetaMigrationConvertingToPB failed twice - builds 236 and 237. See https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/236/testReport/org.apache.hadoop.hbase.catalog/TestMetaMigrationConvertingToPB/org_apache_hadoop_hbase_catalog_TestMetaMigrationConvertingToPB/ {code} java.io.IOException: Failed put; errcode=1 at org.apache.hadoop.hbase.catalog.TestMetaMigrationConvertingToPB.doFsCommand(TestMetaMigrationConvertingToPB.java:155) at org.apache.hadoop.hbase.catalog.TestMetaMigrationConvertingToPB.setUpBeforeClass(TestMetaMigrationConvertingToPB.java:108) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15) {code} -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7143) TestMetaMigrationRemovingHTD fails when used with Hadoop 0.23/2.x
[ https://issues.apache.org/jira/browse/HBASE-7143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13494901#comment-13494901 ] Hudson commented on HBASE-7143: --- Integrated in HBase-TRUNK #3530 (See [https://builds.apache.org/job/HBase-TRUNK/3530/]) HBASE-7143 TestMetaMigrationRemovingHTD fails when used with Hadoop 0.23/2.x (Andrey Klochlov) (Revision 1408011) Result = FAILURE tedyu : Files : * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/catalog/TestMetaMigrationConvertingToPB.java TestMetaMigrationRemovingHTD fails when used with Hadoop 0.23/2.x - Key: HBASE-7143 URL: https://issues.apache.org/jira/browse/HBASE-7143 Project: HBase Issue Type: Bug Components: test Affects Versions: 0.94.2 Reporter: Andrey Klochkov Assignee: Andrey Klochkov Fix For: 0.94.3, 0.96.0 Attachments: 7143-trunk-v2.txt, HBASE-7143-0.94.patch, HBASE-7143-0.94.patch, HBASE-7143-trunk.patch, HBASE-7143-trunk.patch TestMetaMigrationRemovingHTD fails when build is done with -Dhadoop.profile=23 option. The reason is the changes of defaults in -mkdir CLI call. In 0.23/2.x, it doesn't create parent directories by default anymore. The patch will be submitted shortly. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7143) TestMetaMigrationRemovingHTD fails when used with Hadoop 0.23/2.x
[ https://issues.apache.org/jira/browse/HBASE-7143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13494902#comment-13494902 ] Hudson commented on HBASE-7143: --- Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #256 (See [https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/256/]) HBASE-7143 TestMetaMigrationRemovingHTD fails when used with Hadoop 0.23/2.x (Andrey Klochlov) (Revision 1408011) Result = FAILURE tedyu : Files : * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/catalog/TestMetaMigrationConvertingToPB.java TestMetaMigrationRemovingHTD fails when used with Hadoop 0.23/2.x - Key: HBASE-7143 URL: https://issues.apache.org/jira/browse/HBASE-7143 Project: HBase Issue Type: Bug Components: test Affects Versions: 0.94.2 Reporter: Andrey Klochkov Assignee: Andrey Klochkov Fix For: 0.94.3, 0.96.0 Attachments: 7143-trunk-v2.txt, HBASE-7143-0.94.patch, HBASE-7143-0.94.patch, HBASE-7143-trunk.patch, HBASE-7143-trunk.patch TestMetaMigrationRemovingHTD fails when build is done with -Dhadoop.profile=23 option. The reason is the changes of defaults in -mkdir CLI call. In 0.23/2.x, it doesn't create parent directories by default anymore. The patch will be submitted shortly. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7143) TestMetaMigrationRemovingHTD fails when used with Hadoop 0.23/2.x
[ https://issues.apache.org/jira/browse/HBASE-7143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13494911#comment-13494911 ] Hudson commented on HBASE-7143: --- Integrated in HBase-0.94 #578 (See [https://builds.apache.org/job/HBase-0.94/578/]) HBASE-7143 TestMetaMigrationRemovingHTD fails when used with Hadoop 0.23/2.x (Andrey Klochlov) (Revision 1408012) Result = SUCCESS tedyu : Files : * /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/client/TestMetaMigrationRemovingHTD.java TestMetaMigrationRemovingHTD fails when used with Hadoop 0.23/2.x - Key: HBASE-7143 URL: https://issues.apache.org/jira/browse/HBASE-7143 Project: HBase Issue Type: Bug Components: test Affects Versions: 0.94.2 Reporter: Andrey Klochkov Assignee: Andrey Klochkov Fix For: 0.94.3, 0.96.0 Attachments: 7143-trunk-v2.txt, HBASE-7143-0.94.patch, HBASE-7143-0.94.patch, HBASE-7143-trunk.patch, HBASE-7143-trunk.patch TestMetaMigrationRemovingHTD fails when build is done with -Dhadoop.profile=23 option. The reason is the changes of defaults in -mkdir CLI call. In 0.23/2.x, it doesn't create parent directories by default anymore. The patch will be submitted shortly. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7143) TestMetaMigrationRemovingHTD fails when used with Hadoop 0.23/2.x
[ https://issues.apache.org/jira/browse/HBASE-7143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-7143: -- Resolution: Fixed Status: Resolved (was: Patch Available) TestMetaMigrationConvertingToPB passed in Jenkins build against hadoop 2.0 TestMetaMigrationRemovingHTD fails when used with Hadoop 0.23/2.x - Key: HBASE-7143 URL: https://issues.apache.org/jira/browse/HBASE-7143 Project: HBase Issue Type: Bug Components: test Affects Versions: 0.94.2 Reporter: Andrey Klochkov Assignee: Andrey Klochkov Fix For: 0.94.3, 0.96.0 Attachments: 7143-trunk-v2.txt, HBASE-7143-0.94.patch, HBASE-7143-0.94.patch, HBASE-7143-trunk.patch, HBASE-7143-trunk.patch TestMetaMigrationRemovingHTD fails when build is done with -Dhadoop.profile=23 option. The reason is the changes of defaults in -mkdir CLI call. In 0.23/2.x, it doesn't create parent directories by default anymore. The patch will be submitted shortly. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6793) Make hbase-examples module
[ https://issues.apache.org/jira/browse/HBASE-6793?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13494916#comment-13494916 ] Ted Yu commented on HBASE-6793: --- Trunk build 3530 got to building hbase-examples module but failed: {code} [INFO] HBase - Examples .. FAILURE [3.222s] [INFO] [INFO] BUILD FAILURE [INFO] [INFO] Total time: 29:21.569s [INFO] Finished at: Sun Nov 11 15:17:35 UTC 2012 [INFO] Final Memory: 68M/642M [INFO] [ERROR] Failed to execute goal org.apache.rat:apache-rat-plugin:0.8:check (default) on project hbase-examples: Too many unapproved licenses: 20 - [Help 1] {code} Looks like license headers are missing in some of the files in hbase-examples module Make hbase-examples module -- Key: HBASE-6793 URL: https://issues.apache.org/jira/browse/HBASE-6793 Project: HBase Issue Type: Improvement Affects Versions: 0.96.0 Reporter: Enis Soztutar Assignee: Sergey Shelukhin Labels: noob Fix For: 0.96.0 Attachments: HBASE-6793.patch, HBASE-6793-v2.patch, HBASE-6793-v3-thrift-0.9.0.patch, HBASE-6793-v4.1-squashed.patch, HBASE-6793-v5-squashed.patch There are some examples under /examples/, which are not compiled as a part of the build. We can move them to an hbase-examples module. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HBASE-7142) TestSplitLogManager#testDeadWorker may fail because of hard limit on the TimeoutMonitor's timeout period
[ https://issues.apache.org/jira/browse/HBASE-7142?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl reassigned HBASE-7142: Assignee: Himanshu Vashishtha TestSplitLogManager#testDeadWorker may fail because of hard limit on the TimeoutMonitor's timeout period Key: HBASE-7142 URL: https://issues.apache.org/jira/browse/HBASE-7142 Project: HBase Issue Type: Test Components: test Affects Versions: 0.94.2 Reporter: Himanshu Vashishtha Assignee: Himanshu Vashishtha Priority: Minor Fix For: 0.94.3 Attachments: HBASE-7142.patch The timeout in testDeadWorker is set to 1 sec, it is the same as the TimeoutMonitor thread timeout. In some case, this may fail: {code} java.lang.AssertionError at org.junit.Assert.fail(Assert.java:92) at org.junit.Assert.assertTrue(Assert.java:43) at org.junit.Assert.assertTrue(Assert.java:54) at org.apache.hadoop.hbase.master.TestSplitLogManager.waitForCounter(TestSplitLogManager.java:147) at org.apache.hadoop.hbase.master.TestSplitLogManager.waitForCounter(TestSplitLogManager.java:127) at org.apache.hadoop.hbase.master.TestSplitLogManager.testDeadWorker(TestSplitLogManager.java:433) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) {code} Fix is to increase the timeout for this test. Its not needed in trunk as the timeout is 3 seconds. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HBASE-7142) TestSplitLogManager#testDeadWorker may fail because of hard limit on the TimeoutMonitor's timeout period
[ https://issues.apache.org/jira/browse/HBASE-7142?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl resolved HBASE-7142. -- Resolution: Fixed Hadoop Flags: Reviewed Committed to 0.94. Thanks for the patch Himanshu. TestSplitLogManager#testDeadWorker may fail because of hard limit on the TimeoutMonitor's timeout period Key: HBASE-7142 URL: https://issues.apache.org/jira/browse/HBASE-7142 Project: HBase Issue Type: Test Components: test Affects Versions: 0.94.2 Reporter: Himanshu Vashishtha Assignee: Himanshu Vashishtha Priority: Minor Fix For: 0.94.3 Attachments: HBASE-7142.patch The timeout in testDeadWorker is set to 1 sec, it is the same as the TimeoutMonitor thread timeout. In some case, this may fail: {code} java.lang.AssertionError at org.junit.Assert.fail(Assert.java:92) at org.junit.Assert.assertTrue(Assert.java:43) at org.junit.Assert.assertTrue(Assert.java:54) at org.apache.hadoop.hbase.master.TestSplitLogManager.waitForCounter(TestSplitLogManager.java:147) at org.apache.hadoop.hbase.master.TestSplitLogManager.waitForCounter(TestSplitLogManager.java:127) at org.apache.hadoop.hbase.master.TestSplitLogManager.testDeadWorker(TestSplitLogManager.java:433) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) {code} Fix is to increase the timeout for this test. Its not needed in trunk as the timeout is 3 seconds. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6642) enable_all,disable_all,drop_all can call list command with regex directly.
[ https://issues.apache.org/jira/browse/HBASE-6642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13494995#comment-13494995 ] Lars Hofhansl commented on HBASE-6642: -- Should we also fix the misleading messaging? It should not print a message indicating successful execution when in fact it was not. enable_all,disable_all,drop_all can call list command with regex directly. Key: HBASE-6642 URL: https://issues.apache.org/jira/browse/HBASE-6642 Project: HBase Issue Type: Bug Components: shell Affects Versions: 0.92.2, 0.94.0 Reporter: Y. SREENIVASULU REDDY Assignee: rajeshbabu Fix For: 0.92.3, 0.94.3, 0.96.0 Attachments: HBASE-6642_trunk.patch created few tables. then performing disable_all operation in shell prompt. but it is not performing operation successfully. {noformat} hbase(main):043:0 disable_all '*' table12 zk0113 zk0114 Disable the above 3 tables (y/n)? y/ 3 tables successfully disabled just it is showing the message but operation is not success. but the following way only performing successfully hbase(main):043:0 disable_all '*.*' table12 zk0113 zk0114 Disable the above 3 tables (y/n)? y 3 tables successfully disabled {noformat} -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5898) Consider double-checked locking for block cache lock
[ https://issues.apache.org/jira/browse/HBASE-5898?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl updated HBASE-5898: - Attachment: 5898-v2.txt Rebased trunk patch. (want to get 0.94.3 RC out) Consider double-checked locking for block cache lock Key: HBASE-5898 URL: https://issues.apache.org/jira/browse/HBASE-5898 Project: HBase Issue Type: Improvement Components: Performance Affects Versions: 0.94.1 Reporter: Todd Lipcon Assignee: Todd Lipcon Priority: Critical Fix For: 0.94.3, 0.96.0 Attachments: 5898-TestBlocksRead.txt, 5898-v2.txt, HBASE-5898-0.patch, HBASE-5898-1.patch, HBASE-5898-1.patch, hbase-5898.txt Running a workload with a high query rate against a dataset that fits in cache, I saw a lot of CPU being used in IdLock.getLockEntry, being called by HFileReaderV2.readBlock. Even though it was all cache hits, it was wasting a lot of CPU doing lock management here. I wrote a quick patch to switch to a double-checked locking and it improved throughput substantially for this workload. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5898) Consider double-checked locking for block cache lock
[ https://issues.apache.org/jira/browse/HBASE-5898?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl updated HBASE-5898: - Status: Open (was: Patch Available) Oops. No, not quite the same. Consider double-checked locking for block cache lock Key: HBASE-5898 URL: https://issues.apache.org/jira/browse/HBASE-5898 Project: HBase Issue Type: Improvement Components: Performance Affects Versions: 0.94.1 Reporter: Todd Lipcon Assignee: Todd Lipcon Priority: Critical Fix For: 0.94.3, 0.96.0 Attachments: 5898-TestBlocksRead.txt, 5898-v2.txt, HBASE-5898-0.patch, HBASE-5898-1.patch, HBASE-5898-1.patch, hbase-5898.txt Running a workload with a high query rate against a dataset that fits in cache, I saw a lot of CPU being used in IdLock.getLockEntry, being called by HFileReaderV2.readBlock. Even though it was all cache hits, it was wasting a lot of CPU doing lock management here. I wrote a quick patch to switch to a double-checked locking and it improved throughput substantially for this workload. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5898) Consider double-checked locking for block cache lock
[ https://issues.apache.org/jira/browse/HBASE-5898?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl updated HBASE-5898: - Status: Patch Available (was: Open) Arggh... NM. This is the right patch. Consider double-checked locking for block cache lock Key: HBASE-5898 URL: https://issues.apache.org/jira/browse/HBASE-5898 Project: HBase Issue Type: Improvement Components: Performance Affects Versions: 0.94.1 Reporter: Todd Lipcon Assignee: Todd Lipcon Priority: Critical Fix For: 0.94.3, 0.96.0 Attachments: 5898-TestBlocksRead.txt, 5898-v2.txt, HBASE-5898-0.patch, HBASE-5898-1.patch, HBASE-5898-1.patch, hbase-5898.txt Running a workload with a high query rate against a dataset that fits in cache, I saw a lot of CPU being used in IdLock.getLockEntry, being called by HFileReaderV2.readBlock. Even though it was all cache hits, it was wasting a lot of CPU doing lock management here. I wrote a quick patch to switch to a double-checked locking and it improved throughput substantially for this workload. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7101) HBase stuck in Region SPLIT
[ https://issues.apache.org/jira/browse/HBASE-7101?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl updated HBASE-7101: - Fix Version/s: (was: 0.94.3) 0.94.4 Pushing to 0.94.4 after all. HBase stuck in Region SPLIT Key: HBASE-7101 URL: https://issues.apache.org/jira/browse/HBASE-7101 Project: HBase Issue Type: Bug Affects Versions: 0.92.1 Reporter: Bing Jiang Fix For: 0.96.0, 0.94.4 I found this issue from a zknode which has existed for a long time in the unassigned parent.And HMaster report warnning log increasingly.The loop log is at below. WARN org.apache.hadoop.hbase.master.AssignmentManager: Region 1a1c950ad45812d7b4b9b90ebf268468 not found on server sev0040,60020,1350378314041; failed processing WARN org.apache.hadoop.hbase.master.AssignmentManager: Received SPLIT for region 1a1c950ad45812d7b4b9b90ebf268468 from server sev0040,60020,1350378314041 but it doesn't exist anymore, probably already processed its split WARN org.apache.hadoop.hbase.master.AssignmentManager: Region 1a1c950ad45812d7b4b9b90ebf268468 not found on server gs-dpo-sev0040,60020,1350378314041; failed processing WARN org.apache.hadoop.hbase.master.AssignmentManager: Received SPLIT for region 1a1c950ad45812d7b4b9b90ebf268468 from server sev0040,60020,1350378314041 but it doesn't exist anymore, probably already processed its split we use Hbase-0.92.1, and I trace back to the source code. HMaster AssignmentManager have already deleted the SPLIT_Region in its memory structure,but HRegionServer SplitTransaction has found the unassigned/parent-node existed in a transient state, precisely SplitTransaction executes tickleNodeSplit to update a new version a little later than AssignmentManager deleting unassigned/parent-znode. After updating a version of the znode, it will intrigue the handleRegion operation again, however, AssignmentManager assert that the RegionState in Memory has been deleted, and transaction goes into a retry loop. In the SplitTransaction, transitionZKNode will retry tickleNodeSplit after sleeping 100ms. In my opinion, if the time is much longger than 100ms, all the operation from AssignmentManagement will finish off completely. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5898) Consider double-checked locking for block cache lock
[ https://issues.apache.org/jira/browse/HBASE-5898?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl updated HBASE-5898: - Attachment: 5898-v2.txt Figured my confusion. Took the patch from the wrong directory. This version is good now. Consider double-checked locking for block cache lock Key: HBASE-5898 URL: https://issues.apache.org/jira/browse/HBASE-5898 Project: HBase Issue Type: Improvement Components: Performance Affects Versions: 0.94.1 Reporter: Todd Lipcon Assignee: Todd Lipcon Priority: Critical Fix For: 0.94.3, 0.96.0 Attachments: 5898-TestBlocksRead.txt, 5898-v2.txt, 5898-v2.txt, HBASE-5898-0.patch, HBASE-5898-1.patch, HBASE-5898-1.patch, hbase-5898.txt Running a workload with a high query rate against a dataset that fits in cache, I saw a lot of CPU being used in IdLock.getLockEntry, being called by HFileReaderV2.readBlock. Even though it was all cache hits, it was wasting a lot of CPU doing lock management here. I wrote a quick patch to switch to a double-checked locking and it improved throughput substantially for this workload. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5898) Consider double-checked locking for block cache lock
[ https://issues.apache.org/jira/browse/HBASE-5898?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl updated HBASE-5898: - Status: Open (was: Patch Available) I can't make patches. Consider double-checked locking for block cache lock Key: HBASE-5898 URL: https://issues.apache.org/jira/browse/HBASE-5898 Project: HBase Issue Type: Improvement Components: Performance Affects Versions: 0.94.1 Reporter: Todd Lipcon Assignee: Todd Lipcon Priority: Critical Fix For: 0.94.3, 0.96.0 Attachments: 5898-TestBlocksRead.txt, 5898-v2.txt, HBASE-5898-0.patch, HBASE-5898-1.patch, HBASE-5898-1.patch, hbase-5898.txt Running a workload with a high query rate against a dataset that fits in cache, I saw a lot of CPU being used in IdLock.getLockEntry, being called by HFileReaderV2.readBlock. Even though it was all cache hits, it was wasting a lot of CPU doing lock management here. I wrote a quick patch to switch to a double-checked locking and it improved throughput substantially for this workload. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5898) Consider double-checked locking for block cache lock
[ https://issues.apache.org/jira/browse/HBASE-5898?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl updated HBASE-5898: - Attachment: (was: 5898-v2.txt) Consider double-checked locking for block cache lock Key: HBASE-5898 URL: https://issues.apache.org/jira/browse/HBASE-5898 Project: HBase Issue Type: Improvement Components: Performance Affects Versions: 0.94.1 Reporter: Todd Lipcon Assignee: Todd Lipcon Priority: Critical Fix For: 0.94.3, 0.96.0 Attachments: 5898-TestBlocksRead.txt, 5898-v2.txt, HBASE-5898-0.patch, HBASE-5898-1.patch, HBASE-5898-1.patch, hbase-5898.txt Running a workload with a high query rate against a dataset that fits in cache, I saw a lot of CPU being used in IdLock.getLockEntry, being called by HFileReaderV2.readBlock. Even though it was all cache hits, it was wasting a lot of CPU doing lock management here. I wrote a quick patch to switch to a double-checked locking and it improved throughput substantially for this workload. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5898) Consider double-checked locking for block cache lock
[ https://issues.apache.org/jira/browse/HBASE-5898?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl updated HBASE-5898: - Attachment: (was: 5898-v2.txt) Consider double-checked locking for block cache lock Key: HBASE-5898 URL: https://issues.apache.org/jira/browse/HBASE-5898 Project: HBase Issue Type: Improvement Components: Performance Affects Versions: 0.94.1 Reporter: Todd Lipcon Assignee: Todd Lipcon Priority: Critical Fix For: 0.94.3, 0.96.0 Attachments: 5898-TestBlocksRead.txt, 5898-v2.txt, HBASE-5898-0.patch, HBASE-5898-1.patch, HBASE-5898-1.patch, hbase-5898.txt Running a workload with a high query rate against a dataset that fits in cache, I saw a lot of CPU being used in IdLock.getLockEntry, being called by HFileReaderV2.readBlock. Even though it was all cache hits, it was wasting a lot of CPU doing lock management here. I wrote a quick patch to switch to a double-checked locking and it improved throughput substantially for this workload. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5898) Consider double-checked locking for block cache lock
[ https://issues.apache.org/jira/browse/HBASE-5898?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl updated HBASE-5898: - Attachment: 5898-v2.txt Last try. Consider double-checked locking for block cache lock Key: HBASE-5898 URL: https://issues.apache.org/jira/browse/HBASE-5898 Project: HBase Issue Type: Improvement Components: Performance Affects Versions: 0.94.1 Reporter: Todd Lipcon Assignee: Todd Lipcon Priority: Critical Fix For: 0.94.3, 0.96.0 Attachments: 5898-TestBlocksRead.txt, 5898-v2.txt, HBASE-5898-0.patch, HBASE-5898-1.patch, HBASE-5898-1.patch, hbase-5898.txt Running a workload with a high query rate against a dataset that fits in cache, I saw a lot of CPU being used in IdLock.getLockEntry, being called by HFileReaderV2.readBlock. Even though it was all cache hits, it was wasting a lot of CPU doing lock management here. I wrote a quick patch to switch to a double-checked locking and it improved throughput substantially for this workload. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5898) Consider double-checked locking for block cache lock
[ https://issues.apache.org/jira/browse/HBASE-5898?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl updated HBASE-5898: - Status: Patch Available (was: Open) Consider double-checked locking for block cache lock Key: HBASE-5898 URL: https://issues.apache.org/jira/browse/HBASE-5898 Project: HBase Issue Type: Improvement Components: Performance Affects Versions: 0.94.1 Reporter: Todd Lipcon Assignee: Todd Lipcon Priority: Critical Fix For: 0.94.3, 0.96.0 Attachments: 5898-TestBlocksRead.txt, 5898-v2.txt, HBASE-5898-0.patch, HBASE-5898-1.patch, HBASE-5898-1.patch, hbase-5898.txt Running a workload with a high query rate against a dataset that fits in cache, I saw a lot of CPU being used in IdLock.getLockEntry, being called by HFileReaderV2.readBlock. Even though it was all cache hits, it was wasting a lot of CPU doing lock management here. I wrote a quick patch to switch to a double-checked locking and it improved throughput substantially for this workload. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6721) RegionServer Group based Assignment
[ https://issues.apache.org/jira/browse/HBASE-6721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13495007#comment-13495007 ] Ted Yu commented on HBASE-6721: --- I went over GroupBasedLoadBalancer. In correctAssignments(): {code} +}catch(IOException exp){ + LOG.debug(Group information null for region of table + region.getTableNameAsString(), + exp); +} +if ((info == null) || (!info.containsServer(sName.getHostAndPort( { {code} If groupManager.getGroup() fails to return group information, it is likely the exception would occur multiple times inside the loop. We should distinguish this from the case of !info.containsServer(sName.getHostAndPort()). Meaning, in case of exception, we don't conclude that the regions are misplaced. In roundRobinAssignment(): {code} + regionGroup.get(groupKey), getServerToAssign(info, servers))); {code} Rename getServerToAssign() - getServersToAssign() {code} + assignments.put(ServerName.parseServerName(127.0.0.1:1),regions); {code} The above statement would be executed many times. Consider creating a singleton for ServerName.parseServerName(127.0.0.1:1). {code} + throw new IllegalStateException(Failed to access group store, e); {code} The above would cause master to exit, right ? If you don't want to change the signature of roundRobinAssignment(), consider returning null. {code} +if(nulled.size() 0) { {code} style: insert space between if and (. offlineRetainAssignment() contains only one method call. Consider merging it into retainAssignment(). {code} +//chances are most are not, then we just use balance to correct {code} 'are not': not correct or not incorrect ? Please clarify. It would be nice to end each line of comment with a period. {code} + ListMultimapString, HRegionInfo rGroup = ArrayListMultimap.create(); {code} Would groupToRegionMap be better name for the above variable ? RegionServer Group based Assignment --- Key: HBASE-6721 URL: https://issues.apache.org/jira/browse/HBASE-6721 Project: HBase Issue Type: New Feature Reporter: Francis Liu Assignee: Vandana Ayyalasomayajula Fix For: 0.96.0 Attachments: HBASE-6721_94_2.patch, HBASE-6721_94_3.patch, HBASE-6721_94_3.patch, HBASE-6721_94.patch, HBASE-6721_94.patch, HBASE-6721-DesigDoc.pdf In multi-tenant deployments of HBase, it is likely that a RegionServer will be serving out regions from a number of different tables owned by various client applications. Being able to group a subset of running RegionServers and assign specific tables to it, provides a client application a level of isolation and resource allocation. The proposal essentially is to have an AssignmentManager which is aware of RegionServer groups and assigns tables to region servers based on groupings. Load balancing will occur on a per group basis as well. This is essentially a simplification of the approach taken in HBASE-4120. See attached document. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7103) Need to fail split if SPLIT znode is deleted even before the split is completed.
[ https://issues.apache.org/jira/browse/HBASE-7103?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13495008#comment-13495008 ] Lars Hofhansl commented on HBASE-7103: -- This is not really my area of expertise, but the patch makes sense. We should probably remove STARTED_SPLITTING from JournalEntry (it's not use anywhere after this patch). Otherwise +1. @Stack: Wanna have a look? Need to fail split if SPLIT znode is deleted even before the split is completed. Key: HBASE-7103 URL: https://issues.apache.org/jira/browse/HBASE-7103 Project: HBase Issue Type: Bug Reporter: ramkrishna.s.vasudevan Assignee: ramkrishna.s.vasudevan Fix For: 0.94.3, 0.96.0 Attachments: 7103-6088-revert.txt, HBASE-7103_0.94.patch, HBASE-7103_testcase.patch This came up after the following mail in dev list 'infinite loop of RS_ZK_REGION_SPLIT on .94.2'. The following is the reason for the problem The following steps happen - Initially the parent region P1 starts splitting. - The split is going on normally. - Another split starts at the same time for the same region P1. (Not sure why this started). - Rollback happens seeing an already existing node. - This node gets deleted in rollback and nodeDeleted Event starts. - In nodeDeleted event the RIT for the region P1 gets deleted. - Because of this there is no region in RIT. - Now the first split gets over. Here the problem is we try to transit the node to SPLITTING to SPLIT. But the node even does not exist. But we don take any action on this. We think it is successful. - Because of this SplitRegionHandler never gets invoked. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7142) TestSplitLogManager#testDeadWorker may fail because of hard limit on the TimeoutMonitor's timeout period
[ https://issues.apache.org/jira/browse/HBASE-7142?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13495010#comment-13495010 ] Hudson commented on HBASE-7142: --- Integrated in HBase-0.94 #579 (See [https://builds.apache.org/job/HBase-0.94/579/]) HBASE-7142 TestSplitLogManager#testDeadWorker may fail because of hard limit on the TimeoutMonitor's timeout period (Himanshu) (Revision 1408119) Result = SUCCESS larsh : Files : * /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/master/TestSplitLogManager.java TestSplitLogManager#testDeadWorker may fail because of hard limit on the TimeoutMonitor's timeout period Key: HBASE-7142 URL: https://issues.apache.org/jira/browse/HBASE-7142 Project: HBase Issue Type: Test Components: test Affects Versions: 0.94.2 Reporter: Himanshu Vashishtha Assignee: Himanshu Vashishtha Priority: Minor Fix For: 0.94.3 Attachments: HBASE-7142.patch The timeout in testDeadWorker is set to 1 sec, it is the same as the TimeoutMonitor thread timeout. In some case, this may fail: {code} java.lang.AssertionError at org.junit.Assert.fail(Assert.java:92) at org.junit.Assert.assertTrue(Assert.java:43) at org.junit.Assert.assertTrue(Assert.java:54) at org.apache.hadoop.hbase.master.TestSplitLogManager.waitForCounter(TestSplitLogManager.java:147) at org.apache.hadoop.hbase.master.TestSplitLogManager.waitForCounter(TestSplitLogManager.java:127) at org.apache.hadoop.hbase.master.TestSplitLogManager.testDeadWorker(TestSplitLogManager.java:433) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) {code} Fix is to increase the timeout for this test. Its not needed in trunk as the timeout is 3 seconds. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5898) Consider double-checked locking for block cache lock
[ https://issues.apache.org/jira/browse/HBASE-5898?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13495013#comment-13495013 ] Hadoop QA commented on HBASE-5898: -- {color:red}-1 overall{color}. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12553048/5898-v2.txt against trunk revision . {color:green}+1 @author{color}. The patch does not contain any @author tags. {color:red}-1 tests included{color}. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. {color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop 2.0 profile. {color:red}-1 javadoc{color}. The javadoc tool appears to have generated 87 warning messages. {color:green}+1 javac{color}. The applied patch does not increase the total number of javac compiler warnings. {color:red}-1 findbugs{color}. The patch appears to introduce 17 new Findbugs (version 1.3.9) warnings. {color:green}+1 release audit{color}. The applied patch does not increase the total number of release audit warnings. {color:red}-1 core tests{color}. The patch failed these unit tests: org.apache.hadoop.hbase.regionserver.TestStoreFile Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/3307//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3307//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3307//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3307//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3307//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3307//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3307//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/3307//console This message is automatically generated. Consider double-checked locking for block cache lock Key: HBASE-5898 URL: https://issues.apache.org/jira/browse/HBASE-5898 Project: HBase Issue Type: Improvement Components: Performance Affects Versions: 0.94.1 Reporter: Todd Lipcon Assignee: Todd Lipcon Priority: Critical Fix For: 0.94.3, 0.96.0 Attachments: 5898-TestBlocksRead.txt, 5898-v2.txt, HBASE-5898-0.patch, HBASE-5898-1.patch, HBASE-5898-1.patch, hbase-5898.txt Running a workload with a high query rate against a dataset that fits in cache, I saw a lot of CPU being used in IdLock.getLockEntry, being called by HFileReaderV2.readBlock. Even though it was all cache hits, it was wasting a lot of CPU doing lock management here. I wrote a quick patch to switch to a double-checked locking and it improved throughput substantially for this workload. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6721) RegionServer Group based Assignment
[ https://issues.apache.org/jira/browse/HBASE-6721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13495015#comment-13495015 ] Ted Yu commented on HBASE-6721: --- More comments about GroupBasedLoadBalancer. In randomAssignment(): {code} + } else if(SPECIAL_TABLES.contains(region.getTableName())){ +candidateList = servers; {code} If you limit special tables to default group, there is no need to move them after system becomes online. For GroupStartupWorker, isOnline is set to true after waitForGroupTableOnline() returns. In waitForGroupTableOnline(): {code} + if(found.get()) { + ((GroupBasedLoadBalancer)masterServices.getLoadBalancer()).getGroupInfoManager(); + } {code} Would the above call be effective (isOnline is false at this point) ? RegionServer Group based Assignment --- Key: HBASE-6721 URL: https://issues.apache.org/jira/browse/HBASE-6721 Project: HBase Issue Type: New Feature Reporter: Francis Liu Assignee: Vandana Ayyalasomayajula Fix For: 0.96.0 Attachments: HBASE-6721_94_2.patch, HBASE-6721_94_3.patch, HBASE-6721_94_3.patch, HBASE-6721_94.patch, HBASE-6721_94.patch, HBASE-6721-DesigDoc.pdf In multi-tenant deployments of HBase, it is likely that a RegionServer will be serving out regions from a number of different tables owned by various client applications. Being able to group a subset of running RegionServers and assign specific tables to it, provides a client application a level of isolation and resource allocation. The proposal essentially is to have an AssignmentManager which is aware of RegionServer groups and assigns tables to region servers based on groupings. Load balancing will occur on a per group basis as well. This is essentially a simplification of the approach taken in HBASE-4120. See attached document. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5898) Consider double-checked locking for block cache lock
[ https://issues.apache.org/jira/browse/HBASE-5898?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13495032#comment-13495032 ] Lars Hofhansl commented on HBASE-5898: -- TestStoreFile fails because it checks CacheStats.getMissCount(), which will be wrong even after modified patch. CacheStats.miss updates two counters: missCount and missCachingCount. Only missCachingCount is counted correctly. So, we can change the test and accept the wrong number for missCount, or we change the CacheStats API. Consider double-checked locking for block cache lock Key: HBASE-5898 URL: https://issues.apache.org/jira/browse/HBASE-5898 Project: HBase Issue Type: Improvement Components: Performance Affects Versions: 0.94.1 Reporter: Todd Lipcon Assignee: Todd Lipcon Priority: Critical Fix For: 0.94.3, 0.96.0 Attachments: 5898-TestBlocksRead.txt, 5898-v2.txt, HBASE-5898-0.patch, HBASE-5898-1.patch, HBASE-5898-1.patch, hbase-5898.txt Running a workload with a high query rate against a dataset that fits in cache, I saw a lot of CPU being used in IdLock.getLockEntry, being called by HFileReaderV2.readBlock. Even though it was all cache hits, it was wasting a lot of CPU doing lock management here. I wrote a quick patch to switch to a double-checked locking and it improved throughput substantially for this workload. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5898) Consider double-checked locking for block cache lock
[ https://issues.apache.org/jira/browse/HBASE-5898?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl updated HBASE-5898: - Attachment: 5898-v3.txt Version that fixes TestStoreFile (by using getMissCachingCount() instead of getMissCount()) The missCount is still over counted. Consider double-checked locking for block cache lock Key: HBASE-5898 URL: https://issues.apache.org/jira/browse/HBASE-5898 Project: HBase Issue Type: Improvement Components: Performance Affects Versions: 0.94.1 Reporter: Todd Lipcon Assignee: Todd Lipcon Priority: Critical Fix For: 0.94.3, 0.96.0 Attachments: 5898-TestBlocksRead.txt, 5898-v2.txt, 5898-v3.txt, HBASE-5898-0.patch, HBASE-5898-1.patch, HBASE-5898-1.patch, hbase-5898.txt Running a workload with a high query rate against a dataset that fits in cache, I saw a lot of CPU being used in IdLock.getLockEntry, being called by HFileReaderV2.readBlock. Even though it was all cache hits, it was wasting a lot of CPU doing lock management here. I wrote a quick patch to switch to a double-checked locking and it improved throughput substantially for this workload. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7103) Need to fail split if SPLIT znode is deleted even before the split is completed.
[ https://issues.apache.org/jira/browse/HBASE-7103?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13495040#comment-13495040 ] Lars Hofhansl commented on HBASE-7103: -- Also a trunk patch would be awesome so we can run against HadoopQA. Need to fail split if SPLIT znode is deleted even before the split is completed. Key: HBASE-7103 URL: https://issues.apache.org/jira/browse/HBASE-7103 Project: HBase Issue Type: Bug Reporter: ramkrishna.s.vasudevan Assignee: ramkrishna.s.vasudevan Fix For: 0.94.3, 0.96.0 Attachments: 7103-6088-revert.txt, HBASE-7103_0.94.patch, HBASE-7103_testcase.patch This came up after the following mail in dev list 'infinite loop of RS_ZK_REGION_SPLIT on .94.2'. The following is the reason for the problem The following steps happen - Initially the parent region P1 starts splitting. - The split is going on normally. - Another split starts at the same time for the same region P1. (Not sure why this started). - Rollback happens seeing an already existing node. - This node gets deleted in rollback and nodeDeleted Event starts. - In nodeDeleted event the RIT for the region P1 gets deleted. - Because of this there is no region in RIT. - Now the first split gets over. Here the problem is we try to transit the node to SPLITTING to SPLIT. But the node even does not exist. But we don take any action on this. We think it is successful. - Because of this SplitRegionHandler never gets invoked. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6775) Use ZK.multi when available for HBASE-6710 0.92/0.94 compatibility fix
[ https://issues.apache.org/jira/browse/HBASE-6775?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13495048#comment-13495048 ] Lars Hofhansl commented on HBASE-6775: -- I'd be OK with punting on this for 0.92 and 0.94. Use ZK.multi when available for HBASE-6710 0.92/0.94 compatibility fix -- Key: HBASE-6775 URL: https://issues.apache.org/jira/browse/HBASE-6775 Project: HBase Issue Type: Improvement Components: Zookeeper Affects Versions: 0.94.2 Reporter: Gregory Chanan Assignee: Gregory Chanan Priority: Minor Fix For: 0.94.4 HBASE-6710 fixed a 0.92/0.94 compatibility issue by writing two znodes in different formats. If a ZK failure occurs between the writing of the two znodes, strange behavior can result. This issue is a reminder to change these two ZK writes to use ZK.multi when we require ZK 3.4+. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4913) Per-CF compaction Via the Shell
[ https://issues.apache.org/jira/browse/HBASE-4913?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13495051#comment-13495051 ] Lars Hofhansl commented on HBASE-4913: -- Patch looks good +1 (for 0.94.3 if committed soon). Minor nit: we have this: {code} - private void compact(final byte [] tableNameOrRegionName, final boolean major) + private void compact(final byte [] tableNameOrRegionName, +final byte[] columnFamily,final boolean major) {code} and this: {code} private void compact(final ServerName sn, final HRegionInfo hri, - final boolean major) + final boolean major, final byte [] family) {code} Maybe the order should be consistent...? (Also a space missing the former.) But since it's in trunk already let's leave it this way in 0.94. Per-CF compaction Via the Shell --- Key: HBASE-4913 URL: https://issues.apache.org/jira/browse/HBASE-4913 Project: HBase Issue Type: Sub-task Components: Client, regionserver Reporter: Nicolas Spiegelberg Assignee: Mubarak Seyed Fix For: 0.96.0, 0.94.4 Attachments: HBASE-4913-94.patch, HBASE-4913-addendum.patch, HBASE-4913.trunk.v1.patch, HBASE-4913.trunk.v2.patch, HBASE-4913.trunk.v2.patch, HBASE-4913-trunk-v3.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5898) Consider double-checked locking for block cache lock
[ https://issues.apache.org/jira/browse/HBASE-5898?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13495053#comment-13495053 ] Hadoop QA commented on HBASE-5898: -- {color:red}-1 overall{color}. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12553057/5898-v3.txt against trunk revision . {color:green}+1 @author{color}. The patch does not contain any @author tags. {color:green}+1 tests included{color}. The patch appears to include 3 new or modified tests. {color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop 2.0 profile. {color:red}-1 javadoc{color}. The javadoc tool appears to have generated 87 warning messages. {color:green}+1 javac{color}. The applied patch does not increase the total number of javac compiler warnings. {color:red}-1 findbugs{color}. The patch appears to introduce 17 new Findbugs (version 1.3.9) warnings. {color:green}+1 release audit{color}. The applied patch does not increase the total number of release audit warnings. {color:red}-1 core tests{color}. The patch failed these unit tests: org.apache.hadoop.hbase.client.TestFromClientSide org.apache.hadoop.hbase.client.TestFromClientSideWithCoprocessor Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/3308//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3308//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3308//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3308//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3308//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3308//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3308//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/3308//console This message is automatically generated. Consider double-checked locking for block cache lock Key: HBASE-5898 URL: https://issues.apache.org/jira/browse/HBASE-5898 Project: HBase Issue Type: Improvement Components: Performance Affects Versions: 0.94.1 Reporter: Todd Lipcon Assignee: Todd Lipcon Priority: Critical Fix For: 0.94.3, 0.96.0 Attachments: 5898-TestBlocksRead.txt, 5898-v2.txt, 5898-v3.txt, HBASE-5898-0.patch, HBASE-5898-1.patch, HBASE-5898-1.patch, hbase-5898.txt Running a workload with a high query rate against a dataset that fits in cache, I saw a lot of CPU being used in IdLock.getLockEntry, being called by HFileReaderV2.readBlock. Even though it was all cache hits, it was wasting a lot of CPU doing lock management here. I wrote a quick patch to switch to a double-checked locking and it improved throughput substantially for this workload. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5898) Consider double-checked locking for block cache lock
[ https://issues.apache.org/jira/browse/HBASE-5898?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13495098#comment-13495098 ] stack commented on HBASE-5898: -- Seems fine by me Lars. Consider double-checked locking for block cache lock Key: HBASE-5898 URL: https://issues.apache.org/jira/browse/HBASE-5898 Project: HBase Issue Type: Improvement Components: Performance Affects Versions: 0.94.1 Reporter: Todd Lipcon Assignee: Todd Lipcon Priority: Critical Fix For: 0.94.3, 0.96.0 Attachments: 5898-TestBlocksRead.txt, 5898-v2.txt, 5898-v3.txt, HBASE-5898-0.patch, HBASE-5898-1.patch, HBASE-5898-1.patch, hbase-5898.txt Running a workload with a high query rate against a dataset that fits in cache, I saw a lot of CPU being used in IdLock.getLockEntry, being called by HFileReaderV2.readBlock. Even though it was all cache hits, it was wasting a lot of CPU doing lock management here. I wrote a quick patch to switch to a double-checked locking and it improved throughput substantially for this workload. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7103) Need to fail split if SPLIT znode is deleted even before the split is completed.
[ https://issues.apache.org/jira/browse/HBASE-7103?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13495101#comment-13495101 ] stack commented on HBASE-7103: -- +1 on patch. There is still a hole in here (deleting a znode though we're not sure it is ours) but it is narrower than the hole that was there previous. I would suggest removing the unused state as per Lars comment above and adding comment that we could be removing znode that we do not own if the transition from SPLITTING to SPLITTING fails (maybe we should create it w/o data or w/ another state but can do that in another issue... just note the problem in a TODO comment for now). Good on you Ram. Need to fail split if SPLIT znode is deleted even before the split is completed. Key: HBASE-7103 URL: https://issues.apache.org/jira/browse/HBASE-7103 Project: HBase Issue Type: Bug Reporter: ramkrishna.s.vasudevan Assignee: ramkrishna.s.vasudevan Fix For: 0.94.3, 0.96.0 Attachments: 7103-6088-revert.txt, HBASE-7103_0.94.patch, HBASE-7103_testcase.patch This came up after the following mail in dev list 'infinite loop of RS_ZK_REGION_SPLIT on .94.2'. The following is the reason for the problem The following steps happen - Initially the parent region P1 starts splitting. - The split is going on normally. - Another split starts at the same time for the same region P1. (Not sure why this started). - Rollback happens seeing an already existing node. - This node gets deleted in rollback and nodeDeleted Event starts. - In nodeDeleted event the RIT for the region P1 gets deleted. - Because of this there is no region in RIT. - Now the first split gets over. Here the problem is we try to transit the node to SPLITTING to SPLIT. But the node even does not exist. But we don take any action on this. We think it is successful. - Because of this SplitRegionHandler never gets invoked. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6793) Make hbase-examples module
[ https://issues.apache.org/jira/browse/HBASE-6793?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13495103#comment-13495103 ] stack commented on HBASE-6793: -- [~ted_yu] Make a new issue I'd say. Make hbase-examples module -- Key: HBASE-6793 URL: https://issues.apache.org/jira/browse/HBASE-6793 Project: HBase Issue Type: Improvement Affects Versions: 0.96.0 Reporter: Enis Soztutar Assignee: Sergey Shelukhin Labels: noob Fix For: 0.96.0 Attachments: HBASE-6793.patch, HBASE-6793-v2.patch, HBASE-6793-v3-thrift-0.9.0.patch, HBASE-6793-v4.1-squashed.patch, HBASE-6793-v5-squashed.patch There are some examples under /examples/, which are not compiled as a part of the build. We can move them to an hbase-examples module. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7103) Need to fail split if SPLIT znode is deleted even before the split is completed.
[ https://issues.apache.org/jira/browse/HBASE-7103?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramkrishna.s.vasudevan updated HBASE-7103: -- Attachment: HBASE-7103_0.94.patch Need to fail split if SPLIT znode is deleted even before the split is completed. Key: HBASE-7103 URL: https://issues.apache.org/jira/browse/HBASE-7103 Project: HBase Issue Type: Bug Reporter: ramkrishna.s.vasudevan Assignee: ramkrishna.s.vasudevan Fix For: 0.94.3, 0.96.0 Attachments: 7103-6088-revert.txt, HBASE-7103_0.94.patch, HBASE-7103_0.94.patch, HBASE-7103_testcase.patch, HBASE-7103_trunk.patch This came up after the following mail in dev list 'infinite loop of RS_ZK_REGION_SPLIT on .94.2'. The following is the reason for the problem The following steps happen - Initially the parent region P1 starts splitting. - The split is going on normally. - Another split starts at the same time for the same region P1. (Not sure why this started). - Rollback happens seeing an already existing node. - This node gets deleted in rollback and nodeDeleted Event starts. - In nodeDeleted event the RIT for the region P1 gets deleted. - Because of this there is no region in RIT. - Now the first split gets over. Here the problem is we try to transit the node to SPLITTING to SPLIT. But the node even does not exist. But we don take any action on this. We think it is successful. - Because of this SplitRegionHandler never gets invoked. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7103) Need to fail split if SPLIT znode is deleted even before the split is completed.
[ https://issues.apache.org/jira/browse/HBASE-7103?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramkrishna.s.vasudevan updated HBASE-7103: -- Attachment: HBASE-7103_trunk.patch Need to fail split if SPLIT znode is deleted even before the split is completed. Key: HBASE-7103 URL: https://issues.apache.org/jira/browse/HBASE-7103 Project: HBase Issue Type: Bug Reporter: ramkrishna.s.vasudevan Assignee: ramkrishna.s.vasudevan Fix For: 0.94.3, 0.96.0 Attachments: 7103-6088-revert.txt, HBASE-7103_0.94.patch, HBASE-7103_0.94.patch, HBASE-7103_testcase.patch, HBASE-7103_trunk.patch This came up after the following mail in dev list 'infinite loop of RS_ZK_REGION_SPLIT on .94.2'. The following is the reason for the problem The following steps happen - Initially the parent region P1 starts splitting. - The split is going on normally. - Another split starts at the same time for the same region P1. (Not sure why this started). - Rollback happens seeing an already existing node. - This node gets deleted in rollback and nodeDeleted Event starts. - In nodeDeleted event the RIT for the region P1 gets deleted. - Because of this there is no region in RIT. - Now the first split gets over. Here the problem is we try to transit the node to SPLITTING to SPLIT. But the node even does not exist. But we don take any action on this. We think it is successful. - Because of this SplitRegionHandler never gets invoked. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7103) Need to fail split if SPLIT znode is deleted even before the split is completed.
[ https://issues.apache.org/jira/browse/HBASE-7103?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramkrishna.s.vasudevan updated HBASE-7103: -- Status: Open (was: Patch Available) Need to fail split if SPLIT znode is deleted even before the split is completed. Key: HBASE-7103 URL: https://issues.apache.org/jira/browse/HBASE-7103 Project: HBase Issue Type: Bug Reporter: ramkrishna.s.vasudevan Assignee: ramkrishna.s.vasudevan Fix For: 0.94.3, 0.96.0 Attachments: 7103-6088-revert.txt, HBASE-7103_0.94.patch, HBASE-7103_0.94.patch, HBASE-7103_testcase.patch, HBASE-7103_trunk.patch This came up after the following mail in dev list 'infinite loop of RS_ZK_REGION_SPLIT on .94.2'. The following is the reason for the problem The following steps happen - Initially the parent region P1 starts splitting. - The split is going on normally. - Another split starts at the same time for the same region P1. (Not sure why this started). - Rollback happens seeing an already existing node. - This node gets deleted in rollback and nodeDeleted Event starts. - In nodeDeleted event the RIT for the region P1 gets deleted. - Because of this there is no region in RIT. - Now the first split gets over. Here the problem is we try to transit the node to SPLITTING to SPLIT. But the node even does not exist. But we don take any action on this. We think it is successful. - Because of this SplitRegionHandler never gets invoked. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7103) Need to fail split if SPLIT znode is deleted even before the split is completed.
[ https://issues.apache.org/jira/browse/HBASE-7103?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramkrishna.s.vasudevan updated HBASE-7103: -- Status: Patch Available (was: Open) Need to fail split if SPLIT znode is deleted even before the split is completed. Key: HBASE-7103 URL: https://issues.apache.org/jira/browse/HBASE-7103 Project: HBase Issue Type: Bug Reporter: ramkrishna.s.vasudevan Assignee: ramkrishna.s.vasudevan Fix For: 0.94.3, 0.96.0 Attachments: 7103-6088-revert.txt, HBASE-7103_0.94.patch, HBASE-7103_0.94.patch, HBASE-7103_testcase.patch, HBASE-7103_trunk.patch This came up after the following mail in dev list 'infinite loop of RS_ZK_REGION_SPLIT on .94.2'. The following is the reason for the problem The following steps happen - Initially the parent region P1 starts splitting. - The split is going on normally. - Another split starts at the same time for the same region P1. (Not sure why this started). - Rollback happens seeing an already existing node. - This node gets deleted in rollback and nodeDeleted Event starts. - In nodeDeleted event the RIT for the region P1 gets deleted. - Because of this there is no region in RIT. - Now the first split gets over. Here the problem is we try to transit the node to SPLITTING to SPLIT. But the node even does not exist. But we don take any action on this. We think it is successful. - Because of this SplitRegionHandler never gets invoked. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7103) Need to fail split if SPLIT znode is deleted even before the split is completed.
[ https://issues.apache.org/jira/browse/HBASE-7103?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13495120#comment-13495120 ] ramkrishna.s.vasudevan commented on HBASE-7103: --- Attached patch for trunk and 0.94. I think i have removed the unused state Lars. Added the comments and also the TODO. @Stack I was thinking about new state and infact had some idea on mind. But did not want to complicate it now with new states and handling it in master side should be done with proper care. Anyway will come up with some idea sooner. bq.deleting a znode though we're not sure it is ours This could be a problem? Why so? Because now if the znode exists we will not start the split anyway so there is only one split right going on? Anyway the node deletion is done by master. May be am missing something Stack. Thanks a lot for review. Need to fail split if SPLIT znode is deleted even before the split is completed. Key: HBASE-7103 URL: https://issues.apache.org/jira/browse/HBASE-7103 Project: HBase Issue Type: Bug Reporter: ramkrishna.s.vasudevan Assignee: ramkrishna.s.vasudevan Fix For: 0.94.3, 0.96.0 Attachments: 7103-6088-revert.txt, HBASE-7103_0.94.patch, HBASE-7103_0.94.patch, HBASE-7103_testcase.patch, HBASE-7103_trunk.patch This came up after the following mail in dev list 'infinite loop of RS_ZK_REGION_SPLIT on .94.2'. The following is the reason for the problem The following steps happen - Initially the parent region P1 starts splitting. - The split is going on normally. - Another split starts at the same time for the same region P1. (Not sure why this started). - Rollback happens seeing an already existing node. - This node gets deleted in rollback and nodeDeleted Event starts. - In nodeDeleted event the RIT for the region P1 gets deleted. - Because of this there is no region in RIT. - Now the first split gets over. Here the problem is we try to transit the node to SPLITTING to SPLIT. But the node even does not exist. But we don take any action on this. We think it is successful. - Because of this SplitRegionHandler never gets invoked. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6721) RegionServer Group based Assignment
[ https://issues.apache.org/jira/browse/HBASE-6721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13495129#comment-13495129 ] Francis Liu commented on HBASE-6721: {quote} Should we confine these special tables to default group ? {quote} That's a good point, I just realized that it's not possible to add table properties to the catalog tables. Looks like I will have to move group membership into the group metadata table as well. It felt clunky that changing group membership would be considered as a schema change anyway. RegionServer Group based Assignment --- Key: HBASE-6721 URL: https://issues.apache.org/jira/browse/HBASE-6721 Project: HBase Issue Type: New Feature Reporter: Francis Liu Assignee: Vandana Ayyalasomayajula Fix For: 0.96.0 Attachments: HBASE-6721_94_2.patch, HBASE-6721_94_3.patch, HBASE-6721_94_3.patch, HBASE-6721_94.patch, HBASE-6721_94.patch, HBASE-6721-DesigDoc.pdf In multi-tenant deployments of HBase, it is likely that a RegionServer will be serving out regions from a number of different tables owned by various client applications. Being able to group a subset of running RegionServers and assign specific tables to it, provides a client application a level of isolation and resource allocation. The proposal essentially is to have an AssignmentManager which is aware of RegionServer groups and assigns tables to region servers based on groupings. Load balancing will occur on a per group basis as well. This is essentially a simplification of the approach taken in HBASE-4120. See attached document. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7103) Need to fail split if SPLIT znode is deleted even before the split is completed.
[ https://issues.apache.org/jira/browse/HBASE-7103?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13495141#comment-13495141 ] Hadoop QA commented on HBASE-7103: -- {color:red}-1 overall{color}. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12553079/HBASE-7103_trunk.patch against trunk revision . {color:green}+1 @author{color}. The patch does not contain any @author tags. {color:green}+1 tests included{color}. The patch appears to include 3 new or modified tests. {color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop 2.0 profile. {color:red}-1 javadoc{color}. The javadoc tool appears to have generated 87 warning messages. {color:green}+1 javac{color}. The applied patch does not increase the total number of javac compiler warnings. {color:red}-1 findbugs{color}. The patch appears to introduce 18 new Findbugs (version 1.3.9) warnings. {color:green}+1 release audit{color}. The applied patch does not increase the total number of release audit warnings. {color:green}+1 core tests{color}. The patch passed unit tests in . Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/3309//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3309//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3309//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3309//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3309//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3309//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3309//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/3309//console This message is automatically generated. Need to fail split if SPLIT znode is deleted even before the split is completed. Key: HBASE-7103 URL: https://issues.apache.org/jira/browse/HBASE-7103 Project: HBase Issue Type: Bug Reporter: ramkrishna.s.vasudevan Assignee: ramkrishna.s.vasudevan Fix For: 0.94.3, 0.96.0 Attachments: 7103-6088-revert.txt, HBASE-7103_0.94.patch, HBASE-7103_0.94.patch, HBASE-7103_testcase.patch, HBASE-7103_trunk.patch This came up after the following mail in dev list 'infinite loop of RS_ZK_REGION_SPLIT on .94.2'. The following is the reason for the problem The following steps happen - Initially the parent region P1 starts splitting. - The split is going on normally. - Another split starts at the same time for the same region P1. (Not sure why this started). - Rollback happens seeing an already existing node. - This node gets deleted in rollback and nodeDeleted Event starts. - In nodeDeleted event the RIT for the region P1 gets deleted. - Because of this there is no region in RIT. - Now the first split gets over. Here the problem is we try to transit the node to SPLITTING to SPLIT. But the node even does not exist. But we don take any action on this. We think it is successful. - Because of this SplitRegionHandler never gets invoked. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira