[jira] [Commented] (HBASE-7390) Add extra test cases for assignement on the region server and fix the related issues
[ https://issues.apache.org/jira/browse/HBASE-7390?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13541353#comment-13541353 ] Hudson commented on HBASE-7390: --- Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #321 (See [https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/321/]) HBASE-7390 Add extra test cases for assignement on the region server and fix the related issues (Revision 1427064) Result = FAILURE nkeywal : Files : * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/client/HTable.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RegionServerServices.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/handler/CloseRegionHandler.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/handler/OpenRegionHandler.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/zookeeper/ZKAssign.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/master/MockRegionServer.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestRegionServerNoMaster.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/handler/TestOpenRegionHandler.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/util/MockRegionServerServices.java > Add extra test cases for assignement on the region server and fix the related > issues > > > Key: HBASE-7390 > URL: https://issues.apache.org/jira/browse/HBASE-7390 > Project: HBase > Issue Type: Bug > Components: Region Assignment, regionserver >Affects Versions: 0.96.0 >Reporter: nkeywal >Assignee: nkeywal > Fix For: 0.96.0 > > Attachments: 7390.v10.patch, 7390.v11.patch, 7390.v1.patch, > 7390.v2.patch, 7390.v3.patch, 7390.v4.patch, 7390.v6.patch, 7390.v7.patch, > 7390.v8.patch, 7390.v9.patch, 7390.v9.patch, assignment_zk_states.jpg > > > We don't have a lot of tests on the region server itself. > Here are some. > Some of them are failing, feedback welcome. > See as well the attached state diagram for the ZK nodes on assignment. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7390) Add extra test cases for assignement on the region server and fix the related issues
[ https://issues.apache.org/jira/browse/HBASE-7390?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13541330#comment-13541330 ] Hudson commented on HBASE-7390: --- Integrated in HBase-TRUNK #3676 (See [https://builds.apache.org/job/HBase-TRUNK/3676/]) HBASE-7390 Add extra test cases for assignement on the region server and fix the related issues (Revision 1427064) Result = FAILURE nkeywal : Files : * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/client/HTable.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RegionServerServices.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/handler/CloseRegionHandler.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/handler/OpenRegionHandler.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/zookeeper/ZKAssign.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/master/MockRegionServer.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestRegionServerNoMaster.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/handler/TestOpenRegionHandler.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/util/MockRegionServerServices.java > Add extra test cases for assignement on the region server and fix the related > issues > > > Key: HBASE-7390 > URL: https://issues.apache.org/jira/browse/HBASE-7390 > Project: HBase > Issue Type: Bug > Components: Region Assignment, regionserver >Affects Versions: 0.96.0 >Reporter: nkeywal >Assignee: nkeywal > Fix For: 0.96.0 > > Attachments: 7390.v10.patch, 7390.v11.patch, 7390.v1.patch, > 7390.v2.patch, 7390.v3.patch, 7390.v4.patch, 7390.v6.patch, 7390.v7.patch, > 7390.v8.patch, 7390.v9.patch, 7390.v9.patch, assignment_zk_states.jpg > > > We don't have a lot of tests on the region server itself. > Here are some. > Some of them are failing, feedback welcome. > See as well the attached state diagram for the ZK nodes on assignment. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7390) Add extra test cases for assignement on the region server and fix the related issues
[ https://issues.apache.org/jira/browse/HBASE-7390?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13541327#comment-13541327 ] nkeywal commented on HBASE-7390: v11 is what I committed. > Add extra test cases for assignement on the region server and fix the related > issues > > > Key: HBASE-7390 > URL: https://issues.apache.org/jira/browse/HBASE-7390 > Project: HBase > Issue Type: Bug > Components: Region Assignment, regionserver >Affects Versions: 0.96.0 >Reporter: nkeywal >Assignee: nkeywal > Fix For: 0.96.0 > > Attachments: 7390.v10.patch, 7390.v11.patch, 7390.v1.patch, > 7390.v2.patch, 7390.v3.patch, 7390.v4.patch, 7390.v6.patch, 7390.v7.patch, > 7390.v8.patch, 7390.v9.patch, 7390.v9.patch, assignment_zk_states.jpg > > > We don't have a lot of tests on the region server itself. > Here are some. > Some of them are failing, feedback welcome. > See as well the attached state diagram for the ZK nodes on assignment. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7390) Add extra test cases for assignement on the region server and fix the related issues
[ https://issues.apache.org/jira/browse/HBASE-7390?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13541165#comment-13541165 ] stack commented on HBASE-7390: -- Above sounds reasonable to me. Regards encapsulating RIT, Jimmy did this for the server-side nicely makes sense that RS could get same treatment. > Add extra test cases for assignement on the region server and fix the related > issues > > > Key: HBASE-7390 > URL: https://issues.apache.org/jira/browse/HBASE-7390 > Project: HBase > Issue Type: Bug > Components: Region Assignment, regionserver >Affects Versions: 0.96.0 >Reporter: nkeywal >Assignee: nkeywal > Fix For: 0.96.0 > > Attachments: 7390.v10.patch, 7390.v1.patch, 7390.v2.patch, > 7390.v3.patch, 7390.v4.patch, 7390.v6.patch, 7390.v7.patch, 7390.v8.patch, > 7390.v9.patch, 7390.v9.patch, assignment_zk_states.jpg > > > We don't have a lot of tests on the region server itself. > Here are some. > Some of them are failing, feedback welcome. > See as well the attached state diagram for the ZK nodes on assignment. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7390) Add extra test cases for assignement on the region server and fix the related issues
[ https://issues.apache.org/jira/browse/HBASE-7390?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13541164#comment-13541164 ] nkeywal commented on HBASE-7390: bq. You have lines > 100 characters according to the hadoopqa output! Hadoopqa is right :-). I will fix it on commit. bq. This is no longer deprecated? It was a little bit strange to have this one deprecated while the ones with two parameters / same method name were not. So I thought it was better to de-deprecate it. bq. This should not be javadoc: Hum, I thought it made sense to have it in the javadoc: someone using (vs. developing) HBase would need to know that if a method is not private, it's not because it's supposed to be used or extended, but only for internal testing. bq. public ConcurrentSkipListMap getRegionsInTransitionInRS(); I kept the change even if iirc I finally don't need it for this patch. What was useful was pure concurrent oriented stuff like functions putif-absent or removeif-absent. So the best interface would be ConcurrentMap. I will change this on commit. It's beyond the scope of this patch, but the RIT should be more encapsulated, especially now that we sometimes hijack it to cancel an open... This makes me think as well that the best implementation class would be ConcurrentHashMap instead of ConcurrentSkipListMap. But's it will be for another patch as well :-) Thanks for the review, Stack, I will commit this tomorrow (and start to go forward with the other jira I had on assignment: the depend on this one). > Add extra test cases for assignement on the region server and fix the related > issues > > > Key: HBASE-7390 > URL: https://issues.apache.org/jira/browse/HBASE-7390 > Project: HBase > Issue Type: Bug > Components: Region Assignment, regionserver >Affects Versions: 0.96.0 >Reporter: nkeywal >Assignee: nkeywal > Fix For: 0.96.0 > > Attachments: 7390.v10.patch, 7390.v1.patch, 7390.v2.patch, > 7390.v3.patch, 7390.v4.patch, 7390.v6.patch, 7390.v7.patch, 7390.v8.patch, > 7390.v9.patch, 7390.v9.patch, assignment_zk_states.jpg > > > We don't have a lot of tests on the region server itself. > Here are some. > Some of them are failing, feedback welcome. > See as well the attached state diagram for the ZK nodes on assignment. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7390) Add extra test cases for assignement on the region server and fix the related issues
[ https://issues.apache.org/jira/browse/HBASE-7390?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13541162#comment-13541162 ] stack commented on HBASE-7390: -- Patch is great. You have two +1s up in RB. Here is another +1 from me... The tests are great as are the comments. You have lines > 100 characters according to the hadoopqa output! This is no longer deprecated? - * @deprecated use {@link #getRegionLocation(byte [], boolean)} instead This should not be javadoc: + * + * Protected to ease testing. Could be a comment inside the method. Does it have to be this explicit type: + public ConcurrentSkipListMap getRegionsInTransitionInRS(); Can it be NavigableMap or SortedMap? (Not important). > Add extra test cases for assignement on the region server and fix the related > issues > > > Key: HBASE-7390 > URL: https://issues.apache.org/jira/browse/HBASE-7390 > Project: HBase > Issue Type: Bug > Components: Region Assignment, regionserver >Affects Versions: 0.96.0 >Reporter: nkeywal >Assignee: nkeywal > Fix For: 0.96.0 > > Attachments: 7390.v10.patch, 7390.v1.patch, 7390.v2.patch, > 7390.v3.patch, 7390.v4.patch, 7390.v6.patch, 7390.v7.patch, 7390.v8.patch, > 7390.v9.patch, 7390.v9.patch, assignment_zk_states.jpg > > > We don't have a lot of tests on the region server itself. > Here are some. > Some of them are failing, feedback welcome. > See as well the attached state diagram for the ZK nodes on assignment. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7390) Add extra test cases for assignement on the region server and fix the related issues
[ https://issues.apache.org/jira/browse/HBASE-7390?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=1354#comment-1354 ] Hadoop QA commented on HBASE-7390: -- {color:red}-1 overall{color}. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12562721/7390.v10.patch against trunk revision . {color:green}+1 @author{color}. The patch does not contain any @author tags. {color:green}+1 tests included{color}. The patch appears to include 11 new or modified tests. {color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop 2.0 profile. {color:green}+1 javadoc{color}. The javadoc tool did not generate any warning messages. {color:green}+1 javac{color}. The applied patch does not increase the total number of javac compiler warnings. {color:green}+1 findbugs{color}. The patch does not introduce any new Findbugs (version 1.3.9) warnings. {color:green}+1 release audit{color}. The applied patch does not increase the total number of release audit warnings. {color:red}-1 lineLengths{color}. The patch introduces lines longer than 100 {color:red}-1 core tests{color}. The patch failed these unit tests: org.apache.hadoop.hbase.client.TestFromClientSide {color:red}-1 core zombie tests{color}. There are 2 zombie test(s): Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/3766//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3766//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3766//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3766//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3766//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3766//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3766//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3766//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/3766//console This message is automatically generated. > Add extra test cases for assignement on the region server and fix the related > issues > > > Key: HBASE-7390 > URL: https://issues.apache.org/jira/browse/HBASE-7390 > Project: HBase > Issue Type: Bug > Components: Region Assignment, regionserver >Affects Versions: 0.96.0 >Reporter: nkeywal >Assignee: nkeywal > Fix For: 0.96.0 > > Attachments: 7390.v10.patch, 7390.v1.patch, 7390.v2.patch, > 7390.v3.patch, 7390.v4.patch, 7390.v6.patch, 7390.v7.patch, 7390.v8.patch, > 7390.v9.patch, 7390.v9.patch, assignment_zk_states.jpg > > > We don't have a lot of tests on the region server itself. > Here are some. > Some of them are failing, feedback welcome. > See as well the attached state diagram for the ZK nodes on assignment. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7390) Add extra test cases for assignement on the region server and fix the related issues
[ https://issues.apache.org/jira/browse/HBASE-7390?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13540576#comment-13540576 ] Hadoop QA commented on HBASE-7390: -- {color:red}-1 overall{color}. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12562600/7390.v9.patch against trunk revision . {color:green}+1 @author{color}. The patch does not contain any @author tags. {color:green}+1 tests included{color}. The patch appears to include 5 new or modified tests. {color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop 2.0 profile. {color:red}-1 javadoc{color}. The javadoc tool appears to have generated 2 warning messages. {color:green}+1 javac{color}. The applied patch does not increase the total number of javac compiler warnings. {color:green}+1 findbugs{color}. The patch does not introduce any new Findbugs (version 1.3.9) warnings. {color:green}+1 release audit{color}. The applied patch does not increase the total number of release audit warnings. {color:red}-1 lineLengths{color}. The patch introduces lines longer than 100 {color:red}-1 core tests{color}. The patch failed these unit tests: org.apache.hadoop.hbase.regionserver.handler.TestOpenRegionHandler {color:red}-1 core zombie tests{color}. There are 2 zombie test(s): Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/3738//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3738//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3738//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3738//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3738//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3738//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3738//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3738//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/3738//console This message is automatically generated. > Add extra test cases for assignement on the region server and fix the related > issues > > > Key: HBASE-7390 > URL: https://issues.apache.org/jira/browse/HBASE-7390 > Project: HBase > Issue Type: Bug > Components: Region Assignment, regionserver >Affects Versions: 0.96.0 >Reporter: nkeywal >Assignee: nkeywal > Fix For: 0.96.0 > > Attachments: 7390.v1.patch, 7390.v2.patch, 7390.v3.patch, > 7390.v4.patch, 7390.v6.patch, 7390.v7.patch, 7390.v8.patch, 7390.v9.patch, > 7390.v9.patch, assignment_zk_states.jpg > > > We don't have a lot of tests on the region server itself. > Here are some. > Some of them are failing, feedback welcome. > See as well the attached state diagram for the ZK nodes on assignment. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7390) Add extra test cases for assignement on the region server and fix the related issues
[ https://issues.apache.org/jira/browse/HBASE-7390?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13540490#comment-13540490 ] nkeywal commented on HBASE-7390: Thanks Nick & Jimmy. v9 contains the fixes for your points. I'm testing it here, and if it's ok I will commit it. > Add extra test cases for assignement on the region server and fix the related > issues > > > Key: HBASE-7390 > URL: https://issues.apache.org/jira/browse/HBASE-7390 > Project: HBase > Issue Type: Bug > Components: Region Assignment, regionserver >Affects Versions: 0.96.0 >Reporter: nkeywal >Assignee: nkeywal > Fix For: 0.96.0 > > Attachments: 7390.v1.patch, 7390.v2.patch, 7390.v3.patch, > 7390.v4.patch, 7390.v6.patch, 7390.v7.patch, 7390.v8.patch, 7390.v9.patch, > assignment_zk_states.jpg > > > We don't have a lot of tests on the region server itself. > Here are some. > Some of them are failing, feedback welcome. > See as well the attached state diagram for the ZK nodes on assignment. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7390) Add extra test cases for assignement on the region server and fix the related issues
[ https://issues.apache.org/jira/browse/HBASE-7390?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13540064#comment-13540064 ] Jimmy Xiang commented on HBASE-7390: Here is the link to the review: https://reviews.apache.org/r/8754/ I will take a look today. > Add extra test cases for assignement on the region server and fix the related > issues > > > Key: HBASE-7390 > URL: https://issues.apache.org/jira/browse/HBASE-7390 > Project: HBase > Issue Type: Bug > Components: Region Assignment, regionserver >Affects Versions: 0.96.0 >Reporter: nkeywal >Assignee: nkeywal > Fix For: 0.96.0 > > Attachments: 7390.v1.patch, 7390.v2.patch, 7390.v3.patch, > 7390.v4.patch, 7390.v6.patch, 7390.v7.patch, 7390.v8.patch, > assignment_zk_states.jpg > > > We don't have a lot of tests on the region server itself. > Here are some. > Some of them are failing, feedback welcome. > See as well the attached state diagram for the ZK nodes on assignment. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7390) Add extra test cases for assignement on the region server and fix the related issues
[ https://issues.apache.org/jira/browse/HBASE-7390?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13539867#comment-13539867 ] nkeywal commented on HBASE-7390: It's on a review board, there it a +1 from Nick there. Is there anyone else reviewing it? If not, I will commit next Monday. > Add extra test cases for assignement on the region server and fix the related > issues > > > Key: HBASE-7390 > URL: https://issues.apache.org/jira/browse/HBASE-7390 > Project: HBase > Issue Type: Bug > Components: Region Assignment, regionserver >Affects Versions: 0.96.0 >Reporter: nkeywal >Assignee: nkeywal > Fix For: 0.96.0 > > Attachments: 7390.v1.patch, 7390.v2.patch, 7390.v3.patch, > 7390.v4.patch, 7390.v6.patch, 7390.v7.patch, 7390.v8.patch, > assignment_zk_states.jpg > > > We don't have a lot of tests on the region server itself. > Here are some. > Some of them are failing, feedback welcome. > See as well the attached state diagram for the ZK nodes on assignment. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7390) Add extra test cases for assignement on the region server and fix the related issues
[ https://issues.apache.org/jira/browse/HBASE-7390?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13539285#comment-13539285 ] Hadoop QA commented on HBASE-7390: -- {color:red}-1 overall{color}. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12562325/7390.v8.patch against trunk revision . {color:green}+1 @author{color}. The patch does not contain any @author tags. {color:green}+1 tests included{color}. The patch appears to include 5 new or modified tests. {color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop 2.0 profile. {color:red}-1 javadoc{color}. The javadoc tool appears to have generated 2 warning messages. {color:green}+1 javac{color}. The applied patch does not increase the total number of javac compiler warnings. {color:green}+1 findbugs{color}. The patch does not introduce any new Findbugs (version 1.3.9) warnings. {color:green}+1 release audit{color}. The applied patch does not increase the total number of release audit warnings. {color:green}+1 core tests{color}. The patch passed unit tests in . {color:red}-1 core zombie tests{color}. There are 2 zombie test(s): Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/3684//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3684//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3684//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3684//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3684//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3684//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3684//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3684//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/3684//console This message is automatically generated. > Add extra test cases for assignement on the region server and fix the related > issues > > > Key: HBASE-7390 > URL: https://issues.apache.org/jira/browse/HBASE-7390 > Project: HBase > Issue Type: Bug > Components: Region Assignment, regionserver >Affects Versions: 0.96.0 >Reporter: nkeywal >Assignee: nkeywal > Fix For: 0.96.0 > > Attachments: 7390.v1.patch, 7390.v2.patch, 7390.v3.patch, > 7390.v4.patch, 7390.v6.patch, 7390.v7.patch, 7390.v8.patch, > assignment_zk_states.jpg > > > We don't have a lot of tests on the region server itself. > Here are some. > Some of them are failing, feedback welcome. > See as well the attached state diagram for the ZK nodes on assignment. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7390) Add extra test cases for assignement on the region server and fix the related issues
[ https://issues.apache.org/jira/browse/HBASE-7390?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13539278#comment-13539278 ] nkeywal commented on HBASE-7390: v7. More bug fixes & more comments. I'm gonna put it on the review board. > Add extra test cases for assignement on the region server and fix the related > issues > > > Key: HBASE-7390 > URL: https://issues.apache.org/jira/browse/HBASE-7390 > Project: HBase > Issue Type: Bug > Components: Region Assignment, regionserver >Affects Versions: 0.96.0 >Reporter: nkeywal >Assignee: nkeywal > Fix For: 0.96.0 > > Attachments: 7390.v1.patch, 7390.v2.patch, 7390.v3.patch, > 7390.v4.patch, 7390.v6.patch, 7390.v7.patch, 7390.v8.patch, > assignment_zk_states.jpg > > > We don't have a lot of tests on the region server itself. > Here are some. > Some of them are failing, feedback welcome. > See as well the attached state diagram for the ZK nodes on assignment. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7390) Add extra test cases for assignement on the region server and fix the related issues
[ https://issues.apache.org/jira/browse/HBASE-7390?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13538328#comment-13538328 ] Hadoop QA commented on HBASE-7390: -- {color:red}-1 overall{color}. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12562114/7390.v7.patch against trunk revision . {color:green}+1 @author{color}. The patch does not contain any @author tags. {color:green}+1 tests included{color}. The patch appears to include 2 new or modified tests. {color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop 2.0 profile. {color:green}+1 javadoc{color}. The javadoc tool did not generate any warning messages. {color:green}+1 javac{color}. The applied patch does not increase the total number of javac compiler warnings. {color:red}-1 findbugs{color}. The patch appears to introduce 21 new Findbugs (version 1.3.9) warnings. {color:green}+1 release audit{color}. The applied patch does not increase the total number of release audit warnings. {color:red}-1 core tests{color}. The patch failed these unit tests: org.apache.hadoop.hbase.TestDrainingServer Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/3656//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3656//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3656//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3656//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3656//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3656//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3656//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3656//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/3656//console This message is automatically generated. > Add extra test cases for assignement on the region server and fix the related > issues > > > Key: HBASE-7390 > URL: https://issues.apache.org/jira/browse/HBASE-7390 > Project: HBase > Issue Type: Bug > Components: Region Assignment, regionserver >Affects Versions: 0.96.0 >Reporter: nkeywal >Assignee: nkeywal > Fix For: 0.96.0 > > Attachments: 7390.v1.patch, 7390.v2.patch, 7390.v3.patch, > 7390.v4.patch, 7390.v6.patch, 7390.v7.patch, assignment_zk_states.jpg > > > We don't have a lot of tests on the region server itself. > Here are some. > Some of them are failing, feedback welcome. > See as well the attached state diagram for the ZK nodes on assignment. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7390) Add extra test cases for assignement on the region server and fix the related issues
[ https://issues.apache.org/jira/browse/HBASE-7390?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13538257#comment-13538257 ] Hadoop QA commented on HBASE-7390: -- {color:red}-1 overall{color}. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12562109/7390.v6.patch against trunk revision . {color:green}+1 @author{color}. The patch does not contain any @author tags. {color:green}+1 tests included{color}. The patch appears to include 2 new or modified tests. {color:red}-1 patch{color}. The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/3655//console This message is automatically generated. > Add extra test cases for assignement on the region server and fix the related > issues > > > Key: HBASE-7390 > URL: https://issues.apache.org/jira/browse/HBASE-7390 > Project: HBase > Issue Type: Bug > Components: Region Assignment, regionserver >Affects Versions: 0.96.0 >Reporter: nkeywal >Assignee: nkeywal > Fix For: 0.96.0 > > Attachments: 7390.v1.patch, 7390.v2.patch, 7390.v3.patch, > 7390.v4.patch, 7390.v6.patch, assignment_zk_states.jpg > > > We don't have a lot of tests on the region server itself. > Here are some. > Some of them are failing, feedback welcome. > See as well the attached state diagram for the ZK nodes on assignment. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7390) Add extra test cases for assignement on the region server and fix the related issues
[ https://issues.apache.org/jira/browse/HBASE-7390?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13538253#comment-13538253 ] nkeywal commented on HBASE-7390: v6. Likely not final, but enough to be tested and global feedback. > Add extra test cases for assignement on the region server and fix the related > issues > > > Key: HBASE-7390 > URL: https://issues.apache.org/jira/browse/HBASE-7390 > Project: HBase > Issue Type: Bug > Components: Region Assignment, regionserver >Affects Versions: 0.96.0 >Reporter: nkeywal >Assignee: nkeywal > Fix For: 0.96.0 > > Attachments: 7390.v1.patch, 7390.v2.patch, 7390.v3.patch, > 7390.v4.patch, 7390.v6.patch, assignment_zk_states.jpg > > > We don't have a lot of tests on the region server itself. > Here are some. > Some of them are failing, feedback welcome. > See as well the attached state diagram for the ZK nodes on assignment. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7390) Add extra test cases for assignement on the region server and fix the related issues
[ https://issues.apache.org/jira/browse/HBASE-7390?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13538231#comment-13538231 ] nkeywal commented on HBASE-7390: @[~saint@gmail.com] It's quite complicated :-(. I think I'm seeing the end, or at least a checkpoint. It's the region server RIT: 3 lines above, we already checked the content, so this line is likely useless. > Add extra test cases for assignement on the region server and fix the related > issues > > > Key: HBASE-7390 > URL: https://issues.apache.org/jira/browse/HBASE-7390 > Project: HBase > Issue Type: Bug > Components: Region Assignment, regionserver >Affects Versions: 0.96.0 >Reporter: nkeywal >Assignee: nkeywal > Fix For: 0.96.0 > > Attachments: 7390.v1.patch, 7390.v2.patch, 7390.v3.patch, > 7390.v4.patch, assignment_zk_states.jpg > > > We don't have a lot of tests on the region server itself. > Here are some. > Some of them are failing, feedback welcome. > See as well the attached state diagram for the ZK nodes on assignment. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7390) Add extra test cases for assignement on the region server and fix the related issues
[ https://issues.apache.org/jira/browse/HBASE-7390?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13538227#comment-13538227 ] stack commented on HBASE-7390: -- Nice analysis @nkeywal. I am trying to follow but cannot (I think that is your point -- smile). When you say '// If we're there, it means there is nothing in RIT ' -- is the RIT in zk? Under 'unassigned' dir? Can you see who is responsible for the logic w/ git/svn blame? Maybe their insight would help w/ the unravelling? I like your list of problems w/ current implementation. > Add extra test cases for assignement on the region server and fix the related > issues > > > Key: HBASE-7390 > URL: https://issues.apache.org/jira/browse/HBASE-7390 > Project: HBase > Issue Type: Bug > Components: Region Assignment, regionserver >Affects Versions: 0.96.0 >Reporter: nkeywal >Assignee: nkeywal > Fix For: 0.96.0 > > Attachments: 7390.v1.patch, 7390.v2.patch, 7390.v3.patch, > 7390.v4.patch, assignment_zk_states.jpg > > > We don't have a lot of tests on the region server itself. > Here are some. > Some of them are failing, feedback welcome. > See as well the attached state diagram for the ZK nodes on assignment. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7390) Add extra test cases for assignement on the region server and fix the related issues
[ https://issues.apache.org/jira/browse/HBASE-7390?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13537874#comment-13537874 ] nkeywal commented on HBASE-7390: bq. I'm not sure I follow. {code} Boolean openAction = regionsInTransitionInRS.get(encodedName); if (openAction != null) { // If this region is already in transition if (openAction.booleanValue()) { // and is opening regionsInTransitionInRS.replace(encodedName, openAction, Boolean.FALSE); // say it's now closing } checkIfRegionInTransition(encodedName, CLOSE); // throw an exception // we will never go there, we always throw an exception at the previous line // So if we were opening, we now have a OpenRegionHandler running but the RIT state says closing. // What's going to happen to the znode now? } // If we're there, it means there is nothing in RIT requestCount.increment(); LOG.info("Received close region: " + region.getRegionNameAsString() + ". Version of ZK closing node:" + versionOfClosingNode + ". Destination server:" + sn); HRegionInfo regionInfo = region.getRegionInfo(); checkIfRegionInTransition(encodedName, CLOSE); // We've just checked 0.0001 ms ago! {code} I looked at it more, it's actually by design: the OpenRegionHandler checks for the content of this variable. So the behavior is: 'if we received the request to close a region we're currently opening, stop the opening but raise an exception saying we can't close again because we were already closing". Issues are: - What's going to happen with the znode? It seems it won't be changed in the openHandler for example. So it will remain forever. - It's a little bit racy: if we were at the end of the opening, we can get the exception but the opening will be done. - worse, the closeRegion call will be rejected at its very beginning, because when we try to do the "getRegionByEncodedName", it throws a "RegionIsNotOnline" => In many cases, the opening will continue, as if the region is opening it should not be in the onlineRegions list. => It could be an impact of the coprocessor implementation. => As a side node, as the closeRegion closes are nested, the coprocessors "preClose" will be called multiple times. => As another side node, the way it's coded in the other closeRegion is that we don't call the coprocessors when the region is not online => So it means that we don"t check the access rigts if the region is not online, it allows to cancel an opening without the AR. - As well, we had no test case (that's why my patch, while broken, worked). I'm trying to solve this in the last version. bq. + zkw.sync(encoded); It's not absolutely necessary, but it maximizes corectness probability. It's done in all other cases as well, so not doing it will would be bad imho, as it would break consistency. bq. Review comments. Some error. e.g. + * @param expectedVersion expecpted version og the znode It looks like klingon. Done. bq. This below continue is right? I've changed the implementation to remove the continue. I found another issue while doing so. Previously, when the region was already in RIT and closing, we were saying "OPENED" on a bulkAssign (that is: the query was open, and we were saying 'opened' while closing the region). This seems wrong. We now say "FAILED_OPENING" in this case. bq. On the patch, what Sergey said about different closeRegion returns. Could the one that throws an exception be private? I looked at it in more details. Actually, what returns this function is never checked. So I've renamed it and clean stuff around it. bq. Would putIfAbsent be easier to read if it took an enum rather than a boolean to say whether open or close It's was to be compatible with RIT design. I removed the method. I tried to change RIT, but its type is public through the getter, so I prefer not doing it in a patch. I will publish an updated version soon. > Add extra test cases for assignement on the region server and fix the related > issues > > > Key: HBASE-7390 > URL: https://issues.apache.org/jira/browse/HBASE-7390 > Project: HBase > Issue Type: Bug > Components: Region Assignment, regionserver >Affects Versions: 0.96.0 >Reporter: nkeywal >Assignee: nkeywal > Fix For: 0.96.0 > > Attachments: 7390.v1.patch, 7390.v2.patch, 7390.v3.patch, > 7390.v4.patch, assignment_zk_states.jpg > > > We don't have a lot of tests on the region server itself. > Here are some. > Some of them are failing, feedback welcome. > See as well the attached state diagram for the ZK nodes on assignment. -- This message is automatically g
[jira] [Commented] (HBASE-7390) Add extra test cases for assignement on the region server and fix the related issues
[ https://issues.apache.org/jira/browse/HBASE-7390?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13537311#comment-13537311 ] stack commented on HBASE-7390: -- On the patch, what Sergey said about different closeRegion returns. Could the one that throws an exception be private? Would putIfAbsent be easier to read if it took an enum rather than a boolean to say whether open or close? For example, what was there previous was opaque enough and I appreciate your collecting together tests into a single method but the below could be better: -if (this.regionsInTransitionInRS.containsKey(region.getEncodedNameAsBytes())) { - LOG.warn("Received close for region we are already opening or closing; " + -region.getEncodedName()); +if (putIfAbsent(region, false)){ + // We're already closing this region. This below continue is right? + builder.addOpeningState(RegionOpeningState.OPENED); + continue; I see you do it again later if we continue... so above looks like it makes sense. Review comments. Some error. e.g. + * @param expectedVersion expecpted version og the znode It looks like klingon. I appreciate these changes: - String regionName) + String encodedRegionName) I always have to back up to find what what is needed making these calls. I suppose we should make an encodedRegionName type one of these days. Do we need to do this? +zkw.sync(encoded); Its to be 'sure'... Its expensive though... Nice test. Can you say more on this @nkeywal "This code there is very very smart: if there is a open in progress, it changes the internal state to close, then raises an exception through the call to checkIfRegionInTransition. As we changed the state, we will have, if we were currently opening, a message saying that we were trying to close a region already closing." I'm not sure I follow. The patch looks great, cleaning up fuzzy state. Let me get Jimmy to take a looksee too. He LOVEs this stuff. [~jxiang] Any chance of your taking a look here boss? > Add extra test cases for assignement on the region server and fix the related > issues > > > Key: HBASE-7390 > URL: https://issues.apache.org/jira/browse/HBASE-7390 > Project: HBase > Issue Type: Bug > Components: Region Assignment, regionserver >Affects Versions: 0.96.0 >Reporter: nkeywal >Assignee: nkeywal > Fix For: 0.96.0 > > Attachments: 7390.v1.patch, 7390.v2.patch, 7390.v3.patch, > 7390.v4.patch, assignment_zk_states.jpg > > > We don't have a lot of tests on the region server itself. > Here are some. > Some of them are failing, feedback welcome. > See as well the attached state diagram for the ZK nodes on assignment. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7390) Add extra test cases for assignement on the region server and fix the related issues
[ https://issues.apache.org/jira/browse/HBASE-7390?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13537280#comment-13537280 ] stack commented on HBASE-7390: -- bq. What's strange to me is that it seems similar to a RS_ZK_REGION_CLOSED, and in this case we don't retickle at all: we start the close handler immediately. Agree we should do one or the other. The tickle does not seem necessary if when new master joins cluster and it sees a SPLIT, it does the cleanup immediately -- queues the split handler as is don3 for CLOSED (?). > Add extra test cases for assignement on the region server and fix the related > issues > > > Key: HBASE-7390 > URL: https://issues.apache.org/jira/browse/HBASE-7390 > Project: HBase > Issue Type: Bug > Components: Region Assignment, regionserver >Affects Versions: 0.96.0 >Reporter: nkeywal >Assignee: nkeywal > Fix For: 0.96.0 > > Attachments: 7390.v1.patch, 7390.v2.patch, 7390.v3.patch, > 7390.v4.patch, assignment_zk_states.jpg > > > We don't have a lot of tests on the region server itself. > Here are some. > Some of them are failing, feedback welcome. > See as well the attached state diagram for the ZK nodes on assignment. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7390) Add extra test cases for assignement on the region server and fix the related issues
[ https://issues.apache.org/jira/browse/HBASE-7390?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13537232#comment-13537232 ] Sergey Shelukhin commented on HBASE-7390: - +1 on v4 patch > Add extra test cases for assignement on the region server and fix the related > issues > > > Key: HBASE-7390 > URL: https://issues.apache.org/jira/browse/HBASE-7390 > Project: HBase > Issue Type: Bug > Components: Region Assignment, regionserver >Affects Versions: 0.96.0 >Reporter: nkeywal >Assignee: nkeywal > Fix For: 0.96.0 > > Attachments: 7390.v1.patch, 7390.v2.patch, 7390.v3.patch, > 7390.v4.patch, assignment_zk_states.jpg > > > We don't have a lot of tests on the region server itself. > Here are some. > Some of them are failing, feedback welcome. > See as well the attached state diagram for the ZK nodes on assignment. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7390) Add extra test cases for assignement on the region server and fix the related issues
[ https://issues.apache.org/jira/browse/HBASE-7390?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13537185#comment-13537185 ] nkeywal commented on HBASE-7390: [~saint@gmail.com] bq. Yes. In case the master went away on us, so the new master gets the SPLIT callback...and its handler does the SPLIT clean-up. What's strange to me is that it seems similar to a RS_ZK_REGION_CLOSED, and in this case we don't retickle at all: we start the close handler immediately. For this patch, I am waiting for a +1 before committing, as it touches a critical part. > Add extra test cases for assignement on the region server and fix the related > issues > > > Key: HBASE-7390 > URL: https://issues.apache.org/jira/browse/HBASE-7390 > Project: HBase > Issue Type: Bug > Components: Region Assignment, regionserver >Affects Versions: 0.96.0 >Reporter: nkeywal >Assignee: nkeywal > Fix For: 0.96.0 > > Attachments: 7390.v1.patch, 7390.v2.patch, 7390.v3.patch, > 7390.v4.patch, assignment_zk_states.jpg > > > We don't have a lot of tests on the region server itself. > Here are some. > Some of them are failing, feedback welcome. > See as well the attached state diagram for the ZK nodes on assignment. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7390) Add extra test cases for assignement on the region server and fix the related issues
[ https://issues.apache.org/jira/browse/HBASE-7390?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13536634#comment-13536634 ] Hadoop QA commented on HBASE-7390: -- {color:red}-1 overall{color}. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12561800/7390.v4.patch against trunk revision . {color:green}+1 @author{color}. The patch does not contain any @author tags. {color:green}+1 tests included{color}. The patch appears to include 2 new or modified tests. {color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop 2.0 profile. {color:green}+1 javadoc{color}. The javadoc tool did not generate any warning messages. {color:green}+1 javac{color}. The applied patch does not increase the total number of javac compiler warnings. {color:red}-1 findbugs{color}. The patch appears to introduce 28 new Findbugs (version 1.3.9) warnings. {color:green}+1 release audit{color}. The applied patch does not increase the total number of release audit warnings. {color:red}-1 core tests{color}. The patch failed these unit tests: {color:red}-1 core zombie tests{color}. There are zombie tests. See build logs for details. Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/3621//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3621//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3621//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3621//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3621//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3621//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3621//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3621//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/3621//console This message is automatically generated. > Add extra test cases for assignement on the region server and fix the related > issues > > > Key: HBASE-7390 > URL: https://issues.apache.org/jira/browse/HBASE-7390 > Project: HBase > Issue Type: Bug > Components: Region Assignment, regionserver >Affects Versions: 0.96.0 >Reporter: nkeywal >Assignee: nkeywal > Fix For: 0.96.0 > > Attachments: 7390.v1.patch, 7390.v2.patch, 7390.v3.patch, > 7390.v4.patch, assignment_zk_states.jpg > > > We don't have a lot of tests on the region server itself. > Here are some. > Some of them are failing, feedback welcome. > See as well the attached state diagram for the ZK nodes on assignment. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7390) Add extra test cases for assignement on the region server and fix the related issues
[ https://issues.apache.org/jira/browse/HBASE-7390?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13536570#comment-13536570 ] stack commented on HBASE-7390: -- bq. But on master failover, when we iterate on the znodes, we ignore the one int the SPLIT state, that may be the reason. Yes. In case the master went away on us, so the new master gets the SPLIT callback...and its handler does the SPLIT cleanup. Comment doesn't enough: {code} * Transitions an existing node for the specified region which is * currently in the SPLITTING state to be in the SPLIT state. Converts the * ephemeral SPLITTING znode to an ephemeral SPLIT node. Master cleans up * SPLIT znode when it reads it (or if we crash, zk will clean it up). {code} > Add extra test cases for assignement on the region server and fix the related > issues > > > Key: HBASE-7390 > URL: https://issues.apache.org/jira/browse/HBASE-7390 > Project: HBase > Issue Type: Bug > Components: Region Assignment, regionserver >Affects Versions: 0.96.0 >Reporter: nkeywal >Assignee: nkeywal > Fix For: 0.96.0 > > Attachments: 7390.v1.patch, 7390.v2.patch, 7390.v3.patch, > 7390.v4.patch, assignment_zk_states.jpg > > > We don't have a lot of tests on the region server itself. > Here are some. > Some of them are failing, feedback welcome. > See as well the attached state diagram for the ZK nodes on assignment. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7390) Add extra test cases for assignement on the region server and fix the related issues
[ https://issues.apache.org/jira/browse/HBASE-7390?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13536508#comment-13536508 ] nkeywal commented on HBASE-7390: for the tests, we have the usual suspects again it seems. [~sershe] bq. One closeRegion throws, and the other returns false, to indicate the same condition. Is this intended? Yeah... Questionable but acceptable imho. Don't mind changing this, but then I will need to throw an exception from the return somewhere else, and to change the return type from boolean to a 3 states types (no state, same state, different state). bq. What is the RuntimeException here? Over (and bad) design I would say. Changed. bq "of", "expected" Done. > Add extra test cases for assignement on the region server and fix the related > issues > > > Key: HBASE-7390 > URL: https://issues.apache.org/jira/browse/HBASE-7390 > Project: HBase > Issue Type: Bug > Components: Region Assignment, regionserver >Affects Versions: 0.96.0 >Reporter: nkeywal >Assignee: nkeywal > Fix For: 0.96.0 > > Attachments: 7390.v1.patch, 7390.v2.patch, 7390.v3.patch, > 7390.v4.patch, assignment_zk_states.jpg > > > We don't have a lot of tests on the region server itself. > Here are some. > Some of them are failing, feedback welcome. > See as well the attached state diagram for the ZK nodes on assignment. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7390) Add extra test cases for assignement on the region server and fix the related issues
[ https://issues.apache.org/jira/browse/HBASE-7390?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13536490#comment-13536490 ] Hadoop QA commented on HBASE-7390: -- {color:red}-1 overall{color}. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12561787/7390.v3.patch against trunk revision . {color:green}+1 @author{color}. The patch does not contain any @author tags. {color:green}+1 tests included{color}. The patch appears to include 2 new or modified tests. {color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop 2.0 profile. {color:green}+1 javadoc{color}. The javadoc tool did not generate any warning messages. {color:green}+1 javac{color}. The applied patch does not increase the total number of javac compiler warnings. {color:red}-1 findbugs{color}. The patch appears to introduce 28 new Findbugs (version 1.3.9) warnings. {color:green}+1 release audit{color}. The applied patch does not increase the total number of release audit warnings. {color:red}-1 core tests{color}. The patch failed these unit tests: org.apache.hadoop.hbase.client.TestMultiParallel org.apache.hadoop.hbase.replication.TestReplication org.apache.hadoop.hbase.TestDrainingServer {color:red}-1 core zombie tests{color}. There are zombie tests. See build logs for details. Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/3620//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3620//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3620//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3620//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3620//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3620//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3620//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3620//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/3620//console This message is automatically generated. > Add extra test cases for assignement on the region server and fix the related > issues > > > Key: HBASE-7390 > URL: https://issues.apache.org/jira/browse/HBASE-7390 > Project: HBase > Issue Type: Bug > Components: Region Assignment, regionserver >Affects Versions: 0.96.0 >Reporter: nkeywal >Assignee: nkeywal > Fix For: 0.96.0 > > Attachments: 7390.v1.patch, 7390.v2.patch, 7390.v3.patch, > assignment_zk_states.jpg > > > We don't have a lot of tests on the region server itself. > Here are some. > Some of them are failing, feedback welcome. > See as well the attached state diagram for the ZK nodes on assignment. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7390) Add extra test cases for assignement on the region server and fix the related issues
[ https://issues.apache.org/jira/browse/HBASE-7390?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13536474#comment-13536474 ] nkeywal commented on HBASE-7390: Yes, that's what we do. But we should never miss the final event, so it's a little bit strange. But on master failover, when we iterate on the znodes, we ignore the one int the SPLIT state, that may be the reason. I need to look at this as well, I tend to think we may have issues around this part (master crash during a split). > Add extra test cases for assignement on the region server and fix the related > issues > > > Key: HBASE-7390 > URL: https://issues.apache.org/jira/browse/HBASE-7390 > Project: HBase > Issue Type: Bug > Components: Region Assignment, regionserver >Affects Versions: 0.96.0 >Reporter: nkeywal >Assignee: nkeywal > Fix For: 0.96.0 > > Attachments: 7390.v1.patch, 7390.v2.patch, 7390.v3.patch, > assignment_zk_states.jpg > > > We don't have a lot of tests on the region server itself. > Here are some. > Some of them are failing, feedback welcome. > See as well the attached state diagram for the ZK nodes on assignment. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7390) Add extra test cases for assignement on the region server and fix the related issues
[ https://issues.apache.org/jira/browse/HBASE-7390?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13536469#comment-13536469 ] stack commented on HBASE-7390: -- [~nkeywal] I follow you now. Yes, we can miss events and yes the master deals (and yes no harm in improving doc). IIRC (I believe I added SPLIT), we tickle in case the transition to SPLIT from SPLITTING is missed by the master. We keep tickling till master acknowledges the SPLIT message (by removing SPLIT znode -- haven't looked at code). > Add extra test cases for assignement on the region server and fix the related > issues > > > Key: HBASE-7390 > URL: https://issues.apache.org/jira/browse/HBASE-7390 > Project: HBase > Issue Type: Bug > Components: Region Assignment, regionserver >Affects Versions: 0.96.0 >Reporter: nkeywal >Assignee: nkeywal > Fix For: 0.96.0 > > Attachments: 7390.v1.patch, 7390.v2.patch, 7390.v3.patch, > assignment_zk_states.jpg > > > We don't have a lot of tests on the region server itself. > Here are some. > Some of them are failing, feedback welcome. > See as well the attached state diagram for the ZK nodes on assignment. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7390) Add extra test cases for assignement on the region server and fix the related issues
[ https://issues.apache.org/jira/browse/HBASE-7390?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13536454#comment-13536454 ] nkeywal commented on HBASE-7390: @[~saint@gmail.com] For SPLITTING and OPENING, I misread your point in you first comment. Yes, we may miss the state change because of ZooKeeper API: when we receive a watch, we need to do a getData. In the meantime, the value may have changed. The scenario is, for example - Master creates a znode with M_ZK_REGION_OFFLINE and watch it - Regionserver updates the znode to RS_ZK_REGION_OPENING - Master receives the event - Regionserver updates the znode to RS_ZK_REGION_OPENED - Master does a getData, and reads RS_ZK_REGION_OPENED In this scenario, the master missed the RS_ZK_REGION_OPENING. So in the master code you must be ready to miss some transitions. The trunk code is actually compatible with this, but the reason is not documented. However, I don't know why we tickle RS_REGION_SPLIT as we do in the code. > Add extra test cases for assignement on the region server and fix the related > issues > > > Key: HBASE-7390 > URL: https://issues.apache.org/jira/browse/HBASE-7390 > Project: HBase > Issue Type: Bug > Components: Region Assignment, regionserver >Affects Versions: 0.96.0 >Reporter: nkeywal >Assignee: nkeywal > Fix For: 0.96.0 > > Attachments: 7390.v1.patch, 7390.v2.patch, 7390.v3.patch, > assignment_zk_states.jpg > > > We don't have a lot of tests on the region server itself. > Here are some. > Some of them are failing, feedback welcome. > See as well the attached state diagram for the ZK nodes on assignment. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7390) Add extra test cases for assignement on the region server and fix the related issues
[ https://issues.apache.org/jira/browse/HBASE-7390?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13536452#comment-13536452 ] Sergey Shelukhin commented on HBASE-7390: - {code} protected boolean closeRegion(HRegionInfo region, final boolean abort, final boolean zk) { try { return closeRegion(region, abort, zk, -1, null); } catch (RegionAlreadyInTransitionException e) { return false; // The region was already in transition for opening } {code} One closeRegion throws, and the other returns false, to indicate the same condition. Is this intended? {code} }catch (RuntimeException t){ {code} What is the RuntimeException here? {code} * @param expectedVersion expecpted version og the znode {code} "of", "expected" > Add extra test cases for assignement on the region server and fix the related > issues > > > Key: HBASE-7390 > URL: https://issues.apache.org/jira/browse/HBASE-7390 > Project: HBase > Issue Type: Bug > Components: Region Assignment, regionserver >Affects Versions: 0.96.0 >Reporter: nkeywal >Assignee: nkeywal > Fix For: 0.96.0 > > Attachments: 7390.v1.patch, 7390.v2.patch, 7390.v3.patch, > assignment_zk_states.jpg > > > We don't have a lot of tests on the region server itself. > Here are some. > Some of them are failing, feedback welcome. > See as well the attached state diagram for the ZK nodes on assignment. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7390) Add extra test cases for assignement on the region server and fix the related issues
[ https://issues.apache.org/jira/browse/HBASE-7390?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13536439#comment-13536439 ] nkeywal commented on HBASE-7390: v3 with everything included. Ready for review :-) > Add extra test cases for assignement on the region server and fix the related > issues > > > Key: HBASE-7390 > URL: https://issues.apache.org/jira/browse/HBASE-7390 > Project: HBase > Issue Type: Bug > Components: Region Assignment, regionserver >Affects Versions: 0.96.0 >Reporter: nkeywal >Assignee: nkeywal > Fix For: 0.96.0 > > Attachments: 7390.v1.patch, 7390.v2.patch, 7390.v3.patch, > assignment_zk_states.jpg > > > We don't have a lot of tests on the region server itself. > Here are some. > Some of them are failing, feedback welcome. > See as well the attached state diagram for the ZK nodes on assignment. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7390) Add extra test cases for assignement on the region server and fix the related issues
[ https://issues.apache.org/jira/browse/HBASE-7390?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13536403#comment-13536403 ] nkeywal commented on HBASE-7390: Seems to be ok ok imho. For testCloseByMasterWithoutZNode, it fails because we update the znode state after doing the work. So when we discover there is no znode, the close is done. It easy to fix by adding a test at the beginning of the close. As a consequence: - broken situation where we were calling a close without znode will now lead to an error - we will be a little bit slower, as we will have another zk sync & read on the critical path If we're in line with what was done by Jimmy Xiang previously (i.e. checking carefully what we're doing :-) ), we should do it. Toughts? > Add extra test cases for assignement on the region server and fix the related > issues > > > Key: HBASE-7390 > URL: https://issues.apache.org/jira/browse/HBASE-7390 > Project: HBase > Issue Type: Bug > Components: Region Assignment, regionserver >Affects Versions: 0.96.0 >Reporter: nkeywal >Assignee: nkeywal > Fix For: 0.96.0 > > Attachments: 7390.v1.patch, 7390.v2.patch, assignment_zk_states.jpg > > > We don't have a lot of tests on the region server itself. > Here are some. > Some of them are failing, feedback welcome. > See as well the attached state diagram for the ZK nodes on assignment. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7390) Add extra test cases for assignement on the region server and fix the related issues
[ https://issues.apache.org/jira/browse/HBASE-7390?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13536390#comment-13536390 ] Hadoop QA commented on HBASE-7390: -- {color:red}-1 overall{color}. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12561764/7390.v2.patch against trunk revision . {color:green}+1 @author{color}. The patch does not contain any @author tags. {color:green}+1 tests included{color}. The patch appears to include 2 new or modified tests. {color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop 2.0 profile. {color:green}+1 javadoc{color}. The javadoc tool did not generate any warning messages. {color:green}+1 javac{color}. The applied patch does not increase the total number of javac compiler warnings. {color:red}-1 findbugs{color}. The patch appears to introduce 28 new Findbugs (version 1.3.9) warnings. {color:green}+1 release audit{color}. The applied patch does not increase the total number of release audit warnings. {color:red}-1 core tests{color}. The patch failed these unit tests: org.apache.hadoop.hbase.client.TestFromClientSideWithCoprocessor {color:red}-1 core zombie tests{color}. There are zombie tests. See build logs for details. Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/3617//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3617//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3617//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3617//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3617//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3617//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3617//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3617//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/3617//console This message is automatically generated. > Add extra test cases for assignement on the region server and fix the related > issues > > > Key: HBASE-7390 > URL: https://issues.apache.org/jira/browse/HBASE-7390 > Project: HBase > Issue Type: Bug > Components: Region Assignment, regionserver >Affects Versions: 0.96.0 >Reporter: nkeywal >Assignee: nkeywal > Fix For: 0.96.0 > > Attachments: 7390.v1.patch, 7390.v2.patch, assignment_zk_states.jpg > > > We don't have a lot of tests on the region server itself. > Here are some. > Some of them are failing, feedback welcome. > See as well the attached state diagram for the ZK nodes on assignment. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira