[jira] [Commented] (HBASE-10662) RegionScanner should be closed and according lease should be cancelled in regionserver immediately if we find the related region has been re-opened during performing s
[ https://issues.apache.org/jira/browse/HBASE-10662?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13920501#comment-13920501 ] ramkrishna.s.vasudevan commented on HBASE-10662: lgtm. > RegionScanner should be closed and according lease should be cancelled in > regionserver immediately if we find the related region has been re-opened > during performing scan request > -- > > Key: HBASE-10662 > URL: https://issues.apache.org/jira/browse/HBASE-10662 > Project: HBase > Issue Type: Bug > Components: regionserver >Reporter: Feng Honghua >Assignee: Feng Honghua > Attachments: HBASE-10662-trunk_v1.patch > > > During regionserver processes scan request from client, it fails the request > by throwing a wrapped NotServingRegionException to client if it finds the > region related to the passed-in scanner-id has been re-opened, and it also > removes the RegionScannerHolder from the scanners. In fact under this case, > the old and invalid RegionScanner related to the passed-in scanner-id should > be closed and the related lease should be cancelled at the mean time as well. > Currently region's related scanners aren't closed when closing the region, a > region scanner is closed only when requested explicitly by client, or by > expiration of the related lease, in this sense the close of region scanners > is quite passive and lag. > When regionserver processes scan request from client and can't find online > region corresponding to the passed-in scanner-id (due to being moved out) or > find the region has been re-opened, it throws NotServingRegionException and > removes the corresponding RegionScannerHolder from scanners without closing > the related region scanner (nor cancelling the related lease), but when the > lease expires, the related region scanner still doesn't be closed since it > doesn't present in scanners now. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-10662) RegionScanner should be closed and according lease should be cancelled in regionserver immediately if we find the related region has been re-opened during performing s
[ https://issues.apache.org/jira/browse/HBASE-10662?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13920373#comment-13920373 ] Hadoop QA commented on HBASE-10662: --- {color:red}-1 overall{color}. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12632478/HBASE-10662-trunk_v1.patch against trunk revision . ATTACHMENT ID: 12632478 {color:green}+1 @author{color}. The patch does not contain any @author tags. {color:red}-1 tests included{color}. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. {color:green}+1 hadoop1.0{color}. The patch compiles against the hadoop 1.0 profile. {color:green}+1 hadoop1.1{color}. The patch compiles against the hadoop 1.1 profile. {color:green}+1 javadoc{color}. The javadoc tool did not generate any warning messages. {color:green}+1 javac{color}. The applied patch does not increase the total number of javac compiler warnings. {color:green}+1 findbugs{color}. The patch does not introduce any new Findbugs (version 1.3.9) warnings. {color:green}+1 release audit{color}. The applied patch does not increase the total number of release audit warnings. {color:green}+1 lineLengths{color}. The patch does not introduce lines longer than 100 {color:green}+1 site{color}. The mvn site goal succeeds with this patch. {color:green}+1 core tests{color}. The patch passed unit tests in . Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/8887//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/8887//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/8887//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/8887//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/8887//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/8887//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/8887//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/8887//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/8887//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/8887//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/8887//console This message is automatically generated. > RegionScanner should be closed and according lease should be cancelled in > regionserver immediately if we find the related region has been re-opened > during performing scan request > -- > > Key: HBASE-10662 > URL: https://issues.apache.org/jira/browse/HBASE-10662 > Project: HBase > Issue Type: Bug > Components: regionserver >Reporter: Feng Honghua >Assignee: Feng Honghua > Attachments: HBASE-10662-trunk_v1.patch > > > During regionserver processes scan request from client, it fails the request > by throwing a wrapped NotServingRegionException to client if it finds the > region related to the passed-in scanner-id has been re-opened, and it also > removes the RegionScannerHolder from the scanners. In fact under this case, > the old and invalid RegionScanner related to the passed-in scanner-id should > be closed and the related lease should be cancelled at the mean time as well. > Currently region's related scanners aren't closed when closing the region, a > region scanner is closed only when requested explicitly by client, or by > expiration of the related lease, in this sense the close of region scanners > is quite passive and lag. > When regionserver processes scan request from client and can't find online > region corresponding to the passed-in scanner-id (due to being moved out) or > find the region has been re-opened, it throws NotServingRegionException and > removes the corresponding Region
[jira] [Commented] (HBASE-10662) RegionScanner should be closed and according lease should be cancelled in regionserver immediately if we find the related region has been re-opened during performing s
[ https://issues.apache.org/jira/browse/HBASE-10662?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13919331#comment-13919331 ] Feng Honghua commented on HBASE-10662: -- This bug occurs not only when regionserver processes scan request after region re-open, but also when regionserver processes scan request after the region is moved out(due to balance or user's move request) of the regionserver : NotServingRegionException is thrown and the RegionServerHolder is removed from scanners in regionserver, but when leaseExpired is executed due to lease expires, the related region scanner can't be closed due to the according RegionScannerHolder has already been removed from scanners without closing the related regionscanner... > RegionScanner should be closed and according lease should be cancelled in > regionserver immediately if we find the related region has been re-opened > during performing scan request > -- > > Key: HBASE-10662 > URL: https://issues.apache.org/jira/browse/HBASE-10662 > Project: HBase > Issue Type: Bug > Components: regionserver >Reporter: Feng Honghua >Assignee: Feng Honghua > Attachments: HBASE-10662-trunk_v1.patch > > > During regionserver processes scan request from client, it fails the request > by throwing a wrapped NotServingRegionException to client if it finds the > region related to the passed-in scanner-id has been re-opened, and it also > removes the RegionScannerHolder from the scanners. In fact under this case, > the old and invalid RegionScanner related to the passed-in scanner-id should > be closed and the related lease should be cancelled at the mean time as well. > Currently region's related scanners aren't closed when closing the region, a > region scanner is closed only when requested explicitly by client, or by > expiration of the related lease, in this sense the close of region scanners > is quite passive and lag. > Sounds reasonable to cleanup all related scanners and cancel these scanners' > leases after closing a region? -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-10662) RegionScanner should be closed and according lease should be cancelled in regionserver immediately if we find the related region has been re-opened during performing s
[ https://issues.apache.org/jira/browse/HBASE-10662?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13919208#comment-13919208 ] Feng Honghua commented on HBASE-10662: -- When we find the region has been re-opened during serving scan request from client in regionserver, if we only remove RegionScannerHolder from scanners but don't close the related scanner. The related lease will be cancelled when it expires, but the related region scanner won't be closed in leaseExpired as expected: {code} public void leaseExpired() { RegionScannerHolder rsh = scanners.remove(this.scannerName); if (rsh != null) { RegionScanner s = rsh.s; LOG.info("Scanner " + this.scannerName + " lease expired on region " + s.getRegionInfo().getRegionNameAsString()); try { HRegion region = getRegion(s.getRegionInfo().getRegionName()); if (region != null && region.getCoprocessorHost() != null) { region.getCoprocessorHost().preScannerClose(s); } s.close(); if (region != null && region.getCoprocessorHost() != null) { region.getCoprocessorHost().postScannerClose(s); } } catch (IOException e) { LOG.error("Closing scanner for " + s.getRegionInfo().getRegionNameAsString(), e); } } else { LOG.info("Scanner " + this.scannerName + " lease expired"); } } {code} In above code, scanners.remove(this.scannerName) will return a null rsh since it has been removed earlier, so the region scanner can't be closed here, which means the related region scanner doesn't have a chance to be closed ever. > RegionScanner should be closed and according lease should be cancelled in > regionserver immediately if we find the related region has been re-opened > during performing scan request > -- > > Key: HBASE-10662 > URL: https://issues.apache.org/jira/browse/HBASE-10662 > Project: HBase > Issue Type: Bug > Components: regionserver >Reporter: Feng Honghua >Assignee: Feng Honghua > Attachments: HBASE-10662-trunk_v1.patch > > > During regionserver processes scan request from client, it fails the request > by throwing a wrapped NotServingRegionException to client if it finds the > region related to the passed-in scanner-id has been re-opened, and it also > removes the RegionScannerHolder from the scanners. In fact under this case, > the old and invalid RegionScanner related to the passed-in scanner-id should > be closed and the related lease should be cancelled at the mean time as well. > Currently region's related scanners aren't closed when closing the region, a > region scanner is closed only when requested explicitly by client, or by > expiration of the related lease, in this sense the close of region scanners > is quite passive and lag. > Sounds reasonable to cleanup all related scanners and cancel these scanners' > leases after closing a region? -- This message was sent by Atlassian JIRA (v6.2#6252)