[jira] [Commented] (HBASE-6236) Offline meta repair fails if the HBase base mount point is on a different cluster/volume than its parent in a ViewFS or similar FS
[ https://issues.apache.org/jira/browse/HBASE-6236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13400035#comment-13400035 ] Hudson commented on HBASE-6236: --- Integrated in HBase-0.94-security #37 (See [https://builds.apache.org/job/HBase-0.94-security/37/]) HBASE-6236 Addendum adds -sidelineDir option to hbck (Aditya) (Revision 1353066) HBASE-6236 Offline meta repair fails if the HBase base mount point is on a different cluster/volume than its parent in a ViewFS or similar FS (Aditya) (Revision 1353064) HBASE-6236 revert upon Jimmy's request. (Revision 1353014) HBASE-6236 Offline meta repair fails if the HBase base mount point is on a different cluster/volume than its parent in a ViewFS or similar FS (Aditya) (Revision 1353012) Result = SUCCESS tedyu : Files : * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/HConstants.java * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/util/hbck/OfflineMetaRepair.java tedyu : Files : * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/HConstants.java * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/util/hbck/OfflineMetaRepair.java tedyu : Files : * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/util/hbck/OfflineMetaRepair.java tedyu : Files : * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/util/hbck/OfflineMetaRepair.java > Offline meta repair fails if the HBase base mount point is on a different > cluster/volume than its parent in a ViewFS or similar FS > -- > > Key: HBASE-6236 > URL: https://issues.apache.org/jira/browse/HBASE-6236 > Project: HBase > Issue Type: Bug > Components: hbck >Reporter: Aditya Kishore >Assignee: Aditya Kishore > Fix For: 0.96.0, 0.94.1 > > Attachments: 6236-94.addendum, 6236-94.txt, 6236_trunk.patch, > HBASE-6236_94.patch, HBASE-6236_trunk.patch > > > While building the .META. and \-ROOT\- from FS data alone (HBASE-4377), hbck > tries to move the existing .META. and \-ROOT\- directories to a backup folder. > This backup folder is created at the same level as the base HBase folder > (e.g. /hbase-xx if the base HBase folder is '/hbase'). > In a federated HDFS like ViewFS and other similar FS implementations, it is > not possible to rename files/directories across namespace volumes (ViewFS > guide section 3.5) and as a result hbck crashes. > A solution to this problem is to create the backup directory under the folder > where HBase base folder has been mounted. This ensures that source and > destination of rename operation are on the same namespace volume. > Patch for 0.94 and trunk is attached for review. The patch modifies the > location of the backup directory from '/hbase-xxx' to > '/hbase/.hbcktmp-xxx' -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6236) Offline meta repair fails if the HBase base mount point is on a different cluster/volume than its parent in a ViewFS or similar FS
[ https://issues.apache.org/jira/browse/HBASE-6236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13399920#comment-13399920 ] Hudson commented on HBASE-6236: --- Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #66 (See [https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/66/]) HBASE-6236 Offline meta repair fails if the HBase base mount point is on a different cluster/volume than its parent in a ViewFS or sim ilar FS (Aditya) (Revision 1353065) Result = FAILURE tedyu : Files : * /hbase/trunk/hbase-common/src/main/java/org/apache/hadoop/hbase/HConstants.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/util/hbck/OfflineMetaRepair.java > Offline meta repair fails if the HBase base mount point is on a different > cluster/volume than its parent in a ViewFS or similar FS > -- > > Key: HBASE-6236 > URL: https://issues.apache.org/jira/browse/HBASE-6236 > Project: HBase > Issue Type: Bug > Components: hbck >Reporter: Aditya Kishore >Assignee: Aditya Kishore > Fix For: 0.96.0, 0.94.1 > > Attachments: 6236-94.addendum, 6236-94.txt, 6236_trunk.patch, > HBASE-6236_94.patch, HBASE-6236_trunk.patch > > > While building the .META. and \-ROOT\- from FS data alone (HBASE-4377), hbck > tries to move the existing .META. and \-ROOT\- directories to a backup folder. > This backup folder is created at the same level as the base HBase folder > (e.g. /hbase-xx if the base HBase folder is '/hbase'). > In a federated HDFS like ViewFS and other similar FS implementations, it is > not possible to rename files/directories across namespace volumes (ViewFS > guide section 3.5) and as a result hbck crashes. > A solution to this problem is to create the backup directory under the folder > where HBase base folder has been mounted. This ensures that source and > destination of rename operation are on the same namespace volume. > Patch for 0.94 and trunk is attached for review. The patch modifies the > location of the backup directory from '/hbase-xxx' to > '/hbase/.hbcktmp-xxx' -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6236) Offline meta repair fails if the HBase base mount point is on a different cluster/volume than its parent in a ViewFS or similar FS
[ https://issues.apache.org/jira/browse/HBASE-6236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13399842#comment-13399842 ] Hudson commented on HBASE-6236: --- Integrated in HBase-TRUNK #3063 (See [https://builds.apache.org/job/HBase-TRUNK/3063/]) HBASE-6236 Offline meta repair fails if the HBase base mount point is on a different cluster/volume than its parent in a ViewFS or sim ilar FS (Aditya) (Revision 1353065) Result = SUCCESS tedyu : Files : * /hbase/trunk/hbase-common/src/main/java/org/apache/hadoop/hbase/HConstants.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/util/hbck/OfflineMetaRepair.java > Offline meta repair fails if the HBase base mount point is on a different > cluster/volume than its parent in a ViewFS or similar FS > -- > > Key: HBASE-6236 > URL: https://issues.apache.org/jira/browse/HBASE-6236 > Project: HBase > Issue Type: Bug > Components: hbck >Reporter: Aditya Kishore >Assignee: Aditya Kishore > Fix For: 0.96.0, 0.94.1 > > Attachments: 6236-94.addendum, 6236-94.txt, 6236_trunk.patch, > HBASE-6236_94.patch, HBASE-6236_trunk.patch > > > While building the .META. and \-ROOT\- from FS data alone (HBASE-4377), hbck > tries to move the existing .META. and \-ROOT\- directories to a backup folder. > This backup folder is created at the same level as the base HBase folder > (e.g. /hbase-xx if the base HBase folder is '/hbase'). > In a federated HDFS like ViewFS and other similar FS implementations, it is > not possible to rename files/directories across namespace volumes (ViewFS > guide section 3.5) and as a result hbck crashes. > A solution to this problem is to create the backup directory under the folder > where HBase base folder has been mounted. This ensures that source and > destination of rename operation are on the same namespace volume. > Patch for 0.94 and trunk is attached for review. The patch modifies the > location of the backup directory from '/hbase-xxx' to > '/hbase/.hbcktmp-xxx' -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6236) Offline meta repair fails if the HBase base mount point is on a different cluster/volume than its parent in a ViewFS or similar FS
[ https://issues.apache.org/jira/browse/HBASE-6236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13399841#comment-13399841 ] Hudson commented on HBASE-6236: --- Integrated in HBase-0.94 #277 (See [https://builds.apache.org/job/HBase-0.94/277/]) HBASE-6236 Addendum adds -sidelineDir option to hbck (Aditya) (Revision 1353066) HBASE-6236 Offline meta repair fails if the HBase base mount point is on a different cluster/volume than its parent in a ViewFS or similar FS (Aditya) (Revision 1353064) Result = FAILURE tedyu : Files : * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/HConstants.java * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/util/hbck/OfflineMetaRepair.java tedyu : Files : * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/HConstants.java * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/util/hbck/OfflineMetaRepair.java > Offline meta repair fails if the HBase base mount point is on a different > cluster/volume than its parent in a ViewFS or similar FS > -- > > Key: HBASE-6236 > URL: https://issues.apache.org/jira/browse/HBASE-6236 > Project: HBase > Issue Type: Bug > Components: hbck >Reporter: Aditya Kishore >Assignee: Aditya Kishore > Fix For: 0.96.0, 0.94.1 > > Attachments: 6236-94.addendum, 6236-94.txt, 6236_trunk.patch, > HBASE-6236_94.patch, HBASE-6236_trunk.patch > > > While building the .META. and \-ROOT\- from FS data alone (HBASE-4377), hbck > tries to move the existing .META. and \-ROOT\- directories to a backup folder. > This backup folder is created at the same level as the base HBase folder > (e.g. /hbase-xx if the base HBase folder is '/hbase'). > In a federated HDFS like ViewFS and other similar FS implementations, it is > not possible to rename files/directories across namespace volumes (ViewFS > guide section 3.5) and as a result hbck crashes. > A solution to this problem is to create the backup directory under the folder > where HBase base folder has been mounted. This ensures that source and > destination of rename operation are on the same namespace volume. > Patch for 0.94 and trunk is attached for review. The patch modifies the > location of the backup directory from '/hbase-xxx' to > '/hbase/.hbcktmp-xxx' -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6236) Offline meta repair fails if the HBase base mount point is on a different cluster/volume than its parent in a ViewFS or similar FS
[ https://issues.apache.org/jira/browse/HBASE-6236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13399833#comment-13399833 ] Hadoop QA commented on HBASE-6236: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12533163/6236-94.txt against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. -1 patch. The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/2232//console This message is automatically generated. > Offline meta repair fails if the HBase base mount point is on a different > cluster/volume than its parent in a ViewFS or similar FS > -- > > Key: HBASE-6236 > URL: https://issues.apache.org/jira/browse/HBASE-6236 > Project: HBase > Issue Type: Bug > Components: hbck >Reporter: Aditya Kishore >Assignee: Aditya Kishore > Fix For: 0.96.0, 0.94.1 > > Attachments: 6236-94.txt, 6236_trunk.patch, HBASE-6236_94.patch, > HBASE-6236_trunk.patch > > > While building the .META. and \-ROOT\- from FS data alone (HBASE-4377), hbck > tries to move the existing .META. and \-ROOT\- directories to a backup folder. > This backup folder is created at the same level as the base HBase folder > (e.g. /hbase-xx if the base HBase folder is '/hbase'). > In a federated HDFS like ViewFS and other similar FS implementations, it is > not possible to rename files/directories across namespace volumes (ViewFS > guide section 3.5) and as a result hbck crashes. > A solution to this problem is to create the backup directory under the folder > where HBase base folder has been mounted. This ensures that source and > destination of rename operation are on the same namespace volume. > Patch for 0.94 and trunk is attached for review. The patch modifies the > location of the backup directory from '/hbase-xxx' to > '/hbase/.hbcktmp-xxx' -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6236) Offline meta repair fails if the HBase base mount point is on a different cluster/volume than its parent in a ViewFS or similar FS
[ https://issues.apache.org/jira/browse/HBASE-6236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13399832#comment-13399832 ] Zhihong Ted Yu commented on HBASE-6236: --- Integrated to 0.94 and trunk. Thanks for the patch, Aditya. Thanks for the review, Jon and Jimmy. > Offline meta repair fails if the HBase base mount point is on a different > cluster/volume than its parent in a ViewFS or similar FS > -- > > Key: HBASE-6236 > URL: https://issues.apache.org/jira/browse/HBASE-6236 > Project: HBase > Issue Type: Bug > Components: hbck >Reporter: Aditya Kishore >Assignee: Aditya Kishore > Fix For: 0.96.0, 0.94.1 > > Attachments: 6236-94.txt, 6236_trunk.patch, HBASE-6236_94.patch, > HBASE-6236_trunk.patch > > > While building the .META. and \-ROOT\- from FS data alone (HBASE-4377), hbck > tries to move the existing .META. and \-ROOT\- directories to a backup folder. > This backup folder is created at the same level as the base HBase folder > (e.g. /hbase-xx if the base HBase folder is '/hbase'). > In a federated HDFS like ViewFS and other similar FS implementations, it is > not possible to rename files/directories across namespace volumes (ViewFS > guide section 3.5) and as a result hbck crashes. > A solution to this problem is to create the backup directory under the folder > where HBase base folder has been mounted. This ensures that source and > destination of rename operation are on the same namespace volume. > Patch for 0.94 and trunk is attached for review. The patch modifies the > location of the backup directory from '/hbase-xxx' to > '/hbase/.hbcktmp-xxx' -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6236) Offline meta repair fails if the HBase base mount point is on a different cluster/volume than its parent in a ViewFS or similar FS
[ https://issues.apache.org/jira/browse/HBASE-6236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13399810#comment-13399810 ] Hadoop QA commented on HBASE-6236: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12533142/HBASE-6236_trunk.patch against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 hadoop2.0. The patch compiles against the hadoop 2.0 profile. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. -1 findbugs. The patch appears to introduce 11 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in . Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/2229//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/2229//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/2229//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/2229//console This message is automatically generated. > Offline meta repair fails if the HBase base mount point is on a different > cluster/volume than its parent in a ViewFS or similar FS > -- > > Key: HBASE-6236 > URL: https://issues.apache.org/jira/browse/HBASE-6236 > Project: HBase > Issue Type: Bug > Components: hbck >Reporter: Aditya Kishore >Assignee: Aditya Kishore > Fix For: 0.96.0, 0.94.1 > > Attachments: 6236_trunk.patch, HBASE-6236_94.patch, > HBASE-6236_trunk.patch > > > While building the .META. and \-ROOT\- from FS data alone (HBASE-4377), hbck > tries to move the existing .META. and \-ROOT\- directories to a backup folder. > This backup folder is created at the same level as the base HBase folder > (e.g. /hbase-xx if the base HBase folder is '/hbase'). > In a federated HDFS like ViewFS and other similar FS implementations, it is > not possible to rename files/directories across namespace volumes (ViewFS > guide section 3.5) and as a result hbck crashes. > A solution to this problem is to create the backup directory under the folder > where HBase base folder has been mounted. This ensures that source and > destination of rename operation are on the same namespace volume. > Patch for 0.94 and trunk is attached for review. The patch modifies the > location of the backup directory from '/hbase-xxx' to > '/hbase/.hbcktmp-xxx' -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6236) Offline meta repair fails if the HBase base mount point is on a different cluster/volume than its parent in a ViewFS or similar FS
[ https://issues.apache.org/jira/browse/HBASE-6236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13399782#comment-13399782 ] Hadoop QA commented on HBASE-6236: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12533129/HBASE-6236_trunk.patch against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 hadoop2.0. The patch compiles against the hadoop 2.0 profile. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. -1 findbugs. The patch appears to introduce 11 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: org.apache.hadoop.hbase.regionserver.TestSplitTransactionOnCluster Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/2227//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/2227//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/2227//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/2227//console This message is automatically generated. > Offline meta repair fails if the HBase base mount point is on a different > cluster/volume than its parent in a ViewFS or similar FS > -- > > Key: HBASE-6236 > URL: https://issues.apache.org/jira/browse/HBASE-6236 > Project: HBase > Issue Type: Bug > Components: hbck >Reporter: Aditya Kishore >Assignee: Aditya Kishore > Fix For: 0.96.0, 0.94.1 > > Attachments: 6236_trunk.patch, HBASE-6236_94.patch, > HBASE-6236_trunk.patch > > > While building the .META. and \-ROOT\- from FS data alone (HBASE-4377), hbck > tries to move the existing .META. and \-ROOT\- directories to a backup folder. > This backup folder is created at the same level as the base HBase folder > (e.g. /hbase-xx if the base HBase folder is '/hbase'). > In a federated HDFS like ViewFS and other similar FS implementations, it is > not possible to rename files/directories across namespace volumes (ViewFS > guide section 3.5) and as a result hbck crashes. > A solution to this problem is to create the backup directory under the folder > where HBase base folder has been mounted. This ensures that source and > destination of rename operation are on the same namespace volume. > Patch for 0.94 and trunk is attached for review. The patch modifies the > location of the backup directory from '/hbase-xxx' to > '/hbase/.hbcktmp-xxx' -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6236) Offline meta repair fails if the HBase base mount point is on a different cluster/volume than its parent in a ViewFS or similar FS
[ https://issues.apache.org/jira/browse/HBASE-6236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13399734#comment-13399734 ] Hudson commented on HBASE-6236: --- Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #65 (See [https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/65/]) HBASE-6236 revert upon Jimmy's request. (Revision 1353013) HBASE-6236 Offline meta repair fails if the HBase base mount point is on a different cluster/volume than its parent in a ViewFS or similar FS (Aditya) (Revision 1353008) Result = FAILURE tedyu : Files : * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/util/hbck/OfflineMetaRepair.java tedyu : Files : * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/util/hbck/OfflineMetaRepair.java > Offline meta repair fails if the HBase base mount point is on a different > cluster/volume than its parent in a ViewFS or similar FS > -- > > Key: HBASE-6236 > URL: https://issues.apache.org/jira/browse/HBASE-6236 > Project: HBase > Issue Type: Bug > Components: hbck >Reporter: Aditya Kishore >Assignee: Aditya Kishore > Fix For: 0.96.0, 0.94.1 > > Attachments: 6236_trunk.patch, HBASE-6236_94.patch, > HBASE-6236_trunk.patch > > > While building the .META. and \-ROOT\- from FS data alone (HBASE-4377), hbck > tries to move the existing .META. and \-ROOT\- directories to a backup folder. > This backup folder is created at the same level as the base HBase folder > (e.g. /hbase-xx if the base HBase folder is '/hbase'). > In a federated HDFS like ViewFS and other similar FS implementations, it is > not possible to rename files/directories across namespace volumes (ViewFS > guide section 3.5) and as a result hbck crashes. > A solution to this problem is to create the backup directory under the folder > where HBase base folder has been mounted. This ensures that source and > destination of rename operation are on the same namespace volume. > Patch for 0.94 and trunk is attached for review. The patch modifies the > location of the backup directory from '/hbase-xxx' to > '/hbase/.hbcktmp-xxx' -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6236) Offline meta repair fails if the HBase base mount point is on a different cluster/volume than its parent in a ViewFS or similar FS
[ https://issues.apache.org/jira/browse/HBASE-6236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13399727#comment-13399727 ] Jimmy Xiang commented on HBASE-6236: Looks good to me. Should we also add this sideline dir option to HBaseFsck? > Offline meta repair fails if the HBase base mount point is on a different > cluster/volume than its parent in a ViewFS or similar FS > -- > > Key: HBASE-6236 > URL: https://issues.apache.org/jira/browse/HBASE-6236 > Project: HBase > Issue Type: Bug > Components: hbck >Reporter: Aditya Kishore >Assignee: Aditya Kishore > Fix For: 0.96.0, 0.94.1 > > Attachments: 6236_trunk.patch, HBASE-6236_94.patch, > HBASE-6236_trunk.patch > > > While building the .META. and \-ROOT\- from FS data alone (HBASE-4377), hbck > tries to move the existing .META. and \-ROOT\- directories to a backup folder. > This backup folder is created at the same level as the base HBase folder > (e.g. /hbase-xx if the base HBase folder is '/hbase'). > In a federated HDFS like ViewFS and other similar FS implementations, it is > not possible to rename files/directories across namespace volumes (ViewFS > guide section 3.5) and as a result hbck crashes. > A solution to this problem is to create the backup directory under the folder > where HBase base folder has been mounted. This ensures that source and > destination of rename operation are on the same namespace volume. > Patch for 0.94 and trunk is attached for review. The patch modifies the > location of the backup directory from '/hbase-xxx' to > '/hbase/.hbcktmp-xxx' -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6236) Offline meta repair fails if the HBase base mount point is on a different cluster/volume than its parent in a ViewFS or similar FS
[ https://issues.apache.org/jira/browse/HBASE-6236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13399703#comment-13399703 ] Jonathan Hsieh commented on HBASE-6236: --- @Aditya Go ahead and fix it in this instance. If there are other instances that have a similar problem, file and fix them in a follow-up jira? I probably wrote the exemplar which has the bug in the first place. :) > Offline meta repair fails if the HBase base mount point is on a different > cluster/volume than its parent in a ViewFS or similar FS > -- > > Key: HBASE-6236 > URL: https://issues.apache.org/jira/browse/HBASE-6236 > Project: HBase > Issue Type: Bug > Components: hbck >Reporter: Aditya Kishore >Assignee: Aditya Kishore > Fix For: 0.96.0, 0.94.1 > > Attachments: 6236_trunk.patch, HBASE-6236_94.patch, > HBASE-6236_trunk.patch, HBASE-6236_trunk.patch > > > While building the .META. and -ROOT- from FS data alone (HBASE-4377), hbck > tries to move the existing .META. and -ROOT- directories to a backup folder. > This backup folder is created at the same level as the base HBase folder > (e.g. /hbase-xx if the base HBase folder is '/hbase'). > In a federated HDFS like ViewFS and other similar FS implementations, it is > not possible to rename files/directories across namespace volumes (ViewFS > guide section 3.5) and as a result hbck crashes. > A solution to this problem is to create the backup directory under the folder > where HBase base folder has been mounted. This ensures that source and > destination of rename operation are on the same namespace volume. > Patch for 0.94 and trunk is attached for review. The patch modifies the > location of the backup directory from '/hbase-xxx' to > '/hbase/.hbcktmp-xxx' -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6236) Offline meta repair fails if the HBase base mount point is on a different cluster/volume than its parent in a ViewFS or similar FS
[ https://issues.apache.org/jira/browse/HBASE-6236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13399693#comment-13399693 ] Zhihong Ted Yu commented on HBASE-6236: --- I think Aditya was following existing convention: {code} } else if (cmd.equals("-base")) { // update hbase root dir to user-specified base i++; String path = args[i]; {code} > Offline meta repair fails if the HBase base mount point is on a different > cluster/volume than its parent in a ViewFS or similar FS > -- > > Key: HBASE-6236 > URL: https://issues.apache.org/jira/browse/HBASE-6236 > Project: HBase > Issue Type: Bug > Components: hbck >Reporter: Aditya Kishore >Assignee: Aditya Kishore > Fix For: 0.96.0, 0.94.1 > > Attachments: 6236_trunk.patch, HBASE-6236_94.patch, > HBASE-6236_trunk.patch, HBASE-6236_trunk.patch > > > While building the .META. and -ROOT- from FS data alone (HBASE-4377), hbck > tries to move the existing .META. and -ROOT- directories to a backup folder. > This backup folder is created at the same level as the base HBase folder > (e.g. /hbase-xx if the base HBase folder is '/hbase'). > In a federated HDFS like ViewFS and other similar FS implementations, it is > not possible to rename files/directories across namespace volumes (ViewFS > guide section 3.5) and as a result hbck crashes. > A solution to this problem is to create the backup directory under the folder > where HBase base folder has been mounted. This ensures that source and > destination of rename operation are on the same namespace volume. > Patch for 0.94 and trunk is attached for review. The patch modifies the > location of the backup directory from '/hbase-xxx' to > '/hbase/.hbcktmp-xxx' -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6236) Offline meta repair fails if the HBase base mount point is on a different cluster/volume than its parent in a ViewFS or similar FS
[ https://issues.apache.org/jira/browse/HBASE-6236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13399694#comment-13399694 ] Aditya Kishore commented on HBASE-6236: --- @Jonathan: I noticed that too but wasn't sure whether to combine that issue with the current one. I guess there is no harm in fixing that while I am touching this part of code. Will include it in the patch. > Offline meta repair fails if the HBase base mount point is on a different > cluster/volume than its parent in a ViewFS or similar FS > -- > > Key: HBASE-6236 > URL: https://issues.apache.org/jira/browse/HBASE-6236 > Project: HBase > Issue Type: Bug > Components: hbck >Reporter: Aditya Kishore >Assignee: Aditya Kishore > Fix For: 0.96.0, 0.94.1 > > Attachments: 6236_trunk.patch, HBASE-6236_94.patch, > HBASE-6236_trunk.patch, HBASE-6236_trunk.patch > > > While building the .META. and -ROOT- from FS data alone (HBASE-4377), hbck > tries to move the existing .META. and -ROOT- directories to a backup folder. > This backup folder is created at the same level as the base HBase folder > (e.g. /hbase-xx if the base HBase folder is '/hbase'). > In a federated HDFS like ViewFS and other similar FS implementations, it is > not possible to rename files/directories across namespace volumes (ViewFS > guide section 3.5) and as a result hbck crashes. > A solution to this problem is to create the backup directory under the folder > where HBase base folder has been mounted. This ensures that source and > destination of rename operation are on the same namespace volume. > Patch for 0.94 and trunk is attached for review. The patch modifies the > location of the backup directory from '/hbase-xxx' to > '/hbase/.hbcktmp-xxx' -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6236) Offline meta repair fails if the HBase base mount point is on a different cluster/volume than its parent in a ViewFS or similar FS
[ https://issues.apache.org/jira/browse/HBASE-6236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13399695#comment-13399695 ] Aditya Kishore commented on HBASE-6236: --- @Ted, Yes, I was :) > Offline meta repair fails if the HBase base mount point is on a different > cluster/volume than its parent in a ViewFS or similar FS > -- > > Key: HBASE-6236 > URL: https://issues.apache.org/jira/browse/HBASE-6236 > Project: HBase > Issue Type: Bug > Components: hbck >Reporter: Aditya Kishore >Assignee: Aditya Kishore > Fix For: 0.96.0, 0.94.1 > > Attachments: 6236_trunk.patch, HBASE-6236_94.patch, > HBASE-6236_trunk.patch, HBASE-6236_trunk.patch > > > While building the .META. and -ROOT- from FS data alone (HBASE-4377), hbck > tries to move the existing .META. and -ROOT- directories to a backup folder. > This backup folder is created at the same level as the base HBase folder > (e.g. /hbase-xx if the base HBase folder is '/hbase'). > In a federated HDFS like ViewFS and other similar FS implementations, it is > not possible to rename files/directories across namespace volumes (ViewFS > guide section 3.5) and as a result hbck crashes. > A solution to this problem is to create the backup directory under the folder > where HBase base folder has been mounted. This ensures that source and > destination of rename operation are on the same namespace volume. > Patch for 0.94 and trunk is attached for review. The patch modifies the > location of the backup directory from '/hbase-xxx' to > '/hbase/.hbcktmp-xxx' -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6236) Offline meta repair fails if the HBase base mount point is on a different cluster/volume than its parent in a ViewFS or similar FS
[ https://issues.apache.org/jira/browse/HBASE-6236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13399690#comment-13399690 ] Jonathan Hsieh commented on HBASE-6236: --- @Aditya I only looked at the 21/Jun/12 22:49 version -- assuming others have a similar problem. There is a possible array out of bounds exception here. Can we check to make sure we are not out of bounds and provide an appropriate warning message and bail out if we are? {code} conf.set(HConstants.HBASE_DIR, path); conf.set("fs.defaultFS", conf.get(HConstants.HBASE_DIR)); conf.set("fs.default.name", conf.get(HConstants.HBASE_DIR)); + } else if (cmd.equals("-sidelineDir")) { +i++; +fsck.setSidelineDir(args[i]); } else if (cmd.equals("-fixHoles")) { fixHoles = true; } else if (cmd.equals("-fix")) { {code} > Offline meta repair fails if the HBase base mount point is on a different > cluster/volume than its parent in a ViewFS or similar FS > -- > > Key: HBASE-6236 > URL: https://issues.apache.org/jira/browse/HBASE-6236 > Project: HBase > Issue Type: Bug > Components: hbck >Reporter: Aditya Kishore >Assignee: Aditya Kishore > Fix For: 0.96.0, 0.94.1 > > Attachments: 6236_trunk.patch, HBASE-6236_94.patch, > HBASE-6236_trunk.patch, HBASE-6236_trunk.patch > > > While building the .META. and -ROOT- from FS data alone (HBASE-4377), hbck > tries to move the existing .META. and -ROOT- directories to a backup folder. > This backup folder is created at the same level as the base HBase folder > (e.g. /hbase-xx if the base HBase folder is '/hbase'). > In a federated HDFS like ViewFS and other similar FS implementations, it is > not possible to rename files/directories across namespace volumes (ViewFS > guide section 3.5) and as a result hbck crashes. > A solution to this problem is to create the backup directory under the folder > where HBase base folder has been mounted. This ensures that source and > destination of rename operation are on the same namespace volume. > Patch for 0.94 and trunk is attached for review. The patch modifies the > location of the backup directory from '/hbase-xxx' to > '/hbase/.hbcktmp-xxx' -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6236) Offline meta repair fails if the HBase base mount point is on a different cluster/volume than its parent in a ViewFS or similar FS
[ https://issues.apache.org/jira/browse/HBASE-6236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13399659#comment-13399659 ] Aditya Kishore commented on HBASE-6236: --- And it might be a good idea to add ".hbck" to the list "org.apache.hadoop.hbase.HConstants.HBASE_NON_USER_TABLE_DIRS" along with ".logs", ".oldlogs", ".corrupt", etc. > Offline meta repair fails if the HBase base mount point is on a different > cluster/volume than its parent in a ViewFS or similar FS > -- > > Key: HBASE-6236 > URL: https://issues.apache.org/jira/browse/HBASE-6236 > Project: HBase > Issue Type: Bug > Components: hbck >Reporter: Aditya Kishore >Assignee: Aditya Kishore > Fix For: 0.96.0, 0.94.1 > > Attachments: 6236_trunk.patch, HBASE-6236_94.patch, > HBASE-6236_trunk.patch, HBASE-6236_trunk.patch > > > While building the .META. and -ROOT- from FS data alone (HBASE-4377), hbck > tries to move the existing .META. and -ROOT- directories to a backup folder. > This backup folder is created at the same level as the base HBase folder > (e.g. /hbase-xx if the base HBase folder is '/hbase'). > In a federated HDFS like ViewFS and other similar FS implementations, it is > not possible to rename files/directories across namespace volumes (ViewFS > guide section 3.5) and as a result hbck crashes. > A solution to this problem is to create the backup directory under the folder > where HBase base folder has been mounted. This ensures that source and > destination of rename operation are on the same namespace volume. > Patch for 0.94 and trunk is attached for review. The patch modifies the > location of the backup directory from '/hbase-xxx' to > '/hbase/.hbcktmp-xxx' -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6236) Offline meta repair fails if the HBase base mount point is on a different cluster/volume than its parent in a ViewFS or similar FS
[ https://issues.apache.org/jira/browse/HBASE-6236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13399652#comment-13399652 ] Jimmy Xiang commented on HBASE-6236: That's right. I prefer /hbase/.hbck/hbase-xx as a default, with option to change it. > Offline meta repair fails if the HBase base mount point is on a different > cluster/volume than its parent in a ViewFS or similar FS > -- > > Key: HBASE-6236 > URL: https://issues.apache.org/jira/browse/HBASE-6236 > Project: HBase > Issue Type: Bug > Components: hbck >Reporter: Aditya Kishore >Assignee: Aditya Kishore > Fix For: 0.96.0, 0.94.1 > > Attachments: 6236_trunk.patch, HBASE-6236_94.patch, > HBASE-6236_trunk.patch, HBASE-6236_trunk.patch > > > While building the .META. and -ROOT- from FS data alone (HBASE-4377), hbck > tries to move the existing .META. and -ROOT- directories to a backup folder. > This backup folder is created at the same level as the base HBase folder > (e.g. /hbase-xx if the base HBase folder is '/hbase'). > In a federated HDFS like ViewFS and other similar FS implementations, it is > not possible to rename files/directories across namespace volumes (ViewFS > guide section 3.5) and as a result hbck crashes. > A solution to this problem is to create the backup directory under the folder > where HBase base folder has been mounted. This ensures that source and > destination of rename operation are on the same namespace volume. > Patch for 0.94 and trunk is attached for review. The patch modifies the > location of the backup directory from '/hbase-xxx' to > '/hbase/.hbcktmp-xxx' -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6236) Offline meta repair fails if the HBase base mount point is on a different cluster/volume than its parent in a ViewFS or similar FS
[ https://issues.apache.org/jira/browse/HBASE-6236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13399646#comment-13399646 ] Aditya Kishore commented on HBASE-6236: --- @Jimmy To be sure, you want the folders to take the form of "/hbase/.hbck/hbase-xx", right? This way they won't clutter the "/hbase/" when many of them get created during. > Offline meta repair fails if the HBase base mount point is on a different > cluster/volume than its parent in a ViewFS or similar FS > -- > > Key: HBASE-6236 > URL: https://issues.apache.org/jira/browse/HBASE-6236 > Project: HBase > Issue Type: Bug > Components: hbck >Reporter: Aditya Kishore >Assignee: Aditya Kishore > Fix For: 0.96.0, 0.94.1 > > Attachments: 6236_trunk.patch, HBASE-6236_94.patch, > HBASE-6236_trunk.patch, HBASE-6236_trunk.patch > > > While building the .META. and -ROOT- from FS data alone (HBASE-4377), hbck > tries to move the existing .META. and -ROOT- directories to a backup folder. > This backup folder is created at the same level as the base HBase folder > (e.g. /hbase-xx if the base HBase folder is '/hbase'). > In a federated HDFS like ViewFS and other similar FS implementations, it is > not possible to rename files/directories across namespace volumes (ViewFS > guide section 3.5) and as a result hbck crashes. > A solution to this problem is to create the backup directory under the folder > where HBase base folder has been mounted. This ensures that source and > destination of rename operation are on the same namespace volume. > Patch for 0.94 and trunk is attached for review. The patch modifies the > location of the backup directory from '/hbase-xxx' to > '/hbase/.hbcktmp-xxx' -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6236) Offline meta repair fails if the HBase base mount point is on a different cluster/volume than its parent in a ViewFS or similar FS
[ https://issues.apache.org/jira/browse/HBASE-6236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13399634#comment-13399634 ] Jimmy Xiang commented on HBASE-6236: @Aditya, good catch. Please make sure they are the same. Can you use /hbase/.hbck as the default? > Offline meta repair fails if the HBase base mount point is on a different > cluster/volume than its parent in a ViewFS or similar FS > -- > > Key: HBASE-6236 > URL: https://issues.apache.org/jira/browse/HBASE-6236 > Project: HBase > Issue Type: Bug > Components: hbck >Reporter: Aditya Kishore >Assignee: Aditya Kishore > Fix For: 0.96.0, 0.94.1 > > Attachments: 6236_trunk.patch, HBASE-6236_94.patch, > HBASE-6236_trunk.patch, HBASE-6236_trunk.patch > > > While building the .META. and -ROOT- from FS data alone (HBASE-4377), hbck > tries to move the existing .META. and -ROOT- directories to a backup folder. > This backup folder is created at the same level as the base HBase folder > (e.g. /hbase-xx if the base HBase folder is '/hbase'). > In a federated HDFS like ViewFS and other similar FS implementations, it is > not possible to rename files/directories across namespace volumes (ViewFS > guide section 3.5) and as a result hbck crashes. > A solution to this problem is to create the backup directory under the folder > where HBase base folder has been mounted. This ensures that source and > destination of rename operation are on the same namespace volume. > Patch for 0.94 and trunk is attached for review. The patch modifies the > location of the backup directory from '/hbase-xxx' to > '/hbase/.hbcktmp-xxx' -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6236) Offline meta repair fails if the HBase base mount point is on a different cluster/volume than its parent in a ViewFS or similar FS
[ https://issues.apache.org/jira/browse/HBASE-6236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13399628#comment-13399628 ] Aditya Kishore commented on HBASE-6236: --- @Jimmy Currently there are two different locations where hbck sidelines the backup data. First one takes the form '/hbck/hbase-xx' and is used to sideline regions {code:title=HBaseFsck.java} 857 private Path getSidelineDir() throws IOException { 858 Path hbaseDir = FSUtils.getRootDir(conf); 859 Path hbckDir = new Path(hbaseDir.getParent(), "hbck"); 860 Path backupDir = new Path(hbckDir, hbaseDir.getName() + "-" 861 + startMillis); 862 return backupDir; 863 } {code} While the second one takes the form of '/hbase-x' and is used to sideline the current .META. and \-ROOT\- folders {code:title=HBaseFsck.java} 960 Path backupDir = new Path(hbaseDir.getParent(), hbaseDir.getName() + "-" 961 + startMillis); {code} At lease we need to make them standard. Secondly, I am not sure how pre-creating "/hbck" will solve this problem in a federated HDFS in all cases. Suppose the root of a cluster is mounted as "/hbase" in ViewFS then the pre created "/hbck" folder is going to be on a different namespace volume and renaming of a file from "/hbase/" to "/hbck/" is going to fail. > Offline meta repair fails if the HBase base mount point is on a different > cluster/volume than its parent in a ViewFS or similar FS > -- > > Key: HBASE-6236 > URL: https://issues.apache.org/jira/browse/HBASE-6236 > Project: HBase > Issue Type: Bug > Components: hbck >Reporter: Aditya Kishore >Assignee: Aditya Kishore > Fix For: 0.96.0, 0.94.1 > > Attachments: 6236_trunk.patch, HBASE-6236_94.patch, > HBASE-6236_trunk.patch, HBASE-6236_trunk.patch > > > While building the .META. and -ROOT- from FS data alone (HBASE-4377), hbck > tries to move the existing .META. and -ROOT- directories to a backup folder. > This backup folder is created at the same level as the base HBase folder > (e.g. /hbase-xx if the base HBase folder is '/hbase'). > In a federated HDFS like ViewFS and other similar FS implementations, it is > not possible to rename files/directories across namespace volumes (ViewFS > guide section 3.5) and as a result hbck crashes. > A solution to this problem is to create the backup directory under the folder > where HBase base folder has been mounted. This ensures that source and > destination of rename operation are on the same namespace volume. > Patch for 0.94 and trunk is attached for review. The patch modifies the > location of the backup directory from '/hbase-xxx' to > '/hbase/.hbcktmp-xxx' -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6236) Offline meta repair fails if the HBase base mount point is on a different cluster/volume than its parent in a ViewFS or similar FS
[ https://issues.apache.org/jira/browse/HBASE-6236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13399623#comment-13399623 ] Hudson commented on HBASE-6236: --- Integrated in HBase-TRUNK #3060 (See [https://builds.apache.org/job/HBase-TRUNK/3060/]) HBASE-6236 revert upon Jimmy's request. (Revision 1353013) HBASE-6236 Offline meta repair fails if the HBase base mount point is on a different cluster/volume than its parent in a ViewFS or similar FS (Aditya) (Revision 1353008) Result = FAILURE tedyu : Files : * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/util/hbck/OfflineMetaRepair.java tedyu : Files : * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/util/hbck/OfflineMetaRepair.java > Offline meta repair fails if the HBase base mount point is on a different > cluster/volume than its parent in a ViewFS or similar FS > -- > > Key: HBASE-6236 > URL: https://issues.apache.org/jira/browse/HBASE-6236 > Project: HBase > Issue Type: Bug > Components: hbck >Reporter: Aditya Kishore >Assignee: Aditya Kishore > Fix For: 0.96.0, 0.94.1 > > Attachments: 6236_trunk.patch, HBASE-6236_94.patch, > HBASE-6236_trunk.patch, HBASE-6236_trunk.patch > > > While building the .META. and -ROOT- from FS data alone (HBASE-4377), hbck > tries to move the existing .META. and -ROOT- directories to a backup folder. > This backup folder is created at the same level as the base HBase folder > (e.g. /hbase-xx if the base HBase folder is '/hbase'). > In a federated HDFS like ViewFS and other similar FS implementations, it is > not possible to rename files/directories across namespace volumes (ViewFS > guide section 3.5) and as a result hbck crashes. > A solution to this problem is to create the backup directory under the folder > where HBase base folder has been mounted. This ensures that source and > destination of rename operation are on the same namespace volume. > Patch for 0.94 and trunk is attached for review. The patch modifies the > location of the backup directory from '/hbase-xxx' to > '/hbase/.hbcktmp-xxx' -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6236) Offline meta repair fails if the HBase base mount point is on a different cluster/volume than its parent in a ViewFS or similar FS
[ https://issues.apache.org/jira/browse/HBASE-6236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13399603#comment-13399603 ] Hudson commented on HBASE-6236: --- Integrated in HBase-0.94 #272 (See [https://builds.apache.org/job/HBase-0.94/272/]) HBASE-6236 revert upon Jimmy's request. (Revision 1353014) HBASE-6236 Offline meta repair fails if the HBase base mount point is on a different cluster/volume than its parent in a ViewFS or similar FS (Aditya) (Revision 1353012) Result = FAILURE tedyu : Files : * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/util/hbck/OfflineMetaRepair.java tedyu : Files : * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/util/hbck/OfflineMetaRepair.java > Offline meta repair fails if the HBase base mount point is on a different > cluster/volume than its parent in a ViewFS or similar FS > -- > > Key: HBASE-6236 > URL: https://issues.apache.org/jira/browse/HBASE-6236 > Project: HBase > Issue Type: Bug > Components: hbck >Reporter: Aditya Kishore >Assignee: Aditya Kishore > Fix For: 0.96.0, 0.94.1 > > Attachments: 6236_trunk.patch, HBASE-6236_94.patch, > HBASE-6236_trunk.patch, HBASE-6236_trunk.patch > > > While building the .META. and -ROOT- from FS data alone (HBASE-4377), hbck > tries to move the existing .META. and -ROOT- directories to a backup folder. > This backup folder is created at the same level as the base HBase folder > (e.g. /hbase-xx if the base HBase folder is '/hbase'). > In a federated HDFS like ViewFS and other similar FS implementations, it is > not possible to rename files/directories across namespace volumes (ViewFS > guide section 3.5) and as a result hbck crashes. > A solution to this problem is to create the backup directory under the folder > where HBase base folder has been mounted. This ensures that source and > destination of rename operation are on the same namespace volume. > Patch for 0.94 and trunk is attached for review. The patch modifies the > location of the backup directory from '/hbase-xxx' to > '/hbase/.hbcktmp-xxx' -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6236) Offline meta repair fails if the HBase base mount point is on a different cluster/volume than its parent in a ViewFS or similar FS
[ https://issues.apache.org/jira/browse/HBASE-6236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13399562#comment-13399562 ] Jimmy Xiang commented on HBASE-6236: It is a good idea to have sideline dir option. I think we should keep the existing default folder name. > Offline meta repair fails if the HBase base mount point is on a different > cluster/volume than its parent in a ViewFS or similar FS > -- > > Key: HBASE-6236 > URL: https://issues.apache.org/jira/browse/HBASE-6236 > Project: HBase > Issue Type: Bug > Components: hbck >Reporter: Aditya Kishore >Assignee: Aditya Kishore > Fix For: 0.96.0, 0.94.1 > > Attachments: 6236_trunk.patch, HBASE-6236_94.patch, > HBASE-6236_trunk.patch, HBASE-6236_trunk.patch > > > While building the .META. and -ROOT- from FS data alone (HBASE-4377), hbck > tries to move the existing .META. and -ROOT- directories to a backup folder. > This backup folder is created at the same level as the base HBase folder > (e.g. /hbase-xx if the base HBase folder is '/hbase'). > In a federated HDFS like ViewFS and other similar FS implementations, it is > not possible to rename files/directories across namespace volumes (ViewFS > guide section 3.5) and as a result hbck crashes. > A solution to this problem is to create the backup directory under the folder > where HBase base folder has been mounted. This ensures that source and > destination of rename operation are on the same namespace volume. > Patch for 0.94 and trunk is attached for review. The patch modifies the > location of the backup directory from '/hbase-xxx' to > '/hbase/.hbcktmp-xxx' -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6236) Offline meta repair fails if the HBase base mount point is on a different cluster/volume than its parent in a ViewFS or similar FS
[ https://issues.apache.org/jira/browse/HBASE-6236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13399557#comment-13399557 ] Zhihong Ted Yu commented on HBASE-6236: --- Saw Jimmy's comment and reverted the patch. > Offline meta repair fails if the HBase base mount point is on a different > cluster/volume than its parent in a ViewFS or similar FS > -- > > Key: HBASE-6236 > URL: https://issues.apache.org/jira/browse/HBASE-6236 > Project: HBase > Issue Type: Bug > Components: hbck >Reporter: Aditya Kishore >Assignee: Aditya Kishore > Fix For: 0.96.0, 0.94.1 > > Attachments: 6236_trunk.patch, HBASE-6236_94.patch, > HBASE-6236_trunk.patch, HBASE-6236_trunk.patch > > > While building the .META. and -ROOT- from FS data alone (HBASE-4377), hbck > tries to move the existing .META. and -ROOT- directories to a backup folder. > This backup folder is created at the same level as the base HBase folder > (e.g. /hbase-xx if the base HBase folder is '/hbase'). > In a federated HDFS like ViewFS and other similar FS implementations, it is > not possible to rename files/directories across namespace volumes (ViewFS > guide section 3.5) and as a result hbck crashes. > A solution to this problem is to create the backup directory under the folder > where HBase base folder has been mounted. This ensures that source and > destination of rename operation are on the same namespace volume. > Patch for 0.94 and trunk is attached for review. The patch modifies the > location of the backup directory from '/hbase-xxx' to > '/hbase/.hbcktmp-xxx' -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6236) Offline meta repair fails if the HBase base mount point is on a different cluster/volume than its parent in a ViewFS or similar FS
[ https://issues.apache.org/jira/browse/HBASE-6236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13399551#comment-13399551 ] Zhihong Ted Yu commented on HBASE-6236: --- Integrated to 0.94 as well. Thanks for the patch, Aditya. > Offline meta repair fails if the HBase base mount point is on a different > cluster/volume than its parent in a ViewFS or similar FS > -- > > Key: HBASE-6236 > URL: https://issues.apache.org/jira/browse/HBASE-6236 > Project: HBase > Issue Type: Bug > Components: hbck >Reporter: Aditya Kishore >Assignee: Aditya Kishore > Fix For: 0.96.0, 0.94.1 > > Attachments: 6236_trunk.patch, HBASE-6236_94.patch, > HBASE-6236_trunk.patch, HBASE-6236_trunk.patch > > > While building the .META. and -ROOT- from FS data alone (HBASE-4377), hbck > tries to move the existing .META. and -ROOT- directories to a backup folder. > This backup folder is created at the same level as the base HBase folder > (e.g. /hbase-xx if the base HBase folder is '/hbase'). > In a federated HDFS like ViewFS and other similar FS implementations, it is > not possible to rename files/directories across namespace volumes (ViewFS > guide section 3.5) and as a result hbck crashes. > A solution to this problem is to create the backup directory under the folder > where HBase base folder has been mounted. This ensures that source and > destination of rename operation are on the same namespace volume. > Patch for 0.94 and trunk is attached for review. The patch modifies the > location of the backup directory from '/hbase-xxx' to > '/hbase/.hbcktmp-xxx' -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6236) Offline meta repair fails if the HBase base mount point is on a different cluster/volume than its parent in a ViewFS or similar FS
[ https://issues.apache.org/jira/browse/HBASE-6236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13399546#comment-13399546 ] Jimmy Xiang commented on HBASE-6236: We should revert this change. > Offline meta repair fails if the HBase base mount point is on a different > cluster/volume than its parent in a ViewFS or similar FS > -- > > Key: HBASE-6236 > URL: https://issues.apache.org/jira/browse/HBASE-6236 > Project: HBase > Issue Type: Bug > Components: hbck >Reporter: Aditya Kishore >Assignee: Aditya Kishore > Fix For: 0.96.0, 0.94.1 > > Attachments: 6236_trunk.patch, HBASE-6236_94.patch, > HBASE-6236_trunk.patch, HBASE-6236_trunk.patch > > > While building the .META. and -ROOT- from FS data alone (HBASE-4377), hbck > tries to move the existing .META. and -ROOT- directories to a backup folder. > This backup folder is created at the same level as the base HBase folder > (e.g. /hbase-xx if the base HBase folder is '/hbase'). > In a federated HDFS like ViewFS and other similar FS implementations, it is > not possible to rename files/directories across namespace volumes (ViewFS > guide section 3.5) and as a result hbck crashes. > A solution to this problem is to create the backup directory under the folder > where HBase base folder has been mounted. This ensures that source and > destination of rename operation are on the same namespace volume. > Patch for 0.94 and trunk is attached for review. The patch modifies the > location of the backup directory from '/hbase-xxx' to > '/hbase/.hbcktmp-xxx' -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6236) Offline meta repair fails if the HBase base mount point is on a different cluster/volume than its parent in a ViewFS or similar FS
[ https://issues.apache.org/jira/browse/HBASE-6236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13399544#comment-13399544 ] Jimmy Xiang commented on HBASE-6236: hbck sideline the dir to some folder like /hbck/hbase-xx, not /hbase-x In a federated HDFS, you can pre-create a folder like /hbck, which is for this kind of use cases. > Offline meta repair fails if the HBase base mount point is on a different > cluster/volume than its parent in a ViewFS or similar FS > -- > > Key: HBASE-6236 > URL: https://issues.apache.org/jira/browse/HBASE-6236 > Project: HBase > Issue Type: Bug > Components: hbck >Reporter: Aditya Kishore >Assignee: Aditya Kishore > Fix For: 0.96.0, 0.94.1 > > Attachments: 6236_trunk.patch, HBASE-6236_94.patch, > HBASE-6236_trunk.patch, HBASE-6236_trunk.patch > > > While building the .META. and -ROOT- from FS data alone (HBASE-4377), hbck > tries to move the existing .META. and -ROOT- directories to a backup folder. > This backup folder is created at the same level as the base HBase folder > (e.g. /hbase-xx if the base HBase folder is '/hbase'). > In a federated HDFS like ViewFS and other similar FS implementations, it is > not possible to rename files/directories across namespace volumes (ViewFS > guide section 3.5) and as a result hbck crashes. > A solution to this problem is to create the backup directory under the folder > where HBase base folder has been mounted. This ensures that source and > destination of rename operation are on the same namespace volume. > Patch for 0.94 and trunk is attached for review. The patch modifies the > location of the backup directory from '/hbase-xxx' to > '/hbase/.hbcktmp-xxx' -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6236) Offline meta repair fails if the HBase base mount point is on a different cluster/volume than its parent in a ViewFS or similar FS
[ https://issues.apache.org/jira/browse/HBASE-6236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13399543#comment-13399543 ] Zhihong Ted Yu commented on HBASE-6236: --- Patch integrated to trunk. > Offline meta repair fails if the HBase base mount point is on a different > cluster/volume than its parent in a ViewFS or similar FS > -- > > Key: HBASE-6236 > URL: https://issues.apache.org/jira/browse/HBASE-6236 > Project: HBase > Issue Type: Bug > Components: hbck >Reporter: Aditya Kishore >Assignee: Aditya Kishore > Fix For: 0.96.0, 0.94.1 > > Attachments: 6236_trunk.patch, HBASE-6236_94.patch, > HBASE-6236_trunk.patch, HBASE-6236_trunk.patch > > > While building the .META. and -ROOT- from FS data alone (HBASE-4377), hbck > tries to move the existing .META. and -ROOT- directories to a backup folder. > This backup folder is created at the same level as the base HBase folder > (e.g. /hbase-xx if the base HBase folder is '/hbase'). > In a federated HDFS like ViewFS and other similar FS implementations, it is > not possible to rename files/directories across namespace volumes (ViewFS > guide section 3.5) and as a result hbck crashes. > A solution to this problem is to create the backup directory under the folder > where HBase base folder has been mounted. This ensures that source and > destination of rename operation are on the same namespace volume. > Patch for 0.94 and trunk is attached for review. The patch modifies the > location of the backup directory from '/hbase-xxx' to > '/hbase/.hbcktmp-xxx' -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6236) Offline meta repair fails if the HBase base mount point is on a different cluster/volume than its parent in a ViewFS or similar FS
[ https://issues.apache.org/jira/browse/HBASE-6236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13399462#comment-13399462 ] Aditya Kishore commented on HBASE-6236: --- Even though I am running into some random test case failures, all of these HBCK/HBFsck tests have been passing regularly org.apache.hadoop.hbase.util.hbck.TestOfflineMetaRebuildBase org.apache.hadoop.hbase.util.hbck.TestOfflineMetaRebuildHole org.apache.hadoop.hbase.util.hbck.TestOfflineMetaRebuildOverlap org.apache.hadoop.hbase.util.TestHBaseFsck org.apache.hadoop.hbase.util.TestHBaseFsckComparator > Offline meta repair fails if the HBase base mount point is on a different > cluster/volume than its parent in a ViewFS or similar FS > -- > > Key: HBASE-6236 > URL: https://issues.apache.org/jira/browse/HBASE-6236 > Project: HBase > Issue Type: Bug > Components: hbck >Reporter: Aditya Kishore >Assignee: Aditya Kishore > Attachments: 6236_trunk.patch, HBASE-6236_94.patch, > HBASE-6236_trunk.patch, HBASE-6236_trunk.patch > > > While building the .META. and -ROOT- from FS data alone (HBASE-4377), hbck > tries to move the existing .META. and -ROOT- directories to a backup folder. > This backup folder is created at the same level as the base HBase folder > (e.g. /hbase-xx if the base HBase folder is '/hbase'). > In a federated HDFS like ViewFS and other similar FS implementations, it is > not possible to rename files/directories across namespace volumes (ViewFS > guide section 3.5) and as a result hbck crashes. > A solution to this problem is to create the backup directory under the folder > where HBase base folder has been mounted. This ensures that source and > destination of rename operation are on the same namespace volume. > Patch for 0.94 and trunk is attached for review. The patch modifies the > location of the backup directory from '/hbase-xxx' to > '/hbase/.hbcktmp-xxx' -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6236) Offline meta repair fails if the HBase base mount point is on a different cluster/volume than its parent in a ViewFS or similar FS
[ https://issues.apache.org/jira/browse/HBASE-6236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13399450#comment-13399450 ] Zhihong Ted Yu commented on HBASE-6236: --- The failed tests are unrelated to the patch. I ran them and they passed. > Offline meta repair fails if the HBase base mount point is on a different > cluster/volume than its parent in a ViewFS or similar FS > -- > > Key: HBASE-6236 > URL: https://issues.apache.org/jira/browse/HBASE-6236 > Project: HBase > Issue Type: Bug > Components: hbck >Reporter: Aditya Kishore >Assignee: Aditya Kishore > Attachments: 6236_trunk.patch, HBASE-6236_94.patch, > HBASE-6236_trunk.patch, HBASE-6236_trunk.patch > > > While building the .META. and -ROOT- from FS data alone (HBASE-4377), hbck > tries to move the existing .META. and -ROOT- directories to a backup folder. > This backup folder is created at the same level as the base HBase folder > (e.g. /hbase-xx if the base HBase folder is '/hbase'). > In a federated HDFS like ViewFS and other similar FS implementations, it is > not possible to rename files/directories across namespace volumes (ViewFS > guide section 3.5) and as a result hbck crashes. > A solution to this problem is to create the backup directory under the folder > where HBase base folder has been mounted. This ensures that source and > destination of rename operation are on the same namespace volume. > Patch for 0.94 and trunk is attached for review. The patch modifies the > location of the backup directory from '/hbase-xxx' to > '/hbase/.hbcktmp-xxx' -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6236) Offline meta repair fails if the HBase base mount point is on a different cluster/volume than its parent in a ViewFS or similar FS
[ https://issues.apache.org/jira/browse/HBASE-6236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13399444#comment-13399444 ] Hadoop QA commented on HBASE-6236: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12532994/HBASE-6236_trunk.patch against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 hadoop2.0. The patch compiles against the hadoop 2.0 profile. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. -1 findbugs. The patch appears to introduce 11 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: org.apache.hadoop.hbase.regionserver.TestAtomicOperation org.apache.hadoop.hbase.security.access.TestAccessController org.apache.hadoop.hbase.util.TestFSUtils Test results: https://builds.apache.org/job/PreCommit-HBASE-Build///testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build///artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build///artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build///console This message is automatically generated. > Offline meta repair fails if the HBase base mount point is on a different > cluster/volume than its parent in a ViewFS or similar FS > -- > > Key: HBASE-6236 > URL: https://issues.apache.org/jira/browse/HBASE-6236 > Project: HBase > Issue Type: Bug > Components: hbck >Reporter: Aditya Kishore >Assignee: Aditya Kishore > Attachments: 6236_trunk.patch, HBASE-6236_94.patch, > HBASE-6236_trunk.patch, HBASE-6236_trunk.patch > > > While building the .META. and -ROOT- from FS data alone (HBASE-4377), hbck > tries to move the existing .META. and -ROOT- directories to a backup folder. > This backup folder is created at the same level as the base HBase folder > (e.g. /hbase-xx if the base HBase folder is '/hbase'). > In a federated HDFS like ViewFS and other similar FS implementations, it is > not possible to rename files/directories across namespace volumes (ViewFS > guide section 3.5) and as a result hbck crashes. > A solution to this problem is to create the backup directory under the folder > where HBase base folder has been mounted. This ensures that source and > destination of rename operation are on the same namespace volume. > Patch for 0.94 and trunk is attached for review. The patch modifies the > location of the backup directory from '/hbase-xxx' to > '/hbase/.hbcktmp-xxx' -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6236) Offline meta repair fails if the HBase base mount point is on a different cluster/volume than its parent in a ViewFS or similar FS
[ https://issues.apache.org/jira/browse/HBASE-6236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13399201#comment-13399201 ] Zhihong Ted Yu commented on HBASE-6236: --- All Apache build servers are offline. Please list the failing tests individually so that people can help verify. > Offline meta repair fails if the HBase base mount point is on a different > cluster/volume than its parent in a ViewFS or similar FS > -- > > Key: HBASE-6236 > URL: https://issues.apache.org/jira/browse/HBASE-6236 > Project: HBase > Issue Type: Bug > Components: hbck >Reporter: Aditya Kishore >Assignee: Aditya Kishore > Attachments: 6236_trunk.patch, HBASE-6236_94.patch, > HBASE-6236_trunk.patch, HBASE-6236_trunk.patch > > > While building the .META. and -ROOT- from FS data alone (HBASE-4377), hbck > tries to move the existing .META. and -ROOT- directories to a backup folder. > This backup folder is created at the same level as the base HBase folder > (e.g. /hbase-xx if the base HBase folder is '/hbase'). > In a federated HDFS like ViewFS and other similar FS implementations, it is > not possible to rename files/directories across namespace volumes (ViewFS > guide section 3.5) and as a result hbck crashes. > A solution to this problem is to create the backup directory under the folder > where HBase base folder has been mounted. This ensures that source and > destination of rename operation are on the same namespace volume. > Patch for 0.94 and trunk is attached for review. The patch modifies the > location of the backup directory from '/hbase-xxx' to > '/hbase/.hbcktmp-xxx' -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6236) Offline meta repair fails if the HBase base mount point is on a different cluster/volume than its parent in a ViewFS or similar FS
[ https://issues.apache.org/jira/browse/HBASE-6236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13399197#comment-13399197 ] Aditya Kishore commented on HBASE-6236: --- Strangely, the test suit is randomly failing at different places during different runs on my test machine, even without applying the patch :( > Offline meta repair fails if the HBase base mount point is on a different > cluster/volume than its parent in a ViewFS or similar FS > -- > > Key: HBASE-6236 > URL: https://issues.apache.org/jira/browse/HBASE-6236 > Project: HBase > Issue Type: Bug > Components: hbck >Reporter: Aditya Kishore >Assignee: Aditya Kishore > Attachments: 6236_trunk.patch, HBASE-6236_94.patch, > HBASE-6236_trunk.patch, HBASE-6236_trunk.patch > > > While building the .META. and -ROOT- from FS data alone (HBASE-4377), hbck > tries to move the existing .META. and -ROOT- directories to a backup folder. > This backup folder is created at the same level as the base HBase folder > (e.g. /hbase-xx if the base HBase folder is '/hbase'). > In a federated HDFS like ViewFS and other similar FS implementations, it is > not possible to rename files/directories across namespace volumes (ViewFS > guide section 3.5) and as a result hbck crashes. > A solution to this problem is to create the backup directory under the folder > where HBase base folder has been mounted. This ensures that source and > destination of rename operation are on the same namespace volume. > Patch for 0.94 and trunk is attached for review. The patch modifies the > location of the backup directory from '/hbase-xxx' to > '/hbase/.hbcktmp-xxx' -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6236) Offline meta repair fails if the HBase base mount point is on a different cluster/volume than its parent in a ViewFS or similar FS
[ https://issues.apache.org/jira/browse/HBASE-6236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13399135#comment-13399135 ] Aditya Kishore commented on HBASE-6236: --- I am running the test suit at this moment, will update here once it is done > Offline meta repair fails if the HBase base mount point is on a different > cluster/volume than its parent in a ViewFS or similar FS > -- > > Key: HBASE-6236 > URL: https://issues.apache.org/jira/browse/HBASE-6236 > Project: HBase > Issue Type: Bug > Components: hbck >Reporter: Aditya Kishore >Assignee: Aditya Kishore > Attachments: 6236_trunk.patch, HBASE-6236_94.patch, > HBASE-6236_trunk.patch, HBASE-6236_trunk.patch > > > While building the .META. and -ROOT- from FS data alone (HBASE-4377), hbck > tries to move the existing .META. and -ROOT- directories to a backup folder. > This backup folder is created at the same level as the base HBase folder > (e.g. /hbase-xx if the base HBase folder is '/hbase'). > In a federated HDFS like ViewFS and other similar FS implementations, it is > not possible to rename files/directories across namespace volumes (ViewFS > guide section 3.5) and as a result hbck crashes. > A solution to this problem is to create the backup directory under the folder > where HBase base folder has been mounted. This ensures that source and > destination of rename operation are on the same namespace volume. > Patch for 0.94 and trunk is attached for review. The patch modifies the > location of the backup directory from '/hbase-xxx' to > '/hbase/.hbcktmp-xxx' -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6236) Offline meta repair fails if the HBase base mount point is on a different cluster/volume than its parent in a ViewFS or similar FS
[ https://issues.apache.org/jira/browse/HBASE-6236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13399099#comment-13399099 ] Zhihong Ted Yu commented on HBASE-6236: --- {code} +.println(" -backup HDFS path to backup existing meta and root."); {code} How about giving the new option a name which is more consistent with its usage, such as -sidelineDir ? > Offline meta repair fails if the HBase base mount point is on a different > cluster/volume than its parent in a ViewFS or similar FS > -- > > Key: HBASE-6236 > URL: https://issues.apache.org/jira/browse/HBASE-6236 > Project: HBase > Issue Type: Bug > Components: hbck >Reporter: Aditya Kishore >Assignee: Aditya Kishore > Attachments: 6236_trunk.patch, HBASE-6236_94.patch, > HBASE-6236_trunk.patch > > > While building the .META. and -ROOT- from FS data alone (HBASE-4377), hbck > tries to move the existing .META. and -ROOT- directories to a backup folder. > This backup folder is created at the same level as the base HBase folder > (e.g. /hbase-xx if the base HBase folder is '/hbase'). > In a federated HDFS like ViewFS and other similar FS implementations, it is > not possible to rename files/directories across namespace volumes (ViewFS > guide section 3.5) and as a result hbck crashes. > A solution to this problem is to create the backup directory under the folder > where HBase base folder has been mounted. This ensures that source and > destination of rename operation are on the same namespace volume. > Patch for 0.94 and trunk is attached for review. The patch modifies the > location of the backup directory from '/hbase-xxx' to > '/hbase/.hbcktmp-xxx' -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6236) Offline meta repair fails if the HBase base mount point is on a different cluster/volume than its parent in a ViewFS or similar FS
[ https://issues.apache.org/jira/browse/HBASE-6236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13399095#comment-13399095 ] Zhihong Ted Yu commented on HBASE-6236: --- @Aditya: Hadoop QA is not functioning. Please run test suite locally. @Jonathan H, @Jimmy: Can you take a look at the patch ? Thanks > Offline meta repair fails if the HBase base mount point is on a different > cluster/volume than its parent in a ViewFS or similar FS > -- > > Key: HBASE-6236 > URL: https://issues.apache.org/jira/browse/HBASE-6236 > Project: HBase > Issue Type: Bug > Components: hbck >Reporter: Aditya Kishore >Assignee: Aditya Kishore > Attachments: 6236_trunk.patch, HBASE-6236_94.patch, > HBASE-6236_trunk.patch > > > While building the .META. and -ROOT- from FS data alone (HBASE-4377), hbck > tries to move the existing .META. and -ROOT- directories to a backup folder. > This backup folder is created at the same level as the base HBase folder > (e.g. /hbase-xx if the base HBase folder is '/hbase'). > In a federated HDFS like ViewFS and other similar FS implementations, it is > not possible to rename files/directories across namespace volumes (ViewFS > guide section 3.5) and as a result hbck crashes. > A solution to this problem is to create the backup directory under the folder > where HBase base folder has been mounted. This ensures that source and > destination of rename operation are on the same namespace volume. > Patch for 0.94 and trunk is attached for review. The patch modifies the > location of the backup directory from '/hbase-xxx' to > '/hbase/.hbcktmp-xxx' -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6236) Offline meta repair fails if the HBase base mount point is on a different cluster/volume than its parent in a ViewFS or similar FS
[ https://issues.apache.org/jira/browse/HBASE-6236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13399003#comment-13399003 ] Hadoop QA commented on HBASE-6236: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12532967/HBASE-6236_94.patch against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. -1 patch. The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/2217//console This message is automatically generated. > Offline meta repair fails if the HBase base mount point is on a different > cluster/volume than its parent in a ViewFS or similar FS > -- > > Key: HBASE-6236 > URL: https://issues.apache.org/jira/browse/HBASE-6236 > Project: HBase > Issue Type: Bug > Components: hbck >Reporter: Aditya Kishore >Assignee: Aditya Kishore > Attachments: HBASE-6236_94.patch, HBASE-6236_trunk.patch > > > While building the .META. and -ROOT- from FS data alone (HBASE-4377), hbck > tries to move the existing .META. and -ROOT- directories to a backup folder. > This backup folder is created at the same level as the base HBase folder > (e.g. /hbase-xx if the base HBase folder is '/hbase'). > In a federated HDFS like ViewFS and other similar FS implementations, it is > not possible to rename files/directories across namespace volumes (ViewFS > guide section 3.5) and as a result hbck crashes. > A solution to this problem is to create the backup directory under the folder > where HBase base folder has been mounted. This ensures that source and > destination of rename operation are on the same namespace volume. > Patch for 0.94 and trunk is attached for review. The patch modifies the > location of the backup directory from '/hbase-xxx' to > '/hbase/.hbcktmp-xxx' -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6236) Offline meta repair fails if the HBase base mount point is on a different cluster/volume than its parent in a ViewFS or similar FS
[ https://issues.apache.org/jira/browse/HBASE-6236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13396564#comment-13396564 ] Zhihong Ted Yu commented on HBASE-6236: --- Makes sense. New configuration would increase the possibility of satisfying future use case. > Offline meta repair fails if the HBase base mount point is on a different > cluster/volume than its parent in a ViewFS or similar FS > -- > > Key: HBASE-6236 > URL: https://issues.apache.org/jira/browse/HBASE-6236 > Project: HBase > Issue Type: Bug > Components: hbck >Reporter: Aditya Kishore >Assignee: Aditya Kishore > Attachments: HBASE-6236_0.94.patch, HBASE-6236_trunk.patch > > > While building the .META. and -ROOT- from FS data alone (HBASE-4377), hbck > tries to move the existing .META. and -ROOT- directories to a backup folder. > This backup folder is created at the same level as the base HBase folder > (e.g. /hbase-xx if the base HBase folder is '/hbase'). > In a federated HDFS like ViewFS and other similar FS implementations, it is > not possible to rename files/directories across namespace volumes (ViewFS > guide section 3.5) and as a result hbck crashes. > A solution to this problem is to create the backup directory under the folder > where HBase base folder has been mounted. This ensures that source and > destination of rename operation are on the same namespace volume. > Patch for 0.94 and trunk is attached for review. The patch modifies the > location of the backup directory from '/hbase-xxx' to > '/hbase/.hbcktmp-xxx' -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6236) Offline meta repair fails if the HBase base mount point is on a different cluster/volume than its parent in a ViewFS or similar FS
[ https://issues.apache.org/jira/browse/HBASE-6236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13396531#comment-13396531 ] Aditya Kishore commented on HBASE-6236: --- @[~zhi...@ebaysf.com] Agree that it would be a clean to move the prefix to a constant. However, we would still want to enforce 2 criteria on the backup folder name/location: 1. The folder must be immediate child of HBase base folder to ensure that fs.rename() works, and 2. Prefix starts with a "." to ensure that it does not conflict with or presumed to be a table directory. Instead of making it configurable, how about print a message at the end notifying the user the location of the backup folder? > Offline meta repair fails if the HBase base mount point is on a different > cluster/volume than its parent in a ViewFS or similar FS > -- > > Key: HBASE-6236 > URL: https://issues.apache.org/jira/browse/HBASE-6236 > Project: HBase > Issue Type: Bug > Components: hbck >Reporter: Aditya Kishore >Assignee: Aditya Kishore > Attachments: HBASE-6236_0.94.patch, HBASE-6236_trunk.patch > > > While building the .META. and -ROOT- from FS data alone (HBASE-4377), hbck > tries to move the existing .META. and -ROOT- directories to a backup folder. > This backup folder is created at the same level as the base HBase folder > (e.g. /hbase-xx if the base HBase folder is '/hbase'). > In a federated HDFS like ViewFS and other similar FS implementations, it is > not possible to rename files/directories across namespace volumes (ViewFS > guide section 3.5) and as a result hbck crashes. > A solution to this problem is to create the backup directory under the folder > where HBase base folder has been mounted. This ensures that source and > destination of rename operation are on the same namespace volume. > Patch for 0.94 and trunk is attached for review. The patch modifies the > location of the backup directory from '/hbase-xxx' to > '/hbase/.hbcktmp-xxx' -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6236) Offline meta repair fails if the HBase base mount point is on a different cluster/volume than its parent in a ViewFS or similar FS
[ https://issues.apache.org/jira/browse/HBASE-6236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13396526#comment-13396526 ] Aditya Kishore commented on HBASE-6236: --- @Hadoop QA Build seems to have attempted to apply 0.94 patch to trunk > Offline meta repair fails if the HBase base mount point is on a different > cluster/volume than its parent in a ViewFS or similar FS > -- > > Key: HBASE-6236 > URL: https://issues.apache.org/jira/browse/HBASE-6236 > Project: HBase > Issue Type: Bug > Components: hbck >Reporter: Aditya Kishore >Assignee: Aditya Kishore > Attachments: HBASE-6236_0.94.patch, HBASE-6236_trunk.patch > > > While building the .META. and -ROOT- from FS data alone (HBASE-4377), hbck > tries to move the existing .META. and -ROOT- directories to a backup folder. > This backup folder is created at the same level as the base HBase folder > (e.g. /hbase-xx if the base HBase folder is '/hbase'). > In a federated HDFS like ViewFS and other similar FS implementations, it is > not possible to rename files/directories across namespace volumes (ViewFS > guide section 3.5) and as a result hbck crashes. > A solution to this problem is to create the backup directory under the folder > where HBase base folder has been mounted. This ensures that source and > destination of rename operation are on the same namespace volume. > Patch for 0.94 and trunk is attached for review. The patch modifies the > location of the backup directory from '/hbase-xxx' to > '/hbase/.hbcktmp-xxx' -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6236) Offline meta repair fails if the HBase base mount point is on a different cluster/volume than its parent in a ViewFS or similar FS
[ https://issues.apache.org/jira/browse/HBASE-6236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13396523#comment-13396523 ] Zhihong Ted Yu commented on HBASE-6236: --- {code} +return new Path(hbaseDir, ".hbcktmp" + "-" + startMillis); {code} Introduce some constant and make the above configurable ? > Offline meta repair fails if the HBase base mount point is on a different > cluster/volume than its parent in a ViewFS or similar FS > -- > > Key: HBASE-6236 > URL: https://issues.apache.org/jira/browse/HBASE-6236 > Project: HBase > Issue Type: Bug > Components: hbck >Reporter: Aditya Kishore >Assignee: Aditya Kishore > Attachments: HBASE-6236_0.94.patch, HBASE-6236_trunk.patch > > > While building the .META. and -ROOT- from FS data alone (HBASE-4377), hbck > tries to move the existing .META. and -ROOT- directories to a backup folder. > This backup folder is created at the same level as the base HBase folder > (e.g. /hbase-xx if the base HBase folder is '/hbase'). > In a federated HDFS like ViewFS and other similar FS implementations, it is > not possible to rename files/directories across namespace volumes (ViewFS > guide section 3.5) and as a result hbck crashes. > A solution to this problem is to create the backup directory under the folder > where HBase base folder has been mounted. This ensures that source and > destination of rename operation are on the same namespace volume. > Patch for 0.94 and trunk is attached for review. The patch modifies the > location of the backup directory from '/hbase-xxx' to > '/hbase/.hbcktmp-xxx' -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6236) Offline meta repair fails if the HBase base mount point is on a different cluster/volume than its parent in a ViewFS or similar FS
[ https://issues.apache.org/jira/browse/HBASE-6236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13396522#comment-13396522 ] Hadoop QA commented on HBASE-6236: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12532496/HBASE-6236_0.94.patch against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. -1 patch. The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/2188//console This message is automatically generated. > Offline meta repair fails if the HBase base mount point is on a different > cluster/volume than its parent in a ViewFS or similar FS > -- > > Key: HBASE-6236 > URL: https://issues.apache.org/jira/browse/HBASE-6236 > Project: HBase > Issue Type: Bug > Components: hbck >Reporter: Aditya Kishore >Assignee: Aditya Kishore > Attachments: HBASE-6236_0.94.patch, HBASE-6236_trunk.patch > > > While building the .META. and -ROOT- from FS data alone (HBASE-4377), hbck > tries to move the existing .META. and -ROOT- directories to a backup folder. > This backup folder is created at the same level as the base HBase folder > (e.g. /hbase-xx if the base HBase folder is '/hbase'). > In a federated HDFS like ViewFS and other similar FS implementations, it is > not possible to rename files/directories across namespace volumes (ViewFS > guide section 3.5) and as a result hbck crashes. > A solution to this problem is to create the backup directory under the folder > where HBase base folder has been mounted. This ensures that source and > destination of rename operation are on the same namespace volume. > Patch for 0.94 and trunk is attached for review. The patch modifies the > location of the backup directory from '/hbase-xxx' to > '/hbase/.hbcktmp-xxx' -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira