subject:"\[jira\] \[Commented\] \(HBASE\-6236\) Offline meta repair fails if the HBase base mount point is on a different cluster\/volume than its parent in a ViewFS or similar FS"


[ 
https://issues.apache.org/jira/browse/HBASE-6236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13399197#comment-13399197
 ] 

Aditya Kishore commented on HBASE-6236:
---

Strangely, the test suit is randomly failing at different places during 
different runs on my test machine, even without applying the patch :(

 Offline meta repair fails if the HBase base mount point is on a different 
 cluster/volume than its parent in a ViewFS or similar FS
 --

 Key: HBASE-6236
 URL: https://issues.apache.org/jira/browse/HBASE-6236
 Project: HBase
  Issue Type: Bug
  Components: hbck
Reporter: Aditya Kishore
Assignee: Aditya Kishore
 Attachments: 6236_trunk.patch, HBASE-6236_94.patch, 
 HBASE-6236_trunk.patch, HBASE-6236_trunk.patch


 While building the .META. and -ROOT- from FS data alone (HBASE-4377), hbck 
 tries to move the existing .META. and -ROOT- directories to a backup folder.
 This backup folder is created at the same level as the base HBase folder 
 (e.g. /hbase-xx if the base HBase folder is '/hbase').
 In a federated HDFS like ViewFS and other similar FS implementations, it is 
 not possible to rename files/directories across namespace volumes (ViewFS 
 guide section 3.5) and as a result hbck crashes.
 A solution to this problem is to create the backup directory under the folder 
 where HBase base folder has been mounted. This ensures that source and 
 destination of rename operation are on the same namespace volume.
 Patch for 0.94 and trunk is attached for review. The patch modifies the 
 location of the backup directory from '/hbase-xxx' to 
 '/hbase/.hbcktmp-xxx'

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-6236) Offline meta repair fails if the HBase base mount point is on a different cluster/volume than its parent in a ViewFS or similar FS


[ 
https://issues.apache.org/jira/browse/HBASE-6236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13399201#comment-13399201
 ] 

Zhihong Ted Yu commented on HBASE-6236:
---

All Apache build servers are offline.
Please list the failing tests individually so that people can help verify.

 Offline meta repair fails if the HBase base mount point is on a different 
 cluster/volume than its parent in a ViewFS or similar FS
 --

 Key: HBASE-6236
 URL: https://issues.apache.org/jira/browse/HBASE-6236
 Project: HBase
  Issue Type: Bug
  Components: hbck
Reporter: Aditya Kishore
Assignee: Aditya Kishore
 Attachments: 6236_trunk.patch, HBASE-6236_94.patch, 
 HBASE-6236_trunk.patch, HBASE-6236_trunk.patch


 While building the .META. and -ROOT- from FS data alone (HBASE-4377), hbck 
 tries to move the existing .META. and -ROOT- directories to a backup folder.
 This backup folder is created at the same level as the base HBase folder 
 (e.g. /hbase-xx if the base HBase folder is '/hbase').
 In a federated HDFS like ViewFS and other similar FS implementations, it is 
 not possible to rename files/directories across namespace volumes (ViewFS 
 guide section 3.5) and as a result hbck crashes.
 A solution to this problem is to create the backup directory under the folder 
 where HBase base folder has been mounted. This ensures that source and 
 destination of rename operation are on the same namespace volume.
 Patch for 0.94 and trunk is attached for review. The patch modifies the 
 location of the backup directory from '/hbase-xxx' to 
 '/hbase/.hbcktmp-xxx'

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-6236) Offline meta repair fails if the HBase base mount point is on a different cluster/volume than its parent in a ViewFS or similar FS

2012-06-22 Thread Hadoop QA (JIRA)

[
https://issues.apache.org/jira/browse/HBASE-6236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13399444#comment-13399444
]

Hadoop QA commented on HBASE-6236:
--

-1 overall. Here are the results of testing the latest attachment

http://issues.apache.org/jira/secure/attachment/12532994/HBASE-6236_trunk.patch
against trunk revision .

+1 @author. The patch does not contain any @author tags.

-1 tests included. The patch doesn't appear to include any new or modified
tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.

+1 hadoop2.0. The patch compiles against the hadoop 2.0 profile.

+1 javadoc. The javadoc tool did not generate any warning messages.

+1 javac. The applied patch does not increase the total number of javac
compiler warnings.

-1 findbugs. The patch appears to introduce 11 new Findbugs (version
1.3.9) warnings.

+1 release audit. The applied patch does not increase the total number of
release audit warnings.

-1 core tests. The patch failed these unit tests:
org.apache.hadoop.hbase.regionserver.TestAtomicOperation
org.apache.hadoop.hbase.security.access.TestAccessController
org.apache.hadoop.hbase.util.TestFSUtils

Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build///testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build///artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build///artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build///console

This message is automatically generated.

Offline meta repair fails if the HBase base mount point is on a different
cluster/volume than its parent in a ViewFS or similar FS
--

Key: HBASE-6236
URL: https://issues.apache.org/jira/browse/HBASE-6236
Project: HBase
Issue Type: Bug
Components: hbck
Reporter: Aditya Kishore
Assignee: Aditya Kishore
Attachments: 6236_trunk.patch, HBASE-6236_94.patch,
HBASE-6236_trunk.patch, HBASE-6236_trunk.patch

While building the .META. and -ROOT- from FS data alone (HBASE-4377), hbck
tries to move the existing .META. and -ROOT- directories to a backup folder.
This backup folder is created at the same level as the base HBase folder
(e.g. /hbase-xx if the base HBase folder is '/hbase').
In a federated HDFS like ViewFS and other similar FS implementations, it is
not possible to rename files/directories across namespace volumes (ViewFS
guide section 3.5) and as a result hbck crashes.
A solution to this problem is to create the backup directory under the folder
where HBase base folder has been mounted. This ensures that source and
destination of rename operation are on the same namespace volume.
Patch for 0.94 and trunk is attached for review. The patch modifies the
location of the backup directory from '/hbase-xxx' to
'/hbase/.hbcktmp-xxx'

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-6236) Offline meta repair fails if the HBase base mount point is on a different cluster/volume than its parent in a ViewFS or similar FS


[ 
https://issues.apache.org/jira/browse/HBASE-6236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13399450#comment-13399450
 ] 

Zhihong Ted Yu commented on HBASE-6236:
---

The failed tests are unrelated to the patch.
I ran them and they passed.

 Offline meta repair fails if the HBase base mount point is on a different 
 cluster/volume than its parent in a ViewFS or similar FS
 --

 Key: HBASE-6236
 URL: https://issues.apache.org/jira/browse/HBASE-6236
 Project: HBase
  Issue Type: Bug
  Components: hbck
Reporter: Aditya Kishore
Assignee: Aditya Kishore
 Attachments: 6236_trunk.patch, HBASE-6236_94.patch, 
 HBASE-6236_trunk.patch, HBASE-6236_trunk.patch


 While building the .META. and -ROOT- from FS data alone (HBASE-4377), hbck 
 tries to move the existing .META. and -ROOT- directories to a backup folder.
 This backup folder is created at the same level as the base HBase folder 
 (e.g. /hbase-xx if the base HBase folder is '/hbase').
 In a federated HDFS like ViewFS and other similar FS implementations, it is 
 not possible to rename files/directories across namespace volumes (ViewFS 
 guide section 3.5) and as a result hbck crashes.
 A solution to this problem is to create the backup directory under the folder 
 where HBase base folder has been mounted. This ensures that source and 
 destination of rename operation are on the same namespace volume.
 Patch for 0.94 and trunk is attached for review. The patch modifies the 
 location of the backup directory from '/hbase-xxx' to 
 '/hbase/.hbcktmp-xxx'

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-6236) Offline meta repair fails if the HBase base mount point is on a different cluster/volume than its parent in a ViewFS or similar FS


[ 
https://issues.apache.org/jira/browse/HBASE-6236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13399462#comment-13399462
 ] 

Aditya Kishore commented on HBASE-6236:
---

Even though I am running into some random test case failures, all of these 
HBCK/HBFsck tests have been passing regularly

org.apache.hadoop.hbase.util.hbck.TestOfflineMetaRebuildBase
org.apache.hadoop.hbase.util.hbck.TestOfflineMetaRebuildHole
org.apache.hadoop.hbase.util.hbck.TestOfflineMetaRebuildOverlap
org.apache.hadoop.hbase.util.TestHBaseFsck
org.apache.hadoop.hbase.util.TestHBaseFsckComparator

 Offline meta repair fails if the HBase base mount point is on a different 
 cluster/volume than its parent in a ViewFS or similar FS
 --

 Key: HBASE-6236
 URL: https://issues.apache.org/jira/browse/HBASE-6236
 Project: HBase
  Issue Type: Bug
  Components: hbck
Reporter: Aditya Kishore
Assignee: Aditya Kishore
 Attachments: 6236_trunk.patch, HBASE-6236_94.patch, 
 HBASE-6236_trunk.patch, HBASE-6236_trunk.patch


 While building the .META. and -ROOT- from FS data alone (HBASE-4377), hbck 
 tries to move the existing .META. and -ROOT- directories to a backup folder.
 This backup folder is created at the same level as the base HBase folder 
 (e.g. /hbase-xx if the base HBase folder is '/hbase').
 In a federated HDFS like ViewFS and other similar FS implementations, it is 
 not possible to rename files/directories across namespace volumes (ViewFS 
 guide section 3.5) and as a result hbck crashes.
 A solution to this problem is to create the backup directory under the folder 
 where HBase base folder has been mounted. This ensures that source and 
 destination of rename operation are on the same namespace volume.
 Patch for 0.94 and trunk is attached for review. The patch modifies the 
 location of the backup directory from '/hbase-xxx' to 
 '/hbase/.hbcktmp-xxx'

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-6236) Offline meta repair fails if the HBase base mount point is on a different cluster/volume than its parent in a ViewFS or similar FS


[ 
https://issues.apache.org/jira/browse/HBASE-6236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13399543#comment-13399543
 ] 

Zhihong Ted Yu commented on HBASE-6236:
---

Patch integrated to trunk.

 Offline meta repair fails if the HBase base mount point is on a different 
 cluster/volume than its parent in a ViewFS or similar FS
 --

 Key: HBASE-6236
 URL: https://issues.apache.org/jira/browse/HBASE-6236
 Project: HBase
  Issue Type: Bug
  Components: hbck
Reporter: Aditya Kishore
Assignee: Aditya Kishore
 Fix For: 0.96.0, 0.94.1

 Attachments: 6236_trunk.patch, HBASE-6236_94.patch, 
 HBASE-6236_trunk.patch, HBASE-6236_trunk.patch


 While building the .META. and -ROOT- from FS data alone (HBASE-4377), hbck 
 tries to move the existing .META. and -ROOT- directories to a backup folder.
 This backup folder is created at the same level as the base HBase folder 
 (e.g. /hbase-xx if the base HBase folder is '/hbase').
 In a federated HDFS like ViewFS and other similar FS implementations, it is 
 not possible to rename files/directories across namespace volumes (ViewFS 
 guide section 3.5) and as a result hbck crashes.
 A solution to this problem is to create the backup directory under the folder 
 where HBase base folder has been mounted. This ensures that source and 
 destination of rename operation are on the same namespace volume.
 Patch for 0.94 and trunk is attached for review. The patch modifies the 
 location of the backup directory from '/hbase-xxx' to 
 '/hbase/.hbcktmp-xxx'

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-6236) Offline meta repair fails if the HBase base mount point is on a different cluster/volume than its parent in a ViewFS or similar FS


[ 
https://issues.apache.org/jira/browse/HBASE-6236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13399544#comment-13399544
 ] 

Jimmy Xiang commented on HBASE-6236:


hbck sideline the dir to some folder like /hbck/hbase-xx, not /hbase-x
In a federated HDFS, you can pre-create a folder like /hbck, which is for this 
kind of use cases.

 Offline meta repair fails if the HBase base mount point is on a different 
 cluster/volume than its parent in a ViewFS or similar FS
 --

 Key: HBASE-6236
 URL: https://issues.apache.org/jira/browse/HBASE-6236
 Project: HBase
  Issue Type: Bug
  Components: hbck
Reporter: Aditya Kishore
Assignee: Aditya Kishore
 Fix For: 0.96.0, 0.94.1

 Attachments: 6236_trunk.patch, HBASE-6236_94.patch, 
 HBASE-6236_trunk.patch, HBASE-6236_trunk.patch


 While building the .META. and -ROOT- from FS data alone (HBASE-4377), hbck 
 tries to move the existing .META. and -ROOT- directories to a backup folder.
 This backup folder is created at the same level as the base HBase folder 
 (e.g. /hbase-xx if the base HBase folder is '/hbase').
 In a federated HDFS like ViewFS and other similar FS implementations, it is 
 not possible to rename files/directories across namespace volumes (ViewFS 
 guide section 3.5) and as a result hbck crashes.
 A solution to this problem is to create the backup directory under the folder 
 where HBase base folder has been mounted. This ensures that source and 
 destination of rename operation are on the same namespace volume.
 Patch for 0.94 and trunk is attached for review. The patch modifies the 
 location of the backup directory from '/hbase-xxx' to 
 '/hbase/.hbcktmp-xxx'

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-6236) Offline meta repair fails if the HBase base mount point is on a different cluster/volume than its parent in a ViewFS or similar FS


[ 
https://issues.apache.org/jira/browse/HBASE-6236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13399546#comment-13399546
 ] 

Jimmy Xiang commented on HBASE-6236:


We should revert this change.

 Offline meta repair fails if the HBase base mount point is on a different 
 cluster/volume than its parent in a ViewFS or similar FS
 --

 Key: HBASE-6236
 URL: https://issues.apache.org/jira/browse/HBASE-6236
 Project: HBase
  Issue Type: Bug
  Components: hbck
Reporter: Aditya Kishore
Assignee: Aditya Kishore
 Fix For: 0.96.0, 0.94.1

 Attachments: 6236_trunk.patch, HBASE-6236_94.patch, 
 HBASE-6236_trunk.patch, HBASE-6236_trunk.patch


 While building the .META. and -ROOT- from FS data alone (HBASE-4377), hbck 
 tries to move the existing .META. and -ROOT- directories to a backup folder.
 This backup folder is created at the same level as the base HBase folder 
 (e.g. /hbase-xx if the base HBase folder is '/hbase').
 In a federated HDFS like ViewFS and other similar FS implementations, it is 
 not possible to rename files/directories across namespace volumes (ViewFS 
 guide section 3.5) and as a result hbck crashes.
 A solution to this problem is to create the backup directory under the folder 
 where HBase base folder has been mounted. This ensures that source and 
 destination of rename operation are on the same namespace volume.
 Patch for 0.94 and trunk is attached for review. The patch modifies the 
 location of the backup directory from '/hbase-xxx' to 
 '/hbase/.hbcktmp-xxx'

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-6236) Offline meta repair fails if the HBase base mount point is on a different cluster/volume than its parent in a ViewFS or similar FS


[ 
https://issues.apache.org/jira/browse/HBASE-6236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13399551#comment-13399551
 ] 

Zhihong Ted Yu commented on HBASE-6236:
---

Integrated to 0.94 as well.

Thanks for the patch, Aditya.

 Offline meta repair fails if the HBase base mount point is on a different 
 cluster/volume than its parent in a ViewFS or similar FS
 --

 Key: HBASE-6236
 URL: https://issues.apache.org/jira/browse/HBASE-6236
 Project: HBase
  Issue Type: Bug
  Components: hbck
Reporter: Aditya Kishore
Assignee: Aditya Kishore
 Fix For: 0.96.0, 0.94.1

 Attachments: 6236_trunk.patch, HBASE-6236_94.patch, 
 HBASE-6236_trunk.patch, HBASE-6236_trunk.patch


 While building the .META. and -ROOT- from FS data alone (HBASE-4377), hbck 
 tries to move the existing .META. and -ROOT- directories to a backup folder.
 This backup folder is created at the same level as the base HBase folder 
 (e.g. /hbase-xx if the base HBase folder is '/hbase').
 In a federated HDFS like ViewFS and other similar FS implementations, it is 
 not possible to rename files/directories across namespace volumes (ViewFS 
 guide section 3.5) and as a result hbck crashes.
 A solution to this problem is to create the backup directory under the folder 
 where HBase base folder has been mounted. This ensures that source and 
 destination of rename operation are on the same namespace volume.
 Patch for 0.94 and trunk is attached for review. The patch modifies the 
 location of the backup directory from '/hbase-xxx' to 
 '/hbase/.hbcktmp-xxx'

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-6236) Offline meta repair fails if the HBase base mount point is on a different cluster/volume than its parent in a ViewFS or similar FS


[ 
https://issues.apache.org/jira/browse/HBASE-6236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13399557#comment-13399557
 ] 

Zhihong Ted Yu commented on HBASE-6236:
---

Saw Jimmy's comment and reverted the patch.

 Offline meta repair fails if the HBase base mount point is on a different 
 cluster/volume than its parent in a ViewFS or similar FS
 --

 Key: HBASE-6236
 URL: https://issues.apache.org/jira/browse/HBASE-6236
 Project: HBase
  Issue Type: Bug
  Components: hbck
Reporter: Aditya Kishore
Assignee: Aditya Kishore
 Fix For: 0.96.0, 0.94.1

 Attachments: 6236_trunk.patch, HBASE-6236_94.patch, 
 HBASE-6236_trunk.patch, HBASE-6236_trunk.patch


 While building the .META. and -ROOT- from FS data alone (HBASE-4377), hbck 
 tries to move the existing .META. and -ROOT- directories to a backup folder.
 This backup folder is created at the same level as the base HBase folder 
 (e.g. /hbase-xx if the base HBase folder is '/hbase').
 In a federated HDFS like ViewFS and other similar FS implementations, it is 
 not possible to rename files/directories across namespace volumes (ViewFS 
 guide section 3.5) and as a result hbck crashes.
 A solution to this problem is to create the backup directory under the folder 
 where HBase base folder has been mounted. This ensures that source and 
 destination of rename operation are on the same namespace volume.
 Patch for 0.94 and trunk is attached for review. The patch modifies the 
 location of the backup directory from '/hbase-xxx' to 
 '/hbase/.hbcktmp-xxx'

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-6236) Offline meta repair fails if the HBase base mount point is on a different cluster/volume than its parent in a ViewFS or similar FS


[ 
https://issues.apache.org/jira/browse/HBASE-6236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13399562#comment-13399562
 ] 

Jimmy Xiang commented on HBASE-6236:


It is a good idea to have sideline dir option.  I think we should keep the 
existing default folder name.

 Offline meta repair fails if the HBase base mount point is on a different 
 cluster/volume than its parent in a ViewFS or similar FS
 --

 Key: HBASE-6236
 URL: https://issues.apache.org/jira/browse/HBASE-6236
 Project: HBase
  Issue Type: Bug
  Components: hbck
Reporter: Aditya Kishore
Assignee: Aditya Kishore
 Fix For: 0.96.0, 0.94.1

 Attachments: 6236_trunk.patch, HBASE-6236_94.patch, 
 HBASE-6236_trunk.patch, HBASE-6236_trunk.patch


 While building the .META. and -ROOT- from FS data alone (HBASE-4377), hbck 
 tries to move the existing .META. and -ROOT- directories to a backup folder.
 This backup folder is created at the same level as the base HBase folder 
 (e.g. /hbase-xx if the base HBase folder is '/hbase').
 In a federated HDFS like ViewFS and other similar FS implementations, it is 
 not possible to rename files/directories across namespace volumes (ViewFS 
 guide section 3.5) and as a result hbck crashes.
 A solution to this problem is to create the backup directory under the folder 
 where HBase base folder has been mounted. This ensures that source and 
 destination of rename operation are on the same namespace volume.
 Patch for 0.94 and trunk is attached for review. The patch modifies the 
 location of the backup directory from '/hbase-xxx' to 
 '/hbase/.hbcktmp-xxx'

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-6236) Offline meta repair fails if the HBase base mount point is on a different cluster/volume than its parent in a ViewFS or similar FS


[ 
https://issues.apache.org/jira/browse/HBASE-6236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13399603#comment-13399603
 ] 

Hudson commented on HBASE-6236:
---

Integrated in HBase-0.94 #272 (See 
[https://builds.apache.org/job/HBase-0.94/272/])
HBASE-6236 revert upon Jimmy's request. (Revision 1353014)
HBASE-6236 Offline meta repair fails if the HBase base mount point is on a 
different cluster/volume than its parent in a ViewFS or similar FS (Aditya) 
(Revision 1353012)

 Result = FAILURE
tedyu : 
Files : 
* /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java
* 
/hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/util/hbck/OfflineMetaRepair.java

tedyu : 
Files : 
* /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java
* 
/hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/util/hbck/OfflineMetaRepair.java


 Offline meta repair fails if the HBase base mount point is on a different 
 cluster/volume than its parent in a ViewFS or similar FS
 --

 Key: HBASE-6236
 URL: https://issues.apache.org/jira/browse/HBASE-6236
 Project: HBase
  Issue Type: Bug
  Components: hbck
Reporter: Aditya Kishore
Assignee: Aditya Kishore
 Fix For: 0.96.0, 0.94.1

 Attachments: 6236_trunk.patch, HBASE-6236_94.patch, 
 HBASE-6236_trunk.patch, HBASE-6236_trunk.patch


 While building the .META. and -ROOT- from FS data alone (HBASE-4377), hbck 
 tries to move the existing .META. and -ROOT- directories to a backup folder.
 This backup folder is created at the same level as the base HBase folder 
 (e.g. /hbase-xx if the base HBase folder is '/hbase').
 In a federated HDFS like ViewFS and other similar FS implementations, it is 
 not possible to rename files/directories across namespace volumes (ViewFS 
 guide section 3.5) and as a result hbck crashes.
 A solution to this problem is to create the backup directory under the folder 
 where HBase base folder has been mounted. This ensures that source and 
 destination of rename operation are on the same namespace volume.
 Patch for 0.94 and trunk is attached for review. The patch modifies the 
 location of the backup directory from '/hbase-xxx' to 
 '/hbase/.hbcktmp-xxx'

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-6236) Offline meta repair fails if the HBase base mount point is on a different cluster/volume than its parent in a ViewFS or similar FS


[ 
https://issues.apache.org/jira/browse/HBASE-6236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13399623#comment-13399623
 ] 

Hudson commented on HBASE-6236:
---

Integrated in HBase-TRUNK #3060 (See 
[https://builds.apache.org/job/HBase-TRUNK/3060/])
HBASE-6236 revert upon Jimmy's request. (Revision 1353013)
HBASE-6236 Offline meta repair fails if the HBase base mount point is on a 
different cluster/volume than its parent in a ViewFS or similar FS (Aditya) 
(Revision 1353008)

 Result = FAILURE
tedyu : 
Files : 
* 
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java
* 
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/util/hbck/OfflineMetaRepair.java

tedyu : 
Files : 
* 
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java
* 
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/util/hbck/OfflineMetaRepair.java


 Offline meta repair fails if the HBase base mount point is on a different 
 cluster/volume than its parent in a ViewFS or similar FS
 --

 Key: HBASE-6236
 URL: https://issues.apache.org/jira/browse/HBASE-6236
 Project: HBase
  Issue Type: Bug
  Components: hbck
Reporter: Aditya Kishore
Assignee: Aditya Kishore
 Fix For: 0.96.0, 0.94.1

 Attachments: 6236_trunk.patch, HBASE-6236_94.patch, 
 HBASE-6236_trunk.patch, HBASE-6236_trunk.patch


 While building the .META. and -ROOT- from FS data alone (HBASE-4377), hbck 
 tries to move the existing .META. and -ROOT- directories to a backup folder.
 This backup folder is created at the same level as the base HBase folder 
 (e.g. /hbase-xx if the base HBase folder is '/hbase').
 In a federated HDFS like ViewFS and other similar FS implementations, it is 
 not possible to rename files/directories across namespace volumes (ViewFS 
 guide section 3.5) and as a result hbck crashes.
 A solution to this problem is to create the backup directory under the folder 
 where HBase base folder has been mounted. This ensures that source and 
 destination of rename operation are on the same namespace volume.
 Patch for 0.94 and trunk is attached for review. The patch modifies the 
 location of the backup directory from '/hbase-xxx' to 
 '/hbase/.hbcktmp-xxx'

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-6236) Offline meta repair fails if the HBase base mount point is on a different cluster/volume than its parent in a ViewFS or similar FS


[ 
https://issues.apache.org/jira/browse/HBASE-6236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13399628#comment-13399628
 ] 

Aditya Kishore commented on HBASE-6236:
---

@Jimmy

Currently there are two different locations where hbck sidelines the backup 
data.

First one takes the form '/hbck/hbase-xx' and is used to sideline regions

{code:title=HBaseFsck.java}
857   private Path getSidelineDir() throws IOException {
858 Path hbaseDir = FSUtils.getRootDir(conf);
859 Path hbckDir = new Path(hbaseDir.getParent(), hbck);
860 Path backupDir = new Path(hbckDir, hbaseDir.getName() + -
861 + startMillis);
862 return backupDir;
863   }
{code}

While the second one takes the form of '/hbase-x' and is used to sideline 
the current .META. and \-ROOT\- folders

{code:title=HBaseFsck.java}
960 Path backupDir = new Path(hbaseDir.getParent(), hbaseDir.getName() 
+ -
961 + startMillis);
{code}

At lease we need to make them standard.

Secondly, I am not sure how pre-creating /hbck will solve this problem in a 
federated HDFS in all cases. Suppose the root of a cluster is mounted as 
/hbase in ViewFS then the pre created /hbck folder is going to be on a 
different namespace volume and renaming of a file from /hbase/ to /hbck/ is 
going to fail.

 Offline meta repair fails if the HBase base mount point is on a different 
 cluster/volume than its parent in a ViewFS or similar FS
 --

 Key: HBASE-6236
 URL: https://issues.apache.org/jira/browse/HBASE-6236
 Project: HBase
  Issue Type: Bug
  Components: hbck
Reporter: Aditya Kishore
Assignee: Aditya Kishore
 Fix For: 0.96.0, 0.94.1

 Attachments: 6236_trunk.patch, HBASE-6236_94.patch, 
 HBASE-6236_trunk.patch, HBASE-6236_trunk.patch


 While building the .META. and -ROOT- from FS data alone (HBASE-4377), hbck 
 tries to move the existing .META. and -ROOT- directories to a backup folder.
 This backup folder is created at the same level as the base HBase folder 
 (e.g. /hbase-xx if the base HBase folder is '/hbase').
 In a federated HDFS like ViewFS and other similar FS implementations, it is 
 not possible to rename files/directories across namespace volumes (ViewFS 
 guide section 3.5) and as a result hbck crashes.
 A solution to this problem is to create the backup directory under the folder 
 where HBase base folder has been mounted. This ensures that source and 
 destination of rename operation are on the same namespace volume.
 Patch for 0.94 and trunk is attached for review. The patch modifies the 
 location of the backup directory from '/hbase-xxx' to 
 '/hbase/.hbcktmp-xxx'

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-6236) Offline meta repair fails if the HBase base mount point is on a different cluster/volume than its parent in a ViewFS or similar FS


[ 
https://issues.apache.org/jira/browse/HBASE-6236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13399634#comment-13399634
 ] 

Jimmy Xiang commented on HBASE-6236:


@Aditya, good catch.  Please make sure they are the same.
Can you use /hbase/.hbck as the default?

 Offline meta repair fails if the HBase base mount point is on a different 
 cluster/volume than its parent in a ViewFS or similar FS
 --

 Key: HBASE-6236
 URL: https://issues.apache.org/jira/browse/HBASE-6236
 Project: HBase
  Issue Type: Bug
  Components: hbck
Reporter: Aditya Kishore
Assignee: Aditya Kishore
 Fix For: 0.96.0, 0.94.1

 Attachments: 6236_trunk.patch, HBASE-6236_94.patch, 
 HBASE-6236_trunk.patch, HBASE-6236_trunk.patch


 While building the .META. and -ROOT- from FS data alone (HBASE-4377), hbck 
 tries to move the existing .META. and -ROOT- directories to a backup folder.
 This backup folder is created at the same level as the base HBase folder 
 (e.g. /hbase-xx if the base HBase folder is '/hbase').
 In a federated HDFS like ViewFS and other similar FS implementations, it is 
 not possible to rename files/directories across namespace volumes (ViewFS 
 guide section 3.5) and as a result hbck crashes.
 A solution to this problem is to create the backup directory under the folder 
 where HBase base folder has been mounted. This ensures that source and 
 destination of rename operation are on the same namespace volume.
 Patch for 0.94 and trunk is attached for review. The patch modifies the 
 location of the backup directory from '/hbase-xxx' to 
 '/hbase/.hbcktmp-xxx'

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-6236) Offline meta repair fails if the HBase base mount point is on a different cluster/volume than its parent in a ViewFS or similar FS


[ 
https://issues.apache.org/jira/browse/HBASE-6236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13399646#comment-13399646
 ] 

Aditya Kishore commented on HBASE-6236:
---

@Jimmy

To be sure, you want the folders to take the form of 
/hbase/.hbck/hbase-xx, right? 

This way they won't clutter the /hbase/ when many of them get created during.

 Offline meta repair fails if the HBase base mount point is on a different 
 cluster/volume than its parent in a ViewFS or similar FS
 --

 Key: HBASE-6236
 URL: https://issues.apache.org/jira/browse/HBASE-6236
 Project: HBase
  Issue Type: Bug
  Components: hbck
Reporter: Aditya Kishore
Assignee: Aditya Kishore
 Fix For: 0.96.0, 0.94.1

 Attachments: 6236_trunk.patch, HBASE-6236_94.patch, 
 HBASE-6236_trunk.patch, HBASE-6236_trunk.patch


 While building the .META. and -ROOT- from FS data alone (HBASE-4377), hbck 
 tries to move the existing .META. and -ROOT- directories to a backup folder.
 This backup folder is created at the same level as the base HBase folder 
 (e.g. /hbase-xx if the base HBase folder is '/hbase').
 In a federated HDFS like ViewFS and other similar FS implementations, it is 
 not possible to rename files/directories across namespace volumes (ViewFS 
 guide section 3.5) and as a result hbck crashes.
 A solution to this problem is to create the backup directory under the folder 
 where HBase base folder has been mounted. This ensures that source and 
 destination of rename operation are on the same namespace volume.
 Patch for 0.94 and trunk is attached for review. The patch modifies the 
 location of the backup directory from '/hbase-xxx' to 
 '/hbase/.hbcktmp-xxx'

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-6236) Offline meta repair fails if the HBase base mount point is on a different cluster/volume than its parent in a ViewFS or similar FS


[ 
https://issues.apache.org/jira/browse/HBASE-6236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13399652#comment-13399652
 ] 

Jimmy Xiang commented on HBASE-6236:


That's right.  I prefer /hbase/.hbck/hbase-xx as a default, with option to 
change it.

 Offline meta repair fails if the HBase base mount point is on a different 
 cluster/volume than its parent in a ViewFS or similar FS
 --

 Key: HBASE-6236
 URL: https://issues.apache.org/jira/browse/HBASE-6236
 Project: HBase
  Issue Type: Bug
  Components: hbck
Reporter: Aditya Kishore
Assignee: Aditya Kishore
 Fix For: 0.96.0, 0.94.1

 Attachments: 6236_trunk.patch, HBASE-6236_94.patch, 
 HBASE-6236_trunk.patch, HBASE-6236_trunk.patch


 While building the .META. and -ROOT- from FS data alone (HBASE-4377), hbck 
 tries to move the existing .META. and -ROOT- directories to a backup folder.
 This backup folder is created at the same level as the base HBase folder 
 (e.g. /hbase-xx if the base HBase folder is '/hbase').
 In a federated HDFS like ViewFS and other similar FS implementations, it is 
 not possible to rename files/directories across namespace volumes (ViewFS 
 guide section 3.5) and as a result hbck crashes.
 A solution to this problem is to create the backup directory under the folder 
 where HBase base folder has been mounted. This ensures that source and 
 destination of rename operation are on the same namespace volume.
 Patch for 0.94 and trunk is attached for review. The patch modifies the 
 location of the backup directory from '/hbase-xxx' to 
 '/hbase/.hbcktmp-xxx'

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-6236) Offline meta repair fails if the HBase base mount point is on a different cluster/volume than its parent in a ViewFS or similar FS


[ 
https://issues.apache.org/jira/browse/HBASE-6236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13399659#comment-13399659
 ] 

Aditya Kishore commented on HBASE-6236:
---

And it might be a good idea to add .hbck to the list 
org.apache.hadoop.hbase.HConstants.HBASE_NON_USER_TABLE_DIRS

along with .logs, .oldlogs, .corrupt, etc.

 Offline meta repair fails if the HBase base mount point is on a different 
 cluster/volume than its parent in a ViewFS or similar FS
 --

 Key: HBASE-6236
 URL: https://issues.apache.org/jira/browse/HBASE-6236
 Project: HBase
  Issue Type: Bug
  Components: hbck
Reporter: Aditya Kishore
Assignee: Aditya Kishore
 Fix For: 0.96.0, 0.94.1

 Attachments: 6236_trunk.patch, HBASE-6236_94.patch, 
 HBASE-6236_trunk.patch, HBASE-6236_trunk.patch


 While building the .META. and -ROOT- from FS data alone (HBASE-4377), hbck 
 tries to move the existing .META. and -ROOT- directories to a backup folder.
 This backup folder is created at the same level as the base HBase folder 
 (e.g. /hbase-xx if the base HBase folder is '/hbase').
 In a federated HDFS like ViewFS and other similar FS implementations, it is 
 not possible to rename files/directories across namespace volumes (ViewFS 
 guide section 3.5) and as a result hbck crashes.
 A solution to this problem is to create the backup directory under the folder 
 where HBase base folder has been mounted. This ensures that source and 
 destination of rename operation are on the same namespace volume.
 Patch for 0.94 and trunk is attached for review. The patch modifies the 
 location of the backup directory from '/hbase-xxx' to 
 '/hbase/.hbcktmp-xxx'

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-6236) Offline meta repair fails if the HBase base mount point is on a different cluster/volume than its parent in a ViewFS or similar FS

2012-06-22 Thread Jonathan Hsieh (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-6236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13399690#comment-13399690
 ] 

Jonathan Hsieh commented on HBASE-6236:
---

@Aditya

I only looked at the 21/Jun/12 22:49 version -- assuming others have a similar 
problem.

There is a possible array out of bounds exception here.  Can we check to make 
sure we are not out of bounds and provide an appropriate warning message and 
bail out if we are?

{code}
conf.set(HConstants.HBASE_DIR, path);
 conf.set(fs.defaultFS, conf.get(HConstants.HBASE_DIR));
 conf.set(fs.default.name, conf.get(HConstants.HBASE_DIR));
+  } else if (cmd.equals(-sidelineDir)) {
+i++;
+fsck.setSidelineDir(args[i]);
   } else if (cmd.equals(-fixHoles)) {
 fixHoles = true;
   } else if (cmd.equals(-fix)) {
{code}

 Offline meta repair fails if the HBase base mount point is on a different 
 cluster/volume than its parent in a ViewFS or similar FS
 --

 Key: HBASE-6236
 URL: https://issues.apache.org/jira/browse/HBASE-6236
 Project: HBase
  Issue Type: Bug
  Components: hbck
Reporter: Aditya Kishore
Assignee: Aditya Kishore
 Fix For: 0.96.0, 0.94.1

 Attachments: 6236_trunk.patch, HBASE-6236_94.patch, 
 HBASE-6236_trunk.patch, HBASE-6236_trunk.patch


 While building the .META. and -ROOT- from FS data alone (HBASE-4377), hbck 
 tries to move the existing .META. and -ROOT- directories to a backup folder.
 This backup folder is created at the same level as the base HBase folder 
 (e.g. /hbase-xx if the base HBase folder is '/hbase').
 In a federated HDFS like ViewFS and other similar FS implementations, it is 
 not possible to rename files/directories across namespace volumes (ViewFS 
 guide section 3.5) and as a result hbck crashes.
 A solution to this problem is to create the backup directory under the folder 
 where HBase base folder has been mounted. This ensures that source and 
 destination of rename operation are on the same namespace volume.
 Patch for 0.94 and trunk is attached for review. The patch modifies the 
 location of the backup directory from '/hbase-xxx' to 
 '/hbase/.hbcktmp-xxx'

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-6236) Offline meta repair fails if the HBase base mount point is on a different cluster/volume than its parent in a ViewFS or similar FS


[ 
https://issues.apache.org/jira/browse/HBASE-6236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13399693#comment-13399693
 ] 

Zhihong Ted Yu commented on HBASE-6236:
---

I think Aditya was following existing convention:
{code}
  } else if (cmd.equals(-base)) {
// update hbase root dir to user-specified base
i++;
String path = args[i];
{code}

 Offline meta repair fails if the HBase base mount point is on a different 
 cluster/volume than its parent in a ViewFS or similar FS
 --

 Key: HBASE-6236
 URL: https://issues.apache.org/jira/browse/HBASE-6236
 Project: HBase
  Issue Type: Bug
  Components: hbck
Reporter: Aditya Kishore
Assignee: Aditya Kishore
 Fix For: 0.96.0, 0.94.1

 Attachments: 6236_trunk.patch, HBASE-6236_94.patch, 
 HBASE-6236_trunk.patch, HBASE-6236_trunk.patch


 While building the .META. and -ROOT- from FS data alone (HBASE-4377), hbck 
 tries to move the existing .META. and -ROOT- directories to a backup folder.
 This backup folder is created at the same level as the base HBase folder 
 (e.g. /hbase-xx if the base HBase folder is '/hbase').
 In a federated HDFS like ViewFS and other similar FS implementations, it is 
 not possible to rename files/directories across namespace volumes (ViewFS 
 guide section 3.5) and as a result hbck crashes.
 A solution to this problem is to create the backup directory under the folder 
 where HBase base folder has been mounted. This ensures that source and 
 destination of rename operation are on the same namespace volume.
 Patch for 0.94 and trunk is attached for review. The patch modifies the 
 location of the backup directory from '/hbase-xxx' to 
 '/hbase/.hbcktmp-xxx'

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-6236) Offline meta repair fails if the HBase base mount point is on a different cluster/volume than its parent in a ViewFS or similar FS

[
https://issues.apache.org/jira/browse/HBASE-6236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13399694#comment-13399694
]

Aditya Kishore commented on HBASE-6236:
---

@Jonathan:

I noticed that too but wasn't sure whether to combine that issue with the
current one. I guess there is no harm in fixing that while I am touching this
part of code. Will include it in the patch.

Offline meta repair fails if the HBase base mount point is on a different
cluster/volume than its parent in a ViewFS or similar FS
--

Key: HBASE-6236
URL: https://issues.apache.org/jira/browse/HBASE-6236
Project: HBase
Issue Type: Bug
Components: hbck
Reporter: Aditya Kishore
Assignee: Aditya Kishore
Fix For: 0.96.0, 0.94.1

Attachments: 6236_trunk.patch, HBASE-6236_94.patch,
HBASE-6236_trunk.patch, HBASE-6236_trunk.patch

[jira] [Commented] (HBASE-6236) Offline meta repair fails if the HBase base mount point is on a different cluster/volume than its parent in a ViewFS or similar FS


[ 
https://issues.apache.org/jira/browse/HBASE-6236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13399695#comment-13399695
 ] 

Aditya Kishore commented on HBASE-6236:
---

@Ted,

Yes, I was :)

 Offline meta repair fails if the HBase base mount point is on a different 
 cluster/volume than its parent in a ViewFS or similar FS
 --

 Key: HBASE-6236
 URL: https://issues.apache.org/jira/browse/HBASE-6236
 Project: HBase
  Issue Type: Bug
  Components: hbck
Reporter: Aditya Kishore
Assignee: Aditya Kishore
 Fix For: 0.96.0, 0.94.1

 Attachments: 6236_trunk.patch, HBASE-6236_94.patch, 
 HBASE-6236_trunk.patch, HBASE-6236_trunk.patch


 While building the .META. and -ROOT- from FS data alone (HBASE-4377), hbck 
 tries to move the existing .META. and -ROOT- directories to a backup folder.
 This backup folder is created at the same level as the base HBase folder 
 (e.g. /hbase-xx if the base HBase folder is '/hbase').
 In a federated HDFS like ViewFS and other similar FS implementations, it is 
 not possible to rename files/directories across namespace volumes (ViewFS 
 guide section 3.5) and as a result hbck crashes.
 A solution to this problem is to create the backup directory under the folder 
 where HBase base folder has been mounted. This ensures that source and 
 destination of rename operation are on the same namespace volume.
 Patch for 0.94 and trunk is attached for review. The patch modifies the 
 location of the backup directory from '/hbase-xxx' to 
 '/hbase/.hbcktmp-xxx'

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-6236) Offline meta repair fails if the HBase base mount point is on a different cluster/volume than its parent in a ViewFS or similar FS

2012-06-22 Thread Jonathan Hsieh (JIRA)

[
https://issues.apache.org/jira/browse/HBASE-6236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13399703#comment-13399703
]

Jonathan Hsieh commented on HBASE-6236:
---

@Aditya Go ahead and fix it in this instance. If there are other instances
that have a similar problem, file and fix them in a follow-up jira?

I probably wrote the exemplar which has the bug in the first place. :)

Offline meta repair fails if the HBase base mount point is on a different
cluster/volume than its parent in a ViewFS or similar FS
--

Key: HBASE-6236
URL: https://issues.apache.org/jira/browse/HBASE-6236
Project: HBase
Issue Type: Bug
Components: hbck
Reporter: Aditya Kishore
Assignee: Aditya Kishore
Fix For: 0.96.0, 0.94.1

Attachments: 6236_trunk.patch, HBASE-6236_94.patch,
HBASE-6236_trunk.patch, HBASE-6236_trunk.patch

[jira] [Commented] (HBASE-6236) Offline meta repair fails if the HBase base mount point is on a different cluster/volume than its parent in a ViewFS or similar FS


[ 
https://issues.apache.org/jira/browse/HBASE-6236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13399727#comment-13399727
 ] 

Jimmy Xiang commented on HBASE-6236:


Looks good to me. Should we also add this sideline dir option to HBaseFsck?

 Offline meta repair fails if the HBase base mount point is on a different 
 cluster/volume than its parent in a ViewFS or similar FS
 --

 Key: HBASE-6236
 URL: https://issues.apache.org/jira/browse/HBASE-6236
 Project: HBase
  Issue Type: Bug
  Components: hbck
Reporter: Aditya Kishore
Assignee: Aditya Kishore
 Fix For: 0.96.0, 0.94.1

 Attachments: 6236_trunk.patch, HBASE-6236_94.patch, 
 HBASE-6236_trunk.patch


 While building the .META. and \-ROOT\- from FS data alone (HBASE-4377), hbck 
 tries to move the existing .META. and \-ROOT\- directories to a backup folder.
 This backup folder is created at the same level as the base HBase folder 
 (e.g. /hbase-xx if the base HBase folder is '/hbase').
 In a federated HDFS like ViewFS and other similar FS implementations, it is 
 not possible to rename files/directories across namespace volumes (ViewFS 
 guide section 3.5) and as a result hbck crashes.
 A solution to this problem is to create the backup directory under the folder 
 where HBase base folder has been mounted. This ensures that source and 
 destination of rename operation are on the same namespace volume.
 Patch for 0.94 and trunk is attached for review. The patch modifies the 
 location of the backup directory from '/hbase-xxx' to 
 '/hbase/.hbcktmp-xxx'

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-6236) Offline meta repair fails if the HBase base mount point is on a different cluster/volume than its parent in a ViewFS or similar FS


[ 
https://issues.apache.org/jira/browse/HBASE-6236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13399734#comment-13399734
 ] 

Hudson commented on HBASE-6236:
---

Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #65 (See 
[https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/65/])
HBASE-6236 revert upon Jimmy's request. (Revision 1353013)
HBASE-6236 Offline meta repair fails if the HBase base mount point is on a 
different cluster/volume than its parent in a ViewFS or similar FS (Aditya) 
(Revision 1353008)

 Result = FAILURE
tedyu : 
Files : 
* 
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java
* 
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/util/hbck/OfflineMetaRepair.java

tedyu : 
Files : 
* 
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java
* 
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/util/hbck/OfflineMetaRepair.java


 Offline meta repair fails if the HBase base mount point is on a different 
 cluster/volume than its parent in a ViewFS or similar FS
 --

 Key: HBASE-6236
 URL: https://issues.apache.org/jira/browse/HBASE-6236
 Project: HBase
  Issue Type: Bug
  Components: hbck
Reporter: Aditya Kishore
Assignee: Aditya Kishore
 Fix For: 0.96.0, 0.94.1

 Attachments: 6236_trunk.patch, HBASE-6236_94.patch, 
 HBASE-6236_trunk.patch


 While building the .META. and \-ROOT\- from FS data alone (HBASE-4377), hbck 
 tries to move the existing .META. and \-ROOT\- directories to a backup folder.
 This backup folder is created at the same level as the base HBase folder 
 (e.g. /hbase-xx if the base HBase folder is '/hbase').
 In a federated HDFS like ViewFS and other similar FS implementations, it is 
 not possible to rename files/directories across namespace volumes (ViewFS 
 guide section 3.5) and as a result hbck crashes.
 A solution to this problem is to create the backup directory under the folder 
 where HBase base folder has been mounted. This ensures that source and 
 destination of rename operation are on the same namespace volume.
 Patch for 0.94 and trunk is attached for review. The patch modifies the 
 location of the backup directory from '/hbase-xxx' to 
 '/hbase/.hbcktmp-xxx'

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-6236) Offline meta repair fails if the HBase base mount point is on a different cluster/volume than its parent in a ViewFS or similar FS

2012-06-22 Thread Hadoop QA (JIRA)

[
https://issues.apache.org/jira/browse/HBASE-6236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13399782#comment-13399782
]

Hadoop QA commented on HBASE-6236:
--

-1 overall. Here are the results of testing the latest attachment

http://issues.apache.org/jira/secure/attachment/12533129/HBASE-6236_trunk.patch
against trunk revision .

+1 @author. The patch does not contain any @author tags.

+1 hadoop2.0. The patch compiles against the hadoop 2.0 profile.

+1 javadoc. The javadoc tool did not generate any warning messages.

+1 javac. The applied patch does not increase the total number of javac
compiler warnings.

-1 findbugs. The patch appears to introduce 11 new Findbugs (version
1.3.9) warnings.

+1 release audit. The applied patch does not increase the total number of
release audit warnings.

-1 core tests. The patch failed these unit tests:

org.apache.hadoop.hbase.regionserver.TestSplitTransactionOnCluster

Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/2227//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/2227//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/2227//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/2227//console

This message is automatically generated.

Offline meta repair fails if the HBase base mount point is on a different
cluster/volume than its parent in a ViewFS or similar FS
--

Key: HBASE-6236
URL: https://issues.apache.org/jira/browse/HBASE-6236
Project: HBase
Issue Type: Bug
Components: hbck
Reporter: Aditya Kishore
Assignee: Aditya Kishore
Fix For: 0.96.0, 0.94.1

Attachments: 6236_trunk.patch, HBASE-6236_94.patch,
HBASE-6236_trunk.patch

While building the .META. and \-ROOT\- from FS data alone (HBASE-4377), hbck
tries to move the existing .META. and \-ROOT\- directories to a backup folder.
This backup folder is created at the same level as the base HBase folder
(e.g. /hbase-xx if the base HBase folder is '/hbase').
In a federated HDFS like ViewFS and other similar FS implementations, it is
not possible to rename files/directories across namespace volumes (ViewFS
guide section 3.5) and as a result hbck crashes.
A solution to this problem is to create the backup directory under the folder
where HBase base folder has been mounted. This ensures that source and
destination of rename operation are on the same namespace volume.
Patch for 0.94 and trunk is attached for review. The patch modifies the
location of the backup directory from '/hbase-xxx' to
'/hbase/.hbcktmp-xxx'

[jira] [Commented] (HBASE-6236) Offline meta repair fails if the HBase base mount point is on a different cluster/volume than its parent in a ViewFS or similar FS

2012-06-22 Thread Hadoop QA (JIRA)

[
https://issues.apache.org/jira/browse/HBASE-6236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13399810#comment-13399810
]

Hadoop QA commented on HBASE-6236:
--

-1 overall. Here are the results of testing the latest attachment

http://issues.apache.org/jira/secure/attachment/12533142/HBASE-6236_trunk.patch
against trunk revision .

+1 @author. The patch does not contain any @author tags.

+1 hadoop2.0. The patch compiles against the hadoop 2.0 profile.

+1 javadoc. The javadoc tool did not generate any warning messages.

+1 javac. The applied patch does not increase the total number of javac
compiler warnings.

-1 findbugs. The patch appears to introduce 11 new Findbugs (version
1.3.9) warnings.

+1 release audit. The applied patch does not increase the total number of
release audit warnings.

+1 core tests. The patch passed unit tests in .

Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/2229//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/2229//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/2229//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/2229//console

This message is automatically generated.

Offline meta repair fails if the HBase base mount point is on a different
cluster/volume than its parent in a ViewFS or similar FS
--

Key: HBASE-6236
URL: https://issues.apache.org/jira/browse/HBASE-6236
Project: HBase
Issue Type: Bug
Components: hbck
Reporter: Aditya Kishore
Assignee: Aditya Kishore
Fix For: 0.96.0, 0.94.1

Attachments: 6236_trunk.patch, HBASE-6236_94.patch,
HBASE-6236_trunk.patch

[jira] [Commented] (HBASE-6236) Offline meta repair fails if the HBase base mount point is on a different cluster/volume than its parent in a ViewFS or similar FS


[ 
https://issues.apache.org/jira/browse/HBASE-6236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13399841#comment-13399841
 ] 

Hudson commented on HBASE-6236:
---

Integrated in HBase-0.94 #277 (See 
[https://builds.apache.org/job/HBase-0.94/277/])
HBASE-6236 Addendum adds -sidelineDir option to hbck (Aditya) (Revision 
1353066)
HBASE-6236 Offline meta repair fails if the HBase base mount point is on a 
different cluster/volume than its parent in a ViewFS or similar FS (Aditya) 
(Revision 1353064)

 Result = FAILURE
tedyu : 
Files : 
* /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/HConstants.java
* /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java
* 
/hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/util/hbck/OfflineMetaRepair.java

tedyu : 
Files : 
* /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/HConstants.java
* /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java
* 
/hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/util/hbck/OfflineMetaRepair.java


 Offline meta repair fails if the HBase base mount point is on a different 
 cluster/volume than its parent in a ViewFS or similar FS
 --

 Key: HBASE-6236
 URL: https://issues.apache.org/jira/browse/HBASE-6236
 Project: HBase
  Issue Type: Bug
  Components: hbck
Reporter: Aditya Kishore
Assignee: Aditya Kishore
 Fix For: 0.96.0, 0.94.1

 Attachments: 6236-94.addendum, 6236-94.txt, 6236_trunk.patch, 
 HBASE-6236_94.patch, HBASE-6236_trunk.patch


 While building the .META. and \-ROOT\- from FS data alone (HBASE-4377), hbck 
 tries to move the existing .META. and \-ROOT\- directories to a backup folder.
 This backup folder is created at the same level as the base HBase folder 
 (e.g. /hbase-xx if the base HBase folder is '/hbase').
 In a federated HDFS like ViewFS and other similar FS implementations, it is 
 not possible to rename files/directories across namespace volumes (ViewFS 
 guide section 3.5) and as a result hbck crashes.
 A solution to this problem is to create the backup directory under the folder 
 where HBase base folder has been mounted. This ensures that source and 
 destination of rename operation are on the same namespace volume.
 Patch for 0.94 and trunk is attached for review. The patch modifies the 
 location of the backup directory from '/hbase-xxx' to 
 '/hbase/.hbcktmp-xxx'

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-6236) Offline meta repair fails if the HBase base mount point is on a different cluster/volume than its parent in a ViewFS or similar FS