[jira] [Commented] (HBASE-6067) HBase won't start when hbase.rootdir uses ViewFileSystem
[ https://issues.apache.org/jira/browse/HBASE-6067?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13289082#comment-13289082 ] Hudson commented on HBASE-6067: --- Integrated in HBase-0.92-security #111 (See [https://builds.apache.org/job/HBase-0.92-security/111/]) HBASE-6067 HBase won't start when hbase.rootdir uses ViewFileSystem (Revision 1346098) Result = FAILURE tedyu : Files : * /hbase/branches/0.92/CHANGES.txt * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/regionserver/wal/HLog.java > HBase won't start when hbase.rootdir uses ViewFileSystem > > > Key: HBASE-6067 > URL: https://issues.apache.org/jira/browse/HBASE-6067 > Project: HBase > Issue Type: Improvement > Components: regionserver >Reporter: Eli Collins >Assignee: Zhihong Ted Yu > Fix For: 0.92.2, 0.96.0, 0.94.1 > > Attachments: 6067-addendum.txt, 6067-v2.txt, 6067.txt > > > HBase currently doesn't work with HDFS federation (hbase.rootdir with a > client that uses viewfs) because HLog#init uses > FileSystem#getDefaultBlockSize and getDefaultReplication. These throw an > exception because there is no default filesystem in a viewfs client so > there's no way to determine a default block size or replication factor. They > could use the versions of these methods that take a path, however these were > introduced in HADOOP-8014 and are not yet available in Hadoop 1.x. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6067) HBase won't start when hbase.rootdir uses ViewFileSystem
[ https://issues.apache.org/jira/browse/HBASE-6067?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13289039#comment-13289039 ] Hudson commented on HBASE-6067: --- Integrated in HBase-0.94-security #34 (See [https://builds.apache.org/job/HBase-0.94-security/34/]) HBASE-6067 Addendum adopts suggestions from Daryn (Revision 1346055) HBASE-6067 HBase won't start when hbase.rootdir uses ViewFileSystem (Revision 1345753) Result = SUCCESS tedyu : Files : * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/wal/HLog.java tedyu : Files : * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/wal/HLog.java > HBase won't start when hbase.rootdir uses ViewFileSystem > > > Key: HBASE-6067 > URL: https://issues.apache.org/jira/browse/HBASE-6067 > Project: HBase > Issue Type: Improvement > Components: regionserver >Reporter: Eli Collins >Assignee: Zhihong Ted Yu > Fix For: 0.92.2, 0.96.0, 0.94.1 > > Attachments: 6067-addendum.txt, 6067-v2.txt, 6067.txt > > > HBase currently doesn't work with HDFS federation (hbase.rootdir with a > client that uses viewfs) because HLog#init uses > FileSystem#getDefaultBlockSize and getDefaultReplication. These throw an > exception because there is no default filesystem in a viewfs client so > there's no way to determine a default block size or replication factor. They > could use the versions of these methods that take a path, however these were > introduced in HADOOP-8014 and are not yet available in Hadoop 1.x. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6067) HBase won't start when hbase.rootdir uses ViewFileSystem
[ https://issues.apache.org/jira/browse/HBASE-6067?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13289012#comment-13289012 ] Hudson commented on HBASE-6067: --- Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #40 (See [https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/40/]) HBASE-6067 Addendum adopts suggestions from Daryn (Revision 1346057) Result = FAILURE tedyu : Files : * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/HLog.java > HBase won't start when hbase.rootdir uses ViewFileSystem > > > Key: HBASE-6067 > URL: https://issues.apache.org/jira/browse/HBASE-6067 > Project: HBase > Issue Type: Improvement > Components: regionserver >Reporter: Eli Collins >Assignee: Zhihong Ted Yu > Fix For: 0.92.2, 0.96.0, 0.94.1 > > Attachments: 6067-addendum.txt, 6067-v2.txt, 6067.txt > > > HBase currently doesn't work with HDFS federation (hbase.rootdir with a > client that uses viewfs) because HLog#init uses > FileSystem#getDefaultBlockSize and getDefaultReplication. These throw an > exception because there is no default filesystem in a viewfs client so > there's no way to determine a default block size or replication factor. They > could use the versions of these methods that take a path, however these were > introduced in HADOOP-8014 and are not yet available in Hadoop 1.x. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6067) HBase won't start when hbase.rootdir uses ViewFileSystem
[ https://issues.apache.org/jira/browse/HBASE-6067?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13288996#comment-13288996 ] Hudson commented on HBASE-6067: --- Integrated in HBase-0.92 #446 (See [https://builds.apache.org/job/HBase-0.92/446/]) HBASE-6067 HBase won't start when hbase.rootdir uses ViewFileSystem (Revision 1346098) Result = FAILURE tedyu : Files : * /hbase/branches/0.92/CHANGES.txt * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/regionserver/wal/HLog.java > HBase won't start when hbase.rootdir uses ViewFileSystem > > > Key: HBASE-6067 > URL: https://issues.apache.org/jira/browse/HBASE-6067 > Project: HBase > Issue Type: Improvement > Components: regionserver >Reporter: Eli Collins >Assignee: Zhihong Ted Yu > Fix For: 0.92.2, 0.96.0, 0.94.1 > > Attachments: 6067-addendum.txt, 6067-v2.txt, 6067.txt > > > HBase currently doesn't work with HDFS federation (hbase.rootdir with a > client that uses viewfs) because HLog#init uses > FileSystem#getDefaultBlockSize and getDefaultReplication. These throw an > exception because there is no default filesystem in a viewfs client so > there's no way to determine a default block size or replication factor. They > could use the versions of these methods that take a path, however these were > introduced in HADOOP-8014 and are not yet available in Hadoop 1.x. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6067) HBase won't start when hbase.rootdir uses ViewFileSystem
[ https://issues.apache.org/jira/browse/HBASE-6067?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13288828#comment-13288828 ] Hudson commented on HBASE-6067: --- Integrated in HBase-0.94 #245 (See [https://builds.apache.org/job/HBase-0.94/245/]) HBASE-6067 Addendum adopts suggestions from Daryn (Revision 1346055) Result = FAILURE tedyu : Files : * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/wal/HLog.java > HBase won't start when hbase.rootdir uses ViewFileSystem > > > Key: HBASE-6067 > URL: https://issues.apache.org/jira/browse/HBASE-6067 > Project: HBase > Issue Type: Improvement > Components: regionserver >Reporter: Eli Collins >Assignee: Zhihong Ted Yu > Fix For: 0.96.0, 0.94.1, 0.92.3 > > Attachments: 6067-addendum.txt, 6067-v2.txt, 6067.txt > > > HBase currently doesn't work with HDFS federation (hbase.rootdir with a > client that uses viewfs) because HLog#init uses > FileSystem#getDefaultBlockSize and getDefaultReplication. These throw an > exception because there is no default filesystem in a viewfs client so > there's no way to determine a default block size or replication factor. They > could use the versions of these methods that take a path, however these were > introduced in HADOOP-8014 and are not yet available in Hadoop 1.x. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6067) HBase won't start when hbase.rootdir uses ViewFileSystem
[ https://issues.apache.org/jira/browse/HBASE-6067?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13288825#comment-13288825 ] stack commented on HBASE-6067: -- +1 on getting it into 0.92.2. I'll wait on it before cutting next RC > HBase won't start when hbase.rootdir uses ViewFileSystem > > > Key: HBASE-6067 > URL: https://issues.apache.org/jira/browse/HBASE-6067 > Project: HBase > Issue Type: Improvement > Components: regionserver >Reporter: Eli Collins >Assignee: Zhihong Ted Yu > Fix For: 0.96.0, 0.94.1, 0.92.3 > > Attachments: 6067-addendum.txt, 6067-v2.txt, 6067.txt > > > HBase currently doesn't work with HDFS federation (hbase.rootdir with a > client that uses viewfs) because HLog#init uses > FileSystem#getDefaultBlockSize and getDefaultReplication. These throw an > exception because there is no default filesystem in a viewfs client so > there's no way to determine a default block size or replication factor. They > could use the versions of these methods that take a path, however these were > introduced in HADOOP-8014 and are not yet available in Hadoop 1.x. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6067) HBase won't start when hbase.rootdir uses ViewFileSystem
[ https://issues.apache.org/jira/browse/HBASE-6067?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13288803#comment-13288803 ] Hudson commented on HBASE-6067: --- Integrated in HBase-TRUNK #2985 (See [https://builds.apache.org/job/HBase-TRUNK/2985/]) HBASE-6067 Addendum adopts suggestions from Daryn (Revision 1346057) Result = FAILURE tedyu : Files : * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/HLog.java > HBase won't start when hbase.rootdir uses ViewFileSystem > > > Key: HBASE-6067 > URL: https://issues.apache.org/jira/browse/HBASE-6067 > Project: HBase > Issue Type: Improvement > Components: regionserver >Reporter: Eli Collins >Assignee: Zhihong Ted Yu > Fix For: 0.96.0, 0.94.1, 0.92.3 > > Attachments: 6067-addendum.txt, 6067-v2.txt, 6067.txt > > > HBase currently doesn't work with HDFS federation (hbase.rootdir with a > client that uses viewfs) because HLog#init uses > FileSystem#getDefaultBlockSize and getDefaultReplication. These throw an > exception because there is no default filesystem in a viewfs client so > there's no way to determine a default block size or replication factor. They > could use the versions of these methods that take a path, however these were > introduced in HADOOP-8014 and are not yet available in Hadoop 1.x. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6067) HBase won't start when hbase.rootdir uses ViewFileSystem
[ https://issues.apache.org/jira/browse/HBASE-6067?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13288755#comment-13288755 ] Hadoop QA commented on HBASE-6067: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12530821/6067-addendum.txt against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. -1 patch. The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/2100//console This message is automatically generated. > HBase won't start when hbase.rootdir uses ViewFileSystem > > > Key: HBASE-6067 > URL: https://issues.apache.org/jira/browse/HBASE-6067 > Project: HBase > Issue Type: Improvement > Components: regionserver >Reporter: Eli Collins >Assignee: Zhihong Ted Yu > Fix For: 0.96.0, 0.94.1, 0.92.3 > > Attachments: 6067-addendum.txt, 6067-v2.txt, 6067.txt > > > HBase currently doesn't work with HDFS federation (hbase.rootdir with a > client that uses viewfs) because HLog#init uses > FileSystem#getDefaultBlockSize and getDefaultReplication. These throw an > exception because there is no default filesystem in a viewfs client so > there's no way to determine a default block size or replication factor. They > could use the versions of these methods that take a path, however these were > introduced in HADOOP-8014 and are not yet available in Hadoop 1.x. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6067) HBase won't start when hbase.rootdir uses ViewFileSystem
[ https://issues.apache.org/jira/browse/HBASE-6067?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13288740#comment-13288740 ] Zhihong Ted Yu commented on HBASE-6067: --- Addendum integrated to 0.94 and trunk. Thanks for the tip, Daryn. > HBase won't start when hbase.rootdir uses ViewFileSystem > > > Key: HBASE-6067 > URL: https://issues.apache.org/jira/browse/HBASE-6067 > Project: HBase > Issue Type: Improvement > Components: regionserver >Reporter: Eli Collins >Assignee: Zhihong Ted Yu > Fix For: 0.96.0, 0.94.1, 0.92.3 > > Attachments: 6067-addendum.txt, 6067-v2.txt, 6067.txt > > > HBase currently doesn't work with HDFS federation (hbase.rootdir with a > client that uses viewfs) because HLog#init uses > FileSystem#getDefaultBlockSize and getDefaultReplication. These throw an > exception because there is no default filesystem in a viewfs client so > there's no way to determine a default block size or replication factor. They > could use the versions of these methods that take a path, however these were > introduced in HADOOP-8014 and are not yet available in Hadoop 1.x. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6067) HBase won't start when hbase.rootdir uses ViewFileSystem
[
https://issues.apache.org/jira/browse/HBASE-6067?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13288708#comment-13288708
]
Daryn Sharp commented on HBASE-6067:
This code demonstrates that {{getDeclaredMethod}} only returns a method
explicitly defined in the class.
{code}
import java.lang.reflect.*;
public class Test {
public static void main(String[] args) throws Exception {
String m;
try {
m = B.class.getDeclaredMethod("method").toString();
} catch (Exception e) {
m = e.toString();
}
System.out.println("getDeclaredMethod: "+m);
try {
m = B.class.getMethod("method").toString();
} catch (Exception e) {
m = e.toString();
}
System.out.println("getMethod: "+m);
}
static class A { public void method() {} }
static class B extends A { }
}
{code}
Produces the output:
{code}
getDeclaredMethod: java.lang.NoSuchMethodException: Test$B.method()
getMethod: public void Test$A.method()
{code}
Per the docs and my testing, the get method calls only return public methods,
so the {{setAccessible}} is a no-op.
> HBase won't start when hbase.rootdir uses ViewFileSystem
>
>
> Key: HBASE-6067
> URL: https://issues.apache.org/jira/browse/HBASE-6067
> Project: HBase
> Issue Type: Improvement
> Components: regionserver
>Reporter: Eli Collins
>Assignee: Zhihong Ted Yu
> Fix For: 0.96.0, 0.94.1, 0.92.3
>
> Attachments: 6067-v2.txt, 6067.txt
>
>
> HBase currently doesn't work with HDFS federation (hbase.rootdir with a
> client that uses viewfs) because HLog#init uses
> FileSystem#getDefaultBlockSize and getDefaultReplication. These throw an
> exception because there is no default filesystem in a viewfs client so
> there's no way to determine a default block size or replication factor. They
> could use the versions of these methods that take a path, however these were
> introduced in HADOOP-8014 and are not yet available in Hadoop 1.x.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6067) HBase won't start when hbase.rootdir uses ViewFileSystem
[ https://issues.apache.org/jira/browse/HBASE-6067?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13288636#comment-13288636 ] Zhihong Ted Yu commented on HBASE-6067: --- @Stack: Do you think this JIRA should be in 0.92.2 RC ? > HBase won't start when hbase.rootdir uses ViewFileSystem > > > Key: HBASE-6067 > URL: https://issues.apache.org/jira/browse/HBASE-6067 > Project: HBase > Issue Type: Improvement > Components: regionserver >Reporter: Eli Collins >Assignee: Zhihong Ted Yu > Fix For: 0.96.0, 0.94.1, 0.92.3 > > Attachments: 6067-v2.txt, 6067.txt > > > HBase currently doesn't work with HDFS federation (hbase.rootdir with a > client that uses viewfs) because HLog#init uses > FileSystem#getDefaultBlockSize and getDefaultReplication. These throw an > exception because there is no default filesystem in a viewfs client so > there's no way to determine a default block size or replication factor. They > could use the versions of these methods that take a path, however these were > introduced in HADOOP-8014 and are not yet available in Hadoop 1.x. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6067) HBase won't start when hbase.rootdir uses ViewFileSystem
[
https://issues.apache.org/jira/browse/HBASE-6067?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13288606#comment-13288606
]
Zhihong Ted Yu commented on HBASE-6067:
---
@Daryn:
Looking at:
http://docs.oracle.com/javase/1.4.2/docs/api/java/lang/Class.html#getDeclaredMethod%28java.lang.String,%20java.lang.Class[]%29
I don't see why getDeclaredMethod wouldn't find the desired method from this.fs
w.r.t. setAccessible(), I didn't include it in patch v1 for this reason.
Then I found that getGetNumCurrentReplicas() was using the call. And:
{code}
public int getNumCurrentReplicas() throws IOException {
./src/hdfs/org/apache/hadoop/hdfs/DFSClient.java
{code}
> HBase won't start when hbase.rootdir uses ViewFileSystem
>
>
> Key: HBASE-6067
> URL: https://issues.apache.org/jira/browse/HBASE-6067
> Project: HBase
> Issue Type: Improvement
> Components: regionserver
>Reporter: Eli Collins
>Assignee: Zhihong Ted Yu
> Fix For: 0.96.0, 0.94.1, 0.92.3
>
> Attachments: 6067-v2.txt, 6067.txt
>
>
> HBase currently doesn't work with HDFS federation (hbase.rootdir with a
> client that uses viewfs) because HLog#init uses
> FileSystem#getDefaultBlockSize and getDefaultReplication. These throw an
> exception because there is no default filesystem in a viewfs client so
> there's no way to determine a default block size or replication factor. They
> could use the versions of these methods that take a path, however these were
> introduced in HADOOP-8014 and are not yet available in Hadoop 1.x.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6067) HBase won't start when hbase.rootdir uses ViewFileSystem
[
https://issues.apache.org/jira/browse/HBASE-6067?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13288542#comment-13288542
]
Daryn Sharp commented on HBASE-6067:
I forgot to mention, is {{setAccessible}} needed? The method is public.
> HBase won't start when hbase.rootdir uses ViewFileSystem
>
>
> Key: HBASE-6067
> URL: https://issues.apache.org/jira/browse/HBASE-6067
> Project: HBase
> Issue Type: Improvement
> Components: regionserver
>Reporter: Eli Collins
>Assignee: Zhihong Ted Yu
> Fix For: 0.96.0, 0.94.1, 0.92.3
>
> Attachments: 6067-v2.txt, 6067.txt
>
>
> HBase currently doesn't work with HDFS federation (hbase.rootdir with a
> client that uses viewfs) because HLog#init uses
> FileSystem#getDefaultBlockSize and getDefaultReplication. These throw an
> exception because there is no default filesystem in a viewfs client so
> there's no way to determine a default block size or replication factor. They
> could use the versions of these methods that take a path, however these were
> introduced in HADOOP-8014 and are not yet available in Hadoop 1.x.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6067) HBase won't start when hbase.rootdir uses ViewFileSystem
[
https://issues.apache.org/jira/browse/HBASE-6067?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13288541#comment-13288541
]
Daryn Sharp commented on HBASE-6067:
{{getMethod}} should be used instead of {{getDeclaredMethod}} since most
filesystems, like DFS, currently rely on the default impl of the path accepting
methods in {{FileSystem}} to call the non-path methods (which are or should be
deprecated). I don't think {{getDeclaredMethod}} will search the superclasses,
so it will always fallback to the non-path version when the fs isn't viewfs.
> HBase won't start when hbase.rootdir uses ViewFileSystem
>
>
> Key: HBASE-6067
> URL: https://issues.apache.org/jira/browse/HBASE-6067
> Project: HBase
> Issue Type: Improvement
> Components: regionserver
>Reporter: Eli Collins
>Assignee: Zhihong Ted Yu
> Fix For: 0.96.0, 0.94.1, 0.92.3
>
> Attachments: 6067-v2.txt, 6067.txt
>
>
> HBase currently doesn't work with HDFS federation (hbase.rootdir with a
> client that uses viewfs) because HLog#init uses
> FileSystem#getDefaultBlockSize and getDefaultReplication. These throw an
> exception because there is no default filesystem in a viewfs client so
> there's no way to determine a default block size or replication factor. They
> could use the versions of these methods that take a path, however these were
> introduced in HADOOP-8014 and are not yet available in Hadoop 1.x.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6067) HBase won't start when hbase.rootdir uses ViewFileSystem
[ https://issues.apache.org/jira/browse/HBASE-6067?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13288283#comment-13288283 ] Hudson commented on HBASE-6067: --- Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #38 (See [https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/38/]) HBASE-6067 HBase won't start when hbase.rootdir uses ViewFileSystem (Revision 1345727) Result = FAILURE tedyu : Files : * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/HLog.java > HBase won't start when hbase.rootdir uses ViewFileSystem > > > Key: HBASE-6067 > URL: https://issues.apache.org/jira/browse/HBASE-6067 > Project: HBase > Issue Type: Improvement > Components: regionserver >Reporter: Eli Collins >Assignee: Zhihong Ted Yu > Fix For: 0.96.0, 0.94.1, 0.92.3 > > Attachments: 6067-v2.txt, 6067.txt > > > HBase currently doesn't work with HDFS federation (hbase.rootdir with a > client that uses viewfs) because HLog#init uses > FileSystem#getDefaultBlockSize and getDefaultReplication. These throw an > exception because there is no default filesystem in a viewfs client so > there's no way to determine a default block size or replication factor. They > could use the versions of these methods that take a path, however these were > introduced in HADOOP-8014 and are not yet available in Hadoop 1.x. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6067) HBase won't start when hbase.rootdir uses ViewFileSystem
[ https://issues.apache.org/jira/browse/HBASE-6067?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13288256#comment-13288256 ] Hudson commented on HBASE-6067: --- Integrated in HBase-0.94 #243 (See [https://builds.apache.org/job/HBase-0.94/243/]) HBASE-6067 HBase won't start when hbase.rootdir uses ViewFileSystem (Revision 1345753) Result = SUCCESS tedyu : Files : * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/wal/HLog.java > HBase won't start when hbase.rootdir uses ViewFileSystem > > > Key: HBASE-6067 > URL: https://issues.apache.org/jira/browse/HBASE-6067 > Project: HBase > Issue Type: Improvement > Components: regionserver >Reporter: Eli Collins >Assignee: Zhihong Ted Yu > Fix For: 0.96.0, 0.94.1, 0.92.3 > > Attachments: 6067-v2.txt, 6067.txt > > > HBase currently doesn't work with HDFS federation (hbase.rootdir with a > client that uses viewfs) because HLog#init uses > FileSystem#getDefaultBlockSize and getDefaultReplication. These throw an > exception because there is no default filesystem in a viewfs client so > there's no way to determine a default block size or replication factor. They > could use the versions of these methods that take a path, however these were > introduced in HADOOP-8014 and are not yet available in Hadoop 1.x. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6067) HBase won't start when hbase.rootdir uses ViewFileSystem
[ https://issues.apache.org/jira/browse/HBASE-6067?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13288248#comment-13288248 ] Zhihong Ted Yu commented on HBASE-6067: --- Integrated to 0.94 Will wait for release of 0.92.2 before integrating to 0.92 branch. > HBase won't start when hbase.rootdir uses ViewFileSystem > > > Key: HBASE-6067 > URL: https://issues.apache.org/jira/browse/HBASE-6067 > Project: HBase > Issue Type: Improvement > Components: regionserver >Reporter: Eli Collins >Assignee: Zhihong Ted Yu > Fix For: 0.96.0, 0.94.1, 0.92.3 > > Attachments: 6067-v2.txt, 6067.txt > > > HBase currently doesn't work with HDFS federation (hbase.rootdir with a > client that uses viewfs) because HLog#init uses > FileSystem#getDefaultBlockSize and getDefaultReplication. These throw an > exception because there is no default filesystem in a viewfs client so > there's no way to determine a default block size or replication factor. They > could use the versions of these methods that take a path, however these were > introduced in HADOOP-8014 and are not yet available in Hadoop 1.x. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6067) HBase won't start when hbase.rootdir uses ViewFileSystem
[ https://issues.apache.org/jira/browse/HBASE-6067?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13288234#comment-13288234 ] Hudson commented on HBASE-6067: --- Integrated in HBase-TRUNK #2979 (See [https://builds.apache.org/job/HBase-TRUNK/2979/]) HBASE-6067 HBase won't start when hbase.rootdir uses ViewFileSystem (Revision 1345727) Result = SUCCESS tedyu : Files : * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/HLog.java > HBase won't start when hbase.rootdir uses ViewFileSystem > > > Key: HBASE-6067 > URL: https://issues.apache.org/jira/browse/HBASE-6067 > Project: HBase > Issue Type: Improvement > Components: regionserver >Reporter: Eli Collins >Assignee: Zhihong Ted Yu > Fix For: 0.96.0, 0.94.1, 0.92.3 > > Attachments: 6067-v2.txt, 6067.txt > > > HBase currently doesn't work with HDFS federation (hbase.rootdir with a > client that uses viewfs) because HLog#init uses > FileSystem#getDefaultBlockSize and getDefaultReplication. These throw an > exception because there is no default filesystem in a viewfs client so > there's no way to determine a default block size or replication factor. They > could use the versions of these methods that take a path, however these were > introduced in HADOOP-8014 and are not yet available in Hadoop 1.x. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6067) HBase won't start when hbase.rootdir uses ViewFileSystem
[ https://issues.apache.org/jira/browse/HBASE-6067?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13288218#comment-13288218 ] Zhihong Ted Yu commented on HBASE-6067: --- Integrated to trunk first. Thanks for the review, Eli. > HBase won't start when hbase.rootdir uses ViewFileSystem > > > Key: HBASE-6067 > URL: https://issues.apache.org/jira/browse/HBASE-6067 > Project: HBase > Issue Type: Improvement > Components: regionserver >Reporter: Eli Collins >Assignee: Zhihong Ted Yu > Fix For: 0.96.0, 0.94.1, 0.92.3 > > Attachments: 6067-v2.txt, 6067.txt > > > HBase currently doesn't work with HDFS federation (hbase.rootdir with a > client that uses viewfs) because HLog#init uses > FileSystem#getDefaultBlockSize and getDefaultReplication. These throw an > exception because there is no default filesystem in a viewfs client so > there's no way to determine a default block size or replication factor. They > could use the versions of these methods that take a path, however these were > introduced in HADOOP-8014 and are not yet available in Hadoop 1.x. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6067) HBase won't start when hbase.rootdir uses ViewFileSystem
[ https://issues.apache.org/jira/browse/HBASE-6067?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13288214#comment-13288214 ] Eli Collins commented on HBASE-6067: I'd merge it to both 92 and 94. > HBase won't start when hbase.rootdir uses ViewFileSystem > > > Key: HBASE-6067 > URL: https://issues.apache.org/jira/browse/HBASE-6067 > Project: HBase > Issue Type: Improvement > Components: regionserver >Reporter: Eli Collins >Assignee: Zhihong Ted Yu > Fix For: 0.96.0 > > Attachments: 6067-v2.txt, 6067.txt > > > HBase currently doesn't work with HDFS federation (hbase.rootdir with a > client that uses viewfs) because HLog#init uses > FileSystem#getDefaultBlockSize and getDefaultReplication. These throw an > exception because there is no default filesystem in a viewfs client so > there's no way to determine a default block size or replication factor. They > could use the versions of these methods that take a path, however these were > introduced in HADOOP-8014 and are not yet available in Hadoop 1.x. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6067) HBase won't start when hbase.rootdir uses ViewFileSystem
[ https://issues.apache.org/jira/browse/HBASE-6067?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13288166#comment-13288166 ] Zhihong Yu commented on HBASE-6067: --- I ran TestRegionRebalancing on MacBook and it passed. @Eli: Do you think the patch should go to 0.92 or 0.94 as well ? > HBase won't start when hbase.rootdir uses ViewFileSystem > > > Key: HBASE-6067 > URL: https://issues.apache.org/jira/browse/HBASE-6067 > Project: HBase > Issue Type: Improvement > Components: regionserver >Reporter: Eli Collins >Assignee: Zhihong Yu > Fix For: 0.96.0 > > Attachments: 6067-v2.txt, 6067.txt > > > HBase currently doesn't work with HDFS federation (hbase.rootdir with a > client that uses viewfs) because HLog#init uses > FileSystem#getDefaultBlockSize and getDefaultReplication. These throw an > exception because there is no default filesystem in a viewfs client so > there's no way to determine a default block size or replication factor. They > could use the versions of these methods that take a path, however these were > introduced in HADOOP-8014 and are not yet available in Hadoop 1.x. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6067) HBase won't start when hbase.rootdir uses ViewFileSystem
[ https://issues.apache.org/jira/browse/HBASE-6067?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13288061#comment-13288061 ] Hadoop QA commented on HBASE-6067: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12530672/6067-v2.txt against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 hadoop2.0. The patch compiles against the hadoop 2.0 profile. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. -1 findbugs. The patch appears to cause Findbugs (version 1.3.9) to fail. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: org.apache.hadoop.hbase.TestRegionRebalancing Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/2090//testReport/ Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/2090//console This message is automatically generated. > HBase won't start when hbase.rootdir uses ViewFileSystem > > > Key: HBASE-6067 > URL: https://issues.apache.org/jira/browse/HBASE-6067 > Project: HBase > Issue Type: Improvement > Components: regionserver >Reporter: Eli Collins >Assignee: Zhihong Yu > Attachments: 6067-v2.txt, 6067.txt > > > HBase currently doesn't work with HDFS federation (hbase.rootdir with a > client that uses viewfs) because HLog#init uses > FileSystem#getDefaultBlockSize and getDefaultReplication. These throw an > exception because there is no default filesystem in a viewfs client so > there's no way to determine a default block size or replication factor. They > could use the versions of these methods that take a path, however these were > introduced in HADOOP-8014 and are not yet available in Hadoop 1.x. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6067) HBase won't start when hbase.rootdir uses ViewFileSystem
[
https://issues.apache.org/jira/browse/HBASE-6067?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13288059#comment-13288059
]
Zhihong Yu commented on HBASE-6067:
---
findbugs functionality hasn't been fixed
(https://builds.apache.org/job/PreCommit-HBASE-Build/2088//console):
{code}
[ERROR] Could not find resource
'${parent.basedir}/../dev-support/findbugs-exclude.xml'. -> [Help 1]
{code}
> HBase won't start when hbase.rootdir uses ViewFileSystem
>
>
> Key: HBASE-6067
> URL: https://issues.apache.org/jira/browse/HBASE-6067
> Project: HBase
> Issue Type: Improvement
> Components: regionserver
>Reporter: Eli Collins
>Assignee: Zhihong Yu
> Attachments: 6067-v2.txt, 6067.txt
>
>
> HBase currently doesn't work with HDFS federation (hbase.rootdir with a
> client that uses viewfs) because HLog#init uses
> FileSystem#getDefaultBlockSize and getDefaultReplication. These throw an
> exception because there is no default filesystem in a viewfs client so
> there's no way to determine a default block size or replication factor. They
> could use the versions of these methods that take a path, however these were
> introduced in HADOOP-8014 and are not yet available in Hadoop 1.x.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6067) HBase won't start when hbase.rootdir uses ViewFileSystem
[
https://issues.apache.org/jira/browse/HBASE-6067?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13288057#comment-13288057
]
Zhihong Yu commented on HBASE-6067:
---
fs.getDefaultBlockSize() is only called in one place:
{code}
this.blocksize = conf.getLong("hbase.regionserver.hlog.blocksize",
getDefaultBlockSize());
{code}
So I didn't a Method member.
I will upload a new patch with setAccessible() call.
> HBase won't start when hbase.rootdir uses ViewFileSystem
>
>
> Key: HBASE-6067
> URL: https://issues.apache.org/jira/browse/HBASE-6067
> Project: HBase
> Issue Type: Improvement
> Components: regionserver
>Reporter: Eli Collins
>Assignee: Eli Collins
> Attachments: 6067.txt
>
>
> HBase currently doesn't work with HDFS federation (hbase.rootdir with a
> client that uses viewfs) because HLog#init uses
> FileSystem#getDefaultBlockSize and getDefaultReplication. These throw an
> exception because there is no default filesystem in a viewfs client so
> there's no way to determine a default block size or replication factor. They
> could use the versions of these methods that take a path, however these were
> introduced in HADOOP-8014 and are not yet available in Hadoop 1.x.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6067) HBase won't start when hbase.rootdir uses ViewFileSystem
[ https://issues.apache.org/jira/browse/HBASE-6067?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13288045#comment-13288045 ] Eli Collins commented on HBASE-6067: Zhihong, Approach seems reasonable to me. I'd make it consistent with getNumCurrentReplicas and use a Method member. Also, think you're missing a call to setAccessible? Worth checking out why there's a findbugs warning as well. HBase gang - can someone make Zhihong a contributor and assign this to him? > HBase won't start when hbase.rootdir uses ViewFileSystem > > > Key: HBASE-6067 > URL: https://issues.apache.org/jira/browse/HBASE-6067 > Project: HBase > Issue Type: Improvement > Components: regionserver >Reporter: Eli Collins >Assignee: Eli Collins > Attachments: 6067.txt > > > HBase currently doesn't work with HDFS federation (hbase.rootdir with a > client that uses viewfs) because HLog#init uses > FileSystem#getDefaultBlockSize and getDefaultReplication. These throw an > exception because there is no default filesystem in a viewfs client so > there's no way to determine a default block size or replication factor. They > could use the versions of these methods that take a path, however these were > introduced in HADOOP-8014 and are not yet available in Hadoop 1.x. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6067) HBase won't start when hbase.rootdir uses ViewFileSystem
[ https://issues.apache.org/jira/browse/HBASE-6067?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13288017#comment-13288017 ] Zhihong Yu commented on HBASE-6067: --- @Eli: Do you think the patch is Okay. > HBase won't start when hbase.rootdir uses ViewFileSystem > > > Key: HBASE-6067 > URL: https://issues.apache.org/jira/browse/HBASE-6067 > Project: HBase > Issue Type: Improvement > Components: regionserver >Reporter: Eli Collins >Assignee: Eli Collins > Attachments: 6067.txt > > > HBase currently doesn't work with HDFS federation (hbase.rootdir with a > client that uses viewfs) because HLog#init uses > FileSystem#getDefaultBlockSize and getDefaultReplication. These throw an > exception because there is no default filesystem in a viewfs client so > there's no way to determine a default block size or replication factor. They > could use the versions of these methods that take a path, however these were > introduced in HADOOP-8014 and are not yet available in Hadoop 1.x. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6067) HBase won't start when hbase.rootdir uses ViewFileSystem
[ https://issues.apache.org/jira/browse/HBASE-6067?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13287984#comment-13287984 ] Hadoop QA commented on HBASE-6067: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12530653/6067.txt against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 hadoop2.0. The patch compiles against the hadoop 2.0 profile. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. -1 findbugs. The patch appears to cause Findbugs (version 1.3.9) to fail. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in . Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/2088//testReport/ Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/2088//console This message is automatically generated. > HBase won't start when hbase.rootdir uses ViewFileSystem > > > Key: HBASE-6067 > URL: https://issues.apache.org/jira/browse/HBASE-6067 > Project: HBase > Issue Type: Improvement > Components: regionserver >Reporter: Eli Collins >Assignee: Eli Collins > Attachments: 6067.txt > > > HBase currently doesn't work with HDFS federation (hbase.rootdir with a > client that uses viewfs) because HLog#init uses > FileSystem#getDefaultBlockSize and getDefaultReplication. These throw an > exception because there is no default filesystem in a viewfs client so > there's no way to determine a default block size or replication factor. They > could use the versions of these methods that take a path, however these were > introduced in HADOOP-8014 and are not yet available in Hadoop 1.x. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6067) HBase won't start when hbase.rootdir uses ViewFileSystem
[
https://issues.apache.org/jira/browse/HBASE-6067?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13281894#comment-13281894
]
Zhihong Yu commented on HBASE-6067:
---
+1 on backporting new methods.
We can use reflection to detect the existence of the new method:
{code}
+ public long getDefaultBlockSize(Path f) {
{code}
If the above method exists, we pass hbase rootdir to it. Otherwise fall back to
calling getDefaultBlockSize().
> HBase won't start when hbase.rootdir uses ViewFileSystem
>
>
> Key: HBASE-6067
> URL: https://issues.apache.org/jira/browse/HBASE-6067
> Project: HBase
> Issue Type: Improvement
> Components: regionserver
>Reporter: Eli Collins
>Assignee: Eli Collins
>
> HBase currently doesn't work with HDFS federation (hbase.rootdir with a
> client that uses viewfs) because HLog#init uses
> FileSystem#getDefaultBlockSize and getDefaultReplication. These throw an
> exception because there is no default filesystem in a viewfs client so
> there's no way to determine a default block size or replication factor. They
> could use the versions of these methods that take a path, however these were
> introduced in HADOOP-8014 and are not yet available in Hadoop 1.x.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6067) HBase won't start when hbase.rootdir uses ViewFileSystem
[ https://issues.apache.org/jira/browse/HBASE-6067?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13281878#comment-13281878 ] Eli Collins commented on HBASE-6067: It points to a the hbase directory on a viewfs mount (the URI is viewfs://cluster2/hbase). Per HADOOP-8422 I think we should backport the new methods to branch-1 and modify HBase here to use them. This means the next HBase release with this patch would require Hadoop 1.1 or later. This might be reasonable given that Hadoop 1.1 is the first 1.x release that will support HBase out of the box, obviously would like the HBase guys input on this before proceeding. > HBase won't start when hbase.rootdir uses ViewFileSystem > > > Key: HBASE-6067 > URL: https://issues.apache.org/jira/browse/HBASE-6067 > Project: HBase > Issue Type: Improvement > Components: regionserver >Reporter: Eli Collins > > HBase currently doesn't work with HDFS federation (hbase.rootdir with a > client that uses viewfs) because HLog#init uses > FileSystem#getDefaultBlockSize and getDefaultReplication. These throw an > exception because there is no default filesystem in a viewfs client so > there's no way to determine a default block size or replication factor. They > could use the versions of these methods that take a path, however these were > introduced in HADOOP-8014 and are not yet available in Hadoop 1.x. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6067) HBase won't start when hbase.rootdir uses ViewFileSystem
[
https://issues.apache.org/jira/browse/HBASE-6067?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13281807#comment-13281807
]
Daryn Sharp commented on HBASE-6067:
Does {{hbase.rootdir}} point to just the root directory of viewfs, or is it a
lower level directory? If the latter, I'm assuming this is how you could use
the path-based variants after we fix 1.x?
> HBase won't start when hbase.rootdir uses ViewFileSystem
>
>
> Key: HBASE-6067
> URL: https://issues.apache.org/jira/browse/HBASE-6067
> Project: HBase
> Issue Type: Improvement
> Components: regionserver
>Reporter: Eli Collins
>
> HBase currently doesn't work with HDFS federation (hbase.rootdir with a
> client that uses viewfs) because HLog#init uses
> FileSystem#getDefaultBlockSize and getDefaultReplication. These throw an
> exception because there is no default filesystem in a viewfs client so
> there's no way to determine a default block size or replication factor. They
> could use the versions of these methods that take a path, however these were
> introduced in HADOOP-8014 and are not yet available in Hadoop 1.x.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6067) HBase won't start when hbase.rootdir uses ViewFileSystem
[ https://issues.apache.org/jira/browse/HBASE-6067?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13281745#comment-13281745 ] Eli Collins commented on HBASE-6067: Should mentioned that I considered defining hbase.regionserver.hlog.blocksize and hbase.regionserver.hlog.tolerable.lowreplication in hbase-default.xml and use constants in the code like the other parameters but then (1) you can have a mismatch between hbase and hdfs' parameter values and (2) HBase supports non-HDFS file systems which may want different default values. > HBase won't start when hbase.rootdir uses ViewFileSystem > > > Key: HBASE-6067 > URL: https://issues.apache.org/jira/browse/HBASE-6067 > Project: HBase > Issue Type: Improvement > Components: regionserver >Reporter: Eli Collins > > HBase currently doesn't work with HDFS federation (hbase.rootdir with a > client that uses viewfs) because HLog#init uses > FileSystem#getDefaultBlockSize and getDefaultReplication. These throw an > exception because there is no default filesystem in a viewfs client so > there's no way to determine a default block size or replication factor. They > could use the versions of these methods that take a path, however these were > introduced in HADOOP-8014 and are not yet available in Hadoop 1.x. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6067) HBase won't start when hbase.rootdir uses ViewFileSystem
[
https://issues.apache.org/jira/browse/HBASE-6067?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13281276#comment-13281276
]
Eli Collins commented on HBASE-6067:
Here's the exception btw when configuring hbase.rootdir to use viewfs:
{noformat}
2012-05-17 14:02:14,327 INFO
org.apache.hadoop.hbase.regionserver.HRegionServer: STOPPED: Failed
initialization
2012-05-17 14:02:14,328 ERROR
org.apache.hadoop.hbase.regionserver.HRegionServer: Failed init
org.apache.hadoop.fs.viewfs.NotInMountpointException: getDefaultBlockSize on
empty path is invalid
at
org.apache.hadoop.fs.viewfs.ViewFileSystem.getDefaultBlockSize(ViewFileSystem.java:488)
at org.apache.hadoop.hbase.regionserver.wal.HLog.(HLog.java:377)
at org.apache.hadoop.hbase.regionserver.wal.HLog.(HLog.java:341)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.instantiateHLog(HRegionServer.java:1229)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.setupWALAndReplication(HRegionServer.java:1218)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.handleReportForDutyResponse(HRegionServer.java:937)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:648)
at java.lang.Thread.run(Thread.java:662)
{noformat}
> HBase won't start when hbase.rootdir uses ViewFileSystem
>
>
> Key: HBASE-6067
> URL: https://issues.apache.org/jira/browse/HBASE-6067
> Project: HBase
> Issue Type: Improvement
> Components: regionserver
>Reporter: Eli Collins
>
> HBase currently doesn't work with HDFS federation (hbase.rootdir with a
> client that uses viewfs) because HLog#init uses
> FileSystem#getDefaultBlockSize and getDefaultReplication. These throw an
> exception because there is no default filesystem in a viewfs client so
> there's no way to determine a default block size or replication factor. They
> could use the versions of these methods that take a path, however these were
> introduced in HADOOP-8014 and are not yet available in Hadoop 1.x.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
