[jira] [Updated] (HDFS-6210) Support GETACLSTATUS operation in WebImageViewer
[
https://issues.apache.org/jira/browse/HDFS-6210?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Akira AJISAKA updated HDFS-6210:
Attachment: HDFS-6210.2.patch
> Support GETACLSTATUS operation in WebImageViewer
>
>
> Key: HDFS-6210
> URL: https://issues.apache.org/jira/browse/HDFS-6210
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: tools
>Affects Versions: 2.5.0
>Reporter: Akira AJISAKA
>Assignee: Akira AJISAKA
> Attachments: HDFS-6210.2.patch, HDFS-6210.patch, HDFS-6210.patch
>
>
> In HDFS-6170, I found {{GETACLSTATUS}} operation support is also required to
> execute "hdfs dfs -ls" to WebImageViewer.
> {code}
> [root@trunk ~]# hdfs dfs -ls webhdfs://localhost:5978/
> 14/04/09 11:53:04 WARN util.NativeCodeLoader: Unable to load native-hadoop
> library for your platform... using builtin-java classes where applicable
> Found 1 items
> ls: Unexpected HTTP response: code=400 != 200, op=GETACLSTATUS, message=Bad
> Request
> {code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HDFS-6210) Support GETACLSTATUS operation in WebImageViewer
[
https://issues.apache.org/jira/browse/HDFS-6210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13973892#comment-13973892
]
Akira AJISAKA commented on HDFS-6210:
-
Thanks [~wheat9] for the comment. I updated the patch.
> Support GETACLSTATUS operation in WebImageViewer
>
>
> Key: HDFS-6210
> URL: https://issues.apache.org/jira/browse/HDFS-6210
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: tools
>Affects Versions: 2.5.0
>Reporter: Akira AJISAKA
>Assignee: Akira AJISAKA
> Attachments: HDFS-6210.2.patch, HDFS-6210.patch, HDFS-6210.patch
>
>
> In HDFS-6170, I found {{GETACLSTATUS}} operation support is also required to
> execute "hdfs dfs -ls" to WebImageViewer.
> {code}
> [root@trunk ~]# hdfs dfs -ls webhdfs://localhost:5978/
> 14/04/09 11:53:04 WARN util.NativeCodeLoader: Unable to load native-hadoop
> library for your platform... using builtin-java classes where applicable
> Found 1 items
> ls: Unexpected HTTP response: code=400 != 200, op=GETACLSTATUS, message=Bad
> Request
> {code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HDFS-6210) Support GETACLSTATUS operation in WebImageViewer
[
https://issues.apache.org/jira/browse/HDFS-6210?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Akira AJISAKA updated HDFS-6210:
Attachment: HDFS-6210.3.patch
Removed unnecessarily logging from {{testWebImageViewerForAcl()}}.
> Support GETACLSTATUS operation in WebImageViewer
>
>
> Key: HDFS-6210
> URL: https://issues.apache.org/jira/browse/HDFS-6210
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: tools
>Affects Versions: 2.5.0
>Reporter: Akira AJISAKA
>Assignee: Akira AJISAKA
> Attachments: HDFS-6210.2.patch, HDFS-6210.3.patch, HDFS-6210.patch,
> HDFS-6210.patch
>
>
> In HDFS-6170, I found {{GETACLSTATUS}} operation support is also required to
> execute "hdfs dfs -ls" to WebImageViewer.
> {code}
> [root@trunk ~]# hdfs dfs -ls webhdfs://localhost:5978/
> 14/04/09 11:53:04 WARN util.NativeCodeLoader: Unable to load native-hadoop
> library for your platform... using builtin-java classes where applicable
> Found 1 items
> ls: Unexpected HTTP response: code=400 != 200, op=GETACLSTATUS, message=Bad
> Request
> {code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HDFS-6256) Clean up ImageVisitor and SpotCheckImageVisitor
[
https://issues.apache.org/jira/browse/HDFS-6256?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13973899#comment-13973899
]
Hadoop QA commented on HDFS-6256:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12640778/HDFS-6256.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-hdfs-project/hadoop-hdfs.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-HDFS-Build/6681//testReport/
Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/6681//console
This message is automatically generated.
> Clean up ImageVisitor and SpotCheckImageVisitor
> ---
>
> Key: HDFS-6256
> URL: https://issues.apache.org/jira/browse/HDFS-6256
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: tools
>Reporter: Akira AJISAKA
>Assignee: Akira AJISAKA
> Labels: newbie
> Attachments: HDFS-6256.patch
>
>
> Dead code in OfflineImageViewer was removed by HDFS-6158, but
> {{ImageVisitor.java}} and {{SpotCheckImageVisitor.java}} are still exist.
> They have become dead and should be removed.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HDFS-6256) Clean up ImageVisitor and SpotCheckImageVisitor
[
https://issues.apache.org/jira/browse/HDFS-6256?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13973902#comment-13973902
]
Akira AJISAKA commented on HDFS-6256:
-
The patch is to remove the dead code, so new tests are not needed.
> Clean up ImageVisitor and SpotCheckImageVisitor
> ---
>
> Key: HDFS-6256
> URL: https://issues.apache.org/jira/browse/HDFS-6256
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: tools
>Reporter: Akira AJISAKA
>Assignee: Akira AJISAKA
> Labels: newbie
> Attachments: HDFS-6256.patch
>
>
> Dead code in OfflineImageViewer was removed by HDFS-6158, but
> {{ImageVisitor.java}} and {{SpotCheckImageVisitor.java}} are still exist.
> They have become dead and should be removed.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HDFS-6153) Document "fileId" and "childrenNum" fields in the FileStatus Json schema
[ https://issues.apache.org/jira/browse/HDFS-6153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13973976#comment-13973976 ] Hudson commented on HDFS-6153: -- FAILURE: Integrated in Hadoop-Yarn-trunk #544 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk/544/]) HDFS-6153. Document "fileId" and "childrenNum" fields in the FileStatus Json schema (vinayakumarb: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1588258) * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/site/apt/WebHDFS.apt.vm > Document "fileId" and "childrenNum" fields in the FileStatus Json schema > > > Key: HDFS-6153 > URL: https://issues.apache.org/jira/browse/HDFS-6153 > Project: Hadoop HDFS > Issue Type: Bug > Components: documentation, webhdfs >Affects Versions: 2.3.0 >Reporter: Akira AJISAKA >Assignee: Akira AJISAKA >Priority: Minor > Labels: newbie > Fix For: 2.5.0 > > Attachments: HDFS-6153.patch, HDFS-6153.patch > > > Now WebHDFS returns FileStatus Json objects include "fileId" and > "childrenNum" fields but these fields are not documented. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HDFS-6210) Support GETACLSTATUS operation in WebImageViewer
[
https://issues.apache.org/jira/browse/HDFS-6210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13973986#comment-13973986
]
Hadoop QA commented on HDFS-6210:
-
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12640790/HDFS-6210.3.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-hdfs-project/hadoop-hdfs.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-HDFS-Build/6682//testReport/
Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/6682//console
This message is automatically generated.
> Support GETACLSTATUS operation in WebImageViewer
>
>
> Key: HDFS-6210
> URL: https://issues.apache.org/jira/browse/HDFS-6210
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: tools
>Affects Versions: 2.5.0
>Reporter: Akira AJISAKA
>Assignee: Akira AJISAKA
> Attachments: HDFS-6210.2.patch, HDFS-6210.3.patch, HDFS-6210.patch,
> HDFS-6210.patch
>
>
> In HDFS-6170, I found {{GETACLSTATUS}} operation support is also required to
> execute "hdfs dfs -ls" to WebImageViewer.
> {code}
> [root@trunk ~]# hdfs dfs -ls webhdfs://localhost:5978/
> 14/04/09 11:53:04 WARN util.NativeCodeLoader: Unable to load native-hadoop
> library for your platform... using builtin-java classes where applicable
> Found 1 items
> ls: Unexpected HTTP response: code=400 != 200, op=GETACLSTATUS, message=Bad
> Request
> {code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HDFS-6153) Document "fileId" and "childrenNum" fields in the FileStatus Json schema
[ https://issues.apache.org/jira/browse/HDFS-6153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13974073#comment-13974073 ] Hudson commented on HDFS-6153: -- FAILURE: Integrated in Hadoop-Hdfs-trunk #1736 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/1736/]) HDFS-6153. Document "fileId" and "childrenNum" fields in the FileStatus Json schema (vinayakumarb: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1588258) * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/site/apt/WebHDFS.apt.vm > Document "fileId" and "childrenNum" fields in the FileStatus Json schema > > > Key: HDFS-6153 > URL: https://issues.apache.org/jira/browse/HDFS-6153 > Project: Hadoop HDFS > Issue Type: Bug > Components: documentation, webhdfs >Affects Versions: 2.3.0 >Reporter: Akira AJISAKA >Assignee: Akira AJISAKA >Priority: Minor > Labels: newbie > Fix For: 2.5.0 > > Attachments: HDFS-6153.patch, HDFS-6153.patch > > > Now WebHDFS returns FileStatus Json objects include "fileId" and > "childrenNum" fields but these fields are not documented. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HDFS-2006) ability to support storing extended attributes per file
[ https://issues.apache.org/jira/browse/HDFS-2006?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Liu updated HDFS-2006: - Attachment: HDFS-XAttrs-Design-1.pdf > ability to support storing extended attributes per file > --- > > Key: HDFS-2006 > URL: https://issues.apache.org/jira/browse/HDFS-2006 > Project: Hadoop HDFS > Issue Type: Improvement > Components: namenode >Reporter: dhruba borthakur >Assignee: Yi Liu > Attachments: HDFS-XAttrs-Design-1.pdf, xattrs.patch > > > It would be nice if HDFS provides a feature to store extended attributes for > files, similar to the one described here: > http://en.wikipedia.org/wiki/Extended_file_attributes. > The challenge is that it has to be done in such a way that a site not using > this feature does not waste precious memory resources in the namenode. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HDFS-6153) Document "fileId" and "childrenNum" fields in the FileStatus Json schema
[ https://issues.apache.org/jira/browse/HDFS-6153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13974147#comment-13974147 ] Hudson commented on HDFS-6153: -- SUCCESS: Integrated in Hadoop-Mapreduce-trunk #1761 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1761/]) HDFS-6153. Document "fileId" and "childrenNum" fields in the FileStatus Json schema (vinayakumarb: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1588258) * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/site/apt/WebHDFS.apt.vm > Document "fileId" and "childrenNum" fields in the FileStatus Json schema > > > Key: HDFS-6153 > URL: https://issues.apache.org/jira/browse/HDFS-6153 > Project: Hadoop HDFS > Issue Type: Bug > Components: documentation, webhdfs >Affects Versions: 2.3.0 >Reporter: Akira AJISAKA >Assignee: Akira AJISAKA >Priority: Minor > Labels: newbie > Fix For: 2.5.0 > > Attachments: HDFS-6153.patch, HDFS-6153.patch > > > Now WebHDFS returns FileStatus Json objects include "fileId" and > "childrenNum" fields but these fields are not documented. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Created] (HDFS-6257) TestCacheDirectives#testExceedsCapacity fails occasionally in trunk
Ted Yu created HDFS-6257:
Summary: TestCacheDirectives#testExceedsCapacity fails
occasionally in trunk
Key: HDFS-6257
URL: https://issues.apache.org/jira/browse/HDFS-6257
Project: Hadoop HDFS
Issue Type: Test
Reporter: Ted Yu
Priority: Minor
>From https://builds.apache.org/job/Hadoop-Hdfs-trunk/1736/ :
REGRESSION:
org.apache.hadoop.hdfs.server.namenode.TestCacheDirectives.testExceedsCapacity
{code}
Error Message:
Namenode should not send extra CACHE commands expected:<0> but was:<2>
Stack Trace:
java.lang.AssertionError: Namenode should not send extra CACHE commands
expected:<0> but was:<2>
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:472)
at
org.apache.hadoop.hdfs.server.namenode.TestCacheDirectives.testExceedsCapacity(TestCacheDirectives.java:1419)
{code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Created] (HDFS-6258) Implementation of extended Attributes in HDFS
Yi Liu created HDFS-6258: Summary: Implementation of extended Attributes in HDFS Key: HDFS-6258 URL: https://issues.apache.org/jira/browse/HDFS-6258 Project: Hadoop HDFS Issue Type: New Feature Components: namenode Reporter: Yi Liu Assignee: Yi Liu This JIRA is to implement extended attributes in HDFS: support XAttrs from NameNode, implements XAttr APIs for DistributedFileSystem and so on. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HDFS-6258) Implementation of extended Attributes in HDFS
[ https://issues.apache.org/jira/browse/HDFS-6258?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Liu updated HDFS-6258: - Issue Type: Sub-task (was: New Feature) Parent: HDFS-2006 > Implementation of extended Attributes in HDFS > - > > Key: HDFS-6258 > URL: https://issues.apache.org/jira/browse/HDFS-6258 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: namenode >Reporter: Yi Liu >Assignee: Yi Liu > Attachments: HDFS-6258.patch > > > This JIRA is to implement extended attributes in HDFS: support XAttrs from > NameNode, implements XAttr APIs for DistributedFileSystem and so on. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HDFS-2006) ability to support storing extended attributes per file
[ https://issues.apache.org/jira/browse/HDFS-2006?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Liu updated HDFS-2006: - Attachment: xattrs.1.patch > ability to support storing extended attributes per file > --- > > Key: HDFS-2006 > URL: https://issues.apache.org/jira/browse/HDFS-2006 > Project: Hadoop HDFS > Issue Type: Improvement > Components: namenode >Reporter: dhruba borthakur >Assignee: Yi Liu > Attachments: HDFS-XAttrs-Design-1.pdf, xattrs.1.patch, xattrs.patch > > > It would be nice if HDFS provides a feature to store extended attributes for > files, similar to the one described here: > http://en.wikipedia.org/wiki/Extended_file_attributes. > The challenge is that it has to be done in such a way that a site not using > this feature does not waste precious memory resources in the namenode. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HDFS-6258) Implementation of extended Attributes in HDFS
[ https://issues.apache.org/jira/browse/HDFS-6258?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Liu updated HDFS-6258: - Attachment: HDFS-6258.patch > Implementation of extended Attributes in HDFS > - > > Key: HDFS-6258 > URL: https://issues.apache.org/jira/browse/HDFS-6258 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: namenode >Reporter: Yi Liu >Assignee: Yi Liu > Attachments: HDFS-6258.patch > > > This JIRA is to implement extended attributes in HDFS: support XAttrs from > NameNode, implements XAttr APIs for DistributedFileSystem and so on. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Work started] (HDFS-6258) Implementation of extended Attributes in HDFS
[ https://issues.apache.org/jira/browse/HDFS-6258?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HDFS-6258 started by Yi Liu. > Implementation of extended Attributes in HDFS > - > > Key: HDFS-6258 > URL: https://issues.apache.org/jira/browse/HDFS-6258 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: namenode >Reporter: Yi Liu >Assignee: Yi Liu > Attachments: HDFS-6258.patch > > > This JIRA is to implement extended attributes in HDFS: support XAttrs from > NameNode, implements XAttr APIs for DistributedFileSystem and so on. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Created] (HDFS-6259) Support extended attributes via WebHDFS/HttpFS
Yi Liu created HDFS-6259: Summary: Support extended attributes via WebHDFS/HttpFS Key: HDFS-6259 URL: https://issues.apache.org/jira/browse/HDFS-6259 Project: Hadoop HDFS Issue Type: New Feature Reporter: Yi Liu Assignee: Yi Liu -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HDFS-6259) Support extended attributes via WebHDFS/HttpFS
[ https://issues.apache.org/jira/browse/HDFS-6259?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Liu updated HDFS-6259: - Issue Type: Sub-task (was: New Feature) Parent: HDFS-2006 > Support extended attributes via WebHDFS/HttpFS > -- > > Key: HDFS-6259 > URL: https://issues.apache.org/jira/browse/HDFS-6259 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Yi Liu >Assignee: Yi Liu > -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Created] (HDFS-6260) Support extended attributes via libHDFS
Yi Liu created HDFS-6260: Summary: Support extended attributes via libHDFS Key: HDFS-6260 URL: https://issues.apache.org/jira/browse/HDFS-6260 Project: Hadoop HDFS Issue Type: New Feature Components: libhdfs Reporter: Yi Liu Assignee: Yi Liu -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HDFS-6260) Support extended attributes via libHDFS
[ https://issues.apache.org/jira/browse/HDFS-6260?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Liu updated HDFS-6260: - Issue Type: Sub-task (was: New Feature) Parent: HDFS-2006 > Support extended attributes via libHDFS > --- > > Key: HDFS-6260 > URL: https://issues.apache.org/jira/browse/HDFS-6260 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: libhdfs >Reporter: Yi Liu >Assignee: Yi Liu > -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HDFS-6259) Support extended attributes via WebHDFS/HttpFS
[ https://issues.apache.org/jira/browse/HDFS-6259?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Liu updated HDFS-6259: - Component/s: webhdfs > Support extended attributes via WebHDFS/HttpFS > -- > > Key: HDFS-6259 > URL: https://issues.apache.org/jira/browse/HDFS-6259 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: webhdfs >Reporter: Yi Liu >Assignee: Yi Liu > -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HDFS-2006) ability to support storing extended attributes per file
[ https://issues.apache.org/jira/browse/HDFS-2006?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Liu updated HDFS-2006: - Status: Patch Available (was: Open) > ability to support storing extended attributes per file > --- > > Key: HDFS-2006 > URL: https://issues.apache.org/jira/browse/HDFS-2006 > Project: Hadoop HDFS > Issue Type: Improvement > Components: namenode >Reporter: dhruba borthakur >Assignee: Yi Liu > Attachments: HDFS-XAttrs-Design-1.pdf, xattrs.1.patch, xattrs.patch > > > It would be nice if HDFS provides a feature to store extended attributes for > files, similar to the one described here: > http://en.wikipedia.org/wiki/Extended_file_attributes. > The challenge is that it has to be done in such a way that a site not using > this feature does not waste precious memory resources in the namenode. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HDFS-6258) Support XAttrs from NameNode and implements XAttr APIs for DistributedFileSystem
[ https://issues.apache.org/jira/browse/HDFS-6258?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Liu updated HDFS-6258: - Summary: Support XAttrs from NameNode and implements XAttr APIs for DistributedFileSystem (was: Implementation of extended Attributes in HDFS) > Support XAttrs from NameNode and implements XAttr APIs for > DistributedFileSystem > > > Key: HDFS-6258 > URL: https://issues.apache.org/jira/browse/HDFS-6258 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: namenode >Reporter: Yi Liu >Assignee: Yi Liu > Attachments: HDFS-6258.patch > > > This JIRA is to implement extended attributes in HDFS: support XAttrs from > NameNode, implements XAttr APIs for DistributedFileSystem and so on. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HDFS-6256) Clean up ImageVisitor and SpotCheckImageVisitor
[
https://issues.apache.org/jira/browse/HDFS-6256?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13974281#comment-13974281
]
Haohui Mai commented on HDFS-6256:
--
+1
> Clean up ImageVisitor and SpotCheckImageVisitor
> ---
>
> Key: HDFS-6256
> URL: https://issues.apache.org/jira/browse/HDFS-6256
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: tools
>Reporter: Akira AJISAKA
>Assignee: Akira AJISAKA
> Labels: newbie
> Attachments: HDFS-6256.patch
>
>
> Dead code in OfflineImageViewer was removed by HDFS-6158, but
> {{ImageVisitor.java}} and {{SpotCheckImageVisitor.java}} are still exist.
> They have become dead and should be removed.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HDFS-6256) Clean up ImageVisitor and SpotCheckImageVisitor
[
https://issues.apache.org/jira/browse/HDFS-6256?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Haohui Mai updated HDFS-6256:
-
Resolution: Fixed
Fix Version/s: 2.5.0
Hadoop Flags: Reviewed
Status: Resolved (was: Patch Available)
I've committed the patch to trunk and branch-2. Thanks [~ajisakaa] for the
contribution.
> Clean up ImageVisitor and SpotCheckImageVisitor
> ---
>
> Key: HDFS-6256
> URL: https://issues.apache.org/jira/browse/HDFS-6256
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: tools
>Reporter: Akira AJISAKA
>Assignee: Akira AJISAKA
> Labels: newbie
> Fix For: 2.5.0
>
> Attachments: HDFS-6256.patch
>
>
> Dead code in OfflineImageViewer was removed by HDFS-6158, but
> {{ImageVisitor.java}} and {{SpotCheckImageVisitor.java}} are still exist.
> They have become dead and should be removed.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HDFS-6256) Clean up ImageVisitor and SpotCheckImageVisitor
[
https://issues.apache.org/jira/browse/HDFS-6256?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13974295#comment-13974295
]
Hudson commented on HDFS-6256:
--
SUCCESS: Integrated in Hadoop-trunk-Commit #5536 (See
[https://builds.apache.org/job/Hadoop-trunk-Commit/5536/])
HDFS-6256. Clean up ImageVisitor and SpotCheckImageVisitor. Contributed by
Akira Ajisaka. (wheat9:
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1588524)
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
*
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/offlineImageViewer/ImageVisitor.java
*
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/tools/offlineImageViewer/SpotCheckImageVisitor.java
> Clean up ImageVisitor and SpotCheckImageVisitor
> ---
>
> Key: HDFS-6256
> URL: https://issues.apache.org/jira/browse/HDFS-6256
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: tools
>Reporter: Akira AJISAKA
>Assignee: Akira AJISAKA
> Labels: newbie
> Fix For: 2.5.0
>
> Attachments: HDFS-6256.patch
>
>
> Dead code in OfflineImageViewer was removed by HDFS-6158, but
> {{ImageVisitor.java}} and {{SpotCheckImageVisitor.java}} are still exist.
> They have become dead and should be removed.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Moved] (HDFS-6261) Add document for enabling node group layer in HDFS
[ https://issues.apache.org/jira/browse/HDFS-6261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du moved HADOOP-10497 to HDFS-6261: --- Component/s: (was: documentation) documentation Fix Version/s: (was: site) Key: HDFS-6261 (was: HADOOP-10497) Project: Hadoop HDFS (was: Hadoop Common) > Add document for enabling node group layer in HDFS > -- > > Key: HDFS-6261 > URL: https://issues.apache.org/jira/browse/HDFS-6261 > Project: Hadoop HDFS > Issue Type: Task > Components: documentation >Reporter: Wenwu Peng >Assignee: Binglin Chang > Labels: documentation > > Most of patches from Umbrella JIRA HADOOP-8468 have committed, However there > is no site to introduce NodeGroup-aware(HADOOP Virtualization Extensisons) > and how to do configuration. so we need to doc it. > 1. Doc NodeGroup-aware relate in http://hadoop.apache.org/docs/current > 2. Doc NodeGroup-aware properties in core-default.xml. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HDFS-6261) Add document for enabling node group layer in HDFS
[ https://issues.apache.org/jira/browse/HDFS-6261?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13974349#comment-13974349 ] Junping Du commented on HDFS-6261: -- NodeGroup Topology feature is completed in HDFS part, but haven't finished in YARN (pending on YARN-18 and YARN-19). Agree we can do HDFS first, so move this JIRA to HDFS. > Add document for enabling node group layer in HDFS > -- > > Key: HDFS-6261 > URL: https://issues.apache.org/jira/browse/HDFS-6261 > Project: Hadoop HDFS > Issue Type: Task > Components: documentation >Reporter: Wenwu Peng >Assignee: Binglin Chang > Labels: documentation > > Most of patches from Umbrella JIRA HADOOP-8468 have committed, However there > is no site to introduce NodeGroup-aware(HADOOP Virtualization Extensisons) > and how to do configuration. so we need to doc it. > 1. Doc NodeGroup-aware relate in http://hadoop.apache.org/docs/current > 2. Doc NodeGroup-aware properties in core-default.xml. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HDFS-2006) ability to support storing extended attributes per file
[
https://issues.apache.org/jira/browse/HDFS-2006?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13974373#comment-13974373
]
Hadoop QA commented on HDFS-2006:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12640836/xattrs.1.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 3 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests in
hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs:
org.apache.hadoop.fs.TestHarFileSystem
org.apache.hadoop.hdfs.tools.offlineEditsViewer.TestOfflineEditsViewer
org.apache.hadoop.hdfs.server.namenode.ha.TestFailureToReadEdits
org.apache.hadoop.hdfs.protocol.TestLayoutVersion
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-HDFS-Build/6683//testReport/
Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/6683//console
This message is automatically generated.
> ability to support storing extended attributes per file
> ---
>
> Key: HDFS-2006
> URL: https://issues.apache.org/jira/browse/HDFS-2006
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: namenode
>Reporter: dhruba borthakur
>Assignee: Yi Liu
> Attachments: HDFS-XAttrs-Design-1.pdf, xattrs.1.patch, xattrs.patch
>
>
> It would be nice if HDFS provides a feature to store extended attributes for
> files, similar to the one described here:
> http://en.wikipedia.org/wiki/Extended_file_attributes.
> The challenge is that it has to be done in such a way that a site not using
> this feature does not waste precious memory resources in the namenode.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HDFS-5851) Support memory as a storage medium
[ https://issues.apache.org/jira/browse/HDFS-5851?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13974472#comment-13974472 ] Arpit Agarwal commented on HDFS-5851: - Hi Andrew, bq. There are also plans to move towards sub-block caching. Whole-block caching is wasteful for columnar formats like ORC and Parquet. With sub-block caching, automatic cache replacement looks a lot more attractive (another planned feature). These are both things we can support with HDFS-4949's infrastructure. I'm not sure about with HSM. Does CCM support block-level caching or is it just _cache all blocks in a file_? bq. Anyway, either pulling this towards HDFS-4949 or vice versa, we should figure out these details before moving ahead. I'll echo Colin's desire for a meeting to discuss this. We're willing to host at our Palo Alto office. Thanks for the offer! We are discussing use cases and proposed API at high level. I am not sure there is much overlap between CCM and our work and I expect them to solve different use cases. However I am also open to discussing how and whether we can align them more closely once we have shared our initial proposal. > Support memory as a storage medium > -- > > Key: HDFS-5851 > URL: https://issues.apache.org/jira/browse/HDFS-5851 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: datanode >Affects Versions: 3.0.0 >Reporter: Arpit Agarwal >Assignee: Arpit Agarwal > > Memory can be used as a storage medium for smaller/transient files for fast > write throughput. > More information/design will be added later. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HDFS-5851) Support memory as a storage medium
[ https://issues.apache.org/jira/browse/HDFS-5851?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13974489#comment-13974489 ] Andrew Wang commented on HDFS-5851: --- bq. Does CCM support block-level caching or is it just cache all blocks in a file? Right now, the user APIs are all in terms of files, but the backend could do single blocks pretty easily. We didn't add block-level cache directives because we feel automatic cache eviction is a better solution (and could operate at the block or subblock level). Looking forward to the design doc! Feel free to ping us even with preliminary usecases/ideas. > Support memory as a storage medium > -- > > Key: HDFS-5851 > URL: https://issues.apache.org/jira/browse/HDFS-5851 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: datanode >Affects Versions: 3.0.0 >Reporter: Arpit Agarwal >Assignee: Arpit Agarwal > > Memory can be used as a storage medium for smaller/transient files for fast > write throughput. > More information/design will be added later. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HDFS-6141) WebHdfsFileSystem#toUrl does not perform character escaping.
[
https://issues.apache.org/jira/browse/HDFS-6141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13974491#comment-13974491
]
Suresh Srinivas commented on HDFS-6141:
---
+1 for the patch.
> WebHdfsFileSystem#toUrl does not perform character escaping.
>
>
> Key: HDFS-6141
> URL: https://issues.apache.org/jira/browse/HDFS-6141
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: webhdfs
>Affects Versions: 1.2.1
>Reporter: Chris Nauroth
>Assignee: Chris Nauroth
> Attachments: HDFS-6141.1.patch
>
>
> On branch-1, {{WebHdfsFileSystem#toUrl}} does not perform character escaping,
> which will cause it to try to create invalid URLs for things like spaces in a
> file name. This bug does not exist on trunk or branch-2.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Resolved] (HDFS-6141) WebHdfsFileSystem#toUrl does not perform character escaping.
[
https://issues.apache.org/jira/browse/HDFS-6141?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chris Nauroth resolved HDFS-6141.
-
Resolution: Fixed
Fix Version/s: 1.3.0
Hadoop Flags: Reviewed
Thank you, Suresh. I committed this to branch-1.
> WebHdfsFileSystem#toUrl does not perform character escaping.
>
>
> Key: HDFS-6141
> URL: https://issues.apache.org/jira/browse/HDFS-6141
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: webhdfs
>Affects Versions: 1.2.1
>Reporter: Chris Nauroth
>Assignee: Chris Nauroth
> Fix For: 1.3.0
>
> Attachments: HDFS-6141.1.patch
>
>
> On branch-1, {{WebHdfsFileSystem#toUrl}} does not perform character escaping,
> which will cause it to try to create invalid URLs for things like spaces in a
> file name. This bug does not exist on trunk or branch-2.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Work started] (HDFS-6245) datanode fails to start with a bad disk even when failed volumes is set
[ https://issues.apache.org/jira/browse/HDFS-6245?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HDFS-6245 started by Arpit Agarwal. > datanode fails to start with a bad disk even when failed volumes is set > --- > > Key: HDFS-6245 > URL: https://issues.apache.org/jira/browse/HDFS-6245 > Project: Hadoop HDFS > Issue Type: Bug >Affects Versions: 2.4.0 >Reporter: Arpit Gupta >Assignee: Arpit Agarwal > > Data node startup failed even when failed volumes was set. Had to remove the > bad disk from the config to get it to boot. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HDFS-6245) datanode fails to start with a bad disk even when failed volumes is set
[ https://issues.apache.org/jira/browse/HDFS-6245?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arpit Agarwal updated HDFS-6245: Attachment: HDFS-6245.01.patch > datanode fails to start with a bad disk even when failed volumes is set > --- > > Key: HDFS-6245 > URL: https://issues.apache.org/jira/browse/HDFS-6245 > Project: Hadoop HDFS > Issue Type: Bug >Affects Versions: 2.4.0 >Reporter: Arpit Gupta >Assignee: Arpit Agarwal > Attachments: HDFS-6245.01.patch > > > Data node startup failed even when failed volumes was set. Had to remove the > bad disk from the config to get it to boot. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HDFS-6245) datanode fails to start with a bad disk even when failed volumes is set
[ https://issues.apache.org/jira/browse/HDFS-6245?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arpit Agarwal updated HDFS-6245: Status: Patch Available (was: In Progress) > datanode fails to start with a bad disk even when failed volumes is set > --- > > Key: HDFS-6245 > URL: https://issues.apache.org/jira/browse/HDFS-6245 > Project: Hadoop HDFS > Issue Type: Bug >Affects Versions: 2.4.0 >Reporter: Arpit Gupta >Assignee: Arpit Agarwal > Attachments: HDFS-6245.01.patch > > > Data node startup failed even when failed volumes was set. Had to remove the > bad disk from the config to get it to boot. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HDFS-6245) datanode fails to start with a bad disk even when failed volumes is set
[
https://issues.apache.org/jira/browse/HDFS-6245?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13974599#comment-13974599
]
Arpit Agarwal commented on HDFS-6245:
-
This is effectively a regression since 2.3. {{File.getCanonicalPath}} can throw
IOException which should be ignored by {{getStorageLocations}}.
URISyntaxException is already consumed by the callee.
> datanode fails to start with a bad disk even when failed volumes is set
> ---
>
> Key: HDFS-6245
> URL: https://issues.apache.org/jira/browse/HDFS-6245
> Project: Hadoop HDFS
> Issue Type: Bug
>Affects Versions: 2.4.0
>Reporter: Arpit Gupta
>Assignee: Arpit Agarwal
> Attachments: HDFS-6245.01.patch
>
>
> Data node startup failed even when failed volumes was set. Had to remove the
> bad disk from the config to get it to boot.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HDFS-6245) datanode fails to start with a bad disk even when failed volumes is set
[ https://issues.apache.org/jira/browse/HDFS-6245?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Nauroth updated HDFS-6245: Hadoop Flags: Reviewed +1 for the patch, pending Jenkins. Thanks, Arpit! > datanode fails to start with a bad disk even when failed volumes is set > --- > > Key: HDFS-6245 > URL: https://issues.apache.org/jira/browse/HDFS-6245 > Project: Hadoop HDFS > Issue Type: Bug >Affects Versions: 2.4.0 >Reporter: Arpit Gupta >Assignee: Arpit Agarwal > Attachments: HDFS-6245.01.patch > > > Data node startup failed even when failed volumes was set. Had to remove the > bad disk from the config to get it to boot. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HDFS-6245) datanode fails to start with a bad disk even when failed volumes is set
[
https://issues.apache.org/jira/browse/HDFS-6245?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13974676#comment-13974676
]
Hadoop QA commented on HDFS-6245:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12640908/HDFS-6245.01.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-hdfs-project/hadoop-hdfs.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-HDFS-Build/6684//testReport/
Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/6684//console
This message is automatically generated.
> datanode fails to start with a bad disk even when failed volumes is set
> ---
>
> Key: HDFS-6245
> URL: https://issues.apache.org/jira/browse/HDFS-6245
> Project: Hadoop HDFS
> Issue Type: Bug
>Affects Versions: 2.4.0
>Reporter: Arpit Gupta
>Assignee: Arpit Agarwal
> Attachments: HDFS-6245.01.patch
>
>
> Data node startup failed even when failed volumes was set. Had to remove the
> bad disk from the config to get it to boot.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
