[jira] [Updated] (HDFS-7902) Expose truncate API for libwebhdfs
[ https://issues.apache.org/jira/browse/HDFS-7902?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Liu updated HDFS-7902: - Attachment: HDFS-7902.001.patch Upload the patch. Will submit it after HDFS-7838. > Expose truncate API for libwebhdfs > -- > > Key: HDFS-7902 > URL: https://issues.apache.org/jira/browse/HDFS-7902 > Project: Hadoop HDFS > Issue Type: Improvement > Components: native, webhdfs >Affects Versions: 2.7.0 >Reporter: Yi Liu >Assignee: Yi Liu > Attachments: HDFS-7902.001.patch > > > As Colin suggested in HDFS-7838, we will add truncate support for libwebhdfs. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (HDFS-6037) TestIncrementalBlockReports#testReplaceReceivedBlock fails occasionally in trunk
[
https://issues.apache.org/jira/browse/HDFS-6037?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu resolved HDFS-6037.
--
Resolution: Cannot Reproduce
> TestIncrementalBlockReports#testReplaceReceivedBlock fails occasionally in
> trunk
>
>
> Key: HDFS-6037
> URL: https://issues.apache.org/jira/browse/HDFS-6037
> Project: Hadoop HDFS
> Issue Type: Test
>Reporter: Ted Yu
>
> From
> https://builds.apache.org/job/Hadoop-Hdfs-trunk/1688/testReport/junit/org.apache.hadoop.hdfs.server.datanode/TestIncrementalBlockReports/testReplaceReceivedBlock/
> :
> {code}
> datanodeProtocolClientSideTranslatorPB.blockReceivedAndDeleted(
> ,
> ,
>
> );
> Wanted 1 time:
> -> at
> org.apache.hadoop.hdfs.server.datanode.TestIncrementalBlockReports.testReplaceReceivedBlock(TestIncrementalBlockReports.java:198)
> But was 2 times. Undesired invocation:
> -> at
> org.apache.hadoop.hdfs.server.datanode.BPServiceActor.reportReceivedDeletedBlocks(BPServiceActor.java:303)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HDFS-7460) Rewrite httpfs to use new shell framework
[ https://issues.apache.org/jira/browse/HDFS-7460?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer updated HDFS-7460: --- Release Note: This deprecates the following environment variables: | Old | New | |: |: | | HTTPFS_LOG | HADOOP_LOG_DIR| | HTTPFS_CONFG | HADOOP_CONF_DIR | was: This deprecates the following environment variables: HTTPFS_CONFIG HTTPFS_LOG > Rewrite httpfs to use new shell framework > - > > Key: HDFS-7460 > URL: https://issues.apache.org/jira/browse/HDFS-7460 > Project: Hadoop HDFS > Issue Type: Improvement >Affects Versions: 3.0.0 >Reporter: Allen Wittenauer >Assignee: John Smith > Labels: security > Fix For: 3.0.0 > > Attachments: HDFS-7460-01.patch, HDFS-7460.patch > > > httpfs shell code was not rewritten during HADOOP-9902. It should be modified > to take advantage of the common shell framework. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-7885) Datanode should not trust the generation stamp provided by client
[ https://issues.apache.org/jira/browse/HDFS-7885?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14351637#comment-14351637 ] Hudson commented on HDFS-7885: -- SUCCESS: Integrated in Hadoop-Mapreduce-trunk #2075 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2075/]) HDFS-7885. Datanode should not trust the generation stamp provided by client. Contributed by Tsz Wo Nicholas Sze. (jing9: rev 24db0812be64e83a48ade01fc1eaaeaedad4dec0) * hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestBlockReaderLocalLegacy.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImpl.java > Datanode should not trust the generation stamp provided by client > - > > Key: HDFS-7885 > URL: https://issues.apache.org/jira/browse/HDFS-7885 > Project: Hadoop HDFS > Issue Type: Bug > Components: datanode >Affects Versions: 2.2.0 >Reporter: vitthal (Suhas) Gogate >Assignee: Tsz Wo Nicholas Sze >Priority: Critical > Fix For: 2.7.0 > > Attachments: h7885_20150305.patch, h7885_20150306.patch > > > Datanode should not trust the generation stamp provided by client, since it > is prefetched and buffered in client, and concurrent append may increase it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-6488) Support HDFS superuser in NFSv3 gateway
[
https://issues.apache.org/jira/browse/HDFS-6488?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14351643#comment-14351643
]
Hudson commented on HDFS-6488:
--
SUCCESS: Integrated in Hadoop-Mapreduce-trunk #2075 (See
[https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2075/])
HDFS-6488. Support HDFS superuser in NFSv3 gateway. Contributed by Brandon Li
(brandonli: rev 0f8ecb1d0ce6d3ee9a7caf5b15b299210c2b8875)
* hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
*
hadoop-hdfs-project/hadoop-hdfs-nfs/src/main/java/org/apache/hadoop/hdfs/nfs/nfs3/RpcProgramNfs3.java
*
hadoop-hdfs-project/hadoop-hdfs-nfs/src/main/java/org/apache/hadoop/hdfs/nfs/conf/NfsConfigKeys.java
* hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HdfsNfsGateway.md
> Support HDFS superuser in NFSv3 gateway
> ---
>
> Key: HDFS-6488
> URL: https://issues.apache.org/jira/browse/HDFS-6488
> Project: Hadoop HDFS
> Issue Type: New Feature
> Components: nfs
>Affects Versions: 2.3.0
>Reporter: Stephen Chu
>Assignee: Brandon Li
> Fix For: 2.7.0
>
> Attachments: HDFS-6488.001.patch, HDFS-6488.002.patch,
> HDFS-6488.003.patch
>
>
> As hdfs superuseruser on the NFS mount, I cannot cd or ls the
> /user/schu/.Trash directory:
> {code}
> bash-4.1$ cd .Trash/
> bash: cd: .Trash/: Permission denied
> bash-4.1$ ls -la
> total 2
> drwxr-xr-x 4 schu 2584148964 128 Jan 7 10:42 .
> drwxr-xr-x 4 hdfs 2584148964 128 Jan 6 16:59 ..
> drwx-- 2 schu 2584148964 64 Jan 7 10:45 .Trash
> drwxr-xr-x 2 hdfs hdfs64 Jan 7 10:42 tt
> bash-4.1$ ls .Trash
> ls: cannot open directory .Trash: Permission denied
> bash-4.1$
> {code}
> When using FsShell as hdfs superuser, I have superuser permissions to schu's
> .Trash contents:
> {code}
> bash-4.1$ hdfs dfs -ls -R /user/schu/.Trash
> drwx-- - schu supergroup 0 2014-01-07 10:48
> /user/schu/.Trash/Current
> drwx-- - schu supergroup 0 2014-01-07 10:48
> /user/schu/.Trash/Current/user
> drwx-- - schu supergroup 0 2014-01-07 10:48
> /user/schu/.Trash/Current/user/schu
> -rw-r--r-- 1 schu supergroup 4 2014-01-07 10:48
> /user/schu/.Trash/Current/user/schu/tf1
> {code}
> The NFSv3 logs don't produce any error when superuser tries to access schu
> Trash contents. However, for other permission errors (e.g. schu tries to
> delete a directory owned by hdfs), there will be a permission error in the
> logs.
> I think this is not specific to the .Trash directory perhaps.
> I created a /user/schu/dir1 which has the same permissions as .Trash (700).
> When I try cd'ing into the directory from the NFSv3 mount as hdfs superuser,
> I get the same permission denied.
> {code}
> [schu@hdfs-nfs ~]$ hdfs dfs -ls
> Found 4 items
> drwx-- - schu supergroup 0 2014-01-07 10:57 .Trash
> drwx-- - schu supergroup 0 2014-01-07 11:05 dir1
> -rw-r--r-- 1 schu supergroup 4 2014-01-07 11:05 tf1
> drwxr-xr-x - hdfs hdfs0 2014-01-07 10:42 tt
> bash-4.1$ whoami
> hdfs
> bash-4.1$ pwd
> /hdfs_nfs_mount/user/schu
> bash-4.1$ cd dir1
> bash: cd: dir1: Permission denied
> bash-4.1$
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-7818) OffsetParam should return the default value instead of throwing NPE when the value is unspecified
[
https://issues.apache.org/jira/browse/HDFS-7818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14351644#comment-14351644
]
Hudson commented on HDFS-7818:
--
SUCCESS: Integrated in Hadoop-Mapreduce-trunk #2075 (See
[https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2075/])
HDFS-7818. OffsetParam should return the default value instead of throwing NPE
when the value is unspecified. Contributed by Eric Payne. (wheat9: rev
c79710302ee51e1a9ee17dadb161c69bb3aba5c9)
*
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/resources/OffsetParam.java
*
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/web/webhdfs/ParameterParser.java
*
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/web/webhdfs/TestParameterParser.java
* hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
> OffsetParam should return the default value instead of throwing NPE when the
> value is unspecified
> -
>
> Key: HDFS-7818
> URL: https://issues.apache.org/jira/browse/HDFS-7818
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: webhdfs
>Affects Versions: 2.7.0
>Reporter: Eric Payne
>Assignee: Eric Payne
>Priority: Blocker
> Fix For: 2.7.0
>
> Attachments: HDFS-7818.v1.txt, HDFS-7818.v2.txt, HDFS-7818.v3.txt,
> HDFS-7818.v4.txt, HDFS-7818.v5.txt
>
>
> This is a regression in 2.7 and later.
> {{hadoop fs -cat}} over webhdfs works, but {{hadoop fs -text}} does not:
> {code}
> $ hadoop fs -cat webhdfs://myhost.com/tmp/test.1
> ... output ...
> $ hadoop fs -text webhdfs://myhost.com/tmp/test.1
> text: org.apache.hadoop.ipc.RemoteException(java.lang.NullPointerException):
> null
> at
> org.apache.hadoop.hdfs.web.JsonUtil.toRemoteException(JsonUtil.java:165)
> at
> org.apache.hadoop.hdfs.web.WebHdfsFileSystem.validateResponse(WebHdfsFileSystem.java:358)
> at
> org.apache.hadoop.hdfs.web.WebHdfsFileSystem.access$200(WebHdfsFileSystem.java:91)
> at
> org.apache.hadoop.hdfs.web.WebHdfsFileSystem$AbstractRunner.runWithRetry(WebHdfsFileSystem.java:615)
> at
> org.apache.hadoop.hdfs.web.WebHdfsFileSystem$AbstractRunner.access$100(WebHdfsFileSystem.java:463)
> at
> org.apache.hadoop.hdfs.web.WebHdfsFileSystem$AbstractRunner$1.run(WebHdfsFileSystem.java:492)
> ...
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-7818) OffsetParam should return the default value instead of throwing NPE when the value is unspecified
[
https://issues.apache.org/jira/browse/HDFS-7818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14351626#comment-14351626
]
Hudson commented on HDFS-7818:
--
FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #125 (See
[https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/125/])
HDFS-7818. OffsetParam should return the default value instead of throwing NPE
when the value is unspecified. Contributed by Eric Payne. (wheat9: rev
c79710302ee51e1a9ee17dadb161c69bb3aba5c9)
*
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/web/webhdfs/TestParameterParser.java
*
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/resources/OffsetParam.java
* hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
*
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/web/webhdfs/ParameterParser.java
> OffsetParam should return the default value instead of throwing NPE when the
> value is unspecified
> -
>
> Key: HDFS-7818
> URL: https://issues.apache.org/jira/browse/HDFS-7818
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: webhdfs
>Affects Versions: 2.7.0
>Reporter: Eric Payne
>Assignee: Eric Payne
>Priority: Blocker
> Fix For: 2.7.0
>
> Attachments: HDFS-7818.v1.txt, HDFS-7818.v2.txt, HDFS-7818.v3.txt,
> HDFS-7818.v4.txt, HDFS-7818.v5.txt
>
>
> This is a regression in 2.7 and later.
> {{hadoop fs -cat}} over webhdfs works, but {{hadoop fs -text}} does not:
> {code}
> $ hadoop fs -cat webhdfs://myhost.com/tmp/test.1
> ... output ...
> $ hadoop fs -text webhdfs://myhost.com/tmp/test.1
> text: org.apache.hadoop.ipc.RemoteException(java.lang.NullPointerException):
> null
> at
> org.apache.hadoop.hdfs.web.JsonUtil.toRemoteException(JsonUtil.java:165)
> at
> org.apache.hadoop.hdfs.web.WebHdfsFileSystem.validateResponse(WebHdfsFileSystem.java:358)
> at
> org.apache.hadoop.hdfs.web.WebHdfsFileSystem.access$200(WebHdfsFileSystem.java:91)
> at
> org.apache.hadoop.hdfs.web.WebHdfsFileSystem$AbstractRunner.runWithRetry(WebHdfsFileSystem.java:615)
> at
> org.apache.hadoop.hdfs.web.WebHdfsFileSystem$AbstractRunner.access$100(WebHdfsFileSystem.java:463)
> at
> org.apache.hadoop.hdfs.web.WebHdfsFileSystem$AbstractRunner$1.run(WebHdfsFileSystem.java:492)
> ...
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-7885) Datanode should not trust the generation stamp provided by client
[ https://issues.apache.org/jira/browse/HDFS-7885?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14351619#comment-14351619 ] Hudson commented on HDFS-7885: -- FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #125 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/125/]) HDFS-7885. Datanode should not trust the generation stamp provided by client. Contributed by Tsz Wo Nicholas Sze. (jing9: rev 24db0812be64e83a48ade01fc1eaaeaedad4dec0) * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestBlockReaderLocalLegacy.java * hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImpl.java > Datanode should not trust the generation stamp provided by client > - > > Key: HDFS-7885 > URL: https://issues.apache.org/jira/browse/HDFS-7885 > Project: Hadoop HDFS > Issue Type: Bug > Components: datanode >Affects Versions: 2.2.0 >Reporter: vitthal (Suhas) Gogate >Assignee: Tsz Wo Nicholas Sze >Priority: Critical > Fix For: 2.7.0 > > Attachments: h7885_20150305.patch, h7885_20150306.patch > > > Datanode should not trust the generation stamp provided by client, since it > is prefetched and buffered in client, and concurrent append may increase it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-6488) Support HDFS superuser in NFSv3 gateway
[
https://issues.apache.org/jira/browse/HDFS-6488?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14351625#comment-14351625
]
Hudson commented on HDFS-6488:
--
FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #125 (See
[https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/125/])
HDFS-6488. Support HDFS superuser in NFSv3 gateway. Contributed by Brandon Li
(brandonli: rev 0f8ecb1d0ce6d3ee9a7caf5b15b299210c2b8875)
*
hadoop-hdfs-project/hadoop-hdfs-nfs/src/main/java/org/apache/hadoop/hdfs/nfs/nfs3/RpcProgramNfs3.java
* hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HdfsNfsGateway.md
*
hadoop-hdfs-project/hadoop-hdfs-nfs/src/main/java/org/apache/hadoop/hdfs/nfs/conf/NfsConfigKeys.java
* hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
> Support HDFS superuser in NFSv3 gateway
> ---
>
> Key: HDFS-6488
> URL: https://issues.apache.org/jira/browse/HDFS-6488
> Project: Hadoop HDFS
> Issue Type: New Feature
> Components: nfs
>Affects Versions: 2.3.0
>Reporter: Stephen Chu
>Assignee: Brandon Li
> Fix For: 2.7.0
>
> Attachments: HDFS-6488.001.patch, HDFS-6488.002.patch,
> HDFS-6488.003.patch
>
>
> As hdfs superuseruser on the NFS mount, I cannot cd or ls the
> /user/schu/.Trash directory:
> {code}
> bash-4.1$ cd .Trash/
> bash: cd: .Trash/: Permission denied
> bash-4.1$ ls -la
> total 2
> drwxr-xr-x 4 schu 2584148964 128 Jan 7 10:42 .
> drwxr-xr-x 4 hdfs 2584148964 128 Jan 6 16:59 ..
> drwx-- 2 schu 2584148964 64 Jan 7 10:45 .Trash
> drwxr-xr-x 2 hdfs hdfs64 Jan 7 10:42 tt
> bash-4.1$ ls .Trash
> ls: cannot open directory .Trash: Permission denied
> bash-4.1$
> {code}
> When using FsShell as hdfs superuser, I have superuser permissions to schu's
> .Trash contents:
> {code}
> bash-4.1$ hdfs dfs -ls -R /user/schu/.Trash
> drwx-- - schu supergroup 0 2014-01-07 10:48
> /user/schu/.Trash/Current
> drwx-- - schu supergroup 0 2014-01-07 10:48
> /user/schu/.Trash/Current/user
> drwx-- - schu supergroup 0 2014-01-07 10:48
> /user/schu/.Trash/Current/user/schu
> -rw-r--r-- 1 schu supergroup 4 2014-01-07 10:48
> /user/schu/.Trash/Current/user/schu/tf1
> {code}
> The NFSv3 logs don't produce any error when superuser tries to access schu
> Trash contents. However, for other permission errors (e.g. schu tries to
> delete a directory owned by hdfs), there will be a permission error in the
> logs.
> I think this is not specific to the .Trash directory perhaps.
> I created a /user/schu/dir1 which has the same permissions as .Trash (700).
> When I try cd'ing into the directory from the NFSv3 mount as hdfs superuser,
> I get the same permission denied.
> {code}
> [schu@hdfs-nfs ~]$ hdfs dfs -ls
> Found 4 items
> drwx-- - schu supergroup 0 2014-01-07 10:57 .Trash
> drwx-- - schu supergroup 0 2014-01-07 11:05 dir1
> -rw-r--r-- 1 schu supergroup 4 2014-01-07 11:05 tf1
> drwxr-xr-x - hdfs hdfs0 2014-01-07 10:42 tt
> bash-4.1$ whoami
> hdfs
> bash-4.1$ pwd
> /hdfs_nfs_mount/user/schu
> bash-4.1$ cd dir1
> bash: cd: dir1: Permission denied
> bash-4.1$
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-6488) Support HDFS superuser in NFSv3 gateway
[
https://issues.apache.org/jira/browse/HDFS-6488?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14351612#comment-14351612
]
Hudson commented on HDFS-6488:
--
SUCCESS: Integrated in Hadoop-Hdfs-trunk-Java8 #116 (See
[https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/116/])
HDFS-6488. Support HDFS superuser in NFSv3 gateway. Contributed by Brandon Li
(brandonli: rev 0f8ecb1d0ce6d3ee9a7caf5b15b299210c2b8875)
* hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HdfsNfsGateway.md
* hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
*
hadoop-hdfs-project/hadoop-hdfs-nfs/src/main/java/org/apache/hadoop/hdfs/nfs/nfs3/RpcProgramNfs3.java
*
hadoop-hdfs-project/hadoop-hdfs-nfs/src/main/java/org/apache/hadoop/hdfs/nfs/conf/NfsConfigKeys.java
> Support HDFS superuser in NFSv3 gateway
> ---
>
> Key: HDFS-6488
> URL: https://issues.apache.org/jira/browse/HDFS-6488
> Project: Hadoop HDFS
> Issue Type: New Feature
> Components: nfs
>Affects Versions: 2.3.0
>Reporter: Stephen Chu
>Assignee: Brandon Li
> Fix For: 2.7.0
>
> Attachments: HDFS-6488.001.patch, HDFS-6488.002.patch,
> HDFS-6488.003.patch
>
>
> As hdfs superuseruser on the NFS mount, I cannot cd or ls the
> /user/schu/.Trash directory:
> {code}
> bash-4.1$ cd .Trash/
> bash: cd: .Trash/: Permission denied
> bash-4.1$ ls -la
> total 2
> drwxr-xr-x 4 schu 2584148964 128 Jan 7 10:42 .
> drwxr-xr-x 4 hdfs 2584148964 128 Jan 6 16:59 ..
> drwx-- 2 schu 2584148964 64 Jan 7 10:45 .Trash
> drwxr-xr-x 2 hdfs hdfs64 Jan 7 10:42 tt
> bash-4.1$ ls .Trash
> ls: cannot open directory .Trash: Permission denied
> bash-4.1$
> {code}
> When using FsShell as hdfs superuser, I have superuser permissions to schu's
> .Trash contents:
> {code}
> bash-4.1$ hdfs dfs -ls -R /user/schu/.Trash
> drwx-- - schu supergroup 0 2014-01-07 10:48
> /user/schu/.Trash/Current
> drwx-- - schu supergroup 0 2014-01-07 10:48
> /user/schu/.Trash/Current/user
> drwx-- - schu supergroup 0 2014-01-07 10:48
> /user/schu/.Trash/Current/user/schu
> -rw-r--r-- 1 schu supergroup 4 2014-01-07 10:48
> /user/schu/.Trash/Current/user/schu/tf1
> {code}
> The NFSv3 logs don't produce any error when superuser tries to access schu
> Trash contents. However, for other permission errors (e.g. schu tries to
> delete a directory owned by hdfs), there will be a permission error in the
> logs.
> I think this is not specific to the .Trash directory perhaps.
> I created a /user/schu/dir1 which has the same permissions as .Trash (700).
> When I try cd'ing into the directory from the NFSv3 mount as hdfs superuser,
> I get the same permission denied.
> {code}
> [schu@hdfs-nfs ~]$ hdfs dfs -ls
> Found 4 items
> drwx-- - schu supergroup 0 2014-01-07 10:57 .Trash
> drwx-- - schu supergroup 0 2014-01-07 11:05 dir1
> -rw-r--r-- 1 schu supergroup 4 2014-01-07 11:05 tf1
> drwxr-xr-x - hdfs hdfs0 2014-01-07 10:42 tt
> bash-4.1$ whoami
> hdfs
> bash-4.1$ pwd
> /hdfs_nfs_mount/user/schu
> bash-4.1$ cd dir1
> bash: cd: dir1: Permission denied
> bash-4.1$
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-7885) Datanode should not trust the generation stamp provided by client
[ https://issues.apache.org/jira/browse/HDFS-7885?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14351606#comment-14351606 ] Hudson commented on HDFS-7885: -- SUCCESS: Integrated in Hadoop-Hdfs-trunk-Java8 #116 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/116/]) HDFS-7885. Datanode should not trust the generation stamp provided by client. Contributed by Tsz Wo Nicholas Sze. (jing9: rev 24db0812be64e83a48ade01fc1eaaeaedad4dec0) * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImpl.java * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestBlockReaderLocalLegacy.java * hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt > Datanode should not trust the generation stamp provided by client > - > > Key: HDFS-7885 > URL: https://issues.apache.org/jira/browse/HDFS-7885 > Project: Hadoop HDFS > Issue Type: Bug > Components: datanode >Affects Versions: 2.2.0 >Reporter: vitthal (Suhas) Gogate >Assignee: Tsz Wo Nicholas Sze >Priority: Critical > Fix For: 2.7.0 > > Attachments: h7885_20150305.patch, h7885_20150306.patch > > > Datanode should not trust the generation stamp provided by client, since it > is prefetched and buffered in client, and concurrent append may increase it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-7818) OffsetParam should return the default value instead of throwing NPE when the value is unspecified
[
https://issues.apache.org/jira/browse/HDFS-7818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14351613#comment-14351613
]
Hudson commented on HDFS-7818:
--
SUCCESS: Integrated in Hadoop-Hdfs-trunk-Java8 #116 (See
[https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/116/])
HDFS-7818. OffsetParam should return the default value instead of throwing NPE
when the value is unspecified. Contributed by Eric Payne. (wheat9: rev
c79710302ee51e1a9ee17dadb161c69bb3aba5c9)
*
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/resources/OffsetParam.java
* hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
*
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/web/webhdfs/TestParameterParser.java
*
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/web/webhdfs/ParameterParser.java
> OffsetParam should return the default value instead of throwing NPE when the
> value is unspecified
> -
>
> Key: HDFS-7818
> URL: https://issues.apache.org/jira/browse/HDFS-7818
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: webhdfs
>Affects Versions: 2.7.0
>Reporter: Eric Payne
>Assignee: Eric Payne
>Priority: Blocker
> Fix For: 2.7.0
>
> Attachments: HDFS-7818.v1.txt, HDFS-7818.v2.txt, HDFS-7818.v3.txt,
> HDFS-7818.v4.txt, HDFS-7818.v5.txt
>
>
> This is a regression in 2.7 and later.
> {{hadoop fs -cat}} over webhdfs works, but {{hadoop fs -text}} does not:
> {code}
> $ hadoop fs -cat webhdfs://myhost.com/tmp/test.1
> ... output ...
> $ hadoop fs -text webhdfs://myhost.com/tmp/test.1
> text: org.apache.hadoop.ipc.RemoteException(java.lang.NullPointerException):
> null
> at
> org.apache.hadoop.hdfs.web.JsonUtil.toRemoteException(JsonUtil.java:165)
> at
> org.apache.hadoop.hdfs.web.WebHdfsFileSystem.validateResponse(WebHdfsFileSystem.java:358)
> at
> org.apache.hadoop.hdfs.web.WebHdfsFileSystem.access$200(WebHdfsFileSystem.java:91)
> at
> org.apache.hadoop.hdfs.web.WebHdfsFileSystem$AbstractRunner.runWithRetry(WebHdfsFileSystem.java:615)
> at
> org.apache.hadoop.hdfs.web.WebHdfsFileSystem$AbstractRunner.access$100(WebHdfsFileSystem.java:463)
> at
> org.apache.hadoop.hdfs.web.WebHdfsFileSystem$AbstractRunner$1.run(WebHdfsFileSystem.java:492)
> ...
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-7818) OffsetParam should return the default value instead of throwing NPE when the value is unspecified
[
https://issues.apache.org/jira/browse/HDFS-7818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14351602#comment-14351602
]
Hudson commented on HDFS-7818:
--
FAILURE: Integrated in Hadoop-Hdfs-trunk #2057 (See
[https://builds.apache.org/job/Hadoop-Hdfs-trunk/2057/])
HDFS-7818. OffsetParam should return the default value instead of throwing NPE
when the value is unspecified. Contributed by Eric Payne. (wheat9: rev
c79710302ee51e1a9ee17dadb161c69bb3aba5c9)
*
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/web/webhdfs/ParameterParser.java
* hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
*
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/resources/OffsetParam.java
*
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/web/webhdfs/TestParameterParser.java
> OffsetParam should return the default value instead of throwing NPE when the
> value is unspecified
> -
>
> Key: HDFS-7818
> URL: https://issues.apache.org/jira/browse/HDFS-7818
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: webhdfs
>Affects Versions: 2.7.0
>Reporter: Eric Payne
>Assignee: Eric Payne
>Priority: Blocker
> Fix For: 2.7.0
>
> Attachments: HDFS-7818.v1.txt, HDFS-7818.v2.txt, HDFS-7818.v3.txt,
> HDFS-7818.v4.txt, HDFS-7818.v5.txt
>
>
> This is a regression in 2.7 and later.
> {{hadoop fs -cat}} over webhdfs works, but {{hadoop fs -text}} does not:
> {code}
> $ hadoop fs -cat webhdfs://myhost.com/tmp/test.1
> ... output ...
> $ hadoop fs -text webhdfs://myhost.com/tmp/test.1
> text: org.apache.hadoop.ipc.RemoteException(java.lang.NullPointerException):
> null
> at
> org.apache.hadoop.hdfs.web.JsonUtil.toRemoteException(JsonUtil.java:165)
> at
> org.apache.hadoop.hdfs.web.WebHdfsFileSystem.validateResponse(WebHdfsFileSystem.java:358)
> at
> org.apache.hadoop.hdfs.web.WebHdfsFileSystem.access$200(WebHdfsFileSystem.java:91)
> at
> org.apache.hadoop.hdfs.web.WebHdfsFileSystem$AbstractRunner.runWithRetry(WebHdfsFileSystem.java:615)
> at
> org.apache.hadoop.hdfs.web.WebHdfsFileSystem$AbstractRunner.access$100(WebHdfsFileSystem.java:463)
> at
> org.apache.hadoop.hdfs.web.WebHdfsFileSystem$AbstractRunner$1.run(WebHdfsFileSystem.java:492)
> ...
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-6488) Support HDFS superuser in NFSv3 gateway
[
https://issues.apache.org/jira/browse/HDFS-6488?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14351601#comment-14351601
]
Hudson commented on HDFS-6488:
--
FAILURE: Integrated in Hadoop-Hdfs-trunk #2057 (See
[https://builds.apache.org/job/Hadoop-Hdfs-trunk/2057/])
HDFS-6488. Support HDFS superuser in NFSv3 gateway. Contributed by Brandon Li
(brandonli: rev 0f8ecb1d0ce6d3ee9a7caf5b15b299210c2b8875)
* hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HdfsNfsGateway.md
* hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
*
hadoop-hdfs-project/hadoop-hdfs-nfs/src/main/java/org/apache/hadoop/hdfs/nfs/nfs3/RpcProgramNfs3.java
*
hadoop-hdfs-project/hadoop-hdfs-nfs/src/main/java/org/apache/hadoop/hdfs/nfs/conf/NfsConfigKeys.java
> Support HDFS superuser in NFSv3 gateway
> ---
>
> Key: HDFS-6488
> URL: https://issues.apache.org/jira/browse/HDFS-6488
> Project: Hadoop HDFS
> Issue Type: New Feature
> Components: nfs
>Affects Versions: 2.3.0
>Reporter: Stephen Chu
>Assignee: Brandon Li
> Fix For: 2.7.0
>
> Attachments: HDFS-6488.001.patch, HDFS-6488.002.patch,
> HDFS-6488.003.patch
>
>
> As hdfs superuseruser on the NFS mount, I cannot cd or ls the
> /user/schu/.Trash directory:
> {code}
> bash-4.1$ cd .Trash/
> bash: cd: .Trash/: Permission denied
> bash-4.1$ ls -la
> total 2
> drwxr-xr-x 4 schu 2584148964 128 Jan 7 10:42 .
> drwxr-xr-x 4 hdfs 2584148964 128 Jan 6 16:59 ..
> drwx-- 2 schu 2584148964 64 Jan 7 10:45 .Trash
> drwxr-xr-x 2 hdfs hdfs64 Jan 7 10:42 tt
> bash-4.1$ ls .Trash
> ls: cannot open directory .Trash: Permission denied
> bash-4.1$
> {code}
> When using FsShell as hdfs superuser, I have superuser permissions to schu's
> .Trash contents:
> {code}
> bash-4.1$ hdfs dfs -ls -R /user/schu/.Trash
> drwx-- - schu supergroup 0 2014-01-07 10:48
> /user/schu/.Trash/Current
> drwx-- - schu supergroup 0 2014-01-07 10:48
> /user/schu/.Trash/Current/user
> drwx-- - schu supergroup 0 2014-01-07 10:48
> /user/schu/.Trash/Current/user/schu
> -rw-r--r-- 1 schu supergroup 4 2014-01-07 10:48
> /user/schu/.Trash/Current/user/schu/tf1
> {code}
> The NFSv3 logs don't produce any error when superuser tries to access schu
> Trash contents. However, for other permission errors (e.g. schu tries to
> delete a directory owned by hdfs), there will be a permission error in the
> logs.
> I think this is not specific to the .Trash directory perhaps.
> I created a /user/schu/dir1 which has the same permissions as .Trash (700).
> When I try cd'ing into the directory from the NFSv3 mount as hdfs superuser,
> I get the same permission denied.
> {code}
> [schu@hdfs-nfs ~]$ hdfs dfs -ls
> Found 4 items
> drwx-- - schu supergroup 0 2014-01-07 10:57 .Trash
> drwx-- - schu supergroup 0 2014-01-07 11:05 dir1
> -rw-r--r-- 1 schu supergroup 4 2014-01-07 11:05 tf1
> drwxr-xr-x - hdfs hdfs0 2014-01-07 10:42 tt
> bash-4.1$ whoami
> hdfs
> bash-4.1$ pwd
> /hdfs_nfs_mount/user/schu
> bash-4.1$ cd dir1
> bash: cd: dir1: Permission denied
> bash-4.1$
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-7885) Datanode should not trust the generation stamp provided by client
[ https://issues.apache.org/jira/browse/HDFS-7885?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14351595#comment-14351595 ] Hudson commented on HDFS-7885: -- FAILURE: Integrated in Hadoop-Hdfs-trunk #2057 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/2057/]) HDFS-7885. Datanode should not trust the generation stamp provided by client. Contributed by Tsz Wo Nicholas Sze. (jing9: rev 24db0812be64e83a48ade01fc1eaaeaedad4dec0) * hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestBlockReaderLocalLegacy.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImpl.java > Datanode should not trust the generation stamp provided by client > - > > Key: HDFS-7885 > URL: https://issues.apache.org/jira/browse/HDFS-7885 > Project: Hadoop HDFS > Issue Type: Bug > Components: datanode >Affects Versions: 2.2.0 >Reporter: vitthal (Suhas) Gogate >Assignee: Tsz Wo Nicholas Sze >Priority: Critical > Fix For: 2.7.0 > > Attachments: h7885_20150305.patch, h7885_20150306.patch > > > Datanode should not trust the generation stamp provided by client, since it > is prefetched and buffered in client, and concurrent append may increase it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-7885) Datanode should not trust the generation stamp provided by client
[ https://issues.apache.org/jira/browse/HDFS-7885?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14351525#comment-14351525 ] Hudson commented on HDFS-7885: -- FAILURE: Integrated in Hadoop-Yarn-trunk #859 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk/859/]) HDFS-7885. Datanode should not trust the generation stamp provided by client. Contributed by Tsz Wo Nicholas Sze. (jing9: rev 24db0812be64e83a48ade01fc1eaaeaedad4dec0) * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImpl.java * hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestBlockReaderLocalLegacy.java > Datanode should not trust the generation stamp provided by client > - > > Key: HDFS-7885 > URL: https://issues.apache.org/jira/browse/HDFS-7885 > Project: Hadoop HDFS > Issue Type: Bug > Components: datanode >Affects Versions: 2.2.0 >Reporter: vitthal (Suhas) Gogate >Assignee: Tsz Wo Nicholas Sze >Priority: Critical > Fix For: 2.7.0 > > Attachments: h7885_20150305.patch, h7885_20150306.patch > > > Datanode should not trust the generation stamp provided by client, since it > is prefetched and buffered in client, and concurrent append may increase it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-7818) OffsetParam should return the default value instead of throwing NPE when the value is unspecified
[
https://issues.apache.org/jira/browse/HDFS-7818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14351532#comment-14351532
]
Hudson commented on HDFS-7818:
--
FAILURE: Integrated in Hadoop-Yarn-trunk #859 (See
[https://builds.apache.org/job/Hadoop-Yarn-trunk/859/])
HDFS-7818. OffsetParam should return the default value instead of throwing NPE
when the value is unspecified. Contributed by Eric Payne. (wheat9: rev
c79710302ee51e1a9ee17dadb161c69bb3aba5c9)
*
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/resources/OffsetParam.java
* hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
*
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/web/webhdfs/TestParameterParser.java
*
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/web/webhdfs/ParameterParser.java
> OffsetParam should return the default value instead of throwing NPE when the
> value is unspecified
> -
>
> Key: HDFS-7818
> URL: https://issues.apache.org/jira/browse/HDFS-7818
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: webhdfs
>Affects Versions: 2.7.0
>Reporter: Eric Payne
>Assignee: Eric Payne
>Priority: Blocker
> Fix For: 2.7.0
>
> Attachments: HDFS-7818.v1.txt, HDFS-7818.v2.txt, HDFS-7818.v3.txt,
> HDFS-7818.v4.txt, HDFS-7818.v5.txt
>
>
> This is a regression in 2.7 and later.
> {{hadoop fs -cat}} over webhdfs works, but {{hadoop fs -text}} does not:
> {code}
> $ hadoop fs -cat webhdfs://myhost.com/tmp/test.1
> ... output ...
> $ hadoop fs -text webhdfs://myhost.com/tmp/test.1
> text: org.apache.hadoop.ipc.RemoteException(java.lang.NullPointerException):
> null
> at
> org.apache.hadoop.hdfs.web.JsonUtil.toRemoteException(JsonUtil.java:165)
> at
> org.apache.hadoop.hdfs.web.WebHdfsFileSystem.validateResponse(WebHdfsFileSystem.java:358)
> at
> org.apache.hadoop.hdfs.web.WebHdfsFileSystem.access$200(WebHdfsFileSystem.java:91)
> at
> org.apache.hadoop.hdfs.web.WebHdfsFileSystem$AbstractRunner.runWithRetry(WebHdfsFileSystem.java:615)
> at
> org.apache.hadoop.hdfs.web.WebHdfsFileSystem$AbstractRunner.access$100(WebHdfsFileSystem.java:463)
> at
> org.apache.hadoop.hdfs.web.WebHdfsFileSystem$AbstractRunner$1.run(WebHdfsFileSystem.java:492)
> ...
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-6488) Support HDFS superuser in NFSv3 gateway
[
https://issues.apache.org/jira/browse/HDFS-6488?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14351531#comment-14351531
]
Hudson commented on HDFS-6488:
--
FAILURE: Integrated in Hadoop-Yarn-trunk #859 (See
[https://builds.apache.org/job/Hadoop-Yarn-trunk/859/])
HDFS-6488. Support HDFS superuser in NFSv3 gateway. Contributed by Brandon Li
(brandonli: rev 0f8ecb1d0ce6d3ee9a7caf5b15b299210c2b8875)
* hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HdfsNfsGateway.md
* hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
*
hadoop-hdfs-project/hadoop-hdfs-nfs/src/main/java/org/apache/hadoop/hdfs/nfs/conf/NfsConfigKeys.java
*
hadoop-hdfs-project/hadoop-hdfs-nfs/src/main/java/org/apache/hadoop/hdfs/nfs/nfs3/RpcProgramNfs3.java
> Support HDFS superuser in NFSv3 gateway
> ---
>
> Key: HDFS-6488
> URL: https://issues.apache.org/jira/browse/HDFS-6488
> Project: Hadoop HDFS
> Issue Type: New Feature
> Components: nfs
>Affects Versions: 2.3.0
>Reporter: Stephen Chu
>Assignee: Brandon Li
> Fix For: 2.7.0
>
> Attachments: HDFS-6488.001.patch, HDFS-6488.002.patch,
> HDFS-6488.003.patch
>
>
> As hdfs superuseruser on the NFS mount, I cannot cd or ls the
> /user/schu/.Trash directory:
> {code}
> bash-4.1$ cd .Trash/
> bash: cd: .Trash/: Permission denied
> bash-4.1$ ls -la
> total 2
> drwxr-xr-x 4 schu 2584148964 128 Jan 7 10:42 .
> drwxr-xr-x 4 hdfs 2584148964 128 Jan 6 16:59 ..
> drwx-- 2 schu 2584148964 64 Jan 7 10:45 .Trash
> drwxr-xr-x 2 hdfs hdfs64 Jan 7 10:42 tt
> bash-4.1$ ls .Trash
> ls: cannot open directory .Trash: Permission denied
> bash-4.1$
> {code}
> When using FsShell as hdfs superuser, I have superuser permissions to schu's
> .Trash contents:
> {code}
> bash-4.1$ hdfs dfs -ls -R /user/schu/.Trash
> drwx-- - schu supergroup 0 2014-01-07 10:48
> /user/schu/.Trash/Current
> drwx-- - schu supergroup 0 2014-01-07 10:48
> /user/schu/.Trash/Current/user
> drwx-- - schu supergroup 0 2014-01-07 10:48
> /user/schu/.Trash/Current/user/schu
> -rw-r--r-- 1 schu supergroup 4 2014-01-07 10:48
> /user/schu/.Trash/Current/user/schu/tf1
> {code}
> The NFSv3 logs don't produce any error when superuser tries to access schu
> Trash contents. However, for other permission errors (e.g. schu tries to
> delete a directory owned by hdfs), there will be a permission error in the
> logs.
> I think this is not specific to the .Trash directory perhaps.
> I created a /user/schu/dir1 which has the same permissions as .Trash (700).
> When I try cd'ing into the directory from the NFSv3 mount as hdfs superuser,
> I get the same permission denied.
> {code}
> [schu@hdfs-nfs ~]$ hdfs dfs -ls
> Found 4 items
> drwx-- - schu supergroup 0 2014-01-07 10:57 .Trash
> drwx-- - schu supergroup 0 2014-01-07 11:05 dir1
> -rw-r--r-- 1 schu supergroup 4 2014-01-07 11:05 tf1
> drwxr-xr-x - hdfs hdfs0 2014-01-07 10:42 tt
> bash-4.1$ whoami
> hdfs
> bash-4.1$ pwd
> /hdfs_nfs_mount/user/schu
> bash-4.1$ cd dir1
> bash: cd: dir1: Permission denied
> bash-4.1$
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-7885) Datanode should not trust the generation stamp provided by client
[ https://issues.apache.org/jira/browse/HDFS-7885?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14351517#comment-14351517 ] Hudson commented on HDFS-7885: -- FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #125 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/125/]) HDFS-7885. Datanode should not trust the generation stamp provided by client. Contributed by Tsz Wo Nicholas Sze. (jing9: rev 24db0812be64e83a48ade01fc1eaaeaedad4dec0) * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestBlockReaderLocalLegacy.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImpl.java * hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt > Datanode should not trust the generation stamp provided by client > - > > Key: HDFS-7885 > URL: https://issues.apache.org/jira/browse/HDFS-7885 > Project: Hadoop HDFS > Issue Type: Bug > Components: datanode >Affects Versions: 2.2.0 >Reporter: vitthal (Suhas) Gogate >Assignee: Tsz Wo Nicholas Sze >Priority: Critical > Fix For: 2.7.0 > > Attachments: h7885_20150305.patch, h7885_20150306.patch > > > Datanode should not trust the generation stamp provided by client, since it > is prefetched and buffered in client, and concurrent append may increase it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-6488) Support HDFS superuser in NFSv3 gateway
[
https://issues.apache.org/jira/browse/HDFS-6488?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14351523#comment-14351523
]
Hudson commented on HDFS-6488:
--
FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #125 (See
[https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/125/])
HDFS-6488. Support HDFS superuser in NFSv3 gateway. Contributed by Brandon Li
(brandonli: rev 0f8ecb1d0ce6d3ee9a7caf5b15b299210c2b8875)
* hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HdfsNfsGateway.md
* hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
*
hadoop-hdfs-project/hadoop-hdfs-nfs/src/main/java/org/apache/hadoop/hdfs/nfs/conf/NfsConfigKeys.java
*
hadoop-hdfs-project/hadoop-hdfs-nfs/src/main/java/org/apache/hadoop/hdfs/nfs/nfs3/RpcProgramNfs3.java
> Support HDFS superuser in NFSv3 gateway
> ---
>
> Key: HDFS-6488
> URL: https://issues.apache.org/jira/browse/HDFS-6488
> Project: Hadoop HDFS
> Issue Type: New Feature
> Components: nfs
>Affects Versions: 2.3.0
>Reporter: Stephen Chu
>Assignee: Brandon Li
> Fix For: 2.7.0
>
> Attachments: HDFS-6488.001.patch, HDFS-6488.002.patch,
> HDFS-6488.003.patch
>
>
> As hdfs superuseruser on the NFS mount, I cannot cd or ls the
> /user/schu/.Trash directory:
> {code}
> bash-4.1$ cd .Trash/
> bash: cd: .Trash/: Permission denied
> bash-4.1$ ls -la
> total 2
> drwxr-xr-x 4 schu 2584148964 128 Jan 7 10:42 .
> drwxr-xr-x 4 hdfs 2584148964 128 Jan 6 16:59 ..
> drwx-- 2 schu 2584148964 64 Jan 7 10:45 .Trash
> drwxr-xr-x 2 hdfs hdfs64 Jan 7 10:42 tt
> bash-4.1$ ls .Trash
> ls: cannot open directory .Trash: Permission denied
> bash-4.1$
> {code}
> When using FsShell as hdfs superuser, I have superuser permissions to schu's
> .Trash contents:
> {code}
> bash-4.1$ hdfs dfs -ls -R /user/schu/.Trash
> drwx-- - schu supergroup 0 2014-01-07 10:48
> /user/schu/.Trash/Current
> drwx-- - schu supergroup 0 2014-01-07 10:48
> /user/schu/.Trash/Current/user
> drwx-- - schu supergroup 0 2014-01-07 10:48
> /user/schu/.Trash/Current/user/schu
> -rw-r--r-- 1 schu supergroup 4 2014-01-07 10:48
> /user/schu/.Trash/Current/user/schu/tf1
> {code}
> The NFSv3 logs don't produce any error when superuser tries to access schu
> Trash contents. However, for other permission errors (e.g. schu tries to
> delete a directory owned by hdfs), there will be a permission error in the
> logs.
> I think this is not specific to the .Trash directory perhaps.
> I created a /user/schu/dir1 which has the same permissions as .Trash (700).
> When I try cd'ing into the directory from the NFSv3 mount as hdfs superuser,
> I get the same permission denied.
> {code}
> [schu@hdfs-nfs ~]$ hdfs dfs -ls
> Found 4 items
> drwx-- - schu supergroup 0 2014-01-07 10:57 .Trash
> drwx-- - schu supergroup 0 2014-01-07 11:05 dir1
> -rw-r--r-- 1 schu supergroup 4 2014-01-07 11:05 tf1
> drwxr-xr-x - hdfs hdfs0 2014-01-07 10:42 tt
> bash-4.1$ whoami
> hdfs
> bash-4.1$ pwd
> /hdfs_nfs_mount/user/schu
> bash-4.1$ cd dir1
> bash: cd: dir1: Permission denied
> bash-4.1$
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-7818) OffsetParam should return the default value instead of throwing NPE when the value is unspecified
[
https://issues.apache.org/jira/browse/HDFS-7818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14351524#comment-14351524
]
Hudson commented on HDFS-7818:
--
FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #125 (See
[https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/125/])
HDFS-7818. OffsetParam should return the default value instead of throwing NPE
when the value is unspecified. Contributed by Eric Payne. (wheat9: rev
c79710302ee51e1a9ee17dadb161c69bb3aba5c9)
* hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
*
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/resources/OffsetParam.java
*
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/web/webhdfs/ParameterParser.java
*
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/web/webhdfs/TestParameterParser.java
> OffsetParam should return the default value instead of throwing NPE when the
> value is unspecified
> -
>
> Key: HDFS-7818
> URL: https://issues.apache.org/jira/browse/HDFS-7818
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: webhdfs
>Affects Versions: 2.7.0
>Reporter: Eric Payne
>Assignee: Eric Payne
>Priority: Blocker
> Fix For: 2.7.0
>
> Attachments: HDFS-7818.v1.txt, HDFS-7818.v2.txt, HDFS-7818.v3.txt,
> HDFS-7818.v4.txt, HDFS-7818.v5.txt
>
>
> This is a regression in 2.7 and later.
> {{hadoop fs -cat}} over webhdfs works, but {{hadoop fs -text}} does not:
> {code}
> $ hadoop fs -cat webhdfs://myhost.com/tmp/test.1
> ... output ...
> $ hadoop fs -text webhdfs://myhost.com/tmp/test.1
> text: org.apache.hadoop.ipc.RemoteException(java.lang.NullPointerException):
> null
> at
> org.apache.hadoop.hdfs.web.JsonUtil.toRemoteException(JsonUtil.java:165)
> at
> org.apache.hadoop.hdfs.web.WebHdfsFileSystem.validateResponse(WebHdfsFileSystem.java:358)
> at
> org.apache.hadoop.hdfs.web.WebHdfsFileSystem.access$200(WebHdfsFileSystem.java:91)
> at
> org.apache.hadoop.hdfs.web.WebHdfsFileSystem$AbstractRunner.runWithRetry(WebHdfsFileSystem.java:615)
> at
> org.apache.hadoop.hdfs.web.WebHdfsFileSystem$AbstractRunner.access$100(WebHdfsFileSystem.java:463)
> at
> org.apache.hadoop.hdfs.web.WebHdfsFileSystem$AbstractRunner$1.run(WebHdfsFileSystem.java:492)
> ...
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-7838) Expose truncate API for libhdfs
[
https://issues.apache.org/jira/browse/HDFS-7838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14351498#comment-14351498
]
Hadoop QA commented on HDFS-7838:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12703223/HDFS-7838.002.patch
against trunk revision 608ebd5.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests in
hadoop-hdfs-project/hadoop-hdfs:
org.apache.hadoop.hdfs.server.balancer.TestBalancer
org.apache.hadoop.hdfs.server.namenode.TestFileTruncate
org.apache.hadoop.hdfs.web.TestWebHDFS
Test results:
https://builds.apache.org/job/PreCommit-HDFS-Build/9787//testReport/
Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/9787//console
This message is automatically generated.
> Expose truncate API for libhdfs
> ---
>
> Key: HDFS-7838
> URL: https://issues.apache.org/jira/browse/HDFS-7838
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: datanode, namenode
>Affects Versions: 2.7.0
>Reporter: Yi Liu
>Assignee: Yi Liu
> Fix For: 2.7.0
>
> Attachments: HDFS-7838.001.patch, HDFS-7838.002.patch
>
>
> It's good to expose truncate in libhdfs.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
