[jira] [Created] (HDFS-7088) Archival Storage: fix TestBalancer and TestBalancerWithMultipleNameNodes

2014-09-18 Thread Tsz Wo Nicholas Sze (JIRA)
Tsz Wo Nicholas Sze created HDFS-7088: - Summary: Archival Storage: fix TestBalancer and TestBalancerWithMultipleNameNodes Key: HDFS-7088 URL: https://issues.apache.org/jira/browse/HDFS-7088

[jira] [Resolved] (HDFS-2932) Under replicated block after the pipeline recovery.

2014-09-18 Thread Srikanth Upputuri (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-2932?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Srikanth Upputuri resolved HDFS-2932. - Resolution: Duplicate Fix Version/s: (was: 0.24.0) Closed as duplicate of

[DISCUSS] Allow continue reading from being-written file using same stream

2014-09-18 Thread Vinayakumar B
Hi all, Currently *DFSInputStream *doen't allow reading a write-inprogress file, once all written bytes, by the time of opening an input stream, are read. To read further update on the same file, needs to be read by opening another stream to the same file again. Instead how about refreshing

Re: [DISCUSS] Allow continue reading from being-written file using same stream

2014-09-18 Thread Steve Loughran
I don't know about the merits of this, but I do know that native filesystems implement this by not raising the EOF exception on the seek() but only on the read ... some of the non-HDFS filesystems Hadoop support work this way. -I haven't ever looked to see what code assumes that it is the seek

Hadoop-Hdfs-trunk - Build # 1875 - Still Failing

2014-09-18 Thread Apache Jenkins Server
See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1875/ ### ## LAST 60 LINES OF THE CONSOLE ### [...truncated 6035 lines...] [INFO] [INFO] ---

Build failed in Jenkins: Hadoop-Hdfs-trunk #1875

2014-09-18 Thread Apache Jenkins Server
See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1875/changes Changes: [tucu] HADOOP-11099. KMS return HTTP UNAUTHORIZED 401 on ACL failure. (tucu) [tucu] HADOOP-11016. KMS should support signing cookies with zookeeper secret manager. (tucu) [tucu] HADOOP-10982 [tucu] Revert HADOOP-10982

Restarting HDFS datanode process

2014-09-18 Thread Biju N
Hello There, Can individual datanodes restarted (through service command) or do we need to add it to the exclude file in NN, restart DN it, remove it from the exclude file in NN as if the data node is being decommissioned and put back? Any feedback on this is much appreciated. Thanks, Biju

[jira] [Resolved] (HDFS-7088) Archival Storage: fix TestBalancer and TestBalancerWithMultipleNameNodes

2014-09-18 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7088?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao resolved HDFS-7088. - Resolution: Fixed Hadoop Flags: Reviewed I've committed this. Archival Storage: fix TestBalancer

Re: Restarting HDFS datanode process

2014-09-18 Thread Ted Yu
How long would your datanode be out of service ? If the duration is long (say 1 day), you'd better decommission it first. Otherwise, datanode restart can be done without going through decommissioning. On Thu, Sep 18, 2014 at 8:25 AM, Biju N bijuatapa...@gmail.com wrote: Hello There, Can

Re: [DISCUSS] Allow continue reading from being-written file using same stream

2014-09-18 Thread Vinayakumar B
bq. I don't know about the merits of this, but I do know that native filesystems implement this by not raising the EOF exception on the seek() but only on the read ... some of the non-HDFS filesystems Hadoop support work this way. I agree with you steve. read only will throw EOF. But when we know

[jira] [Resolved] (HDFS-7084) FsDatasetImpl#copyBlockFiles debug log can be improved

2014-09-18 Thread Arpit Agarwal (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7084?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arpit Agarwal resolved HDFS-7084. - Resolution: Fixed Fix Version/s: HDFS-6581 Hadoop Flags: Reviewed +1, committed to the

Re: Restarting HDFS datanode process

2014-09-18 Thread Biju N
It is for the duration of the restart. The concern is what will happen to any reads which is inflight that is using the data node. Will HDFS client know the offset to start reading from where the read failed using another DN? On Thu, Sep 18, 2014 at 1:57 PM, Ted Yu yuzhih...@gmail.com wrote:

Re: Restarting HDFS datanode process

2014-09-18 Thread Ted Yu
DFSClient would resume reading from where it stops from the next data node. Cheers On Thu, Sep 18, 2014 at 11:53 AM, Biju N bijuatapa...@gmail.com wrote: It is for the duration of the restart. The concern is what will happen to any reads which is inflight that is using the data node. Will

[jira] [Created] (HDFS-7089) Fix findbugs and release audit warnings in the branch

2014-09-18 Thread Arpit Agarwal (JIRA)
Arpit Agarwal created HDFS-7089: --- Summary: Fix findbugs and release audit warnings in the branch Key: HDFS-7089 URL: https://issues.apache.org/jira/browse/HDFS-7089 Project: Hadoop HDFS Issue

[jira] [Resolved] (HDFS-6815) Verify that alternate access methods work properly with Data at Rest Encryption

2014-09-18 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6815?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang resolved HDFS-6815. --- Resolution: Done We've gone through the various other HDFS access methods at this point, and have

Re: [VOTE] Merge HDFS-6584 Archival Storage to trunk

2014-09-18 Thread Tsz Wo Sze
Hi HDFS dev, We have four +1's (Vinay, Jitendra, Arpit and me) and no -1's. The VOTE passed. PS: Sorry that my email box seems having some problem. I somehow did not receive Arpit's VOTE and some other emails. The email I sent last time (forwarded below) did not go out. Thanks everyone.

[jira] [Created] (HDFS-7090) Use unbuffered writes when persisting in-memory replicas

2014-09-18 Thread Arpit Agarwal (JIRA)
Arpit Agarwal created HDFS-7090: --- Summary: Use unbuffered writes when persisting in-memory replicas Key: HDFS-7090 URL: https://issues.apache.org/jira/browse/HDFS-7090 Project: Hadoop HDFS

[jira] [Created] (HDFS-7091) Prepare branch for merging with HDFS-6584

2014-09-18 Thread Arpit Agarwal (JIRA)
Arpit Agarwal created HDFS-7091: --- Summary: Prepare branch for merging with HDFS-6584 Key: HDFS-7091 URL: https://issues.apache.org/jira/browse/HDFS-7091 Project: Hadoop HDFS Issue Type:

[jira] [Created] (HDFS-7092) Document the logic about Delegation Token is generated and used in both HA and non-HA setup

2014-09-18 Thread Jing Zhao (JIRA)
Jing Zhao created HDFS-7092: --- Summary: Document the logic about Delegation Token is generated and used in both HA and non-HA setup Key: HDFS-7092 URL: https://issues.apache.org/jira/browse/HDFS-7092

[jira] [Created] (HDFS-7093) Allow unprivileged users to set storage policy on a directory

2014-09-18 Thread Arpit Agarwal (JIRA)
Arpit Agarwal created HDFS-7093: --- Summary: Allow unprivileged users to set storage policy on a directory Key: HDFS-7093 URL: https://issues.apache.org/jira/browse/HDFS-7093 Project: Hadoop HDFS

[jira] [Resolved] (HDFS-7091) Add forwarding constructor for INodeFile for existing callers

2014-09-18 Thread Arpit Agarwal (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7091?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arpit Agarwal resolved HDFS-7091. - Resolution: Fixed Release Note: Thanks Nicholas! Revised title and committed to the feature