[jira] [Created] (HDFS-12423) Ozone: TopN container choosing policy should ignore unnecessary containers

2017-09-11 Thread Yiqun Lin (JIRA)
Yiqun Lin created HDFS-12423: Summary: Ozone: TopN container choosing policy should ignore unnecessary containers Key: HDFS-12423 URL: https://issues.apache.org/jira/browse/HDFS-12423 Project: Hadoop HDFS

[jira] [Resolved] (HDFS-11772) Ozone: KSM: set creationTime for volume/bucket/key

2017-09-11 Thread Anu Engineer (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11772?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anu Engineer resolved HDFS-11772. - Resolution: Duplicate > Ozone: KSM: set creationTime for volume/bucket/key > -

Re: [VOTE] Release Apache Hadoop 2.8.2 (RC0)

2017-09-11 Thread Junping Du
Miklos just ping me offline that a security fix should get landed to resolve a docker runtime issue. I will retrieval RC0 for security fixes landing. In the mean while, if people here really think a document here is necessary (although not enough verification to work as an alpha feature) and can

Re: [VOTE] Release Apache Hadoop 2.8.2 (RC0)

2017-09-11 Thread Junping Du
Thanks Daniel for volunteering for documentation effort. I suspect the problem we are facing here is not a simply documentation truncate effort, but indeed a incomplete feature problem. The umbrella JIRA YARN-3611 has 31 issues are marked as resolved but only 9 patches get backport to branch-2.

Re: Release 2.9.0 update

2017-09-11 Thread Wangda Tan
Thanks Arun Suresh for creating the page. +1 to have a page to track all planned releases, I also updated 3.1 release plans. Several suggestions: 1) Only put large features / changes which needed to be included by the release, otherwise we will have too long list of JIRAs. 2) For release updates,

Re: [VOTE] Release Apache Hadoop 2.8.2 (RC0)

2017-09-11 Thread Daniel Templeton
YARN-6622 is now committed to 2.9.  We could backport YARN-5258 and YARN-6622 for 2.8, but it'll take some editing.  We'll have to check to see what features are unsupported in 2.8 and remove those from the docs.  Not a huge effort overall, though.  Probably a hour's work.  I may have time to t

Release 2.9.0 update

2017-09-11 Thread Arun Suresh
Hello Folks Thanks to everyone who chimed in in the discussion thread: https://www.mail-archive.com/yarn-dev@hadoop.apache.org/msg27104.html We have updated the roadmap wiki based on the feedback: https://cwiki.apache.org/confluence/display/HADOOP/Roadmap#Roadmap-Version2.9 Given the explosion o

Re: [VOTE] Release Apache Hadoop 2.8.2 (RC0)

2017-09-11 Thread Chris Douglas
On Mon, Sep 11, 2017 at 2:52 PM, Junping Du wrote: > I don't think this -1 is reasonable, because: > - If you look at YARN-6622 closely, it targets to fix a problematic > documentation work on YARN-5258 which get checked into 2.9 and 3.0 branch > only. It means it targets to fix a problem that 2

Re: [VOTE] Release Apache Hadoop 2.8.2 (RC0)

2017-09-11 Thread Junping Du
I don't think this -1 is reasonable, because: - If you look at YARN-6622 closely, it targets to fix a problematic documentation work on YARN-5258 which get checked into 2.9 and 3.0 branch only. It means it targets to fix a problem that 2.8.2 never exists. - New docker container support (replace o

[jira] [Created] (HDFS-12422) Replace DataNode in Pipeline for Last Packet

2017-09-11 Thread Lukas Majercak (JIRA)
Lukas Majercak created HDFS-12422: - Summary: Replace DataNode in Pipeline for Last Packet Key: HDFS-12422 URL: https://issues.apache.org/jira/browse/HDFS-12422 Project: Hadoop HDFS Issue Type

[jira] [Resolved] (HDFS-11982) Add markdown documentation about Ozone

2017-09-11 Thread Anu Engineer (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11982?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anu Engineer resolved HDFS-11982. - Resolution: Later > Add markdown documentation about Ozone > -

[jira] [Resolved] (HDFS-12239) Ozone: OzoneClient : Remove createContainer handling from client

2017-09-11 Thread Anu Engineer (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12239?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anu Engineer resolved HDFS-12239. - Resolution: Duplicate > Ozone: OzoneClient : Remove createContainer handling from client > ---

[jira] [Resolved] (HDFS-12252) Ozone: SCM: Container State Machine -1- Track container creation state in SCM

2017-09-11 Thread Anu Engineer (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12252?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anu Engineer resolved HDFS-12252. - Resolution: Duplicate > Ozone: SCM: Container State Machine -1- Track container creation state in

Re: [VOTE] Merge yarn-native-services branch into trunk

2017-09-11 Thread Andrew Wang
Thanks for your consideration Jian, let's track this for GA then. Best, Andrew On Fri, Sep 8, 2017 at 3:02 PM, Jian He wrote: > Hi Andrew, > > At this point, there are no more release blockers including documentations > from our side - all work done. > But I agree it is too close to the release

[jira] [Created] (HDFS-12421) Balancer to emit standard metrics

2017-09-11 Thread Erik Krogen (JIRA)
Erik Krogen created HDFS-12421: -- Summary: Balancer to emit standard metrics Key: HDFS-12421 URL: https://issues.apache.org/jira/browse/HDFS-12421 Project: Hadoop HDFS Issue Type: Improvement

Re: If primary replica is unresponsive, hsync() hangs

2017-09-11 Thread Yongjun Zhang
Thanks for finding the issue Wei-Chiu. I agree hsync should be handling DN failure similarly as write-pipeline recovery, as you stated. If it's not doing that, it should be fixed. --Yongjun On Mon, Sep 11, 2017 at 10:53 AM, Wei-Chiu Chuang wrote: > Hello my dear HDFS dev colleagues, > > It app

Re: [VOTE] Release Apache Hadoop 2.8.2 (RC0)

2017-09-11 Thread Chris Douglas
-1 (binding) I don't think we should release this without YARN-6622. Since this doesn't happen often: a -1 in this case is NOT a veto. Releases are approved by majority vote of the PMC. -C On Mon, Sep 11, 2017 at 11:45 AM, Junping Du wrote: > Thanks Mikols for notifying on this. I think docker

Re: [VOTE] Release Apache Hadoop 2.8.2 (RC0)

2017-09-11 Thread Junping Du
Thanks Mikols for notifying on this. I think docker support is general known as alpha feature so document it as experimental is nice to have but not a blocker for 2.8.2. I also noticed that our 2.7.x document (https://hadoop.apache.org/docs/r2.7.4/hadoop-yarn/hadoop-yarn-site/DockerContainerExec

[jira] [Created] (HDFS-12420) Improve error message on Namenode format

2017-09-11 Thread Ajay Kumar (JIRA)
Ajay Kumar created HDFS-12420: - Summary: Improve error message on Namenode format Key: HDFS-12420 URL: https://issues.apache.org/jira/browse/HDFS-12420 Project: Hadoop HDFS Issue Type: Improvemen

[jira] [Created] (HDFS-12419) Fix TestDistCpViewFs failures on branch-2

2017-09-11 Thread Xiaoyu Yao (JIRA)
Xiaoyu Yao created HDFS-12419: - Summary: Fix TestDistCpViewFs failures on branch-2 Key: HDFS-12419 URL: https://issues.apache.org/jira/browse/HDFS-12419 Project: Hadoop HDFS Issue Type: Bug

Re: [VOTE] Release Apache Hadoop 2.8.2 (RC0)

2017-09-11 Thread Brahma Reddy Battula
Thanks Junping Du. +1 (non-binding) -Built from source -Installed HA cluster -Executed basic shell commands -Ran sample jobs -Verified HDFS/YARN UI. --Brahma Reddy Battula On Mon, 11 Sep 2017 at 5:30 AM, Junping Du wrote: > Hi folks, > With fix of HADOOP-14842 get in, I've created our

If primary replica is unresponsive, hsync() hangs

2017-09-11 Thread Wei-Chiu Chuang
Hello my dear HDFS dev colleagues, It appears that when a dfs client writes and hsync(), and if the primary replica (that is, the first DataNode in the write pipeline) is unresponsive to the hsync() request, the hsync() would wait at DataStreamer#waitForAckedSeqno(). In one scenario, we saw this

[jira] [Created] (HDFS-12418) Fix BackupNode UT broken by HDFS-10391

2017-09-11 Thread Arpit Agarwal (JIRA)
Arpit Agarwal created HDFS-12418: Summary: Fix BackupNode UT broken by HDFS-10391 Key: HDFS-12418 URL: https://issues.apache.org/jira/browse/HDFS-12418 Project: Hadoop HDFS Issue Type: Bug

[jira] [Created] (HDFS-12417) Disable flaky TestDFSStripedOutputStreamWithFailure

2017-09-11 Thread Chris Douglas (JIRA)
Chris Douglas created HDFS-12417: Summary: Disable flaky TestDFSStripedOutputStreamWithFailure Key: HDFS-12417 URL: https://issues.apache.org/jira/browse/HDFS-12417 Project: Hadoop HDFS Issue

[jira] [Reopened] (HDFS-12349) Improve log message when it could not alloc enough blocks for EC

2017-09-11 Thread Chris Douglas (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12349?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Douglas reopened HDFS-12349: -- > Improve log message when it could not alloc enough blocks for EC > --

Apache Hadoop qbt Report: trunk+JDK8 on Linux/x86

2017-09-11 Thread Apache Jenkins Server
For more details, see https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/520/ [Sep 11, 2017 4:58:10 AM] (sunilg) YARN-7163. RMContext need not to be injected to webapp and other Always [Sep 11, 2017 6:17:59 AM] (yufei) YARN-6799. Remove the duplicated code in CGroupsHandlerImp.java.

[jira] [Created] (HDFS-12416) BlockPlacementPolicyDefault will cause NN shutdown if log level is changed

2017-09-11 Thread Suhan Mao (JIRA)
Suhan Mao created HDFS-12416: Summary: BlockPlacementPolicyDefault will cause NN shutdown if log level is changed Key: HDFS-12416 URL: https://issues.apache.org/jira/browse/HDFS-12416 Project: Hadoop HDFS

[jira] [Created] (HDFS-12415) Ozone: TestXceiverClientManager occasionally fails

2017-09-11 Thread Weiwei Yang (JIRA)
Weiwei Yang created HDFS-12415: -- Summary: Ozone: TestXceiverClientManager occasionally fails Key: HDFS-12415 URL: https://issues.apache.org/jira/browse/HDFS-12415 Project: Hadoop HDFS Issue Type

[jira] [Resolved] (HDFS-11922) Ozone: KSM: Garbage collect deleted blocks

2017-09-11 Thread Weiwei Yang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11922?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Weiwei Yang resolved HDFS-11922. Resolution: Done Fix Version/s: HDFS-7240 Target Version/s: HDFS-7240 > Ozone: KSM: