Re: Planning Hadoop 2.6.1 release
To clarify, there is already a branch-2.6.1 created separately. So, if you want something in 2.6.2, you can continue putting it in branch-2.6. But no 2.6.1 related commits please. Thanks +Vinod On Aug 14, 2015, at 12:09 PM, Vinod Kumar Vavilapalli vino...@hortonworks.com wrote: Everyone, Please stop committing to 2.6.1 (branch-2.6 or branch-2.6.1), we have a fairly elaborate parallel release-process going on for 2.6.1 (with me / Akira and Sangjin participating offline) and any outside cherry-picks are going to disrupt our progress. If you want something added to 2.6.1, please post it in the mailing lists and we will consider it. Thanks. +Vinod On Aug 14, 2015, at 12:17 AM, Vinayakumar B vinayakum...@apache.org wrote: FYI, Some of the issues in https://wiki.apache.org/hadoop/Release-2.6.1-Working-Notes are already merged in 2.6.1. Regards, Vinay On Fri, Aug 14, 2015 at 12:43 PM, Sangjin Lee sj...@apache.org wrote: HDFS-8850 https://issues.apache.org/jira/browse/HDFS-8850 is a fix for VolumeScanner which was added to 2.7 (HDFS-7430 https://issues.apache.org/jira/browse/HDFS-7430). I don't see it applicable to 2.6. Could you please confirm? Thanks, Sangjin On Wed, Aug 5, 2015 at 2:34 PM, Sean Busbey bus...@cloudera.com wrote: If we haven't frozen yet, HDFS-8850 is a straight forward fix that is currently only in 2.8+ and would benefit 2.6 and 2.7. On Wed, Aug 5, 2015 at 2:56 PM, Junping Du j...@hortonworks.com wrote: I would like to nominate YARN-3832 as 2.6.1 candidate which is critical and I also saw it happened recently on a 2.6.0 cluster. Hope this is not too late. Thanks, Junping From: Rich Haase rha...@pandora.com Sent: Thursday, August 06, 2015 1:52 AM To: mapreduce-dev@hadoop.apache.org; yarn-...@hadoop.apache.org Cc: common-...@hadoop.apache.org; hdfs-...@hadoop.apache.org Subject: Re: Planning Hadoop 2.6.1 release +1 to add those fixes. Rich Haase | Sr. Software Engineer | Pandora m 303.887.1146 | rha...@pandora.com On 8/5/15, 11:42 AM, Wangda Tan wheele...@gmail.com wrote: Can we add following two fixes to 2.6.1? https://issues.apache.org/jira/browse/YARN-2922 and https://issues.apache.org/jira/browse/YARN-3487. They're not fatal issue, but they can cause lots of issue in a large cluster. Thanks, Wangda On Mon, Aug 3, 2015 at 1:21 PM, Sangjin Lee sj...@apache.org wrote: See my later update in the thread. HDFS-7704 is in the list. Thanks, Sangjin On Mon, Aug 3, 2015 at 1:19 PM, Vinod Kumar Vavilapalli vino...@hortonworks.com wrote: Makes sense, it was caused by HDFS-7704 which got into 2.7.0 only and is not part of the candidate list. Removed HDFS-7916 from the list. Thanks +Vinod On Jul 24, 2015, at 6:32 PM, Sangjin Lee sj...@apache.org wrote: Out of the JIRAs we proposed, please remove HDFS-7916. I don't think it applies to 2.6. Thanks, Sangjin -- Sean
Re: Planning Hadoop 2.6.1 release
Everyone, Please stop committing to 2.6.1 (branch-2.6 or branch-2.6.1), we have a fairly elaborate parallel release-process going on for 2.6.1 (with me / Akira and Sangjin participating offline) and any outside cherry-picks are going to disrupt our progress. If you want something added to 2.6.1, please post it in the mailing lists and we will consider it. Thanks. +Vinod On Aug 14, 2015, at 12:17 AM, Vinayakumar B vinayakum...@apache.org wrote: FYI, Some of the issues in https://wiki.apache.org/hadoop/Release-2.6.1-Working-Notes are already merged in 2.6.1. Regards, Vinay On Fri, Aug 14, 2015 at 12:43 PM, Sangjin Lee sj...@apache.org wrote: HDFS-8850 https://issues.apache.org/jira/browse/HDFS-8850 is a fix for VolumeScanner which was added to 2.7 (HDFS-7430 https://issues.apache.org/jira/browse/HDFS-7430). I don't see it applicable to 2.6. Could you please confirm? Thanks, Sangjin On Wed, Aug 5, 2015 at 2:34 PM, Sean Busbey bus...@cloudera.com wrote: If we haven't frozen yet, HDFS-8850 is a straight forward fix that is currently only in 2.8+ and would benefit 2.6 and 2.7. On Wed, Aug 5, 2015 at 2:56 PM, Junping Du j...@hortonworks.com wrote: I would like to nominate YARN-3832 as 2.6.1 candidate which is critical and I also saw it happened recently on a 2.6.0 cluster. Hope this is not too late. Thanks, Junping From: Rich Haase rha...@pandora.com Sent: Thursday, August 06, 2015 1:52 AM To: mapreduce-dev@hadoop.apache.org; yarn-...@hadoop.apache.org Cc: common-...@hadoop.apache.org; hdfs-...@hadoop.apache.org Subject: Re: Planning Hadoop 2.6.1 release +1 to add those fixes. Rich Haase | Sr. Software Engineer | Pandora m 303.887.1146 | rha...@pandora.com On 8/5/15, 11:42 AM, Wangda Tan wheele...@gmail.com wrote: Can we add following two fixes to 2.6.1? https://issues.apache.org/jira/browse/YARN-2922 and https://issues.apache.org/jira/browse/YARN-3487. They're not fatal issue, but they can cause lots of issue in a large cluster. Thanks, Wangda On Mon, Aug 3, 2015 at 1:21 PM, Sangjin Lee sj...@apache.org wrote: See my later update in the thread. HDFS-7704 is in the list. Thanks, Sangjin On Mon, Aug 3, 2015 at 1:19 PM, Vinod Kumar Vavilapalli vino...@hortonworks.com wrote: Makes sense, it was caused by HDFS-7704 which got into 2.7.0 only and is not part of the candidate list. Removed HDFS-7916 from the list. Thanks +Vinod On Jul 24, 2015, at 6:32 PM, Sangjin Lee sj...@apache.org wrote: Out of the JIRAs we proposed, please remove HDFS-7916. I don't think it applies to 2.6. Thanks, Sangjin -- Sean
Re: Planning Hadoop 2.6.1 release
Can we include YARN-3999 too, if the list is not yet frozen. On Aug 11, 2015, at 1:49 PM, Hitesh Shah hit...@apache.org wrote: Jason Lowe just filed YARN-4047 ( patch available too ) which probably is also a good candidate for 2.6.1 or 2.6.2 if the list is frozen. I have labelled the jira as a 2.6.1 candidate. Feel free to switch to 2.6.2 if needed. thanks — Hitesh On Aug 10, 2015, at 4:15 PM, Vinod Kumar Vavilapalli vino...@hortonworks.com wrote: As of now, I am freezing the list. We have 139 tickets to go through, find the right order, get them committed etc. I’ll start a separate thread for this one-time long release-process of 2.6.1. Thanks everyone for pitching in. +Vinod On Aug 10, 2015, at 4:13 PM, Vinod Kumar Vavilapalli vino...@hortonworks.com wrote: Added them all, tx Rohit. +Vinod On Aug 5, 2015, at 9:51 PM, Rohith Sharma K S rohithsharm...@huawei.com wrote: Can we add following fixes to 2.6.1? YARN-3733 YARN-2865 YARN-3990 YARN-2894 Thanks Regards Rohith Sharma K S -Original Message- From: Allan Wilson [mailto:awils...@pandora.com] Sent: 05 August 2015 23:25 To: common-...@hadoop.apache.org; mapreduce-dev@hadoop.apache.org; yarn-...@hadoop.apache.org Cc: hdfs-...@hadoop.apache.org Subject: Re: Planning Hadoop 2.6.1 release Another +1 to add those fixes. YARN-3487 bug can grind a large cluster to a halt repeatedly -Allan Allan Wilson | Sr. Software Engineer | Pandora m 919.841.2449 | awils...@pandora.com On 8/5/15, 1:52 PM, Rich Haase rha...@pandora.com wrote: +1 to add those fixes. Rich Haase | Sr. Software Engineer | Pandora m 303.887.1146 | rha...@pandora.com On 8/5/15, 11:42 AM, Wangda Tan wheele...@gmail.com wrote: Can we add following two fixes to 2.6.1? https://issues.apache.org/jira/browse/YARN-2922 and https://issues.apache.org/jira/browse/YARN-3487. They're not fatal issue, but they can cause lots of issue in a large cluster. Thanks, Wangda On Mon, Aug 3, 2015 at 1:21 PM, Sangjin Lee sj...@apache.org wrote: See my later update in the thread. HDFS-7704 is in the list. Thanks, Sangjin On Mon, Aug 3, 2015 at 1:19 PM, Vinod Kumar Vavilapalli vino...@hortonworks.com wrote: Makes sense, it was caused by HDFS-7704 which got into 2.7.0 only and is not part of the candidate list. Removed HDFS-7916 from the list. Thanks +Vinod On Jul 24, 2015, at 6:32 PM, Sangjin Lee sj...@apache.org wrote: Out of the JIRAs we proposed, please remove HDFS-7916. I don't think it applies to 2.6. Thanks, Sangjin
Re: Planning Hadoop 2.6.1 release
Jason Lowe just filed YARN-4047 ( patch available too ) which probably is also a good candidate for 2.6.1 or 2.6.2 if the list is frozen. I have labelled the jira as a 2.6.1 candidate. Feel free to switch to 2.6.2 if needed. thanks — Hitesh On Aug 10, 2015, at 4:15 PM, Vinod Kumar Vavilapalli vino...@hortonworks.com wrote: As of now, I am freezing the list. We have 139 tickets to go through, find the right order, get them committed etc. I’ll start a separate thread for this one-time long release-process of 2.6.1. Thanks everyone for pitching in. +Vinod On Aug 10, 2015, at 4:13 PM, Vinod Kumar Vavilapalli vino...@hortonworks.com wrote: Added them all, tx Rohit. +Vinod On Aug 5, 2015, at 9:51 PM, Rohith Sharma K S rohithsharm...@huawei.com wrote: Can we add following fixes to 2.6.1? YARN-3733 YARN-2865 YARN-3990 YARN-2894 Thanks Regards Rohith Sharma K S -Original Message- From: Allan Wilson [mailto:awils...@pandora.com] Sent: 05 August 2015 23:25 To: common-...@hadoop.apache.org; mapreduce-dev@hadoop.apache.org; yarn-...@hadoop.apache.org Cc: hdfs-...@hadoop.apache.org Subject: Re: Planning Hadoop 2.6.1 release Another +1 to add those fixes. YARN-3487 bug can grind a large cluster to a halt repeatedly -Allan Allan Wilson | Sr. Software Engineer | Pandora m 919.841.2449 | awils...@pandora.com On 8/5/15, 1:52 PM, Rich Haase rha...@pandora.com wrote: +1 to add those fixes. Rich Haase | Sr. Software Engineer | Pandora m 303.887.1146 | rha...@pandora.com On 8/5/15, 11:42 AM, Wangda Tan wheele...@gmail.com wrote: Can we add following two fixes to 2.6.1? https://issues.apache.org/jira/browse/YARN-2922 and https://issues.apache.org/jira/browse/YARN-3487. They're not fatal issue, but they can cause lots of issue in a large cluster. Thanks, Wangda On Mon, Aug 3, 2015 at 1:21 PM, Sangjin Lee sj...@apache.org wrote: See my later update in the thread. HDFS-7704 is in the list. Thanks, Sangjin On Mon, Aug 3, 2015 at 1:19 PM, Vinod Kumar Vavilapalli vino...@hortonworks.com wrote: Makes sense, it was caused by HDFS-7704 which got into 2.7.0 only and is not part of the candidate list. Removed HDFS-7916 from the list. Thanks +Vinod On Jul 24, 2015, at 6:32 PM, Sangjin Lee sj...@apache.org wrote: Out of the JIRAs we proposed, please remove HDFS-7916. I don't think it applies to 2.6. Thanks, Sangjin
Re: Planning Hadoop 2.6.1 release
Added them all, tx Rohit. +Vinod On Aug 5, 2015, at 9:51 PM, Rohith Sharma K S rohithsharm...@huawei.com wrote: Can we add following fixes to 2.6.1? YARN-3733 YARN-2865 YARN-3990 YARN-2894 Thanks Regards Rohith Sharma K S -Original Message- From: Allan Wilson [mailto:awils...@pandora.com] Sent: 05 August 2015 23:25 To: common-...@hadoop.apache.org; mapreduce-dev@hadoop.apache.org; yarn-...@hadoop.apache.org Cc: hdfs-...@hadoop.apache.org Subject: Re: Planning Hadoop 2.6.1 release Another +1 to add those fixes. YARN-3487 bug can grind a large cluster to a halt repeatedly -Allan Allan Wilson | Sr. Software Engineer | Pandora m 919.841.2449 | awils...@pandora.com On 8/5/15, 1:52 PM, Rich Haase rha...@pandora.com wrote: +1 to add those fixes. Rich Haase | Sr. Software Engineer | Pandora m 303.887.1146 | rha...@pandora.com On 8/5/15, 11:42 AM, Wangda Tan wheele...@gmail.com wrote: Can we add following two fixes to 2.6.1? https://issues.apache.org/jira/browse/YARN-2922 and https://issues.apache.org/jira/browse/YARN-3487. They're not fatal issue, but they can cause lots of issue in a large cluster. Thanks, Wangda On Mon, Aug 3, 2015 at 1:21 PM, Sangjin Lee sj...@apache.org wrote: See my later update in the thread. HDFS-7704 is in the list. Thanks, Sangjin On Mon, Aug 3, 2015 at 1:19 PM, Vinod Kumar Vavilapalli vino...@hortonworks.com wrote: Makes sense, it was caused by HDFS-7704 which got into 2.7.0 only and is not part of the candidate list. Removed HDFS-7916 from the list. Thanks +Vinod On Jul 24, 2015, at 6:32 PM, Sangjin Lee sj...@apache.org wrote: Out of the JIRAs we proposed, please remove HDFS-7916. I don't think it applies to 2.6. Thanks, Sangjin
Re: Planning Hadoop 2.6.1 release
Added them both. Tx Wangda! +Vinod On Aug 5, 2015, at 10:42 AM, Wangda Tan wheele...@gmail.com wrote: Can we add following two fixes to 2.6.1? https://issues.apache.org/jira/browse/YARN-2922 and https://issues.apache.org/jira/browse/YARN-3487. They're not fatal issue, but they can cause lots of issue in a large cluster. Thanks, Wangda On Mon, Aug 3, 2015 at 1:21 PM, Sangjin Lee sj...@apache.org wrote: See my later update in the thread. HDFS-7704 is in the list. Thanks, Sangjin On Mon, Aug 3, 2015 at 1:19 PM, Vinod Kumar Vavilapalli vino...@hortonworks.com wrote: Makes sense, it was caused by HDFS-7704 which got into 2.7.0 only and is not part of the candidate list. Removed HDFS-7916 from the list. Thanks +Vinod On Jul 24, 2015, at 6:32 PM, Sangjin Lee sj...@apache.org wrote: Out of the JIRAs we proposed, please remove HDFS-7916. I don't think it applies to 2.6. Thanks, Sangjin
Re: Planning Hadoop 2.6.1 release
Added this, Tx Zhijie! +Vinod On Aug 5, 2015, at 10:49 AM, Zhijie Shen zs...@hortonworks.com wrote: +YARN-3011: without which, NM is likely to crash, and cannot come back with recovery enabled. Thanks, Zhijie From: Wangda Tan wheele...@gmail.com Sent: Wednesday, August 05, 2015 10:42 AM To: yarn-...@hadoop.apache.org Cc: mapreduce-dev@hadoop.apache.org; common-...@hadoop.apache.org; hdfs-...@hadoop.apache.org Subject: Re: Planning Hadoop 2.6.1 release Can we add following two fixes to 2.6.1? https://issues.apache.org/jira/browse/YARN-2922 and https://issues.apache.org/jira/browse/YARN-3487. They're not fatal issue, but they can cause lots of issue in a large cluster. Thanks, Wangda On Mon, Aug 3, 2015 at 1:21 PM, Sangjin Lee sj...@apache.org wrote: See my later update in the thread. HDFS-7704 is in the list. Thanks, Sangjin On Mon, Aug 3, 2015 at 1:19 PM, Vinod Kumar Vavilapalli vino...@hortonworks.com wrote: Makes sense, it was caused by HDFS-7704 which got into 2.7.0 only and is not part of the candidate list. Removed HDFS-7916 from the list. Thanks +Vinod On Jul 24, 2015, at 6:32 PM, Sangjin Lee sj...@apache.org wrote: Out of the JIRAs we proposed, please remove HDFS-7916. I don't think it applies to 2.6. Thanks, Sangjin
Re: Planning Hadoop 2.6.1 release
As of now, I am freezing the list. We have 139 tickets to go through, find the right order, get them committed etc. I’ll start a separate thread for this one-time long release-process of 2.6.1. Thanks everyone for pitching in. +Vinod On Aug 10, 2015, at 4:13 PM, Vinod Kumar Vavilapalli vino...@hortonworks.com wrote: Added them all, tx Rohit. +Vinod On Aug 5, 2015, at 9:51 PM, Rohith Sharma K S rohithsharm...@huawei.com wrote: Can we add following fixes to 2.6.1? YARN-3733 YARN-2865 YARN-3990 YARN-2894 Thanks Regards Rohith Sharma K S -Original Message- From: Allan Wilson [mailto:awils...@pandora.com] Sent: 05 August 2015 23:25 To: common-...@hadoop.apache.org; mapreduce-dev@hadoop.apache.org; yarn-...@hadoop.apache.org Cc: hdfs-...@hadoop.apache.org Subject: Re: Planning Hadoop 2.6.1 release Another +1 to add those fixes. YARN-3487 bug can grind a large cluster to a halt repeatedly -Allan Allan Wilson | Sr. Software Engineer | Pandora m 919.841.2449 | awils...@pandora.com On 8/5/15, 1:52 PM, Rich Haase rha...@pandora.com wrote: +1 to add those fixes. Rich Haase | Sr. Software Engineer | Pandora m 303.887.1146 | rha...@pandora.com On 8/5/15, 11:42 AM, Wangda Tan wheele...@gmail.com wrote: Can we add following two fixes to 2.6.1? https://issues.apache.org/jira/browse/YARN-2922 and https://issues.apache.org/jira/browse/YARN-3487. They're not fatal issue, but they can cause lots of issue in a large cluster. Thanks, Wangda On Mon, Aug 3, 2015 at 1:21 PM, Sangjin Lee sj...@apache.org wrote: See my later update in the thread. HDFS-7704 is in the list. Thanks, Sangjin On Mon, Aug 3, 2015 at 1:19 PM, Vinod Kumar Vavilapalli vino...@hortonworks.com wrote: Makes sense, it was caused by HDFS-7704 which got into 2.7.0 only and is not part of the candidate list. Removed HDFS-7916 from the list. Thanks +Vinod On Jul 24, 2015, at 6:32 PM, Sangjin Lee sj...@apache.org wrote: Out of the JIRAs we proposed, please remove HDFS-7916. I don't think it applies to 2.6. Thanks, Sangjin
Re: Planning Hadoop 2.6.1 release
Added, tx Junping. +Vinod On Aug 5, 2015, at 12:56 PM, Junping Du j...@hortonworks.com wrote: I would like to nominate YARN-3832 as 2.6.1 candidate which is critical and I also saw it happened recently on a 2.6.0 cluster. Hope this is not too late. Thanks, Junping From: Rich Haase rha...@pandora.com Sent: Thursday, August 06, 2015 1:52 AM To: mapreduce-dev@hadoop.apache.org; yarn-...@hadoop.apache.org Cc: common-...@hadoop.apache.org; hdfs-...@hadoop.apache.org Subject: Re: Planning Hadoop 2.6.1 release +1 to add those fixes. Rich Haase | Sr. Software Engineer | Pandora m 303.887.1146 | rha...@pandora.com On 8/5/15, 11:42 AM, Wangda Tan wheele...@gmail.com wrote: Can we add following two fixes to 2.6.1? https://issues.apache.org/jira/browse/YARN-2922 and https://issues.apache.org/jira/browse/YARN-3487. They're not fatal issue, but they can cause lots of issue in a large cluster. Thanks, Wangda On Mon, Aug 3, 2015 at 1:21 PM, Sangjin Lee sj...@apache.org wrote: See my later update in the thread. HDFS-7704 is in the list. Thanks, Sangjin On Mon, Aug 3, 2015 at 1:19 PM, Vinod Kumar Vavilapalli vino...@hortonworks.com wrote: Makes sense, it was caused by HDFS-7704 which got into 2.7.0 only and is not part of the candidate list. Removed HDFS-7916 from the list. Thanks +Vinod On Jul 24, 2015, at 6:32 PM, Sangjin Lee sj...@apache.org wrote: Out of the JIRAs we proposed, please remove HDFS-7916. I don't think it applies to 2.6. Thanks, Sangjin
Re: Planning Hadoop 2.6.1 release
Added to the list. Can you get it committed to 2.7.2 please? Thanks +Vinod On Aug 5, 2015, at 2:34 PM, Sean Busbey bus...@cloudera.com wrote: If we haven't frozen yet, HDFS-8850 is a straight forward fix that is currently only in 2.8+ and would benefit 2.6 and 2.7. On Wed, Aug 5, 2015 at 2:56 PM, Junping Du j...@hortonworks.com wrote: I would like to nominate YARN-3832 as 2.6.1 candidate which is critical and I also saw it happened recently on a 2.6.0 cluster. Hope this is not too late. Thanks, Junping From: Rich Haase rha...@pandora.com Sent: Thursday, August 06, 2015 1:52 AM To: mapreduce-dev@hadoop.apache.org; yarn-...@hadoop.apache.org Cc: common-...@hadoop.apache.org; hdfs-...@hadoop.apache.org Subject: Re: Planning Hadoop 2.6.1 release +1 to add those fixes. Rich Haase | Sr. Software Engineer | Pandora m 303.887.1146 | rha...@pandora.com On 8/5/15, 11:42 AM, Wangda Tan wheele...@gmail.com wrote: Can we add following two fixes to 2.6.1? https://issues.apache.org/jira/browse/YARN-2922 and https://issues.apache.org/jira/browse/YARN-3487. They're not fatal issue, but they can cause lots of issue in a large cluster. Thanks, Wangda On Mon, Aug 3, 2015 at 1:21 PM, Sangjin Lee sj...@apache.org wrote: See my later update in the thread. HDFS-7704 is in the list. Thanks, Sangjin On Mon, Aug 3, 2015 at 1:19 PM, Vinod Kumar Vavilapalli vino...@hortonworks.com wrote: Makes sense, it was caused by HDFS-7704 which got into 2.7.0 only and is not part of the candidate list. Removed HDFS-7916 from the list. Thanks +Vinod On Jul 24, 2015, at 6:32 PM, Sangjin Lee sj...@apache.org wrote: Out of the JIRAs we proposed, please remove HDFS-7916. I don't think it applies to 2.6. Thanks, Sangjin -- Sean
Re: Planning Hadoop 2.6.1 release
Can we add following two fixes to 2.6.1? https://issues.apache.org/jira/browse/YARN-2922 and https://issues.apache.org/jira/browse/YARN-3487. They're not fatal issue, but they can cause lots of issue in a large cluster. Thanks, Wangda On Mon, Aug 3, 2015 at 1:21 PM, Sangjin Lee sj...@apache.org wrote: See my later update in the thread. HDFS-7704 is in the list. Thanks, Sangjin On Mon, Aug 3, 2015 at 1:19 PM, Vinod Kumar Vavilapalli vino...@hortonworks.com wrote: Makes sense, it was caused by HDFS-7704 which got into 2.7.0 only and is not part of the candidate list. Removed HDFS-7916 from the list. Thanks +Vinod On Jul 24, 2015, at 6:32 PM, Sangjin Lee sj...@apache.org wrote: Out of the JIRAs we proposed, please remove HDFS-7916. I don't think it applies to 2.6. Thanks, Sangjin
Re: Planning Hadoop 2.6.1 release
+YARN-3011: without which, NM is likely to crash, and cannot come back with recovery enabled. Thanks, Zhijie From: Wangda Tan wheele...@gmail.com Sent: Wednesday, August 05, 2015 10:42 AM To: yarn-...@hadoop.apache.org Cc: mapreduce-dev@hadoop.apache.org; common-...@hadoop.apache.org; hdfs-...@hadoop.apache.org Subject: Re: Planning Hadoop 2.6.1 release Can we add following two fixes to 2.6.1? https://issues.apache.org/jira/browse/YARN-2922 and https://issues.apache.org/jira/browse/YARN-3487. They're not fatal issue, but they can cause lots of issue in a large cluster. Thanks, Wangda On Mon, Aug 3, 2015 at 1:21 PM, Sangjin Lee sj...@apache.org wrote: See my later update in the thread. HDFS-7704 is in the list. Thanks, Sangjin On Mon, Aug 3, 2015 at 1:19 PM, Vinod Kumar Vavilapalli vino...@hortonworks.com wrote: Makes sense, it was caused by HDFS-7704 which got into 2.7.0 only and is not part of the candidate list. Removed HDFS-7916 from the list. Thanks +Vinod On Jul 24, 2015, at 6:32 PM, Sangjin Lee sj...@apache.org wrote: Out of the JIRAs we proposed, please remove HDFS-7916. I don't think it applies to 2.6. Thanks, Sangjin
Re: Planning Hadoop 2.6.1 release
+1 to add those fixes. Rich Haase | Sr. Software Engineer | Pandora m 303.887.1146 | rha...@pandora.com On 8/5/15, 11:42 AM, Wangda Tan wheele...@gmail.com wrote: Can we add following two fixes to 2.6.1? https://issues.apache.org/jira/browse/YARN-2922 and https://issues.apache.org/jira/browse/YARN-3487. They're not fatal issue, but they can cause lots of issue in a large cluster. Thanks, Wangda On Mon, Aug 3, 2015 at 1:21 PM, Sangjin Lee sj...@apache.org wrote: See my later update in the thread. HDFS-7704 is in the list. Thanks, Sangjin On Mon, Aug 3, 2015 at 1:19 PM, Vinod Kumar Vavilapalli vino...@hortonworks.com wrote: Makes sense, it was caused by HDFS-7704 which got into 2.7.0 only and is not part of the candidate list. Removed HDFS-7916 from the list. Thanks +Vinod On Jul 24, 2015, at 6:32 PM, Sangjin Lee sj...@apache.org wrote: Out of the JIRAs we proposed, please remove HDFS-7916. I don't think it applies to 2.6. Thanks, Sangjin
Re: Planning Hadoop 2.6.1 release
I would like to nominate YARN-3832 as 2.6.1 candidate which is critical and I also saw it happened recently on a 2.6.0 cluster. Hope this is not too late. Thanks, Junping From: Rich Haase rha...@pandora.com Sent: Thursday, August 06, 2015 1:52 AM To: mapreduce-dev@hadoop.apache.org; yarn-...@hadoop.apache.org Cc: common-...@hadoop.apache.org; hdfs-...@hadoop.apache.org Subject: Re: Planning Hadoop 2.6.1 release +1 to add those fixes. Rich Haase | Sr. Software Engineer | Pandora m 303.887.1146 | rha...@pandora.com On 8/5/15, 11:42 AM, Wangda Tan wheele...@gmail.com wrote: Can we add following two fixes to 2.6.1? https://issues.apache.org/jira/browse/YARN-2922 and https://issues.apache.org/jira/browse/YARN-3487. They're not fatal issue, but they can cause lots of issue in a large cluster. Thanks, Wangda On Mon, Aug 3, 2015 at 1:21 PM, Sangjin Lee sj...@apache.org wrote: See my later update in the thread. HDFS-7704 is in the list. Thanks, Sangjin On Mon, Aug 3, 2015 at 1:19 PM, Vinod Kumar Vavilapalli vino...@hortonworks.com wrote: Makes sense, it was caused by HDFS-7704 which got into 2.7.0 only and is not part of the candidate list. Removed HDFS-7916 from the list. Thanks +Vinod On Jul 24, 2015, at 6:32 PM, Sangjin Lee sj...@apache.org wrote: Out of the JIRAs we proposed, please remove HDFS-7916. I don't think it applies to 2.6. Thanks, Sangjin
Re: Planning Hadoop 2.6.1 release
If we haven't frozen yet, HDFS-8850 is a straight forward fix that is currently only in 2.8+ and would benefit 2.6 and 2.7. On Wed, Aug 5, 2015 at 2:56 PM, Junping Du j...@hortonworks.com wrote: I would like to nominate YARN-3832 as 2.6.1 candidate which is critical and I also saw it happened recently on a 2.6.0 cluster. Hope this is not too late. Thanks, Junping From: Rich Haase rha...@pandora.com Sent: Thursday, August 06, 2015 1:52 AM To: mapreduce-dev@hadoop.apache.org; yarn-...@hadoop.apache.org Cc: common-...@hadoop.apache.org; hdfs-...@hadoop.apache.org Subject: Re: Planning Hadoop 2.6.1 release +1 to add those fixes. Rich Haase | Sr. Software Engineer | Pandora m 303.887.1146 | rha...@pandora.com On 8/5/15, 11:42 AM, Wangda Tan wheele...@gmail.com wrote: Can we add following two fixes to 2.6.1? https://issues.apache.org/jira/browse/YARN-2922 and https://issues.apache.org/jira/browse/YARN-3487. They're not fatal issue, but they can cause lots of issue in a large cluster. Thanks, Wangda On Mon, Aug 3, 2015 at 1:21 PM, Sangjin Lee sj...@apache.org wrote: See my later update in the thread. HDFS-7704 is in the list. Thanks, Sangjin On Mon, Aug 3, 2015 at 1:19 PM, Vinod Kumar Vavilapalli vino...@hortonworks.com wrote: Makes sense, it was caused by HDFS-7704 which got into 2.7.0 only and is not part of the candidate list. Removed HDFS-7916 from the list. Thanks +Vinod On Jul 24, 2015, at 6:32 PM, Sangjin Lee sj...@apache.org wrote: Out of the JIRAs we proposed, please remove HDFS-7916. I don't think it applies to 2.6. Thanks, Sangjin -- Sean
RE: Planning Hadoop 2.6.1 release
Can we add following fixes to 2.6.1? YARN-3733 YARN-2865 YARN-3990 YARN-2894 Thanks Regards Rohith Sharma K S -Original Message- From: Allan Wilson [mailto:awils...@pandora.com] Sent: 05 August 2015 23:25 To: common-...@hadoop.apache.org; mapreduce-dev@hadoop.apache.org; yarn-...@hadoop.apache.org Cc: hdfs-...@hadoop.apache.org Subject: Re: Planning Hadoop 2.6.1 release Another +1 to add those fixes. YARN-3487 bug can grind a large cluster to a halt repeatedly -Allan Allan Wilson | Sr. Software Engineer | Pandora m 919.841.2449 | awils...@pandora.com On 8/5/15, 1:52 PM, Rich Haase rha...@pandora.com wrote: +1 to add those fixes. Rich Haase | Sr. Software Engineer | Pandora m 303.887.1146 | rha...@pandora.com On 8/5/15, 11:42 AM, Wangda Tan wheele...@gmail.com wrote: Can we add following two fixes to 2.6.1? https://issues.apache.org/jira/browse/YARN-2922 and https://issues.apache.org/jira/browse/YARN-3487. They're not fatal issue, but they can cause lots of issue in a large cluster. Thanks, Wangda On Mon, Aug 3, 2015 at 1:21 PM, Sangjin Lee sj...@apache.org wrote: See my later update in the thread. HDFS-7704 is in the list. Thanks, Sangjin On Mon, Aug 3, 2015 at 1:19 PM, Vinod Kumar Vavilapalli vino...@hortonworks.com wrote: Makes sense, it was caused by HDFS-7704 which got into 2.7.0 only and is not part of the candidate list. Removed HDFS-7916 from the list. Thanks +Vinod On Jul 24, 2015, at 6:32 PM, Sangjin Lee sj...@apache.org wrote: Out of the JIRAs we proposed, please remove HDFS-7916. I don't think it applies to 2.6. Thanks, Sangjin
Re: Planning Hadoop 2.6.1 release
Makes sense, it was caused by HDFS-7704 which got into 2.7.0 only and is not part of the candidate list. Removed HDFS-7916 from the list. Thanks +Vinod On Jul 24, 2015, at 6:32 PM, Sangjin Lee sj...@apache.org wrote: Out of the JIRAs we proposed, please remove HDFS-7916. I don't think it applies to 2.6. Thanks, Sangjin
Re: Planning Hadoop 2.6.1 release
Tx for the list, Joep! Jiras not yet marked with 2.6.1-candidate that we'd like to see in 2.6.1: A whole bunch of them already are marked so, presumably from my list. I’ve added the remaining for discussion. HDFS-7281 (committed in 3.0) This is an incompatible change, so not adding it. YARN-3176 (not yet committed in OSS) We should let it naturally go into 2.6.1 and 2.7.2 depending on when it goes in. We shouldn't block the release for this till it gets reviewed/committed in case the later can’t come in time. Thanks +Vinod
Re: Planning Hadoop 2.6.1 release
See my later update in the thread. HDFS-7704 is in the list. Thanks, Sangjin On Mon, Aug 3, 2015 at 1:19 PM, Vinod Kumar Vavilapalli vino...@hortonworks.com wrote: Makes sense, it was caused by HDFS-7704 which got into 2.7.0 only and is not part of the candidate list. Removed HDFS-7916 from the list. Thanks +Vinod On Jul 24, 2015, at 6:32 PM, Sangjin Lee sj...@apache.org wrote: Out of the JIRAs we proposed, please remove HDFS-7916. I don't think it applies to 2.6. Thanks, Sangjin
Re: Planning Hadoop 2.6.1 release
Thanks Joep and your team members for creating the list. I really appreciate your work. I looked your 'not yet marked with 2.6.1-candidate' list and categorized them. 1) Now marked as 2.6.1-candidate and I agree with you to keep it marked. * HDFS-7213 * HDFS-7788 * HDFS-7884 * HDFS-7930 * YARN-2856 * YARN-3222 * YARN-3238 * YARN-3464 * YARN-3526 * YARN-3850 Thanks Sangjin for creating patches for branch-2.6. 2) Not yet marked as 2.6.1-candidate that I'd like to see in 2.6.1 * HDFS-7182 * HDFS-7314 * HDFS-7704 * HDFS-7894 * HDFS-7929 and HDFS-8480 (they are related) * HDFS-7980 * HDFS-8245 * HDFS-8270 * HDFS-8404 * HDFS-8486 * MAPREDUCE-5465 * MAPREDUCE-5649 * MAPREDUCE-6166 * MAPREDUCE-6238 * MAPREDUCE-6300 * YARN-2952 * YARN-2997 * YARN-3094 * YARN-3176 (to be fixed soon, I think) * YARN-3231 * HADOOP-11295 * HADOOP-11812 3) Not yet marked as 2.6.1-candidate. I'd like to drop * HDFS-7281 (incompatible change) * HDFS-7446 (this looks to be an improvement) * HDFS-7916 (cannot apply to branch-2.6 as Sangjin mentioned) Hi Vinod, could you mark the issues in 2) as 2.6.1-candidate? I'd like to freeze the candidate list in about 7 days and start backporting them. Do you have any thoughts? Regards, Akira On 7/25/15 10:32, Sangjin Lee wrote: Out of the JIRAs we proposed, please remove HDFS-7916. I don't think it applies to 2.6. Thanks, Sangjin On Wed, Jul 22, 2015 at 4:02 PM, Vinod Kumar Vavilapalli vino...@hortonworks.com wrote: I’ve added them all to the 2.6.1-candidate list. I included everything even though some of them are major tickets. The list is getting large, we will have to cut these down once we get down to the next phase of figuring out what to include and what not to. Thanks +Vinod On Jul 21, 2015, at 2:15 AM, Akira AJISAKA ajisa...@oss.nttdata.co.jp wrote: Thanks Vinod for updating the candidate list. I'd like to include the followings 12 JIRAs: * YARN-3641 * YARN-3585 * YARN-2910 * HDFS-8431 * HDFS-7830 * HDFS-7763 * HDFS-7742 * HDFS-7235 * HDFS-7225 * MAPREDUCE-6324 * HADOOP-11934 * HADOOP-11491 Thanks, Akira On 7/18/15 11:13, Vinod Kumar Vavilapalli wrote: - I also have a bunch of patches that I’d like to include, will update them right away. I’ve just finished this. The latest 2.6.1-candidate list is up at 64 JIRAs. Others, please look at the list and post anything else you’d like to get included for 2.6.1. Thanks +Vinod On Jul 15, 2015, at 6:24 PM, Vinod Kumar Vavilapalli vino...@hortonworks.commailto:vino...@hortonworks.com wrote: Alright, I’d like to make progress while the issue is hot. I created a label to discuss on the candidate list of patches: https://issues.apache.org/jira/issues/?jql=labels%20%3D%202.6.1-candidate https://issues.apache.org/jira/issues/?jql=labels%20=%202.6.1-candidate Next steps, I’ll do the following - Review 2.7 and 2.8 blocker/critical tickets and see what makes sense for 2.6.1 and add as candidates - I haven’t reviewed the current list yet, the seed list is from this email thread. Will review them. - I also have a bunch of patches that I’d like to include, will update them right away. Others, please look at the current list and let me know what else you’d like to include. I’d like to keep this ‘candidate-collection’ cycle’ for a max of a week and then start the release process. @Akira, let’s sync up offline on how to take this forward in terms of the release process. Thanks +Vinod
Re: Planning Hadoop 2.6.1 release
Thanks Akira. I'd like to make one small correction. If we're getting HDFS-7704, then we should also get HDFS-7916. My earlier comment was assuming HDFS-7704 was not included in the list. But if is (and I think it should), then we should also get HDFS-7916 as it addresses an important issue related to HDFS-7704. Hope it makes it clear. Sangjin On Fri, Jul 31, 2015 at 10:01 AM, Akira AJISAKA ajisa...@oss.nttdata.co.jp wrote: Thanks Joep and your team members for creating the list. I really appreciate your work. I looked your 'not yet marked with 2.6.1-candidate' list and categorized them. 1) Now marked as 2.6.1-candidate and I agree with you to keep it marked. * HDFS-7213 * HDFS-7788 * HDFS-7884 * HDFS-7930 * YARN-2856 * YARN-3222 * YARN-3238 * YARN-3464 * YARN-3526 * YARN-3850 Thanks Sangjin for creating patches for branch-2.6. 2) Not yet marked as 2.6.1-candidate that I'd like to see in 2.6.1 * HDFS-7182 * HDFS-7314 * HDFS-7704 * HDFS-7894 * HDFS-7929 and HDFS-8480 (they are related) * HDFS-7980 * HDFS-8245 * HDFS-8270 * HDFS-8404 * HDFS-8486 * MAPREDUCE-5465 * MAPREDUCE-5649 * MAPREDUCE-6166 * MAPREDUCE-6238 * MAPREDUCE-6300 * YARN-2952 * YARN-2997 * YARN-3094 * YARN-3176 (to be fixed soon, I think) * YARN-3231 * HADOOP-11295 * HADOOP-11812 3) Not yet marked as 2.6.1-candidate. I'd like to drop * HDFS-7281 (incompatible change) * HDFS-7446 (this looks to be an improvement) * HDFS-7916 (cannot apply to branch-2.6 as Sangjin mentioned) Hi Vinod, could you mark the issues in 2) as 2.6.1-candidate? I'd like to freeze the candidate list in about 7 days and start backporting them. Do you have any thoughts? Regards, Akira On 7/25/15 10:32, Sangjin Lee wrote: Out of the JIRAs we proposed, please remove HDFS-7916. I don't think it applies to 2.6. Thanks, Sangjin On Wed, Jul 22, 2015 at 4:02 PM, Vinod Kumar Vavilapalli vino...@hortonworks.com wrote: I’ve added them all to the 2.6.1-candidate list. I included everything even though some of them are major tickets. The list is getting large, we will have to cut these down once we get down to the next phase of figuring out what to include and what not to. Thanks +Vinod On Jul 21, 2015, at 2:15 AM, Akira AJISAKA ajisa...@oss.nttdata.co.jp wrote: Thanks Vinod for updating the candidate list. I'd like to include the followings 12 JIRAs: * YARN-3641 * YARN-3585 * YARN-2910 * HDFS-8431 * HDFS-7830 * HDFS-7763 * HDFS-7742 * HDFS-7235 * HDFS-7225 * MAPREDUCE-6324 * HADOOP-11934 * HADOOP-11491 Thanks, Akira On 7/18/15 11:13, Vinod Kumar Vavilapalli wrote: - I also have a bunch of patches that I’d like to include, will update them right away. I’ve just finished this. The latest 2.6.1-candidate list is up at 64 JIRAs. Others, please look at the list and post anything else you’d like to get included for 2.6.1. Thanks +Vinod On Jul 15, 2015, at 6:24 PM, Vinod Kumar Vavilapalli vino...@hortonworks.commailto:vino...@hortonworks.com wrote: Alright, I’d like to make progress while the issue is hot. I created a label to discuss on the candidate list of patches: https://issues.apache.org/jira/issues/?jql=labels%20%3D%202.6.1-candidate https://issues.apache.org/jira/issues/?jql=labels%20=%202.6.1-candidate Next steps, I’ll do the following - Review 2.7 and 2.8 blocker/critical tickets and see what makes sense for 2.6.1 and add as candidates - I haven’t reviewed the current list yet, the seed list is from this email thread. Will review them. - I also have a bunch of patches that I’d like to include, will update them right away. Others, please look at the current list and let me know what else you’d like to include. I’d like to keep this ‘candidate-collection’ cycle’ for a max of a week and then start the release process. @Akira, let’s sync up offline on how to take this forward in terms of the release process. Thanks +Vinod
Re: Planning Hadoop 2.6.1 release
Just for the completeness, the following JIRAs have already been committed to branch-2.6 and thus will be part of 2.6.1: HADOOP-11307 YARN-2375 HDFS-7425 HDFS-4882 HDFS-7489 HDFS-7503 HDFS-7443 HDFS-3443 HADOOP-11466 HDFS-7733 MAPREDUCE-6237 YARN-3251 HDFS-6153 reverted On Fri, Jul 31, 2015 at 10:07 AM, Sangjin Lee sj...@apache.org wrote: Thanks Akira. I'd like to make one small correction. If we're getting HDFS-7704, then we should also get HDFS-7916. My earlier comment was assuming HDFS-7704 was not included in the list. But if is (and I think it should), then we should also get HDFS-7916 as it addresses an important issue related to HDFS-7704. Hope it makes it clear. Sangjin On Fri, Jul 31, 2015 at 10:01 AM, Akira AJISAKA ajisa...@oss.nttdata.co.jp wrote: Thanks Joep and your team members for creating the list. I really appreciate your work. I looked your 'not yet marked with 2.6.1-candidate' list and categorized them. 1) Now marked as 2.6.1-candidate and I agree with you to keep it marked. * HDFS-7213 * HDFS-7788 * HDFS-7884 * HDFS-7930 * YARN-2856 * YARN-3222 * YARN-3238 * YARN-3464 * YARN-3526 * YARN-3850 Thanks Sangjin for creating patches for branch-2.6. 2) Not yet marked as 2.6.1-candidate that I'd like to see in 2.6.1 * HDFS-7182 * HDFS-7314 * HDFS-7704 * HDFS-7894 * HDFS-7929 and HDFS-8480 (they are related) * HDFS-7980 * HDFS-8245 * HDFS-8270 * HDFS-8404 * HDFS-8486 * MAPREDUCE-5465 * MAPREDUCE-5649 * MAPREDUCE-6166 * MAPREDUCE-6238 * MAPREDUCE-6300 * YARN-2952 * YARN-2997 * YARN-3094 * YARN-3176 (to be fixed soon, I think) * YARN-3231 * HADOOP-11295 * HADOOP-11812 3) Not yet marked as 2.6.1-candidate. I'd like to drop * HDFS-7281 (incompatible change) * HDFS-7446 (this looks to be an improvement) * HDFS-7916 (cannot apply to branch-2.6 as Sangjin mentioned) Hi Vinod, could you mark the issues in 2) as 2.6.1-candidate? I'd like to freeze the candidate list in about 7 days and start backporting them. Do you have any thoughts? Regards, Akira On 7/25/15 10:32, Sangjin Lee wrote: Out of the JIRAs we proposed, please remove HDFS-7916. I don't think it applies to 2.6. Thanks, Sangjin On Wed, Jul 22, 2015 at 4:02 PM, Vinod Kumar Vavilapalli vino...@hortonworks.com wrote: I’ve added them all to the 2.6.1-candidate list. I included everything even though some of them are major tickets. The list is getting large, we will have to cut these down once we get down to the next phase of figuring out what to include and what not to. Thanks +Vinod On Jul 21, 2015, at 2:15 AM, Akira AJISAKA ajisa...@oss.nttdata.co.jp wrote: Thanks Vinod for updating the candidate list. I'd like to include the followings 12 JIRAs: * YARN-3641 * YARN-3585 * YARN-2910 * HDFS-8431 * HDFS-7830 * HDFS-7763 * HDFS-7742 * HDFS-7235 * HDFS-7225 * MAPREDUCE-6324 * HADOOP-11934 * HADOOP-11491 Thanks, Akira On 7/18/15 11:13, Vinod Kumar Vavilapalli wrote: - I also have a bunch of patches that I’d like to include, will update them right away. I’ve just finished this. The latest 2.6.1-candidate list is up at 64 JIRAs. Others, please look at the list and post anything else you’d like to get included for 2.6.1. Thanks +Vinod On Jul 15, 2015, at 6:24 PM, Vinod Kumar Vavilapalli vino...@hortonworks.commailto:vino...@hortonworks.com wrote: Alright, I’d like to make progress while the issue is hot. I created a label to discuss on the candidate list of patches: https://issues.apache.org/jira/issues/?jql=labels%20%3D%202.6.1-candidate https://issues.apache.org/jira/issues/?jql=labels%20=%202.6.1-candidate Next steps, I’ll do the following - Review 2.7 and 2.8 blocker/critical tickets and see what makes sense for 2.6.1 and add as candidates - I haven’t reviewed the current list yet, the seed list is from this email thread. Will review them. - I also have a bunch of patches that I’d like to include, will update them right away. Others, please look at the current list and let me know what else you’d like to include. I’d like to keep this ‘candidate-collection’ cycle’ for a max of a week and then start the release process. @Akira, let’s sync up offline on how to take this forward in terms of the release process. Thanks +Vinod
Re: Planning Hadoop 2.6.1 release
Out of the JIRAs we proposed, please remove HDFS-7916. I don't think it applies to 2.6. Thanks, Sangjin On Wed, Jul 22, 2015 at 4:02 PM, Vinod Kumar Vavilapalli vino...@hortonworks.com wrote: I’ve added them all to the 2.6.1-candidate list. I included everything even though some of them are major tickets. The list is getting large, we will have to cut these down once we get down to the next phase of figuring out what to include and what not to. Thanks +Vinod On Jul 21, 2015, at 2:15 AM, Akira AJISAKA ajisa...@oss.nttdata.co.jp wrote: Thanks Vinod for updating the candidate list. I'd like to include the followings 12 JIRAs: * YARN-3641 * YARN-3585 * YARN-2910 * HDFS-8431 * HDFS-7830 * HDFS-7763 * HDFS-7742 * HDFS-7235 * HDFS-7225 * MAPREDUCE-6324 * HADOOP-11934 * HADOOP-11491 Thanks, Akira On 7/18/15 11:13, Vinod Kumar Vavilapalli wrote: - I also have a bunch of patches that I’d like to include, will update them right away. I’ve just finished this. The latest 2.6.1-candidate list is up at 64 JIRAs. Others, please look at the list and post anything else you’d like to get included for 2.6.1. Thanks +Vinod On Jul 15, 2015, at 6:24 PM, Vinod Kumar Vavilapalli vino...@hortonworks.commailto:vino...@hortonworks.com wrote: Alright, I’d like to make progress while the issue is hot. I created a label to discuss on the candidate list of patches: https://issues.apache.org/jira/issues/?jql=labels%20%3D%202.6.1-candidate https://issues.apache.org/jira/issues/?jql=labels%20=%202.6.1-candidate Next steps, I’ll do the following - Review 2.7 and 2.8 blocker/critical tickets and see what makes sense for 2.6.1 and add as candidates - I haven’t reviewed the current list yet, the seed list is from this email thread. Will review them. - I also have a bunch of patches that I’d like to include, will update them right away. Others, please look at the current list and let me know what else you’d like to include. I’d like to keep this ‘candidate-collection’ cycle’ for a max of a week and then start the release process. @Akira, let’s sync up offline on how to take this forward in terms of the release process. Thanks +Vinod
Re: Planning Hadoop 2.6.1 release
Hi Vinod, We've gone through the various lists of upstream jiras and wanted to provide our take on what we'd like to see in 2.6.1 release. In addition, we have ~58 jiras (some a group of work such as DN maintenance state) that we already have in production in a pre-2.6 release. I've gone through and selected those that seem relevant for 2.6.1. Jiras already marked with 2.6.1-candidate (we agree should be part of 2.6.1): HADOOP-11802 HDFS-7489 HDFS-7533 HDFS-7587 HDFS-7596 HDFS-7707 HDFS-7742 HDFS-8046 HDFS-8072 HDFS-8127 MAPREDUCE-6303 MAPREDUCE-6361 YARN-2414 YARN-2905 YARN-3369 YARN-3485 YARN-3585 YARN-3641 HDFS-7443 HDFS-7575 Jiras not yet marked with 2.6.1-candidate that we'd like to see in 2.6.1: HDFS-7213 HDFS-7446 HDFS-7704 HDFS-7788 HDFS-7884 HDFS-7894 HDFS-7916 HDFS-7929 HDFS-7930 HDFS-7980 HDFS-8245 HDFS-8270 HDFS-8404 HDFS-8480 HDFS-8486 MAPREDUCE-6238 YARN-2856 MAPREDUCE-6300 YARN-2952 YARN-2997 YARN-3094 YARN-3222 YARN-3238 YARN-3464 YARN-3526 YARN-3850 Jiras that we're already running in production pre-2.6 that we'd like in 2.6.1-candidate: HADOOP-11812 (committed in 2.8) MAPREDUCE-5649 (committed in 2.8) YARN-3231 (committed in 2.7) MAPREDUCE-6166 (committed in 2.7) HADOOP-11295 (committed in 2.7) HDFS-7314 (committed in 2.8) HDFS-7182 (committed in 2.7) MAPREDUCE-5465 (committed in 2.8) Lower priority jira that we're already running in production in pre-2.6 that are more a nice to have: HDFS-7281 (committed in 3.0) YARN-3176 (not yet committed in OSS) Note that these lists are a culmination of lots of work by many people in our team, so credit goes to all of them. Any possible typo or mistake in copying jira ids are entirely to be blamed on me. Thanks, Joep On Tue, Jul 21, 2015 at 2:15 AM, Akira AJISAKA ajisa...@oss.nttdata.co.jp wrote: Thanks Vinod for updating the candidate list. I'd like to include the followings 12 JIRAs: * YARN-3641 * YARN-3585 * YARN-2910 * HDFS-8431 * HDFS-7830 * HDFS-7763 * HDFS-7742 * HDFS-7235 * HDFS-7225 * MAPREDUCE-6324 * HADOOP-11934 * HADOOP-11491 Thanks, Akira On 7/18/15 11:13, Vinod Kumar Vavilapalli wrote: - I also have a bunch of patches that I’d like to include, will update them right away. I’ve just finished this. The latest 2.6.1-candidate list is up at 64 JIRAs. Others, please look at the list and post anything else you’d like to get included for 2.6.1. Thanks +Vinod On Jul 15, 2015, at 6:24 PM, Vinod Kumar Vavilapalli vino...@hortonworks.commailto:vino...@hortonworks.com wrote: Alright, I’d like to make progress while the issue is hot. I created a label to discuss on the candidate list of patches: https://issues.apache.org/jira/issues/?jql=labels%20%3D%202.6.1-candidate https://issues.apache.org/jira/issues/?jql=labels%20=%202.6.1-candidate Next steps, I’ll do the following - Review 2.7 and 2.8 blocker/critical tickets and see what makes sense for 2.6.1 and add as candidates - I haven’t reviewed the current list yet, the seed list is from this email thread. Will review them. - I also have a bunch of patches that I’d like to include, will update them right away. Others, please look at the current list and let me know what else you’d like to include. I’d like to keep this ‘candidate-collection’ cycle’ for a max of a week and then start the release process. @Akira, let’s sync up offline on how to take this forward in terms of the release process. Thanks +Vinod
Re: Planning Hadoop 2.6.1 release
I’ve added them all to the 2.6.1-candidate list. I included everything even though some of them are major tickets. The list is getting large, we will have to cut these down once we get down to the next phase of figuring out what to include and what not to. Thanks +Vinod On Jul 21, 2015, at 2:15 AM, Akira AJISAKA ajisa...@oss.nttdata.co.jp wrote: Thanks Vinod for updating the candidate list. I'd like to include the followings 12 JIRAs: * YARN-3641 * YARN-3585 * YARN-2910 * HDFS-8431 * HDFS-7830 * HDFS-7763 * HDFS-7742 * HDFS-7235 * HDFS-7225 * MAPREDUCE-6324 * HADOOP-11934 * HADOOP-11491 Thanks, Akira On 7/18/15 11:13, Vinod Kumar Vavilapalli wrote: - I also have a bunch of patches that I’d like to include, will update them right away. I’ve just finished this. The latest 2.6.1-candidate list is up at 64 JIRAs. Others, please look at the list and post anything else you’d like to get included for 2.6.1. Thanks +Vinod On Jul 15, 2015, at 6:24 PM, Vinod Kumar Vavilapalli vino...@hortonworks.commailto:vino...@hortonworks.com wrote: Alright, I’d like to make progress while the issue is hot. I created a label to discuss on the candidate list of patches: https://issues.apache.org/jira/issues/?jql=labels%20%3D%202.6.1-candidatehttps://issues.apache.org/jira/issues/?jql=labels%20=%202.6.1-candidate Next steps, I’ll do the following - Review 2.7 and 2.8 blocker/critical tickets and see what makes sense for 2.6.1 and add as candidates - I haven’t reviewed the current list yet, the seed list is from this email thread. Will review them. - I also have a bunch of patches that I’d like to include, will update them right away. Others, please look at the current list and let me know what else you’d like to include. I’d like to keep this ‘candidate-collection’ cycle’ for a max of a week and then start the release process. @Akira, let’s sync up offline on how to take this forward in terms of the release process. Thanks +Vinod
Re: Planning Hadoop 2.6.1 release
Thanks Vinod for updating the candidate list. I'd like to include the followings 12 JIRAs: * YARN-3641 * YARN-3585 * YARN-2910 * HDFS-8431 * HDFS-7830 * HDFS-7763 * HDFS-7742 * HDFS-7235 * HDFS-7225 * MAPREDUCE-6324 * HADOOP-11934 * HADOOP-11491 Thanks, Akira On 7/18/15 11:13, Vinod Kumar Vavilapalli wrote: - I also have a bunch of patches that I’d like to include, will update them right away. I’ve just finished this. The latest 2.6.1-candidate list is up at 64 JIRAs. Others, please look at the list and post anything else you’d like to get included for 2.6.1. Thanks +Vinod On Jul 15, 2015, at 6:24 PM, Vinod Kumar Vavilapalli vino...@hortonworks.commailto:vino...@hortonworks.com wrote: Alright, I’d like to make progress while the issue is hot. I created a label to discuss on the candidate list of patches: https://issues.apache.org/jira/issues/?jql=labels%20%3D%202.6.1-candidatehttps://issues.apache.org/jira/issues/?jql=labels%20=%202.6.1-candidate Next steps, I’ll do the following - Review 2.7 and 2.8 blocker/critical tickets and see what makes sense for 2.6.1 and add as candidates - I haven’t reviewed the current list yet, the seed list is from this email thread. Will review them. - I also have a bunch of patches that I’d like to include, will update them right away. Others, please look at the current list and let me know what else you’d like to include. I’d like to keep this ‘candidate-collection’ cycle’ for a max of a week and then start the release process. @Akira, let’s sync up offline on how to take this forward in terms of the release process. Thanks +Vinod
Re: Planning Hadoop 2.6.1 release
- I also have a bunch of patches that I’d like to include, will update them right away. I’ve just finished this. The latest 2.6.1-candidate list is up at 64 JIRAs. Others, please look at the list and post anything else you’d like to get included for 2.6.1. Thanks +Vinod On Jul 15, 2015, at 6:24 PM, Vinod Kumar Vavilapalli vino...@hortonworks.commailto:vino...@hortonworks.com wrote: Alright, I’d like to make progress while the issue is hot. I created a label to discuss on the candidate list of patches: https://issues.apache.org/jira/issues/?jql=labels%20%3D%202.6.1-candidatehttps://issues.apache.org/jira/issues/?jql=labels%20=%202.6.1-candidate Next steps, I’ll do the following - Review 2.7 and 2.8 blocker/critical tickets and see what makes sense for 2.6.1 and add as candidates - I haven’t reviewed the current list yet, the seed list is from this email thread. Will review them. - I also have a bunch of patches that I’d like to include, will update them right away. Others, please look at the current list and let me know what else you’d like to include. I’d like to keep this ‘candidate-collection’ cycle’ for a max of a week and then start the release process. @Akira, let’s sync up offline on how to take this forward in terms of the release process. Thanks +Vinod
Re: Planning Hadoop 2.6.1 release
Thanks Vinod for creating the list! @Akira, let’s sync up offline on how to take this forward in terms of the release process. Okay. Thanks, Akira On 7/16/15 10:24, Vinod Kumar Vavilapalli wrote: Alright, I’d like to make progress while the issue is hot. I created a label to discuss on the candidate list of patches: https://issues.apache.org/jira/issues/?jql=labels%20%3D%202.6.1-candidatehttps://issues.apache.org/jira/issues/?jql=labels%20=%202.6.1-candidate Next steps, I’ll do the following - Review 2.7 and 2.8 blocker/critical tickets and see what makes sense for 2.6.1 and add as candidates - I haven’t reviewed the current list yet, the seed list is from this email thread. Will review them. - I also have a bunch of patches that I’d like to include, will update them right away. Others, please look at the current list and let me know what else you’d like to include. I’d like to keep this ‘candidate-collection’ cycle’ for a max of a week and then start the release process. @Akira, let’s sync up offline on how to take this forward in terms of the release process. Thanks +Vinod On Jul 15, 2015, at 1:12 PM, Vinod Kumar Vavilapalli vino...@hortonworks.commailto:vino...@hortonworks.com wrote: Got pinged on a recent thread on this one. As I mentioned there, I had many offline discussions re 2.6.1. The biggest problem I found offline was about what bug-fixes are acceptable and what aren’t for everyone wishing to consume 2.6.1. Given the number of bug-fixes that went into 2.7.x and into branch-2.8, figuring out a set of patches that is acceptable for everyone is a huge challenge which kind of stalled my attempts. Thanks +Vinod On Jul 1, 2015, at 12:41 PM, Sean Busbey bus...@cloudera.commailto:bus...@cloudera.com wrote: Any update on a release plan for 2.6.1? On Wed, Jun 10, 2015 at 1:25 AM, Brahma Reddy Battula brahmareddy.batt...@huawei.commailto:brahmareddy.batt...@huawei.com wrote: HI vinod any update on this..? are we planning to give 2.6.1 Or can we make 2.7.1 as stable give..? Thanks Regards Brahma Reddy Battula From: Zhihai Xu [z...@cloudera.commailto:z...@cloudera.com] Sent: Wednesday, May 13, 2015 12:04 PM To: mapreduce-dev@hadoop.apache.orgmailto:mapreduce-dev@hadoop.apache.org Cc: common-...@hadoop.apache.orgmailto:common-...@hadoop.apache.org; yarn-...@hadoop.apache.orgmailto:yarn-...@hadoop.apache.org; hdfs-...@hadoop.apache.orgmailto:hdfs-...@hadoop.apache.org Subject: Re: Planning Hadoop 2.6.1 release Hi Akira, Can we also include YARN-3242? YARN-3242 fixed a critical ZKRMStateStore bug. It will work better with YARN-2992. thanks zhihai On Tue, May 12, 2015 at 10:38 PM, Akira AJISAKA ajisa...@oss.nttdata.co.jp wrote: Thanks all for collecting jiras for 2.6.1 release. In addition, I'd like to include the following: * HADOOP-11343. Overflow is not properly handled in calculating final iv for AES CTR * YARN-2874. Dead lock in DelegationTokenRenewer which blocks RM to execute any further apps * YARN-2992. ZKRMStateStore crashes due to session expiry * YARN-3013. AMRMClientImpl does not update AMRM token properly * YARN-3369. Missing NullPointer check in AppSchedulingInfo causes RM to die * MAPREDUCE-6303. Read timeout when retrying a fetch error can be fatal to a reducer All of these are marked as blocker bug for 2.7.0 but not fixed in 2.6.0. Regards, Akira On 5/4/15 11:15, Brahma Reddy Battula wrote: Hello Vinod, I am thinking,can we include HADOOP-11491 also..? wihout this jira harfs will not be usable when cluster installed in HA mode and try to get filecontext like below.. Path path = new Path(har:///archivedLogs/application_1428917727658_0005-application_1428917727658_0008-1428927448352.har); FileSystem fs = path.getFileSystem(new Configuration()); path = fs.makeQualified(path); FileContext fc = FileContext.getFileContext(path.toUri(),new Configuration()); Thanks Regards Brahma Reddy Battula From: Chris Nauroth [cnaur...@hortonworks.com] Sent: Friday, May 01, 2015 4:32 AM To: mapreduce-dev@hadoop.apache.org; common-...@hadoop.apache.org; yarn-...@hadoop.apache.org; hdfs-...@hadoop.apache.org Subject: Re: Planning Hadoop 2.6.1 release Thank you, Arpit. In addition, I suggest we include the following: HADOOP-11333. Fix deadlock in DomainSocketWatcher when the notification pipe is full HADOOP-11604. Prevent ConcurrentModificationException while closing domain sockets during shutdown of DomainSocketWatcher thread. HADOOP-11648. Set DomainSocketWatcher thread name explicitly HADOOP-11802. DomainSocketWatcher thread terminates sometimes after there is an I/O error during requestShortCircuitShm HADOOP-11604 and 11648 are not critical by themselves, but they are pre-requisites to getting a clean cherry-pick of 11802, which we believe finally fixes the root cause of this issue. --Chris Nauroth On 4/30/15, 3:55 PM, Arpit Agarwal aagar
Re: Planning Hadoop 2.6.1 release
Alright, I’d like to make progress while the issue is hot. I created a label to discuss on the candidate list of patches: https://issues.apache.org/jira/issues/?jql=labels%20%3D%202.6.1-candidatehttps://issues.apache.org/jira/issues/?jql=labels%20=%202.6.1-candidate Next steps, I’ll do the following - Review 2.7 and 2.8 blocker/critical tickets and see what makes sense for 2.6.1 and add as candidates - I haven’t reviewed the current list yet, the seed list is from this email thread. Will review them. - I also have a bunch of patches that I’d like to include, will update them right away. Others, please look at the current list and let me know what else you’d like to include. I’d like to keep this ‘candidate-collection’ cycle’ for a max of a week and then start the release process. @Akira, let’s sync up offline on how to take this forward in terms of the release process. Thanks +Vinod On Jul 15, 2015, at 1:12 PM, Vinod Kumar Vavilapalli vino...@hortonworks.commailto:vino...@hortonworks.com wrote: Got pinged on a recent thread on this one. As I mentioned there, I had many offline discussions re 2.6.1. The biggest problem I found offline was about what bug-fixes are acceptable and what aren’t for everyone wishing to consume 2.6.1. Given the number of bug-fixes that went into 2.7.x and into branch-2.8, figuring out a set of patches that is acceptable for everyone is a huge challenge which kind of stalled my attempts. Thanks +Vinod On Jul 1, 2015, at 12:41 PM, Sean Busbey bus...@cloudera.commailto:bus...@cloudera.com wrote: Any update on a release plan for 2.6.1? On Wed, Jun 10, 2015 at 1:25 AM, Brahma Reddy Battula brahmareddy.batt...@huawei.commailto:brahmareddy.batt...@huawei.com wrote: HI vinod any update on this..? are we planning to give 2.6.1 Or can we make 2.7.1 as stable give..? Thanks Regards Brahma Reddy Battula From: Zhihai Xu [z...@cloudera.commailto:z...@cloudera.com] Sent: Wednesday, May 13, 2015 12:04 PM To: mapreduce-dev@hadoop.apache.orgmailto:mapreduce-dev@hadoop.apache.org Cc: common-...@hadoop.apache.orgmailto:common-...@hadoop.apache.org; yarn-...@hadoop.apache.orgmailto:yarn-...@hadoop.apache.org; hdfs-...@hadoop.apache.orgmailto:hdfs-...@hadoop.apache.org Subject: Re: Planning Hadoop 2.6.1 release Hi Akira, Can we also include YARN-3242? YARN-3242 fixed a critical ZKRMStateStore bug. It will work better with YARN-2992. thanks zhihai On Tue, May 12, 2015 at 10:38 PM, Akira AJISAKA ajisa...@oss.nttdata.co.jp wrote: Thanks all for collecting jiras for 2.6.1 release. In addition, I'd like to include the following: * HADOOP-11343. Overflow is not properly handled in calculating final iv for AES CTR * YARN-2874. Dead lock in DelegationTokenRenewer which blocks RM to execute any further apps * YARN-2992. ZKRMStateStore crashes due to session expiry * YARN-3013. AMRMClientImpl does not update AMRM token properly * YARN-3369. Missing NullPointer check in AppSchedulingInfo causes RM to die * MAPREDUCE-6303. Read timeout when retrying a fetch error can be fatal to a reducer All of these are marked as blocker bug for 2.7.0 but not fixed in 2.6.0. Regards, Akira On 5/4/15 11:15, Brahma Reddy Battula wrote: Hello Vinod, I am thinking,can we include HADOOP-11491 also..? wihout this jira harfs will not be usable when cluster installed in HA mode and try to get filecontext like below.. Path path = new Path(har:///archivedLogs/application_1428917727658_0005-application_1428917727658_0008-1428927448352.har); FileSystem fs = path.getFileSystem(new Configuration()); path = fs.makeQualified(path); FileContext fc = FileContext.getFileContext(path.toUri(),new Configuration()); Thanks Regards Brahma Reddy Battula From: Chris Nauroth [cnaur...@hortonworks.com] Sent: Friday, May 01, 2015 4:32 AM To: mapreduce-dev@hadoop.apache.org; common-...@hadoop.apache.org; yarn-...@hadoop.apache.org; hdfs-...@hadoop.apache.org Subject: Re: Planning Hadoop 2.6.1 release Thank you, Arpit. In addition, I suggest we include the following: HADOOP-11333. Fix deadlock in DomainSocketWatcher when the notification pipe is full HADOOP-11604. Prevent ConcurrentModificationException while closing domain sockets during shutdown of DomainSocketWatcher thread. HADOOP-11648. Set DomainSocketWatcher thread name explicitly HADOOP-11802. DomainSocketWatcher thread terminates sometimes after there is an I/O error during requestShortCircuitShm HADOOP-11604 and 11648 are not critical by themselves, but they are pre-requisites to getting a clean cherry-pick of 11802, which we believe finally fixes the root cause of this issue. --Chris Nauroth On 4/30/15, 3:55 PM, Arpit Agarwal aagar...@hortonworks.com wrote: HDFS candidates for back-porting to Hadoop 2.6.1. The first two were requested in [1]. HADOOP-11674. oneByteBuf in CryptoInputStream and CryptoOutputStream should be non static HADOOP-11710
Re: Planning Hadoop 2.6.1 release
Got pinged on a recent thread on this one. As I mentioned there, I had many offline discussions re 2.6.1. The biggest problem I found offline was about what bug-fixes are acceptable and what aren’t for everyone wishing to consume 2.6.1. Given the number of bug-fixes that went into 2.7.x and into branch-2.8, figuring out a set of patches that is acceptable for everyone is a huge challenge which kind of stalled my attempts. Thanks +Vinod On Jul 1, 2015, at 12:41 PM, Sean Busbey bus...@cloudera.com wrote: Any update on a release plan for 2.6.1? On Wed, Jun 10, 2015 at 1:25 AM, Brahma Reddy Battula brahmareddy.batt...@huawei.com wrote: HI vinod any update on this..? are we planning to give 2.6.1 Or can we make 2.7.1 as stable give..? Thanks Regards Brahma Reddy Battula From: Zhihai Xu [z...@cloudera.com] Sent: Wednesday, May 13, 2015 12:04 PM To: mapreduce-dev@hadoop.apache.org Cc: common-...@hadoop.apache.org; yarn-...@hadoop.apache.org; hdfs-...@hadoop.apache.org Subject: Re: Planning Hadoop 2.6.1 release Hi Akira, Can we also include YARN-3242? YARN-3242 fixed a critical ZKRMStateStore bug. It will work better with YARN-2992. thanks zhihai On Tue, May 12, 2015 at 10:38 PM, Akira AJISAKA ajisa...@oss.nttdata.co.jp wrote: Thanks all for collecting jiras for 2.6.1 release. In addition, I'd like to include the following: * HADOOP-11343. Overflow is not properly handled in calculating final iv for AES CTR * YARN-2874. Dead lock in DelegationTokenRenewer which blocks RM to execute any further apps * YARN-2992. ZKRMStateStore crashes due to session expiry * YARN-3013. AMRMClientImpl does not update AMRM token properly * YARN-3369. Missing NullPointer check in AppSchedulingInfo causes RM to die * MAPREDUCE-6303. Read timeout when retrying a fetch error can be fatal to a reducer All of these are marked as blocker bug for 2.7.0 but not fixed in 2.6.0. Regards, Akira On 5/4/15 11:15, Brahma Reddy Battula wrote: Hello Vinod, I am thinking,can we include HADOOP-11491 also..? wihout this jira harfs will not be usable when cluster installed in HA mode and try to get filecontext like below.. Path path = new Path(har:///archivedLogs/application_1428917727658_0005-application_1428917727658_0008-1428927448352.har); FileSystem fs = path.getFileSystem(new Configuration()); path = fs.makeQualified(path); FileContext fc = FileContext.getFileContext(path.toUri(),new Configuration()); Thanks Regards Brahma Reddy Battula From: Chris Nauroth [cnaur...@hortonworks.com] Sent: Friday, May 01, 2015 4:32 AM To: mapreduce-dev@hadoop.apache.org; common-...@hadoop.apache.org; yarn-...@hadoop.apache.org; hdfs-...@hadoop.apache.org Subject: Re: Planning Hadoop 2.6.1 release Thank you, Arpit. In addition, I suggest we include the following: HADOOP-11333. Fix deadlock in DomainSocketWatcher when the notification pipe is full HADOOP-11604. Prevent ConcurrentModificationException while closing domain sockets during shutdown of DomainSocketWatcher thread. HADOOP-11648. Set DomainSocketWatcher thread name explicitly HADOOP-11802. DomainSocketWatcher thread terminates sometimes after there is an I/O error during requestShortCircuitShm HADOOP-11604 and 11648 are not critical by themselves, but they are pre-requisites to getting a clean cherry-pick of 11802, which we believe finally fixes the root cause of this issue. --Chris Nauroth On 4/30/15, 3:55 PM, Arpit Agarwal aagar...@hortonworks.com wrote: HDFS candidates for back-porting to Hadoop 2.6.1. The first two were requested in [1]. HADOOP-11674. oneByteBuf in CryptoInputStream and CryptoOutputStream should be non static HADOOP-11710. Make CryptoOutputStream behave like DFSOutputStream wrt synchronization HDFS-7009. Active NN and standby NN have different live nodes. HDFS-7035. Make adding a new data directory to the DataNode an atomic and improve error handling HDFS-7425. NameNode block deletion logging uses incorrect appender. HDFS-7443. Datanode upgrade to BLOCKID_BASED_LAYOUT fails if duplicate block files are present in the same volume. HDFS-7489. Incorrect locking in FsVolumeList#checkDirs can hang datanodes HDFS-7503. Namenode restart after large deletions can cause slow processReport. HDFS-7575. Upgrade should generate a unique storage ID for each volume. HDFS-7579. Improve log reporting during block report rpc failure. HDFS-7587. Edit log corruption can happen if append fails with a quota violation. HDFS-7596. NameNode should prune dead storages from storageMap. HDFS-7611. deleteSnapshot and delete of a file can leave orphaned blocks in the blocksMap on NameNode restart. HDFS-7714. Simultaneous restart of HA NameNodes and DataNode can cause DataNode to register successfully with only one NameNode. HDFS-7733. NFS: readdir
Re: Planning Hadoop 2.6.1 release
Any update on a release plan for 2.6.1? On Wed, Jun 10, 2015 at 1:25 AM, Brahma Reddy Battula brahmareddy.batt...@huawei.com wrote: HI vinod any update on this..? are we planning to give 2.6.1 Or can we make 2.7.1 as stable give..? Thanks Regards Brahma Reddy Battula From: Zhihai Xu [z...@cloudera.com] Sent: Wednesday, May 13, 2015 12:04 PM To: mapreduce-dev@hadoop.apache.org Cc: common-...@hadoop.apache.org; yarn-...@hadoop.apache.org; hdfs-...@hadoop.apache.org Subject: Re: Planning Hadoop 2.6.1 release Hi Akira, Can we also include YARN-3242? YARN-3242 fixed a critical ZKRMStateStore bug. It will work better with YARN-2992. thanks zhihai On Tue, May 12, 2015 at 10:38 PM, Akira AJISAKA ajisa...@oss.nttdata.co.jp wrote: Thanks all for collecting jiras for 2.6.1 release. In addition, I'd like to include the following: * HADOOP-11343. Overflow is not properly handled in calculating final iv for AES CTR * YARN-2874. Dead lock in DelegationTokenRenewer which blocks RM to execute any further apps * YARN-2992. ZKRMStateStore crashes due to session expiry * YARN-3013. AMRMClientImpl does not update AMRM token properly * YARN-3369. Missing NullPointer check in AppSchedulingInfo causes RM to die * MAPREDUCE-6303. Read timeout when retrying a fetch error can be fatal to a reducer All of these are marked as blocker bug for 2.7.0 but not fixed in 2.6.0. Regards, Akira On 5/4/15 11:15, Brahma Reddy Battula wrote: Hello Vinod, I am thinking,can we include HADOOP-11491 also..? wihout this jira harfs will not be usable when cluster installed in HA mode and try to get filecontext like below.. Path path = new Path(har:///archivedLogs/application_1428917727658_0005-application_1428917727658_0008-1428927448352.har); FileSystem fs = path.getFileSystem(new Configuration()); path = fs.makeQualified(path); FileContext fc = FileContext.getFileContext(path.toUri(),new Configuration()); Thanks Regards Brahma Reddy Battula From: Chris Nauroth [cnaur...@hortonworks.com] Sent: Friday, May 01, 2015 4:32 AM To: mapreduce-dev@hadoop.apache.org; common-...@hadoop.apache.org; yarn-...@hadoop.apache.org; hdfs-...@hadoop.apache.org Subject: Re: Planning Hadoop 2.6.1 release Thank you, Arpit. In addition, I suggest we include the following: HADOOP-11333. Fix deadlock in DomainSocketWatcher when the notification pipe is full HADOOP-11604. Prevent ConcurrentModificationException while closing domain sockets during shutdown of DomainSocketWatcher thread. HADOOP-11648. Set DomainSocketWatcher thread name explicitly HADOOP-11802. DomainSocketWatcher thread terminates sometimes after there is an I/O error during requestShortCircuitShm HADOOP-11604 and 11648 are not critical by themselves, but they are pre-requisites to getting a clean cherry-pick of 11802, which we believe finally fixes the root cause of this issue. --Chris Nauroth On 4/30/15, 3:55 PM, Arpit Agarwal aagar...@hortonworks.com wrote: HDFS candidates for back-porting to Hadoop 2.6.1. The first two were requested in [1]. HADOOP-11674. oneByteBuf in CryptoInputStream and CryptoOutputStream should be non static HADOOP-11710. Make CryptoOutputStream behave like DFSOutputStream wrt synchronization HDFS-7009. Active NN and standby NN have different live nodes. HDFS-7035. Make adding a new data directory to the DataNode an atomic and improve error handling HDFS-7425. NameNode block deletion logging uses incorrect appender. HDFS-7443. Datanode upgrade to BLOCKID_BASED_LAYOUT fails if duplicate block files are present in the same volume. HDFS-7489. Incorrect locking in FsVolumeList#checkDirs can hang datanodes HDFS-7503. Namenode restart after large deletions can cause slow processReport. HDFS-7575. Upgrade should generate a unique storage ID for each volume. HDFS-7579. Improve log reporting during block report rpc failure. HDFS-7587. Edit log corruption can happen if append fails with a quota violation. HDFS-7596. NameNode should prune dead storages from storageMap. HDFS-7611. deleteSnapshot and delete of a file can leave orphaned blocks in the blocksMap on NameNode restart. HDFS-7714. Simultaneous restart of HA NameNodes and DataNode can cause DataNode to register successfully with only one NameNode. HDFS-7733. NFS: readdir/readdirplus return null directory attribute on failure. HDFS-7831. Fix the starting index and end condition of the loop in FileDiffList.findEarlierSnapshotBlocks(). HDFS-7885. Datanode should not trust the generation stamp provided by client. HDFS-7960. The full block report should prune zombie storages even if they're not empty. HDFS-8072. Reserved RBW space is not released if client terminates while writing block. HDFS-8127. NameNode
RE: Planning Hadoop 2.6.1 release
HI vinod any update on this..? are we planning to give 2.6.1 Or can we make 2.7.1 as stable give..? Thanks Regards Brahma Reddy Battula From: Zhihai Xu [z...@cloudera.com] Sent: Wednesday, May 13, 2015 12:04 PM To: mapreduce-dev@hadoop.apache.org Cc: common-...@hadoop.apache.org; yarn-...@hadoop.apache.org; hdfs-...@hadoop.apache.org Subject: Re: Planning Hadoop 2.6.1 release Hi Akira, Can we also include YARN-3242? YARN-3242 fixed a critical ZKRMStateStore bug. It will work better with YARN-2992. thanks zhihai On Tue, May 12, 2015 at 10:38 PM, Akira AJISAKA ajisa...@oss.nttdata.co.jp wrote: Thanks all for collecting jiras for 2.6.1 release. In addition, I'd like to include the following: * HADOOP-11343. Overflow is not properly handled in calculating final iv for AES CTR * YARN-2874. Dead lock in DelegationTokenRenewer which blocks RM to execute any further apps * YARN-2992. ZKRMStateStore crashes due to session expiry * YARN-3013. AMRMClientImpl does not update AMRM token properly * YARN-3369. Missing NullPointer check in AppSchedulingInfo causes RM to die * MAPREDUCE-6303. Read timeout when retrying a fetch error can be fatal to a reducer All of these are marked as blocker bug for 2.7.0 but not fixed in 2.6.0. Regards, Akira On 5/4/15 11:15, Brahma Reddy Battula wrote: Hello Vinod, I am thinking,can we include HADOOP-11491 also..? wihout this jira harfs will not be usable when cluster installed in HA mode and try to get filecontext like below.. Path path = new Path(har:///archivedLogs/application_1428917727658_0005-application_1428917727658_0008-1428927448352.har); FileSystem fs = path.getFileSystem(new Configuration()); path = fs.makeQualified(path); FileContext fc = FileContext.getFileContext(path.toUri(),new Configuration()); Thanks Regards Brahma Reddy Battula From: Chris Nauroth [cnaur...@hortonworks.com] Sent: Friday, May 01, 2015 4:32 AM To: mapreduce-dev@hadoop.apache.org; common-...@hadoop.apache.org; yarn-...@hadoop.apache.org; hdfs-...@hadoop.apache.org Subject: Re: Planning Hadoop 2.6.1 release Thank you, Arpit. In addition, I suggest we include the following: HADOOP-11333. Fix deadlock in DomainSocketWatcher when the notification pipe is full HADOOP-11604. Prevent ConcurrentModificationException while closing domain sockets during shutdown of DomainSocketWatcher thread. HADOOP-11648. Set DomainSocketWatcher thread name explicitly HADOOP-11802. DomainSocketWatcher thread terminates sometimes after there is an I/O error during requestShortCircuitShm HADOOP-11604 and 11648 are not critical by themselves, but they are pre-requisites to getting a clean cherry-pick of 11802, which we believe finally fixes the root cause of this issue. --Chris Nauroth On 4/30/15, 3:55 PM, Arpit Agarwal aagar...@hortonworks.com wrote: HDFS candidates for back-porting to Hadoop 2.6.1. The first two were requested in [1]. HADOOP-11674. oneByteBuf in CryptoInputStream and CryptoOutputStream should be non static HADOOP-11710. Make CryptoOutputStream behave like DFSOutputStream wrt synchronization HDFS-7009. Active NN and standby NN have different live nodes. HDFS-7035. Make adding a new data directory to the DataNode an atomic and improve error handling HDFS-7425. NameNode block deletion logging uses incorrect appender. HDFS-7443. Datanode upgrade to BLOCKID_BASED_LAYOUT fails if duplicate block files are present in the same volume. HDFS-7489. Incorrect locking in FsVolumeList#checkDirs can hang datanodes HDFS-7503. Namenode restart after large deletions can cause slow processReport. HDFS-7575. Upgrade should generate a unique storage ID for each volume. HDFS-7579. Improve log reporting during block report rpc failure. HDFS-7587. Edit log corruption can happen if append fails with a quota violation. HDFS-7596. NameNode should prune dead storages from storageMap. HDFS-7611. deleteSnapshot and delete of a file can leave orphaned blocks in the blocksMap on NameNode restart. HDFS-7714. Simultaneous restart of HA NameNodes and DataNode can cause DataNode to register successfully with only one NameNode. HDFS-7733. NFS: readdir/readdirplus return null directory attribute on failure. HDFS-7831. Fix the starting index and end condition of the loop in FileDiffList.findEarlierSnapshotBlocks(). HDFS-7885. Datanode should not trust the generation stamp provided by client. HDFS-7960. The full block report should prune zombie storages even if they're not empty. HDFS-8072. Reserved RBW space is not released if client terminates while writing block. HDFS-8127. NameNode Failover during HA upgrade can cause DataNode to finalize upgrade. Arpit [1] Will Hadoop 2.6.1 be released soon? http://markmail.org/thread/zlsr6prejyogdyvh On 4/27/15, 11:47 AM, Vinod Kumar Vavilapalli vino...@apache.org wrote: There were several
Re: Planning Hadoop 2.6.1 release
Hi Akira, Can we also include YARN-3242? YARN-3242 fixed a critical ZKRMStateStore bug. It will work better with YARN-2992. thanks zhihai On Tue, May 12, 2015 at 10:38 PM, Akira AJISAKA ajisa...@oss.nttdata.co.jp wrote: Thanks all for collecting jiras for 2.6.1 release. In addition, I'd like to include the following: * HADOOP-11343. Overflow is not properly handled in calculating final iv for AES CTR * YARN-2874. Dead lock in DelegationTokenRenewer which blocks RM to execute any further apps * YARN-2992. ZKRMStateStore crashes due to session expiry * YARN-3013. AMRMClientImpl does not update AMRM token properly * YARN-3369. Missing NullPointer check in AppSchedulingInfo causes RM to die * MAPREDUCE-6303. Read timeout when retrying a fetch error can be fatal to a reducer All of these are marked as blocker bug for 2.7.0 but not fixed in 2.6.0. Regards, Akira On 5/4/15 11:15, Brahma Reddy Battula wrote: Hello Vinod, I am thinking,can we include HADOOP-11491 also..? wihout this jira harfs will not be usable when cluster installed in HA mode and try to get filecontext like below.. Path path = new Path(har:///archivedLogs/application_1428917727658_0005-application_1428917727658_0008-1428927448352.har); FileSystem fs = path.getFileSystem(new Configuration()); path = fs.makeQualified(path); FileContext fc = FileContext.getFileContext(path.toUri(),new Configuration()); Thanks Regards Brahma Reddy Battula From: Chris Nauroth [cnaur...@hortonworks.com] Sent: Friday, May 01, 2015 4:32 AM To: mapreduce-dev@hadoop.apache.org; common-...@hadoop.apache.org; yarn-...@hadoop.apache.org; hdfs-...@hadoop.apache.org Subject: Re: Planning Hadoop 2.6.1 release Thank you, Arpit. In addition, I suggest we include the following: HADOOP-11333. Fix deadlock in DomainSocketWatcher when the notification pipe is full HADOOP-11604. Prevent ConcurrentModificationException while closing domain sockets during shutdown of DomainSocketWatcher thread. HADOOP-11648. Set DomainSocketWatcher thread name explicitly HADOOP-11802. DomainSocketWatcher thread terminates sometimes after there is an I/O error during requestShortCircuitShm HADOOP-11604 and 11648 are not critical by themselves, but they are pre-requisites to getting a clean cherry-pick of 11802, which we believe finally fixes the root cause of this issue. --Chris Nauroth On 4/30/15, 3:55 PM, Arpit Agarwal aagar...@hortonworks.com wrote: HDFS candidates for back-porting to Hadoop 2.6.1. The first two were requested in [1]. HADOOP-11674. oneByteBuf in CryptoInputStream and CryptoOutputStream should be non static HADOOP-11710. Make CryptoOutputStream behave like DFSOutputStream wrt synchronization HDFS-7009. Active NN and standby NN have different live nodes. HDFS-7035. Make adding a new data directory to the DataNode an atomic and improve error handling HDFS-7425. NameNode block deletion logging uses incorrect appender. HDFS-7443. Datanode upgrade to BLOCKID_BASED_LAYOUT fails if duplicate block files are present in the same volume. HDFS-7489. Incorrect locking in FsVolumeList#checkDirs can hang datanodes HDFS-7503. Namenode restart after large deletions can cause slow processReport. HDFS-7575. Upgrade should generate a unique storage ID for each volume. HDFS-7579. Improve log reporting during block report rpc failure. HDFS-7587. Edit log corruption can happen if append fails with a quota violation. HDFS-7596. NameNode should prune dead storages from storageMap. HDFS-7611. deleteSnapshot and delete of a file can leave orphaned blocks in the blocksMap on NameNode restart. HDFS-7714. Simultaneous restart of HA NameNodes and DataNode can cause DataNode to register successfully with only one NameNode. HDFS-7733. NFS: readdir/readdirplus return null directory attribute on failure. HDFS-7831. Fix the starting index and end condition of the loop in FileDiffList.findEarlierSnapshotBlocks(). HDFS-7885. Datanode should not trust the generation stamp provided by client. HDFS-7960. The full block report should prune zombie storages even if they're not empty. HDFS-8072. Reserved RBW space is not released if client terminates while writing block. HDFS-8127. NameNode Failover during HA upgrade can cause DataNode to finalize upgrade. Arpit [1] Will Hadoop 2.6.1 be released soon? http://markmail.org/thread/zlsr6prejyogdyvh On 4/27/15, 11:47 AM, Vinod Kumar Vavilapalli vino...@apache.org wrote: There were several requests on the user lists [1] for a 2.6.1 release. I got many offline comments too. Planning to do a 2.6.1 release in a few weeks time. We already have a bunch of tickets committed to 2.7.1. I created a filter [2] to tracking pending tickets. We need to collectively come up with a list of critical issues. We can use the JIRA Target Version field for the same. I see some but not a whole lot of new work for this release
Re: Planning Hadoop 2.6.1 release
Thanks all for collecting jiras for 2.6.1 release. In addition, I'd like to include the following: * HADOOP-11343. Overflow is not properly handled in calculating final iv for AES CTR * YARN-2874. Dead lock in DelegationTokenRenewer which blocks RM to execute any further apps * YARN-2992. ZKRMStateStore crashes due to session expiry * YARN-3013. AMRMClientImpl does not update AMRM token properly * YARN-3369. Missing NullPointer check in AppSchedulingInfo causes RM to die * MAPREDUCE-6303. Read timeout when retrying a fetch error can be fatal to a reducer All of these are marked as blocker bug for 2.7.0 but not fixed in 2.6.0. Regards, Akira On 5/4/15 11:15, Brahma Reddy Battula wrote: Hello Vinod, I am thinking,can we include HADOOP-11491 also..? wihout this jira harfs will not be usable when cluster installed in HA mode and try to get filecontext like below.. Path path = new Path(har:///archivedLogs/application_1428917727658_0005-application_1428917727658_0008-1428927448352.har); FileSystem fs = path.getFileSystem(new Configuration()); path = fs.makeQualified(path); FileContext fc = FileContext.getFileContext(path.toUri(),new Configuration()); Thanks Regards Brahma Reddy Battula From: Chris Nauroth [cnaur...@hortonworks.com] Sent: Friday, May 01, 2015 4:32 AM To: mapreduce-dev@hadoop.apache.org; common-...@hadoop.apache.org; yarn-...@hadoop.apache.org; hdfs-...@hadoop.apache.org Subject: Re: Planning Hadoop 2.6.1 release Thank you, Arpit. In addition, I suggest we include the following: HADOOP-11333. Fix deadlock in DomainSocketWatcher when the notification pipe is full HADOOP-11604. Prevent ConcurrentModificationException while closing domain sockets during shutdown of DomainSocketWatcher thread. HADOOP-11648. Set DomainSocketWatcher thread name explicitly HADOOP-11802. DomainSocketWatcher thread terminates sometimes after there is an I/O error during requestShortCircuitShm HADOOP-11604 and 11648 are not critical by themselves, but they are pre-requisites to getting a clean cherry-pick of 11802, which we believe finally fixes the root cause of this issue. --Chris Nauroth On 4/30/15, 3:55 PM, Arpit Agarwal aagar...@hortonworks.com wrote: HDFS candidates for back-porting to Hadoop 2.6.1. The first two were requested in [1]. HADOOP-11674. oneByteBuf in CryptoInputStream and CryptoOutputStream should be non static HADOOP-11710. Make CryptoOutputStream behave like DFSOutputStream wrt synchronization HDFS-7009. Active NN and standby NN have different live nodes. HDFS-7035. Make adding a new data directory to the DataNode an atomic and improve error handling HDFS-7425. NameNode block deletion logging uses incorrect appender. HDFS-7443. Datanode upgrade to BLOCKID_BASED_LAYOUT fails if duplicate block files are present in the same volume. HDFS-7489. Incorrect locking in FsVolumeList#checkDirs can hang datanodes HDFS-7503. Namenode restart after large deletions can cause slow processReport. HDFS-7575. Upgrade should generate a unique storage ID for each volume. HDFS-7579. Improve log reporting during block report rpc failure. HDFS-7587. Edit log corruption can happen if append fails with a quota violation. HDFS-7596. NameNode should prune dead storages from storageMap. HDFS-7611. deleteSnapshot and delete of a file can leave orphaned blocks in the blocksMap on NameNode restart. HDFS-7714. Simultaneous restart of HA NameNodes and DataNode can cause DataNode to register successfully with only one NameNode. HDFS-7733. NFS: readdir/readdirplus return null directory attribute on failure. HDFS-7831. Fix the starting index and end condition of the loop in FileDiffList.findEarlierSnapshotBlocks(). HDFS-7885. Datanode should not trust the generation stamp provided by client. HDFS-7960. The full block report should prune zombie storages even if they're not empty. HDFS-8072. Reserved RBW space is not released if client terminates while writing block. HDFS-8127. NameNode Failover during HA upgrade can cause DataNode to finalize upgrade. Arpit [1] Will Hadoop 2.6.1 be released soon? http://markmail.org/thread/zlsr6prejyogdyvh On 4/27/15, 11:47 AM, Vinod Kumar Vavilapalli vino...@apache.org wrote: There were several requests on the user lists [1] for a 2.6.1 release. I got many offline comments too. Planning to do a 2.6.1 release in a few weeks time. We already have a bunch of tickets committed to 2.7.1. I created a filter [2] to tracking pending tickets. We need to collectively come up with a list of critical issues. We can use the JIRA Target Version field for the same. I see some but not a whole lot of new work for this release, most of it is likely going to be pulling in critical patches from 2.7.1/2.8 etc. Thoughts? Thanks +Vinod [1] Will Hadoop 2.6.1 be released soon? http://markmail.org/thread/zlsr6prejyogdyvh [2] 2.6.1 pending tickets https://issues.apache.org/jira/issues/?filter=12331711
RE: Planning Hadoop 2.6.1 release
Hello Vinod, I am thinking,can we include HADOOP-11491 also..? wihout this jira harfs will not be usable when cluster installed in HA mode and try to get filecontext like below.. Path path = new Path(har:///archivedLogs/application_1428917727658_0005-application_1428917727658_0008-1428927448352.har); FileSystem fs = path.getFileSystem(new Configuration()); path = fs.makeQualified(path); FileContext fc = FileContext.getFileContext(path.toUri(),new Configuration()); Thanks Regards Brahma Reddy Battula From: Chris Nauroth [cnaur...@hortonworks.com] Sent: Friday, May 01, 2015 4:32 AM To: mapreduce-dev@hadoop.apache.org; common-...@hadoop.apache.org; yarn-...@hadoop.apache.org; hdfs-...@hadoop.apache.org Subject: Re: Planning Hadoop 2.6.1 release Thank you, Arpit. In addition, I suggest we include the following: HADOOP-11333. Fix deadlock in DomainSocketWatcher when the notification pipe is full HADOOP-11604. Prevent ConcurrentModificationException while closing domain sockets during shutdown of DomainSocketWatcher thread. HADOOP-11648. Set DomainSocketWatcher thread name explicitly HADOOP-11802. DomainSocketWatcher thread terminates sometimes after there is an I/O error during requestShortCircuitShm HADOOP-11604 and 11648 are not critical by themselves, but they are pre-requisites to getting a clean cherry-pick of 11802, which we believe finally fixes the root cause of this issue. --Chris Nauroth On 4/30/15, 3:55 PM, Arpit Agarwal aagar...@hortonworks.com wrote: HDFS candidates for back-porting to Hadoop 2.6.1. The first two were requested in [1]. HADOOP-11674. oneByteBuf in CryptoInputStream and CryptoOutputStream should be non static HADOOP-11710. Make CryptoOutputStream behave like DFSOutputStream wrt synchronization HDFS-7009. Active NN and standby NN have different live nodes. HDFS-7035. Make adding a new data directory to the DataNode an atomic and improve error handling HDFS-7425. NameNode block deletion logging uses incorrect appender. HDFS-7443. Datanode upgrade to BLOCKID_BASED_LAYOUT fails if duplicate block files are present in the same volume. HDFS-7489. Incorrect locking in FsVolumeList#checkDirs can hang datanodes HDFS-7503. Namenode restart after large deletions can cause slow processReport. HDFS-7575. Upgrade should generate a unique storage ID for each volume. HDFS-7579. Improve log reporting during block report rpc failure. HDFS-7587. Edit log corruption can happen if append fails with a quota violation. HDFS-7596. NameNode should prune dead storages from storageMap. HDFS-7611. deleteSnapshot and delete of a file can leave orphaned blocks in the blocksMap on NameNode restart. HDFS-7714. Simultaneous restart of HA NameNodes and DataNode can cause DataNode to register successfully with only one NameNode. HDFS-7733. NFS: readdir/readdirplus return null directory attribute on failure. HDFS-7831. Fix the starting index and end condition of the loop in FileDiffList.findEarlierSnapshotBlocks(). HDFS-7885. Datanode should not trust the generation stamp provided by client. HDFS-7960. The full block report should prune zombie storages even if they're not empty. HDFS-8072. Reserved RBW space is not released if client terminates while writing block. HDFS-8127. NameNode Failover during HA upgrade can cause DataNode to finalize upgrade. Arpit [1] Will Hadoop 2.6.1 be released soon? http://markmail.org/thread/zlsr6prejyogdyvh On 4/27/15, 11:47 AM, Vinod Kumar Vavilapalli vino...@apache.org wrote: There were several requests on the user lists [1] for a 2.6.1 release. I got many offline comments too. Planning to do a 2.6.1 release in a few weeks time. We already have a bunch of tickets committed to 2.7.1. I created a filter [2] to tracking pending tickets. We need to collectively come up with a list of critical issues. We can use the JIRA Target Version field for the same. I see some but not a whole lot of new work for this release, most of it is likely going to be pulling in critical patches from 2.7.1/2.8 etc. Thoughts? Thanks +Vinod [1] Will Hadoop 2.6.1 be released soon? http://markmail.org/thread/zlsr6prejyogdyvh [2] 2.6.1 pending tickets https://issues.apache.org/jira/issues/?filter=12331711
Re: Planning Hadoop 2.6.1 release
Thank you, Arpit. In addition, I suggest we include the following: HADOOP-11333. Fix deadlock in DomainSocketWatcher when the notification pipe is full HADOOP-11604. Prevent ConcurrentModificationException while closing domain sockets during shutdown of DomainSocketWatcher thread. HADOOP-11648. Set DomainSocketWatcher thread name explicitly HADOOP-11802. DomainSocketWatcher thread terminates sometimes after there is an I/O error during requestShortCircuitShm HADOOP-11604 and 11648 are not critical by themselves, but they are pre-requisites to getting a clean cherry-pick of 11802, which we believe finally fixes the root cause of this issue. --Chris Nauroth On 4/30/15, 3:55 PM, Arpit Agarwal aagar...@hortonworks.com wrote: HDFS candidates for back-porting to Hadoop 2.6.1. The first two were requested in [1]. HADOOP-11674. oneByteBuf in CryptoInputStream and CryptoOutputStream should be non static HADOOP-11710. Make CryptoOutputStream behave like DFSOutputStream wrt synchronization HDFS-7009. Active NN and standby NN have different live nodes. HDFS-7035. Make adding a new data directory to the DataNode an atomic and improve error handling HDFS-7425. NameNode block deletion logging uses incorrect appender. HDFS-7443. Datanode upgrade to BLOCKID_BASED_LAYOUT fails if duplicate block files are present in the same volume. HDFS-7489. Incorrect locking in FsVolumeList#checkDirs can hang datanodes HDFS-7503. Namenode restart after large deletions can cause slow processReport. HDFS-7575. Upgrade should generate a unique storage ID for each volume. HDFS-7579. Improve log reporting during block report rpc failure. HDFS-7587. Edit log corruption can happen if append fails with a quota violation. HDFS-7596. NameNode should prune dead storages from storageMap. HDFS-7611. deleteSnapshot and delete of a file can leave orphaned blocks in the blocksMap on NameNode restart. HDFS-7714. Simultaneous restart of HA NameNodes and DataNode can cause DataNode to register successfully with only one NameNode. HDFS-7733. NFS: readdir/readdirplus return null directory attribute on failure. HDFS-7831. Fix the starting index and end condition of the loop in FileDiffList.findEarlierSnapshotBlocks(). HDFS-7885. Datanode should not trust the generation stamp provided by client. HDFS-7960. The full block report should prune zombie storages even if they're not empty. HDFS-8072. Reserved RBW space is not released if client terminates while writing block. HDFS-8127. NameNode Failover during HA upgrade can cause DataNode to finalize upgrade. Arpit [1] Will Hadoop 2.6.1 be released soon? http://markmail.org/thread/zlsr6prejyogdyvh On 4/27/15, 11:47 AM, Vinod Kumar Vavilapalli vino...@apache.org wrote: There were several requests on the user lists [1] for a 2.6.1 release. I got many offline comments too. Planning to do a 2.6.1 release in a few weeks time. We already have a bunch of tickets committed to 2.7.1. I created a filter [2] to tracking pending tickets. We need to collectively come up with a list of critical issues. We can use the JIRA Target Version field for the same. I see some but not a whole lot of new work for this release, most of it is likely going to be pulling in critical patches from 2.7.1/2.8 etc. Thoughts? Thanks +Vinod [1] Will Hadoop 2.6.1 be released soon? http://markmail.org/thread/zlsr6prejyogdyvh [2] 2.6.1 pending tickets https://issues.apache.org/jira/issues/?filter=12331711
Re: Planning Hadoop 2.6.1 release
HDFS candidates for back-porting to Hadoop 2.6.1. The first two were requested in [1]. HADOOP-11674. oneByteBuf in CryptoInputStream and CryptoOutputStream should be non static HADOOP-11710. Make CryptoOutputStream behave like DFSOutputStream wrt synchronization HDFS-7009. Active NN and standby NN have different live nodes. HDFS-7035. Make adding a new data directory to the DataNode an atomic and improve error handling HDFS-7425. NameNode block deletion logging uses incorrect appender. HDFS-7443. Datanode upgrade to BLOCKID_BASED_LAYOUT fails if duplicate block files are present in the same volume. HDFS-7489. Incorrect locking in FsVolumeList#checkDirs can hang datanodes HDFS-7503. Namenode restart after large deletions can cause slow processReport. HDFS-7575. Upgrade should generate a unique storage ID for each volume. HDFS-7579. Improve log reporting during block report rpc failure. HDFS-7587. Edit log corruption can happen if append fails with a quota violation. HDFS-7596. NameNode should prune dead storages from storageMap. HDFS-7611. deleteSnapshot and delete of a file can leave orphaned blocks in the blocksMap on NameNode restart. HDFS-7714. Simultaneous restart of HA NameNodes and DataNode can cause DataNode to register successfully with only one NameNode. HDFS-7733. NFS: readdir/readdirplus return null directory attribute on failure. HDFS-7831. Fix the starting index and end condition of the loop in FileDiffList.findEarlierSnapshotBlocks(). HDFS-7885. Datanode should not trust the generation stamp provided by client. HDFS-7960. The full block report should prune zombie storages even if they're not empty. HDFS-8072. Reserved RBW space is not released if client terminates while writing block. HDFS-8127. NameNode Failover during HA upgrade can cause DataNode to finalize upgrade. Arpit [1] Will Hadoop 2.6.1 be released soon? http://markmail.org/thread/zlsr6prejyogdyvh On 4/27/15, 11:47 AM, Vinod Kumar Vavilapalli vino...@apache.org wrote: There were several requests on the user lists [1] for a 2.6.1 release. I got many offline comments too. Planning to do a 2.6.1 release in a few weeks time. We already have a bunch of tickets committed to 2.7.1. I created a filter [2] to tracking pending tickets. We need to collectively come up with a list of critical issues. We can use the JIRA Target Version field for the same. I see some but not a whole lot of new work for this release, most of it is likely going to be pulling in critical patches from 2.7.1/2.8 etc. Thoughts? Thanks +Vinod [1] Will Hadoop 2.6.1 be released soon? http://markmail.org/thread/zlsr6prejyogdyvh [2] 2.6.1 pending tickets https://issues.apache.org/jira/issues/?filter=12331711
