Re: [VOTE] Release Apache Hadoop 2.6.1 RC0

[Vinod Vavilapalli]

Thanks for the verification Nick!

Regarding that one issue, we saw this before itself in 2.7 and deemed it to be 
a compatible change given the expected usage of the API. The error in your 
report is “a client class C is not abstract and does not override abstract 
method in ApplicationBaseProtocol.”, but we do not expect clients to extend 
this communication protocol, only bind to it as a calling interface.

Your report was useful in general otherwise though as I forgot to run jdiff 
again! With this, things are looking good on the latest RC that I am going to 
push out now.

Thanks
+Vinod

On Sep 16, 2015, at 12:05 PM, Nick Dimiduk 
mailto:ndimi...@apache.org>> wrote:

Hi folks,

I ran a modified version [0] of the compatibility checking tool we use in HBase 
over this release. There's one source-related issue [1] in YARN API that may 
need to be addressed in the next candidate. Otherwise, it's looking pretty 
clean for a patch release. Nice job!

[0]: http://people.apache.org/~ndimiduk/check_compat_hadoop.patch
[1]: 
http://people.apache.org/~ndimiduk/release-2.6.0_release-2.6.1-RC0_compat_report.html

On Mon, Sep 14, 2015 at 11:28 AM, Vinod Kumar Vavilapalli 
mailto:vino...@hortonworks.com>> wrote:
One more popped in from  Jeff Zhang from Apache Tez: YARN-4154.

Looking at that too for RC1.

Thanks
+Vinod

> On Sep 11, 2015, at 11:37 AM, Vinod Kumar Vavilapalli 
> mailto:vino...@hortonworks.com>> wrote:
>
> Thanks Sangjin and Akira for finding the bug and the fix!
>
> This fix is needed because YARN-1809 was pulled in. I further found 3 more 
> related fixes that need to be pulled in because of YARN-1809: YARN-1884 and 
> YARN-3544 (UI fixes), YARN-3740 (to avoid incompat).
>
> This VOTE thread is closed now, I’ll spin a new RC with those patches.
>
> Thanks
> +Vinod
>
>
>> On Sep 11, 2015, at 2:40 AM, Akira AJISAKA 
>> mailto:ajisa...@oss.nttdata.co.jp>> wrote:
>>
>> I could reproduce the issue, and I found YARN-3171 will fix it.
>>
>> Hi Vinod, would you include YARN-3171?
>> I applied the patch to branch-2.6.1 locally and confirmed the issue was 
>> fixed.
>>
>> Regards,
>> Akira
>>
>> On 9/11/15 09:16, Sangjin Lee wrote:
>>> I verified the signatures for both source and the binary tarballs. I
>>> started up a pseudo-distributed cluster, and tested simple apps such as
>>> sleep and terasort.
>>>
>>> I do see one issue with the RM UI where the sorting by id is broken. The
>>> table is not rendered in the expected id-descending order, and when I click
>>> the sort control, nothing happens. Sorting by other columns works fine.
>>>
>>> Is anyone else able to reproduce the issue? I checked 2.6.0, and it works
>>> fine on 2.6.0.
>>>
>>> On Wed, Sep 9, 2015 at 6:00 PM, Vinod Kumar Vavilapalli 
>>> mailto:vino...@apache.org>>
>>> wrote:
>>>
 Hi all,

 After a nearly month long [1] toil, with loads of help from Sangjin Lee
 and Akira Ajisaka, and 153 commits later, I've created a release candidate
 RC0 for hadoop-2.6.1.

 The RC is available at:
 http://people.apache.org/~vinodkv/hadoop-2.6.1-RC0/

 The RC tag in git is: release-2.6.1-RC0

 The maven artifacts are available via 
 repository.apache.org at
 https://repository.apache.org/content/repositories/orgapachehadoop-1020

 Some notes from our release process
 -  - Sangjin and I moved out a bunch of items pending from 2.6.1 [2] -
 non-committed but desired patches. 2.6.1 is already big as is and is late
 by any standard, we can definitely include them in the next release.
 - The 2.6.1 wiki page [3] captures some (but not all) of the context of
 the patches that we pushed in.
 - Given the number of fixes pushed [4] in, we had to make a bunch of
 changes to our original plan - we added a few improvements that helped us
 backport patches easier (or in many cases made backports possible), and we
 dropped a few that didn't make sense (HDFS-7831, HDFS-7926, HDFS-7676,
 HDFS-7611, HDFS-7843, HDFS-8850).
 - I ran all the unit tests which (surprisingly?) passed. (Except for one,
 which pointed out a missing fix HDFS-7552).

 As discussed before [5]
 - This release is the first point release after 2.6.0
 - I’d like to use this as a starting release for 2.6.2 in a few weeks
 and then follow up with more of these.

 Please try the release and vote; the vote will run for the usual 5 days.

 Thanks,
 Vinod

 [1] Hadoop 2.6.1 Release process thread:
 http://markmail.org/thread/wkbgkxkhntx5tlux
 [2] 2.6.1 Pending tickets:
 https://issues.apache.org/jira/issues/?filter=12331711
 [3] 2.6.1 Wiki page:
 https://wiki.apache.org/hadoop/Release-2.6.1-Working-Notes
 [4] List of 2.6.1 patches pushed:
 https://issues.apache.org/jira/issues/?jql=fixVersion%20%3D%202.6.1%20and%20labels%20%3D%20%222.6.1-candidate%22
 [5] Planning Hadoop 2.6.1 release:
 http:/

Re: [VOTE] Release Apache Hadoop 2.6.1 RC1

[Vinod Vavilapalli]

Tx Eric, given the long cycle for 2.6.1, I’ll put MAPREDUCE-6334 into 2.6.2.

Thanks
+Vinod

On Sep 21, 2015, at 11:19 AM, Eric Payne 
mailto:eric.payne1...@yahoo.com>> wrote:

Hi Vinod and everyone else who helped on this release!

Thank you very much for going through the work and effort to put this release 
together.

While doing my testing for Hadoop 2.6.1 RC1, I encountered the following issue:
[MAPREDUCE-6334] Fetcher#copyMapOutput is leaking usedMemory upon IOException 
during InMemoryMapOutput shuffle handler - ASF 
JIRA












[MAPREDUCE-6334] Fetcher#copyMapOutput is leaking usedMemory upon IOException 
during ...
We are seeing this happen when an NM's disk goes bad during the creation of map 
output(s) the reducer's fetcher can read the shuffle header and reserve the 
memory


View on issues.apache.org

Preview by Yahoo





This is actually something we ran across frequently when running 2.6. You may 
want to consider pulling that in to 2.6.

Other than that, it looks fine. I did the following manual testing on a 
one-node cluster:
- successfully downloaded and compiled the code
- successfully ran streaming jobs
- successfully ran distributed shell jobs
-- as part of the distributed shell testing, I started the jobs with the 
'-keep_containers_across_application_attempts' property set, killed the AM 
container, and verified that it would in fact keep the containers running 
across AM restarts.
- sucessfully ran wordcount jobs.

Thank you,
-Eric Payne


From: Vinod Kumar Vavilapalli mailto:vino...@apache.org>>
To: common-...@hadoop.apache.org; 
yarn-...@hadoop.apache.org; 
hdfs-...@hadoop.apache.org; 
mapreduce-dev@hadoop.apache.org
Cc: vino...@apache.org; Sangjin Lee 
mailto:sjl...@gmail.com>>; Akira AJISAKA 
mailto:ajisa...@oss.nttdata.co.jp>>
Sent: Wednesday, September 16, 2015 9:10 PM
Subject: [VOTE] Release Apache Hadoop 2.6.1 RC1

Hi all,

After a nearly month long [1] toil, with loads of help from Sangjin Lee and
Akira Ajisaka, and 153 (RC0)+7(RC1) commits later, I've created a release
candidate RC1 for hadoop-2.6.1.

RC1 is RC0 [0] (for which I opened and closed a vote last week) + UI fixes
for the issue Sangjin raised (YARN-3171 and the dependencies YARN-3779,
YARN-3248), additional fix to avoid incompatibility (YARN-3740), other UI
bugs (YARN-1884, YARN-3544) and the MiniYARNCluster issue (right patch for
YARN-2890) that Jeff Zhang raised.

The RC is available at: http://people.apache.org/~vinodkv/hadoop-2.6.1-RC1/

The RC tag in git is: release-2.6.1-RC1

The maven artifacts are available via 
repository.apache.org at
https://repository.apache.org/content/repositories/orgapachehadoop-1021

Some notes from our release process
-  - Sangjin and I moved out a bunch of items pending from 2.6.1 [2] -
non-committed but desired patches. 2.6.1 is already big as is and is late
by any standard, we can definitely include them in the next release.
- The 2.6.1 wiki page [3] captures some (but not all) of the context of
the patches that we pushed in.
- Given the number of fixes pushed [4] in, we had to make a bunch of
changes to our original plan - we added a few improvements that helped us
backport patches easier (or in many cases made backports possible), and we
dropped a few that didn't make sense (HDFS-7831, HDFS-7926, HDFS-7676,
HDFS-7611, HDFS-7843, HDFS-8850).
- I ran all the unit tests which (surprisingly?) passed. (Except for one,
which pointed out a missing fix HDFS-7552).

As discussed before [5]
- This release is the first point release after 2.6.0
- I’d like to use this as a starting release for 2.6.2 in a few weeks and
then follow up with more of these.

Please try the release and vote; the vote will run for the usual 5 days.

Thanks,
Vinod

[0] Hadoop 2.6.1 RC0 vote: http://markmail.org/thread/ubut2rn3lodc55iy
[1] Hadoop 2.6.1 Release process thread:
http://markmail.org/thread/wkbgkxkhntx5tlux
[2] 2.6.1 Pending tickets:
https://issues.apache.org/jira/issues/?filter=12331711
[3] 2.6.1 Wiki page: https://wiki.apache.org/hadoop/Release-2.6.1
-Working-Notes
[4] List of 2.6.1 patches pushed:
https://issues.apache.org/jira/issues/?jql=fixVersion%20%3D%202.6.1
%20and%20labels%20%3D%20%222.6.1-candidate%22
[5] Planning Hadoop 2.6.1 release:
http://markmail.org/thread/sbykjn5xgnksh6wg

PS:
- Note that branch-2.6 which will be the base for 2.6.2 doesn't have these
fixes yet. Once 2.6.1 goes through, I plan to rebase branch-2.6 based off
2.6.1.
- The additional patches in RC1 that got into 2.6.1 all the way from 2.8
are NOT in 2.7.2 yet, this will be done as a followup.

Re: [VOTE] Release Apache Hadoop 2.6.1 RC1

[Vinod Vavilapalli]

I’ve been out sick for a while, so couldn’t close this in time. Doing so now.

+Vinod

On Sep 16, 2015, at 7:10 PM, Vinod Kumar Vavilapalli 
mailto:vino...@apache.org>> wrote:

Hi all,

After a nearly month long [1] toil, with loads of help from Sangjin Lee and 
Akira Ajisaka, and 153 (RC0)+7(RC1) commits later, I've created a release 
candidate RC1 for hadoop-2.6.1.

RC1 is RC0 [0] (for which I opened and closed a vote last week) + UI fixes for 
the issue Sangjin raised (YARN-3171 and the dependencies YARN-3779, YARN-3248), 
additional fix to avoid incompatibility (YARN-3740), other UI bugs (YARN-1884, 
YARN-3544) and the MiniYARNCluster issue (right patch for YARN-2890) that Jeff 
Zhang raised.

The RC is available at: http://people.apache.org/~vinodkv/hadoop-2.6.1-RC1/

The RC tag in git is: release-2.6.1-RC1

The maven artifacts are available via 
repository.apache.org at 
https://repository.apache.org/content/repositories/orgapachehadoop-1021

Some notes from our release process
 -  - Sangjin and I moved out a bunch of items pending from 2.6.1 [2] - 
non-committed but desired patches. 2.6.1 is already big as is and is late by 
any standard, we can definitely include them in the next release.
 - The 2.6.1 wiki page [3] captures some (but not all) of the context of the 
patches that we pushed in.
 - Given the number of fixes pushed [4] in, we had to make a bunch of changes 
to our original plan - we added a few improvements that helped us backport 
patches easier (or in many cases made backports possible), and we dropped a few 
that didn't make sense (HDFS-7831, HDFS-7926, HDFS-7676, HDFS-7611, HDFS-7843, 
HDFS-8850).
 - I ran all the unit tests which (surprisingly?) passed. (Except for one, 
which pointed out a missing fix HDFS-7552).

As discussed before [5]
 - This release is the first point release after 2.6.0
 - I’d like to use this as a starting release for 2.6.2 in a few weeks and then 
follow up with more of these.

Please try the release and vote; the vote will run for the usual 5 days.

Thanks,
Vinod

[0] Hadoop 2.6.1 RC0 vote: http://markmail.org/thread/ubut2rn3lodc55iy
[1] Hadoop 2.6.1 Release process thread: 
http://markmail.org/thread/wkbgkxkhntx5tlux
[2] 2.6.1 Pending tickets: 
https://issues.apache.org/jira/issues/?filter=12331711
[3] 2.6.1 Wiki page: https://wiki.apache.org/hadoop/Release-2.6.1-Working-Notes
[4] List of 2.6.1 patches pushed: 
https://issues.apache.org/jira/issues/?jql=fixVersion%20%3D%202.6.1%20and%20labels%20%3D%20%222.6.1-candidate%22
[5] Planning Hadoop 2.6.1 release: http://markmail.org/thread/sbykjn5xgnksh6wg

PS:
 - Note that branch-2.6 which will be the base for 2.6.2 doesn't have these 
fixes yet. Once 2.6.1 goes through, I plan to rebase branch-2.6 based off 2.6.1.
 - The additional patches in RC1 that got into 2.6.1 all the way from 2.8 are 
NOT in 2.7.2 yet, this will be done as a followup.

[RESULT][VOTE] Release Apache Hadoop 2.6.1 RC1

[Vinod Vavilapalli]

Here’s my +1 to conclude the vote.

With 5 binding +1s (Xuan, Jian, Junping, Karthik, myself), 8 non-binding +1s 
(Wangda, Sanjin, Rohith, Brahma, Mit, Akira, Tsuyoshi, Masatake ), 4 
non-committal yes’s (Ted, Eric, Chang, Kuhu), this vote passes.

I’ll push out the bits.

Thanks to everyone who voted on this release!

Thanks
+Vinod


On Sep 16, 2015, at 7:10 PM, Vinod Kumar Vavilapalli 
mailto:vino...@apache.org>> wrote:

Hi all,

After a nearly month long [1] toil, with loads of help from Sangjin Lee and 
Akira Ajisaka, and 153 (RC0)+7(RC1) commits later, I've created a release 
candidate RC1 for hadoop-2.6.1.

RC1 is RC0 [0] (for which I opened and closed a vote last week) + UI fixes for 
the issue Sangjin raised (YARN-3171 and the dependencies YARN-3779, YARN-3248), 
additional fix to avoid incompatibility (YARN-3740), other UI bugs (YARN-1884, 
YARN-3544) and the MiniYARNCluster issue (right patch for YARN-2890) that Jeff 
Zhang raised.

The RC is available at: http://people.apache.org/~vinodkv/hadoop-2.6.1-RC1/

The RC tag in git is: release-2.6.1-RC1

The maven artifacts are available via 
repository.apache.org at 
https://repository.apache.org/content/repositories/orgapachehadoop-1021

Some notes from our release process
 -  - Sangjin and I moved out a bunch of items pending from 2.6.1 [2] - 
non-committed but desired patches. 2.6.1 is already big as is and is late by 
any standard, we can definitely include them in the next release.
 - The 2.6.1 wiki page [3] captures some (but not all) of the context of the 
patches that we pushed in.
 - Given the number of fixes pushed [4] in, we had to make a bunch of changes 
to our original plan - we added a few improvements that helped us backport 
patches easier (or in many cases made backports possible), and we dropped a few 
that didn't make sense (HDFS-7831, HDFS-7926, HDFS-7676, HDFS-7611, HDFS-7843, 
HDFS-8850).
 - I ran all the unit tests which (surprisingly?) passed. (Except for one, 
which pointed out a missing fix HDFS-7552).

As discussed before [5]
 - This release is the first point release after 2.6.0
 - I’d like to use this as a starting release for 2.6.2 in a few weeks and then 
follow up with more of these.

Please try the release and vote; the vote will run for the usual 5 days.

Thanks,
Vinod

[0] Hadoop 2.6.1 RC0 vote: http://markmail.org/thread/ubut2rn3lodc55iy
[1] Hadoop 2.6.1 Release process thread: 
http://markmail.org/thread/wkbgkxkhntx5tlux
[2] 2.6.1 Pending tickets: 
https://issues.apache.org/jira/issues/?filter=12331711
[3] 2.6.1 Wiki page: https://wiki.apache.org/hadoop/Release-2.6.1-Working-Notes
[4] List of 2.6.1 patches pushed: 
https://issues.apache.org/jira/issues/?jql=fixVersion%20%3D%202.6.1%20and%20labels%20%3D%20%222.6.1-candidate%22
[5] Planning Hadoop 2.6.1 release: http://markmail.org/thread/sbykjn5xgnksh6wg

PS:
 - Note that branch-2.6 which will be the base for 2.6.2 doesn't have these 
fixes yet. Once 2.6.1 goes through, I plan to rebase branch-2.6 based off 2.6.1.
 - The additional patches in RC1 that got into 2.6.1 all the way from 2.8 are 
NOT in 2.7.2 yet, this will be done as a followup.

Re: [RESULT][VOTE] Release Apache Hadoop 2.6.1 RC1

[Vinod Vavilapalli]

Done with all the release activities. Waiting for the bits to propagate to all 
the mirrors.

Thanks
+Vinod

On Sep 23, 2015, at 1:18 PM, Vinod Kumar Vavilapalli 
mailto:vino...@hortonworks.com>> wrote:

Here’s my +1 to conclude the vote.

With 5 binding +1s (Xuan, Jian, Junping, Karthik, myself), 8 non-binding +1s 
(Wangda, Sanjin, Rohith, Brahma, Mit, Akira, Tsuyoshi, Masatake ), 4 
non-committal yes’s (Ted, Eric, Chang, Kuhu), this vote passes.

I’ll push out the bits.

Thanks to everyone who voted on this release!

Thanks
+Vinod


On Sep 16, 2015, at 7:10 PM, Vinod Kumar Vavilapalli 
mailto:vino...@apache.org>> wrote:

Hi all,

After a nearly month long [1] toil, with loads of help from Sangjin Lee and 
Akira Ajisaka, and 153 (RC0)+7(RC1) commits later, I've created a release 
candidate RC1 for hadoop-2.6.1.

RC1 is RC0 [0] (for which I opened and closed a vote last week) + UI fixes for 
the issue Sangjin raised (YARN-3171 and the dependencies YARN-3779, YARN-3248), 
additional fix to avoid incompatibility (YARN-3740), other UI bugs (YARN-1884, 
YARN-3544) and the MiniYARNCluster issue (right patch for YARN-2890) that Jeff 
Zhang raised.

The RC is available at: http://people.apache.org/~vinodkv/hadoop-2.6.1-RC1/

The RC tag in git is: release-2.6.1-RC1

The maven artifacts are available via 
repository.apache.org at 
https://repository.apache.org/content/repositories/orgapachehadoop-1021

Some notes from our release process
 -  - Sangjin and I moved out a bunch of items pending from 2.6.1 [2] - 
non-committed but desired patches. 2.6.1 is already big as is and is late by 
any standard, we can definitely include them in the next release.
 - The 2.6.1 wiki page [3] captures some (but not all) of the context of the 
patches that we pushed in.
 - Given the number of fixes pushed [4] in, we had to make a bunch of changes 
to our original plan - we added a few improvements that helped us backport 
patches easier (or in many cases made backports possible), and we dropped a few 
that didn't make sense (HDFS-7831, HDFS-7926, HDFS-7676, HDFS-7611, HDFS-7843, 
HDFS-8850).
 - I ran all the unit tests which (surprisingly?) passed. (Except for one, 
which pointed out a missing fix HDFS-7552).

As discussed before [5]
 - This release is the first point release after 2.6.0
 - I’d like to use this as a starting release for 2.6.2 in a few weeks and then 
follow up with more of these.

Please try the release and vote; the vote will run for the usual 5 days.

Thanks,
Vinod

[0] Hadoop 2.6.1 RC0 vote: http://markmail.org/thread/ubut2rn3lodc55iy
[1] Hadoop 2.6.1 Release process thread: 
http://markmail.org/thread/wkbgkxkhntx5tlux
[2] 2.6.1 Pending tickets: 
https://issues.apache.org/jira/issues/?filter=12331711
[3] 2.6.1 Wiki page: https://wiki.apache.org/hadoop/Release-2.6.1-Working-Notes
[4] List of 2.6.1 patches pushed: 
https://issues.apache.org/jira/issues/?jql=fixVersion%20%3D%202.6.1%20and%20labels%20%3D%20%222.6.1-candidate%22
[5] Planning Hadoop 2.6.1 release: http://markmail.org/thread/sbykjn5xgnksh6wg

PS:
 - Note that branch-2.6 which will be the base for 2.6.2 doesn't have these 
fixes yet. Once 2.6.1 goes through, I plan to rebase branch-2.6 based off 2.6.1.
 - The additional patches in RC1 that got into 2.6.1 all the way from 2.8 are 
NOT in 2.7.2 yet, this will be done as a followup.

Re: Planning for Apache Hadoop 2.6.2

[Vinod Vavilapalli]

+1. Please take it over, I’ll standby for any help needed.

Thanks
+Vinod


On Sep 24, 2015, at 11:34 AM, Sangjin Lee 
mailto:sj...@apache.org>> wrote:

I'd like to volunteer as the release manager for 2.6.2 unless there is an
objection.

Re: [RESULT][VOTE] Release Apache Hadoop 2.6.1 RC1

[Vinod Vavilapalli]

The website and mirrors updates came back. Sending out an announcement now.

Thanks
+Vinod

On Sep 23, 2015, at 4:59 PM, Vinod Kumar Vavilapalli 
mailto:vino...@hortonworks.com>> wrote:

Done with all the release activities. Waiting for the bits to propagate to all 
the mirrors.

Thanks
+Vinod

On Sep 23, 2015, at 1:18 PM, Vinod Kumar Vavilapalli 
mailto:vino...@hortonworks.com>> wrote:

Here’s my +1 to conclude the vote.

With 5 binding +1s (Xuan, Jian, Junping, Karthik, myself), 8 non-binding +1s 
(Wangda, Sanjin, Rohith, Brahma, Mit, Akira, Tsuyoshi, Masatake ), 4 
non-committal yes’s (Ted, Eric, Chang, Kuhu), this vote passes.

I’ll push out the bits.

Thanks to everyone who voted on this release!

Thanks
+Vinod


On Sep 16, 2015, at 7:10 PM, Vinod Kumar Vavilapalli 
mailto:vino...@apache.org>> wrote:

Hi all,

After a nearly month long [1] toil, with loads of help from Sangjin Lee and 
Akira Ajisaka, and 153 (RC0)+7(RC1) commits later, I've created a release 
candidate RC1 for hadoop-2.6.1.

RC1 is RC0 [0] (for which I opened and closed a vote last week) + UI fixes for 
the issue Sangjin raised (YARN-3171 and the dependencies YARN-3779, YARN-3248), 
additional fix to avoid incompatibility (YARN-3740), other UI bugs (YARN-1884, 
YARN-3544) and the MiniYARNCluster issue (right patch for YARN-2890) that Jeff 
Zhang raised.

The RC is available at: http://people.apache.org/~vinodkv/hadoop-2.6.1-RC1/

The RC tag in git is: release-2.6.1-RC1

The maven artifacts are available via 
repository.apache.org at 
https://repository.apache.org/content/repositories/orgapachehadoop-1021

Some notes from our release process
 -  - Sangjin and I moved out a bunch of items pending from 2.6.1 [2] - 
non-committed but desired patches. 2.6.1 is already big as is and is late by 
any standard, we can definitely include them in the next release.
 - The 2.6.1 wiki page [3] captures some (but not all) of the context of the 
patches that we pushed in.
 - Given the number of fixes pushed [4] in, we had to make a bunch of changes 
to our original plan - we added a few improvements that helped us backport 
patches easier (or in many cases made backports possible), and we dropped a few 
that didn't make sense (HDFS-7831, HDFS-7926, HDFS-7676, HDFS-7611, HDFS-7843, 
HDFS-8850).
 - I ran all the unit tests which (surprisingly?) passed. (Except for one, 
which pointed out a missing fix HDFS-7552).

As discussed before [5]
 - This release is the first point release after 2.6.0
 - I’d like to use this as a starting release for 2.6.2 in a few weeks and then 
follow up with more of these.

Please try the release and vote; the vote will run for the usual 5 days.

Thanks,
Vinod

[0] Hadoop 2.6.1 RC0 vote: http://markmail.org/thread/ubut2rn3lodc55iy
[1] Hadoop 2.6.1 Release process thread: 
http://markmail.org/thread/wkbgkxkhntx5tlux
[2] 2.6.1 Pending tickets: 
https://issues.apache.org/jira/issues/?filter=12331711
[3] 2.6.1 Wiki page: https://wiki.apache.org/hadoop/Release-2.6.1-Working-Notes
[4] List of 2.6.1 patches pushed: 
https://issues.apache.org/jira/issues/?jql=fixVersion%20%3D%202.6.1%20and%20labels%20%3D%20%222.6.1-candidate%22
[5] Planning Hadoop 2.6.1 release: http://markmail.org/thread/sbykjn5xgnksh6wg

PS:
 - Note that branch-2.6 which will be the base for 2.6.2 doesn't have these 
fixes yet. Once 2.6.1 goes through, I plan to rebase branch-2.6 based off 2.6.1.
 - The additional patches in RC1 that got into 2.6.1 all the way from 2.8 are 
NOT in 2.7.2 yet, this will be done as a followup.

Re: [DISCUSS] Looking to a 2.8.0 release

[Vinod Vavilapalli]

As you may have noted, 2.8.0 got completely derailed what with 2.7.x and the 
unusually long 2.6.1 release.

With 2.6.1 out of the way, and two parallel threads in progress for 2.6.2 and 
2.7.2, it’s time for us to look back at where we are with Hadoop 2.8.

I’ll do a quick survey of where the individual features are and the amount of 
content already present in 2.8 and kick-start 2.8.0 process again.

+Vinod


> On Apr 21, 2015, at 2:39 PM, vino...@apache.org wrote:
> 
> With 2.7.0 out of the way, and with more maintenance releases to stabilize 
> it, I propose we start thinking about 2.8.0.
> 
> Here's my first cut of the proposal, will update the Roadmap wiki.
>  - Support *both* JDK7 and JDK8 runtimes: HADOOP-11090
>  - Compatibility tools to catch backwards, forwards compatibility issues at 
> patch submission, release times. Some of it is captured at YARN-3292. This 
> also involves resurrecting jdiff (HADOOP-11776/YARN-3426/MAPREDUCE-6310) 
> and/or investing in new tools.
>  - HADOOP-11656 Classpath isolation for downstream clients
>  - Support for Erasure Codes in HDFS HDFS-7285
>  - Early work for disk and network isolation in YARN: YARN-2139, YARN-2140
>  - YARN Timeline Service Next generation: YARN-2928. At least branch-merge + 
> early peek.
>  - Supporting non-exclusive node-labels: YARN-3214
> 
> I'm experimenting with more agile 2.7.x releases and would like to continue 
> the same by volunteering as the RM for 2.8.x too.
> 
> Given the long time we took with 2.7.0, the timeline I am looking at is 8-12 
> weeks. We can pick as many features as they finish along and make a more 
> predictable releases instead of holding up releases for ever.
> 
> Thoughts?
> 
> Thanks
> +Vinod

Re: [DISCUSS] About the details of JDK-8 support

[Vinod Vavilapalli]

If you see the community discussion thread on 2.8, my proposal was to support 
*both* JDK 7 and JDK 8 first. The last time we had discussion about dropping 
JDKs it wasn’t fun, so let’s not go there for now.

In terms of runtime support for JDK 8, yes, there is vast evidence that things 
already work as they are in Hadoop and all the way up to dependent projects. 
This was already the case as early as Hadoop 2.6/2.7 if I remember correctly, 
so nothing new here.

We should target source-level support of JDK 8 too, around which you outlined a 
bunch of issues around dependencies. I also found a bunch of issues around 
generating documentation, site etc. I propose that we track them under the 
umbrella JIRA and make progress there first.

Thanks
+Vinod


> On Oct 13, 2015, at 1:17 AM, Tsuyoshi Ozawa  wrote:
> 
> Thank you for sharing lots information and joining discussion.
> 
> About the runtime support of JDK-8,  let's describe it on wiki. It
> would be great for users to clarify which JDK version they should use.
> It's also useful to describe to use "1.8.0_40 or above" explicitly.
> Andrew, Elliott, could you update wiki or can I update wiki to add the
> description?
> 
> https://wiki.apache.org/hadoop/HadoopJavaVersions
> 
> About the source code level, I summarize opinions by users and
> developers as follows:
> 
> 1. Moving trunk to the java-8 compatible dependencies.
> 2. Creating branch-2-java-8 after 1.
> 3. Dropping Java 7 support (maybe when we support JDK 9?) after 2.
> Currently, we don't have any consensus.
> 
>> I'd  be +1 for moving trunk to the java-8 compatible dependencies, and to 
>> have the jenkins nighly builds only before java 8; we'd still have the patch 
>> and branch-2 runs on Java 7. That way, people will only have one nightly 
>> mail from Jenkins saying the build is broken, and maybe pay more attention 
>> to the fact.
> 
> This way looks good to me.
> Steve, do you know how to do this? In fact, I don't know so much about
> how to change and update nightly builds. Should we contact to Yetus
> community or Apache Infra team?
> 
> Best,
> - Tsuyoshi
> 
> 
> 
> On Sat, Oct 10, 2015 at 7:17 AM, Sangjin Lee  wrote:
>> Yes, at least for us, dropping the java 7 support (e.g. moving to java 8
>> source-wise) **at this point** would be an issue. I concur with the
>> sentiment that we should preserve the java 7 support on branch-2 (not not
>> move to java 8 source level) but can consider it for trunk. My 2 cents.
>> 
>> Thanks,
>> Sangjin
>> 
>> On Fri, Oct 9, 2015 at 10:43 AM, Steve Loughran 
>> wrote:
>> 
>>> 
 On 7 Oct 2015, at 22:39, Elliott Clark  wrote:
 
 On Mon, Oct 5, 2015 at 5:35 PM, Tsuyoshi Ozawa  wrote:
 
> Do you have any concern about this? I’ve not
> tested with HBase yet.
> 
 
 We've been running JDK 8u60 in production with Hadoop 2.6.X and HBase for
 quite a while. Everything has been very stable for us. We're running and
 compiling with jdk8.
 
 We had to turn off yarn.nodemanager.vmem-check-enabled. Otherwise mr jobs
 didn't do too well.
 
>>> 
>>> maybe related to the initial memory requirements being higher?
>>> 
>>> otherwise: did you file a JIRA?
>>> 
>>> 
 I'd be +1 on dropping jdk7 support. However as downstream developer it
 would be very weird for that to happen on anything but a major release.
>>> 
>>> 
>>> Past discussion (including a big contrib from twitter) was that breaking
>>> Java 7 support breaks all client apps too, which is not something people
>>> were ready for.
>>> 
>>> I'd  be +1 for moving trunk to the java-8 compatible dependencies, and to
>>> have the jenkins nighly builds only before java 8; we'd still have the
>>> patch and branch-2 runs on Java 7. That way, people will only have one
>>> nightly mail from Jenkins saying the build is broken, and maybe pay more
>>> attention to the fact.
> 

Re: 2.7.2 release plan

[Vinod Vavilapalli]

+1.

Thanks
+Vinod

> On Jul 16, 2015, at 8:18 AM, Chris Nauroth  wrote:
> 
> I'd be comfortable with inclusion of any doc-only patch in minor releases.
> There is a lot of value to end users in pushing documentation fixes as
> quickly as possible, and they don't bear the same risk of regressions or
> incompatibilities as code changes.
> 
> --Chris Nauroth
> 
> 
> 
> 
> On 7/16/15, 12:38 AM, "Tsuyoshi Ozawa"  wrote:
> 
>> Hi,
>> 
>> thank you for starting the discussion about 2.7.2 release.
>> 
>>> The focus obviously is to have blocker issues [2], bug-fixes and *no*
>> features / improvements.
>> 
>> I've committed YARN-3170, which is an improvement of documentation. I
>> thought documentation pages which can be fit into branch-2.7 can be
>> included easily. Should I revert it?
>> 
 I need help from all committers in automatically
>> merging in any patch that fits the above criterion into 2.7.2 instead of
>> only on trunk or 2.8.
>> 
>> Sure, I'll try my best.
>> 
>>> That way we can include not only blocker but also critical bug fixes to
>>> 2.7.2 release.
>> 
>> As Vinod mentioned, we should also apply major bug fixes into branch-2.7.
>> 
>> Thanks,
>> - Tsuyoshi
>> 
>> On Thu, Jul 16, 2015 at 3:52 PM, Akira AJISAKA
>>  wrote:
>>> Thanks Vinod for starting 2.7.2 release plan.
>>> 
 The focus obviously is to have blocker issues [2], bug-fixes and *no*
 features / improvements.
>>> 
>>> Can we adopt the plan as Karthik mentioned in "Additional maintenance
>>> releases for Hadoop 2.y versions" thread? That way we can include not
>>> only
>>> blocker but also critical bug fixes to 2.7.2 release.
>>> 
>>> In addition, branch-2.7 is a special case. (2.7.1 is the first stable
>>> release) Therefore I'm thinking we can include major bug fixes as well.
>>> 
>>> Regards,
>>> Akira
>>> 
>>> 
>>> On 7/16/15 04:13, Vinod Kumar Vavilapalli wrote:
 
 Hi all,
 
 
 Thanks everyone for the push on 2.7.1! Branch-2.7 is now open for
 commits
 to a 2.7.2 release. JIRA also now has a 2.7.2 version for all the
 sub-projects.
 
 
 Continuing the previous 2.7.1 thread on steady maintenance releases
 [1],
 we
 should follow up 2.7.1 with a 2.7.2 within 4 weeks. Earlier I tried a
 2-3
 week cycle for 2.7.1, but it seems to be impractical given the
 community
 size. So, I propose we target a release by the end for 4 weeks from
 now,
 starting the release close-down within 2-3 weeks.
 
 The focus obviously is to have blocker issues [2], bug-fixes and *no*
 features / improvements. I need help from all committers in
 automatically
 merging in any patch that fits the above criterion into 2.7.2 instead
 of
 only on trunk or 2.8.
 
 Thoughts?
 
 Thanks,
 
 +Vinod
 
 [1] A 2.7.1 release to follow up 2.7.0
 http://markmail.org/message/zwzze6cqqgwq4rmw
 
 [2] 2.7.2 release blockers:
 https://issues.apache.org/jira/issues/?filter=12332867
 
>>> 
>> 
> 
> 

Re: Planning Apache Hadoop 2.7.2

[Vinod Vavilapalli]

Got swamped again.

We already have about 112 patches in 2.7.2 already.

There are 15 open tickets in progress. I’ll push progress on them for an RC 
towards end of this week.

Thanks
+Vinod

On Sep 25, 2015, at 3:27 PM, Vinod Kumar Vavilapalli 
mailto:vino...@apache.org>> wrote:

Hi all,

We released 2.7.1 nearly 2.5 months ago. I got caught up with a very long 
release process for 2.6.1 so couldn't make progress on a 2.7.2. Now is the time!

Things to do

 (#1) Branch
-- Branch 2.7 has been open to 2.7.2 commits for a while.
-- In order to converge on a release, I will branch out 2.7.2 soon.

  (#2) Patches
-- 2.7.2 already has a boat load [1] of fixes.
-- The list of open blocker / critical tickets [2] is not small. I'll start 
triaging and see what can make it in a week or so of time.

  (#3) Release
-- Even if we can get half of the blocker / critical tickets in, the full 
list [3] will be big enough for us to start voting on an RC in no less than a 
week.
-- Leaving aside some buffer time, I plan to start RC process by end of 
first week of October.

Thoughts?

Appreciate help in moving open tickets [2] forward.

A general note:  Please consider putting any critical / blocker tickets on 2.8 
into 2.6.2 and 2.7.2 releases.

Thanks
+Vinod

[1] 2.7.2 Fixed Tickets: https://issues.apache.org/jira/issues/?filter=12333473
[2] 2.7.2 Open Blockers / Critical Tickets: 
https://issues.apache.org/jira/issues/?filter=12332867
[3] 2.7.2 Release Tickets: 
https://issues.apache.org/jira/issues/?filter=12333461

Re: [VOTE] Release Apache Hadoop 2.6.2

[Vinod Vavilapalli]

I was helping Sangjin offline with the release.

We briefly discussed the KEYS problem before, but it missed my attention.

I will get his KEYS committed right-away, the release is testable right away 
though.

Regarding the voting period, let’s continue voting for two more days, the 
period also had the weekend during which a lot of people (atleast myself and 
team) didn’t pay attention to this vote.

Thanks
+Vinod


> On Oct 26, 2015, at 1:50 PM, Andrew Wang  wrote:
> 
> Hey Sangjin, did you add your release signing keys to the KEYS file? I
> don't see it here:
> 
> https://dist.apache.org/repos/dist/release/hadoop/common/KEYS
> 
> Also only PMC votes are binding on releases, so I think we currently still
> stand at 0 binding +1s.
> 
> On Mon, Oct 26, 2015 at 1:28 PM, Sangjin Lee  wrote:
> 
>> That makes sense. Thanks for pointing that out. The git commit id is
>> 0cfd050febe4a30b1ee1551dcc527589509fb681.
>> 
>> On Mon, Oct 26, 2015 at 12:25 PM, Steve Loughran 
>> wrote:
>> 
>>> 
 On 22 Oct 2015, at 22:14, Sangjin Lee  wrote:
 
 Hi all,
 
 I have created a release candidate (RC0) for Hadoop 2.6.2.
 
 The RC is available at:
>>> http://people.apache.org/~sjlee/hadoop-2.6.2-RC0/
 
 The RC tag in git is: release-2.6.2-RC0
 
>>> 
>>> Tags can move; we should *never* vote for a release of one.
>>> 
>>> What is the git commit #  ?
>>> 
 The list of JIRAs committed for 2.6.2:
 
>>> 
>> https://issues.apache.org/jira/browse/YARN-4101?jql=project%20in%20(HADOOP%2C%20HDFS%2C%20YARN%2C%20MAPREDUCE)%20AND%20fixVersion%20%3D%202.6.2
 
 The maven artifacts are staged at
 
>> https://repository.apache.org/content/repositories/orgapachehadoop-1022/
 
 Please try out the release candidate and vote. The vote will run for 5
>>> days.
 
 Thanks,
 Sangjin
>>> 
>>> 
>> 

Re: [VOTE] Release Apache Hadoop 2.6.2

[Vinod Vavilapalli]

This is done: https://dist.apache.org/repos/dist/release/hadoop/common/KEYS is 
updated with Sangjin’s keys now.

Thanks
+Vinod

On Oct 26, 2015, at 2:06 PM, Vinod Kumar Vavilapalli 
mailto:vino...@hortonworks.com>> wrote:

I will get his KEYS committed right-away

Re: 2.7.2 release plan

[Vinod Vavilapalli]

I see you already put in both HDFS-8950 and HDFS-7725, so we are good there.

Can you please do a pass on HDFS-8871 and may be help nudge Daryn on HDFS-8893, 
HDFS-8674 and HDFS-8498?

Thanks
+Vinod
 
> On Oct 27, 2015, at 8:12 AM, Kihwal Lee  wrote:
> 
> I think we need HDFS-8950 and HDFS-7725 in 2.7.2.It should be easy to 
> backport/cherry-pick HDFS-7725. For HDFS-8950, it will be nice if Ming can 
> chime in.
> Kihwal
> 
>  From: Tsuyoshi Ozawa 
> To: "common-...@hadoop.apache.org"  
> Cc: Chris Nauroth ; "yarn-...@hadoop.apache.org" 
> ; "hdfs-...@hadoop.apache.org" 
> ; "mapreduce-dev@hadoop.apache.org" 
> ; Vinod Kumar Vavilapalli 
>  
> Sent: Tuesday, October 27, 2015 2:39 AM
> Subject: Re: 2.7.2 release plan
> 
> Vinod and Chris,
> 
> Thanks for your reply. I'll do also backport not only bug fixes but
> also documentations(I think 2.7.2 includes them). It helps users a lot.
> 
> Best,
> - Tsuyoshi
> 
> On Tuesday, 27 October 2015, Vinod Vavilapalli 
> wrote:
> 
>> +1.
>> 
>> Thanks
>> +Vinod
>> 
>>> On Jul 16, 2015, at 8:18 AM, Chris Nauroth > > wrote:
>>> 
>>> I'd be comfortable with inclusion of any doc-only patch in minor
>> releases.
>>> There is a lot of value to end users in pushing documentation fixes as
>>> quickly as possible, and they don't bear the same risk of regressions or
>>> incompatibilities as code changes.
>>> 
>>> --Chris Nauroth
>>> 
>>> 
>>> 
>>> 
>>> On 7/16/15, 12:38 AM, "Tsuyoshi Ozawa" >
>> wrote:
>>> 
>>>> Hi,
>>>> 
>>>> thank you for starting the discussion about 2.7.2 release.
>>>> 
>>>>> The focus obviously is to have blocker issues [2], bug-fixes and *no*
>>>> features / improvements.
>>>> 
>>>> I've committed YARN-3170, which is an improvement of documentation. I
>>>> thought documentation pages which can be fit into branch-2.7 can be
>>>> included easily. Should I revert it?
>>>> 
>>>>>> I need help from all committers in automatically
>>>> merging in any patch that fits the above criterion into 2.7.2 instead of
>>>> only on trunk or 2.8.
>>>> 
>>>> Sure, I'll try my best.
>>>> 
>>>>> That way we can include not only blocker but also critical bug fixes to
>>>>> 2.7.2 release.
>>>> 
>>>> As Vinod mentioned, we should also apply major bug fixes into
>> branch-2.7.
>>>> 
>>>> Thanks,
>>>> - Tsuyoshi
>>>> 
>>>> On Thu, Jul 16, 2015 at 3:52 PM, Akira AJISAKA
>>>> > wrote:
> 
> 
>>>>> Thanks Vinod for starting 2.7.2 release plan.
>>>>> 
>>>>>> The focus obviously is to have blocker issues [2], bug-fixes and *no*
>>>>>> features / improvements.
>>>>> 
>>>>> Can we adopt the plan as Karthik mentioned in "Additional maintenance
>>>>> releases for Hadoop 2.y versions" thread? That way we can include not
>>>>> only
>>>>> blocker but also critical bug fixes to 2.7.2 release.
>>>>> 
>>>>> In addition, branch-2.7 is a special case. (2.7.1 is the first stable
>>>>> release) Therefore I'm thinking we can include major bug fixes as well.
>>>>> 
>>>>> Regards,
>>>>> Akira
>>>>> 
>>>>> 
>>>>> On 7/16/15 04:13, Vinod Kumar Vavilapalli wrote:
>>>>>> 
>>>>>> Hi all,
>>>>>> 
>>>>>> 
>>>>>> Thanks everyone for the push on 2.7.1! Branch-2.7 is now open for
>>>>>> commits
>>>>>> to a 2.7.2 release. JIRA also now has a 2.7.2 version for all the
>>>>>> sub-projects.
>>>>>> 
>>>>>> 
>>>>>> Continuing the previous 2.7.1 thread on steady maintenance releases
>>>>>> [1],
>>>>>> we
>>>>>> should follow up 2.7.1 with a 2.7.2 within 4 weeks. Earlier I tried a
>>>>>> 2-3
>>>>>> week cycle for 2.7.1, but it seems to be impractical given the
>>>>>> community
>>>>>> size. So, I propose we target a release by the end for 4 weeks from
>>>>>> now,
>>>>>> starting the release close-down within 2-3 weeks.
>>>>>> 
>>>>>> The focus obviously is to have blocker issues [2], bug-fixes and *no*
>>>>>> features / improvements. I need help from all committers in
>>>>>> automatically
>>>>>> merging in any patch that fits the above criterion into 2.7.2 instead
>>>>>> of
>>>>>> only on trunk or 2.8.
>>>>>> 
>>>>>> Thoughts?
>>>>>> 
>>>>>> Thanks,
>>>>>> 
>>>>>> +Vinod
>>>>>> 
>>>>>> [1] A 2.7.1 release to follow up 2.7.0
>>>>>> http://markmail.org/message/zwzze6cqqgwq4rmw
>>>>>> 
>>>>>> [2] 2.7.2 release blockers:
>>>>>> https://issues.apache.org/jira/issues/?filter=12332867
>>>>>> 
>>>>> 
>>>> 
>>> 
>>> 
>> 
>> 
> 
> 

Re: [VOTE] Release Apache Hadoop 2.6.2

[Vinod Vavilapalli]

Hi,

I checked 2.6.2-RC0, based on my check-list:

 - Signatures and message digests all are good in general.
 - The top level full LICENSE, NOTICE and README for the source artifacts are 
good - CHANGES.txt for common, hdfs and mapped are correctly located.
 - Able to build the tars out of the source tar ball.

Testing: All testing on single node, unsecured, default mode.

- Started HDFS daemons successfully , created directories.
- Successfully started YARN daemons - ResourceManager, NodeManager and Timeline 
Service
- Successfully started MapReduce history server.
- Ran DistributedShell as a native YARN app.
- Ran wordcount, pi, random writer, sort, grep and they all pass just fine.
- Navigated through the RM, NM and Timeline UIs to make sure the views are 
working well.

With that, here's my +1 for the RC.

Thanks,
+Vinod

> On Oct 22, 2015, at 2:14 PM, Sangjin Lee  wrote:
> 
> Hi all,
> 
> I have created a release candidate (RC0) for Hadoop 2.6.2.
> 
> The RC is available at: http://people.apache.org/~sjlee/hadoop-2.6.2-RC0/
> 
> The RC tag in git is: release-2.6.2-RC0
> 
> The list of JIRAs committed for 2.6.2:
> https://issues.apache.org/jira/browse/YARN-4101?jql=project%20in%20(HADOOP%2C%20HDFS%2C%20YARN%2C%20MAPREDUCE)%20AND%20fixVersion%20%3D%202.6.2
> 
> The maven artifacts are staged at
> https://repository.apache.org/content/repositories/orgapachehadoop-1022/
> 
> Please try out the release candidate and vote. The vote will run for 5 days.
> 
> Thanks,
> Sangjin

Re: continuing releases on Apache Hadoop 2.6.x

[Vinod Vavilapalli]

Just to stress on the following, it is very important that any critical 
bug-fixes that we push into 2.8.0 or even trunk, we should consider them for 
2.6.3 and 2.7.3 if it makes sense. This is the only way we can avoid extremely 
long release cycles like that of 2.6.1.

Also, to clarify a little, use Target-version if you want a discussion of the 
backport, but if you do end up backporting patches after that, you should set 
the fix-version to be 2.6.1.

Thanks
+Vinod


> On Nov 2, 2015, at 11:29 AM, Sangjin Lee  wrote:
> 
> As you may have seen, 2.6.2 is out
> . I have also retargeted all
> open issues that were targeted for 2.6.2 to 2.6.3.
> 
> Continuing the discussion in the email thread here
> , I'd like us to maintain the
> cadence of monthly point releases in the 2.6.x line. It would be great if
> we can have 2.6.3 released before the year-end holidays.
> 
> If you have any bugfixes and improvements that are targeted for 2.7.x (or
> 2.8) that you think are applicable to 2.6.x, please *set the target version
> to 2.6.3* and merge them to branch-2.6. Please use your judgment in terms
> of the applicability and quality of the changes so that we can ensure each
> point release is consistently better quality than the previous one. Thanks
> everyone!
> 
> Regards,
> Sangjin

Re: Planning Apache Hadoop 2.7.2

[Vinod Vavilapalli]

We are down to 4 now (https://issues.apache.org/jira/issues/?filter=12332867), 
appreciate help moving forward with them.

Thanks
+Vinod

On Oct 26, 2015, at 11:34 AM, Vinod Kumar Vavilapalli 
mailto:vino...@hortonworks.com>> wrote:

Got swamped again.

We already have about 112 patches in 2.7.2 already.

There are 15 open tickets in progress. I’ll push progress on them for an RC 
towards end of this week.

Thanks
+Vinod

On Sep 25, 2015, at 3:27 PM, Vinod Kumar Vavilapalli 
mailto:vino...@apache.org>> wrote:

Hi all,

We released 2.7.1 nearly 2.5 months ago. I got caught up with a very long 
release process for 2.6.1 so couldn't make progress on a 2.7.2. Now is the time!

Things to do

 (#1) Branch
-- Branch 2.7 has been open to 2.7.2 commits for a while.
-- In order to converge on a release, I will branch out 2.7.2 soon.

  (#2) Patches
-- 2.7.2 already has a boat load [1] of fixes.
-- The list of open blocker / critical tickets [2] is not small. I'll start 
triaging and see what can make it in a week or so of time.

  (#3) Release
-- Even if we can get half of the blocker / critical tickets in, the full 
list [3] will be big enough for us to start voting on an RC in no less than a 
week.
-- Leaving aside some buffer time, I plan to start RC process by end of 
first week of October.

Thoughts?

Appreciate help in moving open tickets [2] forward.

A general note:  Please consider putting any critical / blocker tickets on 2.8 
into 2.6.2 and 2.7.2 releases.

Thanks
+Vinod

[1] 2.7.2 Fixed Tickets: https://issues.apache.org/jira/issues/?filter=12333473
[2] 2.7.2 Open Blockers / Critical Tickets: 
https://issues.apache.org/jira/issues/?filter=12332867
[3] 2.7.2 Release Tickets: 
https://issues.apache.org/jira/issues/?filter=12333461

Re: Planning Apache Hadoop 2.7.2

[Vinod Vavilapalli]

Got this count a bit wrong:
 - There are 13 tickets marked against 2.7.2 but only committed to 2.8.0
 - There are 10 tickets resolved without a patch - duplicates / invalids etc.
 - There are 77 patches marked and committed to 2.7.2.
 - There are a whole bunch of unresolved tickets in that 112 patches count, 
only at non-blocker/non-critical priority - 55 open major / minor / trivial 
tickets

Scanning through these remaining 55 unresolved major / minor / trivial tickets 
to see if there is anything important there.

Thanks
+Vinod


We already have about 112 patches in 2.7.2 already.

Re: Planning Apache Hadoop 2.7.2

[Vinod Vavilapalli]

I’m closing down 2.7.2 now, almost there.

Created a new branch-2.7.2 for the release so that any 2.7.3 patches can go 
into branch-2.7 directly.

Please reach out to me if you have any questions.

Thanks
+Vinod

On Nov 2, 2015, at 2:15 PM, Vinod Kumar Vavilapalli 
mailto:vino...@hortonworks.com>> wrote:

We are down to 4 now (https://issues.apache.org/jira/issues/?filter=12332867), 
appreciate help moving forward with them.

Thanks
+Vinod

On Oct 26, 2015, at 11:34 AM, Vinod Kumar Vavilapalli 
mailto:vino...@hortonworks.com>> wrote:

Got swamped again.

We already have about 112 patches in 2.7.2 already.

There are 15 open tickets in progress. I’ll push progress on them for an RC 
towards end of this week.

Thanks
+Vinod

On Sep 25, 2015, at 3:27 PM, Vinod Kumar Vavilapalli 
mailto:vino...@apache.org>> wrote:

Hi all,

We released 2.7.1 nearly 2.5 months ago. I got caught up with a very long 
release process for 2.6.1 so couldn't make progress on a 2.7.2. Now is the time!

Things to do

 (#1) Branch
-- Branch 2.7 has been open to 2.7.2 commits for a while.
-- In order to converge on a release, I will branch out 2.7.2 soon.

  (#2) Patches
-- 2.7.2 already has a boat load [1] of fixes.
-- The list of open blocker / critical tickets [2] is not small. I'll start 
triaging and see what can make it in a week or so of time.

  (#3) Release
-- Even if we can get half of the blocker / critical tickets in, the full 
list [3] will be big enough for us to start voting on an RC in no less than a 
week.
-- Leaving aside some buffer time, I plan to start RC process by end of 
first week of October.

Thoughts?

Appreciate help in moving open tickets [2] forward.

A general note:  Please consider putting any critical / blocker tickets on 2.8 
into 2.6.2 and 2.7.2 releases.

Thanks
+Vinod

[1] 2.7.2 Fixed Tickets: https://issues.apache.org/jira/issues/?filter=12333473
[2] 2.7.2 Open Blockers / Critical Tickets: 
https://issues.apache.org/jira/issues/?filter=12332867
[3] 2.7.2 Release Tickets: 
https://issues.apache.org/jira/issues/?filter=12333461

Re: Planning Apache Hadoop 2.7.2

[Vinod Vavilapalli]

Just committed the last of the critical issues. Starting the release process 
now.

Thanks
+Vinod

On Nov 3, 2015, at 3:24 PM, Vinod Kumar Vavilapalli 
mailto:vino...@hortonworks.com>> wrote:

I’m closing down 2.7.2 now, almost there.

Created a new branch-2.7.2 for the release so that any 2.7.3 patches can go 
into branch-2.7 directly.

Please reach out to me if you have any questions.

Thanks
+Vinod

On Nov 2, 2015, at 2:15 PM, Vinod Kumar Vavilapalli 
mailto:vino...@hortonworks.com>> wrote:

We are down to 4 now (https://issues.apache.org/jira/issues/?filter=12332867), 
appreciate help moving forward with them.

Thanks
+Vinod

On Oct 26, 2015, at 11:34 AM, Vinod Kumar Vavilapalli 
mailto:vino...@hortonworks.com>> wrote:

Got swamped again.

We already have about 112 patches in 2.7.2 already.

There are 15 open tickets in progress. I’ll push progress on them for an RC 
towards end of this week.

Thanks
+Vinod

On Sep 25, 2015, at 3:27 PM, Vinod Kumar Vavilapalli 
mailto:vino...@apache.org>> wrote:

Hi all,

We released 2.7.1 nearly 2.5 months ago. I got caught up with a very long 
release process for 2.6.1 so couldn't make progress on a 2.7.2. Now is the time!

Things to do

 (#1) Branch
-- Branch 2.7 has been open to 2.7.2 commits for a while.
-- In order to converge on a release, I will branch out 2.7.2 soon.

  (#2) Patches
-- 2.7.2 already has a boat load [1] of fixes.
-- The list of open blocker / critical tickets [2] is not small. I'll start 
triaging and see what can make it in a week or so of time.

  (#3) Release
-- Even if we can get half of the blocker / critical tickets in, the full 
list [3] will be big enough for us to start voting on an RC in no less than a 
week.
-- Leaving aside some buffer time, I plan to start RC process by end of 
first week of October.

Thoughts?

Appreciate help in moving open tickets [2] forward.

A general note:  Please consider putting any critical / blocker tickets on 2.8 
into 2.6.2 and 2.7.2 releases.

Thanks
+Vinod

[1] 2.7.2 Fixed Tickets: https://issues.apache.org/jira/issues/?filter=12333473
[2] 2.7.2 Open Blockers / Critical Tickets: 
https://issues.apache.org/jira/issues/?filter=12332867
[3] 2.7.2 Release Tickets: 
https://issues.apache.org/jira/issues/?filter=12333461

Re: [DISCUSS] Looking to a 2.8.0 release

[Vinod Vavilapalli]

Agreed on not mixing this with major release discussions.

Okay, I just finished my review of 2.8 content.

A quick summary follows.

Current state of originally planned items

 - Nearly Done / Done and so need to close down quickly
— Support *both* JDK7 and JDK8 runtimes HADOOP-11090
— Supporting non-exclusive node-labels: YARN-3214: Done, can push as an 
alpha / beta feature
— Support priorities across applications within the same queue YARN-1963: 
Can push as an alpha / beta feature

 - Definitely have to move out into 2.9 and beyond
— Early work for disk and network isolation in YARN: YARN-2139, YARN-2140: 
Early noise, some critical pieces designed, done but not a lot of movement of 
late
— Classpath isolation for downstream clients HADOOP-11656: Lots of chatter 
a while ago, not much movement of late
— Support for Erasure Codes in HDFS 
HDFS-7285<https://issues.apache.org/jira/browse/HDFS-7285>: Moved out to 2.9 in 
the interest of stability / bake-in

Non-planned features that went into 2.8.0

— The overall list of new features: 
https://issues.apache.org/jira/issues/?filter=12333994
— HDFS-6200 Create a separate jar for hdfs-client: Compatible improvement - 
no dimension of alpha/betaness here.
— HDFS-8155Support OAuth2 in WebHDFS: Alpha / Early feature?
— Stability improvements ready to use:
— HDFS-8008Support client-side back off when the datanodes are 
congested
— HDFS-8009Signal congestion on the DataNode
— YARN-3611 Support Docker Containers In LinuxContainerExecutor: Well most 
of it anyways. Can push as an alpha feature.
— YARN-1197 Support changing resources of an allocated container: Can push 
as an alpha/beta feature

Items in progress to think about in 2 weeks

— YARN Timeline Service v1.5 - YARN-4233: A short term bridge before 
YARN-2928 comes around. I think this should go in given the tremendous activity 
recently.
— YARN Timeline Service Next generation: YARN-2928: Lots of momentum, but 
clearly a work in progress. Two options here
— If it is safe to ship it into 2.8 in a disable manner, we can get the 
early code into trunk and all the way int o2.8.
— If it is not safe, it organically rolls over into 2.9
— Compatibility tools to catch backwards, forwards compatibility issues at 
patch submission, release times. Some of it is captured at YARN-3292. This also 
involves resurrecting jdiff (HADOOP-11776/YARN-3426/MAPREDUCE-6310) and/or 
investing in new tools.

This is my plan of action for now in terms of the release itself

 - Cut a branch about two weeks from now
 - Do an RC mid next month (leaving ~4weeks since branch-cut)
 - As with 2.7.x series, the first release will still be called as early / 
alpha release in the interest of
— gaining downstream adoption
— wider testing,
— yet reserving our right to fix any inadvertent incompatibilities 
introduced.

If we can get answers on “Items to think about now” during this and next week, 
we will overall be in good shape.

Thoughts?

Thanks
+Vinod
PS:As you may have noted above, this time around, I want to do something that 
we’ve always wanted to do, but never explicitly did. I’m calling out readiness 
of each feature as they stand today so we can inform our users better of what 
they can start relying on in production clusters.


On Oct 5, 2015, at 11:53 AM, Colin P. McCabe 
mailto:cmcc...@apache.org>> wrote:

I think it makes sense to have a 2.8 release since there are a
tremendous number of JIRAs in 2.8 that are not in 2.7.  Doing a 3.x
release seems like something we should consider separately since it
would not have the same compatibility guarantees as a 2.8 release.
There's a pretty big delta between trunk and 2.8 as well.

cheers,
Colin

On Sat, Sep 26, 2015 at 1:36 PM, Chris Douglas 
mailto:cdoug...@apache.org>> wrote:
With two active sustaining branches (2.6, 2.7), what would you think
of releasing trunk as 3.x instead of pushing 2.8? There are many new
features (EC, Y1197, etc.), and trunk could be the source of several
alpha/beta releases before we fork the 3.x line. -C

On Sat, Sep 26, 2015 at 12:49 PM, Vinod Vavilapalli
mailto:vino...@hortonworks.com>> wrote:
As you may have noted, 2.8.0 got completely derailed what with 2.7.x and the 
unusually long 2.6.1 release.

With 2.6.1 out of the way, and two parallel threads in progress for 2.6.2 and 
2.7.2, it’s time for us to look back at where we are with Hadoop 2.8.

I’ll do a quick survey of where the individual features are and the amount of 
content already present in 2.8 and kick-start 2.8.0 process again.

+Vinod


On Apr 21, 2015, at 2:39 PM, vino...@apache.org<mailto:vino...@apache.org> 
wrote:

With 2.7.0 out of the way, and with more maintenance releases to stabilize it, 
I propose we start thinking about 2.8.0.

Here's my first cut of the proposal, will update the Roadmap wiki.
- Support *both* JDK7 and JDK8 runt

Re: [DISCUSS] Looking to a 2.8.0 release

[Vinod Vavilapalli]

I’ll let others comment on specific features.

Regarding the 3.x vs 2.x point, as I noted before on other threads, given all 
the incompatibilities in trunk it will be ways off before users can run their 
production workloads on a 3.x release. Therefore, as I was proposing before, we 
should continue the 2.x lines, but soon get started on rolling out a release 
candidate based off trunk.

Like with 2.8, I’d like to go back and prepare some notes on trunk’s content so 
we can objectively discuss about it.

+Vinod

Re: continuing releases on Apache Hadoop 2.6.x

[Vinod Vavilapalli]

I see 18 JIRAs across the sub-projects as of now in 2.6.3. Seems like we will 
have a reasonable number of fixes if we start an RC early december.

In the mean while, we should also review 2.7.3 and 2.8.0 blocker / critical 
list and see if it makes sense to backport any of those into 2.6.3.

+Vinod


On Nov 17, 2015, at 5:10 PM, Sangjin Lee 
mailto:sj...@apache.org>> wrote:

I'd like to pick up this email discussion again. It is time that we started
thinking about the next release in the 2.6.x line. IMO we want to walk the
balance between maintaining a reasonable release cadence and getting a good
amount of high-quality fixes. The timeframe is a little tricky as the
holidays are approaching. If we have enough fixes accumulated in
branch-2.6, some time early December might be a good target for cutting the
first release candidate. Once we miss that window, I think we are looking
at next January. I'd like to hear your thoughts on this.