Re: Branch merges and 3.0.0-beta1 scope

2017-08-29 Thread Vrushali C
bq. I don't think it's a lot to ask that feature leads shoot an email to the release manager of their target version. DISCUSS emails right before a proposed merge VOTE are way too late, it ends up being a fire drill where we need to scramble on many fronts. Yes, I have been thinking about this too

Re: Branch merges and 3.0.0-beta1 scope

2017-08-29 Thread Andrew Wang
Hi Vinod, On Fri, Aug 25, 2017 at 2:42 PM, Vinod Kumar Vavilapalli wrote: > > From a release management perspective, it's *extremely* reasonable to > block the inclusion of new features a month from the planned release date. > A typical software development lifecycle includes weeks of feature fr

Re: Branch merges and 3.0.0-beta1 scope

2017-08-25 Thread Andrew Wang
Jonathan, thanks for the heads up. I don't have much familiarity with YARN, but gave the PBs and pom changes a look, and left a few small comments on the umbrella JIRA. This seems like a smaller change than some of the other branch merges we're discussing, but I'm again reticent about adding scope

Re: Branch merges and 3.0.0-beta1 scope

2017-08-25 Thread Andrew Wang
Here's a summary of some 1-on-1 conversations I had with contributors of the different features I'm tracking. Storage Policy Satisfier (Uma Gangumalla) * Target version: 3.1.0, maybe 3.0.0 GA * We're resolving some design discussions on JIRA. Plan is to do some MVP work on the API to get this into

Re: Branch merges and 3.0.0-beta1 scope

2017-08-25 Thread Vinod Kumar Vavilapalli
> From a release management perspective, it's *extremely* reasonable to block > the inclusion of new features a month from the planned release date. A > typical software development lifecycle includes weeks of feature freeze and > weeks of code freeze. It is no knock on any developer or any feat

Re: Branch merges and 3.0.0-beta1 scope

2017-08-25 Thread Jonathan Hung
Hi Andrew, Thanks for starting the discussion - we have a feature YARN-5734 for API based scheduler configuration that I feel is pretty close to merge (also "a few weeks"). It's almost completely code and API additions and we were careful to design it so that it's compatible (feature is also turne

Re: Branch merges and 3.0.0-beta1 scope

2017-08-25 Thread Wangda Tan
Resource profile is similar to TSv2, the feature is: - Alpha feature, we will not freeze new added APIs. And all added APIs are explicitly marked to @Unstable. - Allow rolling upgrade from branch-2. - Touched existing code, but we have, and will continue tests to make sure changes are safe. Discus

Re: Branch merges and 3.0.0-beta1 scope

2017-08-25 Thread Rohith Sharma K S
On 25 August 2017 at 22:39, Andrew Wang wrote: > Hi Rohith, > > Given that we're advertising TSv2 as an alpha feature, I think we're > allowed to break compatibility. Let's make sure this is clear in the > release notes and documentation. > > That said, with TSv2 phase 2, is the API going to be

Re: Branch merges and 3.0.0-beta1 scope

2017-08-25 Thread Andrew Wang
Hi Jason, I agree with this proposal. I'll start another email thread spelling this out, and gather additional feedback. Best, Andrew On Fri, Aug 25, 2017 at 6:27 AM, Jason Lowe wrote: > Andrew Wang wrote: > > >> This means I'll cut branch-3 and >> branch-3.0, and move trunk to 4.0.0 before th

Re: Branch merges and 3.0.0-beta1 scope

2017-08-25 Thread Andrew Wang
Hi Rohith, Given that we're advertising TSv2 as an alpha feature, I think we're allowed to break compatibility. Let's make sure this is clear in the release notes and documentation. That said, with TSv2 phase 2, is the API going to be frozen? The umbrella JIRA refers to "TSv2 alpha2" which indica

Re: Branch merges and 3.0.0-beta1 scope

2017-08-25 Thread Jason Lowe
Andrew Wang wrote: > This means I'll cut branch-3 and > branch-3.0, and move trunk to 4.0.0 before these VOTEs end. This will open > up development for Hadoop 3.1.0 and 4.0.0. I can see a need for branch-3.0, but please do not create branch-3. Doing so will relegate trunk back to the "patch pu

Re: Branch merges and 3.0.0-beta1 scope

2017-08-24 Thread Rohith Sharma K S
Hi Andrew Thanks for update on release plan! I would like to discuss specifically regarding compatibility of releases. What is the compatibility to be maintained for GA if we don't merge to beta1 release? IIUC, till now all the releases were alpha where compatibility was not that important. All t

Re: Branch merges and 3.0.0-beta1 scope

2017-08-24 Thread Andrew Wang
Glad to see the discussion continued in my absence :) >From a release management perspective, it's *extremely* reasonable to block the inclusion of new features a month from the planned release date. A typical software development lifecycle includes weeks of feature freeze and weeks of code freeze

Re: Branch merges and 3.0.0-beta1 scope

2017-08-23 Thread Ravi Prakash
Also, when people +1 a merge, can they please describe if they did testing / use the feature in addition to what is already described in the thread? On Wed, Aug 23, 2017 at 11:18 AM, Vrushali Channapattan < vrushalic2...@gmail.com> wrote: > For timeline service v2, we have completed all subtasks

Re: Branch merges and 3.0.0-beta1 scope

2017-08-23 Thread Vrushali Channapattan
For timeline service v2, we have completed all subtasks under YARN-5355 [1]. We initiated a merge-to-trunk vote [2] yesterday. thanks Vrushali [1] https://issues.apache.org/jira/browse/YARN-5355 [2] http://mail-archives.apache.org/mod_mbox/hadoop-common-dev/201708.mbox/%3CCAE=b_fblt2j+ezb4wqdn_uw

Re: Branch merges and 3.0.0-beta1 scope

2017-08-23 Thread Vinod Kumar Vavilapalli
Agreed. I was very clearly not advocating for rushing in features. If you have followed my past emails, I have only strongly advocated features be worked in branches and get merged when they are in a reasonable state. Each branch contributor group should look at their readiness and merge stuff i

Re: Branch merges and 3.0.0-beta1 scope

2017-08-22 Thread Allen Wittenauer
We should avoid turning this into a replay of Apache Hadoop 2.6.0 (and to a lesser degree, 2.7.0 and 2.8.0) where a bunch of last minute “experimental” features derail stability for a significantly long period of time. -

Re: Branch merges and 3.0.0-beta1 scope

2017-08-22 Thread Ray Chiang
On 8/22/17 3:20 AM, Steve Loughran wrote: On 21 Aug 2017, at 22:22, Vinod Kumar Vavilapalli wrote: Steve, You can be strict & ruthless about the timelines. Anything that doesn’t get in by mid-September, as was originally planned, can move to the next release - whether it is feature work on

Re: Branch merges and 3.0.0-beta1 scope

2017-08-22 Thread Steve Loughran
> On 21 Aug 2017, at 22:22, Vinod Kumar Vavilapalli wrote: > > Steve, > > You can be strict & ruthless about the timelines. Anything that doesn’t get > in by mid-September, as was originally planned, can move to the next release > - whether it is feature work on branches or feature work on tr

RE: Branch merges and 3.0.0-beta1 scope

2017-08-21 Thread Brahma Reddy Battula
Loughran; Andrew Wang; common-...@hadoop.apache.org; hdfs-...@hadoop.apache.org; mapreduce-dev@hadoop.apache.org; yarn-...@hadoop.apache.org Subject: Re: Branch merges and 3.0.0-beta1 scope Andrew, Thanks for your help to pushing this release. Echoing what Vinod said, all contributors in th

Re: Branch merges and 3.0.0-beta1 scope

2017-08-21 Thread Wangda Tan
Andrew, Thanks for your help to pushing this release. Echoing what Vinod said, all contributors in these branches are putting months to years of time working on these features, we don't have to decide excluded features now since we have 25 days till 3.0-beta1 planned release time. The best appro

Re: Branch merges and 3.0.0-beta1 scope

2017-08-21 Thread Vinod Kumar Vavilapalli
Steve, You can be strict & ruthless about the timelines. Anything that doesn’t get in by mid-September, as was originally planned, can move to the next release - whether it is feature work on branches or feature work on trunk. The problem I see here is that code & branches being worked on for a

Re: Branch merges and 3.0.0-beta1 scope

2017-08-19 Thread Steve Loughran
> On 19 Aug 2017, at 02:22, Andrew Wang wrote: > > * 3.0.0-beta1 includes S3Guard and YARN federation. Target date remains > mid-Sept. > * 3.0.0-GA includes TSv2 alpha2. Target date remains early November. > * Everything else waits for 3.1, approximately March 2018. > probably time to create

Re: Branch merges and 3.0.0-beta1 scope

2017-08-18 Thread Vinod Kumar Vavilapalli
Andrew, Each of the branches below have been created more than a year ago (!) and have been consistently worked upon and are now finally seeing the light of the day. When they are "few weeks” away, pushing them out by 7 *more* months just doesn’t make sense. While I deeply appreciate the push

Branch merges and 3.0.0-beta1 scope

2017-08-18 Thread Andrew Wang
Hi folks, As you might have seen, we've had a number of branch merges floated this past week targeted for 3.0.0-beta1, which is planned for about a month from now. In total, I'm currently tracking these branches: YARN-2915: YARN federation (recently merged) HADOOP-13345: S3Guard (currently being