Re: Branch merges and 3.0.0-beta1 scope

bq. I don't think it's a lot to ask that feature leads shoot an email to the release manager of their target version. DISCUSS emails right before a proposed merge VOTE are way too late, it ends up being a fire drill where we need to scramble on many fronts. Yes, I have been thinking about this

Re: Branch merges and 3.0.0-beta1 scope

Hi Vinod, On Fri, Aug 25, 2017 at 2:42 PM, Vinod Kumar Vavilapalli wrote: > > From a release management perspective, it's *extremely* reasonable to > block the inclusion of new features a month from the planned release date. > A typical software development lifecycle

Re: Branch merges and 3.0.0-beta1 scope

Jonathan, thanks for the heads up. I don't have much familiarity with YARN, but gave the PBs and pom changes a look, and left a few small comments on the umbrella JIRA. This seems like a smaller change than some of the other branch merges we're discussing, but I'm again reticent about adding

Re: Branch merges and 3.0.0-beta1 scope

Here's a summary of some 1-on-1 conversations I had with contributors of the different features I'm tracking. Storage Policy Satisfier (Uma Gangumalla) * Target version: 3.1.0, maybe 3.0.0 GA * We're resolving some design discussions on JIRA. Plan is to do some MVP work on the API to get this

Re: Branch merges and 3.0.0-beta1 scope

> From a release management perspective, it's *extremely* reasonable to block > the inclusion of new features a month from the planned release date. A > typical software development lifecycle includes weeks of feature freeze and > weeks of code freeze. It is no knock on any developer or any

Re: Branch merges and 3.0.0-beta1 scope

Hi Andrew, Thanks for starting the discussion - we have a feature YARN-5734 for API based scheduler configuration that I feel is pretty close to merge (also "a few weeks"). It's almost completely code and API additions and we were careful to design it so that it's compatible (feature is also

Re: Branch merges and 3.0.0-beta1 scope

Resource profile is similar to TSv2, the feature is: - Alpha feature, we will not freeze new added APIs. And all added APIs are explicitly marked to @Unstable. - Allow rolling upgrade from branch-2. - Touched existing code, but we have, and will continue tests to make sure changes are safe.

Re: Branch merges and 3.0.0-beta1 scope

On 25 August 2017 at 22:39, Andrew Wang wrote: > Hi Rohith, > > Given that we're advertising TSv2 as an alpha feature, I think we're > allowed to break compatibility. Let's make sure this is clear in the > release notes and documentation. > > That said, with TSv2 phase

Re: Branch merges and 3.0.0-beta1 scope

Hi Jason, I agree with this proposal. I'll start another email thread spelling this out, and gather additional feedback. Best, Andrew On Fri, Aug 25, 2017 at 6:27 AM, Jason Lowe wrote: > Andrew Wang wrote: > > >> This means I'll cut branch-3 and >> branch-3.0, and move trunk

Re: Branch merges and 3.0.0-beta1 scope

Hi Rohith, Given that we're advertising TSv2 as an alpha feature, I think we're allowed to break compatibility. Let's make sure this is clear in the release notes and documentation. That said, with TSv2 phase 2, is the API going to be frozen? The umbrella JIRA refers to "TSv2 alpha2" which

Re: Branch merges and 3.0.0-beta1 scope

Andrew Wang wrote: > This means I'll cut branch-3 and > branch-3.0, and move trunk to 4.0.0 before these VOTEs end. This will open > up development for Hadoop 3.1.0 and 4.0.0. I can see a need for branch-3.0, but please do not create branch-3. Doing so will relegate trunk back to the "patch

Re: Branch merges and 3.0.0-beta1 scope

Hi Andrew Thanks for update on release plan! I would like to discuss specifically regarding compatibility of releases. What is the compatibility to be maintained for GA if we don't merge to beta1 release? IIUC, till now all the releases were alpha where compatibility was not that important. All

Re: Branch merges and 3.0.0-beta1 scope

Glad to see the discussion continued in my absence :) >From a release management perspective, it's *extremely* reasonable to block the inclusion of new features a month from the planned release date. A typical software development lifecycle includes weeks of feature freeze and weeks of code

Re: Branch merges and 3.0.0-beta1 scope

Also, when people +1 a merge, can they please describe if they did testing / use the feature in addition to what is already described in the thread? On Wed, Aug 23, 2017 at 11:18 AM, Vrushali Channapattan < vrushalic2...@gmail.com> wrote: > For timeline service v2, we have completed all subtasks

Re: Branch merges and 3.0.0-beta1 scope

For timeline service v2, we have completed all subtasks under YARN-5355 [1]. We initiated a merge-to-trunk vote [2] yesterday. thanks Vrushali [1] https://issues.apache.org/jira/browse/YARN-5355 [2]

Re: Branch merges and 3.0.0-beta1 scope

Agreed. I was very clearly not advocating for rushing in features. If you have followed my past emails, I have only strongly advocated features be worked in branches and get merged when they are in a reasonable state. Each branch contributor group should look at their readiness and merge stuff

Re: Branch merges and 3.0.0-beta1 scope

We should avoid turning this into a replay of Apache Hadoop 2.6.0 (and to a lesser degree, 2.7.0 and 2.8.0) where a bunch of last minute “experimental” features derail stability for a significantly long period of time.

Re: Branch merges and 3.0.0-beta1 scope

On 8/22/17 3:20 AM, Steve Loughran wrote: On 21 Aug 2017, at 22:22, Vinod Kumar Vavilapalli wrote: Steve, You can be strict & ruthless about the timelines. Anything that doesn’t get in by mid-September, as was originally planned, can move to the next release - whether

Re: Branch merges and 3.0.0-beta1 scope

> On 21 Aug 2017, at 22:22, Vinod Kumar Vavilapalli wrote: > > Steve, > > You can be strict & ruthless about the timelines. Anything that doesn’t get > in by mid-September, as was originally planned, can move to the next release > - whether it is feature work on branches

RE: Branch merges and 3.0.0-beta1 scope

Andrew Wang; common-dev@hadoop.apache.org; hdfs-...@hadoop.apache.org; mapreduce-...@hadoop.apache.org; yarn-...@hadoop.apache.org Subject: Re: Branch merges and 3.0.0-beta1 scope Andrew, Thanks for your help to pushing this release. Echoing what Vinod said, all contributors in these br

Re: Branch merges and 3.0.0-beta1 scope

Andrew, Thanks for your help to pushing this release. Echoing what Vinod said, all contributors in these branches are putting months to years of time working on these features, we don't have to decide excluded features now since we have 25 days till 3.0-beta1 planned release time. The best

Re: Branch merges and 3.0.0-beta1 scope

Steve, You can be strict & ruthless about the timelines. Anything that doesn’t get in by mid-September, as was originally planned, can move to the next release - whether it is feature work on branches or feature work on trunk. The problem I see here is that code & branches being worked on for

Re: Branch merges and 3.0.0-beta1 scope

> On 19 Aug 2017, at 02:22, Andrew Wang wrote: > > * 3.0.0-beta1 includes S3Guard and YARN federation. Target date remains > mid-Sept. > * 3.0.0-GA includes TSv2 alpha2. Target date remains early November. > * Everything else waits for 3.1, approximately March 2018. >

Re: Branch merges and 3.0.0-beta1 scope

Andrew, Each of the branches below have been created more than a year ago (!) and have been consistently worked upon and are now finally seeing the light of the day. When they are "few weeks” away, pushing them out by 7 *more* months just doesn’t make sense. While I deeply appreciate the push

Branch merges and 3.0.0-beta1 scope

Hi folks, As you might have seen, we've had a number of branch merges floated this past week targeted for 3.0.0-beta1, which is planned for about a month from now. In total, I'm currently tracking these branches: YARN-2915: YARN federation (recently merged) HADOOP-13345: S3Guard (currently