Hi folks,
As you might have seen, we've had a number of branch merges floated this
past week targeted for 3.0.0-beta1, which is planned for about a month from
now.
In total, I'm currently tracking these branches:
YARN-2915: YARN federation (recently merged)
HADOOP-13345: S3Guard (currently being voted on)
YARN-5355: TSv2 alpha2 ("few weeks")
YARN-5079: Native services ("few weeks")
YARN-3926: Resource profiles ("few weeks")
We should effectively be in code freeze (only blockers/criticals), so the
volume of merge proposals at this point came as a surprise. Despite our
best efforts as software engineers, big code movement always comes with
risk.
Since we started the 3.0 release series close to a year ago, I'm also loath
to increase the scope. The main motivation for 3.0 was to deliver JDK8 and
HDFS EC, and our users deserve a production-quality release with these
features. We've also been good about the release cadence thus far in 3.x,
so a 3.1 isn't that far out.
Here's my proposal:
* 3.0.0-beta1 includes S3Guard and YARN federation. Target date remains
mid-Sept.
* 3.0.0-GA includes TSv2 alpha2. Target date remains early November.
* Everything else waits for 3.1, approximately March 2018.
My rationale for inclusion and exclusion is as follows:
Inclusion:
* YARN federation has been run in production, does not touch existing code,
adds no new APIs, and is off by default.
* S3Guard has been run in production and is off by default.
* The first iteration of TSv2 was shipped in 3.0.0-alpha1, so we're
committed to this for 3.0.0 GA. It's off by default and adds no impact.
Exclusion:
* The primary reason for exclusion is to maintain the planned release
schedule. Native services and resource profiles are still a few weeks from
being ready for merge.
* A reminder that 3.1 is only another 3 months after 3.0 given our release
cadence thus far. If there's demand, we could even do a 3.1 immediately
following 3.0.
I'm happy to talk with the contributors of each of these features to
understand their timelines and requirements, with the caveat that I'll be
out through Wednesday next week. Please reach out.
Best,
Andrew
