Thanks Vrushali for being entirely open as to the current status of ATSv2. I appreciate that we want to ensure things are tested at scale, and as you said we are working on that right now on our clusters. We have tested the feature to demonstrate it works at what we consider moderate scale.
I think the criteria for including this feature in the 2.9 release should be if it can be safely turned off and not cause impact to anybody not using the new feature. The confidence for this is high for timeline service v2. Therefore, I think timeline service v2 should definitely be part of 2.9. That is the big draw for us to work on stabilizing a 2.9 release rather than just going to 2.8 and back-porting things ourselves. Thanks, Joep On Tue, Jul 25, 2017 at 11:39 PM, Vrushali Channapattan < vrushalic2...@gmail.com> wrote: > Thanks Subru for initiating this discussion. > > Wanted to share some thoughts in the context of Timeline Service v2. The > current status of this module is that we are ramping up for a second merge > to trunk. We still have a few merge blocker jiras outstanding, which we > think we will finish soon. > > While we have done some testing, we are yet to test at scale. Given all > this, we were thinking of initially targeting a beta release vehicle rather > than a stable release. > > As such, timeline service v2 has branch-2 branch called as > YARN-5355-branch-2 in case anyone wants to try it out. Timeline service v2 > can be turned off and should not affect the cluster. > > thanks > Vrushali > > > > > > On Mon, Jul 24, 2017 at 1:26 PM, Subru Krishnan <su...@apache.org> wrote: > > > Folks, > > > > With the release for 2.8, we would like to look ahead to 2.9 release as > > there are many features/improvements in branch-2 (about 1062 commits), > that > > are in need of a release vechile. > > > > Here's our first cut of the proposal from the YARN side: > > > > 1. Scheduler improvements (decoupling allocation from node heartbeat, > > allocation ID, concurrency fixes, LightResource etc). > > 2. Timeline Service v2 > > 3. Opportunistic containers > > 4. Federation > > > > We would like to hear a formal list from HDFS & Hadoop (& MapReduce if > any) > > and will update the Roadmap wiki accordingly. > > > > Considering our familiarity with the above mentioned YARN features, we > > would like to volunteer as the co-RMs for 2.9.0. > > > > We want to keep the timeline at 8-12 weeks to keep the release pragmatic. > > > > Feedback? > > > > -Subru/Arun > > >
