Timeline Service v2 should be landing on branch2 shortly. thanks Vrushali
On Thu, Sep 7, 2017 at 4:57 PM, Vrushali C <vrushalic2...@gmail.com> wrote: > Thanks everyone. > > It has been over a week (~9 days) since TSv2 has been merged to trunk with > no problems thus far. We are now thinking about merging timeline service v2 > to branch2 some time in the next few weeks. > > So far, we have been maintaining a branch2 based YARN-5355_branch2 along > with our trunk based feature branch YARN-5355. Varun Saxena has been > diligently rebasing it to stay current with branch2. > > Currently we are in the process of testing it just like we did our due > diligence with the trunk based YARN-5355 branch and will ensure the TSv2 > branch2 code is a stable state to be merged. > > We will send out another email when we are ready to merge to branch2. > thanks > Vrushali > > On Thu, Aug 31, 2017 at 12:33 PM, Subramaniam V K <subru...@gmail.com> > wrote: > >> Good to see this merged. I have initiated a separate thread with a >> smaller set of stakeholders to discuss inclusion in 2.9. We'll report back >> to the 2.9 release thread as soon as we reach consensus. >> >> On Thu, Aug 31, 2017 at 10:39 AM, Ravi Prakash <ravihad...@gmail.com> >> wrote: >> >>> +1 to maintaining history. >>> >>> On Wed, Aug 30, 2017 at 11:38 PM, varunsax...@apache.org < >>> varun.saxena.apa...@gmail.com> wrote: >>> >>> > Yes, I had used "git merge --no-ff" while merging ATSv2 to trunk. >>> > Maintaining history I believe can be useful as it can make reverts >>> > easier if at all required. >>> > And can be an easy reference point to look at who had contributed what >>> > without having to go back to the branch. >>> > >>> > Regards, >>> > Varun Saxena. >>> > >>> > On Thu, Aug 31, 2017 at 3:56 AM, Vrushali C <vrushalic2...@gmail.com> >>> > wrote: >>> > >>> > > Thanks Sangjin for the link to the previous discussions on this! I >>> think >>> > > that helps answer Steve's questions. >>> > > >>> > > As decided on that thread [1], YARN-5355 as a feature branch was >>> merged >>> > to >>> > > trunk via "git merge --no-ff" . >>> > > >>> > > Although trunk already had TSv2 code (alpha1) prior to this merge, we >>> > > chose to develop on a feature branch YARN-5355 so that we could >>> control >>> > > when changes went into trunk and didn't inadvertently disrupt trunk. >>> > > >>> > > Is the latest merge causing any conflicts or issues for s3guard, >>> Steve? >>> > > >>> > > thanks >>> > > Vrushali >>> > > [1] https://lists.apache.org/thread.html/43cd65c6b6c3c0e8ac2b3c7 >>> 6afd9ef >>> > > f1f78b177fabe9c4a96d9b3d0b@1440189889@%3Ccommon-dev.hadoop.a >>> pache.org%3E >>> > > >>> > > >>> > > On Wed, Aug 30, 2017 at 2:37 PM, Sangjin Lee <sj...@apache.org> >>> wrote: >>> > > >>> > >> I recall this discussion about a couple of years ago: >>> > >> https://lists.apache.org/thread.html/43cd65c6b6c3c0e8ac >>> > >> 2b3c76afd9eff1f78b177fabe9c4a96d9b3d0b@1440189889@%3Ccommon- >>> > >> dev.hadoop.apache.org%3E >>> > >> >>> > >> On Wed, Aug 30, 2017 at 2:32 PM, Steve Loughran < >>> ste...@hortonworks.com >>> > > >>> > >> wrote: >>> > >> >>> > >>> I'd have assumed it would have gone in as one single patch, rather >>> than >>> > >>> a full history. I don't see why the trunk needs all the >>> evolutionary >>> > >>> history of a build. >>> > >>> >>> > >>> What should our policy/process be here? >>> > >>> >>> > >>> I do currently plan to merge the s3guard in as one single squashed >>> > >>> patch; just getting HADOOP-14809 sorted first. >>> > >>> >>> > >>> >>> > >>> > On 30 Aug 2017, at 07:09, Vrushali C <vrushalic2...@gmail.com> >>> > wrote: >>> > >>> > >>> > >>> > I'm adding my +1 (binding) to conclude the vote. >>> > >>> > >>> > >>> > With 13 +1's (11 binding) and no -1's, the vote passes. We'll >>> get on >>> > >>> with >>> > >>> > the merge to trunk shortly. Thanks everyone! >>> > >>> > >>> > >>> > Regards >>> > >>> > Vrushali >>> > >>> > >>> > >>> > >>> > >>> > On Tue, Aug 29, 2017 at 10:54 AM, varunsax...@apache.org < >>> > >>> > varun.saxena.apa...@gmail.com> wrote: >>> > >>> > >>> > >>> >> +1 (binding). >>> > >>> >> >>> > >>> >> Kudos to all the team members for their great work! >>> > >>> >> >>> > >>> >> Being part of the ATSv2 team, I have been involved with either >>> > >>> development >>> > >>> >> or review of most of the JIRAs'. >>> > >>> >> Tested ATSv2 in both secure and non-secure mode. Also verified >>> that >>> > >>> there >>> > >>> >> is no impact when ATSv2 is turned off. >>> > >>> >> >>> > >>> >> Regards, >>> > >>> >> Varun Saxena. >>> > >>> >> >>> > >>> >> On Tue, Aug 22, 2017 at 12:02 PM, Vrushali Channapattan < >>> > >>> >> vrushalic2...@gmail.com> wrote: >>> > >>> >> >>> > >>> >>> Hi folks, >>> > >>> >>> >>> > >>> >>> Per earlier discussion [1], I'd like to start a formal vote to >>> > merge >>> > >>> >>> feature branch YARN-5355 [2] (Timeline Service v.2) to trunk. >>> The >>> > >>> vote >>> > >>> >>> will >>> > >>> >>> run for 7 days, and will end August 29 11:00 PM PDT. >>> > >>> >>> >>> > >>> >>> We have previously completed one merge onto trunk [3] and >>> Timeline >>> > >>> Service >>> > >>> >>> v2 has been part of Hadoop release 3.0.0-alpha1. >>> > >>> >>> >>> > >>> >>> Since then, we have been working on extending the capabilities >>> of >>> > >>> Timeline >>> > >>> >>> Service v2 in a feature branch [2] for a while, and we are >>> > reasonably >>> > >>> >>> confident that the state of the feature meets the criteria to >>> be >>> > >>> merged >>> > >>> >>> onto trunk and we'd love folks to get their hands on it in a >>> test >>> > >>> capacity >>> > >>> >>> and provide valuable feedback so that we can make it >>> > >>> production-ready. >>> > >>> >>> >>> > >>> >>> In a nutshell, Timeline Service v.2 delivers significant >>> > scalability >>> > >>> and >>> > >>> >>> usability improvements based on a new architecture. What we >>> would >>> > >>> like to >>> > >>> >>> merge to trunk is termed "alpha 2" (milestone 2). The feature >>> has a >>> > >>> >>> complete end-to-end read/write flow with security and read >>> level >>> > >>> >>> authorization via whitelists. You should be able to start >>> setting >>> > it >>> > >>> up >>> > >>> >>> and >>> > >>> >>> testing it. >>> > >>> >>> >>> > >>> >>> At a high level, the following are the key features that have >>> been >>> > >>> >>> implemented since alpha1: >>> > >>> >>> - Security via Kerberos Authentication and delegation tokens >>> > >>> >>> - Read side simple authorization via whitelist >>> > >>> >>> - Client configurable entity sort ordering >>> > >>> >>> - Richer REST APIs for apps, app attempts, containers, fetching >>> > >>> metrics by >>> > >>> >>> timerange, pagination, sub-app entities >>> > >>> >>> - Support for storing sub-application entities (entities that >>> exist >>> > >>> >>> outside >>> > >>> >>> the scope of an application) >>> > >>> >>> - Configurable TTLs (time-to-live) for tables, configurable >>> table >>> > >>> >>> prefixes, >>> > >>> >>> configurable hbase cluster >>> > >>> >>> - Flow level aggregations done as dynamic (table level) >>> > coprocessors >>> > >>> >>> - Uses latest stable HBase release 1.2.6 >>> > >>> >>> >>> > >>> >>> There are a total of 82 subtasks that were completed as part of >>> > this >>> > >>> >>> effort. >>> > >>> >>> >>> > >>> >>> We paid close attention to ensure that once disabled Timeline >>> > >>> Service v.2 >>> > >>> >>> does not impact existing functionality when disabled (by >>> default). >>> > >>> >>> >>> > >>> >>> Special thanks to a team of folks who worked hard and >>> contributed >>> > >>> towards >>> > >>> >>> this effort with patches, reviews and guidance: Rohith Sharma >>> K S, >>> > >>> Varun >>> > >>> >>> Saxena, Haibo Chen, Sangjin Lee, Li Lu, Vinod Kumar >>> Vavilapalli, >>> > Joep >>> > >>> >>> Rottinghuis, Jason Lowe, Jian He, Robert Kanter, Micheal Stack. >>> > >>> >>> >>> > >>> >>> Regards, >>> > >>> >>> Vrushali >>> > >>> >>> >>> > >>> >>> [1] http://www.mail-archive.com/ya >>> rn-...@hadoop.apache.org/msg27 >>> > >>> 383.html >>> > >>> >>> [2] https://issues.apache.org/jira/browse/YARN-5355 >>> > >>> >>> [3] https://issues.apache.org/jira/browse/YARN-2928 >>> > >>> >>> [4] https://github.com/apache/hadoop/commits/YARN-5355 >>> > >>> >>> >>> > >>> >> >>> > >>> >> >>> > >>> >>> > >>> >>> > >>> ------------------------------------------------------------ >>> --------- >>> > >>> To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org >>> > >>> For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org >>> > >>> >>> > >>> >>> > >> >>> > > >>> > >>> >> >> >