Kudos to Vrushali and the team for getting ready for this large and important feature! I know a huge team effort went into this. I look forward to seeing this merged.
I'd like to ask one piece of due diligence. Could you please inspect rigorously to ensure that when disabled Timeline Service v.2 does not impact other features in any way? We did a similar exercise when we had the first drop, and it would be good to repeat that... Thanks! Sangjin On Wed, Aug 16, 2017 at 11:44 AM, Andrew Wang <andrew.w...@cloudera.com> wrote: > Great, thanks Vrushali! Sounds good to me. > > I have a few procedural release notes comments I'll put on YARN-5355, to > make sure we advertise this to our users appropriately. > > On Wed, Aug 16, 2017 at 11:32 AM, Vrushali Channapattan < > vrushal...@gmail.com> wrote: > > > Hi Andrew, > > > > Thanks for your response! > > > > There have been no changes to existing APIs since alpha1. > > > > We at Twitter have tested the feature to demonstrate it works at what we > > consider moderate scale but this did not include the security related > > testing. The security testing is in progress at present by Timeline > Service > > V2 team in the community and we think we will have more details on this > > very soon. > > > > About the jiras under YARN-5355: Only 3 of those sub-tasks are what we > > think of as "merge-blockers". The issues being targeted for merge are in > > [link1] below. There are about 59 jiras of which 56 are completed. > > > > We plan to make a new umbrella jira after the merge to trunk. We will > then > > create a new branch with the new jira name and move these open jiras > under > > YARN-5355 as subtasks of that new umbrella jira. > > > > thanks > > Vrushali > > [link1] https://issues.apache.org/jira/projects/YARN/versions/12337991 > > > > > > On Wed, Aug 16, 2017 at 10:47 AM, Andrew Wang <andrew.w...@cloudera.com> > > wrote: > > > >> Hi Vrushali, > >> > >> Glad to hear this major dev milestone is nearing completion! > >> > >> Repeating my request on other merge [DISCUSS] threads, could you comment > >> on testing and API stability of this merge? Our timeline for beta1 is > about > >> a month out, so there's not much time to fix things beforehand. > >> > >> Looking at YARN-5355 there are also many unresolved subtasks. Should > most > >> of these be moved out to a new umbrella? I'm wondering what needs to be > >> completed before sending the merge vote. > >> > >> Given that TSv2 is committed for 3.0.0 GA, I'm more willing to flex the > >> beta1 release date for this feature than others. Hopefully that won't be > >> necessary though :) > >> > >> Best, > >> Andrew > >> > >> On Wed, Aug 16, 2017 at 10:26 AM, Vrushali Channapattan < > >> vrushalic2...@gmail.com> wrote: > >> > >>> Looks like some of the hyperlinks appear messed up, my apologies, > >>> resending > >>> the same email with hopefully better looking content: > >>> > >>> Hi All, > >>> > >>> I'd like to open a discussion for merging Timeline Service v2 > (YARN-5355) > >>> to trunk in a few weeks. > >>> > >>> We have previously completed one merge onto trunk [1] and Timeline > >>> Service > >>> v2 has been part of Hadoop release 3.0.0-alpha1. > >>> > >>> Since then, we have been working on extending the capabilities of > >>> Timeline > >>> Service v2 in a feature branch [2]. There are a few related issues > >>> pending > >>> that are being actively worked upon and tested. As soon as they are > >>> resolved, we plan on starting a merge vote within the next two weeks. > The > >>> goal is to get this into hadoop3 beta. > >>> > >>> We have paid close attention to ensure that once disabled Timeline > >>> Service > >>> v2 does not impact existing functionality when disabled (by default). > >>> > >>> At a high level, following are the key features that have been > >>> implemented > >>> since 3.0.0-alpha1: > >>> - Security (via Kerberos Authentication & delegation tokens) > [YARN-3053] > >>> - Timeline server usability improvements [timeline-server > >>> <https://issues.apache.org/jira/browse/YARN-5715?jql=project > >>> %20%3D%20YARN%20AND%20fixVersion%20%3D%20YARN-5355%20AND%20c > >>> omponent%20%3D%20timelineserver%20AND%20labels%20!%3D%20atsv > >>> 2-hbase%20ORDER%20BY%20updated%20ASC%2C%20priority%20DESC% > >>> 2C%20created%20ASC> > >>> ] > >>> - HBase specific improvements [atsv2-hbase > >>> <https://issues.apache.org/jira/browse/YARN-6604?jql=project > >>> %20%3D%20YARN%20AND%20fixVersion%20%3D%20YARN-5355%20AND%20l > >>> abels%20%3D%20atsv2-hbase%20ORDER%20BY%20updated%20DESC%2C% > >>> 20affectedVersion%20DESC%2C%20priority%20DESC%2C%20created%20ASC> > >>> ] > >>> - REST API additions and improvements [timeline-reader > >>> <https://issues.apache.org/jira/browse/YARN-5739?jql=project > >>> %20%3D%20YARN%20AND%20fixVersion%20%3D%20YARN-5355%20AND%20c > >>> omponent%20in%20(timelineclient%2C%20timelinereader)% > >>> 20ORDER%20BY%20updated%20ASC%2C%20priority%20DESC%2C%20created%20ASC> > >>> ] > >>> - Reader side simple authorization via whitelist [YARN-6820] > >>> > >>> We would love to get your thoughts on this before we open a real voting > >>> thread. > >>> > >>> Special thanks to a team of folks who worked hard and contributed > towards > >>> this effort via patches, reviews and guidance: Rohith Sharma K S, Varun > >>> Saxena, Haibo Chen, Sangjin Lee, Li Lu, Vinod Kumar Vavilapalli, Joep > >>> Rottinghuis, Jason Lowe, Jian He, Robert Kanter, Micheal Stack. > >>> > >>> Thanks > >>> Vrushali > >>> [1] Merge to trunk: http://www.mail-archive.com/yarn-dev@hadoop.apache > . > >>> org/msg23897.html > >>> <http://www.mail-archive.com/yarn-dev@hadoop.apache.org/msg23897.html> > >>> [2] feature branch YARN-5355 commits: https://github.com/ap > >>> ache/hadoop/commits/YARN-5355 > >>> > >>> <https://github.com/apache/hadoop/commits/YARN-5355> > >>> > >>> > >>> > >>> > >>> > >>> > >>> On Wed, Aug 16, 2017 at 10:02 AM, Vrushali Channapattan < > >>> vrushal...@gmail.com> wrote: > >>> > >>> > Hi All > >>> > > >>> > I’d like to open a discussion for merging Timeline Service v.2 > >>> (YARN-5355) > >>> > to trunk in a few weeks. > >>> > > >>> > We have previously completed one merge onto trunk [1] and Timeline > >>> Service > >>> > v2 has been part of Hadoop release 3.0.0-alpha1. > >>> > > >>> > Since then, we have been working on extending the capabilities of > >>> Timeline > >>> > Service v2 in a feature branch [2]. There are a few related issues > >>> pending > >>> > that are being actively worked upon & tested. As soon as they are > >>> resolved, > >>> > we plan on starting a merge vote within the next 2 weeks. The goal is > >>> to > >>> > get this in for Hadoop3 beta. > >>> > > >>> > We have paid close attention to ensure that once disabled Timeline > >>> Service > >>> > v.2 does not impact existing functionality when disabled (by > default). > >>> > > >>> > At a high level, following are the key features that have been > >>> implemented > >>> > since 3.0.0-alpha1: > >>> > - Security (via Kerberos Authentication & delegation tokens) at the > >>> writer > >>> > [YARN-3053] > >>> > - Timeline server usability improvements [timeline-server > >>> > <https://issues.apache.org/jira/browse/YARN-5715?jql= > >>> > project%20%3D%20YARN%20AND%20fixVersion%20%3D%20YARN- > >>> > 5355%20AND%20component%20%3D%20timelineserver%20AND% > >>> > 20labels%20%21%3D%20atsv2-hbase%20ORDER%20BY%20updated% > >>> > 20ASC%2C%20priority%20DESC%2C%20created%20ASC> > >>> > ] > >>> > - HBase specific improvements [atsv2-hbase > >>> > <https://issues.apache.org/jira/browse/YARN-6604?jql= > >>> > project%20%3D%20YARN%20AND%20fixVersion%20%3D%20YARN- > >>> > 5355%20AND%20labels%20%3D%20atsv2-hbase%20ORDER%20BY%20updat > >>> ed%20DESC%2C% > >>> > 20affectedVersion%20DESC%2C%20priority%20DESC%2C%20created%20ASC> > >>> > ] > >>> > - REST API additions & improvements [timeline-reader > >>> > <https://issues.apache.org/jira/browse/YARN-5739?jql= > >>> > project%20%3D%20YARN%20AND%20fixVersion%20%3D%20YARN- > >>> > 5355%20AND%20component%20in%20%28timelineclient%2C% > >>> > 20timelinereader%29%20ORDER%20BY%20updated%20ASC%2C%20priori > >>> ty%20DESC%2C% > >>> > 20created%20ASC> > >>> > ] > >>> > - Reader side simple authorization via whitelist [YARN-6820] > >>> > > >>> > We would love to get your thoughts on these and more before we open a > >>> real > >>> > voting thread. > >>> > > >>> > Special thanks to a team of folks who worked hard and contributed > >>> towards > >>> > this effort with patches, reviews and guidance: Rohith Sharma K S, > >>> Varun > >>> > Saxena, Haibo Chen, Sangjin Lee, Li Lu, Vinod Kumar Vavilapalli, Joep > >>> > Rottinghuis, Jason Lowe, Jian He, Robert Kanter, Micheal Stack. > >>> > > >>> > Thanks! > >>> > Vrushali > >>> > > >>> > [1] Merge to trunk: http://www.mail-archive.com/yarn-dev@hadoop. > >>> > apache.org/msg23897.html > >>> > [2] feature branch YARN-5355 commits: https://github.com/ > >>> > apache/hadoop/commits/YARN-5355 > >>> > > >>> > >> > >> > > >
