+1 (binding) Thank you very much for the great team work!
Built from source and deployed in secured cluster. The below are the test result. Deployment : Standard hadoop security deployment authentication and authorization as well. Branch-2 Hadoop and Hbase security cluster. Branch-3 Hadoop security cluster. HBase client is pointing to Branch-2 hbase cluster. All security configurations are set in-place. Each service is running with its own user. Say, HDFS is running with hdfs, YARN user is running with yarn, Hbase is running with hbase Smoke test user : test-user Test Cases : Authentication : Verify for all daemons start up successful : OK Run a MR job using test-user : OK Verify for REST API’s with in the scope of application : OK Verify for REST API’s newly added I.e outside scope of application : OK. RM Restart/ NM restart / RM_work-preserving restart has executed and verified for data : OK. (Entity validation is done, but not entity data validation! Token redistribution to AM, NM is verified. Authorization : 1 . Basic whitelisting of users to read has been validated. Works as expected! Disabling TSv2 configuration is also being tested. Thanks & Regards Rohith Sharma K S On 22 August 2017 at 12:02, Vrushali Channapattan <vrushalic2...@gmail.com> wrote: > Hi folks, > > Per earlier discussion [1], I'd like to start a formal vote to merge > feature branch YARN-5355 [2] (Timeline Service v.2) to trunk. The vote will > run for 7 days, and will end August 29 11:00 PM PDT. > > We have previously completed one merge onto trunk [3] and Timeline Service > v2 has been part of Hadoop release 3.0.0-alpha1. > > Since then, we have been working on extending the capabilities of Timeline > Service v2 in a feature branch [2] for a while, and we are reasonably > confident that the state of the feature meets the criteria to be merged > onto trunk and we'd love folks to get their hands on it in a test capacity > and provide valuable feedback so that we can make it production-ready. > > In a nutshell, Timeline Service v.2 delivers significant scalability and > usability improvements based on a new architecture. What we would like to > merge to trunk is termed "alpha 2" (milestone 2). The feature has a > complete end-to-end read/write flow with security and read level > authorization via whitelists. You should be able to start setting it up and > testing it. > > At a high level, the following are the key features that have been > implemented since alpha1: > - Security via Kerberos Authentication and delegation tokens > - Read side simple authorization via whitelist > - Client configurable entity sort ordering > - Richer REST APIs for apps, app attempts, containers, fetching metrics by > timerange, pagination, sub-app entities > - Support for storing sub-application entities (entities that exist outside > the scope of an application) > - Configurable TTLs (time-to-live) for tables, configurable table prefixes, > configurable hbase cluster > - Flow level aggregations done as dynamic (table level) coprocessors > - Uses latest stable HBase release 1.2.6 > > There are a total of 82 subtasks that were completed as part of this > effort. > > We paid close attention to ensure that once disabled Timeline Service v.2 > does not impact existing functionality when disabled (by default). > > Special thanks to a team of folks who worked hard and contributed towards > this effort with patches, reviews and guidance: Rohith Sharma K S, Varun > Saxena, Haibo Chen, Sangjin Lee, Li Lu, Vinod Kumar Vavilapalli, Joep > Rottinghuis, Jason Lowe, Jian He, Robert Kanter, Micheal Stack. > > Regards, > Vrushali > > [1] http://www.mail-archive.com/yarn-dev@hadoop.apache.org/msg27383.html > [2] https://issues.apache.org/jira/browse/YARN-5355 > [3] https://issues.apache.org/jira/browse/YARN-2928 > [4] https://github.com/apache/hadoop/commits/YARN-5355 >