Near the bottom of the bylaws, it states that addition of a "New Branch Committer" requires "Lazy consensus of active PMC members." I think this means that you'll need to get a PMC member to sponsor the vote for you. Regular committer votes happen on the private PMC mailing list, and I assume it would be the same for a branch committer vote.
http://hadoop.apache.org/bylaws.html Chris Nauroth Hortonworks http://hortonworks.com/ On Tue, Aug 6, 2013 at 2:48 PM, Larry McCay <lmc...@hortonworks.com> wrote: > That sounds perfect! > I have been thinking of late that we would maybe need an incubator project > or something for this - which would be unfortunate. > > This would allow us to move much more quickly with a set of patches broken > up into consumable/understandable chunks that are made functional more > easily within the branch. > I assume that we need to start a separate thread for DISCUSS or VOTE to > start that process - correct? > > On Aug 6, 2013, at 4:15 PM, Alejandro Abdelnur <t...@cloudera.com> wrote: > > > yep, that is what I meant. Thanks Chris > > > > > > On Tue, Aug 6, 2013 at 1:12 PM, Chris Nauroth <cnaur...@hortonworks.com > >wrote: > > > >> Perhaps this is also a good opportunity to try out the new "branch > >> committers" clause in the bylaws, enabling non-committers who are > working > >> on this to commit to the feature branch. > >> > >> > >> > http://mail-archives.apache.org/mod_mbox/hadoop-general/201308.mbox/%3CCACO5Y4we4d8knB_xU3a=hr2gbeqo5m3vau+inba0li1i9e2...@mail.gmail.com%3E > >> > >> Chris Nauroth > >> Hortonworks > >> http://hortonworks.com/ > >> > >> > >> > >> On Tue, Aug 6, 2013 at 1:04 PM, Alejandro Abdelnur <t...@cloudera.com > >>> wrote: > >> > >>> Larry, > >>> > >>> Sorry for the delay answering. Thanks for laying down things, yes, it > >> makes > >>> sense. > >>> > >>> Given the large scope of the changes, number of JIRAs and number of > >>> developers involved, wouldn't make sense to create a feature branch for > >> all > >>> this work not to destabilize (more ;) trunk? > >>> > >>> Thanks again. > >>> > >>> > >>> On Tue, Jul 30, 2013 at 9:43 AM, Larry McCay <lmc...@hortonworks.com> > >>> wrote: > >>> > >>>> The following JIRA was filed to provide a token and basic authority > >>>> implementation for this effort: > >>>> https://issues.apache.org/jira/browse/HADOOP-9781 > >>>> > >>>> I have attached an initial patch though have yet to submit it as one > >>> since > >>>> it is dependent on the patch for CMF that was posted to: > >>>> https://issues.apache.org/jira/browse/HADOOP-9534 > >>>> and this patch still has a couple outstanding issues - javac warnings > >> for > >>>> com.sun classes for certification generation and 11 javadoc warnings. > >>>> > >>>> Please feel free to review the patches and raise any questions or > >>> concerns > >>>> related to them. > >>>> > >>>> On Jul 26, 2013, at 8:59 PM, Larry McCay <lmc...@hortonworks.com> > >> wrote: > >>>> > >>>>> Hello All - > >>>>> > >>>>> In an effort to scope an initial iteration that provides value to the > >>>> community while focusing on the pluggable authentication aspects, I've > >>>> written a description for "Iteration 1". It identifies the goal of the > >>>> iteration, the endstate and a set of initial usecases. It also > >> enumerates > >>>> the components that are required for each usecase. There is a scope > >>> section > >>>> that details specific things that should be kept out of the first > >>>> iteration. This is certainly up for discussion. There may be some of > >>> these > >>>> things that can be contributed in short order. If we can add some > >> things > >>> in > >>>> without unnecessary complexity for the identified usecases then we > >>> should. > >>>>> > >>>>> @Alejandro - please review this and see whether it satisfies your > >> point > >>>> for a definition of what we are building. > >>>>> > >>>>> In addition to the document that I will paste here as text and > >> attach a > >>>> pdf version, we have a couple patches for components that are > >> identified > >>> in > >>>> the document. > >>>>> Specifically, COMP-7 and COMP-8. > >>>>> > >>>>> I will be posting COMP-8 patch to the HADOOP-9534 JIRA which was > >> filed > >>>> specifically for that functionality. > >>>>> COMP-7 is a small set of classes to introduce JsonWebToken as the > >> token > >>>> format and a basic JsonWebTokenAuthority that can issue and verify > >> these > >>>> tokens. > >>>>> > >>>>> Since there is no JIRA for this yet, I will likely file a new JIRA > >> for > >>> a > >>>> SSO token implementation. > >>>>> > >>>>> Both of these patches assume to be modules within > >>>> hadoop-common/hadoop-common-project. > >>>>> While they are relatively small, I think that they will be pulled in > >> by > >>>> other modules such as hadoop-auth which would likely not want a > >>> dependency > >>>> on something larger like > >>> hadoop-common/hadoop-common-project/hadoop-common. > >>>>> > >>>>> This is certainly something that we should discuss within the > >> community > >>>> for this effort though - that being, exactly how to add these > libraries > >>> so > >>>> that they are most easily consumed by existing projects. > >>>>> > >>>>> Anyway, the following is the Iteration-1 document - it is also > >> attached > >>>> as a pdf: > >>>>> > >>>>> Iteration 1: Pluggable User Authentication and Federation > >>>>> > >>>>> Introduction > >>>>> The intent of this effort is to bootstrap the development of > >> pluggable > >>>> token-based authentication mechanisms to support certain goals of > >>>> enterprise authentication integrations. By restricting the scope of > >> this > >>>> effort, we hope to provide immediate benefit to the community while > >>> keeping > >>>> the initial contribution to a manageable size that can be easily > >>> reviewed, > >>>> understood and extended with further development through follow up > >> JIRAs > >>>> and related iterations. > >>>>> > >>>>> Iteration Endstate > >>>>> Once complete, this effort will have extended the authentication > >>>> mechanisms - for all client types - from the existing: Simple, > Kerberos > >>> and > >>>> Plain (for RPC) to include LDAP authentication and SAML based > >> federation. > >>>> In addition, the ability to provide additional/custom authentication > >>>> mechanisms will be enabled for users to plug in their preferred > >>> mechanisms. > >>>>> > >>>>> Project Scope > >>>>> The scope of this effort is a subset of the features covered by the > >>>> overviews of HADOOP-9392 and HADOOP-9533. This effort concentrates on > >>>> enabling Hadoop to issue, accept/validate SSO tokens of its own. The > >>>> pluggable authentication mechanism within SASL/RPC layer and the > >>>> authentication filter pluggability for REST and UI components will be > >>>> leveraged and extended to support the results of this effort. > >>>>> > >>>>> Out of Scope > >>>>> In order to scope the initial deliverable as the minimally viable > >>>> product, a handful of things have been simplified or left out of scope > >>> for > >>>> this effort. This is not meant to say that these aspects are not > useful > >>> or > >>>> not needed but that they are not necessary for this iteration. We do > >>>> however need to ensure that we don’t do anything to preclude adding > >> them > >>> in > >>>> future iterations. > >>>>> 1. Additional Attributes - the result of authentication will continue > >>> to > >>>> use the existing hadoop tokens and identity representations. > Additional > >>>> attributes used for finer grained authorization decisions will be > added > >>>> through follow-up efforts. > >>>>> 2. Token revocation - the ability to revoke issued identity tokens > >> will > >>>> be added later > >>>>> 3. Multi-factor authentication - this will likely require additional > >>>> attributes and is not necessary for this iteration. > >>>>> 4. Authorization changes - we will require additional attributes for > >>> the > >>>> fine-grained access control plans. This is not needed for this > >> iteration. > >>>>> 5. Domains - we assume a single flat domain for all users > >>>>> 6. Kinit alternative - we can leverage existing REST clients such as > >>>> cURL to retrieve tokens through authentication and federation for the > >>> time > >>>> being > >>>>> 7. A specific authentication framework isn’t really necessary within > >>> the > >>>> REST endpoints for this iteration. If one is available then we can use > >> it > >>>> otherwise we can leverage existing things like Apache Shiro within a > >>>> servlet filter. > >>>>> > >>>>> In Scope > >>>>> What is in scope for this effort is defined by the usecases described > >>>> below. Components required for supporting the usecases are summarized > >> for > >>>> each client type. Each component is a candidate for a JIRA subtask - > >>> though > >>>> multiple components are likely to be included in a JIRA to represent a > >>> set > >>>> of functionality rather than individual JIRAs per component. > >>>>> > >>>>> Terminology and Naming > >>>>> The terms and names of components within this document are merely > >>>> descriptive of the functionality that they represent. Any similarity > or > >>>> difference in names or terms from those that are found in other > >> documents > >>>> are not intended to make any statement about those other documents or > >> the > >>>> descriptions within. This document represents the pluggable > >>> authentication > >>>> mechanisms and server functionality required to replace Kerberos. > >>>>> > >>>>> Ultimately, the naming of the implementation classes will be a > >> product > >>>> of the patches accepted by the community. > >>>>> > >>>>> Usecases: > >>>>> client types: REST, CLI, UI > >>>>> authentication types: Simple, Kerberos, authentication/LDAP, > >>>> federation/SAML > >>>>> > >>>>> Simple and Kerberos > >>>>> Simple and Kerberos usecases continue to work as they do today. The > >>>> addition of Authentication/LDAP and Federation/SAML are added through > >> the > >>>> existing pluggability points either as they are or with required > >>> extension. > >>>> Either way, continued support for Simple and Kerberos must not require > >>>> changes to existing deployments in the field as a result of this > >> effort. > >>>>> > >>>>> REST > >>>>> USECASE REST-1 Authentication/LDAP: > >>>>> For REST clients, we will provide the ability to: > >>>>> 1. use cURL to Authenticate via LDAP through an IdP endpoint exposed > >> by > >>>> an AuthenticationServer instance via REST calls to: > >>>>> a. authenticate - passing username/password returning a hadoop > >>>> id_token > >>>>> b. get-access-token - from the TokenGrantingService by passing the > >>>> hadoop id_token as an Authorization: Bearer token along with the > >> desired > >>>> service name (master service name) returning a hadoop access token > >>>>> 2. Successfully invoke a hadoop service REST API passing the hadoop > >>>> access token through an HTTP header as an Authorization Bearer token > >>>>> a. validation of the incoming token on the service endpoint is > >>>> accomplished by an SSOAuthenticationHandler > >>>>> 3. Successfully block access to a REST resource when presenting a > >>> hadoop > >>>> access token intended for a different service > >>>>> a. validation of the incoming token on the service endpoint is > >>>> accomplished by an SSOAuthenticationHandler > >>>>> > >>>>> USECASE REST-2 Federation/SAML: > >>>>> We will also provide federation capabilities for REST clients such > >>> that: > >>>>> 1. acquire SAML assertion token from a trusted IdP (shibboleth?) and > >>>> persist in a permissions protected file - ie. > >> ~/.hadoop_tokens/.idp_token > >>>>> 2. use cURL to Federate a token from a trusted IdP through an SP > >>>> endpoint exposed by an AuthenticationServer(FederationServer?) > instance > >>> via > >>>> REST calls to: > >>>>> a. federate - passing a SAML assertion as an Authorization: Bearer > >>>> token returning a hadoop id_token > >>>>> - can copy and paste from commandline or use cat to include > >>>> persisted token through "--Header Authorization: Bearer 'cat > >>>> ~/.hadoop_tokens/.id_token'" > >>>>> b. get-access-token - from the TokenGrantingService by passing the > >>>> hadoop id_token as an Authorization: Bearer token along with the > >> desired > >>>> service name (master service name) to the TokenGrantingService > >> returning > >>> a > >>>> hadoop access token > >>>>> 3. Successfully invoke a hadoop service REST API passing the hadoop > >>>> access token through an HTTP header as an Authorization Bearer token > >>>>> a. validation of the incoming token on the service endpoint is > >>>> accomplished by an SSOAuthenticationHandler > >>>>> 4. Successfully block access to a REST resource when presenting a > >>> hadoop > >>>> access token intended for a different service > >>>>> a. validation of the incoming token on the service endpoint is > >>>> accomplished by an SSOAuthenticationHandler > >>>>> > >>>>> REQUIRED COMPONENTS for REST USECASES: > >>>>> COMP-1. REST client - cURL or similar > >>>>> COMP-2. REST endpoint for BASIC authentication to LDAP - IdP endpoint > >>>> example - returning hadoop id_token > >>>>> COMP-3. REST endpoint for federation with SAML Bearer token - > >>> shibboleth > >>>> SP?|OpenSAML? - returning hadoop id_token > >>>>> COMP-4. REST TokenGrantingServer endpoint for acquiring hadoop access > >>>> tokens from hadoop id_tokens > >>>>> COMP-5. SSOAuthenticationHandler to validate incoming hadoop access > >>>> tokens > >>>>> COMP-6. some source of a SAML assertion - shibboleth IdP? > >>>>> COMP-7. hadoop token and authority implementations > >>>>> COMP-8. core services for crypto support for signing, verifying and > >> PKI > >>>> management > >>>>> > >>>>> CLI > >>>>> USECASE CLI-1 Authentication/LDAP: > >>>>> For CLI/RPC clients, we will provide the ability to: > >>>>> 1. use cURL to Authenticate via LDAP through an IdP endpoint exposed > >> by > >>>> an AuthenticationServer instance via REST calls to: > >>>>> a. authenticate - passing username/password returning a hadoop > >>>> id_token > >>>>> - for RPC clients we need to persist the returned hadoop > >> identity > >>>> token in a file protected by fs permissions so that it may be > leveraged > >>>> until expiry > >>>>> - directing the returned response to a file may suffice for now > >>>> something like ">~/.hadoop_tokens/.id_token" > >>>>> 2. use hadoop CLI to invoke RPC API on a specific hadoop service > >>>>> a. RPC client negotiates a TokenAuth method through SASL layer, > >>>> hadoop id_token is retrieved from ~/.hadoop_tokens/.id_token is passed > >> as > >>>> Authorization: Bearer token to the get-access-token REST endpoint > >> exposed > >>>> by TokenGrantingService returning a hadoop access token > >>>>> b. RPC server side validates the presented hadoop access token and > >>>> continues to serve request > >>>>> c. Successfully invoke a hadoop service RPC API > >>>>> > >>>>> USECASE CLI-2 Federation/SAML: > >>>>> For CLI/RPC clients, we will provide the ability to: > >>>>> 1. acquire SAML assertion token from a trusted IdP (shibboleth?) and > >>>> persist in a permissions protected file - ie. > >> ~/.hadoop_tokens/.idp_token > >>>>> 2. use cURL to Federate a token from a trusted IdP through an SP > >>>> endpoint exposed by an AuthenticationServer(FederationServer?) > instance > >>> via > >>>> REST calls to: > >>>>> a. federate - passing a SAML assertion as an Authorization: Bearer > >>>> token returning a hadoop id_token > >>>>> - can copy and paste from commandline or use cat to include > >>>> previously persisted token through "--Header Authorization: Bearer > 'cat > >>>> ~/.hadoop_tokens/.id_token'" > >>>>> 3. use hadoop CLI to invoke RPC API on a specific hadoop service > >>>>> a. RPC client negotiates a TokenAuth method through SASL layer, > >>>> hadoop id_token is retrieved from ~/.hadoop_tokens/.id_token is passed > >> as > >>>> Authorization: Bearer token to the get-access-token REST endpoint > >> exposed > >>>> by TokenGrantingService returning a hadoop access token > >>>>> b. RPC server side validates the presented hadoop access token and > >>>> continues to serve request > >>>>> c. Successfully invoke a hadoop service RPC API > >>>>> > >>>>> REQUIRED COMPONENTS for CLI USECASES - (beyond those required for > >>> REST): > >>>>> COMP-9. TokenAuth Method negotiation, etc > >>>>> COMP-10. Client side implementation to leverage REST endpoint for > >>>> acquiring hadoop access tokens given a hadoop id_token > >>>>> COMP-11. Server side implementation to validate incoming hadoop > >> access > >>>> tokens > >>>>> > >>>>> UI > >>>>> Various Hadoop services have their own web UI consoles for > >>>> administration and end user interactions. These consoles need to also > >>>> benefit from the pluggability of authentication mechansims to be on > par > >>>> with the access control of the cluster REST and RPC APIs. > >>>>> Web consoles are protected with an WebSSOAuthenticationHandler which > >>>> will be configured for either authentication or federation. > >>>>> > >>>>> USECASE UI-1 Authentication/LDAP: > >>>>> For the authentication usecase: > >>>>> 1. User’s browser requests access to a UI console page > >>>>> 2. WebSSOAuthenticationHandler intercepts the request and redirects > >> the > >>>> browser to an IdP web endpoint exposed by the AuthenticationServer > >>> passing > >>>> the requested url as the redirect_url > >>>>> 3. IdP web endpoint presents the user with a FORM over https > >>>>> a. user provides username/password and submits the FORM > >>>>> 4. AuthenticationServer authenticates the user with provided > >>> credentials > >>>> against the configured LDAP server and: > >>>>> a. leverages a servlet filter or other authentication mechanism > >> for > >>>> the endpoint and authenticates the user with a simple LDAP bind with > >>>> username and password > >>>>> b. acquires a hadoop id_token and uses it to acquire the required > >>>> hadoop access token which is added as a cookie > >>>>> c. redirects the browser to the original service UI resource via > >> the > >>>> provided redirect_url > >>>>> 5. WebSSOAuthenticationHandler for the original UI resource > >>> interrogates > >>>> the incoming request again for an authcookie that contains an access > >>> token > >>>> upon finding one: > >>>>> a. validates the incoming token > >>>>> b. returns the AuthenticationToken as per AuthenticationHandler > >>>> contract > >>>>> c. AuthenticationFilter adds the hadoop auth cookie with the > >>> expected > >>>> token > >>>>> d. serves requested resource for valid tokens > >>>>> e. subsequent requests are handled by the AuthenticationFilter > >>>> recognition of the hadoop auth cookie > >>>>> > >>>>> USECASE UI-2 Federation/SAML: > >>>>> For the federation usecase: > >>>>> 1. User’s browser requests access to a UI console page > >>>>> 2. WebSSOAuthenticationHandler intercepts the request and redirects > >> the > >>>> browser to an SP web endpoint exposed by the AuthenticationServer > >> passing > >>>> the requested url as the redirect_url. This endpoint: > >>>>> a. is dedicated to redirecting to the external IdP passing the > >>>> required parameters which may include a redirect_url back to itself as > >>> well > >>>> as encoding the original redirect_url so that it can determine it on > >> the > >>>> way back to the client > >>>>> 3. the IdP: > >>>>> a. challenges the user for credentials and authenticates the user > >>>>> b. creates appropriate token/cookie and redirects back to the > >>>> AuthenticationServer endpoint > >>>>> 4. AuthenticationServer endpoint: > >>>>> a. extracts the expected token/cookie from the incoming request > >> and > >>>> validates it > >>>>> b. creates a hadoop id_token > >>>>> c. acquires a hadoop access token for the id_token > >>>>> d. creates appropriate cookie and redirects back to the original > >>>> redirect_url - being the requested resource > >>>>> 5. WebSSOAuthenticationHandler for the original UI resource > >>> interrogates > >>>> the incoming request again for an authcookie that contains an access > >>> token > >>>> upon finding one: > >>>>> a. validates the incoming token > >>>>> b. returns the AuthenticationToken as per AuthenticationHandler > >>>> contrac > >>>>> c. AuthenticationFilter adds the hadoop auth cookie with the > >>> expected > >>>> token > >>>>> d. serves requested resource for valid tokens > >>>>> e. subsequent requests are handled by the AuthenticationFilter > >>>> recognition of the hadoop auth cookie > >>>>> REQUIRED COMPONENTS for UI USECASES: > >>>>> COMP-12. WebSSOAuthenticationHandler > >>>>> COMP-13. IdP Web Endpoint within AuthenticationServer for FORM based > >>>> login > >>>>> COMP-14. SP Web Endpoint within AuthenticationServer for 3rd party > >>> token > >>>> federation > >>>>> > >>>>> > >>>>> > >>>>> On Wed, Jul 10, 2013 at 1:59 PM, Brian Swan < > >> brian.s...@microsoft.com> > >>>> wrote: > >>>>> Thanks, Larry. That is what I was trying to say, but you've said it > >>>> better and in more detail. :-) To extract from what you are saying: > "If > >>> we > >>>> were to reframe the immediate scope to the lowest common denominator > of > >>>> what is needed for accepting tokens in authentication plugins then we > >>>> gain... an end-state for the lowest common denominator that enables > >> code > >>>> patches in the near-term is the best of both worlds." > >>>>> > >>>>> -Brian > >>>>> > >>>>> -----Original Message----- > >>>>> From: Larry McCay [mailto:lmc...@hortonworks.com] > >>>>> Sent: Wednesday, July 10, 2013 10:40 AM > >>>>> To: common-dev@hadoop.apache.org > >>>>> Cc: da...@yahoo-inc.com; Kai Zheng; Alejandro Abdelnur > >>>>> Subject: Re: [DISCUSS] Hadoop SSO/Token Server Components > >>>>> > >>>>> It seems to me that we can have the best of both worlds here...it's > >> all > >>>> about the scoping. > >>>>> > >>>>> If we were to reframe the immediate scope to the lowest common > >>>> denominator of what is needed for accepting tokens in authentication > >>>> plugins then we gain: > >>>>> > >>>>> 1. a very manageable scope to define and agree upon 2. a deliverable > >>>> that should be useful in and of itself 3. a foundation for community > >>>> collaboration that we build on for higher level solutions built on > this > >>>> lowest common denominator and experience as a working community > >>>>> > >>>>> So, to Alejandro's point, perhaps we need to define what would make > >> #2 > >>>> above true - this could serve as the "what" we are building instead of > >>> the > >>>> "how" to build it. > >>>>> Including: > >>>>> a. project structure within hadoop-common-project/common-security or > >>> the > >>>> like b. the usecases that would need to be enabled to make it a self > >>>> contained and useful contribution - without higher level solutions c. > >> the > >>>> JIRA/s for contributing patches d. what specific patches will be > needed > >>> to > >>>> accomplished the usecases in #b > >>>>> > >>>>> In other words, an end-state for the lowest common denominator that > >>>> enables code patches in the near-term is the best of both worlds. > >>>>> > >>>>> I think this may be a good way to bootstrap the collaboration process > >>>> for our emerging security community rather than trying to tackle a > huge > >>>> vision all at once. > >>>>> > >>>>> @Alejandro - if you have something else in mind that would bootstrap > >>>> this process - that would great - please advise. > >>>>> > >>>>> thoughts? > >>>>> > >>>>> On Jul 10, 2013, at 1:06 PM, Brian Swan <brian.s...@microsoft.com> > >>>> wrote: > >>>>> > >>>>>> Hi Alejandro, all- > >>>>>> > >>>>>> There seems to be agreement on the broad stroke description of the > >>>> components needed to achieve pluggable token authentication (I'm sure > >>> I'll > >>>> be corrected if that isn't the case). However, discussion of the > >> details > >>> of > >>>> those components doesn't seem to be moving forward. I think this is > >>> because > >>>> the details are really best understood through code. I also see *a* > >> (i.e. > >>>> one of many possible) token format and pluggable authentication > >>> mechanisms > >>>> within the RPC layer as components that can have immediate benefit to > >>>> Hadoop users AND still allow flexibility in the larger design. So, I > >>> think > >>>> the best way to move the conversation of "what we are aiming for" > >> forward > >>>> is to start looking at code for these components. I am especially > >>>> interested in moving forward with pluggable authentication mechanisms > >>>> within the RPC layer and would love to see what others have done in > >> this > >>>> area (if anything). > >>>>>> > >>>>>> Thanks. > >>>>>> > >>>>>> -Brian > >>>>>> > >>>>>> -----Original Message----- > >>>>>> From: Alejandro Abdelnur [mailto:t...@cloudera.com] > >>>>>> Sent: Wednesday, July 10, 2013 8:15 AM > >>>>>> To: Larry McCay > >>>>>> Cc: common-dev@hadoop.apache.org; da...@yahoo-inc.com; Kai Zheng > >>>>>> Subject: Re: [DISCUSS] Hadoop SSO/Token Server Components > >>>>>> > >>>>>> Larry, all, > >>>>>> > >>>>>> Still is not clear to me what is the end state we are aiming for, > >> or > >>>> that we even agree on that. > >>>>>> > >>>>>> IMO, Instead trying to agree what to do, we should first agree on > >>> the > >>>> final state, then we see what should be changed to there there, then > we > >>> see > >>>> how we change things to get there. > >>>>>> > >>>>>> The different documents out there focus more on how. > >>>>>> > >>>>>> We not try to say how before we know what. > >>>>>> > >>>>>> Thx. > >>>>>> > >>>>>> > >>>>>> > >>>>>> > >>>>>> On Wed, Jul 10, 2013 at 6:42 AM, Larry McCay < > >> lmc...@hortonworks.com > >>>> > >>>> wrote: > >>>>>> > >>>>>>> All - > >>>>>>> > >>>>>>> After combing through this thread - as well as the summit session > >>>>>>> summary thread, I think that we have the following two items that > >> we > >>>>>>> can probably move forward with: > >>>>>>> > >>>>>>> 1. TokenAuth method - assuming this means the pluggable > >>>>>>> authentication mechanisms within the RPC layer (2 votes: Kai and > >>>>>>> Kyle) 2. An actual Hadoop Token format (2 votes: Brian and myself) > >>>>>>> > >>>>>>> I propose that we attack both of these aspects as one. Let's > >> provide > >>>>>>> the structure and interfaces of the pluggable framework for use in > >>>>>>> the RPC layer through leveraging Daryn's pluggability work and POC > >>> it > >>>>>>> with a particular token format (not necessarily the only format > >> ever > >>>>>>> supported - we just need one to start). If there has already been > >>>>>>> work done in this area by anyone then please speak up and commit > >> to > >>>>>>> providing a patch - so that we don't duplicate effort. > >>>>>>> > >>>>>>> @Daryn - is there a particular Jira or set of Jiras that we can > >> look > >>>>>>> at to discern the pluggability mechanism details? Documentation of > >>> it > >>>>>>> would be great as well. > >>>>>>> @Kai - do you have existing code for the pluggable token > >>>>>>> authentication mechanism - if not, we can take a stab at > >>> representing > >>>>>>> it with interfaces and/or POC code. > >>>>>>> I can standup and say that we have a token format that we have > >> been > >>>>>>> working with already and can provide a patch that represents it > >> as a > >>>>>>> contribution to test out the pluggable tokenAuth. > >>>>>>> > >>>>>>> These patches will provide progress toward code being the central > >>>>>>> discussion vehicle. As a community, we can then incrementally > >> build > >>>>>>> on that foundation in order to collaboratively deliver the common > >>>> vision. > >>>>>>> > >>>>>>> In the absence of any other home for posting such patches, let's > >>>>>>> assume that they will be attached to HADOOP-9392 - or a dedicated > >>>>>>> subtask for this particular aspect/s - I will leave that detail to > >>>> Kai. > >>>>>>> > >>>>>>> @Alejandro, being the only voice on this thread that isn't > >>>>>>> represented in the votes above, please feel free to agree or > >>> disagree > >>>> with this direction. > >>>>>>> > >>>>>>> thanks, > >>>>>>> > >>>>>>> --larry > >>>>>>> > >>>>>>> On Jul 5, 2013, at 3:24 PM, Larry McCay <lmc...@hortonworks.com> > >>>> wrote: > >>>>>>> > >>>>>>>> Hi Andy - > >>>>>>>> > >>>>>>>>> Happy Fourth of July to you and yours. > >>>>>>>> > >>>>>>>> Same to you and yours. :-) > >>>>>>>> We had some fun in the sun for a change - we've had nothing but > >>> rain > >>>>>>>> on > >>>>>>> the east coast lately. > >>>>>>>> > >>>>>>>>> My concern here is there may have been a misinterpretation or > >> lack > >>>>>>>>> of consensus on what is meant by "clean slate" > >>>>>>>> > >>>>>>>> > >>>>>>>> Apparently so. > >>>>>>>> On the pre-summit call, I stated that I was interested in > >>>>>>>> reconciling > >>>>>>> the jiras so that we had one to work from. > >>>>>>>> > >>>>>>>> You recommended that we set them aside for the time being - with > >>> the > >>>>>>> understanding that work would continue on your side (and our's as > >>>>>>> well) - and approach the community discussion from a clean slate. > >>>>>>>> We seemed to do this at the summit session quite well. > >>>>>>>> It was my understanding that this community discussion would live > >>>>>>>> beyond > >>>>>>> the summit and continue on this list. > >>>>>>>> > >>>>>>>> While closing the summit session we agreed to follow up on > >>>>>>>> common-dev > >>>>>>> with first a summary then a discussion of the moving parts. > >>>>>>>> > >>>>>>>> I never expected the previous work to be abandoned and fully > >>>>>>>> expected it > >>>>>>> to inform the discussion that happened here. > >>>>>>>> > >>>>>>>> If you would like to reframe what clean slate was supposed to > >> mean > >>>>>>>> or > >>>>>>> describe what it means now - that would be welcome - before I > >> waste > >>>>>>> anymore time trying to facilitate a community discussion that is > >>>>>>> apparently not wanted. > >>>>>>>> > >>>>>>>>> Nowhere in this > >>>>>>>>> picture are self appointed "master JIRAs" and such, which have > >>> been > >>>>>>>>> disappointing to see crop up, we should be collaboratively > >> coding > >>>>>>>>> not planting flags. > >>>>>>>> > >>>>>>>> I don't know what you mean by self-appointed master JIRAs. > >>>>>>>> It has certainly not been anyone's intention to disappoint. > >>>>>>>> Any mention of a new JIRA was just to have a clear context to > >>> gather > >>>>>>>> the > >>>>>>> agreed upon points - previous and/or existing JIRAs would easily > >> be > >>>> linked. > >>>>>>>> > >>>>>>>> Planting flags... I need to go back and read my discussion point > >>>>>>>> about the > >>>>>>> JIRA and see how this is the impression that was made. > >>>>>>>> That is not how I define success. The only flags that count is > >>> code. > >>>>>>> What we are lacking is the roadmap on which to put the code. > >>>>>>>> > >>>>>>>>> I read Kai's latest document as something approaching today's > >>>>>>>>> consensus > >>>>>>> (or > >>>>>>>>> at least a common point of view?) rather than a historical > >>> document. > >>>>>>>>> Perhaps he and it can be given equal share of the consideration. > >>>>>>>> > >>>>>>>> I definitely read it as something that has evolved into something > >>>>>>> approaching what we have been talking about so far. There has not > >>>>>>> however been enough discussion anywhere near the level of detail > >> in > >>>>>>> that document and more details are needed for each component in > >> the > >>>> design. > >>>>>>>> Why the work in that document should not be fed into the > >> community > >>>>>>> discussion as anyone else's would be - I fail to understand. > >>>>>>>> > >>>>>>>> My suggestion continues to be that you should take that document > >>> and > >>>>>>> speak to the inventory of moving parts as we agreed. > >>>>>>>> As these are agreed upon, we will ensure that the appropriate > >>>>>>>> subtasks > >>>>>>> are filed against whatever JIRA is to host them - don't really > >> care > >>>>>>> much which it is. > >>>>>>>> > >>>>>>>> I don't really want to continue with two separate JIRAs - as I > >>>>>>>> stated > >>>>>>> long ago - but until we understand what the pieces are and how > >> they > >>>>>>> relate then they can't be consolidated. > >>>>>>>> Even if 9533 ended up being repurposed as the server instance of > >>> the > >>>>>>> work - it should be a subtask of a larger one - if that is to be > >>>>>>> 9392, so be it. > >>>>>>>> We still need to define all the pieces of the larger picture > >> before > >>>>>>>> that > >>>>>>> can be done. > >>>>>>>> > >>>>>>>> What I thought was the clean slate approach to the discussion > >>> seemed > >>>>>>>> a > >>>>>>> very reasonable way to make all this happen. > >>>>>>>> If you would like to restate what you intended by it or something > >>>>>>>> else > >>>>>>> equally as reasonable as a way to move forward that would be > >>> awesome. > >>>>>>>> > >>>>>>>> I will be happy to work toward the roadmap with everyone once it > >> is > >>>>>>> articulated, understood and actionable. > >>>>>>>> In the meantime, I have work to do. > >>>>>>>> > >>>>>>>> thanks, > >>>>>>>> > >>>>>>>> --larry > >>>>>>>> > >>>>>>>> BTW - I meant to quote you in an earlier response and ended up > >>>>>>>> saying it > >>>>>>> was Aaron instead. Not sure what happened there. :-) > >>>>>>>> > >>>>>>>> On Jul 4, 2013, at 2:40 PM, Andrew Purtell <apurt...@apache.org> > >>>> wrote: > >>>>>>>> > >>>>>>>>> Hi Larry (and all), > >>>>>>>>> > >>>>>>>>> Happy Fourth of July to you and yours. > >>>>>>>>> > >>>>>>>>> In our shop Kai and Tianyou are already doing the coding, so I'd > >>>>>>>>> defer > >>>>>>> to > >>>>>>>>> them on the detailed points. > >>>>>>>>> > >>>>>>>>> My concern here is there may have been a misinterpretation or > >> lack > >>>>>>>>> of consensus on what is meant by "clean slate". Hopefully that > >> can > >>>>>>>>> be > >>>>>>> quickly > >>>>>>>>> cleared up. Certainly we did not mean ignore all that came > >> before. > >>>>>>>>> The > >>>>>>> idea > >>>>>>>>> was to reset discussions to find common ground and new direction > >>>>>>>>> where > >>>>>>> we > >>>>>>>>> are working together, not in conflict, on an agreed upon set of > >>>>>>>>> design points and tasks. There's been a lot of good discussion > >> and > >>>>>>>>> design preceeding that we should figure out how to port over. > >>>>>>>>> Nowhere in this picture are self appointed "master JIRAs" and > >>> such, > >>>>>>>>> which have been disappointing to see crop up, we should be > >>>>>>>>> collaboratively coding not planting flags. > >>>>>>>>> > >>>>>>>>> I read Kai's latest document as something approaching today's > >>>>>>>>> consensus > >>>>>>> (or > >>>>>>>>> at least a common point of view?) rather than a historical > >>> document. > >>>>>>>>> Perhaps he and it can be given equal share of the consideration. > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> On Wednesday, July 3, 2013, Larry McCay wrote: > >>>>>>>>> > >>>>>>>>>> Hey Andrew - > >>>>>>>>>> > >>>>>>>>>> I largely agree with that statement. > >>>>>>>>>> My intention was to let the differences be worked out within > >> the > >>>>>>>>>> individual components once they were identified and subtasks > >>>> created. > >>>>>>>>>> > >>>>>>>>>> My reference to HSSO was really referring to a SSO *server* > >> based > >>>>>>> design > >>>>>>>>>> which was not clearly articulated in the earlier documents. > >>>>>>>>>> We aren't trying to compare and contrast one design over > >> another > >>>>>>> anymore. > >>>>>>>>>> > >>>>>>>>>> Let's move this collaboration along as we've mapped out and the > >>>>>>>>>> differences in the details will reveal themselves and be > >>> addressed > >>>>>>> within > >>>>>>>>>> their components. > >>>>>>>>>> > >>>>>>>>>> I've actually been looking forward to you weighing in on the > >>>>>>>>>> actual discussion points in this thread. > >>>>>>>>>> Could you do that? > >>>>>>>>>> > >>>>>>>>>> At this point, I am most interested in your thoughts on a > >> single > >>>>>>>>>> jira > >>>>>>> to > >>>>>>>>>> represent all of this work and whether we should start > >> discussing > >>>>>>>>>> the > >>>>>>> SSO > >>>>>>>>>> Tokens. > >>>>>>>>>> If you think there are discussion points missing from that > >> list, > >>>>>>>>>> feel > >>>>>>> free > >>>>>>>>>> to add to it. > >>>>>>>>>> > >>>>>>>>>> thanks, > >>>>>>>>>> > >>>>>>>>>> --larry > >>>>>>>>>> > >>>>>>>>>> On Jul 3, 2013, at 7:35 PM, Andrew Purtell < > >> apurt...@apache.org> > >>>>>>> wrote: > >>>>>>>>>> > >>>>>>>>>>> Hi Larry, > >>>>>>>>>>> > >>>>>>>>>>> Of course I'll let Kai speak for himself. However, let me > >> point > >>>>>>>>>>> out > >>>>>>> that, > >>>>>>>>>>> while the differences between the competing JIRAs have been > >>>>>>>>>>> reduced > >>>>>>> for > >>>>>>>>>>> sure, there were some key differences that didn't just > >>> disappear. > >>>>>>>>>>> Subsequent discussion will make that clear. I also disagree > >> with > >>>>>>>>>>> your characterization that we have simply endorsed all of the > >>>>>>>>>>> design > >>>>>>> decisions > >>>>>>>>>>> of the so-called HSSO, this is taking a mile from an inch. We > >>> are > >>>>>>> here to > >>>>>>>>>>> engage in a collaborative process as peers. I've been > >> encouraged > >>>>>>>>>>> by > >>>>>>> the > >>>>>>>>>>> spirit of the discussions up to this point and hope that can > >>>>>>>>>>> continue beyond one design summit. > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> On Wed, Jul 3, 2013 at 1:10 PM, Larry McCay > >>>>>>>>>>> <lmc...@hortonworks.com> > >>>>>>>>>> wrote: > >>>>>>>>>>> > >>>>>>>>>>>> Hi Kai - > >>>>>>>>>>>> > >>>>>>>>>>>> I think that I need to clarify something... > >>>>>>>>>>>> > >>>>>>>>>>>> This is not an update for 9533 but a continuation of the > >>>>>>>>>>>> discussions > >>>>>>>>>> that > >>>>>>>>>>>> are focused on a fresh look at a SSO for Hadoop. > >>>>>>>>>>>> We've agreed to leave our previous designs behind and > >> therefore > >>>>>>>>>>>> we > >>>>>>>>>> aren't > >>>>>>>>>>>> really seeing it as an HSSO layered on top of TAS approach or > >>> an > >>>>>>> HSSO vs > >>>>>>>>>>>> TAS discussion. > >>>>>>>>>>>> > >>>>>>>>>>>> Your latest design revision actually makes it clear that you > >>> are > >>>>>>>>>>>> now targeting exactly what was described as HSSO - so > >> comparing > >>>>>>>>>>>> and > >>>>>>>>>> contrasting > >>>>>>>>>>>> is not going to add any value. > >>>>>>>>>>>> > >>>>>>>>>>>> What we need you to do at this point, is to look at those > >>>>>>>>>>>> high-level components described on this thread and comment on > >>>>>>>>>>>> whether we need additional components or any that are listed > >>>>>>>>>>>> that don't seem > >>>>>>> necessary > >>>>>>>>>> to > >>>>>>>>>>>> you and why. > >>>>>>>>>>>> In other words, we need to define and agree on the work that > >>> has > >>>>>>>>>>>> to > >>>>>>> be > >>>>>>>>>>>> done. > >>>>>>>>>>>> > >>>>>>>>>>>> We also need to determine those components that need to be > >> done > >>>>>>> before > >>>>>>>>>>>> anything else can be started. > >>>>>>>>>>>> I happen to agree with Brian that #4 Hadoop SSO Tokens are > >>>>>>>>>>>> central to > >>>>>>>>>> all > >>>>>>>>>>>> the other components and should probably be defined and POC'd > >>> in > >>>>>>> short > >>>>>>>>>>>> order. > >>>>>>>>>>>> > >>>>>>>>>>>> Personally, I think that continuing the separation of 9533 > >> and > >>>>>>>>>>>> 9392 > >>>>>>> will > >>>>>>>>>>>> do this effort a disservice. There doesn't seem to be enough > >>>>>>> differences > >>>>>>>>>>>> between the two to justify separate jiras anymore. It may be > >>>>>>>>>>>> best to > >>>>>>>>>> file a > >>>>>>>>>>>> new one that reflects a single vision without the extra cruft > >>>>>>>>>>>> that > >>>>>>> has > >>>>>>>>>>>> built up in either of the existing ones. We would certainly > >>>>>>>>>>>> reference > >>>>>>>>>> the > >>>>>>>>>>>> existing ones within the new one. This approach would align > >>> with > >>>>>>>>>>>> the > >>>>>>>>>> spirit > >>>>>>>>>>>> of the discussions up to this point. > >>>>>>>>>>>> > >>>>>>>>>>>> I am prepared to start a discussion around the shape of the > >> two > >>>>>>> Hadoop > >>>>>>>>>> SSO > >>>>>>>>>>>> tokens: identity and access. If this is what others feel the > >>>>>>>>>>>> next > >>>>>>> topic > >>>>>>>>>>>> should be. > >>>>>>>>>>>> If we can identify a jira home for it, we can do it there - > >>>>>>> otherwise we > >>>>>>>>>>>> can create another DISCUSS thread for it. > >>>>>>>>>>>> > >>>>>>>>>>>> thanks, > >>>>>>>>>>>> > >>>>>>>>>>>> --larry > >>>>>>>>>>>> > >>>>>>>>>>>> > >>>>>>>>>>>> On Jul 3, 2013, at 2:39 PM, "Zheng, Kai" < > >> kai.zh...@intel.com> > >>>>>>> wrote: > >>>>>>>>>>>> > >>>>>>>>>>>>> Hi Larry, > >>>>>>>>>>>>> > >>>>>>>>>>>>> Thanks for the update. Good to see that with this update we > >>> are > >>>>>>>>>>>>> now > >>>>>>>>>>>> aligned on most points. > >>>>>>>>>>>>> > >>>>>>>>>>>>> I have also updated our TokenAuth design in HADOOP-9392. The > >>>>>>>>>>>>> new > >>>>>>>>>>>> revision incorporates feedback and suggestions in related > >>>>>>>>>>>> discussion > >>>>>>>>>> with > >>>>>>>>>>>> the community, particularly from Microsoft and others > >> attending > >>>>>>>>>>>> the Security design lounge session at the Hadoop summit. > >>> Summary > >>>>>>>>>>>> of the > >>>>>>>>>> changes: > >>>>>>>>>>>>> 1. Revised the approach to now use two tokens, Identity > >>> Token > >>>>>>> plus > >>>>>>>>>>>> Access Token, particularly considering our authorization > >>>>>>>>>>>> framework > >>>>>>> and > >>>>>>>>>>>> compatibility with HSSO; > >>>>>>>>>>>>> 2. Introduced Authorization Server (AS) from our > >>>> authorization > >>>>>>>>>>>> framework into the flow that issues access tokens for clients > >>>>>>>>>>>> with > >>>>>>>>>> identity > >>>>>>>>>>>> tokens to access services; > >>>>>>>>>>>>> 3. Refined proxy access token and the proxy/impersonation > >>>> flow; > >>>>>>>>>>>>> 4. Refined the browser web SSO flow regarding access to > >>>> Hadoop > >>>>>>> web > >>>>>>>>>>>> services; > >>>>>>>>>>>>> 5. Added Hadoop RPC access flow regard > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> -- > >>>>>>>>> Best regards, > >>>>>>>>> > >>>>>>>>> - Andy > >>>>>>>>> > >>>>>>>>> Problems worthy of attack prove their worth by hitting back. - > >>> Piet > >>>>>>>>> Hein (via Tom White) > >>>>>>>> > >>>>>>> > >>>>>>> > >>>>>> > >>>>>> > >>>>>> -- > >>>>>> Alejandro > >>>>>> > >>>>> > >>>>> > >>>>> > >>>>> > >>>>> > >>>>> <Iteration1PluggableUserAuthenticationandFederation.pdf> > >>>> > >>>> > >>> > >>> > >>> -- > >>> Alejandro > >>> > >> > > > > > > > > -- > > Alejandro > >
