It seems to me that we can have the best of both worlds hereā¦it's all about the scoping.
If we were to reframe the immediate scope to the lowest common denominator of what is needed for accepting tokens in authentication plugins then we gain: 1. a very manageable scope to define and agree upon 2. a deliverable that should be useful in and of itself 3. a foundation for community collaboration that we build on for higher level solutions built on this lowest common denominator and experience as a working community So, to Alejandro's point, perhaps we need to define what would make #2 above true - this could serve as the "what" we are building instead of the "how" to build it. Including: a. project structure within hadoop-common-project/common-security or the like b. the usecases that would need to be enabled to make it a self contained and useful contribution - without higher level solutions c. the JIRA/s for contributing patches d. what specific patches will be needed to accomplished the usecases in #b In other words, an end-state for the lowest common denominator that enables code patches in the near-term is the best of both worlds. I think this may be a good way to bootstrap the collaboration process for our emerging security community rather than trying to tackle a huge vision all at once. @Alejandro - if you have something else in mind that would bootstrap this process - that would great - please advise. thoughts? On Jul 10, 2013, at 1:06 PM, Brian Swan <brian.s...@microsoft.com> wrote: > Hi Alejandro, all- > > There seems to be agreement on the broad stroke description of the components > needed to achieve pluggable token authentication (I'm sure I'll be corrected > if that isn't the case). However, discussion of the details of those > components doesn't seem to be moving forward. I think this is because the > details are really best understood through code. I also see *a* (i.e. one of > many possible) token format and pluggable authentication mechanisms within > the RPC layer as components that can have immediate benefit to Hadoop users > AND still allow flexibility in the larger design. So, I think the best way to > move the conversation of "what we are aiming for" forward is to start looking > at code for these components. I am especially interested in moving forward > with pluggable authentication mechanisms within the RPC layer and would love > to see what others have done in this area (if anything). > > Thanks. > > -Brian > > -----Original Message----- > From: Alejandro Abdelnur [mailto:t...@cloudera.com] > Sent: Wednesday, July 10, 2013 8:15 AM > To: Larry McCay > Cc: common-dev@hadoop.apache.org; da...@yahoo-inc.com; Kai Zheng > Subject: Re: [DISCUSS] Hadoop SSO/Token Server Components > > Larry, all, > > Still is not clear to me what is the end state we are aiming for, or that we > even agree on that. > > IMO, Instead trying to agree what to do, we should first agree on the final > state, then we see what should be changed to there there, then we see how we > change things to get there. > > The different documents out there focus more on how. > > We not try to say how before we know what. > > Thx. > > > > > On Wed, Jul 10, 2013 at 6:42 AM, Larry McCay <lmc...@hortonworks.com> wrote: > >> All - >> >> After combing through this thread - as well as the summit session >> summary thread, I think that we have the following two items that we >> can probably move forward with: >> >> 1. TokenAuth method - assuming this means the pluggable authentication >> mechanisms within the RPC layer (2 votes: Kai and Kyle) 2. An actual >> Hadoop Token format (2 votes: Brian and myself) >> >> I propose that we attack both of these aspects as one. Let's provide >> the structure and interfaces of the pluggable framework for use in the >> RPC layer through leveraging Daryn's pluggability work and POC it with >> a particular token format (not necessarily the only format ever >> supported - we just need one to start). If there has already been work >> done in this area by anyone then please speak up and commit to >> providing a patch - so that we don't duplicate effort. >> >> @Daryn - is there a particular Jira or set of Jiras that we can look >> at to discern the pluggability mechanism details? Documentation of it >> would be great as well. >> @Kai - do you have existing code for the pluggable token >> authentication mechanism - if not, we can take a stab at representing >> it with interfaces and/or POC code. >> I can standup and say that we have a token format that we have been >> working with already and can provide a patch that represents it as a >> contribution to test out the pluggable tokenAuth. >> >> These patches will provide progress toward code being the central >> discussion vehicle. As a community, we can then incrementally build on >> that foundation in order to collaboratively deliver the common vision. >> >> In the absence of any other home for posting such patches, let's >> assume that they will be attached to HADOOP-9392 - or a dedicated >> subtask for this particular aspect/s - I will leave that detail to Kai. >> >> @Alejandro, being the only voice on this thread that isn't represented >> in the votes above, please feel free to agree or disagree with this >> direction. >> >> thanks, >> >> --larry >> >> On Jul 5, 2013, at 3:24 PM, Larry McCay <lmc...@hortonworks.com> wrote: >> >>> Hi Andy - >>> >>>> Happy Fourth of July to you and yours. >>> >>> Same to you and yours. :-) >>> We had some fun in the sun for a change - we've had nothing but rain >>> on >> the east coast lately. >>> >>>> My concern here is there may have been a misinterpretation or lack >>>> of consensus on what is meant by "clean slate" >>> >>> >>> Apparently so. >>> On the pre-summit call, I stated that I was interested in >>> reconciling >> the jiras so that we had one to work from. >>> >>> You recommended that we set them aside for the time being - with the >> understanding that work would continue on your side (and our's as >> well) - and approach the community discussion from a clean slate. >>> We seemed to do this at the summit session quite well. >>> It was my understanding that this community discussion would live >>> beyond >> the summit and continue on this list. >>> >>> While closing the summit session we agreed to follow up on >>> common-dev >> with first a summary then a discussion of the moving parts. >>> >>> I never expected the previous work to be abandoned and fully >>> expected it >> to inform the discussion that happened here. >>> >>> If you would like to reframe what clean slate was supposed to mean >>> or >> describe what it means now - that would be welcome - before I waste >> anymore time trying to facilitate a community discussion that is >> apparently not wanted. >>> >>>> Nowhere in this >>>> picture are self appointed "master JIRAs" and such, which have been >>>> disappointing to see crop up, we should be collaboratively coding >>>> not planting flags. >>> >>> I don't know what you mean by self-appointed master JIRAs. >>> It has certainly not been anyone's intention to disappoint. >>> Any mention of a new JIRA was just to have a clear context to gather >>> the >> agreed upon points - previous and/or existing JIRAs would easily be linked. >>> >>> Planting flags... I need to go back and read my discussion point about >>> the >> JIRA and see how this is the impression that was made. >>> That is not how I define success. The only flags that count is code. >> What we are lacking is the roadmap on which to put the code. >>> >>>> I read Kai's latest document as something approaching today's >>>> consensus >> (or >>>> at least a common point of view?) rather than a historical document. >>>> Perhaps he and it can be given equal share of the consideration. >>> >>> I definitely read it as something that has evolved into something >> approaching what we have been talking about so far. There has not >> however been enough discussion anywhere near the level of detail in >> that document and more details are needed for each component in the design. >>> Why the work in that document should not be fed into the community >> discussion as anyone else's would be - I fail to understand. >>> >>> My suggestion continues to be that you should take that document and >> speak to the inventory of moving parts as we agreed. >>> As these are agreed upon, we will ensure that the appropriate >>> subtasks >> are filed against whatever JIRA is to host them - don't really care >> much which it is. >>> >>> I don't really want to continue with two separate JIRAs - as I >>> stated >> long ago - but until we understand what the pieces are and how they >> relate then they can't be consolidated. >>> Even if 9533 ended up being repurposed as the server instance of the >> work - it should be a subtask of a larger one - if that is to be 9392, >> so be it. >>> We still need to define all the pieces of the larger picture before >>> that >> can be done. >>> >>> What I thought was the clean slate approach to the discussion seemed >>> a >> very reasonable way to make all this happen. >>> If you would like to restate what you intended by it or something >>> else >> equally as reasonable as a way to move forward that would be awesome. >>> >>> I will be happy to work toward the roadmap with everyone once it is >> articulated, understood and actionable. >>> In the meantime, I have work to do. >>> >>> thanks, >>> >>> --larry >>> >>> BTW - I meant to quote you in an earlier response and ended up >>> saying it >> was Aaron instead. Not sure what happened there. :-) >>> >>> On Jul 4, 2013, at 2:40 PM, Andrew Purtell <apurt...@apache.org> wrote: >>> >>>> Hi Larry (and all), >>>> >>>> Happy Fourth of July to you and yours. >>>> >>>> In our shop Kai and Tianyou are already doing the coding, so I'd >>>> defer >> to >>>> them on the detailed points. >>>> >>>> My concern here is there may have been a misinterpretation or lack >>>> of consensus on what is meant by "clean slate". Hopefully that can >>>> be >> quickly >>>> cleared up. Certainly we did not mean ignore all that came before. >>>> The >> idea >>>> was to reset discussions to find common ground and new direction >>>> where >> we >>>> are working together, not in conflict, on an agreed upon set of >>>> design points and tasks. There's been a lot of good discussion and >>>> design preceeding that we should figure out how to port over. >>>> Nowhere in this picture are self appointed "master JIRAs" and such, >>>> which have been disappointing to see crop up, we should be >>>> collaboratively coding not planting flags. >>>> >>>> I read Kai's latest document as something approaching today's >>>> consensus >> (or >>>> at least a common point of view?) rather than a historical document. >>>> Perhaps he and it can be given equal share of the consideration. >>>> >>>> >>>> On Wednesday, July 3, 2013, Larry McCay wrote: >>>> >>>>> Hey Andrew - >>>>> >>>>> I largely agree with that statement. >>>>> My intention was to let the differences be worked out within the >>>>> individual components once they were identified and subtasks created. >>>>> >>>>> My reference to HSSO was really referring to a SSO *server* based >> design >>>>> which was not clearly articulated in the earlier documents. >>>>> We aren't trying to compare and contrast one design over another >> anymore. >>>>> >>>>> Let's move this collaboration along as we've mapped out and the >>>>> differences in the details will reveal themselves and be addressed >> within >>>>> their components. >>>>> >>>>> I've actually been looking forward to you weighing in on the >>>>> actual discussion points in this thread. >>>>> Could you do that? >>>>> >>>>> At this point, I am most interested in your thoughts on a single >>>>> jira >> to >>>>> represent all of this work and whether we should start discussing >>>>> the >> SSO >>>>> Tokens. >>>>> If you think there are discussion points missing from that list, >>>>> feel >> free >>>>> to add to it. >>>>> >>>>> thanks, >>>>> >>>>> --larry >>>>> >>>>> On Jul 3, 2013, at 7:35 PM, Andrew Purtell <apurt...@apache.org> >> wrote: >>>>> >>>>>> Hi Larry, >>>>>> >>>>>> Of course I'll let Kai speak for himself. However, let me point >>>>>> out >> that, >>>>>> while the differences between the competing JIRAs have been >>>>>> reduced >> for >>>>>> sure, there were some key differences that didn't just disappear. >>>>>> Subsequent discussion will make that clear. I also disagree with >>>>>> your characterization that we have simply endorsed all of the >>>>>> design >> decisions >>>>>> of the so-called HSSO, this is taking a mile from an inch. We are >> here to >>>>>> engage in a collaborative process as peers. I've been encouraged >>>>>> by >> the >>>>>> spirit of the discussions up to this point and hope that can >>>>>> continue beyond one design summit. >>>>>> >>>>>> >>>>>> >>>>>> On Wed, Jul 3, 2013 at 1:10 PM, Larry McCay >>>>>> <lmc...@hortonworks.com> >>>>> wrote: >>>>>> >>>>>>> Hi Kai - >>>>>>> >>>>>>> I think that I need to clarify something... >>>>>>> >>>>>>> This is not an update for 9533 but a continuation of the >>>>>>> discussions >>>>> that >>>>>>> are focused on a fresh look at a SSO for Hadoop. >>>>>>> We've agreed to leave our previous designs behind and therefore >>>>>>> we >>>>> aren't >>>>>>> really seeing it as an HSSO layered on top of TAS approach or an >> HSSO vs >>>>>>> TAS discussion. >>>>>>> >>>>>>> Your latest design revision actually makes it clear that you are >>>>>>> now targeting exactly what was described as HSSO - so comparing >>>>>>> and >>>>> contrasting >>>>>>> is not going to add any value. >>>>>>> >>>>>>> What we need you to do at this point, is to look at those >>>>>>> high-level components described on this thread and comment on >>>>>>> whether we need additional components or any that are listed >>>>>>> that don't seem >> necessary >>>>> to >>>>>>> you and why. >>>>>>> In other words, we need to define and agree on the work that has >>>>>>> to >> be >>>>>>> done. >>>>>>> >>>>>>> We also need to determine those components that need to be done >> before >>>>>>> anything else can be started. >>>>>>> I happen to agree with Brian that #4 Hadoop SSO Tokens are >>>>>>> central to >>>>> all >>>>>>> the other components and should probably be defined and POC'd in >> short >>>>>>> order. >>>>>>> >>>>>>> Personally, I think that continuing the separation of 9533 and >>>>>>> 9392 >> will >>>>>>> do this effort a disservice. There doesn't seem to be enough >> differences >>>>>>> between the two to justify separate jiras anymore. It may be >>>>>>> best to >>>>> file a >>>>>>> new one that reflects a single vision without the extra cruft >>>>>>> that >> has >>>>>>> built up in either of the existing ones. We would certainly >>>>>>> reference >>>>> the >>>>>>> existing ones within the new one. This approach would align with >>>>>>> the >>>>> spirit >>>>>>> of the discussions up to this point. >>>>>>> >>>>>>> I am prepared to start a discussion around the shape of the two >> Hadoop >>>>> SSO >>>>>>> tokens: identity and access. If this is what others feel the >>>>>>> next >> topic >>>>>>> should be. >>>>>>> If we can identify a jira home for it, we can do it there - >> otherwise we >>>>>>> can create another DISCUSS thread for it. >>>>>>> >>>>>>> thanks, >>>>>>> >>>>>>> --larry >>>>>>> >>>>>>> >>>>>>> On Jul 3, 2013, at 2:39 PM, "Zheng, Kai" <kai.zh...@intel.com> >> wrote: >>>>>>> >>>>>>>> Hi Larry, >>>>>>>> >>>>>>>> Thanks for the update. Good to see that with this update we are >>>>>>>> now >>>>>>> aligned on most points. >>>>>>>> >>>>>>>> I have also updated our TokenAuth design in HADOOP-9392. The >>>>>>>> new >>>>>>> revision incorporates feedback and suggestions in related >>>>>>> discussion >>>>> with >>>>>>> the community, particularly from Microsoft and others attending >>>>>>> the Security design lounge session at the Hadoop summit. Summary >>>>>>> of the >>>>> changes: >>>>>>>> 1. Revised the approach to now use two tokens, Identity Token >> plus >>>>>>> Access Token, particularly considering our authorization >>>>>>> framework >> and >>>>>>> compatibility with HSSO; >>>>>>>> 2. Introduced Authorization Server (AS) from our authorization >>>>>>> framework into the flow that issues access tokens for clients >>>>>>> with >>>>> identity >>>>>>> tokens to access services; >>>>>>>> 3. Refined proxy access token and the proxy/impersonation flow; >>>>>>>> 4. Refined the browser web SSO flow regarding access to Hadoop >> web >>>>>>> services; >>>>>>>> 5. Added Hadoop RPC access flow regard >>>> >>>> >>>> >>>> -- >>>> Best regards, >>>> >>>> - Andy >>>> >>>> Problems worthy of attack prove their worth by hitting back. - Piet >>>> Hein (via Tom White) >>> >> >> > > > -- > Alejandro >