Hi Alejandro -
I missed your #4 in my summary and takeaways of the session in another thread
on this list.
I believe that the points of discussion were along the lines of:
* put common security libraries into common much the same way as hadoop-auth is
today making each available as separate maven modules to be used across the
ecosystem
* the was a concern raised that we need to be cognizant of not using common as
a "dumping grounds"
- I believe this to mean that we need to ensure that the libraries that
are added there are truly cross cutting and can be used by the other projects
across Hadoop
- I think that security related things will largely be of that nature
but we need to keep it in mind
I'm not sure whether #3 is represented in the other summary or not…
There was certainly discussions around the emerging work from Daryn related to
pluggable authentication mechanisms within that layer and we will immediately
have the options of kerberos, simple and plain. There was also talk of how this
can be leveraged to introduce a Hadoop token mechanism as well.
At the same time, there was talk of the possibility of simply making kerberos
easy and a non-issue for intra-cluster use. Certainly we need both of these
approaches.
I believe someone used ApacheDS' KDC support as an example - if we could
standup an ApacheDS based KDC and configure it and related keytabs easily than
the end-to-end story is more palatable to a broader user base. That story being
the choice of authentication mechanisms for user authentication and easy
provisioning and management of kerberos for intra-cluster service
authentication.
If you agree with this extended summary then I can update the other thread with
that recollection.
Thanks for providing it!
--larry
On Jul 4, 2013, at 4:09 PM, Alejandro Abdelnur <t...@cloudera.com> wrote:
> Leaving JIRAs and design docs aside, my recollection from the f2f lounge
> discussion could be summarized as:
>
> ------
> 1* Decouple users-services authentication from (intra) services-services
> authentication.
>
> The main motivation for this is to get pluggable authentication and
> integrated SSO experience for users.
>
> (we never discussed if this is needed for external-apps talking with Hadoop)
>
> 2* We should leave the Hadoop delegation tokens alone
>
> No need to make this pluggable as this is an internal authentication
> mechanism after the 'real' authentication happened.
>
> (this is independent from factoring out all classes we currently have into
> a common implementation for Hadoop and other projects to use)
>
> 3* Being able to replace kerberos with something else for (intra)
> services-services authentication.
>
> It was suggested that to support deployments where stock Kerberos may not
> be an option (i.e. cloud) we should make sure that UserGroupInformation and
> RPC security logic work with a pluggable GSS implementation.
>
> 4* Create a common security component ie 'hadoop-security' to be 'the'
> security lib for all projects to use.
>
> Create a component/project that would provide the common security pieces
> for all projects to use.
>
> ------
>
> If we agree with this, after any necessary corrections, I think we could
> distill clear goals from it and start from there.
>
> Thanks.
>
> Tucu & Alejandro
>
> On Thu, Jul 4, 2013 at 11:40 AM, Andrew Purtell <apurt...@apache.org> wrote:
>
>> Hi Larry (and all),
>>
>> Happy Fourth of July to you and yours.
>>
>> In our shop Kai and Tianyou are already doing the coding, so I'd defer to
>> them on the detailed points.
>>
>> My concern here is there may have been a misinterpretation or lack of
>> consensus on what is meant by "clean slate". Hopefully that can be quickly
>> cleared up. Certainly we did not mean ignore all that came before. The idea
>> was to reset discussions to find common ground and new direction where we
>> are working together, not in conflict, on an agreed upon set of design
>> points and tasks. There's been a lot of good discussion and design
>> preceeding that we should figure out how to port over. Nowhere in this
>> picture are self appointed "master JIRAs" and such, which have been
>> disappointing to see crop up, we should be collaboratively coding not
>> planting flags.
>>
>> I read Kai's latest document as something approaching today's consensus (or
>> at least a common point of view?) rather than a historical document.
>> Perhaps he and it can be given equal share of the consideration.
>>
>>
>> On Wednesday, July 3, 2013, Larry McCay wrote:
>>
>>> Hey Andrew -
>>>
>>> I largely agree with that statement.
>>> My intention was to let the differences be worked out within the
>>> individual components once they were identified and subtasks created.
>>>
>>> My reference to HSSO was really referring to a SSO *server* based design
>>> which was not clearly articulated in the earlier documents.
>>> We aren't trying to compare and contrast one design over another anymore.
>>>
>>> Let's move this collaboration along as we've mapped out and the
>>> differences in the details will reveal themselves and be addressed within
>>> their components.
>>>
>>> I've actually been looking forward to you weighing in on the actual
>>> discussion points in this thread.
>>> Could you do that?
>>>
>>> At this point, I am most interested in your thoughts on a single jira to
>>> represent all of this work and whether we should start discussing the SSO
>>> Tokens.
>>> If you think there are discussion points missing from that list, feel
>> free
>>> to add to it.
>>>
>>> thanks,
>>>
>>> --larry
>>>
>>> On Jul 3, 2013, at 7:35 PM, Andrew Purtell <apurt...@apache.org> wrote:
>>>
>>>> Hi Larry,
>>>>
>>>> Of course I'll let Kai speak for himself. However, let me point out
>> that,
>>>> while the differences between the competing JIRAs have been reduced for
>>>> sure, there were some key differences that didn't just disappear.
>>>> Subsequent discussion will make that clear. I also disagree with your
>>>> characterization that we have simply endorsed all of the design
>> decisions
>>>> of the so-called HSSO, this is taking a mile from an inch. We are here
>> to
>>>> engage in a collaborative process as peers. I've been encouraged by the
>>>> spirit of the discussions up to this point and hope that can continue
>>>> beyond one design summit.
>>>>
>>>>
>>>>
>>>> On Wed, Jul 3, 2013 at 1:10 PM, Larry McCay <lmc...@hortonworks.com>
>>> wrote:
>>>>
>>>>> Hi Kai -
>>>>>
>>>>> I think that I need to clarify something…
>>>>>
>>>>> This is not an update for 9533 but a continuation of the discussions
>>> that
>>>>> are focused on a fresh look at a SSO for Hadoop.
>>>>> We've agreed to leave our previous designs behind and therefore we
>>> aren't
>>>>> really seeing it as an HSSO layered on top of TAS approach or an HSSO
>> vs
>>>>> TAS discussion.
>>>>>
>>>>> Your latest design revision actually makes it clear that you are now
>>>>> targeting exactly what was described as HSSO - so comparing and
>>> contrasting
>>>>> is not going to add any value.
>>>>>
>>>>> What we need you to do at this point, is to look at those high-level
>>>>> components described on this thread and comment on whether we need
>>>>> additional components or any that are listed that don't seem necessary
>>> to
>>>>> you and why.
>>>>> In other words, we need to define and agree on the work that has to be
>>>>> done.
>>>>>
>>>>> We also need to determine those components that need to be done before
>>>>> anything else can be started.
>>>>> I happen to agree with Brian that #4 Hadoop SSO Tokens are central to
>>> all
>>>>> the other components and should probably be defined and POC'd in short
>>>>> order.
>>>>>
>>>>> Personally, I think that continuing the separation of 9533 and 9392
>> will
>>>>> do this effort a disservice. There doesn't seem to be enough
>> differences
>>>>> between the two to justify separate jiras anymore. It may be best to
>>> file a
>>>>> new one that reflects a single vision without the extra cruft that has
>>>>> built up in either of the existing ones. We would certainly reference
>>> the
>>>>> existing ones within the new one. This approach would align with the
>>> spirit
>>>>> of the discussions up to this point.
>>>>>
>>>>> I am prepared to start a discussion around the shape of the two Hadoop
>>> SSO
>>>>> tokens: identity and access. If this is what others feel the next
>> topic
>>>>> should be.
>>>>> If we can identify a jira home for it, we can do it there - otherwise
>> we
>>>>> can create another DISCUSS thread for it.
>>>>>
>>>>> thanks,
>>>>>
>>>>> --larry
>>>>>
>>>>>
>>>>> On Jul 3, 2013, at 2:39 PM, "Zheng, Kai" <kai.zh...@intel.com> wrote:
>>>>>
>>>>>> Hi Larry,
>>>>>>
>>>>>> Thanks for the update. Good to see that with this update we are now
>>>>> aligned on most points.
>>>>>>
>>>>>> I have also updated our TokenAuth design in HADOOP-9392. The new
>>>>> revision incorporates feedback and suggestions in related discussion
>>> with
>>>>> the community, particularly from Microsoft and others attending the
>>>>> Security design lounge session at the Hadoop summit. Summary of the
>>> changes:
>>>>>> 1. Revised the approach to now use two tokens, Identity Token plus
>>>>> Access Token, particularly considering our authorization framework and
>>>>> compatibility with HSSO;
>>>>>> 2. Introduced Authorization Server (AS) from our authorization
>>>>> framework into the flow that issues access tokens for clients with
>>> identity
>>>>> tokens to access services;
>>>>>> 3. Refined proxy access token and the proxy/impersonation flow;
>>>>>> 4. Refined the browser web SSO flow regarding access to Hadoop web
>>>>> services;
>>>>>> 5. Added Hadoop RPC access flow regard
>>
>>
>>
>> --
>> Best regards,
>>
>> - Andy
>>
>> Problems worthy of attack prove their worth by hitting back. - Piet Hein
>> (via Tom White)
>>
>
>
>
> --
> Alejandro
