Good catch. Will test with the right role... 2015-11-10 17:36 GMT+01:00 Kevin Minder <[email protected]>:
> To be a bit more explicit your Pac4jFederationProviderContributor declares > itself to be of role “federation” but when you used it in the sandbox.xml > topology file you were attempting to configure it as an provider with role > authentication. > > > > > On 11/10/15, 11:32 AM, "larry mccay" <[email protected]> wrote: > > >Let's get you past the error that you are getting... > > > >It seems as though you have change the topology for pac4j but didn't > change > >the role to "federation". > >It looks like it is grabbing a random authentication provider and seems to > >be the hadoop-auth provider. > > > >Change that to federation and see if that helps. > > > >On Tue, Nov 10, 2015 at 11:21 AM, larry mccay <[email protected]> wrote: > > > >> Hi Jérôme - > >> > >> Happy to see you here! > >> I apologize for having missed your note on the list a few days ago. > >> > >> Glad to see that the article was helpful and it seems like you are > making > >> great progress. > >> > >> Let me dig into your note a bit deeper and answer your questions. > >> > >> Welcome! > >> > >> --larry > >> > >> On Tue, Nov 10, 2015 at 11:14 AM, Jérôme LELEU <[email protected]> > wrote: > >> > >>> Hi, > >>> > >>> Perfect timing ;-) I have started implemented the pac4j support in > Knox, > >>> it's a very basic attempt: https://github.com/apache/knox/pull/2/files > >>> > >>> This implementation reuses the pac4j implementation for J2E. I have a > >>> ProviderDeploymentContributorBase which registers two filters: > >>> - the first one is a dispatcher filter which uses the j2e-pac4j > >>> CallbackFilter on the /callabck url (it finishes the authentication > process > >>> after a successful authentication at some identity provider) and uses > the > >>> j2e-pac4j RequiresAuthenticationFilter otherwise (redirection to the > >>> identity provider to start the authentication when the user is not > >>> authenticated) > >>> - the second one is an identity adapter which gets the current > >>> authenticated user from the pac4j point of view and populates the J2E > >>> security context. > >>> > >>> > >>> * I have the following error: > >>> > >>> 2015-11-10 16:30:22,347 ERROR hadoop.gateway > >>> (GatewayFilter.java:doFilter(135)) - Gateway processing failed: > >>> javax.servlet.ServletException: javax.servlet.ServletException: > >>> Authentication type must be specified: simple|kerberos|<class> > >>> javax.servlet.ServletException: javax.servlet.ServletException: > >>> Authentication type must be specified: simple|kerberos|<class> > >>> at > >>> > org.apache.hadoop.gateway.GatewayFilter$Holder.getInstance(GatewayFilter.java:354) > >>> > >>> Notice I have changed the sandbox.xml file to use pac4j instead of > Shiro, > >>> maybe it's not the right way: > >>> > https://github.com/apache/knox/pull/2/files#diff-128ca80baa2dfd6c2d25de7b8160b9d9R24 > >>> > >>> Any idea of the problem? > >>> > >>> > >>> * Am I understanding webflows correctly? > >>> > >>> I use the default request to: curl -ivk " > >>> https://localhost:8443/gateway/sandbox/webhdfs/v1/tmp?op=LISTSTATUS"; > >>> > >>> It looks like a REST request, though I'm expecting the following > webflow: > >>> redirection of the user to the identity provider, successful login, > >>> redirection back to the /callback url of the gateway: do I get it > right? > >>> > >>> > >>> * The callback url > >>> > >>> I currently do a check expecting the last part of the url to end with > >>> /callback, but does the gateway will receive requests on this url? Do I > >>> have to do something special? > >>> > >>> It's highly recommended to only have one callback url as this url is > >>> defined on the identity provider side. > >>> > >>> > >>> * How to pass configuration? > >>> > >>> Currently, I hardcoded a FacebookClient for Facebook authentication, > but > >>> we should be able to pass the appropriate client like Facebook or SAML. > >>> Basically, we could do that using filter properties: facebook.key + > >>> facebook.secret means we use Facebook authentication with the > appropriate > >>> key and secret for example. Any recommendation? > >>> > >>> > >>> * Can I use the J2E session to store the requested protected url and > >>> other information? > >>> > >>> Or do I need to do something special? > >>> > >>> > >>> Thanks. > >>> Best regards, > >>> Jérôme > >>> > >>> 2015-11-10 17:03 GMT+01:00 Kevin Minder <[email protected] > >: > >>> > >>>> Hi Jerome, > >>>> This sounds very exciting and is just the sort of thing the open > >>>> architecture of Knox is intended to allow. Larry and I have looked at > >>>> pac4j a number of times thinking that it would be a good fit and would > >>>> provide a great feature set to the Hadoop community. > >>>> Kevin. > >>>> > >>>> > >>>> > >>>> On 11/6/15, 3:31 AM, "Jérôme LELEU" <[email protected]> wrote: > >>>> > >>>> >Hi, > >>>> > > >>>> >Let's open this private discussion on the Knox dev mailing list. > >>>> > > >>>> >I'm Jerome Leleu and the creator of pac4j ( > >>>> https://github.com/pac4j/pac4j), > >>>> >a security engine for Java with many implementations for J2E, Play, > >>>> Spring, > >>>> >Vertx, Ratpack... The idea is to offer something as powerful as > Spring > >>>> >Security, but a lot easier and for all Java frameworks / tools and > all > >>>> >authentication mechanisms. > >>>> > > >>>> >Two years ago, we had a discussion with Larry on how pac4j could be > >>>> used in > >>>> >Knox. > >>>> > > >>>> >Meanwhile, both projects have grown up and I'm back to try to see if > >>>> pac4j > >>>> >can help Knox in terms of security. > >>>> > > >>>> >----- > >>>> > > >>>> >In its latest version, pac4j can be used to build a full security > >>>> library, > >>>> >not only focused on delegated authentication (Facebook, SAML). So we > >>>> have > >>>> >the concepts of direct clients (a client is an authentication > >>>> mechanism): > >>>> >basic auth, header, token... and indirect clients: SAML, Facebook, > >>>> OpenID, > >>>> >CAS... > >>>> >It seems to be very close to the Authentication and Federation > Providers > >>>> >concepts from Knox. > >>>> > > >>>> >The provided article is great and as it's focused on authentication > >>>> with a > >>>> >real example, it's easier to start with it. > >>>> > > >>>> >Thanks for your answers. > >>>> > > >>>> >Like Shiro, pac4j has LDAP support and caching can be done. The only > >>>> >difference is that pac4j relies on ldaptive ( > http://www.ldaptive.org/). > >>>> But > >>>> >I think you're right: it would be better to start by implementing the > >>>> >Federation part which might be the most expected feature for Knox > users > >>>> (a > >>>> >bit like buji-pac4j: https://github.com/bujiio/buji-pac4j is for > >>>> Shiro) and > >>>> >see if it's worth the work to have an authentication provider with > >>>> pac4j in > >>>> >addition to the one of Shiro. > >>>> > > >>>> >I plan to start working on that next week. Do you accept pull > requests > >>>> on > >>>> >Github (it would make discussion easier on source code)? > >>>> > > >>>> >Any feedback will be appreciated. > >>>> > > >>>> >Thanks. > >>>> >Best regards, > >>>> >Jérôme > >>>> > > >>>> > > >>>> > > >>>> > > >>>> > > >>>> > > >>>> >2015-11-05 17:13 GMT+01:00 larry mccay: > >>>> > > >>>> >> Hi Jérôme - > >>>> >> > >>>> >> Contributions coming from the community are owned and maintained by > >>>> the > >>>> >> entire community. > >>>> >> You would certainly not be expected to do all work on it and at the > >>>> same > >>>> >> time any assistance and upgrade messages that you provided would be > >>>> awesome. > >>>> >> > >>>> >> As for an empty module for pac4j, maybe the following example > project > >>>> >> would be helpful to you: > >>>> >> > >>>> >> https://github.com/lmccay/gateway-provider-security-pseudo > >>>> >> > >>>> >> It is in support of an article to illustrate how to go about > adding a > >>>> >> authentication or federation provider - as you would be doing: > >>>> >> > >>>> >> > >>>> > https://github.com/lmccay/gateway-provider-security-pseudo/blob/master/AddingFederationProvider.md > >>>> >> > >>>> >> That may be easier to consume than the entire developers guide. > >>>> >> That said, it would be great if you could describe what makes the > dev > >>>> >> guide too much - if that is the case. > >>>> >> > >>>> >> I'll take a stab at your questions here but we should move more > >>>> detailed > >>>> >> discussions on the topic to the dev@ list in order to keep this in > >>>> the > >>>> >> open and provide insights to others: > >>>> >> > >>>> >> 1) What do I do with the authenticated user: where / how do I fill > >>>> this > >>>> >> identity in Knox? > >>>> >> > >>>> >> We normalize the authentication event into a Java subject to > >>>> represent the > >>>> >> user, groups and impersonated user as appropriate. > >>>> >> In cases - like shiro - where we don't have access to the actual > >>>> >> authenticating code and we need to normalize the provider specific > >>>> security > >>>> >> context, we add another filter. You can look at the shiro provider > >>>> for an > >>>> >> example. The code inside the simple example provider in the article > >>>> also > >>>> >> shows what is expected at least in terms of the PrimaryPrincipal. > >>>> >> > >>>> >> 2) How to handle errors: not authenticated, not authorized? > >>>> >> > >>>> >> We aren't intrusive on what should be done by a provider here. > >>>> >> Generally, authentication failures result in a 401 and > authorizations > >>>> in a > >>>> >> 403 and they are usually sent by the provider. > >>>> >> > >>>> >> 3) How to handle redirections to an external provider? > >>>> >> > >>>> >> Redirects are sent by the providers themselves with a 307 or 302. > >>>> >> You can look at the SSOCookieFederationFilter in the jwt module as > an > >>>> >> example if you like. > >>>> >> > >>>> >> 4) Should the pac4j filter also handle authorizations (pac4j can do > >>>> that)? > >>>> >> > >>>> >> This is an interesting question. > >>>> >> Knox provides a separate slot in the provider chain for > authorization. > >>>> >> It is there because of the ability for identity assertion providers > >>>> to map > >>>> >> principals in order to impersonate another user or map external > >>>> usernames > >>>> >> to those used inside of hadoop clusters. So, the authorization > >>>> decisions > >>>> >> are made based on those identities. > >>>> >> > >>>> >> If we can add the authorization filter as an authorization provider > >>>> rather > >>>> >> than part of the authentication provider than that might make > sense. > >>>> >> Although, the security context is normalized as Knox expects not as > >>>> pac4j > >>>> >> authorization probably does. Now, if there is real value in doing > so, > >>>> the > >>>> >> pac4j Subject adaptor could stuff additional context into the > Subject > >>>> or > >>>> >> the request that can be accessed later by the pac4j authorization > >>>> provider. > >>>> >> > >>>> >> So, it can be done - the question is whether there is compelling > >>>> reason to > >>>> >> do so. > >>>> >> > >>>> >> 5) How to test? > >>>> >> > >>>> >> This can be challenging especially for external authentications and > >>>> web > >>>> >> app flows. > >>>> >> Generally, unit testing as much as possible is required. > >>>> >> We would want to make sure that the security context normalization > >>>> happens > >>>> >> as expected, etc. > >>>> >> > >>>> >> Additional considerations: > >>>> >> > >>>> >> 1. Our Shiro provider does group lookup and caching of the > >>>> authentication > >>>> >> event for optimizing interactions with authentication servers, > LDAP, > >>>> AD, > >>>> >> etc. We wouldn't want to lose these. Doing so would be a > non-starter > >>>> for > >>>> >> most folks replacing what Shiro is used for today. > >>>> >> 2. It is probably best to concentrate on the gaps that pac4j can > fill > >>>> for > >>>> >> openid, oauth, etc for the initial contribution and if the usecases > >>>> handled > >>>> >> by Shiro can be done better, easier, simpler then we can consider a > >>>> >> migration path. > >>>> >> > >>>> >> BTW - we have a couple very interesting possibilities for strong > and > >>>> >> multi-factor authentication that would be enabled by OpenID > >>>> immediately. > >>>> >> > >>>> >> Hope this isn't too long a response and that it is helpful for you. > >>>> >> > >>>> >> Let's determine what an initial contribution would be for and > bring a > >>>> >> proposal to the dev@ list and/or file a jira for the integration. > >>>> >> This would be great for adding features and growing the community - > >>>> so we > >>>> >> are all for it! > >>>> >> > >>>> >> thanks, > >>>> >> > >>>> >> --larry > >>>> >> > >>>> >> > >>>> >> On Thu, Nov 5, 2015 at 9:43 AM, Jérôme LELEU <[email protected]> > >>>> wrote: > >>>> >> > >>>> >>> Hi, > >>>> >>> > >>>> >>> I see that Shiro is already used for basic auth and LDAP > >>>> authentication > >>>> >>> and Picketlink for SAML. pac4j v1.8 can now handle both cases and > >>>> even > >>>> >>> more. So I think we could create a gateway-provider-security-pac4j > >>>> >>> supporting all authentication mechanisms and not only OpenID. > >>>> >>> > >>>> >>> pac4j implementations generally work with two filters: one to > >>>> protect a > >>>> >>> resource and play direct authentication like basic auth (and check > >>>> >>> authorizations) and a callback filter to finish remote > >>>> authentication like > >>>> >>> Facebook, SAML, OpenID. The easiest one is the j2e-pac4j with: > >>>> >>> > >>>> > https://github.com/pac4j/j2e-pac4j/blob/master/src/main/java/org/pac4j/j2e/filter/RequiresAuthenticationFilter.java#L96 > >>>> >>> and > >>>> >>> > >>>> > https://github.com/pac4j/j2e-pac4j/blob/master/src/main/java/org/pac4j/j2e/filter/CallbackFilter.java#L65 > >>>> >>> > >>>> >>> That said, I have already many pac4j implementations to handle so > I'm > >>>> >>> pushing communities to maintain on their own their pac4j > >>>> implementations: > >>>> >>> can we imagine having an official: gateway-provider-security-pac4j > >>>> module > >>>> >>> maintained by the Knox community with my help of course? This is > the > >>>> kind > >>>> >>> of message I deliver to communities when a new version of pac4j is > >>>> >>> available so that they can upgrade: > >>>> >>> https://github.com/ratpack/ratpack/issues/819 > >>>> >>> > >>>> >>> I read: > >>>> https://knox.apache.org/books/knox-0-6-0/dev-guide.html#Providers, > >>>> >>> it might be easy for Knox contributors, but it's a bit hard for me > >>>> to get > >>>> >>> in everything: can you or someone in the Knox community provides > me > >>>> an > >>>> >>> empty gateway-provider-security-pac4j module and the default > >>>> expectations > >>>> >>> from a Knox point of view? > >>>> >>> My main questions: > >>>> >>> 1) What do I do with the authenticated user: where / how do I fill > >>>> this > >>>> >>> identity in Knox? > >>>> >>> 2) How to handle errors: not authenticated, not authorized? > >>>> >>> 3) How to handle redirections to an external provider? > >>>> >>> 4) Should the pac4j filter also handle authorizations (pac4j can > do > >>>> that)? > >>>> >>> 5) How to test? > >>>> >>> > >>>> >>> Thanks. > >>>> >>> Best regards, > >>>> >>> Jérôme > >>>> >>> > >>>> >>> > >>>> >>> > >>>> >>> > >>>> >>> 2015-11-03 16:58 GMT+01:00 larry mccay: > >>>> >>> > >>>> >>>> Hi Jérôme - > >>>> >>>> > >>>> >>>> We were evaluating buji-pac4j for SSO with SAML and ended up > going > >>>> with > >>>> >>>> Picketlink at the time. > >>>> >>>> That said, we do have a pluggable architecture that would allow > for > >>>> a > >>>> >>>> pac4j provider as well. > >>>> >>>> > >>>> >>>> If you are interested in contributing such a provider that would > >>>> >>>> certainly be welcome. > >>>> >>>> There is someone in the community working on an openid provider > but > >>>> >>>> perhaps pac4j would be the way to go there? > >>>> >>>> > >>>> >>>> thanks, > >>>> >>>> > >>>> >>>> --larry > >>>> >>>> > >>>> >>>> > >>>> > >>> > >>> > >> >
