I think I know what it probably is. The gateway-release module pom.xml is what actually defines the classpath of the final packaged server. If you take a look at that you will see it references all of the providers that are intended to be included by default. Try adding yours to the dependencies.
On 11/12/15, 7:49 AM, "larry mccay" <[email protected]> wrote: >We really need to fix this behavior... > >It seems you traded in your random authentication provider for a random >federation provider. :/ > >This is most likely due to not having your new provider on the classpath >for the serviceLoader to find. > >Just to try and articulate how it works... > >The deployment machinery loads all of the federation (whichever role) >providers contributors that is finds on the classpath - via the >src/main/resources/META-INF/services/org.apache.hadoop.gateway.deploy.ProviderDeploymentContributor >- and interrogates each for its name to match to the name in the topology. >Once it finds a match it allows the contributor to "contribute" to the >creation of the topology specific webapp. Part of which includes adding >filters to the gateway.xml. > >We currently have an issue wherein, given a name that doesn't match any >found on the classpath, a random provider is returned. This needs to be >fixed to fail deployment with appropriate error messages. > >Now, this usually comes down to one of 2 things: > >1. there is no >src/main/resources/META-INF/services/org.apache.hadoop.gateway.deploy.ProviderDeploymentContributor >file in the provider or it contains the classname of some other contributor >2. the provider itself isn't on the gateway classpath. I see that your >service file seems to be defined properly - so it is probably not on the >classpath. Did you add the new provider to the parent pom? The article >should provide all you need to do so. > >I will file a JIRA to fix the random provider issue. > >On Thu, Nov 12, 2015 at 7:18 AM, Jérôme LELEU <[email protected]> wrote: > >> Hi, >> >> By changing the role to the appropriate value, the error is gone, but I >> have a new problem: >> >> *Caused by: javax.servlet.ServletException: Required authentication >> provider URL is missing.* >> * at >> >> org.apache.hadoop.gateway.provider.federation.jwt.filter.SSOCookieFederationFilter.init(SSOCookieFederationFilter.java:90)* >> >> My sandbox.xml file only references the pac4j provider as well as the >> Default (role: identity-assertion) and static (role: hostmap) ones. >> >> Though, the SSOCookieFederationFilter is defined in the gateway.xml file of >> my sandbox deployment. Why do I need it? What's the expected authentication >> provider url? How this gatweay.xml is generated? >> >> Thanks. >> Best regards, >> Jérôme >> >> >> >> >> >> 2015-11-10 17:36 GMT+01:00 Kevin Minder <[email protected]>: >> >> > To be a bit more explicit your Pac4jFederationProviderContributor >> declares >> > itself to be of role “federation” but when you used it in the sandbox.xml >> > topology file you were attempting to configure it as an provider with >> role >> > authentication. >> > >> > >> > >> > >> > On 11/10/15, 11:32 AM, "larry mccay" <[email protected]> wrote: >> > >> > >Let's get you past the error that you are getting... >> > > >> > >It seems as though you have change the topology for pac4j but didn't >> > change >> > >the role to "federation". >> > >It looks like it is grabbing a random authentication provider and seems >> to >> > >be the hadoop-auth provider. >> > > >> > >Change that to federation and see if that helps. >> > > >> > >On Tue, Nov 10, 2015 at 11:21 AM, larry mccay <[email protected]> >> wrote: >> > > >> > >> Hi Jérôme - >> > >> >> > >> Happy to see you here! >> > >> I apologize for having missed your note on the list a few days ago. >> > >> >> > >> Glad to see that the article was helpful and it seems like you are >> > making >> > >> great progress. >> > >> >> > >> Let me dig into your note a bit deeper and answer your questions. >> > >> >> > >> Welcome! >> > >> >> > >> --larry >> > >> >> > >> On Tue, Nov 10, 2015 at 11:14 AM, Jérôme LELEU <[email protected]> >> > wrote: >> > >> >> > >>> Hi, >> > >>> >> > >>> Perfect timing ;-) I have started implemented the pac4j support in >> > Knox, >> > >>> it's a very basic attempt: >> https://github.com/apache/knox/pull/2/files >> > >>> >> > >>> This implementation reuses the pac4j implementation for J2E. I have a >> > >>> ProviderDeploymentContributorBase which registers two filters: >> > >>> - the first one is a dispatcher filter which uses the j2e-pac4j >> > >>> CallbackFilter on the /callabck url (it finishes the authentication >> > process >> > >>> after a successful authentication at some identity provider) and uses >> > the >> > >>> j2e-pac4j RequiresAuthenticationFilter otherwise (redirection to the >> > >>> identity provider to start the authentication when the user is not >> > >>> authenticated) >> > >>> - the second one is an identity adapter which gets the current >> > >>> authenticated user from the pac4j point of view and populates the J2E >> > >>> security context. >> > >>> >> > >>> >> > >>> * I have the following error: >> > >>> >> > >>> 2015-11-10 16:30:22,347 ERROR hadoop.gateway >> > >>> (GatewayFilter.java:doFilter(135)) - Gateway processing failed: >> > >>> javax.servlet.ServletException: javax.servlet.ServletException: >> > >>> Authentication type must be specified: simple|kerberos|<class> >> > >>> javax.servlet.ServletException: javax.servlet.ServletException: >> > >>> Authentication type must be specified: simple|kerberos|<class> >> > >>> at >> > >>> >> > >> org.apache.hadoop.gateway.GatewayFilter$Holder.getInstance(GatewayFilter.java:354) >> > >>> >> > >>> Notice I have changed the sandbox.xml file to use pac4j instead of >> > Shiro, >> > >>> maybe it's not the right way: >> > >>> >> > >> https://github.com/apache/knox/pull/2/files#diff-128ca80baa2dfd6c2d25de7b8160b9d9R24 >> > >>> >> > >>> Any idea of the problem? >> > >>> >> > >>> >> > >>> * Am I understanding webflows correctly? >> > >>> >> > >>> I use the default request to: curl -ivk " >> > >>> https://localhost:8443/gateway/sandbox/webhdfs/v1/tmp?op=LISTSTATUS"; >> > >>> >> > >>> It looks like a REST request, though I'm expecting the following >> > webflow: >> > >>> redirection of the user to the identity provider, successful login, >> > >>> redirection back to the /callback url of the gateway: do I get it >> > right? >> > >>> >> > >>> >> > >>> * The callback url >> > >>> >> > >>> I currently do a check expecting the last part of the url to end with >> > >>> /callback, but does the gateway will receive requests on this url? >> Do I >> > >>> have to do something special? >> > >>> >> > >>> It's highly recommended to only have one callback url as this url is >> > >>> defined on the identity provider side. >> > >>> >> > >>> >> > >>> * How to pass configuration? >> > >>> >> > >>> Currently, I hardcoded a FacebookClient for Facebook authentication, >> > but >> > >>> we should be able to pass the appropriate client like Facebook or >> SAML. >> > >>> Basically, we could do that using filter properties: facebook.key + >> > >>> facebook.secret means we use Facebook authentication with the >> > appropriate >> > >>> key and secret for example. Any recommendation? >> > >>> >> > >>> >> > >>> * Can I use the J2E session to store the requested protected url and >> > >>> other information? >> > >>> >> > >>> Or do I need to do something special? >> > >>> >> > >>> >> > >>> Thanks. >> > >>> Best regards, >> > >>> Jérôme >> > >>> >> > >>> 2015-11-10 17:03 GMT+01:00 Kevin Minder < >> [email protected] >> > >: >> > >>> >> > >>>> Hi Jerome, >> > >>>> This sounds very exciting and is just the sort of thing the open >> > >>>> architecture of Knox is intended to allow. Larry and I have looked >> at >> > >>>> pac4j a number of times thinking that it would be a good fit and >> would >> > >>>> provide a great feature set to the Hadoop community. >> > >>>> Kevin. >> > >>>> >> > >>>> >> > >>>> >> > >>>> On 11/6/15, 3:31 AM, "Jérôme LELEU" <[email protected]> wrote: >> > >>>> >> > >>>> >Hi, >> > >>>> > >> > >>>> >Let's open this private discussion on the Knox dev mailing list. >> > >>>> > >> > >>>> >I'm Jerome Leleu and the creator of pac4j ( >> > >>>> https://github.com/pac4j/pac4j), >> > >>>> >a security engine for Java with many implementations for J2E, Play, >> > >>>> Spring, >> > >>>> >Vertx, Ratpack... The idea is to offer something as powerful as >> > Spring >> > >>>> >Security, but a lot easier and for all Java frameworks / tools and >> > all >> > >>>> >authentication mechanisms. >> > >>>> > >> > >>>> >Two years ago, we had a discussion with Larry on how pac4j could be >> > >>>> used in >> > >>>> >Knox. >> > >>>> > >> > >>>> >Meanwhile, both projects have grown up and I'm back to try to see >> if >> > >>>> pac4j >> > >>>> >can help Knox in terms of security. >> > >>>> > >> > >>>> >----- >> > >>>> > >> > >>>> >In its latest version, pac4j can be used to build a full security >> > >>>> library, >> > >>>> >not only focused on delegated authentication (Facebook, SAML). So >> we >> > >>>> have >> > >>>> >the concepts of direct clients (a client is an authentication >> > >>>> mechanism): >> > >>>> >basic auth, header, token... and indirect clients: SAML, Facebook, >> > >>>> OpenID, >> > >>>> >CAS... >> > >>>> >It seems to be very close to the Authentication and Federation >> > Providers >> > >>>> >concepts from Knox. >> > >>>> > >> > >>>> >The provided article is great and as it's focused on authentication >> > >>>> with a >> > >>>> >real example, it's easier to start with it. >> > >>>> > >> > >>>> >Thanks for your answers. >> > >>>> > >> > >>>> >Like Shiro, pac4j has LDAP support and caching can be done. The >> only >> > >>>> >difference is that pac4j relies on ldaptive ( >> > http://www.ldaptive.org/). >> > >>>> But >> > >>>> >I think you're right: it would be better to start by implementing >> the >> > >>>> >Federation part which might be the most expected feature for Knox >> > users >> > >>>> (a >> > >>>> >bit like buji-pac4j: https://github.com/bujiio/buji-pac4j is for >> > >>>> Shiro) and >> > >>>> >see if it's worth the work to have an authentication provider with >> > >>>> pac4j in >> > >>>> >addition to the one of Shiro. >> > >>>> > >> > >>>> >I plan to start working on that next week. Do you accept pull >> > requests >> > >>>> on >> > >>>> >Github (it would make discussion easier on source code)? >> > >>>> > >> > >>>> >Any feedback will be appreciated. >> > >>>> > >> > >>>> >Thanks. >> > >>>> >Best regards, >> > >>>> >Jérôme >> > >>>> > >> > >>>> > >> > >>>> > >> > >>>> > >> > >>>> > >> > >>>> > >> > >>>> >2015-11-05 17:13 GMT+01:00 larry mccay: >> > >>>> > >> > >>>> >> Hi Jérôme - >> > >>>> >> >> > >>>> >> Contributions coming from the community are owned and maintained >> by >> > >>>> the >> > >>>> >> entire community. >> > >>>> >> You would certainly not be expected to do all work on it and at >> the >> > >>>> same >> > >>>> >> time any assistance and upgrade messages that you provided would >> be >> > >>>> awesome. >> > >>>> >> >> > >>>> >> As for an empty module for pac4j, maybe the following example >> > project >> > >>>> >> would be helpful to you: >> > >>>> >> >> > >>>> >> https://github.com/lmccay/gateway-provider-security-pseudo >> > >>>> >> >> > >>>> >> It is in support of an article to illustrate how to go about >> > adding a >> > >>>> >> authentication or federation provider - as you would be doing: >> > >>>> >> >> > >>>> >> >> > >>>> >> > >> https://github.com/lmccay/gateway-provider-security-pseudo/blob/master/AddingFederationProvider.md >> > >>>> >> >> > >>>> >> That may be easier to consume than the entire developers guide. >> > >>>> >> That said, it would be great if you could describe what makes the >> > dev >> > >>>> >> guide too much - if that is the case. >> > >>>> >> >> > >>>> >> I'll take a stab at your questions here but we should move more >> > >>>> detailed >> > >>>> >> discussions on the topic to the dev@ list in order to keep this >> in >> > >>>> the >> > >>>> >> open and provide insights to others: >> > >>>> >> >> > >>>> >> 1) What do I do with the authenticated user: where / how do I >> fill >> > >>>> this >> > >>>> >> identity in Knox? >> > >>>> >> >> > >>>> >> We normalize the authentication event into a Java subject to >> > >>>> represent the >> > >>>> >> user, groups and impersonated user as appropriate. >> > >>>> >> In cases - like shiro - where we don't have access to the actual >> > >>>> >> authenticating code and we need to normalize the provider >> specific >> > >>>> security >> > >>>> >> context, we add another filter. You can look at the shiro >> provider >> > >>>> for an >> > >>>> >> example. The code inside the simple example provider in the >> article >> > >>>> also >> > >>>> >> shows what is expected at least in terms of the PrimaryPrincipal. >> > >>>> >> >> > >>>> >> 2) How to handle errors: not authenticated, not authorized? >> > >>>> >> >> > >>>> >> We aren't intrusive on what should be done by a provider here. >> > >>>> >> Generally, authentication failures result in a 401 and >> > authorizations >> > >>>> in a >> > >>>> >> 403 and they are usually sent by the provider. >> > >>>> >> >> > >>>> >> 3) How to handle redirections to an external provider? >> > >>>> >> >> > >>>> >> Redirects are sent by the providers themselves with a 307 or 302. >> > >>>> >> You can look at the SSOCookieFederationFilter in the jwt module >> as >> > an >> > >>>> >> example if you like. >> > >>>> >> >> > >>>> >> 4) Should the pac4j filter also handle authorizations (pac4j can >> do >> > >>>> that)? >> > >>>> >> >> > >>>> >> This is an interesting question. >> > >>>> >> Knox provides a separate slot in the provider chain for >> > authorization. >> > >>>> >> It is there because of the ability for identity assertion >> providers >> > >>>> to map >> > >>>> >> principals in order to impersonate another user or map external >> > >>>> usernames >> > >>>> >> to those used inside of hadoop clusters. So, the authorization >> > >>>> decisions >> > >>>> >> are made based on those identities. >> > >>>> >> >> > >>>> >> If we can add the authorization filter as an authorization >> provider >> > >>>> rather >> > >>>> >> than part of the authentication provider than that might make >> > sense. >> > >>>> >> Although, the security context is normalized as Knox expects not >> as >> > >>>> pac4j >> > >>>> >> authorization probably does. Now, if there is real value in doing >> > so, >> > >>>> the >> > >>>> >> pac4j Subject adaptor could stuff additional context into the >> > Subject >> > >>>> or >> > >>>> >> the request that can be accessed later by the pac4j authorization >> > >>>> provider. >> > >>>> >> >> > >>>> >> So, it can be done - the question is whether there is compelling >> > >>>> reason to >> > >>>> >> do so. >> > >>>> >> >> > >>>> >> 5) How to test? >> > >>>> >> >> > >>>> >> This can be challenging especially for external authentications >> and >> > >>>> web >> > >>>> >> app flows. >> > >>>> >> Generally, unit testing as much as possible is required. >> > >>>> >> We would want to make sure that the security context >> normalization >> > >>>> happens >> > >>>> >> as expected, etc. >> > >>>> >> >> > >>>> >> Additional considerations: >> > >>>> >> >> > >>>> >> 1. Our Shiro provider does group lookup and caching of the >> > >>>> authentication >> > >>>> >> event for optimizing interactions with authentication servers, >> > LDAP, >> > >>>> AD, >> > >>>> >> etc. We wouldn't want to lose these. Doing so would be a >> > non-starter >> > >>>> for >> > >>>> >> most folks replacing what Shiro is used for today. >> > >>>> >> 2. It is probably best to concentrate on the gaps that pac4j can >> > fill >> > >>>> for >> > >>>> >> openid, oauth, etc for the initial contribution and if the >> usecases >> > >>>> handled >> > >>>> >> by Shiro can be done better, easier, simpler then we can >> consider a >> > >>>> >> migration path. >> > >>>> >> >> > >>>> >> BTW - we have a couple very interesting possibilities for strong >> > and >> > >>>> >> multi-factor authentication that would be enabled by OpenID >> > >>>> immediately. >> > >>>> >> >> > >>>> >> Hope this isn't too long a response and that it is helpful for >> you. >> > >>>> >> >> > >>>> >> Let's determine what an initial contribution would be for and >> > bring a >> > >>>> >> proposal to the dev@ list and/or file a jira for the >> integration. >> > >>>> >> This would be great for adding features and growing the >> community - >> > >>>> so we >> > >>>> >> are all for it! >> > >>>> >> >> > >>>> >> thanks, >> > >>>> >> >> > >>>> >> --larry >> > >>>> >> >> > >>>> >> >> > >>>> >> On Thu, Nov 5, 2015 at 9:43 AM, Jérôme LELEU <[email protected]> >> > >>>> wrote: >> > >>>> >> >> > >>>> >>> Hi, >> > >>>> >>> >> > >>>> >>> I see that Shiro is already used for basic auth and LDAP >> > >>>> authentication >> > >>>> >>> and Picketlink for SAML. pac4j v1.8 can now handle both cases >> and >> > >>>> even >> > >>>> >>> more. So I think we could create a >> gateway-provider-security-pac4j >> > >>>> >>> supporting all authentication mechanisms and not only OpenID. >> > >>>> >>> >> > >>>> >>> pac4j implementations generally work with two filters: one to >> > >>>> protect a >> > >>>> >>> resource and play direct authentication like basic auth (and >> check >> > >>>> >>> authorizations) and a callback filter to finish remote >> > >>>> authentication like >> > >>>> >>> Facebook, SAML, OpenID. The easiest one is the j2e-pac4j with: >> > >>>> >>> >> > >>>> >> > >> https://github.com/pac4j/j2e-pac4j/blob/master/src/main/java/org/pac4j/j2e/filter/RequiresAuthenticationFilter.java#L96 >> > >>>> >>> and >> > >>>> >>> >> > >>>> >> > >> https://github.com/pac4j/j2e-pac4j/blob/master/src/main/java/org/pac4j/j2e/filter/CallbackFilter.java#L65 >> > >>>> >>> >> > >>>> >>> That said, I have already many pac4j implementations to handle >> so >> > I'm >> > >>>> >>> pushing communities to maintain on their own their pac4j >> > >>>> implementations: >> > >>>> >>> can we imagine having an official: >> gateway-provider-security-pac4j >> > >>>> module >> > >>>> >>> maintained by the Knox community with my help of course? This is >> > the >> > >>>> kind >> > >>>> >>> of message I deliver to communities when a new version of pac4j >> is >> > >>>> >>> available so that they can upgrade: >> > >>>> >>> https://github.com/ratpack/ratpack/issues/819 >> > >>>> >>> >> > >>>> >>> I read: >> > >>>> https://knox.apache.org/books/knox-0-6-0/dev-guide.html#Providers, >> > >>>> >>> it might be easy for Knox contributors, but it's a bit hard for >> me >> > >>>> to get >> > >>>> >>> in everything: can you or someone in the Knox community provides >> > me >> > >>>> an >> > >>>> >>> empty gateway-provider-security-pac4j module and the default >> > >>>> expectations >> > >>>> >>> from a Knox point of view? >> > >>>> >>> My main questions: >> > >>>> >>> 1) What do I do with the authenticated user: where / how do I >> fill >> > >>>> this >> > >>>> >>> identity in Knox? >> > >>>> >>> 2) How to handle errors: not authenticated, not authorized? >> > >>>> >>> 3) How to handle redirections to an external provider? >> > >>>> >>> 4) Should the pac4j filter also handle authorizations (pac4j can >> > do >> > >>>> that)? >> > >>>> >>> 5) How to test? >> > >>>> >>> >> > >>>> >>> Thanks. >> > >>>> >>> Best regards, >> > >>>> >>> Jérôme >> > >>>> >>> >> > >>>> >>> >> > >>>> >>> >> > >>>> >>> >> > >>>> >>> 2015-11-03 16:58 GMT+01:00 larry mccay: >> > >>>> >>> >> > >>>> >>>> Hi Jérôme - >> > >>>> >>>> >> > >>>> >>>> We were evaluating buji-pac4j for SSO with SAML and ended up >> > going >> > >>>> with >> > >>>> >>>> Picketlink at the time. >> > >>>> >>>> That said, we do have a pluggable architecture that would allow >> > for >> > >>>> a >> > >>>> >>>> pac4j provider as well. >> > >>>> >>>> >> > >>>> >>>> If you are interested in contributing such a provider that >> would >> > >>>> >>>> certainly be welcome. >> > >>>> >>>> There is someone in the community working on an openid provider >> > but >> > >>>> >>>> perhaps pac4j would be the way to go there? >> > >>>> >>>> >> > >>>> >>>> thanks, >> > >>>> >>>> >> > >>>> >>>> --larry >> > >>>> >>>> >> > >>>> >>>> >> > >>>> >> > >>> >> > >>> >> > >> >> > >>
