Hello again, Lets try to simplify the case and remove the login process with the redirection. Can KNOX be configured to act as a token bearer only client? So that no authentication is done from the user, just use a bearer only JWT token that will get validated from the Idp.
KR, Nick Vidiadakis On Tue, Oct 3, 2017 at 11:18 AM, N. Vidiadakis <[email protected]> wrote: > Hello to all again, > > This is what happens during the redirection process in the Idp (this is > the POST request with authentication), as I've recorded it using developer > tools of Chrome: > > Request URL:https://aegle-keycloak.exodussa.com/auth/realms/ > AEGLE/login-actions/authenticate?code=GURA9HtMgD3qLRMQ5VWB4pIyeVbhiZ > aTyE-FtmeGvIM.475fff9a-dce5-42ce-86d9-9433941f919d& > execution=7e0fa916-c58b-4a07-8200-f822793c5ecd > Request Method:POST > Status Code:302 Found > Remote Address:83.212.114.145:443 > Referrer Policy:no-referrer-when-downgrade > Response Headers > view parsed > HTTP/1.1 302 Found > Server: nginx/1.9.15 > Date: Tue, 03 Oct 2017 08:02:27 GMT > Content-Length: 0 > Connection: keep-alive > Cache-Control: no-store, must-revalidate, max-age=0 > X-Powered-By: Undertow/1 > Set-Cookie: KEYCLOAK_IDENTITY=eyJhbGciOiJSUzI1NiJ9. > eyJqdGkiOiIzOTkyMWM3Yi1lMzBiLTRjZGItOWM3MS1mZTljYTE1Y2NhZmQi > LCJleHAiOjE1MDcwNTM3NDcsIm5iZiI6MCwiaWF0IjoxNTA3MDE3NzQ3LCJp > c3MiOiJodHRwczovL2FlZ2xlLWtleWNsb2FrLmV4b2R1c3NhLmNvbS9hdXRo > L3JlYWxtcy9BRUdMRSIsInN1YiI6IjM3ODdhNDQyLTBiN2QtNGJlYi05YjRi > LWQzNTBjYTEwZDNhOCIsImF1dGhfdGltZSI6MCwic2Vzc2lvbl9zdGF0ZSI6 > ImIzOTNjNzVhLWI4MTktNDM0ZS1iYjRlLTUyODllM2NjZWQyNCIsInJlc291 > cmNlX2FjY2VzcyI6e319.UrefWhhVEQQoI5jVjd71IcK7-4P27MshA54K3o7PWpJ- > 9xzhD156fG1_r8L6cflTkeiSLyHU173SSIWkfo69Xi-aQOPVhtWF3DR9UCPnQu0Nk5OSXFi5g > _PwNCiUQsK_7bPVa8M5fNuxv6JbuoZB7jv-PFpsL4lO1gQs8kxPVW- > ig1KBcpJYuwaoxESBrq5pwov82_ipQijWUVC_5Pzui8eT58e-ubAZBuUZjyfyZXqYFdcgvaFw- > WOXEnZUubHUfOFKavlVwfBLAIDzIjGJJGMghVb7KjK4abx2mfQHl6H_ > qTbEqh1JyFHebuluRB99sYWk82pPzZ2AVP_ejF7Y8g; Version=1; > Path=/auth/realms/AEGLE; HttpOnly > Set-Cookie: KEYCLOAK_SESSION=AEGLE/3787a442-0b7d-4beb-9b4b- > d350ca10d3a8/b393c75a-b819-434e-bb4e-5289e3cced24; Version=1; > Expires=Tue, 03-Oct-2017 18:02:27 GMT; Max-Age=36000; > Path=/auth/realms/AEGLE > P3P: CP="This is not a P3P policy!" > Location: https://83.212.114.145:8443/gateway/knoxsso/api/v1/websso? > pac4jCallback=true&client_name=OidcClient&state= > ll1wYF2tjnjL4FQohTNdjz095OtXdYGUa8b4qsU9sE4&code= > GURA9HtMgD3qLRMQ5VWB4pIyeVbhiZaTyE-FtmeGvIM.475fff9a-dce5- > 42ce-86d9-9433941f919d > Request Headers > Accept:text/html,application/xhtml+xml,application/xml;q=0. > 9,image/webp,image/apng,*/*;q=0.8 > Accept-Encoding:gzip, deflate, br > Accept-Language:en-US,en;q=0.8,el;q=0.6 > Cache-Control:max-age=0 > Connection:keep-alive > Content-Length:52 > Content-Type:application/x-www-form-urlencoded > Cookie:KC_RESTART=eyJhbGciOiJIUzI1NiJ9.eyJjcyI6IjQ3NWZmZjlhLWRjZTUtND > JjZS04NmQ5LTk0MzM5NDFmOTE5ZCIsImNpZCI6ImFlZ2xlX2tub3giLCJwdH > kiOiJvcGVuaWQtY29ubmVjdCIsInJ1cmkiOiJodHRwczovLzgzLjIxMi4xMT > QuMTQ1Ojg0NDMvZ2F0ZXdheS9rbm94c3NvL2FwaS92MS93ZWJzc28_ > cGFjNGpDYWxsYmFjaz10cnVlJmNsaWVudF9uYW1lPU9pZGNDbGllbnQiLCJh > Y3QiOiJBVVRIRU5USUNBVEUiLCJub3RlcyI6eyJhY3Rpb25fa2V5IjoiN2Q3 > ZTVmNjQtZGVhZi00NDA3LTljZDktMGI2ZGVhOGM5MzQ1IiwiYXV0aF90eXBl > IjoiY29kZSIsInNjb3BlIjoib3BlbmlkIiwiaXNzIjoiaHR0cHM6Ly9hZWds > ZS1rZXljbG9hay5leG9kdXNzYS5jb20vYXV0aC9yZWFsbXMvQUVHTEUiLCJy > ZXNwb25zZV90eXBlIjoiY29kZSIsInJlZGlyZWN0X3VyaSI6Imh0dHBzOi8v > ODMuMjEyLjExNC4xNDU6ODQ0My9nYXRld2F5L2tub3hzc28vYXBpL3YxL3dl > YnNzbz9wYWM0akNhbGxiYWNrPXRydWUmY2xpZW50X25hbWU9T2lkY0NsaWVu > dCIsInN0YXRlIjoibGwxd1lGMnRqbmpMNEZRb2hUTmRqejA5NU90WGRZR1Vh > OGI0cXNVOXNFNCJ9fQ.3zHWytk5YI2Uz-Ugl5RslDa3NrzkgA8g2ToXRuzfOgU > Host:aegle-keycloak.exodussa.com > Origin:https://aegle-keycloak.exodussa.com > Referer:https://aegle-keycloak.exodussa.com/auth/ > realms/AEGLE/protocol/openid-connect/auth?response_type= > code&client_id=aegle_knox&redirect_uri=https%3A%2F%2F83. > 212.114.145%3A8443%2Fgateway%2Fknoxsso%2Fapi%2Fv1%2Fwebsso% > 3Fpac4jCallback%3Dtrue%26client_name%3DOidcClient&scope=openid&state= > ll1wYF2tjnjL4FQohTNdjz095OtXdYGUa8b4qsU9sE4 > Upgrade-Insecure-Requests:1 > User-Agent:Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_2) > AppleWebKit/537.36 (KHTML, like Gecko) Chrome/60.0.3112.90 Safari/537.36 > Query String Parameters > code:GURA9HtMgD3qLRMQ5VWB4pIyeVbhiZaTyE-FtmeGvIM.475fff9a-dce5- > 42ce-86d9-9433941f919d > execution:7e0fa916-c58b-4a07-8200-f822793c5ecd > Form Data > username:<something> > password:<password> > login:Log in > > and then, this is the final request to the actual resource I'm supposed to > be allowed access: > > Request URL:https://83.212.114.145:8443/gateway/knoxsso/api/v1/ > websso?pac4jCallback=true&client_name=OidcClient&state= > ll1wYF2tjnjL4FQohTNdjz095OtXdYGUa8b4qsU9sE4&code= > GURA9HtMgD3qLRMQ5VWB4pIyeVbhiZaTyE-FtmeGvIM.475fff9a-dce5- > 42ce-86d9-9433941f919d > Request Method:GET > Status Code:500 Server Error > Remote Address:83.212.114.145:8443 > Referrer Policy:no-referrer-when-downgrade > Response Headers > Cache-Control:must-revalidate,no-cache,no-store > Connection:close > Content-Length:319 > Content-Type:text/html; charset=ISO-8859-1 > Date:Tue, 03 Oct 2017 08:02:28 GMT > Server:Jetty(9.2.15.v20160210) > Set-Cookie:pac4j.session.OidcClient$attemptedAuthentication=; > Version=1;Domain=83.212.114.145;Secure;HttpOnly > Request Headers > Accept:text/html,application/xhtml+xml,application/xml;q=0. > 9,image/webp,image/apng,*/*;q=0.8 > Accept-Encoding:gzip, deflate, br > Accept-Language:en-US,en;q=0.8,el;q=0.6 > Cache-Control:max-age=0 > Connection:keep-alive > Cookie:pac4j.session.pac4jRequestedUrl=AAAACAAAABAAAACgwphJ1gNmSWXb5G > nxdXfVCI0HdCrc3TNYgXxJmec0n9qdmZ8poE3Jb9ND24b3l9Z7CJlrl6vmK7 > tlgQmW91lTGYE4PduRCNAyKfJ76zeIVejgKBrCD4vkVksx+ > 9F43gLxvCVihtZ7WlhmaOfeUunhp5whzZKVvUI8NtwIa66fkxfodRH485XzYQKoS0MaNOu/ > RQbwv28PDnCvaPhashKFuK5+/LQCX0KH+r/vNBFAyij+oJV/5D9T3g==; pac4j.session. > oidcStateAttribute=AAAACAAAABAAAADAwphJ1gNmSWXb5GnxdXfVCI0HdCrc3TNYnq+ > yXxGmEAkOvqi8ANmLzSDY8eHovwMpsei3ZH4m7GEONfq6NHgkeS0vG6blmpu > U9VEcUHVGAHs4vZzbV+V9DFN3NIT9gM66R/wWbMlbs4QkJOF3a8ZLNhvnzOczNAMv > rWra7mmLGfwM6QW4KF8eP4lbn1WME8K6QFeCUYK2huqneYddjaQYVX4pYQaS > 7MgPMvSrjwNimHU20JAIuWzorTALLc1vPqbfIcbBh6JKvDZ0/z7mILTzMTvaBpP5Nxhc > Host:83.212.114.145:8443 > Referer:https://aegle-keycloak.exodussa.com/auth/ > realms/AEGLE/protocol/openid-connect/auth?response_type= > code&client_id=aegle_knox&redirect_uri=https%3A%2F%2F83. > 212.114.145%3A8443%2Fgateway%2Fknoxsso%2Fapi%2Fv1%2Fwebsso% > 3Fpac4jCallback%3Dtrue%26client_name%3DOidcClient&scope=openid&state= > ll1wYF2tjnjL4FQohTNdjz095OtXdYGUa8b4qsU9sE4 > Upgrade-Insecure-Requests:1 > User-Agent:Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_2) > AppleWebKit/537.36 (KHTML, like Gecko) Chrome/60.0.3112.90 Safari/537.36 > Query String Parameters > pac4jCallback:true > client_name:OidcClient > state:ll1wYF2tjnjL4FQohTNdjz095OtXdYGUa8b4qsU9sE4 > code:GURA9HtMgD3qLRMQ5VWB4pIyeVbhiZaTyE-FtmeGvIM.475fff9a-dce5- > 42ce-86d9-9433941f919d > > I dont quite understand what should the proper header be like. > > KR, > Nick Vidiadakis > > > On Tue, Oct 3, 2017 at 8:22 AM, N. Vidiadakis <[email protected]> > wrote: > >> Hello to all, >> >> You say "you set something else in your metadata". You mean on my IDP >> (Keycloak) or can I configure something different on knoxsso topology? >> >> KR, >> Nick >> >> >> On Tue, Oct 3, 2017 at 6:00 AM, Jérôme LELEU <[email protected]> wrote: >> >>> Hi, >>> >>> Indeed, the issue occurs in the Nimbus SDK inside pac4j. >>> >>> The client authentication is built from your metadata configuration and >>> we only support *client_secret_post* and *client_secret_basic*. I guess >>> you set something else in your metadata, certainly private_key_jwt. >>> >>> From the spec: >>> >>> token_endpoint_auth_methods_supportedOPTIONAL. JSON array containing a >>> list of Client Authentication methods supported by this Token Endpoint. The >>> options are client_secret_post, client_secret_basic, client_secret_jwt, >>> and private_key_jwt, as described in Section 9 of OpenID Connect Core >>> 1.0 >>> <https://openid.net/specs/openid-connect-discovery-1_0.html#OpenID.Core> >>> [OpenID.Core]. >>> Other authentication methods MAY be defined by extensions. If omitted, the >>> default is client_secret_basic -- the HTTP Basic Authentication Scheme >>> specified in Section 2.3.1 of OAuth 2.0 >>> <https://openid.net/specs/openid-connect-discovery-1_0.html#RFC6749> >>> [RFC6749]. >>> >>> >>> Thanks. >>> Best regards, >>> Jérôme >>> >>> On Mon, Oct 2, 2017 at 9:49 PM, larry mccay <[email protected]> wrote: >>> >>>> Unfortunately, it seems that you will need to put a breakpoint in the >>>> code at org.apache.hadoop.gateway.pac4j.filter.Pac4jDispatcherFil >>>> ter.doFilter(Pac4jDispatcherFilter.java:205) and walk through - >>>> hopefully into the pac4j code and nimbus to see what is expected and not >>>> being found. >>>> >>>> Explicitly adding Jerome... >>>> >>>> @Jerome - does this error ring any bells for you? >>>> >>>> On Mon, Oct 2, 2017 at 3:09 PM, N. Vidiadakis <[email protected]> >>>> wrote: >>>> >>>>> I've done the modifications and unfortunately, I have the same results: >>>>> >>>>> 2017-10-02 19:06:10,559 ERROR hadoop.gateway >>>>> (AbstractGatewayFilter.java:doFilter(69)) - Failed to execute filter: >>>>> java.lang.IllegalArgumentException: The client authentication must >>>>> not be null >>>>> 2017-10-02 19:06:10,560 ERROR hadoop.gateway >>>>> (GatewayFilter.java:doFilter(146)) - Gateway processing failed: >>>>> javax.servlet.ServletException: java.lang.IllegalArgumentException: >>>>> The client authentication must not be null >>>>> javax.servlet.ServletException: java.lang.IllegalArgumentException: >>>>> The client authentication must not be null >>>>> at org.apache.hadoop.gateway.filter.AbstractGatewayFilter.doFil >>>>> ter(AbstractGatewayFilter.java:70) >>>>> at org.apache.hadoop.gateway.GatewayFilter$Holder.doFilter(Gate >>>>> wayFilter.java:346) >>>>> at org.apache.hadoop.gateway.GatewayFilter$Chain.doFilter(Gatew >>>>> ayFilter.java:246) >>>>> at org.apache.hadoop.gateway.GatewayFilter.doFilter(GatewayFilt >>>>> er.java:140) >>>>> ... >>>>> >>>>> KR, >>>>> Nick >>>>> >>>>> On Mon, Oct 2, 2017 at 9:57 PM, larry mccay <[email protected]> wrote: >>>>> >>>>>> Can you add the following after your discoveryUrl in the knoxsso.xml: >>>>>> >>>>>> <param> >>>>>> <name>oidc.useNonce</name> >>>>>> <value>false</value> >>>>>> </param> >>>>>> <param> >>>>>> <name>oidc.customParamKey1</name> 5. scope >>>>>> <value>scope</value> >>>>>> </param> >>>>>> <param> >>>>>> <name>oidc.customParamValue1</name> >>>>>> <value>openid</value> >>>>>> </param> >>>>>> >>>>>> In the testing that I did the the idp did not require the email and >>>>>> profile scopes that are requested by default by pac4j. Therefore, the >>>>>> customParam was being used here to limit the scopes to just openid. >>>>>> >>>>>> I happen to have the useNonce param in mine - so you might as well >>>>>> try that too. >>>>>> >>>>>> On Mon, Oct 2, 2017 at 2:49 PM, N. Vidiadakis <[email protected]> >>>>>> wrote: >>>>>> >>>>>>> Hi Larry, >>>>>>> >>>>>>> You can find attached the topologies and the stack trace. >>>>>>> >>>>>>> thank you in advance, >>>>>>> Nick >>>>>>> >>>>>>> >>>>>>> On Mon, Oct 2, 2017 at 9:34 PM, larry mccay <[email protected]> >>>>>>> wrote: >>>>>>> >>>>>>>> Hi Nick - >>>>>>>> >>>>>>>> Can you please provide your topologies that you are using for both >>>>>>>> sandbox.xml and knoxsso.xml? >>>>>>>> >>>>>>>> I have tested OIDC usecase before and would like to compare the >>>>>>>> configuration that you have - I did not try it against Keycloak but it >>>>>>>> should be generic OIDC. >>>>>>>> >>>>>>>> Also, can you provide the full stacktrace from the log? >>>>>>>> >>>>>>>> thanks, >>>>>>>> >>>>>>>> --larry >>>>>>>> >>>>>>>> On Mon, Oct 2, 2017 at 2:22 PM, N. Vidiadakis <[email protected] >>>>>>>> > wrote: >>>>>>>> >>>>>>>>> Hello to all, >>>>>>>>> >>>>>>>>> I'm relatively new to the whole Hadoop/KNOX ecosystem but I'm >>>>>>>>> appointed with relatively more complicated task: integrate KNOX with >>>>>>>>> an Idp >>>>>>>>> and specifically with a Keycloak installation which uses OpenID. >>>>>>>>> >>>>>>>>> I've tried following the User Guide and my current state is I get >>>>>>>>> redirected to the Keycloak Login portal, I enter my credentials and >>>>>>>>> then >>>>>>>>> get back to the KnoxSSO urls with an error 500. The log files contain: >>>>>>>>> >>>>>>>>> gateway.log: >>>>>>>>> >>>>>>>>> Caused by: java.lang.IllegalArgumentException: The client >>>>>>>>> authentication must not be null >>>>>>>>> at com.nimbusds.oauth2.sdk.TokenRequest.<init>(TokenRequest.jav >>>>>>>>> a:87) >>>>>>>>> at com.nimbusds.oauth2.sdk.TokenRequest.<init>(TokenRequest.jav >>>>>>>>> a:112) >>>>>>>>> >>>>>>>>> gateway-audit.log: >>>>>>>>> >>>>>>>>> 17/10/02 18:07:17 ||287109de-665e-469e-811e-8991 >>>>>>>>> 550b27e6|audit|91.138.248.128|WEBHDFS||||access|uri|/gateway >>>>>>>>> /sandbox/webhdfs/v1/?op=GETHOMEDIRECTORY|unavailable|Request >>>>>>>>> method: GET >>>>>>>>> 17/10/02 18:07:17 ||287109de-665e-469e-811e-8991 >>>>>>>>> 550b27e6|audit|91.138.248.128|WEBHDFS||||access|uri|/gateway >>>>>>>>> /sandbox/webhdfs/v1/?op=GETHOMEDIRECTORY|success|Response status: >>>>>>>>> 302 >>>>>>>>> 17/10/02 18:07:17 ||a17b49de-dcf6-4bf1-90b1-6f25 >>>>>>>>> 51e5380f|audit|91.138.248.128|KNOXSSO||||access|uri|/gateway >>>>>>>>> /knoxsso/api/v1/websso?originalUrl=https://83.212.114.145:84 >>>>>>>>> 43/gateway/sandbox/webhdfs/v1/?op=GETHOMEDIRECTORY|unavailab >>>>>>>>> le|Request method: GET >>>>>>>>> 17/10/02 18:07:17 ||a17b49de-dcf6-4bf1-90b1-6f25 >>>>>>>>> 51e5380f|audit|91.138.248.128|KNOXSSO||||access|uri|/gateway >>>>>>>>> /knoxsso/api/v1/websso?originalUrl=https://83.212.114.145:84 >>>>>>>>> 43/gateway/sandbox/webhdfs/v1/?op=GETHOMEDIRECTORY|success|R >>>>>>>>> esponse status: 302 >>>>>>>>> 17/10/02 18:07:17 ||0cef72c6-e010-4275-a309-6612 >>>>>>>>> 4e7a1cdb|audit|91.138.248.128|KNOXSSO||||access|uri|/gateway >>>>>>>>> /knoxsso/api/v1/websso?pac4jCallback=true&client_name=OidcCl >>>>>>>>> ient&state=8_-8Ni4pQynijY1ov26rNhXAYkWBWx10GyqJSnZHXYA&code= >>>>>>>>> dFHZBD2zpFbZYFLUArBdHaA1Nb_uEoDzHhULpehX7Sg.cbc5dae7-3532-4e >>>>>>>>> 56-a530-de1ea90b078a|unavailable|Request method: GET >>>>>>>>> 17/10/02 18:07:17 ||0cef72c6-e010-4275-a309-6612 >>>>>>>>> 4e7a1cdb|audit|91.138.248.128|KNOXSSO||||access|uri|/gateway >>>>>>>>> /knoxsso/api/v1/websso?pac4jCallback=true&client_name=OidcCl >>>>>>>>> ient&state=8_-8Ni4pQynijY1ov26rNhXAYkWBWx10GyqJSnZHXYA&code= >>>>>>>>> dFHZBD2zpFbZYFLUArBdHaA1Nb_uEoDzHhULpehX7Sg.cbc5dae7-3532-4e >>>>>>>>> 56-a530-de1ea90b078a|failure| >>>>>>>>> >>>>>>>>> Also, Keycloak does not report something out of the ordinary. >>>>>>>>> >>>>>>>>> My question is if and how to further debug this. I also wanted to >>>>>>>>> try a bearer-only configuration but the documentation is not clear >>>>>>>>> enough >>>>>>>>> for the configuration. >>>>>>>>> >>>>>>>>> Please. Help. >>>>>>>>> >>>>>>>>> KR, >>>>>>>>> Nick Vidiadakis >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>>> >>>> >>> >> >
