So this has nothing to do with Knox sadly. I banged my head against a similar problem a few weeks ago. I had wireshark captures from Knox setting the cookie and it not being available.
The issue is that *.compute.amazonaws.com does not allow cookies to be set. Here are some references that explain that browsers treat certain domains as not allowed to set cookies for. https://stackoverflow.com/questions/43520667/cookies-are-not-being-set-for-amazonaws-com-in-chrome-57-and-58-browsers https://en.wikipedia.org/wiki/Public_Suffix_List https://publicsuffix.org/list/public_suffix_list.dat So you need to have a custom domain instead of relying on the EC2 hostnames directly. Kevin Risden On Fri, May 24, 2019 at 2:00 AM Lokesh Lingarajan <[email protected]> wrote: > Hello all, > > I have been spending several sleepless nights on trying to get around this > infinite login loop for my zeppelin okta integration. > > Following are configurations under knox and zeppelin > > *Knox: knoxsso.xml* > > <?xml version="1.0" encoding="utf-8"?> > <topology> > <gateway> > <provider> > <role>webappsec</role> > <name>WebAppSec</name> > <enabled>true</enabled> > <param><name>xframe.options.enabled</name><value>true</value></param> > </provider> > <provider> > <role>federation</role> > <name>pac4j</name> > <enabled>true</enabled> > <param> > <name>pac4j.callbackUrl</name> > <value>https://**************. > us-west-2.compute.amazonaws.com:8446/gateway/knoxsso/api/v1/websso</value> > </param> > <param> > <name>pac4j.session.store</name> > <value>J2ESessionStore</value> > </param> > <param> > <name>clientName</name> > <value>OidcClient</value> > </param> > <param> > <name>oidc.id</name> > <value>****************</value> > </param> > <param> > <name>oidc.secret</name> > <value>*************************************</value> > </param> > <param> > <name>oidc.preferredJwsAlgorithm</name> > <value>RS256</value> > </param> > <param> > <name>pac4j.id_attribute</name> > <value>email</value> > </param> > <param> > <name>oidc.discoveryUri</name> > <value>https:// > ************************/oauth2/default/.well-known/openid-configuration</value> > </param> > </provider> > <provider> > <role>identity-assertion</role> > <name>Default</name> > <enabled>true</enabled> > <param> > <name>principal.mapping</name> > <value>*=user1;</value> > </param> > <param> > <name>group.principal.mapping</name> > <value>*=role1</value> > </param> > <param> > <name>input</name> > <value>(.*)\.(.*)@example\.com</value> > </param> > <param> > <name>output</name> > <value>{1}_{2}</value> > </param> > </provider> > <provider> > <role>identity-assertion</role> > <name>HadoopGroupProvider</name> > <enabled>true</enabled> > <param> > <name>hadoop.security.group.mapping</name> > <value>org.apache.hadoop.security.ShellBasedUnixGroupsMapping</value> > </param> > </provider> > </gateway> > <application> > <name>knoxauth</name> > </application> > <service> > <role>KNOXSSO</role> > <param> > <name>knoxsso.cookie.secure.only</name> > <value>false</value> > </param> > <param> > <name>knoxsso.token.ttl</name> > <value>3600000</value> > </param> > <param> > <name>knoxsso.redirect.whitelist.regex</name> > > > <value>^https?:\/\/(localhost|127\.0\.0\.1|0:0:0:0:0:0:0:1|::1|********************.us\-west\-2\.compute\.amazonaws\.com):[0-9].*$</value> > </param> > </service> > </topology> > > *Knox: sandbox.xml* > > <?xml version="1.0" encoding="utf-8"?> > <topology> > <gateway> > <provider> > <role>webappsec</role> > <name>WebAppSec</name> > <enabled>true</enabled> > <param> > <name>cors.enabled</name> > <value>true</value> > </param> > </provider> > <provider> > <role>federation</role> > <name>SSOCookieProvider</name> > <enabled>true</enabled> > <param> > <name>sso.authentication.provider.url</name> > <value>https://**********************. > us-west-2.compute.amazonaws.com:8446/gateway/knoxsso/api/v1/websso</value> > </param> > </provider> > </gateway> > </topology> > > > *Zeppelin: shiro.ini* > [main] > ## A same for configuring Knox SSO Realm > knoxJwtRealm = org.apache.zeppelin.realm.jwt.KnoxJwtRealm > knoxJwtRealm.providerUrl = > https://ec2-34-221-96-42.us-west-2.compute.amazonaws.com:8446/ > knoxJwtRealm.login = gateway/knoxsso/api/v1/websso > knoxJwtRealm.logout = gateway/knoxssout/api/v1/webssout > knoxJwtRealm.logoutAPI = true > knoxJwtRealm.redirectParam = originalUrl > knoxJwtRealm.cookieName = hadoop-jwt > knoxJwtRealm.publicKeyPath = /etc/zeppelin/conf.dist/knox-sso.pem > #knoxJwtRealm.groupPrincipalMapping = group.principal.mapping > #knoxJwtRealm.principalMapping = principal.mapping > authc = org.apache.zeppelin.realm.jwt.KnoxAuthenticationFilter > > ##########sessionManager = > org.apache.shiro.web.session.mgt.DefaultWebSessionManager > > ### If caching of user is required then uncomment below lines > #cacheManager = org.apache.shiro.cache.MemoryConstrainedCacheManager > #securityManager.cacheManager = $cacheManager > > ### Enables 'HttpOnly' flag in Zeppelin cookies > ##########cookie = org.apache.shiro.web.servlet.SimpleCookie > ##########cookie.name = JSESSIONID > #cookie.name = hadoop-jwt > #cookie.httpOnly = true > ### Uncomment the below line only when Zeppelin is running over HTTPS > ##########cookie.secure = true > ##########sessionManager.sessionIdCookie = $cookie > > ##########securityManager.sessionManager = $sessionManager > # 86,400,000 milliseconds = 24 hour > ##########securityManager.sessionManager.globalSessionTimeout = 86400000 > shiro.loginUrl = /api/login > > [roles] > role1 = * > > [urls] > /api/version = anon > # Allow all authenticated users to restart interpreters on a notebook page. > # Comment out the following line if you would like to authorize only admin > users to restart interpreters. > /api/interpreter/setting/restart/** = authc > /api/interpreter/** = authc, roles[admin] > /api/configurations/** = authc, roles[admin] > /api/credential/** = authc, roles[admin] > #/** = anon > /** = authc > > > *Knox gateway logs:* > 2019-05-21 17:40:55,161 INFO knox.gateway > (GatewayServer.java:startGateway(330)) - Started gateway on port 8,446. > 2019-05-21 17:41:53,240 WARN service.knoxsso > (WebSSOResource.java:handleCookieSetup(138)) - The SSO cookie SecureOnly > flag is set to FALSE and is therefore insecure. > 2019-05-21 17:41:53,247 INFO service.knoxsso > (WebSSOResource.java:getCookieValue(380)) - Unable to find cookie with > name: original-url > 2019-05-21 17:41:53,261 INFO service.knoxsso > (WebSSOResource.java:addJWTHadoopCookie(354)) - JWT cookie successfully > added. > 2019-05-21 17:41:53,262 INFO service.knoxsso > (WebSSOResource.java:getAuthenticationToken(256)) - About to redirect to > original URL: > https://ec2-34-221-96-42.us-west-2.compute.amazonaws.com:8445/ > 2019-05-21 17:41:55,591 WARN service.knoxsso > (WebSSOResource.java:handleCookieSetup(138)) - The SSO cookie SecureOnly > flag is set to FALSE and is therefore insecure. > 2019-05-21 17:41:55,596 INFO service.knoxsso > (WebSSOResource.java:getCookieValue(380)) - Unable to find cookie with > name: original-url > 2019-05-21 17:41:55,601 INFO service.knoxsso > (WebSSOResource.java:addJWTHadoopCookie(354)) - JWT cookie successfully > added. > 2019-05-21 17:41:55,602 INFO service.knoxsso > (WebSSOResource.java:getAuthenticationToken(256)) - About to redirect to > original URL: > https://ec2-34-221-96-42.us-west-2.compute.amazonaws.com:8445/ > 2019-05-21 17:41:57,280 WARN service.knoxsso > (WebSSOResource.java:handleCookieSetup(138)) - The SSO cookie SecureOnly > flag is set to FALSE and is therefore insecure. > 2019-05-21 17:41:57,281 INFO service.knoxsso > (WebSSOResource.java:getCookieValue(380)) - Unable to find cookie with > name: original-url > 2019-05-21 17:41:57,288 INFO service.knoxsso > (WebSSOResource.java:addJWTHadoopCookie(354)) - JWT cookie successfully > added. > 2019-05-21 17:41:57,288 INFO service.knoxsso > (WebSSOResource.java:getAuthenticationToken(256)) - About to redirect to > original URL: > https://ec2-34-221-96-42.us-west-2.compute.amazonaws.com:8445/ > 2019-05-21 17:41:58,976 WARN service.knoxsso > (WebSSOResource.java:handleCookieSetup(138)) - The SSO cookie SecureOnly > flag is set to FALSE and is therefore insecure. > 2019-05-21 17:41:58,977 INFO service.knoxsso > (WebSSOResource.java:getCookieValue(380)) - Unable to find cookie with > name: original-url > 2019-05-21 17:41:58,982 INFO service.knoxsso > (WebSSOResource.java:addJWTHadoopCookie(354)) - JWT cookie successfully > added. > > *Zeppelin logs:* > INFO [2019-05-21 17:42:02,456] ({qtp1766724936-14} > NotebookServer.java[onClose]:371) - Closed connection to 192.146.154.3 : > 62624. (1001) null > WARN [2019-05-21 17:42:02,456] ({qtp1766724936-14} > NotebookServer.java[removeUserConnection]:382) - Closing connection that is > absent in user connections > INFO [2019-05-21 17:42:03,411] ({qtp1766724936-14} > NotebookServer.java[onOpen]:150) - New connection from 192.146.154.3 : 62411 > INFO [2019-05-21 17:42:04,035] ({qtp1766724936-12} > NotebookServer.java[onClose]:371) - Closed connection to 192.146.154.3 : > 62411. (1001) null > WARN [2019-05-21 17:42:04,036] ({qtp1766724936-12} > NotebookServer.java[removeUserConnection]:382) - Closing connection that is > absent in user connections > INFO [2019-05-21 17:42:05,171] ({qtp1766724936-12} > NotebookServer.java[onOpen]:150) - New connection from 192.146.154.3 : 32535 > INFO [2019-05-21 17:42:05,570] ({qtp1766724936-12} > NotebookServer.java[onClose]:371) - Closed connection to 192.146.154.3 : > 32535. (1001) null > WARN [2019-05-21 17:42:05,571] ({qtp1766724936-12} > NotebookServer.java[removeUserConnection]:382) - Closing connection that is > absent in user connections > INFO [2019-05-21 17:42:06,682] ({qtp1766724936-13} > NotebookServer.java[onOpen]:150) - New connection from 192.146.154.3 : 4085 > > > *Following is the debug logs from knox with pac4j* > > 2019-05-23 05:22:33,826 INFO knox.gateway > (GatewayServer.java:startGateway(330)) - Started gateway on port 8,446. > 2019-05-23 05:23:35,028 DEBUG knox.gateway > (GatewayFilter.java:doFilter(116)) - Received request: GET /api/v1/websso > 2019-05-23 05:24:01,363 DEBUG knox.gateway > (GatewayFilter.java:doFilter(116)) - Received request: GET /api/v1/websso > 2019-05-23 05:24:02,796 DEBUG knox.gateway > (GatewayFilter.java:doFilter(116)) - Received request: GET /api/v1/websso > 2019-05-23 05:24:02,811 DEBUG filter.Pac4jIdentityAdapter > (Pac4jIdentityAdapter.java:doFilter(90)) - User authenticated as: > #OidcProfile# | id: 00ukun6wy78g1pIKG0h7 | attributes: > {at_hash=-EfHIlaKHxFyZE7gU9Im5g, sub=00ukun6wy78g1pIKG0h7, > zoneinfo=America/Los_Angeles, ver=1, email_verified=true, > amr=["pwd","mfa","swk"], > id_token=eyJraWQiOiIzdEFnVEpLRTgxR2l5Sm82M1A0dXpSdWR6RVpSNkdvS05ySDRHYkVUUTY0IiwiYWxnIjoiUlMyNTYifQ.eyJzdWIiOiIwMHVrdW42d3k3OGcxcElLRzBoNyIsIm5hbWUiOiJMb2tlc2ggTGluZ2FyYWphbiIsImVtYWlsIjoibG9rZXNoLmxpbmdhcmFqYW5AbnV0YW5peC5jb20iLCJ2ZXIiOjEsImlzcyI6Imh0dHBzOi8vbnV0YW5peGRldi5va3RhcHJldmlldy5jb20vb2F1dGgyL2RlZmF1bHQiLCJhdWQiOiIwb2FrdWYwbGUyVXZzUHVwQjBoNyIsImlhdCI6MTU1ODU4OTA0MSwiZXhwIjoxNTU4NTkyNjQxLCJqdGkiOiJJRC5EUjFDdFlJZUNNUWlwWFMwTm1rSl9qUnFuQUlVS2xudUZINEtSblRYOTBRIiwiYW1yIjpbInB3ZCIsIm1mYSIsInN3ayJdLCJpZHAiOiIwMG85ZHlwbmRpbHMzZW5qYjBoNyIsInByZWZlcnJlZF91c2VybmFtZSI6Imxva2VzaC5saW5nYXJhamFuQG51dGFuaXguY29tIiwiYXV0aF90aW1lIjoxNTU4NTg5MDMwLCJhdF9oYXNoIjoiLUVmSElsYUtIeEZ5WkU3Z1U5SW01ZyIsImdyb3VwcyI6WyJFdmVyeW9uZSJdfQ.ectAJZIzCEi1g8eR28VYOl6kt8FNLLDCX1KWQjv0N4aWqirQvI1IxTMb1RO4Bud0zmhnBXpb3kjUodTyd3KtQ6dDKmyqU0aWkYFyBQ7XXn-BM5w1PY-yPfII5z4dWE6-zHpev_AuNeBbjiuHRCEgo_bLkPZorknj1g3jJOiDJ_ympS-n5mF2snQbnjA4Wazr25au8o8QpxPQENMHhsBRWlQqmehNu3mfsMxP_7cjOufz2Xt58N4ndcmnKfYbqbc7qSwHOeJTJV8V1wTADUpQ6npy3Zjsvv6vypdpswnLEpfnJG2nNVu-r_W5bS2jG-s5kvogJ8nN_ihke37cpdbeXQ, > iss=https://********************.com/oauth2/default, groups=["Everyone"], > preferred_username=************@********.com, locale=us, > given_name=********, > access_token=eyJraWQiOiIzdEFnVEpLRTgxR2l5Sm82M1A0dXpSdWR6RVpSNkdvS05ySDRHYkVUUTY0IiwiYWxnIjoiUlMyNTYifQ.eyJ2ZXIiOjEsImp0aSI6IkFULmp5QndhNENFUkswMS0zOGVxYlJkSUFXZWFWcklRZFNEa0ZuRG53cjNka3ciLCJpc3MiOiJodHRwczovL251dGFuaXhkZXYub2t0YXByZXZpZXcuY29tL29hdXRoMi9kZWZhdWx0IiwiYXVkIjoiYXBpOi8vZGVmYXVsdCIsImlhdCI6MTU1ODU4OTA0MSwiZXhwIjoxNTU4NTkyNjQxLCJjaWQiOiIwb2FrdWYwbGUyVXZzUHVwQjBoNyIsInVpZCI6IjAwdWt1bjZ3eTc4ZzFwSUtHMGg3Iiwic2NwIjpbIm9wZW5pZCIsInByb2ZpbGUiLCJlbWFpbCJdLCJzdWIiOiJsb2tlc2gubGluZ2FyYWphbkBudXRhbml4LmNvbSIsImdyb3VwcyI6WyJFdmVyeW9uZSJdfQ.M9A8xtYr8dA8QqHD6PR5MkjCrrmC5twtW6xN8oP0TJGB0sHXOmqvTolivbGygQ__b1Hb_dOlnOT8m1jd8xoD8iLdFk58BAzuscmQ50uWcnYrqQ-gpx9qqS8uJmuVbAh2uNdk4lxdvBa62cg6QNt-K0JtNx1D1aw0i9azr_gUvtbsoH_BbCjVD6yW9vq7mDio4OqMyJ0Mpj5RLV4kxwoeTnc_gk-D6u03T6j6mmFRwOoWBk5Kpv0WMNbu5IvOb3TzU7pOjDWibu2_07u_IK6s64760jCQMOwKvrQiXeLXY_R7-LQ_eLnwruNQtdTpjXtlHNutz0c4_Lvp3iHcVRjlrA, > token_expiration_advance=-1, aud=[0oakuf0le2UvsPupB0h7], updated_at=Wed May > 15 06:19:16 UTC 2019, idp=00o9dypndils3enjb0h7, auth_time=Thu May 23 > 05:23:50 UTC 2019, name=*************, exp=Thu May 23 06:24:01 UTC 2019, > family_name=***********, iat=Thu May 23 05:24:01 UTC 2019, > email=*********@********.com, > jti=ID.DR1CtYIeCMQipXS0NmkJ_jRqnAIUKlnuFH4KRnTX90Q} | roles: [] | > permissions: [] | isRemembered: false | clientName: OidcClient | linkedId: > null | > 2019-05-23 05:24:03,983 WARN service.knoxsso > (WebSSOResource.java:handleCookieSetup(138)) - The SSO cookie SecureOnly > flag is set to FALSE and is therefore insecure. > 2019-05-23 05:24:03,990 INFO service.knoxsso > (WebSSOResource.java:getCookieValue(380)) - Unable to find cookie with > name: original-url > 2019-05-23 05:24:04,002 DEBUG service.knoxsso > (WebSSOResource.java:addJWTHadoopCookie(338)) - Adding the following JWT > token as a cookie: > eyJhbGciOiJSUzI1NiJ9.eyJzdWIiOiJsb2tlc2gubGluZ2FyYWphbkBudXRhbml4LmNvbSIsImlzcyI6IktOT1hTU08iLCJleHAiOjE1NTg1OTI2NDN9.VLdgbSz59QBrdUZTyt3jWHB3foIJf4G898e_Gc5VgFI3tx8BA8NRFGaQEdEF0BWhGLMr4PsXtSbq6JG1YXj_nqG1HgdlB1SGCABFbFCjj6r_llPwVSbvmW7qS5Ga4IBsAqDhd8hg-XrJ5WQ-rbg5YA8_XaPP1Fpl7anvzoGwhVU > 2019-05-23 05:24:04,004 INFO service.knoxsso > (WebSSOResource.java:addJWTHadoopCookie(354)) - JWT cookie successfully > added. > 2019-05-23 05:24:04,004 INFO service.knoxsso > (WebSSOResource.java:getAuthenticationToken(256)) - About to redirect to > original URL: > https://ec2-34-221-96-42.us-west-2.compute.amazonaws.com:8445/ > 2019-05-23 05:24:06,412 DEBUG knox.gateway > (GatewayFilter.java:doFilter(116)) - Received request: GET /api/v1/websso > 2019-05-23 05:24:06,706 DEBUG knox.gateway > (GatewayFilter.java:doFilter(116)) - Received request: GET /api/v1/websso > > > > > From inspect https transactions on chrome, I see all of the transactions > in oidc going fine until knox is returning with hadoop-jwt for the final > redirect. > But the final call to Zeppelin url request headers DO NOT contain this > hadoop-jwt cookie. > > > I am very new to SSO and zeppelin and need help in finding out what is > that I am missing here fundamentally > in getting the last leg of the integration working. > > Any help here will be greatly appreciated. > > -- > Regards > Lokesh >
