Hello,
We were trying to enable spark-history UI authentication through keycloak.
>From the spark documentation we found out that we can use javax filters to
enable the UI authentication. Keycloak already provides a java
keycloak-servlet-filter-adapter which can be used.
I have added the following configuration in spark-defaults.conf
spark.ui.filters org.keycloak.adapters.servlet.KeycloakOIDCFilter
spark.org.keycloak.adapters.servlet.KeycloakOIDCFilter.param.keycloak.config.file
/home/ag/spark-2.4.0-bin-2.7.3/conf/keycloak.json
I was facing the below issue while running
java.lang.IllegalStateException: No SessionManager
at
org.spark_project.jetty.server.Request.getSession(Request.java:1544)
at
org.keycloak.adapters.servlet.FilterSessionStore.saveRequest(FilterSessionStore.java:374)
This was because none of the ServletContext in spark-history has
sessionManangment. I have made the below changes
1. Added Session id manager in JettyUtils.scala
server.setSessionIdManager(new HashSessionIdManager())
1. Added session handler for all context -
contextHandler.setSessionHandler(new SessionHandler())
in
* JettyUtils.scala - at createServletHandler, createStaticHandler and
createProxyHandler
* HistoryServer.scala - at initialize for /history context
* ApiRootResource.scala - at getServletHandler for /api context.
1. Placed required Keycloak runtime jars in spark class-path.
Keycloak authentication seems to work, Is this the right approach ? If it is
fine I can submit a PR.
@Vanzin I saw that you have done some refactoring in spark UI code, in
https://github.com/apache/spark/pull/23302 can you please suggest some inputs.
Thanks and Regards,
Ajay G
