Spark-History ACLS

[G, Ajay (Nokia - IN/Bangalore)]

Hello,

I was trying spark-history acl security on spark-2.4. I have written a 
authentication filter which handles user authentication. This is the 
spark-config I have used.

spark.ui.filters com.ag.spark.AuthenticationFilter

spark.acls.enable true
spark.history.ui.acls.enable true
spark.history.ui.admin.acls ajay
spark.history.ui.admin.acls.groups ajay
spark.ui.view.acls ajay
spark.ui.view.acls.groups ajay


When ACLS is enabled all users (users who don't have view permission) can 
access  /api/v1/applications and when I hit any specific 
application-id only then securityManager.setAcl() is internally called.
Is this behaviour expected ? If yes, "Can this be documented in the user guide.


Thanks and Regards,
Ajay

Spark History UI + Keycloak Integration

[G, Ajay (Nokia - IN/Bangalore)]

Hello,

We were trying to enable spark-history UI authentication through keycloak. 
>From the spark documentation we found out that we can use javax filters to 
enable the UI authentication. Keycloak already provides a java 
keycloak-servlet-filter-adapter which can be used.

I have added the following configuration in spark-defaults.conf

spark.ui.filters org.keycloak.adapters.servlet.KeycloakOIDCFilter
spark.org.keycloak.adapters.servlet.KeycloakOIDCFilter.param.keycloak.config.file
 /home/ag/spark-2.4.0-bin-2.7.3/conf/keycloak.json

I was facing the below issue while running

java.lang.IllegalStateException: No SessionManager
   at 
org.spark_project.jetty.server.Request.getSession(Request.java:1544)
   at 
org.keycloak.adapters.servlet.FilterSessionStore.saveRequest(FilterSessionStore.java:374)


This was because none of the ServletContext in spark-history has 
sessionManangment. I have made the below changes

  1.  Added Session id manager in JettyUtils.scala
   server.setSessionIdManager(new HashSessionIdManager())


  1.  Added session handler for all context -   
contextHandler.setSessionHandler(new SessionHandler())

in

 *   JettyUtils.scala - at createServletHandler, createStaticHandler and 
createProxyHandler
 *   HistoryServer.scala - at initialize for /history context
 *   ApiRootResource.scala - at getServletHandler for /api context.


  1.  Placed required Keycloak runtime jars in spark class-path.


Keycloak authentication seems to work, Is this the right approach ? If it is 
fine I can submit a PR.
@Vanzin I saw that you have done some refactoring in spark UI code, in 
https://github.com/apache/spark/pull/23302 can you please suggest some inputs.

Thanks and Regards,
Ajay G