Hello, I am sending this email to the mailing list, to get your help on a problem that I can't seem to resolve myself.
I am trying to secure Spark history ui running with Yarn as master using Apache Knox. >From the Knox configuration point of view I managed to secure the Spark >service, if I go on https://:8443/gateway/default/spark3history I have to >login using SSO then I get redirected to spark history server web ui which >works as expected. But if I directly access Spark without getting logged in I don't get redirected to Knox login page which is what I would like to have, same as HDFS and YarnUI. >From what I see in Spark documentation the webui needs to be protected using >the filter system. I can' t seem to find a filter to protect my Spark history >UI using Knox, I protected both HDFS and Yarn by adding this in core-site.xml >which works fine. < property > < name > hadoop.http.authentication.type </ name > < value > org.apache.hadoop.security.authentication.server.JWTRedirectAuthenticationHandler </ value > </ property > < property > < name > hadoop.http.authentication.authentication.provider.url </ name > < value > https:// < knox-hostname > :8443/gateway/knoxsso/api/v1/websso </ value > </ property > < property > < name > hadoop.http.authentication.public.key.pem </ name > < value > < token > </ value > </ property > Adding those properties allowed me to get redirected to knox host page when I didn' t login yet. I am wondering if you knew how to secure Spark history UI to have the same behavior. Do you know what configuration I am missing to redirect it back to the Knox gateway login page from the Spark history UI as for the other services where the JWT token is passed and used for keeping the user session ? I tried to play with the filters especially org.apache.hadoop.security.authentication.server.AuthenticationFilter but didn' t manage to get anything working, so I don' t even know if this is the right way to do. Thanks for your answer
