zlosim commented on PR #4398: URL: https://github.com/apache/zeppelin/pull/4398#issuecomment-1188746965
Hi @zjffdu thanks for the response. To be honest, I think this fix is doing exactly what you are saying you want to achieve. Right now setting e.g. [spark dependencies via zeppelin interpreter settings](https://i.imgur.com/VeSvIFa.png) is not working in K8s because we are trying to set it in runtime which is too late as driver is already started. This PR picks settings from interpreter settings passed by user that have to be passed directly to spark submit command when creating k8s pod with driver and puts them in env var so interpreter.sh script will start driver correctly. For end user this is what we are trying to achieve - he sets his settings via zeppelin interepreter settings and everything works as expected. As for `SPARK_SUBMIT_OPTIONS` and possibility of zeppelin admin alter configs - [we can alter the append and ignore value](https://github.com/apache/zeppelin/blob/ed251f4b48215f372f437f4490d7f07680793b7b/zeppelin-plugins/launcher/k8s-standard/src/main/java/org/apache/zeppelin/interpreter/launcher/K8sRemoteInterpreterProcess.java#L324) effectively leaving `SPARK_SUBMIT_OPTIONS` unexposed to zeppelin users - same as `ZEPPELIN_SPARK_CONF`. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@zeppelin.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
