zjffdu commented on PR #4398: URL: https://github.com/apache/zeppelin/pull/4398#issuecomment-1188531385
@zlosim [SparkInterpreterLauncher](https://github.com/apache/zeppelin/blob/c4c580a37fde649553d336984a94bcb1b2821201/zeppelin-zengine/src/main/java/org/apache/zeppelin/interpreter/launcher/SparkInterpreterLauncher.java#L80) supports all the spark configurations. It detects all the spark configuration and concatenate them via env variable `ZEPPELIN_SPARK_CONF`. I think we can do similar thing in `K8sRemoteInterpreterProcess`. The reason why we deprecate `SPARK_SUBMIT_OPTIONS` is we would like to encourage user to configure spark in Zeppelin via a unified approach like spark (just configure each property one by one), otherwise it may cause inconsistency (e.g. Zeppelin admin may configure `spark.driver.memory` via `SPARK_SUBMIT_OPTIONS` in `zeppelin-env.sh`, but another user also define it via inline configuration, It is hard for Zeppelin to detect this kind of conflicts). So even for `ZEPPELIN_SPARK_CONF` in `SparkInterpreterLauncher`, it is for internal usage, it is not exposed to zeppelin users. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@zeppelin.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
