[ https://issues.apache.org/jira/browse/SPARK-32223?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17238872#comment-17238872 ]
Dongjoon Hyun commented on SPARK-32223: --------------------------------------- Is driver/executor template not enough for that purpose, [~prashant]? > Support adding a user provided config map. > ------------------------------------------ > > Key: SPARK-32223 > URL: https://issues.apache.org/jira/browse/SPARK-32223 > Project: Spark > Issue Type: Sub-task > Components: Kubernetes > Affects Versions: 3.1.0 > Reporter: Prashant Sharma > Priority: Major > > One of the challenge with this is, spark.properties is not user provided and > is calculated based on certain factors. So a user provided config map, cannot > be used as is to mount as SPARK_CONF_DIR, so it will have to be somehow > augmented with the correct spark.properties. > Q, Do we support update to config map properties for an already running job? > Ans: No, since the spark.properties is calculated at the time of job > submission, it cannot be updated on the fly and it is not supported by Spark > at the moment for all the configuration values. > Q. What are the usecases, where supplying SPARK_CONF_DIR via a config map > helps? > One of the use case, I can think of is programmatically submitting a `spark > on k8s` job - e.g. spark as a service on a cloud deployment may find this > feature useful. -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org