[ https://issues.apache.org/jira/browse/SPARK-32223?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17238617#comment-17238617 ]
Prashant Sharma commented on SPARK-32223: ----------------------------------------- Hi [~dongjoon], Do you think this is useful ? Any other thoughts on this? Thanks ! > Support adding a user provided config map. > ------------------------------------------ > > Key: SPARK-32223 > URL: https://issues.apache.org/jira/browse/SPARK-32223 > Project: Spark > Issue Type: Sub-task > Components: Kubernetes > Affects Versions: 3.1.0 > Reporter: Prashant Sharma > Priority: Major > > One of the challenge with this is, spark.properties is not user provided and > is calculated based on certain factors. So a user provided config map, cannot > be used as is to mount as SPARK_CONF_DIR, so it will have to be somehow > augmented with the correct spark.properties. > Q, Do we support update to config map properties for an already running job? > Ans: No, since the spark.properties is calculated at the time of job > submission, it cannot be updated on the fly and it is not supported by Spark > at the moment for all the configuration values. > Q. What are the usecases, where supplying SPARK_CONF_DIR via a config map > helps? > One of the use case, I can think of is programmatically submitting a spark on > k8s job - e.g. spark as a service on cloud deployment may find this feature > useful. -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org