[
https://issues.apache.org/jira/browse/SPARK-24434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16497009#comment-16497009
]
Anirudh Ramanathan edited comment on SPARK-24434 at 5/31/18 6:54 PM:
---------------------------------------------------------------------
I was basing my suggestion of JSON on allowing specifying JSON strings inline
as configuration, but I guess one could also specify a YAML file with the
template and have spark configuration point to that file. [~skonto], you make a
good point, it is another configuration mechanism that people may have to
learn. This decision should be based more on UX and consistency with what Spark
users expect in general. [~eje], to your point, I think we could support both
if needed, but it might be prudent to find the one that's more intuitive to
users in order to do first.
Sidenote: There's also [jsonpath|https://kubernetes.io/docs/reference/kubectl/]
that kubectl supports but that could be overkill here.
was (Author: foxish):
Good point. I was basing my suggestion of JSON on allowing specifying JSON
strings inline as configuration, but I guess one could also specify a YAML file
with the template and have spark configuration point to that file. [~skonto],
you make a good point, it is another configuration mechanism that people may
have to learn. This decision should be based more on UX and consistency with
what Spark users expect in general. [~eje], to your point, I think we could
support both if needed, but it might be prudent to find the one that's more
intuitive to users in order to do first.
Sidenote: There's also [jsonpath|https://kubernetes.io/docs/reference/kubectl/]
that kubectl supports but that could be overkill here.
> Support user-specified driver and executor pod templates
> --------------------------------------------------------
>
> Key: SPARK-24434
> URL: https://issues.apache.org/jira/browse/SPARK-24434
> Project: Spark
> Issue Type: New Feature
> Components: Kubernetes
> Affects Versions: 2.4.0
> Reporter: Yinan Li
> Priority: Major
>
> With more requests for customizing the driver and executor pods coming, the
> current approach of adding new Spark configuration options has some serious
> drawbacks: 1) it means more Kubernetes specific configuration options to
> maintain, and 2) it widens the gap between the declarative model used by
> Kubernetes and the configuration model used by Spark. We should start
> designing a solution that allows users to specify pod templates as central
> places for all customization needs for the driver and executor pods.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
