[
https://issues.apache.org/jira/browse/SPARK-24434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16496962#comment-16496962
]
Anirudh Ramanathan commented on SPARK-24434:
--------------------------------------------
The way several custom APIs have done this before is having a PodTemplate field
that uses the Kubernetes API to provide a rich type-safe interface to add
arbitrary modifications to pods. It's typically easier with golang structs to
do that, but we should investigate if from openapi, there's a way for the Java
client to expose the same. Given that we will want it to map back to
stringified configuration, supporting JSON strings seems like a good choice
there.
So, the flow I see is JSON strings converted into valid (type-checked) and
supported PodTemplate specifications that are eventually added to driver and
executor pods.
> Support user-specified driver and executor pod templates
> --------------------------------------------------------
>
> Key: SPARK-24434
> URL: https://issues.apache.org/jira/browse/SPARK-24434
> Project: Spark
> Issue Type: New Feature
> Components: Kubernetes
> Affects Versions: 2.4.0
> Reporter: Yinan Li
> Priority: Major
>
> With more requests for customizing the driver and executor pods coming, the
> current approach of adding new Spark configuration options has some serious
> drawbacks: 1) it means more Kubernetes specific configuration options to
> maintain, and 2) it widens the gap between the declarative model used by
> Kubernetes and the configuration model used by Spark. We should start
> designing a solution that allows users to specify pod templates as central
> places for all customization needs for the driver and executor pods.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
