Github user aditanase commented on a diff in the pull request:
https://github.com/apache/spark/pull/22146#discussion_r223596796
--- Diff: docs/running-on-kubernetes.md ---
@@ -799,4 +815,168 @@ specific to Spark on Kubernetes.
This sets the major Python version of the docker image used to run the
driver and executor containers. Can either be 2 or 3.
</td>
</tr>
+<tr>
+ <td><code>spark.kubernetes.driver.podTemplateFile</code></td>
+ <td>(none)</td>
+ <td>
+ Specify the local file that contains the driver [pod
template](#pod-template). For example
+
<code>spark.kubernetes.driver.podTemplateFile=/path/to/driver-pod-template.yaml`</code>
+ </td>
+</tr>
+<tr>
+ <td><code>spark.kubernetes.executor.podTemplateFile</code></td>
+ <td>(none)</td>
+ <td>
+ Specify the local file that contains the executor [pod
template](#pod-template). For example
+
<code>spark.kubernetes.executor.podTemplateFile=/path/to/executor-pod-template.yaml`</code>
+ </td>
+</tr>
+</table>
+
+#### Pod template properties
+
+See the below table for the full list of pod specifications that will be
overwritten by spark.
+
+### Pod Metadata
+
+<table class="table">
+<tr><th>Pod metadata key</th><th>Modified
value</th><th>Description</th></tr>
+<tr>
+ <td>name</td>
+ <td>Value of <code>spark.kubernetes.driver.pod.name</code></td>
+ <td>
+ The driver pod name will be overwritten with either the configured or
default value of
+ <code>spark.kubernetes.driver.pod.name</code>. The executor pod names
will be unaffected.
+ </td>
+</tr>
+<tr>
+ <td>namespace</td>
+ <td>Value of <code>spark.kubernetes.namespace</code></td>
+ <td>
+ Spark makes strong assumptions about the driver and executor
namespaces. Both driver and executor namespaces will
+ be replaced by either the configured or default spark conf value.
+ </td>
+</tr>
+<tr>
+ <td>labels</td>
+ <td>Adds the labels from
<code>spark.kubernetes.{driver,executor}.label.*</code></td>
+ <td>
+ Spark will add additional labels specified by the spark configuration.
+ </td>
+</tr>
+<tr>
+ <td>annotations</td>
+ <td>Adds the annotations from
<code>spark.kubernetes.{driver,executor}.annotation.*</code></td>
+ <td>
+ Spark will add additional labels specified by the spark configuration.
+ </td>
+</tr>
+</table>
+
+### Pod Spec
+
+<table class="table">
+<tr><th>Pod spec key</th><th>Modified value</th><th>Description</th></tr>
+<tr>
+ <td>imagePullSecrets</td>
+ <td>Adds image pull secrets from
<code>spark.kubernetes.container.image.pullSecrets</code></td>
+ <td>
+ Additional pull secrets will be added from the spark configuration to
both executor pods.
+ </td>
+</tr>
+<tr>
+ <td>nodeSelector</td>
+ <td>Adds node selectors from
<code>spark.kubernetes.node.selector.*</code></td>
+ <td>
+ Additional node selectors will be added from the spark configuration
to both executor pods.
+ </td>
+</tr>
+<tr>
+ <td>restartPolicy</td>
+ <td><code>"never"</code></td>
+ <td>
+ Spark assumes that both drivers and executors never restart.
+ </td>
+</tr>
+<tr>
+ <td>serviceAccount</td>
+ <td>Value of
<code>spark.kubernetes.authenticate.driver.serviceAccountName</code></td>
+ <td>
+ Spark will override <code>serviceAccount</code> with the value of the
spark configuration for only
+ driver pods, and only if the spark configuration is specified.
Executor pods will remain unaffected.
+ </td>
+</tr>
+<tr>
+ <td>serviceAccountName</td>
+ <td>Value of
<code>spark.kubernetes.authenticate.driver.serviceAccountName</code></td>
+ <td>
+ Spark will override <code>serviceAccountName</code> with the value of
the spark configuration for only
+ driver pods, and only if the spark configuration is specified.
Executor pods will remain unaffected.
+ </td>
+</tr>
+<tr>
+ <td>volumes</td>
+ <td>Adds volumes from
<code>spark.kubernetes.{driver,executor}.volumes.[VolumeType].[VolumeName].mount.path</code></td>
+ <td>
+ Spark will add volumes as specified by the spark conf, as well as
additional volumes necessary for passing
--- End diff --
I think we should also document the volume names (and optionally mount
points) for all the "internal" spark volumes (pod download, config map, etc).
As people will use the pod template to also mount their own volumes, we
should document the existing names so we can avoid name clashes.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
