[
https://issues.apache.org/jira/browse/SPARK-23857?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-23857:
------------------------------------
Assignee: Apache Spark
> In mesos cluster mode keytab Spark submit requires the keytab to be available
> on the local file system.
> -------------------------------------------------------------------------------------------------------
>
> Key: SPARK-23857
> URL: https://issues.apache.org/jira/browse/SPARK-23857
> Project: Spark
> Issue Type: Bug
> Components: Mesos
> Affects Versions: 2.3.0
> Reporter: Stavros Kontopoulos
> Assignee: Apache Spark
> Priority: Minor
>
> Users could submit their jobs from an external to the cluster host which may
> not have the required keytab locally (also discussed here).
> Moreover, in cluster mode it does not make much sense to reference a local
> resource unless this is uploaded/stored somewhere in the cluster. For yarn
> HDFS is used, on mesos and certainly on DC/OS right now the secret store is
> used for storing secrets and consequently keytabs. There is a check
> [here|https://github.com/apache/spark/blob/7cf9fab33457ccc9b2d548f15dd5700d5e8d08ef/core/src/main/scala/org/apache/spark/deploy/SparkSubmit.scala#L387]
> that makes spark submit difficult to use in such deployment scenarios.
> On DC/OS the workaround is to directly submit to the mesos dispatcher rest
> api by passing the spark.yarn.tab property pointing to a path within the
> driver's container where the keytab will be mounted after its fetched from
> the secret store, at container's launch time. Target is to allow spark submit
> be flexible enough for mesos in cluster mode, as DC/OS users often want to
> deploy using that.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
