[
https://issues.apache.org/jira/browse/SPARK-6017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14338423#comment-14338423
]
Thomas Graves commented on SPARK-6017:
--------------------------------------
when you say:
"Currently the method of distributing the shared secret is not secure. The
secret is
passed to executors on the command line, which means anyone that can run "ps" on
the machine running the executor can see it."
Are you referring to ssl on yarn or standalone mode? Sorry I haven't looked at
the ssl stuff at all. The yarn mode distributed it through the UGI which is
not on the command line.
On Yarn you can use use the UGI to distributed another secret key and no need
for extra files. That obviously wouldn't work for standalone mode though.
> Provide transparent secure communication channel on Yarn
> --------------------------------------------------------
>
> Key: SPARK-6017
> URL: https://issues.apache.org/jira/browse/SPARK-6017
> Project: Spark
> Issue Type: Umbrella
> Components: YARN
> Reporter: Marcelo Vanzin
> Attachments: secure_spark_on_yarn.pdf
>
>
> A quick description:
> Currently driver and executors communicate through an insecure channel, so
> anyone can listen on the network and see what's going on. That prevents Spark
> from adding some features securely (e.g. SPARK-5342, SPARK-5682) without
> resorting to using internal Hadoop APIs.
> Spark 1.3.0 will add SSL support, but properly configuring SSL is not a
> trivial task for operators, let alone users.
> In light of those, we should add a more transparent secure transport layer.
> I've written a short spec to identify the areas in Spark that need work to
> achieve this, and I'll attach the document to this issue shortly.
> Note I'm restricting things to Yarn currently, because as far as I know it's
> the only cluster manager that provides the needed security features to
> bootstrap the secure Spark transport. The design itself doesn't really rely
> on Yarn per se, just on a secure way to distribute the initial secret (which
> the Yarn/HDFS combo provides).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
