Mingyu Kim created SPARK-15974:
----------------------------------
Summary: Create a socket on YARN AM start-up
Key: SPARK-15974
URL: https://issues.apache.org/jira/browse/SPARK-15974
Project: Spark
Issue Type: New Feature
Components: YARN
Reporter: Mingyu Kim
YARN provides a way for AppilcationMaster to register a RPC port so that a
client outside the YARN cluster can reach the application for any RPCs, but
Spark’s YARN AMs simply register a dummy port number of 0. For the Spark
programs that starts up a server, this makes it hard for the submitter to
discover the server port securely. Spark's ApplicationMaster should optionally
create a ServerSocket and pass it to the Spark user program. This socket
initialization should be disabled by default.
Some discussion on dev@spark thread:
http://apache-spark-developers-list.1001551.n3.nabble.com/Utilizing-YARN-AM-RPC-port-field-td17892.html
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
