Sameer Agarwal created SPARK-20531:
--------------------------------------
Summary: Spark master shouldn't send its address back to the
workers over proxied connections
Key: SPARK-20531
URL: https://issues.apache.org/jira/browse/SPARK-20531
Project: Spark
Issue Type: Bug
Components: Spark Core
Affects Versions: 2.0.2, 1.6.3, 2.1.2, 2.2.0
Reporter: Sameer Agarwal
Currently, when a Spark worker connects to Spark master, the master sends its
address back to the worker (as part of the {{RegisteredWorker}} message). The
worker then saves this address and use it to talk to the master. The reason
behind this handshake protocol is that if the master goes down, a new master
can always send back {{RegisteredWorker}} messages to all the workers with its
(new) IP address.
However, if there is a proxy between the master and worker, this unfortunately
ends up bypassing the proxy. A simple fix here is that we should encode the
"destination address" in the {{RegisterWorker}} that can then be sent back to
the to the worker as part of the {{RegisteredWorker}} message.
cc [~zsxwing]
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
