[
https://issues.apache.org/jira/browse/SPARK-4563?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15636802#comment-15636802
]
György Süveges commented on SPARK-4563:
---------------------------------------
+1
However, it's related to SPARK_LOCAL_IP and Spark UI, I think it's similar.
We use AWS EMR5 (Spark 2.0 over YARN)
While developing we use spark from intellij in yarn-client mode. To connect to
AWS, OpenVPN is applied with biderectional routing so the executors in amazon
can access the driver on the local workstation. With OpenVPN there are
*multiple network interfaces* then, so SPARK_LOCAL_IP should be set. But it's
really inconvenient to set SPARK_LOCAL_IP each time we connect to OpenVPN and
get different IP.
So we decided to set it automatically from code. Since SPARK_LOCAL_IP is env
var and env vars are inmutable in java, then we change the spark.driver.host
corresponding java property. But as notices spark.driver.host is only second
order citizen in spark, for some features only the SPARK_LOCAL_IP is taken into
account. For example Spark UI listens on the host set in SPARK_LOCAL_IP, but
ignores the spark.driver.host totally. I hope this improvement will fix this
issue, as I saw in
https://github.com/apache/spark/commit/2cd1bfa4f0c6625b0ab1dbeba2b9586b9a6a9f42
I think SPARK_LOCAL_IP and spark.driver.host should be equivalent counterparts,
shouldn't they?
> Allow spark driver to bind to different ip then advertise ip
> ------------------------------------------------------------
>
> Key: SPARK-4563
> URL: https://issues.apache.org/jira/browse/SPARK-4563
> Project: Spark
> Issue Type: Improvement
> Components: Deploy
> Reporter: Long Nguyen
> Assignee: Marcelo Vanzin
> Priority: Minor
> Fix For: 2.1.0
>
>
> Spark driver bind ip and advertise is not configurable. spark.driver.host is
> only bind ip. SPARK_PUBLIC_DNS does not work for spark driver. Allow option
> to set advertised ip/hostname
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
