[jira] [Commented] (SPARK-4389) Set akka.remote.netty.tcp.bind-hostname="0.0.0.0" so driver can be located behind NAT
[
https://issues.apache.org/jira/browse/SPARK-4389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15399053#comment-15399053
]
Adam Jakubowski commented on SPARK-4389:
Hi [~abraithwaite]
Did you manage to work around this issue somehow? I am also experiencing this
issue.
> Set akka.remote.netty.tcp.bind-hostname="0.0.0.0" so driver can be located
> behind NAT
> -
>
> Key: SPARK-4389
> URL: https://issues.apache.org/jira/browse/SPARK-4389
> Project: Spark
> Issue Type: Improvement
> Components: Spark Core
>Affects Versions: 1.2.0
>Reporter: Josh Rosen
>Priority: Minor
>
> We should set {{akka.remote.netty.tcp.bind-hostname="0.0.0.0"}} in our Akka
> configuration so that Spark drivers can be located behind NATs / work with
> weird DNS setups.
> This is blocked by upgrading our Akka version, since this configuration is
> not present Akka 2.3.4. There might be a different approach / workaround
> that works on our current Akka version, though.
> EDIT: this is blocked by Akka 2.4, since this feature is only available in
> the 2.4 snapshot release.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-4389) Set akka.remote.netty.tcp.bind-hostname="0.0.0.0" so driver can be located behind NAT
[
https://issues.apache.org/jira/browse/SPARK-4389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15092481#comment-15092481
]
Alan Braithwaite commented on SPARK-4389:
-
So is there any hope for running spark behind a transparent proxy then? What
is the preferred method for running a spark-master in an environment where
things get dynamically scheduled (mesos+marathon, kubernetes, etc)?
> Set akka.remote.netty.tcp.bind-hostname="0.0.0.0" so driver can be located
> behind NAT
> -
>
> Key: SPARK-4389
> URL: https://issues.apache.org/jira/browse/SPARK-4389
> Project: Spark
> Issue Type: Improvement
> Components: Spark Core
>Affects Versions: 1.2.0
>Reporter: Josh Rosen
>Priority: Minor
>
> We should set {{akka.remote.netty.tcp.bind-hostname="0.0.0.0"}} in our Akka
> configuration so that Spark drivers can be located behind NATs / work with
> weird DNS setups.
> This is blocked by upgrading our Akka version, since this configuration is
> not present Akka 2.3.4. There might be a different approach / workaround
> that works on our current Akka version, though.
> EDIT: this is blocked by Akka 2.4, since this feature is only available in
> the 2.4 snapshot release.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-4389) Set akka.remote.netty.tcp.bind-hostname=0.0.0.0 so driver can be located behind NAT
[
https://issues.apache.org/jira/browse/SPARK-4389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14600212#comment-14600212
]
Josh Rosen commented on SPARK-4389:
---
Akka 2.4-M1 was released in late May (http://akka.io/news/); not sure what its
current release status is. I doubt that we'll want to depend on a milestone
release of Akka, but you might be able to package a custom Spark build that
uses it.
For 1.5.0, there's an effort to add a new RPC implementation which is not based
on Akka. This custom RPC layer should let us sidestep these hostname issues.
See SPARK-6028 and https://github.com/apache/spark/pull/6457 for more details.
Set akka.remote.netty.tcp.bind-hostname=0.0.0.0 so driver can be located
behind NAT
-
Key: SPARK-4389
URL: https://issues.apache.org/jira/browse/SPARK-4389
Project: Spark
Issue Type: Improvement
Components: Spark Core
Affects Versions: 1.2.0
Reporter: Josh Rosen
Priority: Minor
We should set {{akka.remote.netty.tcp.bind-hostname=0.0.0.0}} in our Akka
configuration so that Spark drivers can be located behind NATs / work with
weird DNS setups.
This is blocked by upgrading our Akka version, since this configuration is
not present Akka 2.3.4. There might be a different approach / workaround
that works on our current Akka version, though.
EDIT: this is blocked by Akka 2.4, since this feature is only available in
the 2.4 snapshot release.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-4389) Set akka.remote.netty.tcp.bind-hostname=0.0.0.0 so driver can be located behind NAT
[
https://issues.apache.org/jira/browse/SPARK-4389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14600206#comment-14600206
]
Harry Brundage commented on SPARK-4389:
---
Any movement on this? Is akka 2.4 happening any time soon?
Set akka.remote.netty.tcp.bind-hostname=0.0.0.0 so driver can be located
behind NAT
-
Key: SPARK-4389
URL: https://issues.apache.org/jira/browse/SPARK-4389
Project: Spark
Issue Type: Improvement
Components: Spark Core
Affects Versions: 1.2.0
Reporter: Josh Rosen
Priority: Minor
We should set {{akka.remote.netty.tcp.bind-hostname=0.0.0.0}} in our Akka
configuration so that Spark drivers can be located behind NATs / work with
weird DNS setups.
This is blocked by upgrading our Akka version, since this configuration is
not present Akka 2.3.4. There might be a different approach / workaround
that works on our current Akka version, though.
EDIT: this is blocked by Akka 2.4, since this feature is only available in
the 2.4 snapshot release.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-4389) Set akka.remote.netty.tcp.bind-hostname=0.0.0.0 so driver can be located behind NAT
[
https://issues.apache.org/jira/browse/SPARK-4389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14259488#comment-14259488
]
Josh Rosen commented on SPARK-4389:
---
I think it's something like that; currently, I think that Akka doesn't give you
full, separate control over the hostname that its advertises and the network
interface(s) that it listens on. In your example, I think the issue is that
Akka would implicitly bind to only one of the IPs / interfaces, which might be
the wrong one if a hostname like {{example.com}} is used.
Set akka.remote.netty.tcp.bind-hostname=0.0.0.0 so driver can be located
behind NAT
-
Key: SPARK-4389
URL: https://issues.apache.org/jira/browse/SPARK-4389
Project: Spark
Issue Type: Improvement
Components: Spark Core
Reporter: Josh Rosen
Priority: Minor
We should set {{akka.remote.netty.tcp.bind-hostname=0.0.0.0}} in our Akka
configuration so that Spark drivers can be located behind NATs / work with
weird DNS setups.
This is blocked by upgrading our Akka version, since this configuration is
not present Akka 2.3.4. There might be a different approach / workaround
that works on our current Akka version, though.
EDIT: this is blocked by Akka 2.4, since this feature is only available in
the 2.4 snapshot release.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-4389) Set akka.remote.netty.tcp.bind-hostname=0.0.0.0 so driver can be located behind NAT
[
https://issues.apache.org/jira/browse/SPARK-4389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14259020#comment-14259020
]
Shixiong Zhu commented on SPARK-4389:
-
Try to make sure I understand this issue:
The driver binds to `example.com` and it has two IPs: `10.11.0.3`,
`192.168.0.3`. In the driver side, `example.com` is resolved to `192.168.0.3`,
so Akka will bind to `192.168.0.3`. But in the executor side, `example.com` is
resolved to `10.11.0.3`. Then the packets sent from the executor side to the
driver will be dropped.
Is it right?
Set akka.remote.netty.tcp.bind-hostname=0.0.0.0 so driver can be located
behind NAT
-
Key: SPARK-4389
URL: https://issues.apache.org/jira/browse/SPARK-4389
Project: Spark
Issue Type: Improvement
Components: Spark Core
Reporter: Josh Rosen
Priority: Minor
We should set {{akka.remote.netty.tcp.bind-hostname=0.0.0.0}} in our Akka
configuration so that Spark drivers can be located behind NATs / work with
weird DNS setups.
This is blocked by upgrading our Akka version, since this configuration is
not present Akka 2.3.4. There might be a different approach / workaround
that works on our current Akka version, though.
EDIT: this is blocked by Akka 2.4, since this feature is only available in
the 2.4 snapshot release.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
