login (or port forwarding) onto the Spark Driver
machine to launch shell or submit spark jobs.
Subject: RE: spark-shell driver interacting with Workers in YARN mode -
firewall blocking communication
To: user@spark.apache.org
From: jeis...@us.ibm.com
Date: Mon, 5 May 2014 12:40:53 -0500
Howdy
with iptables off, but still need to figure out how to
product-ionize the security part.
Subject: RE: spark-shell driver interacting with Workers in YARN mode -
firewall blocking communication
To: user@spark.apache.org
From: jeis...@us.ibm.com
Date: Fri, 2 May 2014 16:07:50 -0500
Howdy Andrew
Hi All,
I encountered this problem when the firewall is enabled between the spark-shell
and the Workers.
When I launch spark-shell in yarn-client mode, I notice that Workers on the
YARN containers are trying to talk to the driver (spark-shell), however, the
firewall is not opened and caused
I think what you want to do is set spark.driver.port to a fixed port.
On Fri, May 2, 2014 at 1:52 PM, Andrew Lee alee...@hotmail.com wrote:
Hi All,
I encountered this problem when the firewall is enabled between the
spark-shell and the Workers.
When I launch spark-shell in yarn-client
-0400
Subject: Re: spark-shell driver interacting with Workers in YARN mode -
firewall blocking communication
From: yana.kadiy...@gmail.com
To: user@spark.apache.org
I think what you want to do is set spark.driver.port to a fixed port.
On Fri, May 2, 2014 at 1:52 PM, Andrew Lee alee...@hotmail.com
From: Andrew Lee alee...@hotmail.com
To: user@spark.apache.org user@spark.apache.org
Date: 05/02/2014 03:15 PM
Subject:RE: spark-shell driver interacting with Workers in YARN mode -
firewall blocking communication
Hi Yana,
I did. I configured the the port in spark-env.sh
