the spark_shuffle via the documentation
https://spark.apache.org/docs/1.4.1/job-scheduling.html ??
I'm testing it on Hadoop 2.4.1.
Any feedback or suggestion are appreciated, thanks.
Date: Fri, 17 Jul 2015 15:35:29 -0700
Subject: Re: The auxService:spark_shuffle does not exist
From: and...@databricks.com
Hi Andrew,
Based on your driver logs, it seems the issue is that the shuffle service
is actually not running on the NodeManagers, but your application is trying
to provide a spark_shuffle secret anyway. One way to verify whether the
shuffle service is actually started is to look at the
is not
enabled.
Appreciate all and the pointers where to look at. Thanks, problem solved.
Date: Tue, 21 Jul 2015 09:31:50 -0700
Subject: Re: The auxService:spark_shuffle does not exist
From: and...@databricks.com
To: alee...@hotmail.com
CC: zjf...@gmail.com; rp...@njit.edu; user@spark.apache.org
:106)
Not sure what else am I missing here or doing wrong?
Appreciate any insights or feedback, thanks.
Date: Wed, 8 Jul 2015 09:25:39 +0800
Subject: Re: The auxService:spark_shuffle does not exist
From: zjf...@gmail.com
To: rp...@njit.edu
CC: user@spark.apache.org
Did you enable the dynamic resource
.
--
Date: Wed, 8 Jul 2015 09:25:39 +0800
Subject: Re: The auxService:spark_shuffle does not exist
From: zjf...@gmail.com
To: rp...@njit.edu
CC: user@spark.apache.org
Did you enable the dynamic resource allocation ? You can refer to this
page for how to configure
Did you enable the dynamic resource allocation ? You can refer to this page
for how to configure spark shuffle service for yarn.
https://spark.apache.org/docs/1.4.0/job-scheduling.html
On Tue, Jul 7, 2015 at 10:55 PM, roy rp...@njit.edu wrote:
we tried --master yarn-client with no different
we tried --master yarn-client with no different result.
--
View this message in context:
http://apache-spark-user-list.1001560.n3.nabble.com/The-auxService-spark-shuffle-does-not-exist-tp23662p23689.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.
