It may have to do with the akka heartbeat interval per SPARK-3923 -
https://issues.apache.org/jira/plugins/servlet/mobile#issue/SPARK-3923 ?
On Tue, Feb 24, 2015 at 16:40 Xi Shen davidshe...@gmail.com wrote:
Hi Sean,
I launched the spark-shell on the same machine as I started YARN service.
I don't think port will be an issue.
I am new to spark. I checked the HDFS web UI and the YARN web UI. But I
don't know how to check the AM. Can you help?
Thanks,
David
On Tue, Feb 24, 2015 at 8:37 PM Sean Owen so...@cloudera.com wrote:
I don't think the build is at issue. The error suggests your App Master
can't be contacted. Is there a network port issue? did the AM fail?
On Tue, Feb 24, 2015 at 9:15 AM, Xi Shen davidshe...@gmail.com wrote:
Hi Arush,
I got the pre-build from https://spark.apache.org/downloads.html. When
I start spark-shell, it prompts:
Spark assembly has been built with Hive, including Datanucleus jars
on classpath
So we don't have pre-build with YARN support? If so, how the
spark-submit work? I checked the YARN log, and job is really submitted and
ran successfully.
Thanks,
David
On Tue Feb 24 2015 at 6:35:38 PM Arush Kharbanda
ar...@sigmoidanalytics.com wrote:
Hi
Are you sure that you built Spark for Yarn.If standalone works, not
sure if its build for Yarn.
Thanks
Arush
On Tue, Feb 24, 2015 at 12:06 PM, Xi Shen davidshe...@gmail.com
wrote:
Hi,
I followed this guide,
http://spark.apache.org/docs/1.2.1/running-on-yarn.html, and tried to
start spark-shell with yarn-client
./bin/spark-shell --master yarn-client
But I got
WARN ReliableDeliverySupervisor: Association with remote system
[akka.tcp://sparkYarnAM@10.0.2.15:38171] has failed, address is now gated
for [5000] ms. Reason is: [Disassociated].
In the spark-shell, and other exceptions in they yarn log. Please see
http://stackoverflow.com/questions/28671171/spark-shell-cannot-connect-to-yarn
for more detail.
However, submitting to the this cluster works. Also, spark-shell as
standalone works.
My system:
- ubuntu amd64
- spark 1.2.1
- yarn from hadoop 2.6 stable
Thanks,
[image: --]
Xi Shen
[image: http://]about.me/davidshen
http://about.me/davidshen?promo=email_sig
http://about.me/davidshen
--
[image: Sigmoid Analytics]
http://htmlsig.com/www.sigmoidanalytics.com
*Arush Kharbanda* || Technical Teamlead
ar...@sigmoidanalytics.com || www.sigmoidanalytics.com
