a Das" <t...@databricks.com>
Sent: 9/23/2015 2:46 AM
To: "Bryan Jeffrey" <bryan.jeff...@gmail.com>
Cc: "user" <user@spark.apache.org>
Subject: Re: Yarn Shutting Down Spark Processing
Does your simple Spark batch jobs work in the same YARN setup? May be
Does your simple Spark batch jobs work in the same YARN setup? May be YARN
is not able to provide resources that you are asking for.
On Tue, Sep 22, 2015 at 5:49 PM, Bryan Jeffrey
wrote:
> Hello.
>
> I have a Spark streaming job running on a cluster managed by Yarn.
CC;ing Hari who may have a better sense of whats going on.
-- Forwarded message --
From: Bryan <bryan.jeff...@gmail.com>
Date: Wed, Sep 23, 2015 at 3:43 AM
Subject: RE: Yarn Shutting Down Spark Processing
To: Tathagata Das <t...@databricks.com>
Cc: user <user@s
;; "Hari Shreedharan"
<hshreedha...@cloudera.com>
Subject: Re: Yarn Shutting Down Spark Processing
CC;ing Hari who may have a better sense of whats going on.
-- Forwarded message --
From: Bryan <bryan.jeff...@gmail.com>
Date: Wed, Sep 23, 2015
Did you look at your application's logs (using the "yarn logs" command?).
That error means your application is failing to create a SparkContext.
So either you have a bug in your code, or there will be some error in
the log pointing at the actual reason for the failure.
On Tue, Sep 22, 2015 at
-Original Message-
From: "Marcelo Vanzin" <van...@cloudera.com>
Sent: 9/23/2015 5:55 PM
To: "Bryan Jeffrey" <bryan.jeff...@gmail.com>
Cc: "user" <user@spark.apache.org>
Subject: Re: Yarn Shutting Down Spark Processing
Did you look at your a
But that's not the complete application log. You say the streaming
context is initialized, but can you show that in the logs? There's
something happening that is causing the SparkContext to not be
registered with the YARN backend, and that's why your application is
being killed.
If you can share
Hello.
I have a Spark streaming job running on a cluster managed by Yarn. The
spark streaming job starts and receives data from Kafka. It is processing
well and then after several seconds I see the following error:
15/09/22 14:53:49 ERROR yarn.ApplicationMaster: SparkContext did not
initialize