Re: Notification on Spark Streaming job failure

On 7 Oct 2015, at 06:28, Krzysztof Zarzycki > wrote: Hi Vikram, So you give up using yarn-cluster mode of launching Spark jobs, is that right? AFAIK when using yarn-cluster mode, the launch process (spark-submit) monitors job running on YARN,

Re: Notification on Spark Streaming job failure

PM To: Krzysztof Zarzycki Cc: user Subject: Re: Notification on Spark Streaming job failure I am also interested specifically in monitoring and alerting on Spark streaming jobs. It will be helpful to get some general guidelines or advice on this, from people who implemented anything

Re: Notification on Spark Streaming job failure

Hi Vikram, So you give up using yarn-cluster mode of launching Spark jobs, is that right? AFAIK when using yarn-cluster mode, the launch process (spark-submit) monitors job running on YARN, but if it is killed/dies, it just stops printing the state (RUNNING usually), without influencing the

Re: Notification on Spark Streaming job failure

We are using Monit to kick off spark streaming jobs n seems to work fine. On Monday, September 28, 2015, Chen Song wrote: > I am also interested specifically in monitoring and alerting on Spark > streaming jobs. It will be helpful to get some general guidelines or advice

Re: Notification on Spark Streaming job failure

I am also interested specifically in monitoring and alerting on Spark streaming jobs. It will be helpful to get some general guidelines or advice on this, from people who implemented anything on this. On Fri, Sep 18, 2015 at 2:35 AM, Krzysztof Zarzycki wrote: > Hi there

Notification on Spark Streaming job failure

Hi there Spark Community, I would like to ask you for an advice: I'm running Spark Streaming jobs in production. Sometimes these jobs fail and I would like to get email notification about it. Do you know how I can set up Spark to notify me by email if my job fails? Or do I have to use external