lto:user@spark.apache.org>"
Subject: Re: Spark on YARN / aws - executor lost on node restart
Hi guys,
Digging up this question after spending some more time trying to replicate it.
It seems to be an issue with the YARN – spark integration, wondering if there
is a bug already tracking this?
dies completely?
If there are no ideas on the list, I’ll prepare some logs and follow up with an
issue.
Thanks,
-adrian
From: Adrian Tanase
Date: Wednesday, September 16, 2015 at 6:01 PM
To: "user@spark.apache.org<mailto:user@spark.apache.org>"
Subject: Spark on YARN / aws
Hi all,
We’re using spark streaming (1.4.0), deployed on AWS through yarn. It’s a
stateful app that reads from kafka (with the new direct API) and we’re
checkpointing to HDFS.
During some resilience testing, we restarted one of the machines and brought it
back online. During the offline
