Thanks for the help :) On Tue, Feb 20, 2018 at 4:22 PM, Femi Anthony <femib...@gmail.com> wrote:
> You can use spark speculation as a way to get around the problem. > > Here is a useful link: > > http://asyncified.io/2016/08/13/leveraging-spark- > speculation-to-identify-and-re-schedule-slow-running-tasks/ > > Sent from my iPhone > > On Feb 20, 2018, at 5:52 PM, Nikhil Goyal <nownik...@gmail.com> wrote: > > Hi guys, > > I have a job which gets stuck if a couple of tasks get killed due to OOM > exception. Spark doesn't kill the job and it keeps on running for hours. > Ideally I would expect Spark to kill the job or restart the killed > executors but nothing seems to be happening. Anybody got idea about this? > > Thanks > Nikhil > >
