Re: automatic start of streaming job on failure on YARN

yes in yarn cluster mode. On 2 October 2015 at 22:10, Ashish Rangole wrote: > Are you running the job in yarn cluster mode? > On Oct 1, 2015 6:30 AM, "Jeetendra Gangele" wrote: > >> We've a streaming application running on yarn and we would like to

Re: automatic start of streaming job on failure on YARN

On 1 Oct 2015, at 16:52, Adrian Tanase > wrote: This happens automatically as long as you submit with cluster mode instead of client mode. (e.g. ./spark-submit —master yarn-cluster …) The property you mention would help right after that, although

Re: automatic start of streaming job on failure on YARN

Are you running the job in yarn cluster mode? On Oct 1, 2015 6:30 AM, "Jeetendra Gangele" wrote: > We've a streaming application running on yarn and we would like to ensure > that is up running 24/7. > > Is there a way to tell yarn to automatically restart a specific >

automatic start of streaming job on failure on YARN

We've a streaming application running on yarn and we would like to ensure that is up running 24/7. Is there a way to tell yarn to automatically restart a specific application on failure? There is property yarn.resourcemanager.am.max-attempts which is default set to 2 setting it to bigger value

Re: automatic start of streaming job on failure on YARN

This happens automatically as long as you submit with cluster mode instead of client mode. (e.g. ./spark-submit —master yarn-cluster …) The property you mention would help right after that, although you will need to set it to a large value (e.g. 1000?) - as there is no “infinite” support.