Unable to stop containers when keep-containers-across-application-attempt is enabled

2020-11-30 Thread Bharath Kumara Subramanian
Hi, I am currently trying to keep the containers from the previous attempts alive across attempts so that when AM restarts happen, the processing containers stay intact. I am achieving this using the keep-containers-across-application-attempt flag. For my use case, I do need to stop the

Re: Application Master High Availability

2020-08-24 Thread Bharath Kumara Subramanian
ve tested this. But the MapReduce framework does not support >> this feature (see https://issues.apache.org/jira/browse/MAPREDUCE-6608). >> I spoke with Jon Eagles, and he believes that the Tez framework does >> support the container-preserving feature. >> >> On Tuesday, Augus

Re: Application Master High Availability

2020-08-14 Thread Bharath Kumara Subramanian
/MAPREDUCE-6608). > I spoke with Jon Eagles, and he believes that the Tez framework does > support the container-preserving feature. > > On Tuesday, August 11, 2020, 5:53:32 PM CDT, Bharath Kumara Subramanian < > codin.mart...@gmail.com> wrote: > > Thanks Eric & Wilfred

Re: Application Master High Availability

2020-08-11 Thread Bharath Kumara Subramanian
e frameworks do support > preserving containers across AM restarts. > In the yarn distributed shell, for example, you can set the > "-keep_containers_across_application_attempts" > parameter. > > -Eric > > > On Monday, August 10, 2020, 7:18:18 PM CDT, Bharath Kumara Subr

Application Master High Availability

2020-08-10 Thread Bharath Kumara Subramanian
Hi, I am looking for more documentation/information on AM high availability. I looked through the documentation and found resources on RM high availability but none for AM. I understand, YARN has provisions to restart AM in case of failure up to a configured number of attempts. However, I wanted