RE: How to make AM terminate if client crashes?
In our application the "client" is actually one step of many in an application. If the "client" goes away entirely, that means the entire application has crashed or has been terminated somehow. Our application doesn't checkpoint state at this step, so the step will have to be restarted anyway. This doesn't break anything, it is just a waste to let it keep running. Thanks for clarifying the stateless nature of the client connection. John From: Vinod Kumar Vavilapalli [mailto:vino...@hortonworks.com] Sent: Monday, January 13, 2014 9:31 AM To: user@hadoop.apache.org Subject: Re: How to make AM terminate if client crashes? The architecture is built around detachable clients. So, no, it doesn't happen automatically. Even if we were to add that feature, it'd be fraught with edge cases - network issues causing app-termination even though client is still alive etc. Any more details on why this is desired? +Vinod On Jan 11, 2014, at 11:37 AM, John Lilley mailto:john.lil...@redpoint.net>> wrote: We have a YARN application that we want to automatically terminate if the YARN client disconnects or crashes. Is it possible to configure the YarnClient-RM connection so that if the client terminates the RM automatically terminates the AM? Or do we need to build our own logic (e.g. a direct client-AM connection) for that? Thanks John CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
Re: How to make AM terminate if client crashes?
You would probably need to bake this into your own application. By default, a client never should need to keep an open active connection with the RM. It could keep an active connection with the AM ( application-specific code required ) but it would then also have to handle failover to a different AM if the first AM crashes and a new attempt is launched by the RM. Does your application have access to a zookeeper cluster within your installation? It would be simple enough for someone to build a library which either uses an ephemeral node or updates some form of an applicationId specific keep-alive flag that the AM can monitor to make its decisions. @Vinod, on a different note, does it make sense for the RM to run an embedded ZK to aid fencing across multiple AM attempts which could then be used by the above to piggyback on? -- Hitesh On Jan 13, 2014, at 8:30 AM, Vinod Kumar Vavilapalli wrote: > The architecture is built around detachable clients. So, no, it doesn't > happen automatically. Even if we were to add that feature, it'd be fraught > with edge cases - network issues causing app-termination even though client > is still alive etc. > > Any more details on why this is desired? > > +Vinod > > On Jan 11, 2014, at 11:37 AM, John Lilley wrote: > >> We have a YARN application that we want to automatically terminate if the >> YARN client disconnects or crashes. Is it possible to configure the >> YarnClient-RM connection so that if the client terminates the RM >> automatically terminates the AM? Or do we need to build our own logic (e.g. >> a direct client-AM connection) for that? >> Thanks >> John
Re: How to make AM terminate if client crashes?
The architecture is built around detachable clients. So, no, it doesn't happen automatically. Even if we were to add that feature, it'd be fraught with edge cases - network issues causing app-termination even though client is still alive etc. Any more details on why this is desired? +Vinod On Jan 11, 2014, at 11:37 AM, John Lilley wrote: > We have a YARN application that we want to automatically terminate if the > YARN client disconnects or crashes. Is it possible to configure the > YarnClient-RM connection so that if the client terminates the RM > automatically terminates the AM? Or do we need to build our own logic (e.g. > a direct client-AM connection) for that? > Thanks > John -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
Re: How to make AM terminate if client crashes?
You can set the maximum map and reduce attempts so that if a failure occurred job gets failed and done. On Sat, Jan 11, 2014 at 11:07 PM, John Lilley wrote: > We have a YARN application that we want to automatically terminate if > the YARN client disconnects or crashes. Is it possible to configure the > YarnClient-RM connection so that if the client terminates the RM > automatically terminates the AM? Or do we need to build our own logic > (e.g. a direct client-AM connection) for that? > > Thanks > > John >
How to make AM terminate if client crashes?
We have a YARN application that we want to automatically terminate if the YARN client disconnects or crashes. Is it possible to configure the YarnClient-RM connection so that if the client terminates the RM automatically terminates the AM? Or do we need to build our own logic (e.g. a direct client-AM connection) for that? Thanks John
