Thanks Steve, Is No 1 a new feature in YARN (Not released yet)?
And you mentioned slider saves the location in history files. What are the history files and where is it stored? Is it in HDFS? If the one of the previous machines is gone, will it try to get resource from new labeled machine? Thanks On Wed, Nov 5, 2014 at 12:46 PM, Steve Loughran <ste...@hortonworks.com> wrote: > On 5 November 2014 20:21, hsy...@gmail.com <hsy...@gmail.com> wrote: > > > Hi guys, > > > > I noticed in the code when a container fails it will try to relaunch from > > the same node. My question is if I restart whole application(Ex. AM got > > killed, or manually restart the app). Does slider try to launch all > > containers from the nodes where it was running? > > > > > 1. If the AM crashes then YARN will restart it. The containers will keep > working. When the AM comes back up it will work out its state and all > running containers will stay live. Any containers that were part way > through starting will be released and new ones requested (there's no record > of what state they were in, so a clean destroy is simpler) > > > If you stop/start the app then it asks for the nodes back on the same > machines they were on. It saves the locations (look in the history subdir) > to see the history files. > > Slider tries to read the last entry, going back to previous ones if the > last one doesn't load. It then asks YARN for containers on those machines. > There's no guarantee you get them though. > > Looking at the history code last week I noticed one little quirk: it > doesn't reload the histories if the number of component types has > increased. It just indexes the entries; more entries means it doesn't know > how to handle them. > > To avoid this problem define all your components from the outset, setting > the instances count 0 for ones you don't currently want > > Thanks! > > > > Best, > > Siyuan > > > > -- > CONFIDENTIALITY NOTICE > NOTICE: This message is intended for the use of the individual or entity to > which it is addressed and may contain information that is confidential, > privileged and exempt from disclosure under applicable law. If the reader > of this message is not the intended recipient, you are hereby notified that > any printing, copying, dissemination, distribution, disclosure or > forwarding of this communication is strictly prohibited. If you have > received this communication in error, please contact the sender immediately > and delete it from your system. Thank You. >
