[
https://issues.apache.org/jira/browse/YARN-230?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13501361#comment-13501361
]
Bikas Saha commented on YARN-230:
---------------------------------
Attaching 4 patches that break up the whole change for easy reviewing. They
wont build on their own.
1) PB-impl.patch - Classes and PB impl for objects used to store Application
and ApplicationAttempt data. Pretty straightforward code derived from similar
code for other PB impls.
2) Store.patch - Implementation of RMStateStore abstract class that interfaces
between RM and real store classes. It translates RM objects like RMAppImpl into
PB objects like ApplicationStateData. It also provides a common implementation
of blocking and non-blocking store functions. Non-blocking operations are
performed using an AsyncDispatcher and RMStateStore events that eventually call
implementations of abstract methods which will be provided by real stores. A
memory store is implemented for testing.
3) Test.patch - 1 new test TestRMRestart is a functional test that takes the RM
through storing and recovering state with applications in different states of
execution. The flow should be easy to follow with comments.
TestRMAppAttemptTransitions tests for the RMAppAttemptImpl state machine
changes. Other changes are refactoring and addition of helper methods to MockRM
etc.
4) Recovery.patch - Implements the proposal in the design doc. RM startup loads
old state and recovers from it if recovery is enabled. Each app that is
recovered is submitted to the RMAppManager so that it re-hydrates all
references and tokens like it would normally and then is ready to start its
next attempt. Each recovered application attempt is added to the app's attempts
collection and moved to a RECOVERED state. After that the services, including
the AsyncDispatcher is started which trigger creation of new attempts for the
submitted apps. Existing code in ApplicationMasterService and
ResourceTrackerService reboot NM's and previously running AM's. When a new app
is submitted then before replying to the client its data is saved (blocking).
Just before an attempt is launched, its data is saved (non-blocking). For
non-blocking state store new RMAppAttempt states have been added for regular
and unmanaged attempts. Once an app is finished, its data is removed.
> Make changes for RM restart phase 1
> -----------------------------------
>
> Key: YARN-230
> URL: https://issues.apache.org/jira/browse/YARN-230
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: resourcemanager
> Reporter: Bikas Saha
> Assignee: Bikas Saha
> Attachments: YARN-230.1.patch
>
>
> As described in YARN-128, phase 1 of RM restart puts in place mechanisms to
> save application state and read them back after restart. Upon restart, the
> NM's are asked to reboot and the previously running AM's are restarted.
> After this is done, RM HA and work preserving restart can continue in
> parallel. For more details please refer to the design document in YARN-128
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
