[
https://issues.apache.org/jira/browse/YARN-8255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16466448#comment-16466448
]
Eric Yang commented on YARN-8255:
---------------------------------
Instead of introduce another field to enable or disable flex. We can identify
if the workload can perform flex operation base on restart_policy.
When restart_policy=ON_FAILURE or ALWAYS, this means the data can be
recomputed, or the process can resume from failure. Flex operation can be
enabled.
When restart_policy=NEVER, this means the data is stateful, and can not
reprocess. (i.e. mapreduce writes to HBase without transaction property.) .
This type of containers are not allowed to have flexing operation.
By reasoning deduction, it is possible to reduce combinations that will be
supported. This also implies that restart_policy=NEVER doesn't have to support
upgrade.
> Allow option to disable flex for a service component
> -----------------------------------------------------
>
> Key: YARN-8255
> URL: https://issues.apache.org/jira/browse/YARN-8255
> Project: Hadoop YARN
> Issue Type: Bug
> Components: yarn-native-services
> Reporter: Suma Shivaprasad
> Assignee: Suma Shivaprasad
> Priority: Major
>
> YARN-8080 implements restart capabilities for service component instances.
> YARN service components should add an option to disallow flexing to support
> workloads which are essentially batch/iterative jobs which terminate with
> restart_policy=NEVER/ON_FAILURE. This could be disabled by default for
> components where restart_policy=NEVER/ON_FAILURE and enabled by default when
> restart_policy=ALWAYS(which is the default restart_policy) unless explicitly
> set at the service spec.
> The option could be exposed as part of the component spec as "allow_flexing".
> cc [~billie.rinaldi] [~gsaha] [~eyang] [~csingh] [~wangda]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
