[
https://issues.apache.org/jira/browse/YARN-8255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16466800#comment-16466800
]
Eric Yang edited comment on YARN-8255 at 5/8/18 3:08 AM:
---------------------------------------------------------
[~leftnoteasy] Recompute and expandable are intertwined. They are not the same
thing. At conceptual level, teragen has no dependency of input format. You
can add more partitions to get more data generated. Hadoop's own
implementation limited this from happening, but this does not mean docker
containers should be imposed by the same initialization time limitation. On
the other hand, we must optimize the framework for general purpose usage and
prevent ourselves from giving too many untested and unsupported options. I
think it make sense to reduce the flex options to 2 main types instead of
giving all 6 options.
was (Author: eyang):
[~leftnoteasy] Recompute and expandable are intertwined. They are the same
thing. At conceptual level, teragen has no dependency of input format. You
can add more partitions to get more data generated. Hadoop's own
implementation limited this from happening, but this does not mean docker
containers should be imposed by the same initialization time limitation. On
the other hand, we must optimize the framework for general purpose usage and
prevent ourselves from giving too many untested and unsupported options. I
think it make sense to reduce the flex options to 2 main types instead of
giving all 6 options.
> Allow option to disable flex for a service component
> -----------------------------------------------------
>
> Key: YARN-8255
> URL: https://issues.apache.org/jira/browse/YARN-8255
> Project: Hadoop YARN
> Issue Type: Bug
> Components: yarn-native-services
> Reporter: Suma Shivaprasad
> Assignee: Suma Shivaprasad
> Priority: Major
>
> YARN-8080 implements restart capabilities for service component instances.
> YARN service components should add an option to disallow flexing to support
> workloads which are essentially batch/iterative jobs which terminate with
> restart_policy=NEVER/ON_FAILURE. This could be disabled by default for
> components where restart_policy=NEVER/ON_FAILURE and enabled by default when
> restart_policy=ALWAYS(which is the default restart_policy) unless explicitly
> set at the service spec.
> The option could be exposed as part of the component spec as "allow_flexing".
> cc [~billie.rinaldi] [~gsaha] [~eyang] [~csingh] [~wangda]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
