[
https://issues.apache.org/jira/browse/YARN-6593?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16095217#comment-16095217
]
Chris Douglas commented on YARN-6593:
-------------------------------------
bq. I found it is very important otherwise we cannot support complex placement
request which need to be updated.
Do you have specific use cases in mind? If there's a restricted form we could
use to support them (e.g., adjusting cardinality, as in your example), that
would be easier for us to support and for users to reason about. Since we don't
have applications that use placement constraints yet, it may be difficult for
us to predict where they need to change during execution (if at all).
bq. Regarding to semantics, I prefer to apply to all containers placed
subsequently, this is also the closest behavior of existing YARN. We just need
to verify updated placement request is still valid, probably we don't need to
restricted to some parameters.
I don't have a clear definition of validity across placement requests,
particularly preserving it across a sequence of updates to the constraints. We
could support relaxations of existing constraints, probably. Still, updates
also require the LRA scheduler to maintain lineage for all its internal
structures. A likely implementation will convert users' expressions to some
normal form, combine those with admin constraints, forecast future allocations,
inject requests into the scheduler, etc. Even if we could offer well-defined
semantics for updates, the implementation and maintenance cost could outweigh
the marginal benefit to users. If the workarounds (like submitting a new
application or a new set of constraints) are easier to understand, that's
probably what users will prefer, anyway.
Placement constraint updates also compound the {{ResourceRequest}} problem you
cited in YARN-6594. Which epoch of the placement constraints applied to a
container returned by the RM, and for which RR? If a user's application isn't
getting containers, how is that debugged? If someone wants to reason about a
group of constraints for a production cluster while applications change clauses
programmatically at runtime, then that analysis goes from difficult to
intractable.
You guys are implementing it, but I'd push this to future work.
> [API] Introduce Placement Constraint object
> -------------------------------------------
>
> Key: YARN-6593
> URL: https://issues.apache.org/jira/browse/YARN-6593
> Project: Hadoop YARN
> Issue Type: Sub-task
> Reporter: Konstantinos Karanasos
> Assignee: Konstantinos Karanasos
> Fix For: 3.0.0-alpha3
>
> Attachments: YARN-6593.001.patch, YARN-6593.002.patch,
> YARN-6593.003.patch, YARN-6593.004.patch
>
>
> This JIRA introduces an object for defining placement constraints.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
