[jira] [Comment Edited] (YARN-8007) Support specifying placement constraint for task containers in SLS
[
https://issues.apache.org/jira/browse/YARN-8007?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16389557#comment-16389557
]
Weiwei Yang edited comment on YARN-8007 at 3/7/18 1:37 PM:
---
Hi [~yangjiandan]
Thanks for initiating this, it makes a lot of sense to help test scheduler perf
with placement constraints,
There are a few general comments to your patch
# Instead of using {{SourceTags}} abstraction, lets follow the public API in
scheduling request, which is to provide a Set of source tags and a
PlacementConstraint. I think you will need 2 configs,
allocation_tags: foo,bar
placement_constraint: notin,node,foo
this way we can honor number of tasks in the config.
# I don't think we need to add placement constraint field in
{{ContainerSimulator}}, a placement constraint is request level, not container
level.
# Have you tried to run this and how this works out?
Thanks
was (Author: cheersyang):
Hi [~yangjiandan]
Thanks for initiating this, it makes a lot of sense to help test scheduler perf
with placement constraints,
There are a few general comments to your patch
# Instead of using {{SourceTags}} abstraction, lets follow the public API in
scheduling request, which is to provide a Set of source tags and a
PlacementConstraint. I think you will need 2 configs,
allocation_tags: foo,bar
placement_constraint: notin,node,foo
this way we can honor number of tasks in the config.
# I don't think we need to add placement constraint field in
ContainerSimulator, a placement constraint is request level, not container
level.
# Have you tried to run this and how this works out?
Thanks
> Support specifying placement constraint for task containers in SLS
> --
>
> Key: YARN-8007
> URL: https://issues.apache.org/jira/browse/YARN-8007
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: scheduler-load-simulator
>Reporter: Jiandan Yang
>Assignee: Jiandan Yang
>Priority: Major
> Attachments: YARN-8007.001.patch
>
>
> YARN-6592 introduces placement constraint. Currently SLS does not support
> specify placement constraint.
> In order to help better perf test, we should be able to support specify
> placement for containers in sls configuration.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (YARN-8007) Support specifying placement constraint for task containers in SLS
[ https://issues.apache.org/jira/browse/YARN-8007?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16389181#comment-16389181 ] Jiandan Yang edited comment on YARN-8007 at 3/7/18 9:03 AM: - Because currently MR specify placement constraints in the form of data locality, there is no need to specify placement constraint in MR job. I plan to implement specifying placement constraints in *generic/stream type* of SYNTH JSON format (introduced by YARN-7732) . [~cheersyang], [~leftnoteasy] Please feel free to share your opinions, thanks! was (Author: yangjiandan): Because currently MR specify placement constraints in the form of data locality, there is no need to specify placement constraint in MR job. I plan to implement specifying placement constraints in *generic/stream type* of SYNTH JSON format (introduced by YARN-7732) . [~cheersyang] [~leftnoteasy] Please feel free to share your opinions, thanks! > Support specifying placement constraint for task containers in SLS > -- > > Key: YARN-8007 > URL: https://issues.apache.org/jira/browse/YARN-8007 > Project: Hadoop YARN > Issue Type: Sub-task > Components: scheduler-load-simulator >Reporter: Jiandan Yang >Assignee: Jiandan Yang >Priority: Major > > YARN-6592 introduces placement constraint. Currently SLS does not support > specify placement constraint. > In order to help better perf test, we should be able to support specify > placement for containers in sls configuration. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (YARN-8007) Support specifying placement constraint for task containers in SLS
[ https://issues.apache.org/jira/browse/YARN-8007?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16389181#comment-16389181 ] Jiandan Yang edited comment on YARN-8007 at 3/7/18 7:43 AM: - Because currently MR specify placement constraints in the form of data locality, there is no need to specify placement constraint in MR job. I plan to implement specifying placement constraints in *generic/stream type* of SYNTH JSON format (introduced by YARN-7732) . [~cheersyang] [~leftnoteasy] Please feel free to share your opinions, thanks! was (Author: yangjiandan): Because currently MR specify placement constraints in the form of data locality, there is no need to specify placement constraint in MR job. I plan to implement specifying placement constraints in *generic/stream type* app in SLS introduced by YARN-7732. [~cheersyang] [~leftnoteasy] Please feel free to share your opinions, thanks! > Support specifying placement constraint for task containers in SLS > -- > > Key: YARN-8007 > URL: https://issues.apache.org/jira/browse/YARN-8007 > Project: Hadoop YARN > Issue Type: Sub-task > Components: scheduler-load-simulator >Reporter: Jiandan Yang >Assignee: Jiandan Yang >Priority: Major > > YARN-6592 introduces placement constraint. Currently SLS does not support > specify placement constraint. > In order to help better perf test, we should be able to support specify > placement for containers in sls configuration. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (YARN-8007) Support specifying placement constraint for task containers in SLS
[ https://issues.apache.org/jira/browse/YARN-8007?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16389181#comment-16389181 ] Jiandan Yang edited comment on YARN-8007 at 3/7/18 7:37 AM: - Because currently MR specify placement constraints in the form of data locality, there is no need to specify placement constraint in MR job. I plan to implement specifying placement constraints in *generic/stream type* app in SLS introduced by YARN-7732. [~cheersyang] [~leftnoteasy] Please feel free to share your opinions, thanks! was (Author: yangjiandan): Because currently MR specify placement constraints in the form of data locality, there is no need to specify placement constraint in MR job. I plan to implement specifying placement constraints in *stream type* app in SLS introduced by YARN-7732. [~cheersyang] [~leftnoteasy] Please feel free to share your opinions, thanks! > Support specifying placement constraint for task containers in SLS > -- > > Key: YARN-8007 > URL: https://issues.apache.org/jira/browse/YARN-8007 > Project: Hadoop YARN > Issue Type: Sub-task > Components: scheduler-load-simulator >Reporter: Jiandan Yang >Assignee: Jiandan Yang >Priority: Major > > YARN-6592 introduces placement constraint. Currently SLS does not support > specify placement constraint. > In order to help better perf test, we should be able to support specify > placement for containers in sls configuration. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
