[jira] [Commented] (YARN-7921) Transform a PlacementConstraint to a string expression
[
https://issues.apache.org/jira/browse/YARN-7921?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16375901#comment-16375901
]
Weiwei Yang commented on YARN-7921:
---
Sure, thanks [~kkaranasos].
> Transform a PlacementConstraint to a string expression
> --
>
> Key: YARN-7921
> URL: https://issues.apache.org/jira/browse/YARN-7921
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Weiwei Yang
>Assignee: Weiwei Yang
>Priority: Major
> Attachments: Placement Constraint Expression Syntax
> Specification.pdf, YARN-7921.001.patch, YARN-7921.002.patch
>
>
> Purpose:
> Let placement constraint viewable on UI or log, e.g print app placement
> constraint in RM app page. Help user to use constraints and analysis
> placement issues easier.
> Propose:
> Like what was added for DS, toString is a reversed process of
> {{PlacementConstraintParser}} that transforms a PlacementConstraint to a
> string, using same syntax. E.g
> {code}
> AbstractConstraint constraintExpr = targetIn(NODE, allocationTag("hbase-m"));
> constraint.toString();
> // This prints: IN,NODE,hbase-m
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7921) Transform a PlacementConstraint to a string expression
[
https://issues.apache.org/jira/browse/YARN-7921?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16375853#comment-16375853
]
Konstantinos Karanasos commented on YARN-7921:
--
{quote}from the latest discussion in YARN-3409, we plan to use DNS format for
namespace
{quote}
Okay, we are good on that front then.
{quote}we are using ":" to separate multiple child constraint in a AND or OR
constraint
{quote}
Yes, I meant using ";" there too.
I find ";" more intuitive for delimiting, but we can do it in a separate Jira
later if you prefer.
I will commit this Monday morning.
> Transform a PlacementConstraint to a string expression
> --
>
> Key: YARN-7921
> URL: https://issues.apache.org/jira/browse/YARN-7921
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Weiwei Yang
>Assignee: Weiwei Yang
>Priority: Major
> Attachments: Placement Constraint Expression Syntax
> Specification.pdf, YARN-7921.001.patch, YARN-7921.002.patch
>
>
> Purpose:
> Let placement constraint viewable on UI or log, e.g print app placement
> constraint in RM app page. Help user to use constraints and analysis
> placement issues easier.
> Propose:
> Like what was added for DS, toString is a reversed process of
> {{PlacementConstraintParser}} that transforms a PlacementConstraint to a
> string, using same syntax. E.g
> {code}
> AbstractConstraint constraintExpr = targetIn(NODE, allocationTag("hbase-m"));
> constraint.toString();
> // This prints: IN,NODE,hbase-m
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7921) Transform a PlacementConstraint to a string expression
[
https://issues.apache.org/jira/browse/YARN-7921?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16375787#comment-16375787
]
Weiwei Yang commented on YARN-7921:
---
Hi [~kkaranasos]
Yes, the spec is same as the document, we are using the parser to parse those
for DS from user input to a PC, and we are also able to parse a PC toString
back to PC like what I did in the UT. It should be consistent.
I don't think we need to reserve ":" for node attributes namespace becuase from
the latest discussion in YARN-3409, we plan to use DNS format for namespace,
such as nm.yarn.io. Namespace and attribute name will be delimited by "/".
Second, we are using ":" to separate multiple child constraint in a AND or OR
constraint, so it's better to keep it this way. What do you think?
> Transform a PlacementConstraint to a string expression
> --
>
> Key: YARN-7921
> URL: https://issues.apache.org/jira/browse/YARN-7921
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Weiwei Yang
>Assignee: Weiwei Yang
>Priority: Major
> Attachments: Placement Constraint Expression Syntax
> Specification.pdf, YARN-7921.001.patch, YARN-7921.002.patch
>
>
> Purpose:
> Let placement constraint viewable on UI or log, e.g print app placement
> constraint in RM app page. Help user to use constraints and analysis
> placement issues easier.
> Propose:
> Like what was added for DS, toString is a reversed process of
> {{PlacementConstraintParser}} that transforms a PlacementConstraint to a
> string, using same syntax. E.g
> {code}
> AbstractConstraint constraintExpr = targetIn(NODE, allocationTag("hbase-m"));
> constraint.toString();
> // This prints: IN,NODE,hbase-m
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7921) Transform a PlacementConstraint to a string expression
[
https://issues.apache.org/jira/browse/YARN-7921?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16375780#comment-16375780
]
Konstantinos Karanasos commented on YARN-7921:
--
Thanks [~cheersyang]. Patch looks good.
Specification is the same as we have in the documentation, right?
Only thing I was wondering is whether we want to use ";" instead of ":" as a
delimiter for constraints. Maybe we should keep ":" for the namespaces for node
attributes.
What do you think?
> Transform a PlacementConstraint to a string expression
> --
>
> Key: YARN-7921
> URL: https://issues.apache.org/jira/browse/YARN-7921
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Weiwei Yang
>Assignee: Weiwei Yang
>Priority: Major
> Attachments: Placement Constraint Expression Syntax
> Specification.pdf, YARN-7921.001.patch, YARN-7921.002.patch
>
>
> Purpose:
> Let placement constraint viewable on UI or log, e.g print app placement
> constraint in RM app page. Help user to use constraints and analysis
> placement issues easier.
> Propose:
> Like what was added for DS, toString is a reversed process of
> {{PlacementConstraintParser}} that transforms a PlacementConstraint to a
> string, using same syntax. E.g
> {code}
> AbstractConstraint constraintExpr = targetIn(NODE, allocationTag("hbase-m"));
> constraint.toString();
> // This prints: IN,NODE,hbase-m
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7963) TestServiceAM and TestServiceMonitor test cases are hanging
[ https://issues.apache.org/jira/browse/YARN-7963?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16375691#comment-16375691 ] Eric Yang commented on YARN-7963: - There are new methods introduced in YARN-6619, and causing the unit test (MockServiceAM.java) to call the real implementation instead of mock version of the class. This trigger the unit test to look for resource manager. > TestServiceAM and TestServiceMonitor test cases are hanging > --- > > Key: YARN-7963 > URL: https://issues.apache.org/jira/browse/YARN-7963 > Project: Hadoop YARN > Issue Type: Bug > Components: yarn-native-services >Affects Versions: 3.1.0 >Reporter: Eric Yang >Priority: Major > > There is a regression when merge YARN-6592 that prevents YARN services test > cases from working. The unit tests hang on contacting resource manager at > port 8030. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7929) SLS supports setting container execution
[
https://issues.apache.org/jira/browse/YARN-7929?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16375491#comment-16375491
]
Weiwei Yang commented on YARN-7929:
---
Hi [~yangjiandan]
Thanks for working on this, this is very helpful to eval scheduler performance
when have opportunistic containers enabled. Some comments regarding to your
patch
*ContainerSimulator*
# line 53: Can we leave the old constructor along and create a new one with
additional argument ExcutionType? This can help to minimize the changes
*NMSimulator*
# line 76: can we replace {{waterLevel}} to something like
{{resourceUtilizationRatio}} ?
# line 139: instead of having a check here, we should place a check earlier
during init, if configuration is not valid, fail directly. Maybe move to
SLSRunner#init
# line 140: please use safe cast from float to int
*SynthJob*
# Similar to the first comment, can we add a new constructor instead of
modifying the old one?
*Misc*
# It looks like you have modified syn.json, syn_generic.json and
syn_stream.json to all use opportunistic execution type, will this affect
existing UT coverage on guaranteed containers? If so, I suggest to add new test
case for this.
# Please fix checkstyle issues, there are a lot of lines exceed 80 length
Thanks
> SLS supports setting container execution
>
>
> Key: YARN-7929
> URL: https://issues.apache.org/jira/browse/YARN-7929
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: scheduler-load-simulator
>Reporter: Jiandan Yang
>Assignee: Jiandan Yang
>Priority: Major
> Attachments: YARN-7929.001.patch, YARN-7929.002.patch
>
>
> SLS currently support three tracetype, SYNTH, SLS and RUMEN, but trace file
> can not set execution type of container.
> This jira will introduce execution type in SLS to help better simulation.
> This will help the perf testing with regarding to the Opportunistic
> Containers.
> RUMEN has default execution type GUARANTEED
> SYNTH set execution type by field map_execution_type and
> reduce_execution_type
> SLS set execution type by field container.execution_type
> For compatibility set GUARANTEED as default value when not setting above
> fields in trace file
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
