[jira] [Commented] (YARN-7822) Constraint satisfaction checker support for composite OR and AND constraints
[
https://issues.apache.org/jira/browse/YARN-7822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16347250#comment-16347250
]
Arun Suresh commented on YARN-7822:
---
Cherry-picked to trunk and deleted branch YARN-7812
> Constraint satisfaction checker support for composite OR and AND constraints
>
>
> Key: YARN-7822
> URL: https://issues.apache.org/jira/browse/YARN-7822
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Arun Suresh
>Assignee: Weiwei Yang
>Priority: Major
> Fix For: 3.1.0
>
> Attachments: YARN-7822-YARN-6592.001.patch,
> YARN-7822-YARN-6592.002.patch, YARN-7822-YARN-6592.003.patch,
> YARN-7822-YARN-6592.004.patch, YARN-7822-YARN-6592.005.patch,
> YARN-7822-YARN-6592.006.patch
>
>
> JIRA to track changes to {{PlacementConstraintsUtil#canSatisfyConstraints}}
> handle OR and AND Composite constaints
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7822) Constraint satisfaction checker support for composite OR and AND constraints
[
https://issues.apache.org/jira/browse/YARN-7822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16347161#comment-16347161
]
Hudson commented on YARN-7822:
--
SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #13590 (See
[https://builds.apache.org/job/Hadoop-trunk-Commit/13590/])
YARN-7822. Constraint satisfaction checker support for composite OR and (arun
suresh: rev d4813447831770446399f2d6501860141551ff33)
* (edit)
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/api/resource/TestPlacementConstraintTransformations.java
* (edit)
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/constraint/TestPlacementProcessor.java
* (edit)
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/constraint/TestPlacementConstraintsUtil.java
* (edit)
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/constraint/PlacementConstraintsUtil.java
> Constraint satisfaction checker support for composite OR and AND constraints
>
>
> Key: YARN-7822
> URL: https://issues.apache.org/jira/browse/YARN-7822
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Arun Suresh
>Assignee: Weiwei Yang
>Priority: Major
> Fix For: YARN-7812
>
> Attachments: YARN-7822-YARN-6592.001.patch,
> YARN-7822-YARN-6592.002.patch, YARN-7822-YARN-6592.003.patch,
> YARN-7822-YARN-6592.004.patch, YARN-7822-YARN-6592.005.patch,
> YARN-7822-YARN-6592.006.patch
>
>
> JIRA to track changes to {{PlacementConstraintsUtil#canSatisfyConstraints}}
> handle OR and AND Composite constaints
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7822) Constraint satisfaction checker support for composite OR and AND constraints
[
https://issues.apache.org/jira/browse/YARN-7822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16345944#comment-16345944
]
Weiwei Yang commented on YARN-7822:
---
Thanks [~asuresh], that sounds nice to me.
> Constraint satisfaction checker support for composite OR and AND constraints
>
>
> Key: YARN-7822
> URL: https://issues.apache.org/jira/browse/YARN-7822
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Arun Suresh
>Assignee: Weiwei Yang
>Priority: Major
> Fix For: YARN-7812
>
> Attachments: YARN-7822-YARN-6592.001.patch,
> YARN-7822-YARN-6592.002.patch, YARN-7822-YARN-6592.003.patch,
> YARN-7822-YARN-6592.004.patch, YARN-7822-YARN-6592.005.patch,
> YARN-7822-YARN-6592.006.patch
>
>
> JIRA to track changes to {{PlacementConstraintsUtil#canSatisfyConstraints}}
> handle OR and AND Composite constaints
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7822) Constraint satisfaction checker support for composite OR and AND constraints
[
https://issues.apache.org/jira/browse/YARN-7822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16345523#comment-16345523
]
Arun Suresh commented on YARN-7822:
---
+1 Looks good to me. Thanks for the update [~cheersyang].
I think I will commit this to a temporary YARN-7812 branch (forked from the
current YARN-6592). Once we merge YARN-6592 onto trunk, I will cherry-pick this
to trunk and close the temp YARN-7812 branch. Given that we have only a single
day for the merge vote to end, I don't think anything else from YARN-7812
umbrella will go in - in which case, we can avoid a merge vote (since we are
cherry-picking a single JIRA)
> Constraint satisfaction checker support for composite OR and AND constraints
>
>
> Key: YARN-7822
> URL: https://issues.apache.org/jira/browse/YARN-7822
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Arun Suresh
>Assignee: Weiwei Yang
>Priority: Major
> Attachments: YARN-7822-YARN-6592.001.patch,
> YARN-7822-YARN-6592.002.patch, YARN-7822-YARN-6592.003.patch,
> YARN-7822-YARN-6592.004.patch, YARN-7822-YARN-6592.005.patch,
> YARN-7822-YARN-6592.006.patch
>
>
> JIRA to track changes to {{PlacementConstraintsUtil#canSatisfyConstraints}}
> handle OR and AND Composite constaints
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7822) Constraint satisfaction checker support for composite OR and AND constraints
[
https://issues.apache.org/jira/browse/YARN-7822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16345364#comment-16345364
]
genericqa commented on YARN-7822:
-
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
18s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 3 new or modified test
files. {color} |
|| || || || {color:brown} YARN-6592 Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
11s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 15m
27s{color} | {color:green} YARN-6592 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 7m
26s{color} | {color:green} YARN-6592 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
59s{color} | {color:green} YARN-6592 passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
34s{color} | {color:green} YARN-6592 passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
12m 5s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
15s{color} | {color:green} YARN-6592 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
4s{color} | {color:green} YARN-6592 passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
9s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
9s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 6m
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 6m
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
49s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
15s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
9m 22s{color} | {color:green} patch has no errors when building and testing our
client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
29s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
6s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 3m
2s{color} | {color:green} hadoop-yarn-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 67m
19s{color} | {color:green} hadoop-yarn-server-resourcemanager in the patch
passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
34s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}133m 47s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:5b98639 |
| JIRA Issue | YARN-7822 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12908339/YARN-7822-YARN-6592.006.patch
|
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit shadedclient findbugs checkstyle |
| uname | Linux dd124e839c5e 4.4.0-64-generic #85-Ubuntu SMP Mon Feb 20
11:50:30 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | YARN-6592 / 00e028e |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_151 |
| findbugs | v3.1.0-RC1 |
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/19525/testReport/ |
| Max. process+thread count
[jira] [Commented] (YARN-7822) Constraint satisfaction checker support for composite OR and AND constraints
[
https://issues.apache.org/jira/browse/YARN-7822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16345106#comment-16345106
]
Weiwei Yang commented on YARN-7822:
---
Not sure what happens to the jenkins, re-submit the patch (by copying over to
v6) to trigger the jenkins build.
> Constraint satisfaction checker support for composite OR and AND constraints
>
>
> Key: YARN-7822
> URL: https://issues.apache.org/jira/browse/YARN-7822
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Arun Suresh
>Assignee: Weiwei Yang
>Priority: Major
> Attachments: YARN-7822-YARN-6592.001.patch,
> YARN-7822-YARN-6592.002.patch, YARN-7822-YARN-6592.003.patch,
> YARN-7822-YARN-6592.004.patch, YARN-7822-YARN-6592.005.patch,
> YARN-7822-YARN-6592.006.patch
>
>
> JIRA to track changes to {{PlacementConstraintsUtil#canSatisfyConstraints}}
> handle OR and AND Composite constaints
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7822) Constraint satisfaction checker support for composite OR and AND constraints
[
https://issues.apache.org/jira/browse/YARN-7822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16345057#comment-16345057
]
genericqa commented on YARN-7822:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
0s{color} | {color:blue} Docker mode activated. {color} |
| {color:red}-1{color} | {color:red} docker {color} | {color:red} 5m
14s{color} | {color:red} Docker failed to build yetus/hadoop:5b98639. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | YARN-7822 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12908334/YARN-7822-YARN-6592.005.patch
|
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/19524/console |
| Powered by | Apache Yetus 0.8.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> Constraint satisfaction checker support for composite OR and AND constraints
>
>
> Key: YARN-7822
> URL: https://issues.apache.org/jira/browse/YARN-7822
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Arun Suresh
>Assignee: Weiwei Yang
>Priority: Major
> Attachments: YARN-7822-YARN-6592.001.patch,
> YARN-7822-YARN-6592.002.patch, YARN-7822-YARN-6592.003.patch,
> YARN-7822-YARN-6592.004.patch, YARN-7822-YARN-6592.005.patch
>
>
> JIRA to track changes to {{PlacementConstraintsUtil#canSatisfyConstraints}}
> handle OR and AND Composite constaints
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7822) Constraint satisfaction checker support for composite OR and AND constraints
[
https://issues.apache.org/jira/browse/YARN-7822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16345048#comment-16345048
]
Weiwei Yang commented on YARN-7822:
---
Hi [~kkaranasos]
Attached v5 patch that basically supports all nested forms of constraints, it's
relative easy as when we iterate over the constraint tree, we only need to
compute child constraint result to determine if the composite constraint is
satisfied or not. So it's like
{code}
//Cn can be either a single constraint (target expression, cardinality
expression etc) or a composite constraint (AND or OR)
[AND|OR](C1, ..., Cn)
{code}
Also added test case in {{TestPlacementConstraintsUtil}}.
Please take a look. Thanks.
> Constraint satisfaction checker support for composite OR and AND constraints
>
>
> Key: YARN-7822
> URL: https://issues.apache.org/jira/browse/YARN-7822
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Arun Suresh
>Assignee: Weiwei Yang
>Priority: Major
> Attachments: YARN-7822-YARN-6592.001.patch,
> YARN-7822-YARN-6592.002.patch, YARN-7822-YARN-6592.003.patch,
> YARN-7822-YARN-6592.004.patch, YARN-7822-YARN-6592.005.patch
>
>
> JIRA to track changes to {{PlacementConstraintsUtil#canSatisfyConstraints}}
> handle OR and AND Composite constaints
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7822) Constraint satisfaction checker support for composite OR and AND constraints
[
https://issues.apache.org/jira/browse/YARN-7822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16343897#comment-16343897
]
Konstantinos Karanasos commented on YARN-7822:
--
Patch looks good, thanks [~cheersyang].
I have only one ask, which I don't think is a big change. Can we support DNF?
This is just ORs of ANDs. This means that we can allow the following:
* AND constraints with no nesting (your patch does that already);
* OR constraints, each being either a simple constraint or an AND constraint
with no nesting. So essentially, you only have to allow this additional case in
your patch.
If we do this, then we can transform any compound constraint with any nesting
level to an equivalent DNF form (we can tackle this in a separate JIRA), and we
will be done with the canSatisfyConstraint for all possible constraints.
Regarding the max cardinality, you are right it is a bit confusing in the
design doc, but it is not a code implementation artifact.
The idea is that the constraint has to be satisfied the moment of scheduling:
* When the source and target tags are different, this happens to be the same
both before and after the placement. Assume a constraint that hb-m should be
placed at a node with at most 2 hb-rs. Say node n1 has before scheduling 2
hb-rs, so we can place hb-m there. After placement, it will still have 2 hb-rs.
* Now when the source and target tags are the same, it is more complicated.
Assume a constraint that hb-rs should be placed at a node with at most 2 hb-rs.
Say node n1 has again 2 hb-rs before scheduling. After scheduling, it will have
3.
So to unify the two cases above, we say that the constraints should be
satisfied before the placement of the new container happens. As [~asuresh]
mentioned, at some point we tried to create a special case for when
source==target, but it the semantics were not straightforward (e.g., if a user
was saying max cardinality 2, they were expecting 2 after placement; when a
user says max cardinality 0, they think anti-affinity, but it actually has to
be 1 if we consider cardinality after placement). To unify things, we opted for
cardinality check before placement in all cases.
> Constraint satisfaction checker support for composite OR and AND constraints
>
>
> Key: YARN-7822
> URL: https://issues.apache.org/jira/browse/YARN-7822
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Arun Suresh
>Assignee: Weiwei Yang
>Priority: Major
> Attachments: YARN-7822-YARN-6592.001.patch,
> YARN-7822-YARN-6592.002.patch, YARN-7822-YARN-6592.003.patch,
> YARN-7822-YARN-6592.004.patch
>
>
> JIRA to track changes to {{PlacementConstraintsUtil#canSatisfyConstraints}}
> handle OR and AND Composite constaints
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7822) Constraint satisfaction checker support for composite OR and AND constraints
[
https://issues.apache.org/jira/browse/YARN-7822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16343449#comment-16343449
]
Weiwei Yang commented on YARN-7822:
---
[~asuresh], [~kkaranasos], how about v4 patch looks to you?
> Constraint satisfaction checker support for composite OR and AND constraints
>
>
> Key: YARN-7822
> URL: https://issues.apache.org/jira/browse/YARN-7822
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Arun Suresh
>Assignee: Weiwei Yang
>Priority: Major
> Attachments: YARN-7822-YARN-6592.001.patch,
> YARN-7822-YARN-6592.002.patch, YARN-7822-YARN-6592.003.patch,
> YARN-7822-YARN-6592.004.patch
>
>
> JIRA to track changes to {{PlacementConstraintsUtil#canSatisfyConstraints}}
> handle OR and AND Composite constaints
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7822) Constraint satisfaction checker support for composite OR and AND constraints
[
https://issues.apache.org/jira/browse/YARN-7822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16343244#comment-16343244
]
genericqa commented on YARN-7822:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
34s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 3 new or modified test
files. {color} |
|| || || || {color:brown} YARN-6592 Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
58s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 16m
20s{color} | {color:green} YARN-6592 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 7m
43s{color} | {color:green} YARN-6592 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
50s{color} | {color:green} YARN-6592 passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
25s{color} | {color:green} YARN-6592 passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
10m 54s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
13s{color} | {color:green} YARN-6592 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
3s{color} | {color:green} YARN-6592 passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
10s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
10s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 6m
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 6m
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
49s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
20s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
9m 50s{color} | {color:green} patch has no errors when building and testing our
client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
32s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
5s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 3m
5s{color} | {color:green} hadoop-yarn-common in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 77m 26s{color}
| {color:red} hadoop-yarn-server-resourcemanager in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
30s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}145m 48s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.yarn.server.resourcemanager.TestRMEmbeddedElector
|
| |
hadoop.yarn.server.resourcemanager.scheduler.constraint.TestPlacementProcessor |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:5b98639 |
| JIRA Issue | YARN-7822 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12908115/YARN-7822-YARN-6592.004.patch
|
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit shadedclient findbugs checkstyle |
| uname | Linux 5e7f52067a70 4.4.0-89-generic #112-Ubuntu SMP Mon Jul 31
19:38:41 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | YARN-6592 / 00e028e |
| maven | version:
[jira] [Commented] (YARN-7822) Constraint satisfaction checker support for composite OR and AND constraints
[
https://issues.apache.org/jira/browse/YARN-7822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16343108#comment-16343108
]
Weiwei Yang commented on YARN-7822:
---
Thanks [~asuresh]
{quote}Maybe instead of stating in the doc that "we do not allow more than max
target tags at the node", we should re-word it as "placement will be made only
if, at the time evaluation, only UPTO a max of target tags exist on the node
{quote}
Thanks for the explanation, I don't fully get it but I think this is something
specifically related to the code. In this case, I agree we should make sure the
doc up-to-date. Just took a look at the doc in YARN-7780, it still states like
{noformat}
For example, \{@code cardinality(NODE, 3, 10, "zk")} is satisfied on nodes
where there are no less than 3 allocations with tag "zk" and no more than 10.
{noformat}
v4 patch uploaded, fixed checkstyle and removed FIXME. The failed test case is
not related to this patch, it runs cleanly on my local env.
Please take a look.
Thanks.
> Constraint satisfaction checker support for composite OR and AND constraints
>
>
> Key: YARN-7822
> URL: https://issues.apache.org/jira/browse/YARN-7822
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Arun Suresh
>Assignee: Weiwei Yang
>Priority: Major
> Attachments: YARN-7822-YARN-6592.001.patch,
> YARN-7822-YARN-6592.002.patch, YARN-7822-YARN-6592.003.patch,
> YARN-7822-YARN-6592.004.patch
>
>
> JIRA to track changes to {{PlacementConstraintsUtil#canSatisfyConstraints}}
> handle OR and AND Composite constaints
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7822) Constraint satisfaction checker support for composite OR and AND constraints
[
https://issues.apache.org/jira/browse/YARN-7822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16343062#comment-16343062
]
genericqa commented on YARN-7822:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
14s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 3 new or modified test
files. {color} |
|| || || || {color:brown} YARN-6592 Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
52s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 17m
20s{color} | {color:green} YARN-6592 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 8m
17s{color} | {color:green} YARN-6592 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
1s{color} | {color:green} YARN-6592 passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
33s{color} | {color:green} YARN-6592 passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
12m 51s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
30s{color} | {color:green} YARN-6592 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
17s{color} | {color:green} YARN-6592 passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
10s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
18s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 6m
59s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 6m
59s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
1m 3s{color} | {color:orange} hadoop-yarn-project/hadoop-yarn: The patch
generated 8 new + 20 unchanged - 0 fixed = 28 total (was 20) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
37s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
11m 13s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
44s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
15s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 3m
6s{color} | {color:green} hadoop-yarn-common in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 65m 8s{color}
| {color:red} hadoop-yarn-server-resourcemanager in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
43s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}140m 26s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests |
hadoop.yarn.server.resourcemanager.scheduler.constraint.TestPlacementProcessor |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:5b98639 |
| JIRA Issue | YARN-7822 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12908099/YARN-7822-YARN-6592.003.patch
|
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit shadedclient findbugs checkstyle |
| uname | Linux 24452f2497ea 3.13.0-135-generic #184-Ubuntu SMP Wed Oct 18
11:55:51 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | YARN-6592 /
[jira] [Commented] (YARN-7822) Constraint satisfaction checker support for composite OR and AND constraints
[
https://issues.apache.org/jira/browse/YARN-7822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16342976#comment-16342976
]
Arun Suresh commented on YARN-7822:
---
Thanks for updating the patch [~cheersyang].
I was just thinking - You should maybe try to implement the checker as a
{{PlacementConstraint.Visitor}}.
But yeah, for simple AND/OR, it might be an over-kill, so once we get to nested
AND/OR, we can use a Visitor.
Thanks for the adding the new tests.
+1 pending jenkins
> Constraint satisfaction checker support for composite OR and AND constraints
>
>
> Key: YARN-7822
> URL: https://issues.apache.org/jira/browse/YARN-7822
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Arun Suresh
>Assignee: Weiwei Yang
>Priority: Major
> Attachments: YARN-7822-YARN-6592.001.patch,
> YARN-7822-YARN-6592.002.patch, YARN-7822-YARN-6592.003.patch
>
>
> JIRA to track changes to {{PlacementConstraintsUtil#canSatisfyConstraints}}
> handle OR and AND Composite constaints
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7822) Constraint satisfaction checker support for composite OR and AND constraints
[
https://issues.apache.org/jira/browse/YARN-7822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16342958#comment-16342958
]
Arun Suresh commented on YARN-7822:
---
[~cheersyang], Thanks for the updated patch.
bq. I am expecting n1 cannot satisfy the constraint as it already has 3 spark
containers (reach the max cardinality). However, it passes... allocate 1 more
spark container onto n3 and then the check returns false...
Yes, so we did modify the implementation a bit from the design-doc. The reason
is to accommodate the situation when the source tag is equal to the target tag.
If we go by the original interpretation, then the placement checker will have
to first see if source tag == target tag, and in that case, evaluate the max
cardinality with a '<=' and for all other cases, it would have to evaluate it
with a '<'. We wanted to unify both cases without having to make the code ugly
- which is why we settled on using '<=' for all cases.
Maybe instead of stating in the doc that "we do not allow more than max target
tags at the node", we should re-word it as "placement will be made only if, at
the time evaluation, only UPTO a max of target tags exist on the node"
> Constraint satisfaction checker support for composite OR and AND constraints
>
>
> Key: YARN-7822
> URL: https://issues.apache.org/jira/browse/YARN-7822
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Arun Suresh
>Assignee: Weiwei Yang
>Priority: Major
> Attachments: YARN-7822-YARN-6592.001.patch,
> YARN-7822-YARN-6592.002.patch, YARN-7822-YARN-6592.003.patch
>
>
> JIRA to track changes to {{PlacementConstraintsUtil#canSatisfyConstraints}}
> handle OR and AND Composite constaints
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7822) Constraint satisfaction checker support for composite OR and AND constraints
[
https://issues.apache.org/jira/browse/YARN-7822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16342937#comment-16342937
]
genericqa commented on YARN-7822:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
0s{color} | {color:blue} Docker mode activated. {color} |
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 4s{color}
| {color:red} YARN-7822 does not apply to YARN-6592. Rebase required? Wrong
Branch? See https://wiki.apache.org/hadoop/HowToContribute for help. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | YARN-7822 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12908096/YARN-7822-YARN-6592.002.patch
|
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/19501/console |
| Powered by | Apache Yetus 0.8.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> Constraint satisfaction checker support for composite OR and AND constraints
>
>
> Key: YARN-7822
> URL: https://issues.apache.org/jira/browse/YARN-7822
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Arun Suresh
>Assignee: Weiwei Yang
>Priority: Major
> Attachments: YARN-7822-YARN-6592.001.patch,
> YARN-7822-YARN-6592.002.patch
>
>
> JIRA to track changes to {{PlacementConstraintsUtil#canSatisfyConstraints}}
> handle OR and AND Composite constaints
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7822) Constraint satisfaction checker support for composite OR and AND constraints
[
https://issues.apache.org/jira/browse/YARN-7822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16342933#comment-16342933
]
Weiwei Yang commented on YARN-7822:
---
[~kkaranasos], [~asuresh], please see v2 patch
{quote}Can you replace with SchedulerNode
{quote}
Done.
{quote}validate an end-2-end scenario ?
{quote}
Done. Added \{{TestPlacementProcessor#testAndOrPlacement}} for end-2-end
testing.
{quote}so you would have to make your maxCard=2 to have at most 3
{quote}
I still have some confusions like I posted in [this earlier
comment|https://issues.apache.org/jira/browse/YARN-7822?focusedCommentId=16341907=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16341907],
[~kkaranasos] please take a look.
Thanks
> Constraint satisfaction checker support for composite OR and AND constraints
>
>
> Key: YARN-7822
> URL: https://issues.apache.org/jira/browse/YARN-7822
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Arun Suresh
>Assignee: Weiwei Yang
>Priority: Major
> Attachments: YARN-7822-YARN-6592.001.patch,
> YARN-7822-YARN-6592.002.patch
>
>
> JIRA to track changes to {{PlacementConstraintsUtil#canSatisfyConstraints}}
> handle OR and AND Composite constaints
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7822) Constraint satisfaction checker support for composite OR and AND constraints
[
https://issues.apache.org/jira/browse/YARN-7822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16341960#comment-16341960
]
Arun Suresh commented on YARN-7822:
---
bq. ..thought it might be better not support nested case at first. We can
further investigate nested case moving on.
Makes sense - we can tackle that later.
bq. Do you want me to replace all FiCaSchedulerNode in
TestPlacementConstraintsUtil in next patch?
That would be ideal - thanks.
> Constraint satisfaction checker support for composite OR and AND constraints
>
>
> Key: YARN-7822
> URL: https://issues.apache.org/jira/browse/YARN-7822
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Arun Suresh
>Assignee: Weiwei Yang
>Priority: Major
> Attachments: YARN-7822-YARN-6592.001.patch
>
>
> JIRA to track changes to {{PlacementConstraintsUtil#canSatisfyConstraints}}
> handle OR and AND Composite constaints
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7822) Constraint satisfaction checker support for composite OR and AND constraints
[
https://issues.apache.org/jira/browse/YARN-7822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16341907#comment-16341907
]
Weiwei Yang commented on YARN-7822:
---
Hi [~kkaranasos]
{quote}
I had mentioned this case in a comment somewhere.
{quote}
Yes, you mentioned this in [this
comment|https://issues.apache.org/jira/browse/YARN-7774?focusedCommentId=16331286=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16331286]
and I saw that, but still was confusing. From the design doc, a constraint like
{noformat}
{max-cardinality: 3, scope: host}
{noformat}
this will be interpreted to "don't allocate more than 3 containers per node".
However, what you were saying was I need to specify
{noformat}
{max-cardinality: 2, scope: host}
{noformat}
to achieve not placing more than 3 containers on a node. Is that correct?
[~asuresh] regarding to your comments
bq. nested AND / OR is not supported ... it should be possible right ?
I meant to take this step by step. I saw some existing test case had the same
assumption, e.g
{{TestPlacementConstraintTransformations#testCompositeConstraint}} so thought
it might be better not support nested case at first. We can further investigate
nested case moving on.
bq. Can you replace with SchedulerNode
Sure, will try in next patch. Do you want me to replace all
{{FiCaSchedulerNode}} in {{TestPlacementConstraintsUtil}} in next patch?
bq. validate an end-2-end scenario ?
Will do.
Thanks
> Constraint satisfaction checker support for composite OR and AND constraints
>
>
> Key: YARN-7822
> URL: https://issues.apache.org/jira/browse/YARN-7822
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Arun Suresh
>Assignee: Weiwei Yang
>Priority: Major
> Attachments: YARN-7822-YARN-6592.001.patch
>
>
> JIRA to track changes to {{PlacementConstraintsUtil#canSatisfyConstraints}}
> handle OR and AND Composite constaints
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7822) Constraint satisfaction checker support for composite OR and AND constraints
[
https://issues.apache.org/jira/browse/YARN-7822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16341468#comment-16341468
]
Arun Suresh commented on YARN-7822:
---
[~cheersyang], Thanks for working on this..
Couple of comments:
* I see a TODO stating that nested AND / OR is not supported. Given that the
the children of OR / AND can take any AbstractConstraint, it should be possible
right ?
* In your testcases, I see references to {{FiCaSchedulerNode}}. Can you replace
with SchedulerNode. We would like to keep this module as scheduler agnostic as
possible.
* Can you also add a testcase to the {{TestPlacementProcessor}} to validate an
end-2-end scenario ?
I was thinking we make changes to the DistributedShell as well. Should we do it
here ? or I am open to doing it in another JIRA.
> Constraint satisfaction checker support for composite OR and AND constraints
>
>
> Key: YARN-7822
> URL: https://issues.apache.org/jira/browse/YARN-7822
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Arun Suresh
>Assignee: Weiwei Yang
>Priority: Major
> Attachments: YARN-7822-YARN-6592.001.patch
>
>
> JIRA to track changes to {{PlacementConstraintsUtil#canSatisfyConstraints}}
> handle OR and AND Composite constaints
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7822) Constraint satisfaction checker support for composite OR and AND constraints
[
https://issues.apache.org/jira/browse/YARN-7822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16341147#comment-16341147
]
Konstantinos Karanasos commented on YARN-7822:
--
Hi [~cheersyang], so maxCardinality does a check of <= at the state of the
cluster before the allocation that is currently being attempted. In your case,
you have maxCard=3, so n1 is a valid node as it has 3. Indeed after the
allocation it will end up having 4, so you would have to make your maxCard=2 to
have at most 3, since we are looking at the pre-allocation state. I had
mentioned this case in a comment somewhere.
> Constraint satisfaction checker support for composite OR and AND constraints
>
>
> Key: YARN-7822
> URL: https://issues.apache.org/jira/browse/YARN-7822
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Arun Suresh
>Assignee: Weiwei Yang
>Priority: Major
> Attachments: YARN-7822-YARN-6592.001.patch
>
>
> JIRA to track changes to {{PlacementConstraintsUtil#canSatisfyConstraints}}
> handle OR and AND Composite constaints
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7822) Constraint satisfaction checker support for composite OR and AND constraints
[
https://issues.apache.org/jira/browse/YARN-7822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16341082#comment-16341082
]
genericqa commented on YARN-7822:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
0s{color} | {color:blue} Docker mode activated. {color} |
| {color:red}-1{color} | {color:red} docker {color} | {color:red} 7m
5s{color} | {color:red} Docker failed to build yetus/hadoop:5b98639. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | YARN-7822 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12907881/YARN-7822-YARN-6592.001.patch
|
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/19492/console |
| Powered by | Apache Yetus 0.8.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> Constraint satisfaction checker support for composite OR and AND constraints
>
>
> Key: YARN-7822
> URL: https://issues.apache.org/jira/browse/YARN-7822
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Arun Suresh
>Assignee: Weiwei Yang
>Priority: Major
> Attachments: YARN-7822-YARN-6592.001.patch
>
>
> JIRA to track changes to {{PlacementConstraintsUtil#canSatisfyConstraints}}
> handle OR and AND Composite constaints
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7822) Constraint satisfaction checker support for composite OR and AND constraints
[
https://issues.apache.org/jira/browse/YARN-7822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16341068#comment-16341068
]
Weiwei Yang commented on YARN-7822:
---
Submitted v1 patch targeting to YARN-6592 branch. It seems pretty
straightforward to implement OR/AND composite constraints with existing API.
However I am confused with one thing. Like what marked with {{FIXME}} in
{{TestPlacementConstraintsUtil#testANDConstraintAssignment}}.
Basically I specified a constraint
{code:java}
// Place a "spark" allocation to a node doesn't have "hbase-m" tag and at most
3 allocations on each node
and(targetNotIn(NODE, allocationTag("hbase-m")), maxCardinality(NODE, 3,
"spark"))
{code}
however, during the test, I've allocated 3 "spark" containers over a node, so
it looks like
{noformat}
n0: hbase-m(1)
n1: spark(3)
n2: hbase-m(1)
n3: ""
{noformat}
I am expecting n1 cannot satisfy the constraint as it already has 3 spark
containers (reach the max cardinality). However, it passes... allocate 1 more
spark container onto n3 and then the check returns false...
I think I am confused by this comment ...
{quote}For example, if we are placing a scheduling request with tag {{hb-m}}
and the constraint dictates that there should be no more than two {{hb-rs}} in
a node, then if there are two, placement should happen.
{quote}
[~asuresh], [~kkaranasos], can you please give me some hints?
Thanks
> Constraint satisfaction checker support for composite OR and AND constraints
>
>
> Key: YARN-7822
> URL: https://issues.apache.org/jira/browse/YARN-7822
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Arun Suresh
>Assignee: Weiwei Yang
>Priority: Major
> Attachments: YARN-7822-YARN-6592.001.patch
>
>
> JIRA to track changes to {{PlacementConstraintsUtil#canSatisfyConstraints}}
> handle OR and AND Composite constaints
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
