[jira] [Commented] (MAPREDUCE-6926) Allow MR jobs to opt out of oversubscription
[ https://issues.apache.org/jira/browse/MAPREDUCE-6926?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16321295#comment-16321295 ] Haibo Chen commented on MAPREDUCE-6926: --- Thanks [~miklos.szeg...@cloudera.com] for your reviews! > Allow MR jobs to opt out of oversubscription > > > Key: MAPREDUCE-6926 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6926 > Project: Hadoop Map/Reduce > Issue Type: Task > Components: mrv2 >Affects Versions: 3.0.0-alpha3 >Reporter: Haibo Chen >Assignee: Haibo Chen > Fix For: YARN-1011 > > Attachments: MAPREDUCE-6926-YARN-1011.00.patch, > MAPREDUCE-6926-YARN-1011.01.patch, MAPREDUCE-6926-YARN-1011.02.patch, > MAPREDUCE-6926-YARN-1011.03.patch, MAPREDUCE-6926-YARN-1011.04.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Commented] (MAPREDUCE-6926) Allow MR jobs to opt out of oversubscription
[ https://issues.apache.org/jira/browse/MAPREDUCE-6926?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16321289#comment-16321289 ] Miklos Szegedi commented on MAPREDUCE-6926: --- +1. > Allow MR jobs to opt out of oversubscription > > > Key: MAPREDUCE-6926 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6926 > Project: Hadoop Map/Reduce > Issue Type: Task > Components: mrv2 >Affects Versions: 3.0.0-alpha3 >Reporter: Haibo Chen >Assignee: Haibo Chen > Attachments: MAPREDUCE-6926-YARN-1011.00.patch, > MAPREDUCE-6926-YARN-1011.01.patch, MAPREDUCE-6926-YARN-1011.02.patch, > MAPREDUCE-6926-YARN-1011.03.patch, MAPREDUCE-6926-YARN-1011.04.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Commented] (MAPREDUCE-6926) Allow MR jobs to opt out of oversubscription
[
https://issues.apache.org/jira/browse/MAPREDUCE-6926?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16321083#comment-16321083
]
Miklos Szegedi commented on MAPREDUCE-6926:
---
Thank you for the patch [~haibochen].
{code}
2070Opts out of YARN oversubscription so the the job always wait for
GUARANTEED
2071resources available.
{code}
It should be "...the job always waits..."
> Allow MR jobs to opt out of oversubscription
>
>
> Key: MAPREDUCE-6926
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6926
> Project: Hadoop Map/Reduce
> Issue Type: Task
> Components: mrv2
>Affects Versions: 3.0.0-alpha3
>Reporter: Haibo Chen
>Assignee: Haibo Chen
> Attachments: MAPREDUCE-6926-YARN-1011.00.patch,
> MAPREDUCE-6926-YARN-1011.01.patch, MAPREDUCE-6926-YARN-1011.02.patch,
> MAPREDUCE-6926-YARN-1011.03.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Commented] (MAPREDUCE-6926) Allow MR jobs to opt out of oversubscription
[
https://issues.apache.org/jira/browse/MAPREDUCE-6926?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16319676#comment-16319676
]
Hadoop QA commented on MAPREDUCE-6926:
--
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 10m
57s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} YARN-1011 Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 2m
16s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 17m
38s{color} | {color:green} YARN-1011 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
42s{color} | {color:green} YARN-1011 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
40s{color} | {color:green} YARN-1011 passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
3s{color} | {color:green} YARN-1011 passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
11m 10s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
26s{color} | {color:green} YARN-1011 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
48s{color} | {color:green} YARN-1011 passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
10s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
56s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
38s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
38s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
36s{color} | {color:green} hadoop-mapreduce-project/hadoop-mapreduce-client:
The patch generated 0 new + 714 unchanged - 2 fixed = 714 total (was 716)
{color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
55s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
1s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
1s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
10m 26s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
39s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
42s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 3m
14s{color} | {color:green} hadoop-mapreduce-client-core in the patch passed.
{color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 9m
29s{color} | {color:green} hadoop-mapreduce-client-app in the patch passed.
{color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
22s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 77m 17s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:5b98639 |
| JIRA Issue | MAPREDUCE-6926 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12905384/MAPREDUCE-6926-YARN-1011.03.patch
|
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit shadedclient findbugs checkstyle xml |
| uname | Linux a08da76bf159 4.4.0-43-generic #63-Ubuntu SMP Wed Oct 12
13:48:03 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /te
[jira] [Commented] (MAPREDUCE-6926) Allow MR jobs to opt out of oversubscription
[
https://issues.apache.org/jira/browse/MAPREDUCE-6926?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16319612#comment-16319612
]
Hadoop QA commented on MAPREDUCE-6926:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
0s{color} | {color:blue} Docker mode activated. {color} |
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 9s{color}
| {color:red} MAPREDUCE-6926 does not apply to YARN-1011. Rebase required?
Wrong Branch? See https://wiki.apache.org/hadoop/HowToContribute for help.
{color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | MAPREDUCE-6926 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12904283/MAPREDUCE-6926-YARN-1011.02.patch
|
| Console output |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/7280/console |
| Powered by | Apache Yetus 0.7.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> Allow MR jobs to opt out of oversubscription
>
>
> Key: MAPREDUCE-6926
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6926
> Project: Hadoop Map/Reduce
> Issue Type: Task
> Components: mrv2
>Affects Versions: 3.0.0-alpha3
>Reporter: Haibo Chen
>Assignee: Haibo Chen
> Attachments: MAPREDUCE-6926-YARN-1011.00.patch,
> MAPREDUCE-6926-YARN-1011.01.patch, MAPREDUCE-6926-YARN-1011.02.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Commented] (MAPREDUCE-6926) Allow MR jobs to opt out of oversubscription
[
https://issues.apache.org/jira/browse/MAPREDUCE-6926?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16310317#comment-16310317
]
Hadoop QA commented on MAPREDUCE-6926:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
0s{color} | {color:blue} Docker mode activated. {color} |
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 7s{color}
| {color:red} MAPREDUCE-6926 does not apply to YARN-1011. Rebase required?
Wrong Branch? See https://wiki.apache.org/hadoop/HowToContribute for help.
{color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | MAPREDUCE-6926 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12904283/MAPREDUCE-6926-YARN-1011.02.patch
|
| Console output |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/7272/console |
| Powered by | Apache Yetus 0.7.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> Allow MR jobs to opt out of oversubscription
>
>
> Key: MAPREDUCE-6926
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6926
> Project: Hadoop Map/Reduce
> Issue Type: Task
> Components: mrv2
>Affects Versions: 3.0.0-alpha3
>Reporter: Haibo Chen
>Assignee: Haibo Chen
> Attachments: MAPREDUCE-6926-YARN-1011.00.patch,
> MAPREDUCE-6926-YARN-1011.01.patch, MAPREDUCE-6926-YARN-1011.02.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Commented] (MAPREDUCE-6926) Allow MR jobs to opt out of oversubscription
[ https://issues.apache.org/jira/browse/MAPREDUCE-6926?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16310315#comment-16310315 ] Haibo Chen commented on MAPREDUCE-6926: --- I have instead rebased YARN-1011 branch and manually triggered a Jenkins job > Allow MR jobs to opt out of oversubscription > > > Key: MAPREDUCE-6926 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6926 > Project: Hadoop Map/Reduce > Issue Type: Task > Components: mrv2 >Affects Versions: 3.0.0-alpha3 >Reporter: Haibo Chen >Assignee: Haibo Chen > Attachments: MAPREDUCE-6926-YARN-1011.00.patch, > MAPREDUCE-6926-YARN-1011.01.patch, MAPREDUCE-6926-YARN-1011.02.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Commented] (MAPREDUCE-6926) Allow MR jobs to opt out of oversubscription
[ https://issues.apache.org/jira/browse/MAPREDUCE-6926?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16308813#comment-16308813 ] Haibo Chen commented on MAPREDUCE-6926: --- As pointed by Miklos offiline, HADOOP-15122 should fix the build issue here. Cherry-picking HADOOP-15122 and re triggering another Jenkins job. > Allow MR jobs to opt out of oversubscription > > > Key: MAPREDUCE-6926 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6926 > Project: Hadoop Map/Reduce > Issue Type: Task > Components: mrv2 >Affects Versions: 3.0.0-alpha3 >Reporter: Haibo Chen >Assignee: Haibo Chen > Attachments: MAPREDUCE-6926-YARN-1011.00.patch, > MAPREDUCE-6926-YARN-1011.01.patch, MAPREDUCE-6926-YARN-1011.02.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Commented] (MAPREDUCE-6926) Allow MR jobs to opt out of oversubscription
[
https://issues.apache.org/jira/browse/MAPREDUCE-6926?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16308736#comment-16308736
]
Hadoop QA commented on MAPREDUCE-6926:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
20s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} YARN-1011 Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 2m
32s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m
15s{color} | {color:red} root in YARN-1011 failed. {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red} 0m
32s{color} | {color:red} hadoop-mapreduce-client in YARN-1011 failed. {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
37s{color} | {color:green} YARN-1011 passed {color} |
| {color:red}-1{color} | {color:red} mvnsite {color} | {color:red} 0m
10s{color} | {color:red} hadoop-mapreduce-client-core in YARN-1011 failed.
{color} |
| {color:red}-1{color} | {color:red} mvnsite {color} | {color:red} 0m
17s{color} | {color:red} hadoop-mapreduce-client-app in YARN-1011 failed.
{color} |
| {color:red}-1{color} | {color:red} shadedclient {color} | {color:red} 1m
16s{color} | {color:red} branch has errors when building and testing our client
artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
58s{color} | {color:green} YARN-1011 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
45s{color} | {color:green} YARN-1011 passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
9s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m
32s{color} | {color:red} hadoop-mapreduce-client-core in the patch failed.
{color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m
21s{color} | {color:red} hadoop-mapreduce-client-app in the patch failed.
{color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red} 0m
30s{color} | {color:red} hadoop-mapreduce-client in the patch failed. {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 0m 30s{color}
| {color:red} hadoop-mapreduce-client in the patch failed. {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
37s{color} | {color:green} hadoop-mapreduce-project/hadoop-mapreduce-client:
The patch generated 0 new + 714 unchanged - 2 fixed = 714 total (was 716)
{color} |
| {color:red}-1{color} | {color:red} mvnsite {color} | {color:red} 0m
12s{color} | {color:red} hadoop-mapreduce-client-core in the patch failed.
{color} |
| {color:red}-1{color} | {color:red} mvnsite {color} | {color:red} 0m
9s{color} | {color:red} hadoop-mapreduce-client-app in the patch failed.
{color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
1s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:red}-1{color} | {color:red} shadedclient {color} | {color:red} 0m
9s{color} | {color:red} patch has errors when building and testing our client
artifacts. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
18s{color} | {color:red} hadoop-mapreduce-client-app in the patch failed.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
44s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
59s{color} | {color:green} hadoop-mapreduce-client-core in the patch passed.
{color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 0m 18s{color}
| {color:red} hadoop-mapreduce-client-app in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
20s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color
[jira] [Commented] (MAPREDUCE-6926) Allow MR jobs to opt out of oversubscription
[ https://issues.apache.org/jira/browse/MAPREDUCE-6926?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16308705#comment-16308705 ] Haibo Chen commented on MAPREDUCE-6926: --- Good catch. I updated the message in the new patch. > Allow MR jobs to opt out of oversubscription > > > Key: MAPREDUCE-6926 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6926 > Project: Hadoop Map/Reduce > Issue Type: Task > Components: mrv2 >Affects Versions: 3.0.0-alpha3 >Reporter: Haibo Chen >Assignee: Haibo Chen > Attachments: MAPREDUCE-6926-YARN-1011.00.patch, > MAPREDUCE-6926-YARN-1011.01.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Commented] (MAPREDUCE-6926) Allow MR jobs to opt out of oversubscription
[
https://issues.apache.org/jira/browse/MAPREDUCE-6926?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16308701#comment-16308701
]
Miklos Szegedi commented on MAPREDUCE-6926:
---
Thank you, [~haibochen] for the reply. There is one more thing I found that the
following line is inaccurate. It should mention not enforced or and instead of
or.
{code}
866 Assert.fail("The execution type of ResourceRequest " +
resourceRequest +
867 " is not guaranteed or enforced.");
{code}
> Allow MR jobs to opt out of oversubscription
>
>
> Key: MAPREDUCE-6926
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6926
> Project: Hadoop Map/Reduce
> Issue Type: Task
> Components: mrv2
>Affects Versions: 3.0.0-alpha3
>Reporter: Haibo Chen
>Assignee: Haibo Chen
> Attachments: MAPREDUCE-6926-YARN-1011.00.patch,
> MAPREDUCE-6926-YARN-1011.01.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Commented] (MAPREDUCE-6926) Allow MR jobs to opt out of oversubscription
[ https://issues.apache.org/jira/browse/MAPREDUCE-6926?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16308413#comment-16308413 ] Haibo Chen commented on MAPREDUCE-6926: --- Thanks [~miklos.szeg...@cloudera.com] for the review! I agree with you that Using Clock.getTime() would ease unit test in general. But in this case, there are already other ContainerRequest constructors to allow easy unit test through direct specification of request time. Plus, I believe my change is preserving the current behavior in the code. > Allow MR jobs to opt out of oversubscription > > > Key: MAPREDUCE-6926 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6926 > Project: Hadoop Map/Reduce > Issue Type: Task > Components: mrv2 >Affects Versions: 3.0.0-alpha3 >Reporter: Haibo Chen >Assignee: Haibo Chen > Attachments: MAPREDUCE-6926-YARN-1011.00.patch, > MAPREDUCE-6926-YARN-1011.01.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Commented] (MAPREDUCE-6926) Allow MR jobs to opt out of oversubscription
[
https://issues.apache.org/jira/browse/MAPREDUCE-6926?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16277524#comment-16277524
]
Miklos Szegedi commented on MAPREDUCE-6926:
---
Thank you for the patch [~haibochen].
Using {{org.apache.hadoop.yarn.util.Clock.getTime()}} would me more future
proof, since we could change SystemClock in case of a unit test.
> Allow MR jobs to opt out of oversubscription
>
>
> Key: MAPREDUCE-6926
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6926
> Project: Hadoop Map/Reduce
> Issue Type: Task
> Components: mrv2
>Affects Versions: 3.0.0-alpha3
>Reporter: Haibo Chen
>Assignee: Haibo Chen
> Attachments: MAPREDUCE-6926-YARN-1011.00.patch,
> MAPREDUCE-6926-YARN-1011.01.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Commented] (MAPREDUCE-6926) Allow MR jobs to opt out of oversubscription
[
https://issues.apache.org/jira/browse/MAPREDUCE-6926?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16269323#comment-16269323
]
Hadoop QA commented on MAPREDUCE-6926:
--
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
11s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} YARN-1011 Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 2m
49s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 17m
31s{color} | {color:green} YARN-1011 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
36s{color} | {color:green} YARN-1011 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
33s{color} | {color:green} YARN-1011 passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
57s{color} | {color:green} YARN-1011 passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
10m 11s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
27s{color} | {color:green} YARN-1011 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
41s{color} | {color:green} YARN-1011 passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
8s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
51s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
31s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
31s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
31s{color} | {color:green} hadoop-mapreduce-project/hadoop-mapreduce-client:
The patch generated 0 new + 714 unchanged - 2 fixed = 714 total (was 716)
{color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
51s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
1s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
9m 44s{color} | {color:green} patch has no errors when building and testing our
client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
39s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
36s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
51s{color} | {color:green} hadoop-mapreduce-client-core in the patch passed.
{color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 9m
24s{color} | {color:green} hadoop-mapreduce-client-app in the patch passed.
{color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
20s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 63m 53s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:5b98639 |
| JIRA Issue | MAPREDUCE-6926 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12899656/MAPREDUCE-6926-YARN-1011.01.patch
|
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit shadedclient findbugs checkstyle xml |
| uname | Linux 41d83a34847d 4.4.0-43-generic #63-Ubuntu SMP Wed Oct 12
13:48:03 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /te
[jira] [Commented] (MAPREDUCE-6926) Allow MR jobs to opt out of oversubscription
[
https://issues.apache.org/jira/browse/MAPREDUCE-6926?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16269109#comment-16269109
]
Hadoop QA commented on MAPREDUCE-6926:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 10m
13s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} YARN-1011 Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 2m
40s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 17m
11s{color} | {color:green} YARN-1011 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
37s{color} | {color:green} YARN-1011 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
39s{color} | {color:green} YARN-1011 passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
2s{color} | {color:green} YARN-1011 passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
10m 54s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
28s{color} | {color:green} YARN-1011 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
45s{color} | {color:green} YARN-1011 passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
9s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
55s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
34s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
34s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 36s{color} | {color:orange}
hadoop-mapreduce-project/hadoop-mapreduce-client: The patch generated 3 new +
714 unchanged - 2 fixed = 717 total (was 716) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
54s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
2s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
10m 17s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
38s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
41s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
57s{color} | {color:green} hadoop-mapreduce-client-core in the patch passed.
{color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 9m 26s{color}
| {color:red} hadoop-mapreduce-client-app in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
22s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 75m 28s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.mapreduce.TestMapreduceConfigFields |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:5b98639 |
| JIRA Issue | MAPREDUCE-6926 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12899639/MAPREDUCE-6926-YARN-1011.00.patch
|
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit shadedclient findbugs checkstyle xml |
| uname | Linux 44f05fac7a25 4.4.0-43-generic #63-Ubuntu SMP Wed Oct 12
13:4
