[jira] [Commented] (YARN-7742) [UI2] Duplicated containers are rendered per attempt
[ https://issues.apache.org/jira/browse/YARN-7742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16338879#comment-16338879 ] Sunil G commented on YARN-7742: --- Updated patch. Tested in a cluster. Thanks [~skmvasu]. Now container wont be duplicated from ATSv2 and RM. Fixed a timestamp bug also. cc/[~rohithsharma] [~leftnoteasy], pls help to review the same. > [UI2] Duplicated containers are rendered per attempt > > > Key: YARN-7742 > URL: https://issues.apache.org/jira/browse/YARN-7742 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Rohith Sharma K S >Assignee: Vasudevan Skm >Priority: Major > Attachments: Screen Shot 2018-01-12 at 5.10.48 PM.png, YARN-7742 > .001.patch, YARN-7742.002.patch, YARN-7742.003.patch > > > In UI2, containers are rendered twice with different start and end time. > Attached the screen shot of UI2 -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-7742) [UI2] Duplicated containers are rendered per attempt
[ https://issues.apache.org/jira/browse/YARN-7742?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sunil G updated YARN-7742: -- Attachment: YARN-7742.003.patch > [UI2] Duplicated containers are rendered per attempt > > > Key: YARN-7742 > URL: https://issues.apache.org/jira/browse/YARN-7742 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Rohith Sharma K S >Assignee: Vasudevan Skm >Priority: Major > Attachments: Screen Shot 2018-01-12 at 5.10.48 PM.png, YARN-7742 > .001.patch, YARN-7742.002.patch, YARN-7742.003.patch > > > In UI2, containers are rendered twice with different start and end time. > Attached the screen shot of UI2 -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-7765) [Atsv2] GSS initiate failed exception in NM logs and app collectors are unable to communicate with HBase in secure cluster
[
https://issues.apache.org/jira/browse/YARN-7765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rohith Sharma K S updated YARN-7765:
Summary: [Atsv2] GSS initiate failed exception in NM logs and app
collectors are unable to communicate with HBase in secure cluster (was:
[Atsv2] GSS initiate failed exception in NM logs and unable to communicate with
HBase in secure cluster)
> [Atsv2] GSS initiate failed exception in NM logs and app collectors are
> unable to communicate with HBase in secure cluster
> --
>
> Key: YARN-7765
> URL: https://issues.apache.org/jira/browse/YARN-7765
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Rohith Sharma K S
>Priority: Critical
>
> Secure cluster is deployed and all YARN services are started successfully.
> When application is submitted, app collectors which is started as aux-service
> throwing below exception. But this exception is *NOT* observed from RM
> TimelineCollector.
> {noformat}
> 2018-01-17 11:04:48,017 FATAL ipc.RpcClientImpl (RpcClientImpl.java:run(684))
> - SASL authentication failed. The most likely cause is missing or invalid
> credentials. Consider 'kinit'.
> javax.security.sasl.SaslException: GSS initiate failed [Caused by
> GSSException: No valid credentials provided (Mechanism level: Failed to find
> any Kerberos tgt)]
> {noformat}
> cc :/ [~vrushalic] [~varun_saxena]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7765) [Atsv2] GSS initiate failed exception in NM logs and unable to communicate with HBase in secure cluster
[
https://issues.apache.org/jira/browse/YARN-7765?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16338849#comment-16338849
]
Rohith Sharma K S commented on YARN-7765:
-
To reproduce this issue, configure /etc/krb5.conf file with 10minutes for
ticket lifetime and 1 hours are renew lifetime.
{code}renew_lifetime = 1h
ticket_lifetime = 10m{code}
> [Atsv2] GSS initiate failed exception in NM logs and unable to communicate
> with HBase in secure cluster
> ---
>
> Key: YARN-7765
> URL: https://issues.apache.org/jira/browse/YARN-7765
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Rohith Sharma K S
>Priority: Critical
>
> Secure cluster is deployed and all YARN services are started successfully.
> When application is submitted, app collectors which is started as aux-service
> throwing below exception. But this exception is *NOT* observed from RM
> TimelineCollector.
> {noformat}
> 2018-01-17 11:04:48,017 FATAL ipc.RpcClientImpl (RpcClientImpl.java:run(684))
> - SASL authentication failed. The most likely cause is missing or invalid
> credentials. Consider 'kinit'.
> javax.security.sasl.SaslException: GSS initiate failed [Caused by
> GSSException: No valid credentials provided (Mechanism level: Failed to find
> any Kerberos tgt)]
> {noformat}
> cc :/ [~vrushalic] [~varun_saxena]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7102) NM heartbeat stuck when responseId overflows MAX_INT
[
https://issues.apache.org/jira/browse/YARN-7102?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16338844#comment-16338844
]
genericqa commented on YARN-7102:
-
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
12s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 5 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
17s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 18m
16s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 13m
44s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
15s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
22s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
13m 56s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
58s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
54s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
15s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
2s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 12m
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 12m
0s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
2m 10s{color} | {color:orange} root: The patch generated 1 new + 328 unchanged
- 16 fixed = 329 total (was 344) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
9m 54s{color} | {color:green} patch has no errors when building and testing our
client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
7s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
56s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 61m
28s{color} | {color:green} hadoop-yarn-server-resourcemanager in the patch
passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 8m
38s{color} | {color:green} hadoop-sls in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
34s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}152m 0s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:5b98639 |
| JIRA Issue | YARN-7102 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12907625/YARN-7102.v17.patch |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit shadedclient findbugs checkstyle |
| uname | Linux a266972cc048 3.13.0-133-generic #182-Ubuntu SMP Tue Sep 19
15:49:21 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / 59828be |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_151 |
| findbugs | v3.1.0-RC1 |
| checkstyle |
[jira] [Commented] (YARN-7798) Refactor SLS Reservation Creation
[ https://issues.apache.org/jira/browse/YARN-7798?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16338833#comment-16338833 ] Yufei Gu commented on YARN-7798: Yes. We can use Java8 features in trunk. Please clean the unused imports. Otherwise it looks pretty good. > Refactor SLS Reservation Creation > - > > Key: YARN-7798 > URL: https://issues.apache.org/jira/browse/YARN-7798 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Young Chen >Assignee: Young Chen >Priority: Minor > Attachments: YARN-7798.01.patch, YARN-7798.02.patch > > > Move the reservation request creation out of SLSRunner and delegate to the > AMSimulator instance. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-7765) [Atsv2] GSS initiate failed exception in NM logs and unable to communicate with HBase in secure cluster
[
https://issues.apache.org/jira/browse/YARN-7765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rohith Sharma K S updated YARN-7765:
Summary: [Atsv2] GSS initiate failed exception in NM logs and unable to
communicate with HBase in secure cluster (was: [Atsv2] App collector failed to
authenticate with HBase in secure cluster)
> [Atsv2] GSS initiate failed exception in NM logs and unable to communicate
> with HBase in secure cluster
> ---
>
> Key: YARN-7765
> URL: https://issues.apache.org/jira/browse/YARN-7765
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Rohith Sharma K S
>Priority: Critical
>
> Secure cluster is deployed and all YARN services are started successfully.
> When application is submitted, app collectors which is started as aux-service
> throwing below exception. But this exception is *NOT* observed from RM
> TimelineCollector.
> {noformat}
> 2018-01-17 11:04:48,017 FATAL ipc.RpcClientImpl (RpcClientImpl.java:run(684))
> - SASL authentication failed. The most likely cause is missing or invalid
> credentials. Consider 'kinit'.
> javax.security.sasl.SaslException: GSS initiate failed [Caused by
> GSSException: No valid credentials provided (Mechanism level: Failed to find
> any Kerberos tgt)]
> {noformat}
> cc :/ [~vrushalic] [~varun_saxena]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7765) [Atsv2] GSS initiate failed exception in NM logs and unable to communicate with HBase in secure cluster
[
https://issues.apache.org/jira/browse/YARN-7765?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16338829#comment-16338829
]
Rohith Sharma K S commented on YARN-7765:
-
This is appearing in our test environment regularly!
> [Atsv2] GSS initiate failed exception in NM logs and unable to communicate
> with HBase in secure cluster
> ---
>
> Key: YARN-7765
> URL: https://issues.apache.org/jira/browse/YARN-7765
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Rohith Sharma K S
>Priority: Critical
>
> Secure cluster is deployed and all YARN services are started successfully.
> When application is submitted, app collectors which is started as aux-service
> throwing below exception. But this exception is *NOT* observed from RM
> TimelineCollector.
> {noformat}
> 2018-01-17 11:04:48,017 FATAL ipc.RpcClientImpl (RpcClientImpl.java:run(684))
> - SASL authentication failed. The most likely cause is missing or invalid
> credentials. Consider 'kinit'.
> javax.security.sasl.SaslException: GSS initiate failed [Caused by
> GSSException: No valid credentials provided (Mechanism level: Failed to find
> any Kerberos tgt)]
> {noformat}
> cc :/ [~vrushalic] [~varun_saxena]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6492) Generate queue metrics for each partition
[
https://issues.apache.org/jira/browse/YARN-6492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16338818#comment-16338818
]
Bibin A Chundatt commented on YARN-6492:
Thank you [~maniraj...@gmail.com] working on this JIRA.
Adding more that point which you can handle in next patch.
{{MetricsOverviewTable}} shows resource only on default partition now. Probably
you can cover that too as part of this JIRA. Since after YARN-6467 the resource
overview is not correct.
> Generate queue metrics for each partition
> -
>
> Key: YARN-6492
> URL: https://issues.apache.org/jira/browse/YARN-6492
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: capacity scheduler
>Reporter: Jonathan Hung
>Assignee: Manikandan R
>Priority: Major
> Attachments: YARN-6492.001.patch, YARN-6492.002.patch,
> YARN-6492.003.patch, partition_metrics.txt
>
>
> We are interested in having queue metrics for all partitions. Right now each
> queue has one QueueMetrics object which captures metrics either in default
> partition or across all partitions. (After YARN-6467 it will be in default
> partition)
> But having the partition metrics would be very useful.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7817) Add Resource reference to RM's NodeInfo object so REST API can get non memory/vcore resource usages.
[
https://issues.apache.org/jira/browse/YARN-7817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16338813#comment-16338813
]
genericqa commented on YARN-7817:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
11s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 18m
24s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
41s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
30s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
48s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
11m 16s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
17s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
26s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
43s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
37s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
37s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 25s{color} | {color:orange}
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager:
The patch generated 4 new + 20 unchanged - 0 fixed = 24 total (was 20) {color}
|
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
40s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
11m 9s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
20s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
24s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 60m 11s{color}
| {color:red} hadoop-yarn-server-resourcemanager in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
22s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}109m 1s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests |
hadoop.yarn.server.resourcemanager.webapp.TestRMWebServiceAppsNodelabel |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:5b98639 |
| JIRA Issue | YARN-7817 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12907624/YARN-7817.002.patch |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit shadedclient findbugs checkstyle |
| uname | Linux b17a96bddffb 3.13.0-133-generic #182-Ubuntu SMP Tue Sep 19
15:49:21 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / 59828be |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_151 |
| findbugs | v3.1.0-RC1 |
| checkstyle |
https://builds.apache.org/job/PreCommit-YARN-Build/19457/artifact/out/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
|
| unit |
[jira] [Commented] (YARN-7817) Add Resource reference to RM's NodeInfo object so REST API can get non memory/vcore resource usages.
[
https://issues.apache.org/jira/browse/YARN-7817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16338759#comment-16338759
]
genericqa commented on YARN-7817:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
11s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 18m
31s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
43s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
31s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
48s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
11m 24s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
17s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
26s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
46s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
43s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
43s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 28s{color} | {color:orange}
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager:
The patch generated 2 new + 15 unchanged - 0 fixed = 17 total (was 15) {color}
|
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
44s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
11m 12s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
12s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
27s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 60m 14s{color}
| {color:red} hadoop-yarn-server-resourcemanager in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
19s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}109m 34s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests |
hadoop.yarn.server.resourcemanager.scheduler.TestSchedulingWithAllocationRequestId
|
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:5b98639 |
| JIRA Issue | YARN-7817 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12907620/YARN-7817.001.patch |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit shadedclient findbugs checkstyle |
| uname | Linux 5999cae7646d 3.13.0-135-generic #184-Ubuntu SMP Wed Oct 18
11:55:51 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / 59828be |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_151 |
| findbugs | v3.1.0-RC1 |
| checkstyle |
https://builds.apache.org/job/PreCommit-YARN-Build/19454/artifact/out/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
|
|
[jira] [Commented] (YARN-7792) Merge work for YARN-6592
[ https://issues.apache.org/jira/browse/YARN-7792?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16338751#comment-16338751 ] Sunil G commented on YARN-7792: --- Thanks [~asuresh]. I will keep a track of this today. > Merge work for YARN-6592 > > > Key: YARN-7792 > URL: https://issues.apache.org/jira/browse/YARN-7792 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Sunil G >Priority: Blocker > Attachments: YARN-6592.001.patch, YARN-7792.002.patch, > YARN-7792.003.patch > > > This Jira is to run aggregated YARN-6592 branch patch against trunk and check > for any jenkins issues. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7792) Merge work for YARN-6592
[ https://issues.apache.org/jira/browse/YARN-7792?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16338742#comment-16338742 ] Arun Suresh commented on YARN-7792: --- Rebased with trunk and updated patch again > Merge work for YARN-6592 > > > Key: YARN-7792 > URL: https://issues.apache.org/jira/browse/YARN-7792 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Sunil G >Priority: Blocker > Attachments: YARN-6592.001.patch, YARN-7792.002.patch, > YARN-7792.003.patch > > > This Jira is to run aggregated YARN-6592 branch patch against trunk and check > for any jenkins issues. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-7792) Merge work for YARN-6592
[ https://issues.apache.org/jira/browse/YARN-7792?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun Suresh updated YARN-7792: -- Attachment: YARN-7792.003.patch > Merge work for YARN-6592 > > > Key: YARN-7792 > URL: https://issues.apache.org/jira/browse/YARN-7792 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Sunil G >Priority: Blocker > Attachments: YARN-6592.001.patch, YARN-7792.002.patch, > YARN-7792.003.patch > > > This Jira is to run aggregated YARN-6592 branch patch against trunk and check > for any jenkins issues. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (YARN-7102) NM heartbeat stuck when responseId overflows MAX_INT
[ https://issues.apache.org/jira/browse/YARN-7102?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16338741#comment-16338741 ] Botong Huang edited comment on YARN-7102 at 1/25/18 5:20 AM: - The unit test failures for branch-2.v17 patch seem irrelevant by looking at the exceptions, and I cannot re-pro them locally... was (Author: botong): The unit test failures for branch-2.v17 patch seem irrelevant by looking at the exceptions, and I cannot re-pre them locally... > NM heartbeat stuck when responseId overflows MAX_INT > > > Key: YARN-7102 > URL: https://issues.apache.org/jira/browse/YARN-7102 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Botong Huang >Assignee: Botong Huang >Priority: Critical > Attachments: YARN-7102-branch-2.8.v10.patch, > YARN-7102-branch-2.8.v11.patch, YARN-7102-branch-2.8.v14.patch, > YARN-7102-branch-2.8.v14.patch, YARN-7102-branch-2.8.v17.patch, > YARN-7102-branch-2.8.v9.patch, YARN-7102-branch-2.v14.patch, > YARN-7102-branch-2.v14.patch, YARN-7102-branch-2.v17.patch, > YARN-7102-branch-2.v17.patch, YARN-7102-branch-2.v9.patch, > YARN-7102-branch-2.v9.patch, YARN-7102-branch-2.v9.patch, YARN-7102.v1.patch, > YARN-7102.v12.patch, YARN-7102.v13.patch, YARN-7102.v14.patch, > YARN-7102.v15.patch, YARN-7102.v16.patch, YARN-7102.v17.patch, > YARN-7102.v17.patch, YARN-7102.v17.patch, YARN-7102.v2.patch, > YARN-7102.v3.patch, YARN-7102.v4.patch, YARN-7102.v5.patch, > YARN-7102.v6.patch, YARN-7102.v7.patch, YARN-7102.v8.patch, YARN-7102.v9.patch > > > ResponseId overflow problem in NM-RM heartbeat. This is same as AM-RM > heartbeat in YARN-6640, please refer to YARN-6640 for details. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (YARN-7102) NM heartbeat stuck when responseId overflows MAX_INT
[ https://issues.apache.org/jira/browse/YARN-7102?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16338741#comment-16338741 ] Botong Huang edited comment on YARN-7102 at 1/25/18 5:20 AM: - The unit test failures for branch-2.v17 patch seem irrelevant looking at the exception and logs, and I cannot re-pro them locally... was (Author: botong): The unit test failures for branch-2.v17 patch seem irrelevant by looking at the exceptions, and I cannot re-pro them locally... > NM heartbeat stuck when responseId overflows MAX_INT > > > Key: YARN-7102 > URL: https://issues.apache.org/jira/browse/YARN-7102 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Botong Huang >Assignee: Botong Huang >Priority: Critical > Attachments: YARN-7102-branch-2.8.v10.patch, > YARN-7102-branch-2.8.v11.patch, YARN-7102-branch-2.8.v14.patch, > YARN-7102-branch-2.8.v14.patch, YARN-7102-branch-2.8.v17.patch, > YARN-7102-branch-2.8.v9.patch, YARN-7102-branch-2.v14.patch, > YARN-7102-branch-2.v14.patch, YARN-7102-branch-2.v17.patch, > YARN-7102-branch-2.v17.patch, YARN-7102-branch-2.v9.patch, > YARN-7102-branch-2.v9.patch, YARN-7102-branch-2.v9.patch, YARN-7102.v1.patch, > YARN-7102.v12.patch, YARN-7102.v13.patch, YARN-7102.v14.patch, > YARN-7102.v15.patch, YARN-7102.v16.patch, YARN-7102.v17.patch, > YARN-7102.v17.patch, YARN-7102.v17.patch, YARN-7102.v2.patch, > YARN-7102.v3.patch, YARN-7102.v4.patch, YARN-7102.v5.patch, > YARN-7102.v6.patch, YARN-7102.v7.patch, YARN-7102.v8.patch, YARN-7102.v9.patch > > > ResponseId overflow problem in NM-RM heartbeat. This is same as AM-RM > heartbeat in YARN-6640, please refer to YARN-6640 for details. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7102) NM heartbeat stuck when responseId overflows MAX_INT
[ https://issues.apache.org/jira/browse/YARN-7102?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16338741#comment-16338741 ] Botong Huang commented on YARN-7102: The unit test failures for branch-2.v17 patch seem irrelevant by looking at the exceptions, and I cannot re-pre them locally... > NM heartbeat stuck when responseId overflows MAX_INT > > > Key: YARN-7102 > URL: https://issues.apache.org/jira/browse/YARN-7102 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Botong Huang >Assignee: Botong Huang >Priority: Critical > Attachments: YARN-7102-branch-2.8.v10.patch, > YARN-7102-branch-2.8.v11.patch, YARN-7102-branch-2.8.v14.patch, > YARN-7102-branch-2.8.v14.patch, YARN-7102-branch-2.8.v17.patch, > YARN-7102-branch-2.8.v9.patch, YARN-7102-branch-2.v14.patch, > YARN-7102-branch-2.v14.patch, YARN-7102-branch-2.v17.patch, > YARN-7102-branch-2.v17.patch, YARN-7102-branch-2.v9.patch, > YARN-7102-branch-2.v9.patch, YARN-7102-branch-2.v9.patch, YARN-7102.v1.patch, > YARN-7102.v12.patch, YARN-7102.v13.patch, YARN-7102.v14.patch, > YARN-7102.v15.patch, YARN-7102.v16.patch, YARN-7102.v17.patch, > YARN-7102.v17.patch, YARN-7102.v17.patch, YARN-7102.v2.patch, > YARN-7102.v3.patch, YARN-7102.v4.patch, YARN-7102.v5.patch, > YARN-7102.v6.patch, YARN-7102.v7.patch, YARN-7102.v8.patch, YARN-7102.v9.patch > > > ResponseId overflow problem in NM-RM heartbeat. This is same as AM-RM > heartbeat in YARN-6640, please refer to YARN-6640 for details. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7817) Add Resource reference to RM's NodeInfo object so REST API can get non memory/vcore resource usages.
[ https://issues.apache.org/jira/browse/YARN-7817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16338721#comment-16338721 ] Wangda Tan commented on YARN-7817: -- Fixed more issues, attached ver.2 patch. Verified on a cluster to make sure it works. [~sunilg] could you help to review the patch? > Add Resource reference to RM's NodeInfo object so REST API can get non > memory/vcore resource usages. > > > Key: YARN-7817 > URL: https://issues.apache.org/jira/browse/YARN-7817 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Sumana Sathish >Assignee: Wangda Tan >Priority: Major > Attachments: YARN-7817.001.patch, YARN-7817.002.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-7102) NM heartbeat stuck when responseId overflows MAX_INT
[ https://issues.apache.org/jira/browse/YARN-7102?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Botong Huang updated YARN-7102: --- Attachment: YARN-7102.v17.patch > NM heartbeat stuck when responseId overflows MAX_INT > > > Key: YARN-7102 > URL: https://issues.apache.org/jira/browse/YARN-7102 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Botong Huang >Assignee: Botong Huang >Priority: Critical > Attachments: YARN-7102-branch-2.8.v10.patch, > YARN-7102-branch-2.8.v11.patch, YARN-7102-branch-2.8.v14.patch, > YARN-7102-branch-2.8.v14.patch, YARN-7102-branch-2.8.v17.patch, > YARN-7102-branch-2.8.v9.patch, YARN-7102-branch-2.v14.patch, > YARN-7102-branch-2.v14.patch, YARN-7102-branch-2.v17.patch, > YARN-7102-branch-2.v17.patch, YARN-7102-branch-2.v9.patch, > YARN-7102-branch-2.v9.patch, YARN-7102-branch-2.v9.patch, YARN-7102.v1.patch, > YARN-7102.v12.patch, YARN-7102.v13.patch, YARN-7102.v14.patch, > YARN-7102.v15.patch, YARN-7102.v16.patch, YARN-7102.v17.patch, > YARN-7102.v17.patch, YARN-7102.v17.patch, YARN-7102.v2.patch, > YARN-7102.v3.patch, YARN-7102.v4.patch, YARN-7102.v5.patch, > YARN-7102.v6.patch, YARN-7102.v7.patch, YARN-7102.v8.patch, YARN-7102.v9.patch > > > ResponseId overflow problem in NM-RM heartbeat. This is same as AM-RM > heartbeat in YARN-6640, please refer to YARN-6640 for details. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-7817) Add Resource reference to RM's NodeInfo object so REST API can get non memory/vcore resource usages.
[ https://issues.apache.org/jira/browse/YARN-7817?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated YARN-7817: - Attachment: YARN-7817.002.patch > Add Resource reference to RM's NodeInfo object so REST API can get non > memory/vcore resource usages. > > > Key: YARN-7817 > URL: https://issues.apache.org/jira/browse/YARN-7817 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Sumana Sathish >Assignee: Wangda Tan >Priority: Major > Attachments: YARN-7817.001.patch, YARN-7817.002.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7814) Remove automatic mounting of the cgroups root directory into Docker containers
[
https://issues.apache.org/jira/browse/YARN-7814?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16338705#comment-16338705
]
Miklos Szegedi commented on YARN-7814:
--
Please repurpose and close them as you see fit. I think it is a good idea not
to auto mount cgroups.
> Remove automatic mounting of the cgroups root directory into Docker containers
> --
>
> Key: YARN-7814
> URL: https://issues.apache.org/jira/browse/YARN-7814
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Shane Kumpf
>Assignee: Shane Kumpf
>Priority: Major
>
> Currently, all Docker containers launched by {{DockerLinuxContainerRuntime}}
> get /sys/fs/cgroup automatically mounted. Now that user supplied mounts
> (YARN-5534) are in, containers that require this mount can request it (with a
> properly configured mount whitelist).
> I propose we remove the automatic mounting of /sys/fs/cgroup into Docker
> containers.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-7817) Add Resource reference to RM's NodeInfo object so REST API can get non memory/vcore resource usages.
[ https://issues.apache.org/jira/browse/YARN-7817?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated YARN-7817: - Attachment: YARN-7817.001.patch > Add Resource reference to RM's NodeInfo object so REST API can get non > memory/vcore resource usages. > > > Key: YARN-7817 > URL: https://issues.apache.org/jira/browse/YARN-7817 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Sumana Sathish >Assignee: Wangda Tan >Priority: Major > Attachments: YARN-7817.001.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-2185) Use pipes when localizing archives
[ https://issues.apache.org/jira/browse/YARN-2185?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Szegedi updated YARN-2185: - Attachment: YARN-2185.012.patch > Use pipes when localizing archives > -- > > Key: YARN-2185 > URL: https://issues.apache.org/jira/browse/YARN-2185 > Project: Hadoop YARN > Issue Type: Improvement > Components: nodemanager >Affects Versions: 2.4.0 >Reporter: Jason Lowe >Assignee: Miklos Szegedi >Priority: Major > Attachments: YARN-2185.000.patch, YARN-2185.001.patch, > YARN-2185.002.patch, YARN-2185.003.patch, YARN-2185.004.patch, > YARN-2185.005.patch, YARN-2185.006.patch, YARN-2185.007.patch, > YARN-2185.008.patch, YARN-2185.009.patch, YARN-2185.010.patch, > YARN-2185.011.patch, YARN-2185.012.patch > > > Currently the nodemanager downloads an archive to a local file, unpacks it, > and then removes it. It would be more efficient to stream the data as it's > being unpacked to avoid both the extra disk space requirements and the > additional disk activity from storing the archive. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-7817) Add Resource reference to RM's NodeInfo object so REST API can get non memory/vcore resource usages.
Wangda Tan created YARN-7817: Summary: Add Resource reference to RM's NodeInfo object so REST API can get non memory/vcore resource usages. Key: YARN-7817 URL: https://issues.apache.org/jira/browse/YARN-7817 Project: Hadoop YARN Issue Type: Sub-task Reporter: Sumana Sathish Assignee: Wangda Tan -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Resolved] (YARN-7759) [UI2]GPU chart shows as "Available: 0" even though GPU is available
[ https://issues.apache.org/jira/browse/YARN-7759?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan resolved YARN-7759. -- Resolution: Duplicate Duplicated by YARN-7817 > [UI2]GPU chart shows as "Available: 0" even though GPU is available > --- > > Key: YARN-7759 > URL: https://issues.apache.org/jira/browse/YARN-7759 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Sumana Sathish >Assignee: Vasudevan Skm >Priority: Major > > GPU chart under Node Manager page shows as zero GPU's available even though > GPU s are present. Only when we click 'GPU Information' chart, it shows > correct GPU information -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7817) Add Resource reference to RM's NodeInfo object so REST API can get non memory/vcore resource usages.
[ https://issues.apache.org/jira/browse/YARN-7817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16338687#comment-16338687 ] Wangda Tan commented on YARN-7817: -- Attached ver.1 patch to run Jenkins. > Add Resource reference to RM's NodeInfo object so REST API can get non > memory/vcore resource usages. > > > Key: YARN-7817 > URL: https://issues.apache.org/jira/browse/YARN-7817 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Sumana Sathish >Assignee: Wangda Tan >Priority: Major > Attachments: YARN-7817.001.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-2185) Use pipes when localizing archives
[
https://issues.apache.org/jira/browse/YARN-2185?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16338655#comment-16338655
]
genericqa commented on YARN-2185:
-
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
20s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
20s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 15m
12s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 12m
15s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
55s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
35s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
13m 12s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
36s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
39s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
19s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
14s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 11m
33s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 11m
33s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
51s{color} | {color:green} root: The patch generated 0 new + 151 unchanged - 8
fixed = 151 total (was 159) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
42s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
9m 0s{color} | {color:green} patch has no errors when building and testing our
client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
57s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
28s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 8m
22s{color} | {color:green} hadoop-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 3m
3s{color} | {color:green} hadoop-yarn-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
28s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 90m 45s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:5b98639 |
| JIRA Issue | YARN-2185 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12907592/YARN-2185.011.patch |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit shadedclient findbugs checkstyle |
| uname | Linux 49461a35392b 4.4.0-64-generic #85-Ubuntu SMP Mon Feb 20
11:50:30 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / 051ad9d |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_151 |
| findbugs | v3.1.0-RC1 |
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/19447/testReport/ |
| Max. process+thread count
[jira] [Commented] (YARN-7810) TestDockerContainerRuntime test failures due to UID lookup of a non-existent user
[
https://issues.apache.org/jira/browse/YARN-7810?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16338651#comment-16338651
]
Shane Kumpf commented on YARN-7810:
---
Thanks for the review and commit, [~eyang].
> TestDockerContainerRuntime test failures due to UID lookup of a non-existent
> user
> -
>
> Key: YARN-7810
> URL: https://issues.apache.org/jira/browse/YARN-7810
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Shane Kumpf
>Assignee: Shane Kumpf
>Priority: Major
> Fix For: 3.1.0
>
> Attachments: YARN-7810.001.patch, YARN-7810.002.patch
>
>
> YARN-7782 enabled the Docker runtime feature to remap the username to uid:gid
> form for launching Docker containers. The feature does an {{id -u}} and {{id
> -G}} to get the UID and GIDs. This fails with the test user, as that user
> doesn't actually exist on the host.
> {code:java}
> [ERROR]
> testContainerLaunchWithCustomNetworks(org.apache.hadoop.yarn.server.nodemanager.containermanager.linux.runtime.TestDockerContainerRuntime)
> Time elapsed: 0.411 s <<< ERROR!
> org.apache.hadoop.yarn.server.nodemanager.containermanager.runtime.ContainerExecutionException:
>
> ExitCodeException exitCode=1: id: 'run_as_user': no such user
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.linux.runtime.DockerLinuxContainerRuntime.getUserIdInfo(DockerLinuxContainerRuntime.java:711)
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.linux.runtime.DockerLinuxContainerRuntime.launchContainer(DockerLinuxContainerRuntime.java:757)
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.linux.runtime.TestDockerContainerRuntime.testContainerLaunchWithCustomNetworks(TestDockerContainerRuntime.java:599){code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7798) Refactor SLS Reservation Creation
[
https://issues.apache.org/jira/browse/YARN-7798?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16338643#comment-16338643
]
genericqa commented on YARN-7798:
-
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
23s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 15m
10s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
20s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
15s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
23s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
9m 44s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
31s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
15s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
17s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 10s{color} | {color:orange} hadoop-tools/hadoop-sls: The patch generated 4
new + 50 unchanged - 2 fixed = 54 total (was 52) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
20s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
10m 28s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
38s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
15s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 7m
49s{color} | {color:green} hadoop-sls in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
20s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 47m 52s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:5b98639 |
| JIRA Issue | YARN-7798 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12907608/YARN-7798.02.patch |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit shadedclient findbugs checkstyle |
| uname | Linux c596ecd4d19e 4.4.0-89-generic #112-Ubuntu SMP Mon Jul 31
19:38:41 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / 051ad9d |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_151 |
| findbugs | v3.1.0-RC1 |
| checkstyle |
https://builds.apache.org/job/PreCommit-YARN-Build/19449/artifact/out/diff-checkstyle-hadoop-tools_hadoop-sls.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/19449/testReport/ |
| Max. process+thread count | 471 (vs. ulimit of 5000) |
| modules | C: hadoop-tools/hadoop-sls U: hadoop-tools/hadoop-sls |
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/19449/console |
| Powered by | Apache Yetus 0.8.0-SNAPSHOT http://yetus.apache.org |
This message was automatically
[jira] [Commented] (YARN-7792) Merge work for YARN-6592
[
https://issues.apache.org/jira/browse/YARN-7792?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16338638#comment-16338638
]
genericqa commented on YARN-7792:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
34s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 41 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
19s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 15m
29s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 15m
27s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
17s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 7m
30s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
19m 4s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Skipped patched modules with no Java source:
hadoop-yarn-project/hadoop-yarn {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m
5s{color} | {color:red} hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api in
trunk has 1 extant Findbugs warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 5m
37s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
43s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m
37s{color} | {color:red} hadoop-yarn in the patch failed. {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m
28s{color} | {color:red} hadoop-yarn-api in the patch failed. {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m
20s{color} | {color:red} hadoop-yarn-common in the patch failed. {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m
19s{color} | {color:red} hadoop-yarn-server-common in the patch failed. {color}
|
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m
18s{color} | {color:red} hadoop-yarn-server-nodemanager in the patch failed.
{color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m
21s{color} | {color:red} hadoop-yarn-server-resourcemanager in the patch
failed. {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m
16s{color} | {color:red} hadoop-yarn-client in the patch failed. {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m
16s{color} | {color:red} hadoop-yarn-applications-distributedshell in the patch
failed. {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m
22s{color} | {color:red} hadoop-mapreduce-client-app in the patch failed.
{color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m
15s{color} | {color:red} hadoop-sls in the patch failed. {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 13m
57s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} cc {color} | {color:green} 13m
57s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 13m
57s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
2m 18s{color} | {color:orange} root: The patch generated 142 new + 2431
unchanged - 44 fixed = 2573 total (was 2475) {color} |
| {color:red}-1{color} | {color:red} mvnsite {color} | {color:red} 0m
23s{color} | {color:red} hadoop-yarn-common in the patch failed. {color} |
| {color:red}-1{color} | {color:red} mvnsite {color} | {color:red} 0m
23s{color} | {color:red} hadoop-yarn-server-common in the patch failed. {color}
|
| {color:red}-1{color} | {color:red} mvnsite {color} | {color:red} 0m
[jira] [Commented] (YARN-7810) TestDockerContainerRuntime test failures due to UID lookup of a non-existent user
[
https://issues.apache.org/jira/browse/YARN-7810?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16338637#comment-16338637
]
Hudson commented on YARN-7810:
--
SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #13554 (See
[https://builds.apache.org/job/Hadoop-trunk-Commit/13554/])
YARN-7810. Update TestDockerContainerRuntime to test with current user (eyang:
rev 59828be1978ec942dda38774a1d9f741efa96f71)
* (edit)
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/runtime/TestDockerContainerRuntime.java
> TestDockerContainerRuntime test failures due to UID lookup of a non-existent
> user
> -
>
> Key: YARN-7810
> URL: https://issues.apache.org/jira/browse/YARN-7810
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Shane Kumpf
>Assignee: Shane Kumpf
>Priority: Major
> Fix For: 3.1.0
>
> Attachments: YARN-7810.001.patch, YARN-7810.002.patch
>
>
> YARN-7782 enabled the Docker runtime feature to remap the username to uid:gid
> form for launching Docker containers. The feature does an {{id -u}} and {{id
> -G}} to get the UID and GIDs. This fails with the test user, as that user
> doesn't actually exist on the host.
> {code:java}
> [ERROR]
> testContainerLaunchWithCustomNetworks(org.apache.hadoop.yarn.server.nodemanager.containermanager.linux.runtime.TestDockerContainerRuntime)
> Time elapsed: 0.411 s <<< ERROR!
> org.apache.hadoop.yarn.server.nodemanager.containermanager.runtime.ContainerExecutionException:
>
> ExitCodeException exitCode=1: id: 'run_as_user': no such user
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.linux.runtime.DockerLinuxContainerRuntime.getUserIdInfo(DockerLinuxContainerRuntime.java:711)
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.linux.runtime.DockerLinuxContainerRuntime.launchContainer(DockerLinuxContainerRuntime.java:757)
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.linux.runtime.TestDockerContainerRuntime.testContainerLaunchWithCustomNetworks(TestDockerContainerRuntime.java:599){code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7810) TestDockerContainerRuntime test failures due to UID lookup of a non-existent user
[
https://issues.apache.org/jira/browse/YARN-7810?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16338625#comment-16338625
]
Eric Yang commented on YARN-7810:
-
+1 looks good. I just committed this. Thank you Shane.
> TestDockerContainerRuntime test failures due to UID lookup of a non-existent
> user
> -
>
> Key: YARN-7810
> URL: https://issues.apache.org/jira/browse/YARN-7810
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Shane Kumpf
>Assignee: Shane Kumpf
>Priority: Major
> Attachments: YARN-7810.001.patch, YARN-7810.002.patch
>
>
> YARN-7782 enabled the Docker runtime feature to remap the username to uid:gid
> form for launching Docker containers. The feature does an {{id -u}} and {{id
> -G}} to get the UID and GIDs. This fails with the test user, as that user
> doesn't actually exist on the host.
> {code:java}
> [ERROR]
> testContainerLaunchWithCustomNetworks(org.apache.hadoop.yarn.server.nodemanager.containermanager.linux.runtime.TestDockerContainerRuntime)
> Time elapsed: 0.411 s <<< ERROR!
> org.apache.hadoop.yarn.server.nodemanager.containermanager.runtime.ContainerExecutionException:
>
> ExitCodeException exitCode=1: id: 'run_as_user': no such user
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.linux.runtime.DockerLinuxContainerRuntime.getUserIdInfo(DockerLinuxContainerRuntime.java:711)
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.linux.runtime.DockerLinuxContainerRuntime.launchContainer(DockerLinuxContainerRuntime.java:757)
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.linux.runtime.TestDockerContainerRuntime.testContainerLaunchWithCustomNetworks(TestDockerContainerRuntime.java:599){code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7798) Refactor SLS Reservation Creation
[ https://issues.apache.org/jira/browse/YARN-7798?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16338596#comment-16338596 ] Young Chen commented on YARN-7798: -- Accidentally generated wrong patch - resubmitting [^YARN-7798.02.patch] > Refactor SLS Reservation Creation > - > > Key: YARN-7798 > URL: https://issues.apache.org/jira/browse/YARN-7798 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Young Chen >Assignee: Young Chen >Priority: Minor > Attachments: YARN-7798.01.patch, YARN-7798.02.patch > > > Move the reservation request creation out of SLSRunner and delegate to the > AMSimulator instance. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-7798) Refactor SLS Reservation Creation
[ https://issues.apache.org/jira/browse/YARN-7798?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Young Chen updated YARN-7798: - Attachment: (was: YARN-7798.02.patch) > Refactor SLS Reservation Creation > - > > Key: YARN-7798 > URL: https://issues.apache.org/jira/browse/YARN-7798 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Young Chen >Assignee: Young Chen >Priority: Minor > Attachments: YARN-7798.01.patch, YARN-7798.02.patch > > > Move the reservation request creation out of SLSRunner and delegate to the > AMSimulator instance. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-7798) Refactor SLS Reservation Creation
[ https://issues.apache.org/jira/browse/YARN-7798?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Young Chen updated YARN-7798: - Attachment: YARN-7798.02.patch > Refactor SLS Reservation Creation > - > > Key: YARN-7798 > URL: https://issues.apache.org/jira/browse/YARN-7798 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Young Chen >Assignee: Young Chen >Priority: Minor > Attachments: YARN-7798.01.patch, YARN-7798.02.patch > > > Move the reservation request creation out of SLSRunner and delegate to the > AMSimulator instance. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7798) Refactor SLS Reservation Creation
[
https://issues.apache.org/jira/browse/YARN-7798?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16338585#comment-16338585
]
genericqa commented on YARN-7798:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
0s{color} | {color:blue} Docker mode activated. {color} |
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 5s{color}
| {color:red} YARN-7798 does not apply to trunk. Rebase required? Wrong Branch?
See https://wiki.apache.org/hadoop/HowToContribute for help. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | YARN-7798 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12907606/YARN-7798.02.patch |
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/19448/console |
| Powered by | Apache Yetus 0.8.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> Refactor SLS Reservation Creation
> -
>
> Key: YARN-7798
> URL: https://issues.apache.org/jira/browse/YARN-7798
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Young Chen
>Assignee: Young Chen
>Priority: Minor
> Attachments: YARN-7798.01.patch, YARN-7798.02.patch
>
>
> Move the reservation request creation out of SLSRunner and delegate to the
> AMSimulator instance.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7798) Refactor SLS Reservation Creation
[ https://issues.apache.org/jira/browse/YARN-7798?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16338578#comment-16338578 ] Young Chen commented on YARN-7798: -- Submitted v2 patch - Good catch on the componentwiseMax, that actually would have been a bug. I cleaned up initReservations. Just to clarify, is it okay to use Java 8 features in patches that are going in trunk? I used Java streams when cleaning up the reservation code. > Refactor SLS Reservation Creation > - > > Key: YARN-7798 > URL: https://issues.apache.org/jira/browse/YARN-7798 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Young Chen >Assignee: Young Chen >Priority: Minor > Attachments: YARN-7798.01.patch, YARN-7798.02.patch > > > Move the reservation request creation out of SLSRunner and delegate to the > AMSimulator instance. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-7798) Refactor SLS Reservation Creation
[ https://issues.apache.org/jira/browse/YARN-7798?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Young Chen updated YARN-7798: - Attachment: YARN-7798.02.patch > Refactor SLS Reservation Creation > - > > Key: YARN-7798 > URL: https://issues.apache.org/jira/browse/YARN-7798 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Young Chen >Assignee: Young Chen >Priority: Minor > Attachments: YARN-7798.01.patch, YARN-7798.02.patch > > > Move the reservation request creation out of SLSRunner and delegate to the > AMSimulator instance. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7814) Remove automatic mounting of the cgroups root directory into Docker containers
[
https://issues.apache.org/jira/browse/YARN-7814?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16338552#comment-16338552
]
Shane Kumpf commented on YARN-7814:
---
Thanks, [~miklos.szeg...@cloudera.com]. I did see that one and considered just
re-purposing it, but it seemed it was aiming towards adding other options as
well. If you'd prefer, we can re-purpose that for outright removal of the
automatic mounting. I also think YARN-7231 and YARN-7232 can be closed if
everyone agrees the automatic mounting can be removed.
> Remove automatic mounting of the cgroups root directory into Docker containers
> --
>
> Key: YARN-7814
> URL: https://issues.apache.org/jira/browse/YARN-7814
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Shane Kumpf
>Assignee: Shane Kumpf
>Priority: Major
>
> Currently, all Docker containers launched by {{DockerLinuxContainerRuntime}}
> get /sys/fs/cgroup automatically mounted. Now that user supplied mounts
> (YARN-5534) are in, containers that require this mount can request it (with a
> properly configured mount whitelist).
> I propose we remove the automatic mounting of /sys/fs/cgroup into Docker
> containers.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7523) Introduce description and version field in Service record
[ https://issues.apache.org/jira/browse/YARN-7523?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16338550#comment-16338550 ] Gour Saha commented on YARN-7523: - [~csingh] the artifact at the Service level is for convenience purpose, such that if the Service has multiple components and if most of them use the same artifact, then it can be specified at the global level rather than repeating it at every component. However note, there could be other components which can have different artifacts. So this Service level artifact is not a representation of the version of the Service. For upgrade purposes, it is a good idea to explicitly expose a version attribute which represents the Service version. > Introduce description and version field in Service record > - > > Key: YARN-7523 > URL: https://issues.apache.org/jira/browse/YARN-7523 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Gour Saha >Priority: Major > Fix For: yarn-native-services > > > YARN-7512 would need version field in Service record. It would be good to > introduce a description field also to allow service owners to capture some > details which can be used to display in Service catalog as well. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7102) NM heartbeat stuck when responseId overflows MAX_INT
[
https://issues.apache.org/jira/browse/YARN-7102?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16338543#comment-16338543
]
genericqa commented on YARN-7102:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
13s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 5 new or modified test
files. {color} |
|| || || || {color:brown} branch-2 Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
36s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 9m
52s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 12m
4s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
49s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
26s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
19s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
58s{color} | {color:green} branch-2 passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
17s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
8s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 11m
12s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 11m
12s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
1m 49s{color} | {color:orange} root: The patch generated 1 new + 329 unchanged
- 16 fixed = 330 total (was 345) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
23s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
36s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
6s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 58m
31s{color} | {color:green} hadoop-yarn-server-resourcemanager in the patch
passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 18m 51s{color}
| {color:red} hadoop-sls in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
37s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}129m 51s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Unreaped Processes | hadoop-sls:1 |
| Failed junit tests | hadoop.yarn.sls.appmaster.TestAMSimulator |
| | hadoop.yarn.sls.nodemanager.TestNMSimulator |
| Timed out junit tests | org.apache.hadoop.yarn.sls.TestSLSRunner |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:17213a0 |
| JIRA Issue | YARN-7102 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12907588/YARN-7102-branch-2.v17.patch
|
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit shadedclient findbugs checkstyle |
| uname | Linux 009bc281438d 3.13.0-135-generic #184-Ubuntu SMP Wed Oct 18
11:55:51 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | branch-2 / e9a338f |
| maven | version: Apache Maven 3.3.9
(bb52d8502b132ec0a5a3f4c09453c07478323dc5; 2015-11-10T16:41:47+00:00) |
| Default Java | 1.7.0_151 |
| findbugs | v3.0.0 |
| checkstyle |
https://builds.apache.org/job/PreCommit-YARN-Build/19442/artifact/out/diff-checkstyle-root.txt
|
| Unreaped
[jira] [Commented] (YARN-6648) [GPG] Add SubClusterCleaner in Global Policy Generator
[ https://issues.apache.org/jira/browse/YARN-6648?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16338538#comment-16338538 ] Botong Huang commented on YARN-6648: The unit test failures are irrelevant. Not sure why the edits in findbugs-exclude.xml won't suppress the findbug warning... > [GPG] Add SubClusterCleaner in Global Policy Generator > -- > > Key: YARN-6648 > URL: https://issues.apache.org/jira/browse/YARN-6648 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Botong Huang >Assignee: Botong Huang >Priority: Minor > Labels: federation, gpg > Attachments: YARN-6648-YARN-2915.v1.patch, > YARN-6648-YARN-7402.v2.patch, YARN-6648-YARN-7402.v3.patch, > YARN-6648-YARN-7402.v4.patch, YARN-6648-YARN-7402.v5.patch, > YARN-6648-YARN-7402.v6.patch, YARN-6648-YARN-7402.v7.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7810) TestDockerContainerRuntime test failures due to UID lookup of a non-existent user
[
https://issues.apache.org/jira/browse/YARN-7810?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16338533#comment-16338533
]
genericqa commented on YARN-7810:
-
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
24s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 19m
24s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
3s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
23s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
40s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
11m 54s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
56s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
30s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
38s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
1s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
1s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
20s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
36s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
13m 6s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
6s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
29s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 20m
8s{color} | {color:green} hadoop-yarn-server-nodemanager in the patch passed.
{color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
20s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 73m 8s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:5b98639 |
| JIRA Issue | YARN-7810 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12907596/YARN-7810.002.patch |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit shadedclient findbugs checkstyle |
| uname | Linux df8a98b82b8d 3.13.0-135-generic #184-Ubuntu SMP Wed Oct 18
11:55:51 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / 051ad9d |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_151 |
| findbugs | v3.1.0-RC1 |
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/19445/testReport/ |
| Max. process+thread count | 303 (vs. ulimit of 5000) |
| modules | C:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager
U:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager
|
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/19445/console |
| Powered by | Apache Yetus 0.8.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> TestDockerContainerRuntime test failures due to UID lookup of
[jira] [Commented] (YARN-6648) [GPG] Add SubClusterCleaner in Global Policy Generator
[
https://issues.apache.org/jira/browse/YARN-6648?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16338529#comment-16338529
]
genericqa commented on YARN-6648:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
25s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} YARN-7402 Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 2m
56s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 17m
1s{color} | {color:green} YARN-7402 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 7m
34s{color} | {color:green} YARN-7402 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
57s{color} | {color:green} YARN-7402 passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 4m
53s{color} | {color:green} YARN-7402 passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
14m 35s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Skipped patched modules with no Java source:
hadoop-yarn-project/hadoop-yarn {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m
4s{color} | {color:red} hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api in
YARN-7402 has 1 extant Findbugs warnings. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
25s{color} | {color:red}
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-globalpolicygenerator
in YARN-7402 has 1 extant Findbugs warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 3m
9s{color} | {color:green} YARN-7402 passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
11s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
48s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 8m
49s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 8m
49s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
58s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 4m
57s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
3s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
9m 48s{color} | {color:green} patch has no errors when building and testing our
client artifacts. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Skipped patched modules with no Java source:
hadoop-yarn-project/hadoop-yarn {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
13s{color} | {color:green} hadoop-yarn-api in the patch passed. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
17s{color} | {color:green} hadoop-yarn-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
53s{color} | {color:green} hadoop-yarn-server-common in the patch passed.
{color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
30s{color} | {color:green}
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-globalpolicygenerator
generated 0 new + 0 unchanged - 1 fixed = 0 total (was 1) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 3m
15s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests
[jira] [Commented] (YARN-2185) Use pipes when localizing archives
[
https://issues.apache.org/jira/browse/YARN-2185?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16338524#comment-16338524
]
genericqa commented on YARN-2185:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
11s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
23s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 17m
55s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 13m
28s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 3m
1s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 2m
1s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
15m 32s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
46s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
39s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
23s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m
35s{color} | {color:red} hadoop-yarn-common in the patch failed. {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 12m
6s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 12m
6s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
5s{color} | {color:green} root: The patch generated 0 new + 151 unchanged - 8
fixed = 151 total (was 159) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
46s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
10m 3s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
7s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
38s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 8m
44s{color} | {color:green} hadoop-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 3m
4s{color} | {color:green} hadoop-yarn-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
32s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 99m 6s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:5b98639 |
| JIRA Issue | YARN-2185 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12907592/YARN-2185.011.patch |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit shadedclient findbugs checkstyle |
| uname | Linux 93badcf81dc0 3.13.0-135-generic #184-Ubuntu SMP Wed Oct 18
11:55:51 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / 0c559b2 |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_151 |
| findbugs | v3.1.0-RC1 |
| mvninstall |
[jira] [Commented] (YARN-7523) Introduce description and version field in Service record
[
https://issues.apache.org/jira/browse/YARN-7523?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16338511#comment-16338511
]
Chandni Singh commented on YARN-7523:
-
Service has a field {{artifact}}.
Will that not be sufficient to serve as version?
> Introduce description and version field in Service record
> -
>
> Key: YARN-7523
> URL: https://issues.apache.org/jira/browse/YARN-7523
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Gour Saha
>Priority: Major
> Fix For: yarn-native-services
>
>
> YARN-7512 would need version field in Service record. It would be good to
> introduce a description field also to allow service owners to capture some
> details which can be used to display in Service catalog as well.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (YARN-7798) Refactor SLS Reservation Creation
[
https://issues.apache.org/jira/browse/YARN-7798?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16338495#comment-16338495
]
Yufei Gu edited comment on YARN-7798 at 1/25/18 12:49 AM:
--
Thanks for filing this. [~youchen]. Looks good to me generally. Some nits:
# In class SLSRuner#startAMFromSynthGenerator(), {{maxRedRes}} and
{{maxRedDur}} seems no need to be there anymore, this applies to {{maxMapRes}}
and {{maxMapDur}} as well.
# Why do you use {{Resource#compareTo}} in {{return
o1.getResource().compareTo(o2.getResource());}} instead of
{{Resources.componentwiseMax()}}?
# This could be refactor to remove duplications.
{code:java}
Resource mapRes = Resource.newInstance(0,0);
long mapDur = 0;
if(!allMaps.isEmpty()){
mapRes = Collections.max(allMaps, maxResource).getResource();
mapDur = Collections.max(allMaps, maxDuration).getLifeTime();
}
Resource redRes = Resource.newInstance(0, 0);
long redDur = 0;
if(!allReduces.isEmpty()){
redRes = Collections.max(allReduces, maxResource).getResource();
redDur = Collections.max(allReduces, maxDuration).getLifeTime();
}
{code}
# Please fix the style issue and bugs found by QA bot.
# {{maxResource}} in method {{initReservation}} isn't descriptive. Something
like {{resourceComparator}} would be better.
was (Author: yufeigu):
Thanks for filing this. [~youchen]. Looks good to me generally. Some nits:
# In class SLSRuner#startAMFromSynthGenerator(), {{maxRedRes}} and
{{maxRedDur}} seems no need to be there anymore, this applies to {{maxMapRes}}
and {{maxMapDur}} as well.
# Why do you use {Resource#compareTo} in {{return
o1.getResource().compareTo(o2.getResource());}} instead of
{{Resources.componentwiseMax()}}?
# This could be refactor to remove duplications.
{code:java}
Resource mapRes = Resource.newInstance(0,0);
long mapDur = 0;
if(!allMaps.isEmpty()){
mapRes = Collections.max(allMaps, maxResource).getResource();
mapDur = Collections.max(allMaps, maxDuration).getLifeTime();
}
Resource redRes = Resource.newInstance(0, 0);
long redDur = 0;
if(!allReduces.isEmpty()){
redRes = Collections.max(allReduces, maxResource).getResource();
redDur = Collections.max(allReduces, maxDuration).getLifeTime();
}
{code}
# Please fix the style issue and bugs found by QA bot.
> Refactor SLS Reservation Creation
> -
>
> Key: YARN-7798
> URL: https://issues.apache.org/jira/browse/YARN-7798
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Young Chen
>Assignee: Young Chen
>Priority: Minor
> Attachments: YARN-7798.01.patch
>
>
> Move the reservation request creation out of SLSRunner and delegate to the
> AMSimulator instance.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (YARN-7798) Refactor SLS Reservation Creation
[
https://issues.apache.org/jira/browse/YARN-7798?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16338495#comment-16338495
]
Yufei Gu edited comment on YARN-7798 at 1/25/18 12:42 AM:
--
Thanks for filing this. [~youchen]. Looks good to me generally. Some nits:
# In class SLSRuner#startAMFromSynthGenerator(), {{maxRedRes}} and
{{maxRedDur}} seems no need to be there anymore, this applies to {{maxMapRes}}
and {{maxMapDur}} as well.
# Why do you use {Resource#compareTo} in {{return
o1.getResource().compareTo(o2.getResource());}} instead of
{{Resources.componentwiseMax()}}?
# This could be refactor to remove duplications.
{code:java}
Resource mapRes = Resource.newInstance(0,0);
long mapDur = 0;
if(!allMaps.isEmpty()){
mapRes = Collections.max(allMaps, maxResource).getResource();
mapDur = Collections.max(allMaps, maxDuration).getLifeTime();
}
Resource redRes = Resource.newInstance(0, 0);
long redDur = 0;
if(!allReduces.isEmpty()){
redRes = Collections.max(allReduces, maxResource).getResource();
redDur = Collections.max(allReduces, maxDuration).getLifeTime();
}
{code}
# Please fix the style issue and bugs found by QA bot.
was (Author: yufeigu):
Thanks for filing this. [~youchen]. Looks good to me generally. Some nits:
# In class SLSRuner#startAMFromSynthGenerator(), {{maxRedRes}} and
{{maxRedDur}} seems no need to be there anymore, this applies to {{maxMapRes}}
and {{maxMapDur}} as well.
# Why do you use {Resource#compareTo} in {{return
o1.getResource().compareTo(o2.getResource());}} instead of
{{Resources.componentwiseMax()}}?
# This could be refactor to remove duplications.
{code:java}
Resource mapRes = Resource.newInstance(0,0);
long mapDur = 0;
if(!allMaps.isEmpty()){
mapRes = Collections.max(allMaps, maxResource).getResource();
mapDur = Collections.max(allMaps, maxDuration).getLifeTime();
}
Resource redRes = Resource.newInstance(0, 0);
long redDur = 0;
if(!allReduces.isEmpty()){
redRes = Collections.max(allReduces, maxResource).getResource();
redDur = Collections.max(allReduces, maxDuration).getLifeTime();
}
{code}
# Please fix the style issue found by QA bot.
> Refactor SLS Reservation Creation
> -
>
> Key: YARN-7798
> URL: https://issues.apache.org/jira/browse/YARN-7798
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Young Chen
>Assignee: Young Chen
>Priority: Minor
> Attachments: YARN-7798.01.patch
>
>
> Move the reservation request creation out of SLSRunner and delegate to the
> AMSimulator instance.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7798) Refactor SLS Reservation Creation
[
https://issues.apache.org/jira/browse/YARN-7798?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16338495#comment-16338495
]
Yufei Gu commented on YARN-7798:
Thanks for filing this. [~youchen]. Looks good to me generally. Some nits:
# In class SLSRuner#startAMFromSynthGenerator(), {{maxRedRes}} and
{{maxRedDur}} seems no need to be there anymore, this applies to {{maxMapRes}}
and {{maxMapDur}} as well.
# Why do you use {Resource#compareTo} in {{return
o1.getResource().compareTo(o2.getResource());}} instead of
{{Resources.componentwiseMax()}}?
# This could be refactor to remove duplications.
{code:java}
Resource mapRes = Resource.newInstance(0,0);
long mapDur = 0;
if(!allMaps.isEmpty()){
mapRes = Collections.max(allMaps, maxResource).getResource();
mapDur = Collections.max(allMaps, maxDuration).getLifeTime();
}
Resource redRes = Resource.newInstance(0, 0);
long redDur = 0;
if(!allReduces.isEmpty()){
redRes = Collections.max(allReduces, maxResource).getResource();
redDur = Collections.max(allReduces, maxDuration).getLifeTime();
}
{code}
# Please fix the style issue found by QA bot.
> Refactor SLS Reservation Creation
> -
>
> Key: YARN-7798
> URL: https://issues.apache.org/jira/browse/YARN-7798
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Young Chen
>Assignee: Young Chen
>Priority: Minor
> Attachments: YARN-7798.01.patch
>
>
> Move the reservation request creation out of SLSRunner and delegate to the
> AMSimulator instance.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Assigned] (YARN-7816) YARN Service - Two different users are unable to launch a service of the same name
[ https://issues.apache.org/jira/browse/YARN-7816?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gour Saha reassigned YARN-7816: --- Assignee: Gour Saha I am working on a patch for this > YARN Service - Two different users are unable to launch a service of the same > name > -- > > Key: YARN-7816 > URL: https://issues.apache.org/jira/browse/YARN-7816 > Project: Hadoop YARN > Issue Type: Bug > Components: applications >Reporter: Gour Saha >Assignee: Gour Saha >Priority: Major > > Now that YARN-7605 is committed, I am able to create a service in an > unsecured cluster from cmd line as the logged in user. However when I login > as a different user, I am unable to create a service of the exact same name. > This feature should be supported in a multi-user setup. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-7816) YARN Service - Two different users are unable to launch a service of the same name
Gour Saha created YARN-7816: --- Summary: YARN Service - Two different users are unable to launch a service of the same name Key: YARN-7816 URL: https://issues.apache.org/jira/browse/YARN-7816 Project: Hadoop YARN Issue Type: Bug Components: applications Reporter: Gour Saha Now that YARN-7605 is committed, I am able to create a service in an unsecured cluster from cmd line as the logged in user. However when I login as a different user, I am unable to create a service of the exact same name. This feature should be supported in a multi-user setup. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7801) AmFilterInitializer should addFilter after fill all parameters
[ https://issues.apache.org/jira/browse/YARN-7801?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16338470#comment-16338470 ] Jian He commented on YARN-7801: --- committed to branch-2.9 too > AmFilterInitializer should addFilter after fill all parameters > -- > > Key: YARN-7801 > URL: https://issues.apache.org/jira/browse/YARN-7801 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Sumana Sathish >Assignee: Wangda Tan >Priority: Critical > Attachments: YARN-7801.001.patch > > > Existing AmFilterInitializer cannot successfully pass RM_HA_URLS parameter to > AmIpFitler because of this issue. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7626) allow regular expression matching in container-executor.cfg for devices and volumes
[
https://issues.apache.org/jira/browse/YARN-7626?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16338463#comment-16338463
]
genericqa commented on YARN-7626:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
10s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 17m
40s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
51s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
35s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
29m 18s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
42s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
1s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} cc {color} | {color:green} 1m
1s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
1s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
39s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
12m 8s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 18m 56s{color}
| {color:red} hadoop-yarn-server-nodemanager in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
31s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 63m 48s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests |
hadoop.yarn.server.nodemanager.containermanager.linux.runtime.TestDockerContainerRuntime
|
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:5b98639 |
| JIRA Issue | YARN-7626 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12907587/YARN-7626.004.patch |
| Optional Tests | asflicense compile cc mvnsite javac unit |
| uname | Linux 0a0f9ed7bed6 3.13.0-133-generic #182-Ubuntu SMP Tue Sep 19
15:49:21 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / 0c559b2 |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_151 |
| unit |
https://builds.apache.org/job/PreCommit-YARN-Build/19443/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/19443/testReport/ |
| Max. process+thread count | 329 (vs. ulimit of 5000) |
| modules | C:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager
U:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager
|
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/19443/console |
| Powered by | Apache Yetus 0.8.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> allow regular expression matching in container-executor.cfg for devices and
> volumes
> ---
>
> Key: YARN-7626
> URL: https://issues.apache.org/jira/browse/YARN-7626
> Project: Hadoop YARN
> Issue Type: New Feature
>Reporter: Zian Chen
>Assignee: Zian Chen
>Priority: Major
> Attachments: YARN-7626.001.patch,
[jira] [Updated] (YARN-7810) TestDockerContainerRuntime test failures due to UID lookup of a non-existent user
[
https://issues.apache.org/jira/browse/YARN-7810?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shane Kumpf updated YARN-7810:
--
Attachment: YARN-7810.002.patch
> TestDockerContainerRuntime test failures due to UID lookup of a non-existent
> user
> -
>
> Key: YARN-7810
> URL: https://issues.apache.org/jira/browse/YARN-7810
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Shane Kumpf
>Assignee: Shane Kumpf
>Priority: Major
> Attachments: YARN-7810.001.patch, YARN-7810.002.patch
>
>
> YARN-7782 enabled the Docker runtime feature to remap the username to uid:gid
> form for launching Docker containers. The feature does an {{id -u}} and {{id
> -G}} to get the UID and GIDs. This fails with the test user, as that user
> doesn't actually exist on the host.
> {code:java}
> [ERROR]
> testContainerLaunchWithCustomNetworks(org.apache.hadoop.yarn.server.nodemanager.containermanager.linux.runtime.TestDockerContainerRuntime)
> Time elapsed: 0.411 s <<< ERROR!
> org.apache.hadoop.yarn.server.nodemanager.containermanager.runtime.ContainerExecutionException:
>
> ExitCodeException exitCode=1: id: 'run_as_user': no such user
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.linux.runtime.DockerLinuxContainerRuntime.getUserIdInfo(DockerLinuxContainerRuntime.java:711)
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.linux.runtime.DockerLinuxContainerRuntime.launchContainer(DockerLinuxContainerRuntime.java:757)
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.linux.runtime.TestDockerContainerRuntime.testContainerLaunchWithCustomNetworks(TestDockerContainerRuntime.java:599){code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7810) TestDockerContainerRuntime test failures due to UID lookup of a non-existent user
[
https://issues.apache.org/jira/browse/YARN-7810?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16338455#comment-16338455
]
Shane Kumpf commented on YARN-7810:
---
Thanks for the review, [~eyang]. One less shell call is a good thing, thanks
for the suggestion. Attaching a patch that implements that suggestion.
> TestDockerContainerRuntime test failures due to UID lookup of a non-existent
> user
> -
>
> Key: YARN-7810
> URL: https://issues.apache.org/jira/browse/YARN-7810
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Shane Kumpf
>Assignee: Shane Kumpf
>Priority: Major
> Attachments: YARN-7810.001.patch, YARN-7810.002.patch
>
>
> YARN-7782 enabled the Docker runtime feature to remap the username to uid:gid
> form for launching Docker containers. The feature does an {{id -u}} and {{id
> -G}} to get the UID and GIDs. This fails with the test user, as that user
> doesn't actually exist on the host.
> {code:java}
> [ERROR]
> testContainerLaunchWithCustomNetworks(org.apache.hadoop.yarn.server.nodemanager.containermanager.linux.runtime.TestDockerContainerRuntime)
> Time elapsed: 0.411 s <<< ERROR!
> org.apache.hadoop.yarn.server.nodemanager.containermanager.runtime.ContainerExecutionException:
>
> ExitCodeException exitCode=1: id: 'run_as_user': no such user
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.linux.runtime.DockerLinuxContainerRuntime.getUserIdInfo(DockerLinuxContainerRuntime.java:711)
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.linux.runtime.DockerLinuxContainerRuntime.launchContainer(DockerLinuxContainerRuntime.java:757)
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.linux.runtime.TestDockerContainerRuntime.testContainerLaunchWithCustomNetworks(TestDockerContainerRuntime.java:599){code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7628) [Documentation] Documenting the ability to disable elasticity at leaf queue
[ https://issues.apache.org/jira/browse/YARN-7628?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16338444#comment-16338444 ] Hudson commented on YARN-7628: -- SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #13553 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/13553/]) YARN-7628. [Documentation] Documenting the ability to disable elasticity (wangda: rev 051ad9d63f41ae98be728a7cdbdad9b072316318) * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/CapacityScheduler.md > [Documentation] Documenting the ability to disable elasticity at leaf queue > --- > > Key: YARN-7628 > URL: https://issues.apache.org/jira/browse/YARN-7628 > Project: Hadoop YARN > Issue Type: Bug > Components: capacity scheduler >Reporter: Zian Chen >Assignee: Zian Chen >Priority: Major > Fix For: 3.1.0 > > Attachments: YARN-7628.2.patch, YARN-7628.wip.1.patch > > > Update documentation after YARN-7274 gets in. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7810) TestDockerContainerRuntime test failures due to UID lookup of a non-existent user
[
https://issues.apache.org/jira/browse/YARN-7810?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16338442#comment-16338442
]
Eric Yang commented on YARN-7810:
-
Overall, the patch looks good. One small nitpick, we can use
System.getProperty("user.name") to find the current user id.
> TestDockerContainerRuntime test failures due to UID lookup of a non-existent
> user
> -
>
> Key: YARN-7810
> URL: https://issues.apache.org/jira/browse/YARN-7810
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Shane Kumpf
>Assignee: Shane Kumpf
>Priority: Major
> Attachments: YARN-7810.001.patch
>
>
> YARN-7782 enabled the Docker runtime feature to remap the username to uid:gid
> form for launching Docker containers. The feature does an {{id -u}} and {{id
> -G}} to get the UID and GIDs. This fails with the test user, as that user
> doesn't actually exist on the host.
> {code:java}
> [ERROR]
> testContainerLaunchWithCustomNetworks(org.apache.hadoop.yarn.server.nodemanager.containermanager.linux.runtime.TestDockerContainerRuntime)
> Time elapsed: 0.411 s <<< ERROR!
> org.apache.hadoop.yarn.server.nodemanager.containermanager.runtime.ContainerExecutionException:
>
> ExitCodeException exitCode=1: id: 'run_as_user': no such user
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.linux.runtime.DockerLinuxContainerRuntime.getUserIdInfo(DockerLinuxContainerRuntime.java:711)
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.linux.runtime.DockerLinuxContainerRuntime.launchContainer(DockerLinuxContainerRuntime.java:757)
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.linux.runtime.TestDockerContainerRuntime.testContainerLaunchWithCustomNetworks(TestDockerContainerRuntime.java:599){code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7789) Should fail RM if 3rd resource type is configured but RM uses DefaultResourceCalculator
[ https://issues.apache.org/jira/browse/YARN-7789?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16338433#comment-16338433 ] Wangda Tan commented on YARN-7789: -- Changed reporter to [~ssath...@hortonworks.com], thanks [~ssath...@hortonworks.com] for reporting this issue while doing tests. > Should fail RM if 3rd resource type is configured but RM uses > DefaultResourceCalculator > --- > > Key: YARN-7789 > URL: https://issues.apache.org/jira/browse/YARN-7789 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Sumana Sathish >Assignee: Zian Chen >Priority: Critical > > We may need to revisit this behavior: Currently, RM doesn't fail if 3rd > resource type is configured, allocated containers will be automatically > assigned minimum allocation for all resource types except memory, this makes > really hard for troubleshooting. I prefer to fail RM if 3rd or more resource > type is configured inside resource-types.xml. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-7789) Should fail RM if 3rd resource type is configured but RM uses DefaultResourceCalculator
[ https://issues.apache.org/jira/browse/YARN-7789?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated YARN-7789: - Reporter: Sumana Sathish (was: Wangda Tan) > Should fail RM if 3rd resource type is configured but RM uses > DefaultResourceCalculator > --- > > Key: YARN-7789 > URL: https://issues.apache.org/jira/browse/YARN-7789 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Sumana Sathish >Assignee: Zian Chen >Priority: Critical > > We may need to revisit this behavior: Currently, RM doesn't fail if 3rd > resource type is configured, allocated containers will be automatically > assigned minimum allocation for all resource types except memory, this makes > really hard for troubleshooting. I prefer to fail RM if 3rd or more resource > type is configured inside resource-types.xml. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7801) AmFilterInitializer should addFilter after fill all parameters
[ https://issues.apache.org/jira/browse/YARN-7801?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16338430#comment-16338430 ] Wangda Tan commented on YARN-7801: -- Changed reporter to [~ssath...@hortonworks.com], thanks [~ssath...@hortonworks.com] for reporting this while doing tests. > AmFilterInitializer should addFilter after fill all parameters > -- > > Key: YARN-7801 > URL: https://issues.apache.org/jira/browse/YARN-7801 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Sumana Sathish >Assignee: Wangda Tan >Priority: Critical > Attachments: YARN-7801.001.patch > > > Existing AmFilterInitializer cannot successfully pass RM_HA_URLS parameter to > AmIpFitler because of this issue. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7801) AmFilterInitializer should addFilter after fill all parameters
[ https://issues.apache.org/jira/browse/YARN-7801?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16338428#comment-16338428 ] Wangda Tan commented on YARN-7801: -- [~jianhe], thanks for reviewing and commiting the patch, I think branch-2.9 is also needed. > AmFilterInitializer should addFilter after fill all parameters > -- > > Key: YARN-7801 > URL: https://issues.apache.org/jira/browse/YARN-7801 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Sumana Sathish >Assignee: Wangda Tan >Priority: Critical > Attachments: YARN-7801.001.patch > > > Existing AmFilterInitializer cannot successfully pass RM_HA_URLS parameter to > AmIpFitler because of this issue. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-7801) AmFilterInitializer should addFilter after fill all parameters
[ https://issues.apache.org/jira/browse/YARN-7801?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated YARN-7801: - Reporter: Sumana Sathish (was: Wangda Tan) > AmFilterInitializer should addFilter after fill all parameters > -- > > Key: YARN-7801 > URL: https://issues.apache.org/jira/browse/YARN-7801 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Sumana Sathish >Assignee: Wangda Tan >Priority: Critical > Attachments: YARN-7801.001.patch > > > Existing AmFilterInitializer cannot successfully pass RM_HA_URLS parameter to > AmIpFitler because of this issue. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-2185) Use pipes when localizing archives
[ https://issues.apache.org/jira/browse/YARN-2185?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Szegedi updated YARN-2185: - Attachment: YARN-2185.011.patch > Use pipes when localizing archives > -- > > Key: YARN-2185 > URL: https://issues.apache.org/jira/browse/YARN-2185 > Project: Hadoop YARN > Issue Type: Improvement > Components: nodemanager >Affects Versions: 2.4.0 >Reporter: Jason Lowe >Assignee: Miklos Szegedi >Priority: Major > Attachments: YARN-2185.000.patch, YARN-2185.001.patch, > YARN-2185.002.patch, YARN-2185.003.patch, YARN-2185.004.patch, > YARN-2185.005.patch, YARN-2185.006.patch, YARN-2185.007.patch, > YARN-2185.008.patch, YARN-2185.009.patch, YARN-2185.010.patch, > YARN-2185.011.patch > > > Currently the nodemanager downloads an archive to a local file, unpacks it, > and then removes it. It would be more efficient to stream the data as it's > being unpacked to avoid both the extra disk space requirements and the > additional disk activity from storing the archive. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-2185) Use pipes when localizing archives
[
https://issues.apache.org/jira/browse/YARN-2185?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16338403#comment-16338403
]
genericqa commented on YARN-2185:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
10s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
16s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 16m
37s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 13m
6s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
8s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
52s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
14m 45s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
50s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
43s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
16s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
20s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 11m
52s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 11m
52s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
6s{color} | {color:green} root: The patch generated 0 new + 151 unchanged - 8
fixed = 151 total (was 159) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
50s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
10m 27s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m
40s{color} | {color:red} hadoop-common-project/hadoop-common generated 2 new +
0 unchanged - 0 fixed = 2 total (was 0) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
42s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 8m
18s{color} | {color:green} hadoop-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 3m
5s{color} | {color:green} hadoop-yarn-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
35s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 96m 34s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:hadoop-common-project/hadoop-common |
| | Found reliance on default encoding in
org.apache.hadoop.fs.FileUtil.lambda$runCommandOnStream$0(Process):in
org.apache.hadoop.fs.FileUtil.lambda$runCommandOnStream$0(Process): new
java.io.InputStreamReader(InputStream) At FileUtil.java:[line 716] |
| | Found reliance on default encoding in
org.apache.hadoop.fs.FileUtil.lambda$runCommandOnStream$1(Process):in
org.apache.hadoop.fs.FileUtil.lambda$runCommandOnStream$1(Process): new
java.io.InputStreamReader(InputStream) At FileUtil.java:[line 738] |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:5b98639 |
| JIRA Issue | YARN-2185 |
| JIRA Patch URL |
[jira] [Commented] (YARN-7814) Remove automatic mounting of the cgroups root directory into Docker containers
[
https://issues.apache.org/jira/browse/YARN-7814?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16338401#comment-16338401
]
Miklos Szegedi commented on YARN-7814:
--
Thank you for reporting this. Is not this a dupe of YARN-7233? I think we need
to resolve one or another.
> Remove automatic mounting of the cgroups root directory into Docker containers
> --
>
> Key: YARN-7814
> URL: https://issues.apache.org/jira/browse/YARN-7814
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Shane Kumpf
>Assignee: Shane Kumpf
>Priority: Major
>
> Currently, all Docker containers launched by {{DockerLinuxContainerRuntime}}
> get /sys/fs/cgroup automatically mounted. Now that user supplied mounts
> (YARN-5534) are in, containers that require this mount can request it (with a
> properly configured mount whitelist).
> I propose we remove the automatic mounting of /sys/fs/cgroup into Docker
> containers.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7626) allow regular expression matching in container-executor.cfg for devices and volumes
[ https://issues.apache.org/jira/browse/YARN-7626?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16338393#comment-16338393 ] Zian Chen commented on YARN-7626: - Update patch 004 to fix failed cases > allow regular expression matching in container-executor.cfg for devices and > volumes > --- > > Key: YARN-7626 > URL: https://issues.apache.org/jira/browse/YARN-7626 > Project: Hadoop YARN > Issue Type: New Feature >Reporter: Zian Chen >Assignee: Zian Chen >Priority: Major > Attachments: YARN-7626.001.patch, YARN-7626.002.patch, > YARN-7626.003.patch, YARN-7626.004.patch > > > Currently when we config some of the GPU devices related fields (like ) in > container-executor.cfg, these fields are generated based on different driver > versions or GPU device names. We want to enable regular expression matching > so that user don't need to manually set up these fields when config > container-executor.cfg, -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-7102) NM heartbeat stuck when responseId overflows MAX_INT
[ https://issues.apache.org/jira/browse/YARN-7102?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Botong Huang updated YARN-7102: --- Attachment: YARN-7102-branch-2.v17.patch > NM heartbeat stuck when responseId overflows MAX_INT > > > Key: YARN-7102 > URL: https://issues.apache.org/jira/browse/YARN-7102 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Botong Huang >Assignee: Botong Huang >Priority: Critical > Attachments: YARN-7102-branch-2.8.v10.patch, > YARN-7102-branch-2.8.v11.patch, YARN-7102-branch-2.8.v14.patch, > YARN-7102-branch-2.8.v14.patch, YARN-7102-branch-2.8.v17.patch, > YARN-7102-branch-2.8.v9.patch, YARN-7102-branch-2.v14.patch, > YARN-7102-branch-2.v14.patch, YARN-7102-branch-2.v17.patch, > YARN-7102-branch-2.v17.patch, YARN-7102-branch-2.v9.patch, > YARN-7102-branch-2.v9.patch, YARN-7102-branch-2.v9.patch, YARN-7102.v1.patch, > YARN-7102.v12.patch, YARN-7102.v13.patch, YARN-7102.v14.patch, > YARN-7102.v15.patch, YARN-7102.v16.patch, YARN-7102.v17.patch, > YARN-7102.v17.patch, YARN-7102.v2.patch, YARN-7102.v3.patch, > YARN-7102.v4.patch, YARN-7102.v5.patch, YARN-7102.v6.patch, > YARN-7102.v7.patch, YARN-7102.v8.patch, YARN-7102.v9.patch > > > ResponseId overflow problem in NM-RM heartbeat. This is same as AM-RM > heartbeat in YARN-6640, please refer to YARN-6640 for details. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-7102) NM heartbeat stuck when responseId overflows MAX_INT
[ https://issues.apache.org/jira/browse/YARN-7102?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Botong Huang updated YARN-7102: --- Attachment: YARN-7102.v17.patch > NM heartbeat stuck when responseId overflows MAX_INT > > > Key: YARN-7102 > URL: https://issues.apache.org/jira/browse/YARN-7102 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Botong Huang >Assignee: Botong Huang >Priority: Critical > Attachments: YARN-7102-branch-2.8.v10.patch, > YARN-7102-branch-2.8.v11.patch, YARN-7102-branch-2.8.v14.patch, > YARN-7102-branch-2.8.v14.patch, YARN-7102-branch-2.8.v17.patch, > YARN-7102-branch-2.8.v9.patch, YARN-7102-branch-2.v14.patch, > YARN-7102-branch-2.v14.patch, YARN-7102-branch-2.v17.patch, > YARN-7102-branch-2.v17.patch, YARN-7102-branch-2.v9.patch, > YARN-7102-branch-2.v9.patch, YARN-7102-branch-2.v9.patch, YARN-7102.v1.patch, > YARN-7102.v12.patch, YARN-7102.v13.patch, YARN-7102.v14.patch, > YARN-7102.v15.patch, YARN-7102.v16.patch, YARN-7102.v17.patch, > YARN-7102.v17.patch, YARN-7102.v2.patch, YARN-7102.v3.patch, > YARN-7102.v4.patch, YARN-7102.v5.patch, YARN-7102.v6.patch, > YARN-7102.v7.patch, YARN-7102.v8.patch, YARN-7102.v9.patch > > > ResponseId overflow problem in NM-RM heartbeat. This is same as AM-RM > heartbeat in YARN-6640, please refer to YARN-6640 for details. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-7626) allow regular expression matching in container-executor.cfg for devices and volumes
[ https://issues.apache.org/jira/browse/YARN-7626?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zian Chen updated YARN-7626: Attachment: YARN-7626.004.patch > allow regular expression matching in container-executor.cfg for devices and > volumes > --- > > Key: YARN-7626 > URL: https://issues.apache.org/jira/browse/YARN-7626 > Project: Hadoop YARN > Issue Type: New Feature >Reporter: Zian Chen >Assignee: Zian Chen >Priority: Major > Attachments: YARN-7626.001.patch, YARN-7626.002.patch, > YARN-7626.003.patch, YARN-7626.004.patch > > > Currently when we config some of the GPU devices related fields (like ) in > container-executor.cfg, these fields are generated based on different driver > versions or GPU device names. We want to enable regular expression matching > so that user don't need to manually set up these fields when config > container-executor.cfg, -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7102) NM heartbeat stuck when responseId overflows MAX_INT
[ https://issues.apache.org/jira/browse/YARN-7102?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16338391#comment-16338391 ] Botong Huang commented on YARN-7102: The unit test failures for branch-2.8.v17 patch all seem irrelevant. The only one I can re-pro locally is TestRMWebServiceAppsNodelabel.testAppsRunning, which is already failing before this patch. > NM heartbeat stuck when responseId overflows MAX_INT > > > Key: YARN-7102 > URL: https://issues.apache.org/jira/browse/YARN-7102 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Botong Huang >Assignee: Botong Huang >Priority: Critical > Attachments: YARN-7102-branch-2.8.v10.patch, > YARN-7102-branch-2.8.v11.patch, YARN-7102-branch-2.8.v14.patch, > YARN-7102-branch-2.8.v14.patch, YARN-7102-branch-2.8.v17.patch, > YARN-7102-branch-2.8.v9.patch, YARN-7102-branch-2.v14.patch, > YARN-7102-branch-2.v14.patch, YARN-7102-branch-2.v17.patch, > YARN-7102-branch-2.v9.patch, YARN-7102-branch-2.v9.patch, > YARN-7102-branch-2.v9.patch, YARN-7102.v1.patch, YARN-7102.v12.patch, > YARN-7102.v13.patch, YARN-7102.v14.patch, YARN-7102.v15.patch, > YARN-7102.v16.patch, YARN-7102.v17.patch, YARN-7102.v2.patch, > YARN-7102.v3.patch, YARN-7102.v4.patch, YARN-7102.v5.patch, > YARN-7102.v6.patch, YARN-7102.v7.patch, YARN-7102.v8.patch, YARN-7102.v9.patch > > > ResponseId overflow problem in NM-RM heartbeat. This is same as AM-RM > heartbeat in YARN-6640, please refer to YARN-6640 for details. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7792) Merge work for YARN-6592
[ https://issues.apache.org/jira/browse/YARN-7792?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16338374#comment-16338374 ] Arun Suresh commented on YARN-7792: --- Updating patch after rebasing the patch. > Merge work for YARN-6592 > > > Key: YARN-7792 > URL: https://issues.apache.org/jira/browse/YARN-7792 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Sunil G >Priority: Blocker > Attachments: YARN-6592.001.patch, YARN-7792.002.patch > > > This Jira is to run aggregated YARN-6592 branch patch against trunk and check > for any jenkins issues. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (YARN-7792) Merge work for YARN-6592
[ https://issues.apache.org/jira/browse/YARN-7792?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16338374#comment-16338374 ] Arun Suresh edited comment on YARN-7792 at 1/24/18 10:53 PM: - Updating patch after rebasing with trunk. was (Author: asuresh): Updating patch after rebasing the patch. > Merge work for YARN-6592 > > > Key: YARN-7792 > URL: https://issues.apache.org/jira/browse/YARN-7792 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Sunil G >Priority: Blocker > Attachments: YARN-6592.001.patch, YARN-7792.002.patch > > > This Jira is to run aggregated YARN-6592 branch patch against trunk and check > for any jenkins issues. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-7792) Merge work for YARN-6592
[ https://issues.apache.org/jira/browse/YARN-7792?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun Suresh updated YARN-7792: -- Attachment: YARN-7792.002.patch > Merge work for YARN-6592 > > > Key: YARN-7792 > URL: https://issues.apache.org/jira/browse/YARN-7792 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Sunil G >Priority: Blocker > Attachments: YARN-6592.001.patch, YARN-7792.002.patch > > > This Jira is to run aggregated YARN-6592 branch patch against trunk and check > for any jenkins issues. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7777) Fix user name format in YARN Registry DNS name
[ https://issues.apache.org/jira/browse/YARN-?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16338359#comment-16338359 ] Hudson commented on YARN-: -- SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #13552 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/13552/]) YARN-. Fix user name format in YARN Registry DNS name. Contributed (billie: rev 0c559b27820d12ebe5c315c6e2d1685c6de6bd33) * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/yarn-service/Configurations.md * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-services/hadoop-yarn-services-core/src/main/java/org/apache/hadoop/yarn/service/ServiceScheduler.java * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-registry/src/test/java/org/apache/hadoop/registry/client/binding/TestRegistryPathUtils.java * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-services/hadoop-yarn-services-core/src/main/java/org/apache/hadoop/yarn/service/utils/ServiceApiUtil.java * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-registry/src/main/java/org/apache/hadoop/registry/server/dns/BaseServiceRecordProcessor.java * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-registry/src/main/java/org/apache/hadoop/registry/client/binding/RegistryPathUtils.java * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-registry/src/main/java/org/apache/hadoop/registry/client/binding/RegistryUtils.java > Fix user name format in YARN Registry DNS name > --- > > Key: YARN- > URL: https://issues.apache.org/jira/browse/YARN- > Project: Hadoop YARN > Issue Type: Bug >Reporter: Jian He >Assignee: Jian He >Priority: Major > Fix For: 3.1.0 > > Attachments: YARN-.01.patch, YARN-.02.patch, > YARN-.03.patch, YARN-.04.patch, YARN-.05.patch > > > user name that has "\_" should be converted to user "-", because DNS name > doesn't allow "_" -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Assigned] (YARN-7815) Mount the filecache as read-only in Docker containers
[ https://issues.apache.org/jira/browse/YARN-7815?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shane Kumpf reassigned YARN-7815: - Assignee: Shane Kumpf > Mount the filecache as read-only in Docker containers > - > > Key: YARN-7815 > URL: https://issues.apache.org/jira/browse/YARN-7815 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Shane Kumpf >Assignee: Shane Kumpf >Priority: Major > > Currently, when using the Docker runtime, the filecache directories are > mounted read-write into the Docker containers. Read write access is not > necessary. We should make this more restrictive by changing that mount to > read-only. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7815) Mount the filecache as read-only in Docker containers
[ https://issues.apache.org/jira/browse/YARN-7815?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16338353#comment-16338353 ] Shane Kumpf commented on YARN-7815: --- In my testing, this was the only mount that could be made read-only without impact. I can work on a patch. > Mount the filecache as read-only in Docker containers > - > > Key: YARN-7815 > URL: https://issues.apache.org/jira/browse/YARN-7815 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Shane Kumpf >Priority: Major > > Currently, when using the Docker runtime, the filecache directories are > mounted read-write into the Docker containers. Read write access is not > necessary. We should make this more restrictive by changing that mount to > read-only. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-7815) Mount the filecache as read-only in Docker containers
Shane Kumpf created YARN-7815: - Summary: Mount the filecache as read-only in Docker containers Key: YARN-7815 URL: https://issues.apache.org/jira/browse/YARN-7815 Project: Hadoop YARN Issue Type: Sub-task Reporter: Shane Kumpf Currently, when using the Docker runtime, the filecache directories are mounted read-write into the Docker containers. Read write access is not necessary. We should make this more restrictive by changing that mount to read-only. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Assigned] (YARN-7814) Remove automatic mounting of the cgroups root directory into Docker containers
[
https://issues.apache.org/jira/browse/YARN-7814?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shane Kumpf reassigned YARN-7814:
-
Assignee: Shane Kumpf
> Remove automatic mounting of the cgroups root directory into Docker containers
> --
>
> Key: YARN-7814
> URL: https://issues.apache.org/jira/browse/YARN-7814
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Shane Kumpf
>Assignee: Shane Kumpf
>Priority: Major
>
> Currently, all Docker containers launched by {{DockerLinuxContainerRuntime}}
> get /sys/fs/cgroup automatically mounted. Now that user supplied mounts
> (YARN-5534) are in, containers that require this mount can request it (with a
> properly configured mount whitelist).
> I propose we remove the automatic mounting of /sys/fs/cgroup into Docker
> containers.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7814) Remove automatic mounting of the cgroups root directory into Docker containers
[
https://issues.apache.org/jira/browse/YARN-7814?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16338337#comment-16338337
]
Shane Kumpf commented on YARN-7814:
---
I can take this.
> Remove automatic mounting of the cgroups root directory into Docker containers
> --
>
> Key: YARN-7814
> URL: https://issues.apache.org/jira/browse/YARN-7814
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Shane Kumpf
>Priority: Major
>
> Currently, all Docker containers launched by {{DockerLinuxContainerRuntime}}
> get /sys/fs/cgroup automatically mounted. Now that user supplied mounts
> (YARN-5534) are in, containers that require this mount can request it (with a
> properly configured mount whitelist).
> I propose we remove the automatic mounting of /sys/fs/cgroup into Docker
> containers.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-7814) Remove automatic mounting of the cgroups root directory into Docker containers
[
https://issues.apache.org/jira/browse/YARN-7814?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shane Kumpf updated YARN-7814:
--
Summary: Remove automatic mounting of the cgroups root directory into
Docker containers (was: Remove automatic mounting of cgroups into Docker
containers)
> Remove automatic mounting of the cgroups root directory into Docker containers
> --
>
> Key: YARN-7814
> URL: https://issues.apache.org/jira/browse/YARN-7814
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Shane Kumpf
>Priority: Major
>
> Currently, all Docker containers launched by {{DockerLinuxContainerRuntime}}
> get /sys/fs/cgroup automatically mounted. Now that user supplied mounts
> (YARN-5534) are in, containers that require this mount can request it (with a
> properly configured mount whitelist).
> I propose we remove the automatic mounting of /sys/fs/cgroup into Docker
> containers.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-7814) Remove automatic mounting of cgroups into Docker containers
Shane Kumpf created YARN-7814:
-
Summary: Remove automatic mounting of cgroups into Docker
containers
Key: YARN-7814
URL: https://issues.apache.org/jira/browse/YARN-7814
Project: Hadoop YARN
Issue Type: Sub-task
Reporter: Shane Kumpf
Currently, all Docker containers launched by {{DockerLinuxContainerRuntime}}
get /sys/fs/cgroup automatically mounted. Now that user supplied mounts
(YARN-5534) are in, containers that require this mount can request it (with a
properly configured mount whitelist).
I propose we remove the automatic mounting of /sys/fs/cgroup into Docker
containers.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7795) Fix jenkins issues of YARN-6592 branch
[ https://issues.apache.org/jira/browse/YARN-7795?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16338324#comment-16338324 ] Arun Suresh commented on YARN-7795: --- +1 Committing this shortly > Fix jenkins issues of YARN-6592 branch > -- > > Key: YARN-7795 > URL: https://issues.apache.org/jira/browse/YARN-7795 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Sunil G >Assignee: Sunil G >Priority: Blocker > Attachments: YARN-7795-YARN-6592.001.patch > > > Refer link . Also fix the javadoc errors as well. > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7813) Capacity Scheduler Intra-queue Preemption should be configurable for each queue
[
https://issues.apache.org/jira/browse/YARN-7813?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16338321#comment-16338321
]
Eric Payne commented on YARN-7813:
--
A grid administrator should be able to configure queues as follows. Currently,
there is no way to configure a grid cluster like this:
{panel:title=VALID CONFIGURATION}
|Queue Name|cross-queue preemption enabled|in-queue preemption enabled|
|QueueA|true|true|
|QueueB|true|false|
{panel}
Currently, if system-wide in-queue preemption is enabled, it would be enabled
for both {{QueueA}} and {{QueueB}}. In the above use case, the administrator
wants to turn in-queue preemption on for {{QueueA}} but wants it off for
{{QueueB}}.
A nuance of this feature is that in order for in-queue preemption to be
enabled, cross-queue preemption must also be enabled. That is, the following
should not be allowed:
{panel:title=INVALID CONFIGURATION}
|Queue Name|cross-queue preemption|in-queue preemption|
|QueueA|false|true|
|QueueB|false|true|
{panel}
The reason for this is because there is no guarantee that the scheduler will
give a preempted container back to the queue that had it originally. So, if
{{QueueA}} in the invalid configuration example is over its capacity, there is
no way to tell the scheduler that a container was preempted for the purposes of
balancing {{QueueA}}'s priority or user limit inversions. Instead, the capacity
scheduler would first try to balance each queue's capacity guarantees by giving
the container to {{QueueB}}.
> Capacity Scheduler Intra-queue Preemption should be configurable for each
> queue
> ---
>
> Key: YARN-7813
> URL: https://issues.apache.org/jira/browse/YARN-7813
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: capacity scheduler, scheduler preemption
>Affects Versions: 2.9.0, 2.8.3, 3.0.0
>Reporter: Eric Payne
>Assignee: Eric Payne
>Priority: Major
>
> Just as inter-queue (a.k.a. cross-queue) preemption is configurable per
> queue, intra-queue (a.k.a. in-queue) preemption should be configurable per
> queue. If a queue does not have a setting for intra-queue preemption, it
> should inherit its parents value.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-7810) TestDockerContainerRuntime test failures due to UID lookup of a non-existent user
[
https://issues.apache.org/jira/browse/YARN-7810?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shane Kumpf updated YARN-7810:
--
Issue Type: Sub-task (was: Bug)
Parent: YARN-3611
> TestDockerContainerRuntime test failures due to UID lookup of a non-existent
> user
> -
>
> Key: YARN-7810
> URL: https://issues.apache.org/jira/browse/YARN-7810
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Shane Kumpf
>Assignee: Shane Kumpf
>Priority: Major
> Attachments: YARN-7810.001.patch
>
>
> YARN-7782 enabled the Docker runtime feature to remap the username to uid:gid
> form for launching Docker containers. The feature does an {{id -u}} and {{id
> -G}} to get the UID and GIDs. This fails with the test user, as that user
> doesn't actually exist on the host.
> {code:java}
> [ERROR]
> testContainerLaunchWithCustomNetworks(org.apache.hadoop.yarn.server.nodemanager.containermanager.linux.runtime.TestDockerContainerRuntime)
> Time elapsed: 0.411 s <<< ERROR!
> org.apache.hadoop.yarn.server.nodemanager.containermanager.runtime.ContainerExecutionException:
>
> ExitCodeException exitCode=1: id: 'run_as_user': no such user
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.linux.runtime.DockerLinuxContainerRuntime.getUserIdInfo(DockerLinuxContainerRuntime.java:711)
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.linux.runtime.DockerLinuxContainerRuntime.launchContainer(DockerLinuxContainerRuntime.java:757)
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.linux.runtime.TestDockerContainerRuntime.testContainerLaunchWithCustomNetworks(TestDockerContainerRuntime.java:599){code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7102) NM heartbeat stuck when responseId overflows MAX_INT
[
https://issues.apache.org/jira/browse/YARN-7102?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16338310#comment-16338310
]
genericqa commented on YARN-7102:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 21m
47s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 5 new or modified test
files. {color} |
|| || || || {color:brown} branch-2.8 Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
18s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
55s{color} | {color:green} branch-2.8 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 6m
57s{color} | {color:green} branch-2.8 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
0s{color} | {color:green} branch-2.8 passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
59s{color} | {color:green} branch-2.8 passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
39s{color} | {color:green} branch-2.8 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
38s{color} | {color:green} branch-2.8 passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
15s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
47s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 6m
31s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 6m
31s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 59s{color} | {color:orange} root: The patch generated 1 new + 132 unchanged
- 21 fixed = 133 total (was 153) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
55s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
59s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
44s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 81m 26s{color}
| {color:red} hadoop-yarn-server-resourcemanager in the patch failed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m
2s{color} | {color:green} hadoop-sls in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
26s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}138m 32s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests |
hadoop.yarn.server.resourcemanager.webapp.TestRMWebServiceAppsNodelabel |
| | hadoop.yarn.server.resourcemanager.TestClientRMTokens |
| | hadoop.yarn.server.resourcemanager.TestAMAuthorization |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:c2d96dd |
| JIRA Issue | YARN-7102 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12907552/YARN-7102-branch-2.8.v17.patch
|
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit shadedclient findbugs checkstyle |
| uname | Linux 1f385bac1c94 4.4.0-64-generic #85-Ubuntu SMP Mon Feb 20
11:50:30 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | branch-2.8 / 176613a |
| maven | version: Apache Maven 3.0.5 |
| Default Java | 1.7.0_151 |
| findbugs | v3.0.0 |
| checkstyle |
https://builds.apache.org/job/PreCommit-YARN-Build/19435/artifact/out/diff-checkstyle-root.txt
|
| unit |
[jira] [Commented] (YARN-7102) NM heartbeat stuck when responseId overflows MAX_INT
[
https://issues.apache.org/jira/browse/YARN-7102?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16338298#comment-16338298
]
genericqa commented on YARN-7102:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 20m
10s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 5 new or modified test
files. {color} |
|| || || || {color:brown} branch-2.8 Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
20s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
41s{color} | {color:green} branch-2.8 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 7m
15s{color} | {color:green} branch-2.8 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
7s{color} | {color:green} branch-2.8 passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
8s{color} | {color:green} branch-2.8 passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
49s{color} | {color:green} branch-2.8 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
44s{color} | {color:green} branch-2.8 passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
15s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
54s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 7m
30s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 7m
30s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
1m 6s{color} | {color:orange} root: The patch generated 1 new + 131 unchanged
- 21 fixed = 132 total (was 152) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
8s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
14s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
42s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 81m 31s{color}
| {color:red} hadoop-yarn-server-resourcemanager in the patch failed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
59s{color} | {color:green} hadoop-sls in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
25s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}140m 17s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests |
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerLazyPreemption
|
| | hadoop.yarn.server.resourcemanager.webapp.TestRMWebServiceAppsNodelabel |
| | hadoop.yarn.server.resourcemanager.TestAMAuthorization |
| | hadoop.yarn.server.resourcemanager.TestClientRMTokens |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:17213a0 |
| JIRA Issue | YARN-7102 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12907552/YARN-7102-branch-2.8.v17.patch
|
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit shadedclient findbugs checkstyle |
| uname | Linux 5b77856b0851 3.13.0-137-generic #186-Ubuntu SMP Mon Dec 4
19:09:19 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | branch-2.8 / 176613a |
| maven | version: Apache Maven 3.3.9
(bb52d8502b132ec0a5a3f4c09453c07478323dc5; 2015-11-10T16:41:47+00:00) |
| Default Java | 1.7.0_151 |
| findbugs | v3.0.0 |
| checkstyle |
[jira] [Commented] (YARN-7777) Fix user name format in YARN Registry DNS name
[
https://issues.apache.org/jira/browse/YARN-?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16338294#comment-16338294
]
genericqa commented on YARN-:
-
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
19s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
11s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 15m
41s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 7m
27s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
56s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
25s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
11m 57s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Skipped patched modules with no Java source:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
20s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
13s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
11s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
58s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 6m
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 6m
22s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 58s{color} | {color:orange} hadoop-yarn-project/hadoop-yarn: The patch
generated 2 new + 55 unchanged - 1 fixed = 57 total (was 56) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
26s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
10m 17s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Skipped patched modules with no Java source:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
33s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
4s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m
2s{color} | {color:green} hadoop-yarn-registry in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 3m
36s{color} | {color:green} hadoop-yarn-services-core in the patch passed.
{color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
17s{color} | {color:green} hadoop-yarn-site in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
32s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 68m 12s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:5b98639 |
| JIRA Issue | YARN- |
| JIRA Patch URL |
[jira] [Commented] (YARN-7777) Fix user name format in YARN Registry DNS name
[
https://issues.apache.org/jira/browse/YARN-?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16338287#comment-16338287
]
genericqa commented on YARN-:
-
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
11s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
12s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 18m
21s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 7m
38s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
1s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
24s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
12m 39s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Skipped patched modules with no Java source:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
19s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
3s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
11s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
56s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 6m
21s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 6m
21s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 58s{color} | {color:orange} hadoop-yarn-project/hadoop-yarn: The patch
generated 2 new + 54 unchanged - 1 fixed = 56 total (was 55) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
11m 1s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Skipped patched modules with no Java source:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
31s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
0s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
51s{color} | {color:green} hadoop-yarn-registry in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 3m
33s{color} | {color:green} hadoop-yarn-services-core in the patch passed.
{color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
18s{color} | {color:green} hadoop-yarn-site in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
31s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 71m 35s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:5b98639 |
| JIRA Issue | YARN- |
| JIRA Patch URL |
[jira] [Created] (YARN-7813) Capacity Scheduler Intra-queue Preemption should be configurable for each queue
Eric Payne created YARN-7813: Summary: Capacity Scheduler Intra-queue Preemption should be configurable for each queue Key: YARN-7813 URL: https://issues.apache.org/jira/browse/YARN-7813 Project: Hadoop YARN Issue Type: Improvement Components: capacity scheduler, scheduler preemption Affects Versions: 3.0.0, 2.8.3, 2.9.0 Reporter: Eric Payne Assignee: Eric Payne Just as inter-queue (a.k.a. cross-queue) preemption is configurable per queue, intra-queue (a.k.a. in-queue) preemption should be configurable per queue. If a queue does not have a setting for intra-queue preemption, it should inherit its parents value. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7777) Fix user name format in YARN Registry DNS name
[ https://issues.apache.org/jira/browse/YARN-?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16338285#comment-16338285 ] Billie Rinaldi commented on YARN-: -- +1 for patch 05. I tested the httpd example with a username containing a dot, underscore, and capital letters. The example ran successfully; it created valid hostnames and provided the correct hostnames to the docker containers and to the user. > Fix user name format in YARN Registry DNS name > --- > > Key: YARN- > URL: https://issues.apache.org/jira/browse/YARN- > Project: Hadoop YARN > Issue Type: Bug >Reporter: Jian He >Assignee: Jian He >Priority: Major > Attachments: YARN-.01.patch, YARN-.02.patch, > YARN-.03.patch, YARN-.04.patch, YARN-.05.patch > > > user name that has "\_" should be converted to user "-", because DNS name > doesn't allow "_" -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-2185) Use pipes when localizing archives
[
https://issues.apache.org/jira/browse/YARN-2185?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16338279#comment-16338279
]
Miklos Szegedi commented on YARN-2185:
--
Thank you, [~jlowe]. I updated the patch.
{quote}makeShellPath is a pre-existing, public function,...
{quote}
If we want to be conservative with this jira I would be more conservative with
security rather than compatibility. I used both Windows and Linux but I have
never seen a legitimate path with ' or ". That said, I would rather throw an
exception in these cases rather than trying to be backward compatible for
existing malware. :) I changed the patch to have a new makeSecureShellPath
function, and I ignore Windows, since it is dead code right now, and it is not
so trivial to avoid code injection.
{quote}Attempting to get the futures from the executor could result in an
ExecutionException...
{quote}
I caught the only IOException thrown there but anyone can change the code
later, so I refactored the it a little bit to protect this scenario. I also log
now on the fly, to avoid OOM errors. Let me know, what you think.
> Use pipes when localizing archives
> --
>
> Key: YARN-2185
> URL: https://issues.apache.org/jira/browse/YARN-2185
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: nodemanager
>Affects Versions: 2.4.0
>Reporter: Jason Lowe
>Assignee: Miklos Szegedi
>Priority: Major
> Attachments: YARN-2185.000.patch, YARN-2185.001.patch,
> YARN-2185.002.patch, YARN-2185.003.patch, YARN-2185.004.patch,
> YARN-2185.005.patch, YARN-2185.006.patch, YARN-2185.007.patch,
> YARN-2185.008.patch, YARN-2185.009.patch, YARN-2185.010.patch
>
>
> Currently the nodemanager downloads an archive to a local file, unpacks it,
> and then removes it. It would be more efficient to stream the data as it's
> being unpacked to avoid both the extra disk space requirements and the
> additional disk activity from storing the archive.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-7800) Bind node constraint once a container is proposed to be placed on this node
[ https://issues.apache.org/jira/browse/YARN-7800?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun Suresh updated YARN-7800: -- Parent Issue: YARN-7812 (was: YARN-6592) > Bind node constraint once a container is proposed to be placed on this node > --- > > Key: YARN-7800 > URL: https://issues.apache.org/jira/browse/YARN-7800 > Project: Hadoop YARN > Issue Type: Sub-task > Components: RM >Reporter: Weiwei Yang >Assignee: Weiwei Yang >Priority: Major > Attachments: bind_node_constraint.pdf > > > We found when there is circular dependency between multiple scheduling > requests, allocation decisions made by placement constraint algorithm might > be conflicting with related tags. More describing of the issue please refer > to YARN-7783. > To solve this issue, a possible solution is to bind *node constraint*. If the > algorithm wants to place any new container to this node, except for checking > if it satisfies the placement constraint, it also check if it satisfies the > node constraint. For example > 1) "foo", anti-affinity with "foo" > +Implies node constraint:+ on each node, it cannot have more than 1 foo tags > 2) "bar", anti-affinity with "foo" > +Implies node constraint:+ on each node, it cannot have both "bar" and "foo" > tags > With such constraint, it works like > * req2 is placed on any of nodes, e.g n2, +a node constraint[1] is added to > n2 that constrains this node cannot have both "bar" and "foo" tags+ > * when the algorithm wants to place req1 on n2, it checks if its placement > constraint is satisfied. It should be as there is no foo container on this > node yet. > * Then the algorithm checks if the node constraint is satisfied. It is not > because it violate node constraint [1]. > This avoids to do additional re-attempt like what was done in YARN-7783. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-6621) Validate Placement Constraints
[ https://issues.apache.org/jira/browse/YARN-6621?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun Suresh updated YARN-6621: -- Parent Issue: YARN-7812 (was: YARN-6592) > Validate Placement Constraints > -- > > Key: YARN-6621 > URL: https://issues.apache.org/jira/browse/YARN-6621 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Konstantinos Karanasos >Assignee: Konstantinos Karanasos >Priority: Major > > This library will be used to validate placement constraints. > It can serve multiple validation purposes: > 1) Check if the placement constraint has a valid form (e.g., a cardinality > constraint should not have an associated target expression, a DELAYED_OR > compound expression should only appear in specific places in a constraint > tree, etc.) > 2) Check if the constraints given by a user are conflicting (e.g., > cardinality more than 5 in a host and less than 3 in a rack). > 3) Check that the constraints are properly added in the Placement Constraint > Manager. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-7778) Merging of constraints defined at different levels
[ https://issues.apache.org/jira/browse/YARN-7778?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun Suresh updated YARN-7778: -- Parent Issue: YARN-7812 (was: YARN-6592) > Merging of constraints defined at different levels > -- > > Key: YARN-7778 > URL: https://issues.apache.org/jira/browse/YARN-7778 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Konstantinos Karanasos >Assignee: Weiwei Yang >Priority: Major > > When we have multiple constraints defined for a given set of allocation tags > at different levels (i.e., at the cluster, the application or the scheduling > request level), we need to merge those constraints. > Defining constraint levels as cluster > application > scheduling request, > constraints defined at lower levels should only be more restrictive than > those of higher levels. Otherwise the allocation should fail. > For example, if there is an application level constraint that allows no more > than 5 HBase containers per rack, a scheduling request can further restrict > that to 3 containers per rack but not to 7 containers per rack. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-7791) Support submit intra-app placement constraint in Distributed Shell to AppPlacementAllocator
[
https://issues.apache.org/jira/browse/YARN-7791?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Arun Suresh updated YARN-7791:
--
Parent Issue: YARN-7812 (was: YARN-6592)
> Support submit intra-app placement constraint in Distributed Shell to
> AppPlacementAllocator
> ---
>
> Key: YARN-7791
> URL: https://issues.apache.org/jira/browse/YARN-7791
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: client
> Environment: Set
> {{yarn.resourcemanager.placement-constraints.enabled}} to {{false}}
> Submit a job with placement constraint spec, e.g
> {code}
> in/yarn org.apache.hadoop.yarn.applications.distributedshell.Client -jar
> share/hadoop/yarn/hadoop-yarn-applications-distributedshell-3.1.0-SNAPSHOT.jar
> -shell_command sleep -shell_args 30 -num_containers 2 -master_memory 500
> -container_memory 200 -placement_spec foo=4,NOTIN,NODE,foo
> {code}
> got following errors in RM log
> {noformat}
> Exception message:As of now, the only accepted target key for targetKey of
> allocation_tag target expression is: [yarn_application_label/%intra_app%].
> Please make changes to placement constraints accordingly.
> {noformat}
> Looks like DS needs some modification to support submitting proper scheduling
> requests to app placement allocators.
>Reporter: Weiwei Yang
>Assignee: Sunil G
>Priority: Major
> Labels: distributedshell
> Attachments: YARN-7791-YARN-6592.001.patch
>
>
> Set {{yarn.resourcemanager.placement-constraints.enabled}} to {{false}}
> Submit a job with placement constraint spec, e.g
> {code}
> in/yarn org.apache.hadoop.yarn.applications.distributedshell.Client -jar
> share/hadoop/yarn/hadoop-yarn-applications-distributedshell-3.1.0-SNAPSHOT.jar
> -shell_command sleep -shell_args 30 -num_containers 2 -master_memory 500
> -container_memory 200 -placement_spec foo=4,NOTIN,NODE,foo
> {code}
> got following errors in RM log
> {noformat}
> Exception message:As of now, the only accepted target key for targetKey of
> allocation_tag target expression is: [yarn_application_label/%intra_app%].
> Please make changes to placement constraints accordingly.
> {noformat}
> Looks like DS needs some modification to support submitting proper scheduling
> requests to app placement allocators.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-7746) Fix PlacementProcessor to support app priority
[ https://issues.apache.org/jira/browse/YARN-7746?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun Suresh updated YARN-7746: -- Parent Issue: YARN-7812 (was: YARN-6592) > Fix PlacementProcessor to support app priority > -- > > Key: YARN-7746 > URL: https://issues.apache.org/jira/browse/YARN-7746 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Arun Suresh >Assignee: Arun Suresh >Priority: Major > > The Threadpools used in the Processor should be modified to take a priority > blocking queue that respects application priority. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-2185) Use pipes when localizing archives
[ https://issues.apache.org/jira/browse/YARN-2185?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Szegedi updated YARN-2185: - Attachment: YARN-2185.010.patch > Use pipes when localizing archives > -- > > Key: YARN-2185 > URL: https://issues.apache.org/jira/browse/YARN-2185 > Project: Hadoop YARN > Issue Type: Improvement > Components: nodemanager >Affects Versions: 2.4.0 >Reporter: Jason Lowe >Assignee: Miklos Szegedi >Priority: Major > Attachments: YARN-2185.000.patch, YARN-2185.001.patch, > YARN-2185.002.patch, YARN-2185.003.patch, YARN-2185.004.patch, > YARN-2185.005.patch, YARN-2185.006.patch, YARN-2185.007.patch, > YARN-2185.008.patch, YARN-2185.009.patch, YARN-2185.010.patch > > > Currently the nodemanager downloads an archive to a local file, unpacks it, > and then removes it. It would be more efficient to stream the data as it's > being unpacked to avoid both the extra disk space requirements and the > additional disk activity from storing the archive. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-7812) Improvements to Rich Placement Constraints in YARN
Arun Suresh created YARN-7812: - Summary: Improvements to Rich Placement Constraints in YARN Key: YARN-7812 URL: https://issues.apache.org/jira/browse/YARN-7812 Project: Hadoop YARN Issue Type: Bug Reporter: Arun Suresh -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7810) TestDockerContainerRuntime test failures due to UID lookup of a non-existent user
[
https://issues.apache.org/jira/browse/YARN-7810?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16338252#comment-16338252
]
genericqa commented on YARN-7810:
-
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
24s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 14m
40s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
42s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
15s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
30s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
9m 21s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
43s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
17s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
27s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
41s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
41s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
15s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
26s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
9m 43s{color} | {color:green} patch has no errors when building and testing our
client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
51s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
20s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 19m
5s{color} | {color:green} hadoop-yarn-server-nodemanager in the patch passed.
{color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
16s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 59m 3s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:5b98639 |
| JIRA Issue | YARN-7810 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12907557/YARN-7810.001.patch |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit shadedclient findbugs checkstyle |
| uname | Linux 8e8bd8df8f99 4.4.0-89-generic #112-Ubuntu SMP Mon Jul 31
19:38:41 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / 55c3277 |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_151 |
| findbugs | v3.1.0-RC1 |
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/19436/testReport/ |
| Max. process+thread count | 445 (vs. ulimit of 5000) |
| modules | C:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager
U:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager
|
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/19436/console |
| Powered by | Apache Yetus 0.8.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> TestDockerContainerRuntime test failures due to UID lookup of a
[jira] [Comment Edited] (YARN-7741) Eliminate extra log statement from yarn app -destroy cli
[ https://issues.apache.org/jira/browse/YARN-7741?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16338249#comment-16338249 ] Chandni Singh edited comment on YARN-7741 at 1/24/18 9:28 PM: -- slf4j api doesn't allow to change logger levels programmatically. A way to achieve this is by setting the logging level of org.apache.zookeeper package to WARN in log4j.properties was (Author: csingh): slf4j api doesn't allow to change logger levels programmatically. A way to achieve this is by setting the logging level of org.apache.zookeeper package can be set to WARN in log4j.properties > Eliminate extra log statement from yarn app -destroy cli > > > Key: YARN-7741 > URL: https://issues.apache.org/jira/browse/YARN-7741 > Project: Hadoop YARN > Issue Type: Bug > Components: yarn-native-services >Reporter: Yesha Vora >Assignee: Chandni Singh >Priority: Major > > "Yarn destroy -app " cli prints very long stacktrace from zookeeper client > which is not required. > This cli prints 44009 characters (38 lines and 358 words) > This api should only print the message whether app was destroyed successfully > or not. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
