[jira] [Updated] (YARN-6605) dafasfass
[
https://issues.apache.org/jira/browse/YARN-6605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
wuchang updated YARN-6605:
--
Description:
{code}
10 mb, 30 vcores
25 mb, 100 vcores
5 mb, 20 vcores
10 mb, 50 vcores
-1.0f
10 mb, 30 vcores
30 mb, 100 vcores
-1.0f
3 mb, 20 vcores
6 mb, 50 vcores
-1.0f
300
{code}
{code}
10 mb, 30 vcores
50 mb, 100 vcores
0.35
20
25
0.8
25000 mb, 20 vcores
225000 mb, 70 vcores
0.14
20
25
0.5
-1.0f
20 mb, 30 vcores
60 mb, 100 vcores
0.42
20
25
0.8
-1.0f
5 mb, 20 vcores
12 mb, 30 vcores
0.09
20
25
0.8
-1.0f
{code}
was:
{code}
10 mb, 30 vcores
25 mb, 100 vcores
5 mb, 20 vcores
10 mb, 50 vcores
-1.0f
10 mb, 30 vcores
30 mb, 100 vcores
-1.0f
3 mb, 20 vcores
6 mb, 50 vcores
-1.0f
300
{code}
{code}
yarn.scheduler.fair.preemption
false
{code}
> dafasfass
> -
>
> Key: YARN-6605
> URL: https://issues.apache.org/jira/browse/YARN-6605
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: wuchang
>
> {code}
>
>
>10 mb, 30 vcores
>25 mb, 100 vcores
>
>
>5 mb, 20 vcores
>10 mb, 50 vcores
>-1.0f
>
>
>10 mb, 30 vcores
>30 mb, 100 vcores
>-1.0f
>
>
>3 mb, 20 vcores
>6 mb, 50 vcores
>-1.0f
>
> 300
>
> {code}
> {code}
>
>
>10 mb, 30 vcores
>50 mb, 100 vcores
>0.35
>20
>25
>0.8
>
>
>25000 mb, 20 vcores
>225000 mb, 70 vcores
>0.14
>20
>25
>0.5
>-1.0f
>
>
>20 mb, 30 vcores
>60 mb, 100 vcores
>0.42
>20
>25
>0.8
>-1.0f
>
>
>5 mb, 20 vcores
>12 mb, 30 vcores
>0.09
>20
>25
>0.8
>-1.0f
>
>
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-6605) dafasfass
[
https://issues.apache.org/jira/browse/YARN-6605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
wuchang updated YARN-6605:
--
Description:
{code}
10 mb, 30 vcores
25 mb, 100 vcores
5 mb, 20 vcores
10 mb, 50 vcores
-1.0f
10 mb, 30 vcores
30 mb, 100 vcores
-1.0f
3 mb, 20 vcores
6 mb, 50 vcores
-1.0f
300
{code}
{code}
yarn.scheduler.fair.preemption
false
{code}
was:
{code}
10 mb, 30 vcores
25 mb, 100 vcores
5 mb, 20 vcores
10 mb, 50 vcores
-1.0f
10 mb, 30 vcores
30 mb, 100 vcores
-1.0f
3 mb, 20 vcores
6 mb, 50 vcores
-1.0f
300
{code}
> dafasfass
> -
>
> Key: YARN-6605
> URL: https://issues.apache.org/jira/browse/YARN-6605
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: wuchang
>
> {code}
>
>
>10 mb, 30 vcores
>25 mb, 100 vcores
>
>
>5 mb, 20 vcores
>10 mb, 50 vcores
>-1.0f
>
>
>10 mb, 30 vcores
>30 mb, 100 vcores
>-1.0f
>
>
>3 mb, 20 vcores
>6 mb, 50 vcores
>-1.0f
>
> 300
>
> {code}
> {code}
>
> yarn.scheduler.fair.preemption
> false
>
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-6605) dafasfass
wuchang created YARN-6605:
-
Summary: dafasfass
Key: YARN-6605
URL: https://issues.apache.org/jira/browse/YARN-6605
Project: Hadoop YARN
Issue Type: Bug
Reporter: wuchang
{code}
10 mb, 30 vcores
25 mb, 100 vcores
5 mb, 20 vcores
10 mb, 50 vcores
-1.0f
10 mb, 30 vcores
30 mb, 100 vcores
-1.0f
3 mb, 20 vcores
6 mb, 50 vcores
-1.0f
300
{code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5949) Add pluggable configuration policy interface as a component of MutableCSConfigurationProvider
[
https://issues.apache.org/jira/browse/YARN-5949?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16011610#comment-16011610
]
Hadoop QA commented on YARN-5949:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 14m
57s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
8s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 15m
19s{color} | {color:green} YARN-5734 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 16m
21s{color} | {color:green} YARN-5734 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
24s{color} | {color:green} YARN-5734 passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 2m
19s{color} | {color:green} YARN-5734 passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 1m
16s{color} | {color:green} YARN-5734 passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m
13s{color} | {color:red} hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common in
YARN-5734 has 2 extant Findbugs warnings. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m
15s{color} | {color:red}
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
in YARN-5734 has 8 extant Findbugs warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
35s{color} | {color:green} YARN-5734 passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
11s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
39s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 9m
40s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 9m
40s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 54s{color} | {color:orange} hadoop-yarn-project/hadoop-yarn: The patch
generated 3 new + 324 unchanged - 0 fixed = 327 total (was 324) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
52s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 1m
5s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 1 line(s) that end in whitespace. Use git
apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply
{color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
2s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
40s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
26s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
33s{color} | {color:green} hadoop-yarn-api in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
26s{color} | {color:green} hadoop-yarn-common in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 42m 16s{color}
| {color:red} hadoop-yarn-server-resourcemanager in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
34s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}133m 31s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.yarn.server.resourcemanager.TestRMRestart |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:0ac17dc |
| JIRA Issue | YARN-5949 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12868177/YARN-5949-YARN-5734.005.patch
|
| Optional Tests | asflicense compile javac javadoc mvninstall
[jira] [Commented] (YARN-6493) Print node partition in assignContainer logs
[
https://issues.apache.org/jira/browse/YARN-6493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16011585#comment-16011585
]
Hadoop QA commented on YARN-6493:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
15s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 14m
30s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
35s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
26s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
37s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
17s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
4s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
22s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
37s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
35s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
35s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
24s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
34s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
15s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
11s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
22s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 39m 58s{color}
| {color:red} hadoop-yarn-server-resourcemanager in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
20s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 63m 44s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.yarn.server.resourcemanager.TestRMRestart |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:14b5c93 |
| JIRA Issue | YARN-6493 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12868190/YARN-6493.003.patch |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux d2374fc1037c 3.13.0-106-generic #153-Ubuntu SMP Tue Dec 6
15:44:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / c48f297 |
| Default Java | 1.8.0_131 |
| findbugs | v3.1.0-RC1 |
| unit |
https://builds.apache.org/job/PreCommit-YARN-Build/15937/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/15937/testReport/ |
| modules | C:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
U:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
|
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/15937/console |
| Powered by | Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> Print node partition in assignContainer logs
>
>
>
[jira] [Commented] (YARN-6316) Provide help information and documentation for TimelineSchemaCreator
[
https://issues.apache.org/jira/browse/YARN-6316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16011569#comment-16011569
]
Hadoop QA commented on YARN-6316:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
19s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
43s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 13m
56s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 11m
53s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
55s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
50s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
42s{color} | {color:green} trunk passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Skipped patched modules with no Java source:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
32s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
39s{color} | {color:green} trunk passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
11s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
29s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 9m
23s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 9m
23s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
52s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
45s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
37s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Skipped patched modules with no Java source:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
41s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
37s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
27s{color} | {color:green} hadoop-yarn-server-timelineservice-hbase in the
patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
14s{color} | {color:green} hadoop-yarn-site in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
35s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 53m 43s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:14b5c93 |
| JIRA Issue | YARN-6316 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12868186/YARN-6316.00.patch |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux 509805099a23 3.13.0-106-generic #153-Ubuntu SMP Tue Dec 6
15:44:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / c48f297 |
| Default Java | 1.8.0_131 |
| findbugs | v3.1.0-RC1 |
| Test Results |
[jira] [Commented] (YARN-6603) NPE in RMAppsBlock
[
https://issues.apache.org/jira/browse/YARN-6603?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16011537#comment-16011537
]
Hadoop QA commented on YARN-6603:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 20m
47s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 15m
29s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
34s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
25s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
37s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
18s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
4s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
23s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
35s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
35s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
35s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
24s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
34s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
14s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
8s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
22s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 43m 14s{color}
| {color:red} hadoop-yarn-server-resourcemanager in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
22s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 88m 30s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Timed out junit tests |
org.apache.hadoop.yarn.server.resourcemanager.recovery.TestZKRMStateStore |
| |
org.apache.hadoop.yarn.server.resourcemanager.TestSubmitApplicationWithRMHA |
| | org.apache.hadoop.yarn.server.resourcemanager.TestKillApplicationWithRMHA
|
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:14b5c93 |
| JIRA Issue | YARN-6603 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12868169/YARN-6603.002.patch |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux f40c473f1499 3.13.0-116-generic #163-Ubuntu SMP Fri Mar 31
14:13:22 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / c48f297 |
| Default Java | 1.8.0_131 |
| findbugs | v3.1.0-RC1 |
| unit |
https://builds.apache.org/job/PreCommit-YARN-Build/15934/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/15934/testReport/ |
| modules | C:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
U:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
|
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/15934/console |
| Powered by | Apache
[jira] [Commented] (YARN-6493) Print node partition in assignContainer logs
[
https://issues.apache.org/jira/browse/YARN-6493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16011530#comment-16011530
]
Jonathan Hung commented on YARN-6493:
-
Thanks [~leftnoteasy], uploaded three patches (003 for trunk, 003 for
branch-2.8, 002 for branch-2.7).
Also added some logging in branch-2.8 in case {{allocatedContainer}} null.
> Print node partition in assignContainer logs
>
>
> Key: YARN-6493
> URL: https://issues.apache.org/jira/browse/YARN-6493
> Project: Hadoop YARN
> Issue Type: Improvement
>Affects Versions: 2.8.0, 2.7.4, 2.6.6
>Reporter: Jonathan Hung
>Assignee: Jonathan Hung
> Attachments: YARN-6493.001.patch, YARN-6493.002.patch,
> YARN-6493.003.patch, YARN-6493-branch-2.7.001.patch,
> YARN-6493-branch-2.7.002.patch, YARN-6493-branch-2.8.001.patch,
> YARN-6493-branch-2.8.002.patch, YARN-6493-branch-2.8.003.patch
>
>
> It would be useful to have the node's partition when logging a container
> allocation, for tracking purposes.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-6493) Print node partition in assignContainer logs
[ https://issues.apache.org/jira/browse/YARN-6493?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hung updated YARN-6493: Attachment: YARN-6493.003.patch YARN-6493-branch-2.8.003.patch YARN-6493-branch-2.7.002.patch > Print node partition in assignContainer logs > > > Key: YARN-6493 > URL: https://issues.apache.org/jira/browse/YARN-6493 > Project: Hadoop YARN > Issue Type: Improvement >Affects Versions: 2.8.0, 2.7.4, 2.6.6 >Reporter: Jonathan Hung >Assignee: Jonathan Hung > Attachments: YARN-6493.001.patch, YARN-6493.002.patch, > YARN-6493.003.patch, YARN-6493-branch-2.7.001.patch, > YARN-6493-branch-2.7.002.patch, YARN-6493-branch-2.8.001.patch, > YARN-6493-branch-2.8.002.patch, YARN-6493-branch-2.8.003.patch > > > It would be useful to have the node's partition when logging a container > allocation, for tracking purposes. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-6316) Provide help information and documentation for TimelineSchemaCreator
[ https://issues.apache.org/jira/browse/YARN-6316?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated YARN-6316: - Attachment: YARN-6316.00.patch Thanks for the review, [~vrushalic]. I updated the patch accordingly. Also tested the change manually. > Provide help information and documentation for TimelineSchemaCreator > > > Key: YARN-6316 > URL: https://issues.apache.org/jira/browse/YARN-6316 > Project: Hadoop YARN > Issue Type: Sub-task > Components: timelineserver >Reporter: Li Lu >Assignee: Haibo Chen > Attachments: YARN-6316.00.patch, YARN-6316.prelim.patch > > > Right now there is no help information for timeline schema creator. We may > probably want to provide an option to print help. Also, ideally, if users > passed in no argument, we may want to print out help, instead of directly > create the tables. This will simplify cluster operations and timeline v2 > deployments. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6493) Print node partition in assignContainer logs
[ https://issues.apache.org/jira/browse/YARN-6493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16011474#comment-16011474 ] Wangda Tan commented on YARN-6493: -- Thanks [~jhung] for updating the patch, could you: 1) Rename "partition" to "requestedPartition"? 2) Could you add a null check for rmcontainer in the 2.8 patch? It should not be expected happe, however I don't want it kills RM by mistake. > Print node partition in assignContainer logs > > > Key: YARN-6493 > URL: https://issues.apache.org/jira/browse/YARN-6493 > Project: Hadoop YARN > Issue Type: Improvement >Affects Versions: 2.8.0, 2.7.4, 2.6.6 >Reporter: Jonathan Hung >Assignee: Jonathan Hung > Attachments: YARN-6493.001.patch, YARN-6493.002.patch, > YARN-6493-branch-2.7.001.patch, YARN-6493-branch-2.8.001.patch, > YARN-6493-branch-2.8.002.patch > > > It would be useful to have the node's partition when logging a container > allocation, for tracking purposes. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5949) Add pluggable configuration policy interface as a component of MutableCSConfigurationProvider
[ https://issues.apache.org/jira/browse/YARN-5949?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16011463#comment-16011463 ] Jonathan Hung commented on YARN-5949: - Thanks [~leftnoteasy]! Seems reasonable. Attached 005 to address these comments. > Add pluggable configuration policy interface as a component of > MutableCSConfigurationProvider > - > > Key: YARN-5949 > URL: https://issues.apache.org/jira/browse/YARN-5949 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Jonathan Hung >Assignee: Jonathan Hung > Attachments: YARN-5949-YARN-5734.001.patch, > YARN-5949-YARN-5734.002.patch, YARN-5949-YARN-5734.003.patch, > YARN-5949-YARN-5734.004.patch, YARN-5949-YARN-5734.005.patch > > > This will allow different policies to customize how/if configuration changes > should be applied (for example, a policy might restrict whether a > configuration change by a certain user is allowed). This will be enforced by > the MutableCSConfigurationProvider. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-5949) Add pluggable configuration policy interface as a component of MutableCSConfigurationProvider
[ https://issues.apache.org/jira/browse/YARN-5949?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hung updated YARN-5949: Attachment: YARN-5949-YARN-5734.005.patch > Add pluggable configuration policy interface as a component of > MutableCSConfigurationProvider > - > > Key: YARN-5949 > URL: https://issues.apache.org/jira/browse/YARN-5949 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Jonathan Hung >Assignee: Jonathan Hung > Attachments: YARN-5949-YARN-5734.001.patch, > YARN-5949-YARN-5734.002.patch, YARN-5949-YARN-5734.003.patch, > YARN-5949-YARN-5734.004.patch, YARN-5949-YARN-5734.005.patch > > > This will allow different policies to customize how/if configuration changes > should be applied (for example, a policy might restrict whether a > configuration change by a certain user is allowed). This will be enforced by > the MutableCSConfigurationProvider. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6603) NPE in RMAppsBlock
[
https://issues.apache.org/jira/browse/YARN-6603?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16011445#comment-16011445
]
Hadoop QA commented on YARN-6603:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 19m
58s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 14m
18s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
36s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
27s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
35s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
18s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
2s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
21s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
33s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
30s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
30s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
23s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
34s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
16s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
9s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
22s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 44m 24s{color}
| {color:red} hadoop-yarn-server-resourcemanager in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
20s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 87m 28s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Timed out junit tests |
org.apache.hadoop.yarn.server.resourcemanager.recovery.TestZKRMStateStore |
| |
org.apache.hadoop.yarn.server.resourcemanager.TestSubmitApplicationWithRMHA |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:14b5c93 |
| JIRA Issue | YARN-6603 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12868157/YARN-6603.001.patch |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux 48cb7beeba8e 3.13.0-108-generic #155-Ubuntu SMP Wed Jan 11
16:58:52 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / c48f297 |
| Default Java | 1.8.0_131 |
| findbugs | v3.1.0-RC1 |
| unit |
https://builds.apache.org/job/PreCommit-YARN-Build/15932/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/15932/testReport/ |
| modules | C:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
U:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
|
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/15932/console |
| Powered by | Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org |
This message was automatically
[jira] [Commented] (YARN-6306) NMClient API change for container upgrade
[ https://issues.apache.org/jira/browse/YARN-6306?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16011431#comment-16011431 ] Jian He commented on YARN-6306: --- +1, thanks Arun! > NMClient API change for container upgrade > - > > Key: YARN-6306 > URL: https://issues.apache.org/jira/browse/YARN-6306 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Jian He >Assignee: Arun Suresh > Attachments: YARN-6306.001.patch, YARN-6306.002.patch, > YARN-6306.003.patch, YARN-6306.004.patch > > > This JIRA is track the addition of Upgrade API (Re-Initialize, Restart, > Rollback and Commit) to the NMClient and NMClientAsync -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6409) RM does not blacklist node for AM launch failures
[
https://issues.apache.org/jira/browse/YARN-6409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16011424#comment-16011424
]
Hadoop QA commented on YARN-6409:
-
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
11s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 3 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 12m
20s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
32s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
22s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
31s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
15s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
54s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
16s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
29s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
26s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
26s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 22s{color} | {color:orange}
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager:
The patch generated 1 new + 233 unchanged - 0 fixed = 234 total (was 233)
{color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
31s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
15s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
1s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
58s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
16s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 38m
1s{color} | {color:green} hadoop-yarn-server-resourcemanager in the patch
passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
18s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 58m 19s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:14b5c93 |
| JIRA Issue | YARN-6409 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12868160/YARN-6409.03.patch |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux 9b5e90a59118 4.4.0-43-generic #63-Ubuntu SMP Wed Oct 12
13:48:03 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / c48f297 |
| Default Java | 1.8.0_131 |
| findbugs | v3.1.0-RC1 |
| checkstyle |
https://builds.apache.org/job/PreCommit-YARN-Build/15933/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/15933/testReport/ |
| modules | C:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
U:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
|
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/15933/console |
| Powered by | Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> RM does not blacklist node for AM launch failures
> -
>
> Key: YARN-6409
> URL:
[jira] [Updated] (YARN-6603) NPE in RMAppsBlock
[ https://issues.apache.org/jira/browse/YARN-6603?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Lowe updated YARN-6603: - Attachment: YARN-6603.002.patch Nice catch, Daryn! Updating the patch with a new version that uses getRMAppAttempt rather than getAppAttempts. And yes, I'm sad we're making a copy of the hashset just to compute a size. However in almost all cases the set will be empty or only a few entries, so it shouldn't be a big deal in practice. > NPE in RMAppsBlock > -- > > Key: YARN-6603 > URL: https://issues.apache.org/jira/browse/YARN-6603 > Project: Hadoop YARN > Issue Type: Bug > Components: resourcemanager >Affects Versions: 2.8.0 >Reporter: Jason Lowe >Assignee: Jason Lowe > Attachments: YARN-6603.001.patch, YARN-6603.002.patch > > > We are seeing an intermittent NPE when the RM is trying to render the > /cluster URI. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6603) NPE in RMAppsBlock
[
https://issues.apache.org/jira/browse/YARN-6603?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16011414#comment-16011414
]
Daryn Sharp commented on YARN-6603:
---
After getting the rmApp, you should replace:
{code}
RMAppAttempt appAttempt = rmApp.getAppAttempts().get(appAttemptId);
{code}
with:
{code}
RMAppAttempt appAttempt = rmApp.getAppAttempt(appAttemptdId);
{code}
The current getAppAttempts() returns an unmodifiable collection of a
non-threadsafe map which isn't useful at all. The latter uses proper
synchronization to lookup the attempt.
You may also be saddened to learn that a synchronized copy of the blacklist
hashset is created just to get the size. Bonus points for fixing that, but not
necessary.
> NPE in RMAppsBlock
> --
>
> Key: YARN-6603
> URL: https://issues.apache.org/jira/browse/YARN-6603
> Project: Hadoop YARN
> Issue Type: Bug
> Components: resourcemanager
>Affects Versions: 2.8.0
>Reporter: Jason Lowe
>Assignee: Jason Lowe
> Attachments: YARN-6603.001.patch
>
>
> We are seeing an intermittent NPE when the RM is trying to render the
> /cluster URI.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6493) Print node partition in assignContainer logs
[
https://issues.apache.org/jira/browse/YARN-6493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16011394#comment-16011394
]
Hadoop QA commented on YARN-6493:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 20m
18s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
1s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 9m
17s{color} | {color:green} branch-2.8 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
32s{color} | {color:green} branch-2.8 passed with JDK v1.8.0_131 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
32s{color} | {color:green} branch-2.8 passed with JDK v1.7.0_121 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
18s{color} | {color:green} branch-2.8 passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
38s{color} | {color:green} branch-2.8 passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
17s{color} | {color:green} branch-2.8 passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
12s{color} | {color:green} branch-2.8 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
23s{color} | {color:green} branch-2.8 passed with JDK v1.8.0_131 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
23s{color} | {color:green} branch-2.8 passed with JDK v1.7.0_121 {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
31s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
31s{color} | {color:green} the patch passed with JDK v1.8.0_131 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
31s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
28s{color} | {color:green} the patch passed with JDK v1.7.0_121 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
28s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
15s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
35s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
13s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
21s{color} | {color:green} the patch passed with JDK v1.8.0_131 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
21s{color} | {color:green} the patch passed with JDK v1.7.0_121 {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 80m 48s{color}
| {color:red} hadoop-yarn-server-resourcemanager in the patch failed with JDK
v1.7.0_121. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
20s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}200m 58s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| JDK v1.8.0_131 Failed junit tests |
hadoop.yarn.server.resourcemanager.TestClientRMTokens |
| | hadoop.yarn.server.resourcemanager.TestAMAuthorization |
| JDK v1.7.0_121 Failed junit tests |
hadoop.yarn.server.resourcemanager.TestClientRMTokens |
| | hadoop.yarn.server.resourcemanager.TestAMAuthorization |
| |
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerLazyPreemption
|
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:5970e82 |
| JIRA Issue | YARN-6493 |
| JIRA Patch URL |
[jira] [Commented] (YARN-6306) NMClient API change for container upgrade
[
https://issues.apache.org/jira/browse/YARN-6306?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16011390#comment-16011390
]
Hadoop QA commented on YARN-6306:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 20m
25s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
7s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 13m
0s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 7m
25s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
46s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
53s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
34s{color} | {color:green} trunk passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
43s{color} | {color:red}
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager
in trunk has 5 extant Findbugs warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
37s{color} | {color:green} trunk passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
8s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
41s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 7m
7s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 7m
7s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 44s{color} | {color:orange} hadoop-yarn-project/hadoop-yarn: The patch
generated 12 new + 165 unchanged - 2 fixed = 177 total (was 167) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
51s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
35s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
33s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
21s{color} | {color:green} hadoop-yarn-server-nodemanager in the patch passed.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
18s{color} | {color:green} hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client
generated 0 new + 150 unchanged - 8 fixed = 150 total (was 158) {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 13m
15s{color} | {color:green} hadoop-yarn-server-nodemanager in the patch passed.
{color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 19m
22s{color} | {color:green} hadoop-yarn-client in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
24s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 98m 48s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:14b5c93 |
| JIRA Issue | YARN-6306 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12868125/YARN-6306.004.patch |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux 0196ffa63a5b 4.4.0-43-generic #63-Ubuntu SMP Wed Oct 12
13:48:03 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / c48f297 |
| Default Java | 1.8.0_131 |
| findbugs | v3.1.0-RC1 |
| findbugs |
[jira] [Commented] (YARN-2113) Add cross-user preemption within CapacityScheduler's leaf-queue
[
https://issues.apache.org/jira/browse/YARN-2113?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16011384#comment-16011384
]
Eric Payne commented on YARN-2113:
--
[~sunilg], the unit test was not failing for a couple of reasons. First, it was
still using the {{DefaultResourceCalculator}} even though the
{{TestProportionalCapacityPreemptionPolicyIntraQueueWithDRF#setup}} was setting
{{rc}} to {{DominantResourceCalculator}}.
First, I think {{when(cs.getResourceCalculator()).thenReturn(rc);}} should be
added
{code:title=TestProportionalCapacityPreemptionPolicyIntraQueueWithDRF#setup}
public void setup() {
super.setup();
conf.setBoolean(
CapacitySchedulerConfiguration.INTRAQUEUE_PREEMPTION_ENABLED, true);
rc = new DominantResourceCalculator();
when(cs.getResourceCalculator()).thenReturn(rc);
policy = new ProportionalCapacityPreemptionPolicy(rmContext, cs, mClock);
}
{code}
Second, in each of the tests, {{String labelsConfig = "=100:50,true;";}} should
be changed to {{String labelsConfig = "=100:200,true;";}}
By making these changes, the tests will fail. If I comment out the
{{when...thenReturn}}, the test succeeds again because it goes back to using
the {{DefaultResourceCalculator}}
> Add cross-user preemption within CapacityScheduler's leaf-queue
> ---
>
> Key: YARN-2113
> URL: https://issues.apache.org/jira/browse/YARN-2113
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: scheduler
>Reporter: Vinod Kumar Vavilapalli
>Assignee: Sunil G
> Attachments: IntraQueue Preemption-Impact Analysis.pdf,
> TestNoIntraQueuePreemptionIfBelowUserLimitAndDifferentPrioritiesWithExtraUsers.txt,
> YARN-2113.0001.patch, YARN-2113.0002.patch, YARN-2113.0003.patch,
> YARN-2113.0004.patch, YARN-2113.0005.patch, YARN-2113.0006.patch,
> YARN-2113.0007.patch, YARN-2113.0008.patch, YARN-2113.0009.patch,
> YARN-2113.0010.patch, YARN-2113.0011.patch, YARN-2113.0012.patch,
> YARN-2113.0013.patch, YARN-2113.0014.patch, YARN-2113.0015.patch,
> YARN-2113.0016.patch, YARN-2113.0017.patch,
> YARN-2113.apply.onto.0012.ericp.patch, YARN-2113 Intra-QueuePreemption
> Behavior.pdf, YARN-2113.v0.patch
>
>
> Preemption today only works across queues and moves around resources across
> queues per demand and usage. We should also have user-level preemption within
> a queue, to balance capacity across users in a predictable manner.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6602) Impersonation does not work if standby RM is contacted first
[
https://issues.apache.org/jira/browse/YARN-6602?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16011373#comment-16011373
]
Robert Kanter commented on YARN-6602:
-
I've narrowed down the source of the issue. {{RMProxy#getProxy}} has a
{{doAs}} block to create a {{YarnRPC}} using
{{UserGroupInformation.getCurrentUser()}} to get the UGI. The normal pattern
of a Hadoop client is to do a {{doAs}} block when creating the client, and then
all subsequent calls to that client should happen with that UGI, even if it's
used outside of a {{doAs}}. {{RMProxy#getProxy}} is getting called when
submitting the application in the case where the client first hits the standby
RM so that it can try the next RM, but not in the case where it hits the active
RM first. So when that happens, {{UserGroupInformation.getCurrentUser()}}
returns the oozie user instead of the impersonated user because submitting the
application is done outside of a {{doAs}} block.
> Impersonation does not work if standby RM is contacted first
>
>
> Key: YARN-6602
> URL: https://issues.apache.org/jira/browse/YARN-6602
> Project: Hadoop YARN
> Issue Type: Bug
> Components: client
>Affects Versions: 3.0.0-alpha3
>Reporter: Robert Kanter
>Assignee: Robert Kanter
>Priority: Blocker
>
> When RM HA is enabled, impersonation does not work correctly if the Yarn
> Client connects to the standby RM first. When this happens, the
> impersonation is "lost" and the client does things on behalf of the
> impersonator user. We saw this with the OOZIE-1770 Oozie on Yarn feature.
> I need to investigate this some more, but it appears to be related to
> delegation tokens. When this issue occurs, the tokens have the owner as
> "oozie" instead of the actual user. On a hunch, we found a workaround that
> explicitly adding a correct RM HA delegation token fixes the problem:
> {code:java}
> org.apache.hadoop.yarn.api.records.Token token =
> yarnClient.getRMDelegationToken(ClientRMProxy.getRMDelegationTokenService(conf));
> org.apache.hadoop.security.token.Token token2 = new
> org.apache.hadoop.security.token.Token(token.getIdentifier().array(),
> token.getPassword().array(), new Text(token.getKind()), new
> Text(token.getService()));
> UserGroupInformation.getCurrentUser().addToken(token2);
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-6604) Allow metric TTL for Application table to be specified through cmd
[ https://issues.apache.org/jira/browse/YARN-6604?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated YARN-6604: - Issue Type: Sub-task (was: Improvement) Parent: YARN-5355 > Allow metric TTL for Application table to be specified through cmd > -- > > Key: YARN-6604 > URL: https://issues.apache.org/jira/browse/YARN-6604 > Project: Hadoop YARN > Issue Type: Sub-task > Components: ATSv2 >Affects Versions: 3.0.0-alpha2 >Reporter: Haibo Chen >Assignee: Haibo Chen > > We should allow metrics TTL in application table to be specified in schema > cmd, as we do with metrics TTL in entity table -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-6604) Allow metric TTL for Application table to be specified through cmd
Haibo Chen created YARN-6604: Summary: Allow metric TTL for Application table to be specified through cmd Key: YARN-6604 URL: https://issues.apache.org/jira/browse/YARN-6604 Project: Hadoop YARN Issue Type: Improvement Components: ATSv2 Affects Versions: 3.0.0-alpha2 Reporter: Haibo Chen Assignee: Haibo Chen We should allow metrics TTL in application table to be specified in schema cmd, as we do with metrics TTL in entity table -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6160) Create an agent-less docker-less provider in the native services framework
[
https://issues.apache.org/jira/browse/YARN-6160?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16011357#comment-16011357
]
Hadoop QA commented on YARN-6160:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 12m
30s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 4 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 17m
6s{color} | {color:green} yarn-native-services passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
26s{color} | {color:green} yarn-native-services passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
23s{color} | {color:green} yarn-native-services passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
32s{color} | {color:green} yarn-native-services passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
23s{color} | {color:green} yarn-native-services passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
55s{color} | {color:green} yarn-native-services passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
14s{color} | {color:green} yarn-native-services passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
24s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
24s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 21s{color} | {color:orange}
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-slider/hadoop-yarn-slider-core:
The patch generated 29 new + 424 unchanged - 39 fixed = 453 total (was 463)
{color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
25s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
18s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
1s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
1s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
54s{color} | {color:red}
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-slider/hadoop-yarn-slider-core
generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
11s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
55s{color} | {color:green} hadoop-yarn-slider-core in the patch passed. {color}
|
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
17s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 40m 5s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs |
module:hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-slider/hadoop-yarn-slider-core
|
| | Dead store to config in
org.apache.slider.client.SliderClient.doClientInstall(ActionClientArgs) At
SliderClient.java:org.apache.slider.client.SliderClient.doClientInstall(ActionClientArgs)
At SliderClient.java:[line 1174] |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:0ac17dc |
| JIRA Issue | YARN-6160 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12868133/YARN-6160-yarn-native-services.002.patch
|
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle xml |
| uname | Linux a772caa815b7 4.4.0-43-generic #63-Ubuntu SMP Wed Oct 12
13:48:03 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | yarn-native-services / 8c32344 |
| Default Java | 1.8.0_131 |
| findbugs | v3.1.0-RC1 |
| checkstyle |
[jira] [Updated] (YARN-6409) RM does not blacklist node for AM launch failures
[
https://issues.apache.org/jira/browse/YARN-6409?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Haibo Chen updated YARN-6409:
-
Attachment: YARN-6409.03.patch
> RM does not blacklist node for AM launch failures
> -
>
> Key: YARN-6409
> URL: https://issues.apache.org/jira/browse/YARN-6409
> Project: Hadoop YARN
> Issue Type: Bug
> Components: resourcemanager
>Affects Versions: 3.0.0-alpha2
>Reporter: Haibo Chen
>Assignee: Haibo Chen
> Attachments: YARN-6409.00.patch, YARN-6409.01.patch,
> YARN-6409.02.patch, YARN-6409.03.patch
>
>
> Currently, node blacklisting upon AM failures only handles failures that
> happen after AM container is launched (see
> RMAppAttemptImpl.shouldCountTowardsNodeBlacklisting()). However, AM launch
> can also fail if the NM, where the AM container is allocated, goes
> unresponsive. Because it is not handled, scheduler may continue to allocate
> AM containers on that same NM for the following app attempts.
> {code}
> Application application_1478721503753_0870 failed 2 times due to Error
> launching appattempt_1478721503753_0870_02. Got exception:
> java.io.IOException: Failed on local exception: java.io.IOException:
> java.net.SocketTimeoutException: 6 millis timeout while waiting for
> channel to be ready for read. ch : java.nio.channels.SocketChannel[connected
> local=/17.111.179.113:46702 remote=*.me.com/17.111.178.125:8041]; Host
> Details : local host is: "*.me.com/17.111.179.113"; destination host is:
> "*.me.com":8041;
> at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:772)
> at org.apache.hadoop.ipc.Client.call(Client.java:1475)
> at org.apache.hadoop.ipc.Client.call(Client.java:1408)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:230)
>
> at com.sun.proxy.$Proxy86.startContainers(Unknown Source)
> at
> org.apache.hadoop.yarn.api.impl.pb.client.ContainerManagementProtocolPBClientImpl.startContainers(ContainerManagementProtocolPBClientImpl.java:96)
>
> at sun.reflect.GeneratedMethodAccessor155.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>
> at java.lang.reflect.Method.invoke(Method.java:497)
> at
> org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:256)
>
> at
> org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:104)
>
> at com.sun.proxy.$Proxy87.startContainers(Unknown Source)
> at
> org.apache.hadoop.yarn.server.resourcemanager.amlauncher.AMLauncher.launch(AMLauncher.java:120)
>
> at
> org.apache.hadoop.yarn.server.resourcemanager.amlauncher.AMLauncher.run(AMLauncher.java:256)
>
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.io.IOException: java.net.SocketTimeoutException: 6 millis
> timeout while waiting for channel to be ready for read. ch :
> java.nio.channels.SocketChannel[connected local=/17.111.179.113:46702
> remote=*.me.com/17.111.178.125:8041]
> at org.apache.hadoop.ipc.Client$Connection$1.run(Client.java:687)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1693)
>
> at
> org.apache.hadoop.ipc.Client$Connection.handleSaslConnectionFailure(Client.java:650)
>
> at org.apache.hadoop.ipc.Client$Connection.setupIOstreams(Client.java:738)
> at org.apache.hadoop.ipc.Client$Connection.access$2900(Client.java:375)
> at org.apache.hadoop.ipc.Client.getConnection(Client.java:1524)
> at org.apache.hadoop.ipc.Client.call(Client.java:1447)
> ... 15 more
> Caused by: java.net.SocketTimeoutException: 6 millis timeout while
> waiting for channel to be ready for read. ch :
> java.nio.channels.SocketChannel[connected local=/17.111.179.113:46702
> remote=*.me.com/17.111.178.125:8041]
> at
> org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:164)
> at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161)
> at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131)
> at java.io.FilterInputStream.read(FilterInputStream.java:133)
> at java.io.BufferedInputStream.fill(BufferedInputStream.java:246)
> at java.io.BufferedInputStream.read(BufferedInputStream.java:265)
> at java.io.DataInputStream.readInt(DataInputStream.java:387)
> at
> org.apache.hadoop.security.SaslRpcClient.saslConnect(SaslRpcClient.java:367)
> at
> org.apache.hadoop.ipc.Client$Connection.setupSaslConnection(Client.java:560)
[jira] [Commented] (YARN-6409) RM does not blacklist node for AM launch failures
[
https://issues.apache.org/jira/browse/YARN-6409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16011322#comment-16011322
]
Haibo Chen commented on YARN-6409:
--
Thanks [~rkanter] for your review! I am not familiar with the RPC code, not
sure if the SocketTimeoutException will always be at depth 3. But it looks to
me that the raw exception can be wrapped multiple times.
Do you think if it's safe to assume that NM connection is down once we see a
SocketTimeoutException regardless of at which depth it appears? I'll update the
patch to address checkstyle issue and test failures.
> RM does not blacklist node for AM launch failures
> -
>
> Key: YARN-6409
> URL: https://issues.apache.org/jira/browse/YARN-6409
> Project: Hadoop YARN
> Issue Type: Bug
> Components: resourcemanager
>Affects Versions: 3.0.0-alpha2
>Reporter: Haibo Chen
>Assignee: Haibo Chen
> Attachments: YARN-6409.00.patch, YARN-6409.01.patch,
> YARN-6409.02.patch
>
>
> Currently, node blacklisting upon AM failures only handles failures that
> happen after AM container is launched (see
> RMAppAttemptImpl.shouldCountTowardsNodeBlacklisting()). However, AM launch
> can also fail if the NM, where the AM container is allocated, goes
> unresponsive. Because it is not handled, scheduler may continue to allocate
> AM containers on that same NM for the following app attempts.
> {code}
> Application application_1478721503753_0870 failed 2 times due to Error
> launching appattempt_1478721503753_0870_02. Got exception:
> java.io.IOException: Failed on local exception: java.io.IOException:
> java.net.SocketTimeoutException: 6 millis timeout while waiting for
> channel to be ready for read. ch : java.nio.channels.SocketChannel[connected
> local=/17.111.179.113:46702 remote=*.me.com/17.111.178.125:8041]; Host
> Details : local host is: "*.me.com/17.111.179.113"; destination host is:
> "*.me.com":8041;
> at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:772)
> at org.apache.hadoop.ipc.Client.call(Client.java:1475)
> at org.apache.hadoop.ipc.Client.call(Client.java:1408)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:230)
>
> at com.sun.proxy.$Proxy86.startContainers(Unknown Source)
> at
> org.apache.hadoop.yarn.api.impl.pb.client.ContainerManagementProtocolPBClientImpl.startContainers(ContainerManagementProtocolPBClientImpl.java:96)
>
> at sun.reflect.GeneratedMethodAccessor155.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>
> at java.lang.reflect.Method.invoke(Method.java:497)
> at
> org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:256)
>
> at
> org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:104)
>
> at com.sun.proxy.$Proxy87.startContainers(Unknown Source)
> at
> org.apache.hadoop.yarn.server.resourcemanager.amlauncher.AMLauncher.launch(AMLauncher.java:120)
>
> at
> org.apache.hadoop.yarn.server.resourcemanager.amlauncher.AMLauncher.run(AMLauncher.java:256)
>
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.io.IOException: java.net.SocketTimeoutException: 6 millis
> timeout while waiting for channel to be ready for read. ch :
> java.nio.channels.SocketChannel[connected local=/17.111.179.113:46702
> remote=*.me.com/17.111.178.125:8041]
> at org.apache.hadoop.ipc.Client$Connection$1.run(Client.java:687)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1693)
>
> at
> org.apache.hadoop.ipc.Client$Connection.handleSaslConnectionFailure(Client.java:650)
>
> at org.apache.hadoop.ipc.Client$Connection.setupIOstreams(Client.java:738)
> at org.apache.hadoop.ipc.Client$Connection.access$2900(Client.java:375)
> at org.apache.hadoop.ipc.Client.getConnection(Client.java:1524)
> at org.apache.hadoop.ipc.Client.call(Client.java:1447)
> ... 15 more
> Caused by: java.net.SocketTimeoutException: 6 millis timeout while
> waiting for channel to be ready for read. ch :
> java.nio.channels.SocketChannel[connected local=/17.111.179.113:46702
> remote=*.me.com/17.111.178.125:8041]
> at
> org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:164)
> at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161)
> at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131)
> at
[jira] [Updated] (YARN-6603) NPE in RMAppsBlock
[ https://issues.apache.org/jira/browse/YARN-6603?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Lowe updated YARN-6603: - Attachment: YARN-6603.001.patch Patch that verifies the app is valid before trying to dig into it for attempt information. > NPE in RMAppsBlock > -- > > Key: YARN-6603 > URL: https://issues.apache.org/jira/browse/YARN-6603 > Project: Hadoop YARN > Issue Type: Bug > Components: resourcemanager >Affects Versions: 2.8.0 >Reporter: Jason Lowe > Attachments: YARN-6603.001.patch > > > We are seeing an intermittent NPE when the RM is trying to render the > /cluster URI. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Assigned] (YARN-6603) NPE in RMAppsBlock
[ https://issues.apache.org/jira/browse/YARN-6603?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Lowe reassigned YARN-6603: Assignee: Jason Lowe > NPE in RMAppsBlock > -- > > Key: YARN-6603 > URL: https://issues.apache.org/jira/browse/YARN-6603 > Project: Hadoop YARN > Issue Type: Bug > Components: resourcemanager >Affects Versions: 2.8.0 >Reporter: Jason Lowe >Assignee: Jason Lowe > Attachments: YARN-6603.001.patch > > > We are seeing an intermittent NPE when the RM is trying to render the > /cluster URI. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-2113) Add cross-user preemption within CapacityScheduler's leaf-queue
[
https://issues.apache.org/jira/browse/YARN-2113?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16011304#comment-16011304
]
Eric Payne commented on YARN-2113:
--
I'm sorry [~sunilg], but the new patch doesn't fix the problem. I can still
reproduce it with patch 0017.
Also, the new test
{{TestProportionalCapacityPreemptionPolicyIntraQueueWithDRF}} doesn't actually
test the problem. I applied patch 0016 to trunk and copied
{{TestProportionalCapacityPreemptionPolicyIntraQueueWithDRF.java}} to
{{hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/monitor/capacity/TestProportionalCapacityPreemptionPolicyIntraQueueWithDRF.java}},
and the test still succeeded.
The failure depends on _both_ of the following:
# using {{DominantResourceCalculator}}
# setting {{yarn.nodemanager.resource.cpu-vcores}} to something other than 10.
> Add cross-user preemption within CapacityScheduler's leaf-queue
> ---
>
> Key: YARN-2113
> URL: https://issues.apache.org/jira/browse/YARN-2113
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: scheduler
>Reporter: Vinod Kumar Vavilapalli
>Assignee: Sunil G
> Attachments: IntraQueue Preemption-Impact Analysis.pdf,
> TestNoIntraQueuePreemptionIfBelowUserLimitAndDifferentPrioritiesWithExtraUsers.txt,
> YARN-2113.0001.patch, YARN-2113.0002.patch, YARN-2113.0003.patch,
> YARN-2113.0004.patch, YARN-2113.0005.patch, YARN-2113.0006.patch,
> YARN-2113.0007.patch, YARN-2113.0008.patch, YARN-2113.0009.patch,
> YARN-2113.0010.patch, YARN-2113.0011.patch, YARN-2113.0012.patch,
> YARN-2113.0013.patch, YARN-2113.0014.patch, YARN-2113.0015.patch,
> YARN-2113.0016.patch, YARN-2113.0017.patch,
> YARN-2113.apply.onto.0012.ericp.patch, YARN-2113 Intra-QueuePreemption
> Behavior.pdf, YARN-2113.v0.patch
>
>
> Preemption today only works across queues and moves around resources across
> queues per demand and usage. We should also have user-level preemption within
> a queue, to balance capacity across users in a predictable manner.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6323) Rolling upgrade/config change is broken on timeline v2.
[ https://issues.apache.org/jira/browse/YARN-6323?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16011262#comment-16011262 ] Haibo Chen commented on YARN-6323: -- Thanks for pointing YARN-6555! I forgot that. A few comments: bq. /** default version for any flow. */ The semantic for the default flow version will change in YARN-6555. Let's update it in YARN-6555 Per comment of ApplicationImpl.flowContext, the flow context for an application is set only if the timeline service v.2 is enabled. With your change, even when ATSv2 is disabled, a default flow context is set. How about we return null from recoverFlowContextFromStateStore if ATSv2 is disabled? > Rolling upgrade/config change is broken on timeline v2. > > > Key: YARN-6323 > URL: https://issues.apache.org/jira/browse/YARN-6323 > Project: Hadoop YARN > Issue Type: Sub-task > Components: timelineserver >Reporter: Li Lu >Assignee: Vrushali C > Labels: yarn-5355-merge-blocker > Attachments: YARN-6323.001.patch > > > Found this issue when deploying on real clusters. If there are apps running > when we enable timeline v2 (with work preserving restart enabled), node > managers will fail to start due to missing app context data. We should > probably assign some default names to these "left over" apps. I believe it's > suboptimal to let users clean up the whole cluster before enabling timeline > v2. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6409) RM does not blacklist node for AM launch failures
[
https://issues.apache.org/jira/browse/YARN-6409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16011261#comment-16011261
]
Robert Kanter commented on YARN-6409:
-
The overall approach seems fine to me, but the {{TestRMAppAttemptTransitions}}
failure looks related and doesn't fail without the patch. Can you take a look
at it? Looks like you need to check {{instanceof}} before casting. Also, on
the depth of 3 thing, is there a reason the {{SocketTimeoutException}} won't be
at the same depth each time?
> RM does not blacklist node for AM launch failures
> -
>
> Key: YARN-6409
> URL: https://issues.apache.org/jira/browse/YARN-6409
> Project: Hadoop YARN
> Issue Type: Bug
> Components: resourcemanager
>Affects Versions: 3.0.0-alpha2
>Reporter: Haibo Chen
>Assignee: Haibo Chen
> Attachments: YARN-6409.00.patch, YARN-6409.01.patch,
> YARN-6409.02.patch
>
>
> Currently, node blacklisting upon AM failures only handles failures that
> happen after AM container is launched (see
> RMAppAttemptImpl.shouldCountTowardsNodeBlacklisting()). However, AM launch
> can also fail if the NM, where the AM container is allocated, goes
> unresponsive. Because it is not handled, scheduler may continue to allocate
> AM containers on that same NM for the following app attempts.
> {code}
> Application application_1478721503753_0870 failed 2 times due to Error
> launching appattempt_1478721503753_0870_02. Got exception:
> java.io.IOException: Failed on local exception: java.io.IOException:
> java.net.SocketTimeoutException: 6 millis timeout while waiting for
> channel to be ready for read. ch : java.nio.channels.SocketChannel[connected
> local=/17.111.179.113:46702 remote=*.me.com/17.111.178.125:8041]; Host
> Details : local host is: "*.me.com/17.111.179.113"; destination host is:
> "*.me.com":8041;
> at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:772)
> at org.apache.hadoop.ipc.Client.call(Client.java:1475)
> at org.apache.hadoop.ipc.Client.call(Client.java:1408)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:230)
>
> at com.sun.proxy.$Proxy86.startContainers(Unknown Source)
> at
> org.apache.hadoop.yarn.api.impl.pb.client.ContainerManagementProtocolPBClientImpl.startContainers(ContainerManagementProtocolPBClientImpl.java:96)
>
> at sun.reflect.GeneratedMethodAccessor155.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>
> at java.lang.reflect.Method.invoke(Method.java:497)
> at
> org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:256)
>
> at
> org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:104)
>
> at com.sun.proxy.$Proxy87.startContainers(Unknown Source)
> at
> org.apache.hadoop.yarn.server.resourcemanager.amlauncher.AMLauncher.launch(AMLauncher.java:120)
>
> at
> org.apache.hadoop.yarn.server.resourcemanager.amlauncher.AMLauncher.run(AMLauncher.java:256)
>
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.io.IOException: java.net.SocketTimeoutException: 6 millis
> timeout while waiting for channel to be ready for read. ch :
> java.nio.channels.SocketChannel[connected local=/17.111.179.113:46702
> remote=*.me.com/17.111.178.125:8041]
> at org.apache.hadoop.ipc.Client$Connection$1.run(Client.java:687)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1693)
>
> at
> org.apache.hadoop.ipc.Client$Connection.handleSaslConnectionFailure(Client.java:650)
>
> at org.apache.hadoop.ipc.Client$Connection.setupIOstreams(Client.java:738)
> at org.apache.hadoop.ipc.Client$Connection.access$2900(Client.java:375)
> at org.apache.hadoop.ipc.Client.getConnection(Client.java:1524)
> at org.apache.hadoop.ipc.Client.call(Client.java:1447)
> ... 15 more
> Caused by: java.net.SocketTimeoutException: 6 millis timeout while
> waiting for channel to be ready for read. ch :
> java.nio.channels.SocketChannel[connected local=/17.111.179.113:46702
> remote=*.me.com/17.111.178.125:8041]
> at
> org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:164)
> at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161)
> at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131)
> at java.io.FilterInputStream.read(FilterInputStream.java:133)
> at
[jira] [Commented] (YARN-6603) NPE in RMAppsBlock
[
https://issues.apache.org/jira/browse/YARN-6603?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16011258#comment-16011258
]
Jason Lowe commented on YARN-6603:
--
Same type of line in question as YARN-4763:
{code}
RMAppAttempt appAttempt =
rm.getRMContext().getRMApps().get(appAttemptId.getApplicationId())
.getAppAttempts().get(appAttemptId);
{code}
I believe the issue can occur if an app gets removed form the RMContext after
the app reports are gathered but before the loop ends up trying to lookup the
app. In that case
{{rm.getRMContext().getRMApps().get(appAttemptId.getApplicationId())}} is going
to return null and a subsequent call to getAppAttempts() is going to trigger
the NPE.
> NPE in RMAppsBlock
> --
>
> Key: YARN-6603
> URL: https://issues.apache.org/jira/browse/YARN-6603
> Project: Hadoop YARN
> Issue Type: Bug
> Components: resourcemanager
>Affects Versions: 2.8.0
>Reporter: Jason Lowe
>
> We are seeing an intermittent NPE when the RM is trying to render the
> /cluster URI.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6603) NPE in RMAppsBlock
[
https://issues.apache.org/jira/browse/YARN-6603?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16011255#comment-16011255
]
Jason Lowe commented on YARN-6603:
--
{noformat}
ERROR webapp.Dispatcher: error handling URI: /cluster
java.lang.NullPointerException
at
org.apache.hadoop.yarn.server.resourcemanager.webapp.RMAppsBlock.renderData(RMAppsBlock.java:101)
at
org.apache.hadoop.yarn.server.webapp.AppsBlock.render(AppsBlock.java:140)
at
org.apache.hadoop.yarn.webapp.view.HtmlBlock.render(HtmlBlock.java:69)
at
org.apache.hadoop.yarn.webapp.view.HtmlBlock.renderPartial(HtmlBlock.java:79)
at org.apache.hadoop.yarn.webapp.View.render(View.java:235)
at
org.apache.hadoop.yarn.webapp.view.HtmlBlock$Block.subView(HtmlBlock.java:43)
at org.apache.hadoop.yarn.webapp.hamlet.Hamlet._(Hamlet.java:30354)
at
org.apache.hadoop.yarn.server.resourcemanager.webapp.AppsBlockWithMetrics.render(AppsBlockWithMetrics.java:30)
at
org.apache.hadoop.yarn.webapp.view.HtmlBlock.render(HtmlBlock.java:69)
at
org.apache.hadoop.yarn.webapp.view.HtmlBlock.renderPartial(HtmlBlock.java:79)
at org.apache.hadoop.yarn.webapp.View.render(View.java:235)
at
org.apache.hadoop.yarn.webapp.view.HtmlPage$Page.subView(HtmlPage.java:49)
at
org.apache.hadoop.yarn.webapp.hamlet.HamletImpl$EImp._v(HamletImpl.java:117)
at org.apache.hadoop.yarn.webapp.hamlet.Hamlet$TD._(Hamlet.java:848)
at
org.apache.hadoop.yarn.webapp.view.TwoColumnLayout.render(TwoColumnLayout.java:71)
at org.apache.hadoop.yarn.webapp.view.HtmlPage.render(HtmlPage.java:82)
at org.apache.hadoop.yarn.webapp.Dispatcher.render(Dispatcher.java:206)
at org.apache.hadoop.yarn.webapp.Dispatcher.service(Dispatcher.java:165)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
at
com.google.inject.servlet.ServletDefinition.doService(ServletDefinition.java:263)
at
com.google.inject.servlet.ServletDefinition.service(ServletDefinition.java:178)
at
com.google.inject.servlet.ManagedServletPipeline.service(ManagedServletPipeline.java:91)
at
com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:62)
at
com.sun.jersey.spi.container.servlet.ServletContainer.doFilter(ServletContainer.java:900)
at
com.sun.jersey.spi.container.servlet.ServletContainer.doFilter(ServletContainer.java:834)
at
org.apache.hadoop.yarn.server.resourcemanager.webapp.RMWebAppFilter.doFilter(RMWebAppFilter.java:178)
at
com.sun.jersey.spi.container.servlet.ServletContainer.doFilter(ServletContainer.java:795)
at
com.google.inject.servlet.FilterDefinition.doFilter(FilterDefinition.java:163)
at
com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:58)
at
com.google.inject.servlet.ManagedFilterPipeline.dispatch(ManagedFilterPipeline.java:118)
at com.google.inject.servlet.GuiceFilter.doFilter(GuiceFilter.java:113)
at
org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
at yjava.servlet.filter.BouncerFilter.doFilter(BouncerFilter.java:217)
at
com.yahoo.hadoop.HadoopBouncerFilter.doFilter(HadoopBouncerFilter.java:108)
at
org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
at
org.apache.hadoop.security.http.CrossOriginFilter.doFilter(CrossOriginFilter.java:95)
at
org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
at org.mortbay.servlet.UserAgentFilter.doFilter(UserAgentFilter.java:78)
at com.yahoo.hadoop.GzipFilter.doFilter(GzipFilter.java:197)
at
org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
at
org.apache.hadoop.http.HttpServer2$QuotingInputFilter.doFilter(HttpServer2.java:1350)
at
org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
at org.apache.hadoop.http.NoCacheFilter.doFilter(NoCacheFilter.java:45)
at
org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
at org.apache.hadoop.http.NoCacheFilter.doFilter(NoCacheFilter.java:45)
at
org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
at
org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:399)
at
org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
at
org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
at
org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766)
at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:450)
at
[jira] [Created] (YARN-6603) NPE in RMAppsBlock
Jason Lowe created YARN-6603: Summary: NPE in RMAppsBlock Key: YARN-6603 URL: https://issues.apache.org/jira/browse/YARN-6603 Project: Hadoop YARN Issue Type: Bug Components: resourcemanager Affects Versions: 2.8.0 Reporter: Jason Lowe We are seeing an intermittent NPE when the RM is trying to render the /cluster URI. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-6160) Create an agent-less docker-less provider in the native services framework
[ https://issues.apache.org/jira/browse/YARN-6160?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Billie Rinaldi updated YARN-6160: - Attachment: YARN-6160-yarn-native-services.002.patch Here is an updated patch that also introduces a simple default provider in addition to the new tarball provider. I am planning a follow-on patch that will change ServiceApiUtil to use the new client provider validation methods and to handle the APPLICATION artifact type. I'll add some tests in the follow-on patch as well. I could do this all in the same patch, but thought it might be better to break it up for easier reviewing. > Create an agent-less docker-less provider in the native services framework > -- > > Key: YARN-6160 > URL: https://issues.apache.org/jira/browse/YARN-6160 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Billie Rinaldi >Assignee: Billie Rinaldi > Fix For: yarn-native-services > > Attachments: YARN-6160-yarn-native-services.001.patch, > YARN-6160-yarn-native-services.002.patch > > > The goal of the agent-less docker-less provider is to be able to use the YARN > native services framework when Docker is not installed or other methods of > app resource installation are preferable. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6323) Rolling upgrade/config change is broken on timeline v2.
[ https://issues.apache.org/jira/browse/YARN-6323?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16011130#comment-16011130 ] Vrushali C commented on YARN-6323: -- Thanks [~haibochen] for the review! Yes, you are right, we do want to store the flow context for the work preserving restart and hence YARN-6555. This jira addresses the case when there just isn't any previous flow context stored, which will happen the first time the cluster is upgraded to atsv2 (as the title suggests). I also added in some comments in the code to reflect this. > Rolling upgrade/config change is broken on timeline v2. > > > Key: YARN-6323 > URL: https://issues.apache.org/jira/browse/YARN-6323 > Project: Hadoop YARN > Issue Type: Sub-task > Components: timelineserver >Reporter: Li Lu >Assignee: Vrushali C > Labels: yarn-5355-merge-blocker > Attachments: YARN-6323.001.patch > > > Found this issue when deploying on real clusters. If there are apps running > when we enable timeline v2 (with work preserving restart enabled), node > managers will fail to start due to missing app context data. We should > probably assign some default names to these "left over" apps. I believe it's > suboptimal to let users clean up the whole cluster before enabling timeline > v2. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6602) Impersonation does not work if standby RM is contacted first
[
https://issues.apache.org/jira/browse/YARN-6602?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16011094#comment-16011094
]
Robert Kanter commented on YARN-6602:
-
>From what I can tell, the client doesn't get an RM token even in the working
>case. Somehow, {{UserGroupInformation.getCurrentUser()}} in the active RM
>(when processing the app submission) is different in each case:
{noformat:title=Working Case}
foo (auth:PROXY) via oozie/host@REALM (auth:KERBEROS)
{noformat}
{noformat:title=Non-Working Case}
oozie/host@REALM (auth:KERBEROS)
{noformat}
> Impersonation does not work if standby RM is contacted first
>
>
> Key: YARN-6602
> URL: https://issues.apache.org/jira/browse/YARN-6602
> Project: Hadoop YARN
> Issue Type: Bug
> Components: client
>Affects Versions: 3.0.0-alpha3
>Reporter: Robert Kanter
>Assignee: Robert Kanter
>Priority: Blocker
>
> When RM HA is enabled, impersonation does not work correctly if the Yarn
> Client connects to the standby RM first. When this happens, the
> impersonation is "lost" and the client does things on behalf of the
> impersonator user. We saw this with the OOZIE-1770 Oozie on Yarn feature.
> I need to investigate this some more, but it appears to be related to
> delegation tokens. When this issue occurs, the tokens have the owner as
> "oozie" instead of the actual user. On a hunch, we found a workaround that
> explicitly adding a correct RM HA delegation token fixes the problem:
> {code:java}
> org.apache.hadoop.yarn.api.records.Token token =
> yarnClient.getRMDelegationToken(ClientRMProxy.getRMDelegationTokenService(conf));
> org.apache.hadoop.security.token.Token token2 = new
> org.apache.hadoop.security.token.Token(token.getIdentifier().array(),
> token.getPassword().array(), new Text(token.getKind()), new
> Text(token.getService()));
> UserGroupInformation.getCurrentUser().addToken(token2);
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-6306) NMClient API change for container upgrade
[ https://issues.apache.org/jira/browse/YARN-6306?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun Suresh updated YARN-6306: -- Attachment: YARN-6306.004.patch Updating patch to format the Container diagnostic timestamp > NMClient API change for container upgrade > - > > Key: YARN-6306 > URL: https://issues.apache.org/jira/browse/YARN-6306 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Jian He >Assignee: Arun Suresh > Attachments: YARN-6306.001.patch, YARN-6306.002.patch, > YARN-6306.003.patch, YARN-6306.004.patch > > > This JIRA is track the addition of Upgrade API (Re-Initialize, Restart, > Rollback and Commit) to the NMClient and NMClientAsync -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6585) RM fails to start when upgrading from 2.7 to 2.8 for clusters with node labels.
[
https://issues.apache.org/jira/browse/YARN-6585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16011073#comment-16011073
]
Wangda Tan commented on YARN-6585:
--
Thanks [~eepayne]/[~nroberts]/[~sunilg] for reporting and investigating this
issue.
[~sunilg] I felt this fix is not correct, reversing fields is an incompatible
change.
In branch-2.7, we have string node labels in AddToClusterNodeLabelProto as 1st
field. In existing branch-2.8, we added a NodeLabelProto to 2nd field and
renamed 1st field to "deprecated-". So far this is compatible.
The problem is, existing implementation:
{code}
private void initLocalNodeLabels() {
AddToClusterNodeLabelsRequestProtoOrBuilder p = viaProto ? proto : builder;
List attributesProtoList = p.getNodeLabelsList();
this.updatedNodeLabels = new ArrayList();
for (NodeLabelProto r : attributesProtoList) {
this.updatedNodeLabels.add(convertFromProtoFormat(r));
}
}
{code}
Inside {{AddToClusterNodeLabelsRequestPBImpl}} doesn't read from deprecated
node label string field (1st). In FileSystemNodeLabelStore, YARN read from
serialized PB message and call {{new
AddToClusterNodeLabelsRequestPBImpl(AddToClusterNodeLabelsRequestProto
proto)}}. If it fails to read from 2nd field, it should try to read from the
1st one.
To make sure we have enough coverage, I suggest an unit test to read from
branch-2.7 stored node label file and make sure all fields can be read from
branch-2.8 and above.
Thoughts?
> RM fails to start when upgrading from 2.7 to 2.8 for clusters with node
> labels.
> ---
>
> Key: YARN-6585
> URL: https://issues.apache.org/jira/browse/YARN-6585
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Eric Payne
>Assignee: Sunil G
>Priority: Blocker
> Attachments: YARN-6585.0001.patch
>
>
> {noformat}
> Caused by: java.io.IOException: Not all labels being replaced contained by
> known label collections, please check, new labels=[abc]
> at
> org.apache.hadoop.yarn.nodelabels.CommonNodeLabelsManager.checkReplaceLabelsOnNode(CommonNodeLabelsManager.java:718)
> at
> org.apache.hadoop.yarn.nodelabels.CommonNodeLabelsManager.replaceLabelsOnNode(CommonNodeLabelsManager.java:737)
> at
> org.apache.hadoop.yarn.server.resourcemanager.nodelabels.RMNodeLabelsManager.replaceLabelsOnNode(RMNodeLabelsManager.java:189)
> at
> org.apache.hadoop.yarn.nodelabels.FileSystemNodeLabelsStore.loadFromMirror(FileSystemNodeLabelsStore.java:181)
> at
> org.apache.hadoop.yarn.nodelabels.FileSystemNodeLabelsStore.recover(FileSystemNodeLabelsStore.java:208)
> at
> org.apache.hadoop.yarn.nodelabels.CommonNodeLabelsManager.initNodeLabelStore(CommonNodeLabelsManager.java:251)
> at
> org.apache.hadoop.yarn.nodelabels.CommonNodeLabelsManager.serviceStart(CommonNodeLabelsManager.java:265)
> at
> org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
> ... 13 more
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5287) LinuxContainerExecutor fails to set proper permission
[ https://issues.apache.org/jira/browse/YARN-5287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16011057#comment-16011057 ] Jian He commented on YARN-5287: --- [~Naganarasimha], [~haibo.chen], saw a bunch of comments on YARN-5765. As the patch is reverted, Is there any plan to fix this original issue ? > LinuxContainerExecutor fails to set proper permission > - > > Key: YARN-5287 > URL: https://issues.apache.org/jira/browse/YARN-5287 > Project: Hadoop YARN > Issue Type: Bug > Components: nodemanager >Affects Versions: 2.7.2 >Reporter: Ying Zhang >Assignee: Naganarasimha G R >Priority: Minor > Fix For: 3.0.0-alpha1 > > Attachments: YARN-5287.003.patch, YARN-5287.004.patch, > YARN-5287.005.patch, YARN-5287.branch-2.001.patch, YARN-5287-tmp.patch > > Original Estimate: 48h > Remaining Estimate: 48h > > LinuxContainerExecutor fails to set the proper permissions on the local > directories(i.e., /hadoop/yarn/local/usercache/... by default) if the cluster > has been configured with a restrictive umask, e.g.: umask 077. Job failed due > to the following reason: > Path /hadoop/yarn/local/usercache/ambari-qa/appcache/application_ has > permission 700 but needs permission 750 -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-6602) Impersonation does not work if standby RM is contacted first
Robert Kanter created YARN-6602:
---
Summary: Impersonation does not work if standby RM is contacted
first
Key: YARN-6602
URL: https://issues.apache.org/jira/browse/YARN-6602
Project: Hadoop YARN
Issue Type: Bug
Components: client
Affects Versions: 3.0.0-alpha3
Reporter: Robert Kanter
Assignee: Robert Kanter
Priority: Blocker
When RM HA is enabled, impersonation does not work correctly if the Yarn Client
connects to the standby RM first. When this happens, the impersonation is
"lost" and the client does things on behalf of the impersonator user. We saw
this with the OOZIE-1770 Oozie on Yarn feature.
I need to investigate this some more, but it appears to be related to
delegation tokens. When this issue occurs, the tokens have the owner as
"oozie" instead of the actual user. On a hunch, we found a workaround that
explicitly adding a correct RM HA delegation token fixes the problem:
{code:java}
org.apache.hadoop.yarn.api.records.Token token =
yarnClient.getRMDelegationToken(ClientRMProxy.getRMDelegationTokenService(conf));
org.apache.hadoop.security.token.Token token2 = new
org.apache.hadoop.security.token.Token(token.getIdentifier().array(),
token.getPassword().array(), new Text(token.getKind()), new
Text(token.getService()));
UserGroupInformation.getCurrentUser().addToken(token2);
{code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6492) Generate queue metrics for each partition
[ https://issues.apache.org/jira/browse/YARN-6492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16010970#comment-16010970 ] Jonathan Hung commented on YARN-6492: - Hi [~Naganarasimha] are you still planning to work on this? If not I can work on it. > Generate queue metrics for each partition > - > > Key: YARN-6492 > URL: https://issues.apache.org/jira/browse/YARN-6492 > Project: Hadoop YARN > Issue Type: Improvement > Components: capacity scheduler >Reporter: Jonathan Hung >Assignee: Naganarasimha G R > > We are interested in having queue metrics for all partitions. Right now each > queue has one QueueMetrics object which captures metrics either in default > partition or across all partitions. (After YARN-6467 it will be in default > partition) > But having the partition metrics would be very useful. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6493) Print node partition in assignContainer logs
[ https://issues.apache.org/jira/browse/YARN-6493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16010965#comment-16010965 ] Jonathan Hung commented on YARN-6493: - Hi [~leftnoteasy], does the latest patch look OK to you? Thanks! > Print node partition in assignContainer logs > > > Key: YARN-6493 > URL: https://issues.apache.org/jira/browse/YARN-6493 > Project: Hadoop YARN > Issue Type: Improvement >Affects Versions: 2.8.0, 2.7.4, 2.6.6 >Reporter: Jonathan Hung >Assignee: Jonathan Hung > Attachments: YARN-6493.001.patch, YARN-6493.002.patch, > YARN-6493-branch-2.7.001.patch, YARN-6493-branch-2.8.001.patch, > YARN-6493-branch-2.8.002.patch > > > It would be useful to have the node's partition when logging a container > allocation, for tracking purposes. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6323) Rolling upgrade/config change is broken on timeline v2.
[ https://issues.apache.org/jira/browse/YARN-6323?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16010938#comment-16010938 ] Haibo Chen commented on YARN-6323: -- Thanks for the patch [~vrushalic]! I wonder if we should, to include as part of this jira, store the flow context in the NM state store for a given application. If NM has always been running with work preserving and ATSv2 on, because its flow context is not stored, any application upon recovery will get a default flow context rather than their original flow context. > Rolling upgrade/config change is broken on timeline v2. > > > Key: YARN-6323 > URL: https://issues.apache.org/jira/browse/YARN-6323 > Project: Hadoop YARN > Issue Type: Sub-task > Components: timelineserver >Reporter: Li Lu >Assignee: Vrushali C > Labels: yarn-5355-merge-blocker > Attachments: YARN-6323.001.patch > > > Found this issue when deploying on real clusters. If there are apps running > when we enable timeline v2 (with work preserving restart enabled), node > managers will fail to start due to missing app context data. We should > probably assign some default names to these "left over" apps. I believe it's > suboptimal to let users clean up the whole cluster before enabling timeline > v2. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6598) History server getApplicationReport NPE when fetching report for pre-2.8 job
[
https://issues.apache.org/jira/browse/YARN-6598?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16010934#comment-16010934
]
Hadoop QA commented on YARN-6598:
-
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 14m
6s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 9m
29s{color} | {color:green} branch-2.8 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
18s{color} | {color:green} branch-2.8 passed with JDK v1.8.0_131 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
21s{color} | {color:green} branch-2.8 passed with JDK v1.7.0_121 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
17s{color} | {color:green} branch-2.8 passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
27s{color} | {color:green} branch-2.8 passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
27s{color} | {color:green} branch-2.8 passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
43s{color} | {color:green} branch-2.8 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
14s{color} | {color:green} branch-2.8 passed with JDK v1.8.0_131 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
18s{color} | {color:green} branch-2.8 passed with JDK v1.7.0_121 {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
16s{color} | {color:green} the patch passed with JDK v1.8.0_131 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
16s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
18s{color} | {color:green} the patch passed with JDK v1.7.0_121 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
18s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 14s{color} | {color:orange}
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice:
The patch generated 1 new + 46 unchanged - 1 fixed = 47 total (was 47) {color}
|
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
23s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
14s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
49s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
11s{color} | {color:green} the patch passed with JDK v1.8.0_131 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
16s{color} | {color:green} the patch passed with JDK v1.7.0_121 {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 3m
11s{color} | {color:green} hadoop-yarn-server-applicationhistoryservice in the
patch passed with JDK v1.7.0_121. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
19s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 37m 59s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:5970e82 |
| JIRA Issue | YARN-6598 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12868101/YARN-6598-branch-2.8.001.patch
|
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux ffdbb32b73e3 3.13.0-107-generic #154-Ubuntu SMP Tue Dec 20
09:57:27 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | branch-2.8 / f4a |
[jira] [Commented] (YARN-5006) ResourceManager quit due to ApplicationStateData exceed the limit size of znode in zk
[
https://issues.apache.org/jira/browse/YARN-5006?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16010933#comment-16010933
]
Hadoop QA commented on YARN-5006:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
21s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
11s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 15m
23s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 11m
24s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
55s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
16s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
43s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
16s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
56s{color} | {color:green} trunk passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
9s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
1s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 9m
12s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 9m
12s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 56s{color} | {color:orange} hadoop-yarn-project/hadoop-yarn: The patch
generated 3 new + 400 unchanged - 2 fixed = 403 total (was 402) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
20s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
40s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 1 line(s) that end in whitespace. Use git
apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply
{color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
52s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
52s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 0m 35s{color}
| {color:red} hadoop-yarn-api in the patch failed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 39m 32s{color}
| {color:red} hadoop-yarn-server-resourcemanager in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
37s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 99m 39s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.yarn.conf.TestYarnConfigurationFields |
| | hadoop.yarn.server.resourcemanager.TestRMRestart |
| | hadoop.yarn.server.resourcemanager.security.TestDelegationTokenRenewer |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:14b5c93 |
| JIRA Issue | YARN-5006 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12868092/YARN-5006.002.patch |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux a44daa646f3d 3.13.0-107-generic #154-Ubuntu SMP Tue Dec 20
09:57:27 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / c48f297 |
| Default Java | 1.8.0_121 |
| findbugs | v3.1.0-RC1 |
| checkstyle |
https://builds.apache.org/job/PreCommit-YARN-Build/15927/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn.txt
|
| whitespace |
[jira] [Commented] (YARN-1593) support out-of-proc AuxiliaryServices
[ https://issues.apache.org/jira/browse/YARN-1593?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16010805#comment-16010805 ] Haibo Chen commented on YARN-1593: -- It seems system services are now addressed by YARN-6601 separately. Do you have plans in the short term to update the design doc and maybe work on system container [~vvasudev]? > support out-of-proc AuxiliaryServices > - > > Key: YARN-1593 > URL: https://issues.apache.org/jira/browse/YARN-1593 > Project: Hadoop YARN > Issue Type: Improvement > Components: nodemanager, rolling upgrade >Reporter: Ming Ma >Assignee: Varun Vasudev > Attachments: SystemContainersandSystemServices.pdf > > > AuxiliaryServices such as ShuffleHandler currently run in the same process as > NM. There are some benefits to host them in dedicated processes. > 1. NM rolling restart. If we want to upgrade YARN , NM restart will force the > ShuffleHandler restart. If ShuffleHandler runs as a separate process, > ShuffleHandler can continue to run during NM restart. NM can reconnect the > the running ShuffleHandler after restart. > 2. Resource management. It is possible another type of AuxiliaryServices will > be implemented. AuxiliaryServices are considered YARN application specific > and could consume lots of resources. Running AuxiliaryServices in separate > processes allow easier resource management. NM could potentially stop a > specific AuxiliaryServices process from running if it consumes resource way > above its allocation. > Here are some high level ideas: > 1. NM provides a hosting process for each AuxiliaryService. Existing > AuxiliaryService API doesn't change. > 2. The hosting process provides RPC server for AuxiliaryService proxy object > inside NM to connect to. > 3. When we rolling restart NM, the existing AuxiliaryService processes will > continue to run. NM could reconnect to the running AuxiliaryService processes > upon restart. > 4. Policy and resource management of AuxiliaryServices. So far we don't have > immediate need for this. AuxiliaryService could run inside a container and > its resource utilization could be taken into account by RM and RM could > consider a specific type of applications overutilize cluster resource. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-6598) History server getApplicationReport NPE when fetching report for pre-2.8 job
[ https://issues.apache.org/jira/browse/YARN-6598?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Lowe updated YARN-6598: - Attachment: YARN-6598-branch-2.8.001.patch Thanks, Jon! Here's an equivalent patch for branch-2.8. > History server getApplicationReport NPE when fetching report for pre-2.8 job > > > Key: YARN-6598 > URL: https://issues.apache.org/jira/browse/YARN-6598 > Project: Hadoop YARN > Issue Type: Bug > Components: timelineserver >Affects Versions: 2.8.0 >Reporter: Jason Lowe >Assignee: Jason Lowe >Priority: Blocker > Attachments: YARN-6598.001.patch, YARN-6598-branch-2.8.001.patch > > > ApplicationHistoryManagerOnTimelineStore#convertToApplicationReport can NPE > for a job that was run prior to the cluster upgrading to 2.8. It blindly > assumes preemption metrics are present when CPU metrics are present, and when > they are not it triggers the NPE. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6601) Allow service to be started as System Services during serviceapi start up
[ https://issues.apache.org/jira/browse/YARN-6601?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16010783#comment-16010783 ] Haibo Chen commented on YARN-6601: -- Thanks. +1 on addressing system container and system services separately. > Allow service to be started as System Services during serviceapi start up > - > > Key: YARN-6601 > URL: https://issues.apache.org/jira/browse/YARN-6601 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Rohith Sharma K S > Attachments: SystemServices.pdf > > > This is extended from YARN-1593 focusing only on system services. This > particular JIRA focusing on starting the system services during > native-service-api start up. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6601) Allow service to be started as System Services during serviceapi start up
[ https://issues.apache.org/jira/browse/YARN-6601?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16010764#comment-16010764 ] Rohith Sharma K S commented on YARN-6601: - Yep, this is as part of YARN-1593 mainly intended for system service. The concept of system container vs system services is bit different. May be this JIRA could be a subtask of YARN-1593 too. But I created under YARN-5079 since code commit will be under native service branch. > Allow service to be started as System Services during serviceapi start up > - > > Key: YARN-6601 > URL: https://issues.apache.org/jira/browse/YARN-6601 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Rohith Sharma K S > Attachments: SystemServices.pdf > > > This is extended from YARN-1593 focusing only on system services. This > particular JIRA focusing on starting the system services during > native-service-api start up. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6598) History server getApplicationReport NPE when fetching report for pre-2.8 job
[ https://issues.apache.org/jira/browse/YARN-6598?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16010742#comment-16010742 ] Hudson commented on YARN-6598: -- SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #11734 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/11734/]) YARN-6598. History server getApplicationReport NPE when fetching report (jeagles: rev c48f2976a3de60b95c4a5ada4f0131c4cdde177a) * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/TestApplicationHistoryManagerOnTimelineStore.java * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/ApplicationHistoryManagerOnTimelineStore.java > History server getApplicationReport NPE when fetching report for pre-2.8 job > > > Key: YARN-6598 > URL: https://issues.apache.org/jira/browse/YARN-6598 > Project: Hadoop YARN > Issue Type: Bug > Components: timelineserver >Affects Versions: 2.8.0 >Reporter: Jason Lowe >Assignee: Jason Lowe >Priority: Blocker > Attachments: YARN-6598.001.patch > > > ApplicationHistoryManagerOnTimelineStore#convertToApplicationReport can NPE > for a job that was run prior to the cluster upgrading to 2.8. It blindly > assumes preemption metrics are present when CPU metrics are present, and when > they are not it triggers the NPE. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6598) History server getApplicationReport NPE when fetching report for pre-2.8 job
[ https://issues.apache.org/jira/browse/YARN-6598?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16010729#comment-16010729 ] Jonathan Eagles commented on YARN-6598: --- [~jlowe], can you provide a patch that will work with branch-2.8 and branch-2.8.1. There is a conflict that is more than trivial. > History server getApplicationReport NPE when fetching report for pre-2.8 job > > > Key: YARN-6598 > URL: https://issues.apache.org/jira/browse/YARN-6598 > Project: Hadoop YARN > Issue Type: Bug > Components: timelineserver >Affects Versions: 2.8.0 >Reporter: Jason Lowe >Assignee: Jason Lowe >Priority: Blocker > Attachments: YARN-6598.001.patch > > > ApplicationHistoryManagerOnTimelineStore#convertToApplicationReport can NPE > for a job that was run prior to the cluster upgrading to 2.8. It blindly > assumes preemption metrics are present when CPU metrics are present, and when > they are not it triggers the NPE. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-5006) ResourceManager quit due to ApplicationStateData exceed the limit size of znode in zk
[
https://issues.apache.org/jira/browse/YARN-5006?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bibin A Chundatt updated YARN-5006:
---
Attachment: YARN-5006.002.patch
[~templedf]
Thank you for review comments.
Attaching patch handling review comments
> ResourceManager quit due to ApplicationStateData exceed the limit size of
> znode in zk
> --
>
> Key: YARN-5006
> URL: https://issues.apache.org/jira/browse/YARN-5006
> Project: Hadoop YARN
> Issue Type: Bug
> Components: resourcemanager
>Affects Versions: 2.6.0, 2.7.2
>Reporter: dongtingting
>Assignee: Bibin A Chundatt
>Priority: Critical
> Attachments: YARN-5006.001.patch, YARN-5006.002.patch
>
>
> Client submit a job, this job add 1 file into DistributedCache. when the
> job is submitted, ResourceManager sotre ApplicationStateData into zk.
> ApplicationStateData is exceed the limit size of znode. RM exit 1.
> The related code in RMStateStore.java :
> {code}
> private static class StoreAppTransition
> implements SingleArcTransition {
> @Override
> public void transition(RMStateStore store, RMStateStoreEvent event) {
> if (!(event instanceof RMStateStoreAppEvent)) {
> // should never happen
> LOG.error("Illegal event type: " + event.getClass());
> return;
> }
> ApplicationState appState = ((RMStateStoreAppEvent)
> event).getAppState();
> ApplicationId appId = appState.getAppId();
> ApplicationStateData appStateData = ApplicationStateData
> .newInstance(appState);
> LOG.info("Storing info for app: " + appId);
> try {
> store.storeApplicationStateInternal(appId, appStateData); //store
> the appStateData
> store.notifyApplication(new RMAppEvent(appId,
>RMAppEventType.APP_NEW_SAVED));
> } catch (Exception e) {
> LOG.error("Error storing app: " + appId, e);
> store.notifyStoreOperationFailed(e); //handle fail event, system
> exit
> }
> };
> }
> {code}
> The Exception log:
> {code}
> ...
> 2016-04-20 11:26:35,732 INFO
> org.apache.hadoop.yarn.server.resourcemanager.recovery.ZKRMStateStore
> AsyncDispatcher event handler: Maxed out ZK retries. Giving up!
> 2016-04-20 11:26:35,732 ERROR
> org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStore
> AsyncDispatcher event handler: Error storing app:
> application_1461061795989_17671
> org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode
> = ConnectionLoss
> at
> org.apache.zookeeper.KeeperException.create(KeeperException.java:99)
> at org.apache.zookeeper.ZooKeeper.multiInternal(ZooKeeper.java:931)
> at org.apache.zookeeper.ZooKeeper.multi(ZooKeeper.java:911)
> at
> org.apache.hadoop.yarn.server.resourcemanager.recovery.ZKRMStateStore$4.run(ZKRMStateStore.java:936)
> at
> org.apache.hadoop.yarn.server.resourcemanager.recovery.ZKRMStateStore$4.run(ZKRMStateStore.java:933)
> at
> org.apache.hadoop.yarn.server.resourcemanager.recovery.ZKRMStateStore$ZKAction.runWithCheck(ZKRMStateStore.java:1075)
> at
> org.apache.hadoop.yarn.server.resourcemanager.recovery.ZKRMStateStore$ZKAction.runWithRetries(ZKRMStateStore.java:1096)
> at
> org.apache.hadoop.yarn.server.resourcemanager.recovery.ZKRMStateStore.doMultiWithRetries(ZKRMStateStore.java:933)
> at
> org.apache.hadoop.yarn.server.resourcemanager.recovery.ZKRMStateStore.doMultiWithRetries(ZKRMStateStore.java:947)
> at
> org.apache.hadoop.yarn.server.resourcemanager.recovery.ZKRMStateStore.createWithRetries(ZKRMStateStore.java:956)
> at
> org.apache.hadoop.yarn.server.resourcemanager.recovery.ZKRMStateStore.storeApplicationStateInternal(ZKRMStateStore.java:626)
> at
> org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStore$StoreAppTransition.transition(RMStateStore.java:138)
> at
> org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStore$StoreAppTransition.transition(RMStateStore.java:123)
> at
> org.apache.hadoop.yarn.state.StateMachineFactory$SingleInternalArc.doTransition(StateMachineFactory.java:362)
> at
> org.apache.hadoop.yarn.state.StateMachineFactory.doTransition(StateMachineFactory.java:302)
> at
> org.apache.hadoop.yarn.state.StateMachineFactory.access$300(StateMachineFactory.java:46)
> at
> org.apache.hadoop.yarn.state.StateMachineFactory$InternalStateMachine.doTransition(StateMachineFactory.java:448)
> at
> org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStore.handleStoreEvent(RMStateStore.java:806)
> at
>
[jira] [Commented] (YARN-6598) History server getApplicationReport NPE when fetching report for pre-2.8 job
[ https://issues.apache.org/jira/browse/YARN-6598?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16010699#comment-16010699 ] Jonathan Eagles commented on YARN-6598: --- +1. Thanks for verifying this extra compatibility, [~jlowe]. > History server getApplicationReport NPE when fetching report for pre-2.8 job > > > Key: YARN-6598 > URL: https://issues.apache.org/jira/browse/YARN-6598 > Project: Hadoop YARN > Issue Type: Bug > Components: timelineserver >Affects Versions: 2.8.0 >Reporter: Jason Lowe >Assignee: Jason Lowe >Priority: Blocker > Attachments: YARN-6598.001.patch > > > ApplicationHistoryManagerOnTimelineStore#convertToApplicationReport can NPE > for a job that was run prior to the cluster upgrading to 2.8. It blindly > assumes preemption metrics are present when CPU metrics are present, and when > they are not it triggers the NPE. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6601) Allow service to be started as System Services during serviceapi start up
[ https://issues.apache.org/jira/browse/YARN-6601?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16010695#comment-16010695 ] Haibo Chen commented on YARN-6601: -- [~rohithsharma], the doc seems to cover part of YARN-1593. Is the intent to split system services out and leave YARN-1593 to only do system containers? > Allow service to be started as System Services during serviceapi start up > - > > Key: YARN-6601 > URL: https://issues.apache.org/jira/browse/YARN-6601 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Rohith Sharma K S > Attachments: SystemServices.pdf > > > This is extended from YARN-1593 focusing only on system services. This > particular JIRA focusing on starting the system services during > native-service-api start up. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6598) History server getApplicationReport NPE when fetching report for pre-2.8 job
[ https://issues.apache.org/jira/browse/YARN-6598?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16010673#comment-16010673 ] Jason Lowe commented on YARN-6598: -- bq. We should exhaustively check the Enum.valueOf to ensure backwards compatibility in those cases as well. I checked all the enums referenced for valueOf (i.e.: YarnApplicationState, FinalApplicationStatus, YarnApplicationAttemptState, ). YarnApplicationAttemptState had the FINAL_SAVING enum removed in YARN-1345, but that change was committed way back in Hadoop 2.4.0 which did not support rolling upgrades. So we should be good there. ContainerState had a QUEUED state that was changed to the SCHEDULED state in YARN-4597. However this change, and YARN-2883 that added the QUEUED state originally, only went into Hadoop 2.9.0 and later, so the removed item was never in an official release. So we should be fine there as well. > History server getApplicationReport NPE when fetching report for pre-2.8 job > > > Key: YARN-6598 > URL: https://issues.apache.org/jira/browse/YARN-6598 > Project: Hadoop YARN > Issue Type: Bug > Components: timelineserver >Affects Versions: 2.8.0 >Reporter: Jason Lowe >Assignee: Jason Lowe >Priority: Blocker > Attachments: YARN-6598.001.patch > > > ApplicationHistoryManagerOnTimelineStore#convertToApplicationReport can NPE > for a job that was run prior to the cluster upgrading to 2.8. It blindly > assumes preemption metrics are present when CPU metrics are present, and when > they are not it triggers the NPE. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Assigned] (YARN-6585) RM fails to start when upgrading from 2.7 to 2.8 for clusters with node labels.
[
https://issues.apache.org/jira/browse/YARN-6585?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sunil G reassigned YARN-6585:
-
Assignee: Sunil G
> RM fails to start when upgrading from 2.7 to 2.8 for clusters with node
> labels.
> ---
>
> Key: YARN-6585
> URL: https://issues.apache.org/jira/browse/YARN-6585
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Eric Payne
>Assignee: Sunil G
>Priority: Blocker
> Attachments: YARN-6585.0001.patch
>
>
> {noformat}
> Caused by: java.io.IOException: Not all labels being replaced contained by
> known label collections, please check, new labels=[abc]
> at
> org.apache.hadoop.yarn.nodelabels.CommonNodeLabelsManager.checkReplaceLabelsOnNode(CommonNodeLabelsManager.java:718)
> at
> org.apache.hadoop.yarn.nodelabels.CommonNodeLabelsManager.replaceLabelsOnNode(CommonNodeLabelsManager.java:737)
> at
> org.apache.hadoop.yarn.server.resourcemanager.nodelabels.RMNodeLabelsManager.replaceLabelsOnNode(RMNodeLabelsManager.java:189)
> at
> org.apache.hadoop.yarn.nodelabels.FileSystemNodeLabelsStore.loadFromMirror(FileSystemNodeLabelsStore.java:181)
> at
> org.apache.hadoop.yarn.nodelabels.FileSystemNodeLabelsStore.recover(FileSystemNodeLabelsStore.java:208)
> at
> org.apache.hadoop.yarn.nodelabels.CommonNodeLabelsManager.initNodeLabelStore(CommonNodeLabelsManager.java:251)
> at
> org.apache.hadoop.yarn.nodelabels.CommonNodeLabelsManager.serviceStart(CommonNodeLabelsManager.java:265)
> at
> org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
> ... 13 more
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-6585) RM fails to start when upgrading from 2.7 to 2.8 for clusters with node labels.
[
https://issues.apache.org/jira/browse/YARN-6585?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sunil G updated YARN-6585:
--
Attachment: YARN-6585.0001.patch
updating basic patch for review.
cc/[~leftnoteasy]
> RM fails to start when upgrading from 2.7 to 2.8 for clusters with node
> labels.
> ---
>
> Key: YARN-6585
> URL: https://issues.apache.org/jira/browse/YARN-6585
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Eric Payne
>Priority: Blocker
> Attachments: YARN-6585.0001.patch
>
>
> {noformat}
> Caused by: java.io.IOException: Not all labels being replaced contained by
> known label collections, please check, new labels=[abc]
> at
> org.apache.hadoop.yarn.nodelabels.CommonNodeLabelsManager.checkReplaceLabelsOnNode(CommonNodeLabelsManager.java:718)
> at
> org.apache.hadoop.yarn.nodelabels.CommonNodeLabelsManager.replaceLabelsOnNode(CommonNodeLabelsManager.java:737)
> at
> org.apache.hadoop.yarn.server.resourcemanager.nodelabels.RMNodeLabelsManager.replaceLabelsOnNode(RMNodeLabelsManager.java:189)
> at
> org.apache.hadoop.yarn.nodelabels.FileSystemNodeLabelsStore.loadFromMirror(FileSystemNodeLabelsStore.java:181)
> at
> org.apache.hadoop.yarn.nodelabels.FileSystemNodeLabelsStore.recover(FileSystemNodeLabelsStore.java:208)
> at
> org.apache.hadoop.yarn.nodelabels.CommonNodeLabelsManager.initNodeLabelStore(CommonNodeLabelsManager.java:251)
> at
> org.apache.hadoop.yarn.nodelabels.CommonNodeLabelsManager.serviceStart(CommonNodeLabelsManager.java:265)
> at
> org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
> ... 13 more
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6585) RM fails to start when upgrading from 2.7 to 2.8 for clusters with node labels.
[
https://issues.apache.org/jira/browse/YARN-6585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16010597#comment-16010597
]
Sunil G commented on YARN-6585:
---
Yes. Order is changed somehow.
{code}
message AddToClusterNodeLabelsRequestProto {
repeated string deprecatedNodeLabels = 1;
repeated NodeLabelProto nodeLabels = 2;
}
{code}
This should be changed like
{code}
message AddToClusterNodeLabelsRequestProto {
repeated NodeLabelProto nodeLabels = 1;
repeated string deprecatedNodeLabels = 2;
}
{code}
I ll update a patch now.
> RM fails to start when upgrading from 2.7 to 2.8 for clusters with node
> labels.
> ---
>
> Key: YARN-6585
> URL: https://issues.apache.org/jira/browse/YARN-6585
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Eric Payne
>Priority: Blocker
>
> {noformat}
> Caused by: java.io.IOException: Not all labels being replaced contained by
> known label collections, please check, new labels=[abc]
> at
> org.apache.hadoop.yarn.nodelabels.CommonNodeLabelsManager.checkReplaceLabelsOnNode(CommonNodeLabelsManager.java:718)
> at
> org.apache.hadoop.yarn.nodelabels.CommonNodeLabelsManager.replaceLabelsOnNode(CommonNodeLabelsManager.java:737)
> at
> org.apache.hadoop.yarn.server.resourcemanager.nodelabels.RMNodeLabelsManager.replaceLabelsOnNode(RMNodeLabelsManager.java:189)
> at
> org.apache.hadoop.yarn.nodelabels.FileSystemNodeLabelsStore.loadFromMirror(FileSystemNodeLabelsStore.java:181)
> at
> org.apache.hadoop.yarn.nodelabels.FileSystemNodeLabelsStore.recover(FileSystemNodeLabelsStore.java:208)
> at
> org.apache.hadoop.yarn.nodelabels.CommonNodeLabelsManager.initNodeLabelStore(CommonNodeLabelsManager.java:251)
> at
> org.apache.hadoop.yarn.nodelabels.CommonNodeLabelsManager.serviceStart(CommonNodeLabelsManager.java:265)
> at
> org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
> ... 13 more
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5924) Resource Manager fails to load state with InvalidProtocolBufferException
[
https://issues.apache.org/jira/browse/YARN-5924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16010362#comment-16010362
]
ASF GitHub Bot commented on YARN-5924:
--
Github user ameks94 commented on the issue:
https://github.com/apache/hadoop/pull/164
I realized that current solution is not good (to allow RM's launch even
with broken app's data).
It's better to crash RM in case application's file with app's state is
broken. This case we can specify more detailed information about which file is
broken (Maybe to give the recommendation to remove application's folder with
broken data to allow RM to be launched successfully)
Second, the most important part of the fix should be to find the reason of
file's crashing and to find the way to prevent file's crash.
> Resource Manager fails to load state with InvalidProtocolBufferException
>
>
> Key: YARN-5924
> URL: https://issues.apache.org/jira/browse/YARN-5924
> Project: Hadoop YARN
> Issue Type: Bug
> Components: resourcemanager
>Affects Versions: 3.0.0-alpha1
>Reporter: Oleksii Dymytrov
>Assignee: Oleksii Dymytrov
> Attachments: YARN-5924.002.patch
>
>
> InvalidProtocolBufferException is thrown during recovering of the
> application's state if application's data has invalid format (or is broken)
> under FSRMStateRoot/RMAppRoot/application_1477986176766_0134/ directory in
> HDFS:
> {noformat}
> com.google.protobuf.InvalidProtocolBufferException: Protocol message
> end-group tag did not match expected tag.
> at
> com.google.protobuf.InvalidProtocolBufferException.invalidEndTag(InvalidProtocolBufferException.java:94)
> at
> com.google.protobuf.CodedInputStream.checkLastTagWas(CodedInputStream.java:124)
> at
> com.google.protobuf.AbstractParser.parsePartialFrom(AbstractParser.java:143)
> at com.google.protobuf.AbstractParser.parseFrom(AbstractParser.java:176)
> at com.google.protobuf.AbstractParser.parseFrom(AbstractParser.java:188)
> at com.google.protobuf.AbstractParser.parseFrom(AbstractParser.java:193)
> at com.google.protobuf.AbstractParser.parseFrom(AbstractParser.java:49)
> at
> org.apache.hadoop.yarn.proto.YarnServerResourceManagerRecoveryProtos$ApplicationStateDataProto.parseFrom(YarnServerResourceManagerRecoveryProtos.java:1028)
> at
> org.apache.hadoop.yarn.server.resourcemanager.recovery.FileSystemRMStateStore$RMAppStateFileProcessor.processChildNode(FileSystemRMStateStore.java:966)
> at
> org.apache.hadoop.yarn.server.resourcemanager.recovery.FileSystemRMStateStore.processDirectoriesOfFiles(FileSystemRMStateStore.java:317)
> at
> org.apache.hadoop.yarn.server.resourcemanager.recovery.FileSystemRMStateStore.loadRMAppState(FileSystemRMStateStore.java:281)
> at
> org.apache.hadoop.yarn.server.resourcemanager.recovery.FileSystemRMStateStore.loadState(FileSystemRMStateStore.java:232)
> {noformat}
> The solution can be to catch "InvalidProtocolBufferException", show warning
> and remove application's folder that contains invalid data to prevent RM
> restart failure.
> Additionally, I've added catch for other exceptions that can appear during
> recovering of the specific application, to avoid RM failure even if the only
> one application's state can't be loaded.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5924) Resource Manager fails to load state with InvalidProtocolBufferException
[
https://issues.apache.org/jira/browse/YARN-5924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16010363#comment-16010363
]
ASF GitHub Bot commented on YARN-5924:
--
Github user ameks94 closed the pull request at:
https://github.com/apache/hadoop/pull/164
> Resource Manager fails to load state with InvalidProtocolBufferException
>
>
> Key: YARN-5924
> URL: https://issues.apache.org/jira/browse/YARN-5924
> Project: Hadoop YARN
> Issue Type: Bug
> Components: resourcemanager
>Affects Versions: 3.0.0-alpha1
>Reporter: Oleksii Dymytrov
>Assignee: Oleksii Dymytrov
> Attachments: YARN-5924.002.patch
>
>
> InvalidProtocolBufferException is thrown during recovering of the
> application's state if application's data has invalid format (or is broken)
> under FSRMStateRoot/RMAppRoot/application_1477986176766_0134/ directory in
> HDFS:
> {noformat}
> com.google.protobuf.InvalidProtocolBufferException: Protocol message
> end-group tag did not match expected tag.
> at
> com.google.protobuf.InvalidProtocolBufferException.invalidEndTag(InvalidProtocolBufferException.java:94)
> at
> com.google.protobuf.CodedInputStream.checkLastTagWas(CodedInputStream.java:124)
> at
> com.google.protobuf.AbstractParser.parsePartialFrom(AbstractParser.java:143)
> at com.google.protobuf.AbstractParser.parseFrom(AbstractParser.java:176)
> at com.google.protobuf.AbstractParser.parseFrom(AbstractParser.java:188)
> at com.google.protobuf.AbstractParser.parseFrom(AbstractParser.java:193)
> at com.google.protobuf.AbstractParser.parseFrom(AbstractParser.java:49)
> at
> org.apache.hadoop.yarn.proto.YarnServerResourceManagerRecoveryProtos$ApplicationStateDataProto.parseFrom(YarnServerResourceManagerRecoveryProtos.java:1028)
> at
> org.apache.hadoop.yarn.server.resourcemanager.recovery.FileSystemRMStateStore$RMAppStateFileProcessor.processChildNode(FileSystemRMStateStore.java:966)
> at
> org.apache.hadoop.yarn.server.resourcemanager.recovery.FileSystemRMStateStore.processDirectoriesOfFiles(FileSystemRMStateStore.java:317)
> at
> org.apache.hadoop.yarn.server.resourcemanager.recovery.FileSystemRMStateStore.loadRMAppState(FileSystemRMStateStore.java:281)
> at
> org.apache.hadoop.yarn.server.resourcemanager.recovery.FileSystemRMStateStore.loadState(FileSystemRMStateStore.java:232)
> {noformat}
> The solution can be to catch "InvalidProtocolBufferException", show warning
> and remove application's folder that contains invalid data to prevent RM
> restart failure.
> Additionally, I've added catch for other exceptions that can appear during
> recovering of the specific application, to avoid RM failure even if the only
> one application's state can't be loaded.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6601) Allow service to be started as System Services during serviceapi start up
[ https://issues.apache.org/jira/browse/YARN-6601?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16010346#comment-16010346 ] Rohith Sharma K S commented on YARN-6601: - Updated documentation that give brief about system services and design. > Allow service to be started as System Services during serviceapi start up > - > > Key: YARN-6601 > URL: https://issues.apache.org/jira/browse/YARN-6601 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Rohith Sharma K S > Attachments: SystemServices.pdf > > > This is extended from YARN-1593 focusing only on system services. This > particular JIRA focusing on starting the system services during > native-service-api start up. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-6601) Allow service to be started as System Services during serviceapi start up
[ https://issues.apache.org/jira/browse/YARN-6601?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rohith Sharma K S updated YARN-6601: Description: This is extended from YARN-1593 focusing only on system services. This particular JIRA focusing on starting the system services during native-service-api start up. > Allow service to be started as System Services during serviceapi start up > - > > Key: YARN-6601 > URL: https://issues.apache.org/jira/browse/YARN-6601 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Rohith Sharma K S > Attachments: SystemServices.pdf > > > This is extended from YARN-1593 focusing only on system services. This > particular JIRA focusing on starting the system services during > native-service-api start up. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-6601) Allow service to be started as System Services during serviceapi start up
[ https://issues.apache.org/jira/browse/YARN-6601?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rohith Sharma K S updated YARN-6601: Attachment: SystemServices.pdf > Allow service to be started as System Services during serviceapi start up > - > > Key: YARN-6601 > URL: https://issues.apache.org/jira/browse/YARN-6601 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Rohith Sharma K S > Attachments: SystemServices.pdf > > -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-6601) Allow service to be started as System Services during serviceapi start up
Rohith Sharma K S created YARN-6601: --- Summary: Allow service to be started as System Services during serviceapi start up Key: YARN-6601 URL: https://issues.apache.org/jira/browse/YARN-6601 Project: Hadoop YARN Issue Type: Sub-task Reporter: Rohith Sharma K S -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-6600) Enhance default lifetime of application at LeafQueue level.
Rohith Sharma K S created YARN-6600: --- Summary: Enhance default lifetime of application at LeafQueue level. Key: YARN-6600 URL: https://issues.apache.org/jira/browse/YARN-6600 Project: Hadoop YARN Issue Type: Sub-task Components: capacity scheduler Reporter: Rohith Sharma K S Assignee: Rohith Sharma K S Setting timeout at LeafQueue level allows admin to control from bad apps which uses most of the resources for all the time. Example : Any application submitted to particular queue i.e QUEUE-1 should not run more than N hours. Even user set lifetime as N+1 hour, the application will be killed after N hours. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-2113) Add cross-user preemption within CapacityScheduler's leaf-queue
[
https://issues.apache.org/jira/browse/YARN-2113?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16010253#comment-16010253
]
Hadoop QA commented on YARN-2113:
-
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
22s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 4 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 12m
18s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
29s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
28s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
32s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
17s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
53s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
22s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
31s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
29s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
29s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 25s{color} | {color:orange}
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager:
The patch generated 22 new + 180 unchanged - 3 fixed = 202 total (was 183)
{color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
32s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
15s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
18s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 39m
36s{color} | {color:green} hadoop-yarn-server-resourcemanager in the patch
passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 60m 31s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:14b5c93 |
| JIRA Issue | YARN-2113 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12868029/YARN-2113.0017.patch |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux f60b076d1bcd 4.4.0-43-generic #63-Ubuntu SMP Wed Oct 12
13:48:03 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / 6600abb |
| Default Java | 1.8.0_121 |
| findbugs | v3.1.0-RC1 |
| checkstyle |
https://builds.apache.org/job/PreCommit-YARN-Build/15926/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/15926/testReport/ |
| modules | C:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
U:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
|
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/15926/console |
| Powered by | Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> Add cross-user preemption within CapacityScheduler's leaf-queue
> ---
>
> Key:
[jira] [Resolved] (YARN-6459) Clean up MiniYARNCluster and MiniMRYarnCluster
[
https://issues.apache.org/jira/browse/YARN-6459?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andras Bokor resolved YARN-6459.
Resolution: Later
It was planned as a follow up aftter YARN-5007. YARN-5007 was reverted and
closed as "Later". Doing the same with this one.
> Clean up MiniYARNCluster and MiniMRYarnCluster
> --
>
> Key: YARN-6459
> URL: https://issues.apache.org/jira/browse/YARN-6459
> Project: Hadoop YARN
> Issue Type: Task
>Reporter: Andras Bokor
>Assignee: Andras Bokor
>Priority: Trivial
>
> Since I was working with {{MiniYARNCluster}} and {{MiniMRYarnCluster}} I
> would like to clean that two classes up as of "The Boy Scout Rule".
> They have either checkstyle or IDE warnings.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6141) ppc64le on Linux doesn't trigger __linux get_executable codepath
[ https://issues.apache.org/jira/browse/YARN-6141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16010181#comment-16010181 ] Sonia Garudi commented on YARN-6141: Any update on this ? > ppc64le on Linux doesn't trigger __linux get_executable codepath > > > Key: YARN-6141 > URL: https://issues.apache.org/jira/browse/YARN-6141 > Project: Hadoop YARN > Issue Type: Bug > Components: nodemanager >Affects Versions: 3.0.0-alpha3 > Environment: $ uname -a > Linux f8eef0f055cf 3.16.0-30-generic #40~14.04.1-Ubuntu SMP Thu Jan 15 > 17:42:36 UTC 2015 ppc64le ppc64le ppc64le GNU/Linux >Reporter: Sonia Garudi > Labels: ppc64le > Attachments: YARN-6141.patch > > > On ppc64le architecture, the build fails in the 'Hadoop YARN NodeManager' > project with the below error: > Cannot safely determine executable path with a relative HADOOP_CONF_DIR on > this operating system. > [WARNING] #error Cannot safely determine executable path with a relative > HADOOP_CONF_DIR on this operating system. > [WARNING] ^ > [WARNING] make[2]: *** > [CMakeFiles/container.dir/main/native/container-executor/impl/get_executable.c.o] > Error 1 > [WARNING] make[2]: *** Waiting for unfinished jobs > [WARNING] make[1]: *** [CMakeFiles/container.dir/all] Error 2 > [WARNING] make: *** [all] Error 2 > [INFO] > > [INFO] BUILD FAILURE > [INFO] > > Cmake version used : > $ /usr/bin/cmake --version > cmake version 2.8.12.2 -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-2113) Add cross-user preemption within CapacityScheduler's leaf-queue
[
https://issues.apache.org/jira/browse/YARN-2113?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sunil G updated YARN-2113:
--
Attachment: YARN-2113.0017.patch
[~eepayne]
Thank you very much.
Yes, i think i got the pblm. Many places we were using
{{Resources.subtractFrom}}. In few cases, its possible that vcores/memory could
become -ve due to this api. Hence {{Resources.subtractFromNonNegative}} should
be used in place of subtractFrom. A similar issue is handled here as well
YARN-6538
Attaching latest patch.
> Add cross-user preemption within CapacityScheduler's leaf-queue
> ---
>
> Key: YARN-2113
> URL: https://issues.apache.org/jira/browse/YARN-2113
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: scheduler
>Reporter: Vinod Kumar Vavilapalli
>Assignee: Sunil G
> Attachments: IntraQueue Preemption-Impact Analysis.pdf,
> TestNoIntraQueuePreemptionIfBelowUserLimitAndDifferentPrioritiesWithExtraUsers.txt,
> YARN-2113.0001.patch, YARN-2113.0002.patch, YARN-2113.0003.patch,
> YARN-2113.0004.patch, YARN-2113.0005.patch, YARN-2113.0006.patch,
> YARN-2113.0007.patch, YARN-2113.0008.patch, YARN-2113.0009.patch,
> YARN-2113.0010.patch, YARN-2113.0011.patch, YARN-2113.0012.patch,
> YARN-2113.0013.patch, YARN-2113.0014.patch, YARN-2113.0015.patch,
> YARN-2113.0016.patch, YARN-2113.0017.patch,
> YARN-2113.apply.onto.0012.ericp.patch, YARN-2113 Intra-QueuePreemption
> Behavior.pdf, YARN-2113.v0.patch
>
>
> Preemption today only works across queues and moves around resources across
> queues per demand and usage. We should also have user-level preemption within
> a queue, to balance capacity across users in a predictable manner.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
