[jira] [Commented] (YARN-6731) Add ability to export scheduler configuration XML
[
https://issues.apache.org/jira/browse/YARN-6731?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16171104#comment-16171104
]
Hadoop QA commented on YARN-6731:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 21m
36s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} YARN-5734 Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 18m
28s{color} | {color:green} YARN-5734 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
34s{color} | {color:green} YARN-5734 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
28s{color} | {color:green} YARN-5734 passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
39s{color} | {color:green} YARN-5734 passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
2s{color} | {color:green} YARN-5734 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
23s{color} | {color:green} YARN-5734 passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
33s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
32s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
32s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
24s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
35s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
6s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
18s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 50m 4s{color}
| {color:red} hadoop-yarn-server-resourcemanager in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
18s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 98m 24s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests |
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestContainerAllocation |
| Timed out junit tests |
org.apache.hadoop.yarn.server.resourcemanager.TestSubmitApplicationWithRMHA |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:14b5c93 |
| JIRA Issue | YARN-6731 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12887813/YARN-6731-YARN-5734.004.patch
|
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux ca79b4732812 3.13.0-123-generic #172-Ubuntu SMP Mon Jun 26
18:04:35 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | YARN-5734 / 9726e1f |
| Default Java | 1.8.0_144 |
| findbugs | v3.1.0-RC1 |
| unit |
https://builds.apache.org/job/PreCommit-YARN-Build/17511/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/17511/testReport/ |
| modules | C:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
U:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
|
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/17511/console |
| Powered by | Apache Yetus 0.6.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> Add ability to export scheduler configuration XML
> -
[jira] [Commented] (YARN-6731) Add ability to export scheduler configuration XML
[ https://issues.apache.org/jira/browse/YARN-6731?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16171031#comment-16171031 ] Jonathan Hung commented on YARN-6731: - Thanks [~csexz] for the rebase. Seems it no longer applies after YARN-6840. Attached 004 to address this - this patch exposes the conf through the conf provider instead of the scheduler, similar to the modifications in YARN-6840. > Add ability to export scheduler configuration XML > - > > Key: YARN-6731 > URL: https://issues.apache.org/jira/browse/YARN-6731 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Jonathan Hung >Assignee: Jonathan Hung > Attachments: YARN-6731-YARN-5734.001.patch, > YARN-6731-YARN-5734.002.patch, YARN-6731-YARN-5734.003.patch, > YARN-6731-YARN-5734.004.patch > > > This is useful for debugging/cluster migration/peace of mind. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-6731) Add ability to export scheduler configuration XML
[ https://issues.apache.org/jira/browse/YARN-6731?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hung updated YARN-6731: Attachment: YARN-6731-YARN-5734.004.patch > Add ability to export scheduler configuration XML > - > > Key: YARN-6731 > URL: https://issues.apache.org/jira/browse/YARN-6731 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Jonathan Hung >Assignee: Jonathan Hung > Attachments: YARN-6731-YARN-5734.001.patch, > YARN-6731-YARN-5734.002.patch, YARN-6731-YARN-5734.003.patch, > YARN-6731-YARN-5734.004.patch > > > This is useful for debugging/cluster migration/peace of mind. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6620) [YARN-6223] NM Java side code changes to support isolate GPU devices by using CGroups
[
https://issues.apache.org/jira/browse/YARN-6620?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16171027#comment-16171027
]
Hadoop QA commented on YARN-6620:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
16s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 15 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
31s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 13m
36s{color} | {color:red} root in trunk failed. {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 14m
26s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
7s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 2m
17s{color} | {color:green} trunk passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
51s{color} | {color:red}
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager
in trunk has 1 extant Findbugs warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
3s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
18s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
47s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 10m
53s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 10m
53s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
2m 13s{color} | {color:orange} root: The patch generated 74 new + 484 unchanged
- 24 fixed = 558 total (was 508) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 2m
30s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch 523 line(s) with tabs. {color} |
| {color:red}-1{color} | {color:red} xml {color} | {color:red} 0m 3s{color} |
{color:red} The patch has 1 ill-formed XML file(s). {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m
8s{color} | {color:red}
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager
generated 1 new + 1 unchanged - 0 fixed = 2 total (was 1) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
19s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 0m 44s{color}
| {color:red} hadoop-yarn-api in the patch failed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
49s{color} | {color:green} hadoop-yarn-common in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 14m 55s{color}
| {color:red} hadoop-yarn-server-nodemanager in the patch failed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 1m 54s{color}
| {color:red} hadoop-sls in the patch failed. {color} |
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m
36s{color} | {color:red} The patch generated 3 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black}112m 54s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs |
module:hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager
|
| | Field only ever set to null:null:
org.apache.hadoop.yarn.server.nodemanager.containermanager.resourceplugin.gpu.GpuResourcePlugin.gpuResourceHandler
In GpuResourcePlugin.java |
| Failed junit tests | hadoop.yarn.conf.TestYarnConfigurationFields |
| | hadoop.yarn.server.nodemanager.containermanager.TestContainerManager |
| | hadoop.yarn.server.nodemanager.TestLinuxContainerExecutorWithMocks |
| |
hadoop.yarn.server
[jira] [Commented] (YARN-6620) [YARN-6223] NM Java side code changes to support isolate GPU devices by using CGroups
[ https://issues.apache.org/jira/browse/YARN-6620?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16171026#comment-16171026 ] Zhankun Tang commented on YARN-6620: [~wangda], Thanks for the 008 patch. LGTM. > [YARN-6223] NM Java side code changes to support isolate GPU devices by using > CGroups > - > > Key: YARN-6620 > URL: https://issues.apache.org/jira/browse/YARN-6620 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Wangda Tan >Assignee: Wangda Tan > Attachments: YARN-6620.001.patch, YARN-6620.002.patch, > YARN-6620.003.patch, YARN-6620.004.patch, YARN-6620.005.patch, > YARN-6620.006-WIP.patch, YARN-6620.007.patch, YARN-6620.008.patch > > > This JIRA plan to add support of: > 1) GPU configuration for NodeManagers > 2) Isolation in CGroups. (Java side). > 3) NM restart and recovery allocated GPU devices -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6943) Update Yarn to YARN in documentation
[ https://issues.apache.org/jira/browse/YARN-6943?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16170986#comment-16170986 ] Chetna Chaudhari commented on YARN-6943: [~miklos.szeg...@cloudera.com]: Are you working on it, if not, can I pick it up? > Update Yarn to YARN in documentation > > > Key: YARN-6943 > URL: https://issues.apache.org/jira/browse/YARN-6943 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Miklos Szegedi >Priority: Minor > Labels: newbie > > Based on the discussion with [~templedf] in YARN-6757 the official case of > YARN is YARN, not Yarn, so we should update all the md files. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-2497) Changes for fair scheduler to support allocate resource respect labels
[
https://issues.apache.org/jira/browse/YARN-2497?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16170971#comment-16170971
]
Hadoop QA commented on YARN-2497:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
26s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 14 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
21s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 27m
57s{color} | {color:red} root in trunk failed. {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 16m
9s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
28s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 3m
28s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 5m
21s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
44s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
23s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
43s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 11m
21s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 11m
21s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
2m 32s{color} | {color:orange} hadoop-yarn-project/hadoop-yarn: The patch
generated 20 new + 1709 unchanged - 9 fixed = 1729 total (was 1718) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 3m
23s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 2 line(s) that end in whitespace. Use git
apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply
{color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 6m
21s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
29s{color} | {color:green} hadoop-yarn-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
7s{color} | {color:green}
hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager
generated 0 new + 296 unchanged - 1 fixed = 296 total (was 297) {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 5m
13s{color} | {color:green} hadoop-yarn-common in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 68m 55s{color}
| {color:red} hadoop-yarn-server-resourcemanager in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
57s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}174m 30s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests |
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestLeafQueue |
| | hadoop.yarn.server.resourcemanager.scheduler.capacity.TestQueueState |
| | hadoop.yarn.server.resourcemanager.monitor.TestSchedulingMonitor |
| |
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestContainerResizing |
| |
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestIncreaseAllocationExpirer
|
| |
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestContainerAllocation |
| | hadoop.yarn.server.resourcemanager.scheduler.capacity.TestReservations |
| | hadoop.yarn.server.resourcemanager.scheduler.capacity.TestParentQueue |
| |
hadoop.yarn.server.resourcemanager.reservation.TestFairSchedulerPlanFollowe
[jira] [Updated] (YARN-6620) [YARN-6223] NM Java side code changes to support isolate GPU devices by using CGroups
[ https://issues.apache.org/jira/browse/YARN-6620?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated YARN-6620: - Attachment: YARN-6620.008.patch Rebased to trunk (008) > [YARN-6223] NM Java side code changes to support isolate GPU devices by using > CGroups > - > > Key: YARN-6620 > URL: https://issues.apache.org/jira/browse/YARN-6620 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Wangda Tan >Assignee: Wangda Tan > Attachments: YARN-6620.001.patch, YARN-6620.002.patch, > YARN-6620.003.patch, YARN-6620.004.patch, YARN-6620.005.patch, > YARN-6620.006-WIP.patch, YARN-6620.007.patch, YARN-6620.008.patch > > > This JIRA plan to add support of: > 1) GPU configuration for NodeManagers > 2) Isolation in CGroups. (Java side). > 3) NM restart and recovery allocated GPU devices -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6620) [YARN-6223] NM Java side code changes to support isolate GPU devices by using CGroups
[
https://issues.apache.org/jira/browse/YARN-6620?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16170951#comment-16170951
]
Hadoop QA commented on YARN-6620:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
0s{color} | {color:blue} Docker mode activated. {color} |
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 6s{color}
| {color:red} YARN-6620 does not apply to trunk. Rebase required? Wrong Branch?
See https://wiki.apache.org/hadoop/HowToContribute for help. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | YARN-6620 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12887788/YARN-6620.007.patch |
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/17509/console |
| Powered by | Apache Yetus 0.6.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> [YARN-6223] NM Java side code changes to support isolate GPU devices by using
> CGroups
> -
>
> Key: YARN-6620
> URL: https://issues.apache.org/jira/browse/YARN-6620
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Wangda Tan
>Assignee: Wangda Tan
> Attachments: YARN-6620.001.patch, YARN-6620.002.patch,
> YARN-6620.003.patch, YARN-6620.004.patch, YARN-6620.005.patch,
> YARN-6620.006-WIP.patch, YARN-6620.007.patch
>
>
> This JIRA plan to add support of:
> 1) GPU configuration for NodeManagers
> 2) Isolation in CGroups. (Java side).
> 3) NM restart and recovery allocated GPU devices
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-6620) [YARN-6223] NM Java side code changes to support isolate GPU devices by using CGroups
[ https://issues.apache.org/jira/browse/YARN-6620?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated YARN-6620: - Attachment: YARN-6620.007.patch Attached ver.7 patch, added unit tests / javadocs. Also this patch is on top of YARN-3926 which can get GPU request from resource vector and set GPU resource value while registering to RM. > [YARN-6223] NM Java side code changes to support isolate GPU devices by using > CGroups > - > > Key: YARN-6620 > URL: https://issues.apache.org/jira/browse/YARN-6620 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Wangda Tan >Assignee: Wangda Tan > Attachments: YARN-6620.001.patch, YARN-6620.002.patch, > YARN-6620.003.patch, YARN-6620.004.patch, YARN-6620.005.patch, > YARN-6620.006-WIP.patch, YARN-6620.007.patch > > > This JIRA plan to add support of: > 1) GPU configuration for NodeManagers > 2) Isolation in CGroups. (Java side). > 3) NM restart and recovery allocated GPU devices -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-7201) Add more sophisticated example YARN service
[ https://issues.apache.org/jira/browse/YARN-7201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Yang updated YARN-7201: Attachment: YARN-7201.yarn-native-services.005.patch Added example document. > Add more sophisticated example YARN service > --- > > Key: YARN-7201 > URL: https://issues.apache.org/jira/browse/YARN-7201 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Jian He > Attachments: YARN-7201.yarn-native-services.001.patch, > YARN-7201.yarn-native-services.002.patch, > YARN-7201.yarn-native-services.003.patch, > YARN-7201.yarn-native-services.004.patch, > YARN-7201.yarn-native-services.005.patch > > > We can show case the following capabilities in the YARN service examples: > # Description of the service > # Component dependencies > # How to mount HDFS volume via NFS Gateway > # Enable privileged container > # Quick link to web application > # Queue to submit > # Use private docker registry -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Assigned] (YARN-7211) Task in SLS does't work
[
https://issues.apache.org/jira/browse/YARN-7211?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yufei Gu reassigned YARN-7211:
--
Assignee: Yufei Gu
> Task in SLS does't work
> ---
>
> Key: YARN-7211
> URL: https://issues.apache.org/jira/browse/YARN-7211
> Project: Hadoop YARN
> Issue Type: Bug
> Components: scheduler-load-simulator
>Affects Versions: 3.0.0-beta1, 3.1.0
>Reporter: Yufei Gu
>Assignee: Yufei Gu
>
> {code}
> java.lang.reflect.UndeclaredThrowableException
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1980)
> at
> org.apache.hadoop.yarn.sls.appmaster.MRAMSimulator.sendContainerRequest(MRAMSimulator.java:339)
> at
> org.apache.hadoop.yarn.sls.appmaster.AMSimulator.middleStep(AMSimulator.java:201)
> at
> org.apache.hadoop.yarn.sls.scheduler.TaskRunner$Task.run(TaskRunner.java:94)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Caused by:
> org.apache.hadoop.yarn.exceptions.InvalidApplicationMasterRequestException:
> Invalid responseId in AllocateRequest from application attempt:
> appattempt_1505762809623_0001_01, expect responseId to be 0, but get 1
> at
> org.apache.hadoop.yarn.server.resourcemanager.ApplicationMasterService.allocate(ApplicationMasterService.java:377)
> at
> org.apache.hadoop.yarn.sls.appmaster.MRAMSimulator$1.run(MRAMSimulator.java:343)
> at
> org.apache.hadoop.yarn.sls.appmaster.MRAMSimulator$1.run(MRAMSimulator.java:340)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1962)
> ... 6 more
> Exception in thread "pool-4-thread-8" java.lang.NullPointerException
> at
> org.apache.hadoop.yarn.sls.scheduler.TaskRunner$Task.run(TaskRunner.java:103)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> {code}
> Seems like it is broken by YARN-6640.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-7201) Add more sophisticated example YARN service
[ https://issues.apache.org/jira/browse/YARN-7201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Yang updated YARN-7201: Attachment: YARN-7201.yarn-native-services.004.patch > Add more sophisticated example YARN service > --- > > Key: YARN-7201 > URL: https://issues.apache.org/jira/browse/YARN-7201 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Jian He > Attachments: YARN-7201.yarn-native-services.001.patch, > YARN-7201.yarn-native-services.002.patch, > YARN-7201.yarn-native-services.003.patch, > YARN-7201.yarn-native-services.004.patch > > > We can show case the following capabilities in the YARN service examples: > # Description of the service > # Component dependencies > # How to mount HDFS volume via NFS Gateway > # Enable privileged container > # Quick link to web application > # Queue to submit > # Use private docker registry -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-7201) Add more sophisticated example YARN service
[ https://issues.apache.org/jira/browse/YARN-7201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Yang updated YARN-7201: Attachment: (was: YARN-7201.yarn-native-services.004.patch) > Add more sophisticated example YARN service > --- > > Key: YARN-7201 > URL: https://issues.apache.org/jira/browse/YARN-7201 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Jian He > Attachments: YARN-7201.yarn-native-services.001.patch, > YARN-7201.yarn-native-services.002.patch, > YARN-7201.yarn-native-services.003.patch > > > We can show case the following capabilities in the YARN service examples: > # Description of the service > # Component dependencies > # How to mount HDFS volume via NFS Gateway > # Enable privileged container > # Quick link to web application > # Queue to submit > # Use private docker registry -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-7201) Add more sophisticated example YARN service
[ https://issues.apache.org/jira/browse/YARN-7201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Yang updated YARN-7201: Attachment: YARN-7201.yarn-native-services.004.patch Remove mounts because YARN-4266 and YARN-7066 are not committed. > Add more sophisticated example YARN service > --- > > Key: YARN-7201 > URL: https://issues.apache.org/jira/browse/YARN-7201 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Jian He > Attachments: YARN-7201.yarn-native-services.001.patch, > YARN-7201.yarn-native-services.002.patch, > YARN-7201.yarn-native-services.003.patch, > YARN-7201.yarn-native-services.004.patch > > > We can show case the following capabilities in the YARN service examples: > # Description of the service > # Component dependencies > # How to mount HDFS volume via NFS Gateway > # Enable privileged container > # Quick link to web application > # Queue to submit > # Use private docker registry -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-2664) Improve RM webapp to expose info about reservations.
[
https://issues.apache.org/jira/browse/YARN-2664?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16170898#comment-16170898
]
Hadoop QA commented on YARN-2664:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
19s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 3 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
36s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 15m
34s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 16m
52s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
19s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 10m
46s{color} | {color:green} trunk passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Skipped patched modules with no Java source: . {color}
|
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
28s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 5m
9s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
19s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 18m
44s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 16m
9s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} cc {color} | {color:green} 16m
9s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 16m
9s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
36s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 12m
34s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Skipped patched modules with no Java source: . {color}
|
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 6m
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 6m
27s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red}171m 53s{color}
| {color:red} root in the patch failed. {color} |
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m
46s{color} | {color:red} The patch generated 5 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black}318m 26s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests |
hadoop.yarn.server.nodemanager.containermanager.TestContainerManager |
| | hadoop.hdfs.server.namenode.metrics.TestNameNodeMetrics |
| | hadoop.hdfs.TestLeaseRecovery |
| | hadoop.hdfs.TestReconstructStripedFile |
| | hadoop.hdfs.TestFileConcurrentReader |
| | hadoop.hdfs.TestLeaseRecoveryStriped |
| | hadoop.hdfs.TestRollingUpgrade |
| | hadoop.hdfs.server.namenode.TestReencryptionWithKMS |
| Timed out junit tests | org.apache.hadoop.hdfs.TestWriteReadStripedFile |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:71bbb86 |
| JIRA Issue | YARN-2664 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12887709/YARN-2664.22.patch |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle cc |
| uname | Linux b627cc274909 3.13.0-123-generic #172-Ubuntu SMP Mon Jun 26
18:04:35 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven
[jira] [Updated] (YARN-7201) Add more sophisticated example YARN service
[ https://issues.apache.org/jira/browse/YARN-7201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Yang updated YARN-7201: Attachment: YARN-7201.yarn-native-services.003.patch Fix typo. > Add more sophisticated example YARN service > --- > > Key: YARN-7201 > URL: https://issues.apache.org/jira/browse/YARN-7201 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Jian He > Attachments: YARN-7201.yarn-native-services.001.patch, > YARN-7201.yarn-native-services.002.patch, > YARN-7201.yarn-native-services.003.patch > > > We can show case the following capabilities in the YARN service examples: > # Description of the service > # Component dependencies > # How to mount HDFS volume via NFS Gateway > # Enable privileged container > # Quick link to web application > # Queue to submit > # Use private docker registry -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-7201) Add more sophisticated example YARN service
[ https://issues.apache.org/jira/browse/YARN-7201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Yang updated YARN-7201: Attachment: YARN-7201.yarn-native-services.002.patch Remove organization, description, icon metadata field to match the current state of REST API. > Add more sophisticated example YARN service > --- > > Key: YARN-7201 > URL: https://issues.apache.org/jira/browse/YARN-7201 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Jian He > Attachments: YARN-7201.yarn-native-services.001.patch, > YARN-7201.yarn-native-services.002.patch > > > We can show case the following capabilities in the YARN service examples: > # Description of the service > # Component dependencies > # How to mount HDFS volume via NFS Gateway > # Enable privileged container > # Quick link to web application > # Queue to submit > # Use private docker registry -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7203) Add container ExecutionType into ContainerReport
[ https://issues.apache.org/jira/browse/YARN-7203?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16170874#comment-16170874 ] Hudson commented on YARN-7203: -- FAILURE: Integrated in Jenkins build Hadoop-trunk-Commit #12905 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/12905/]) YARN-7203. Add container ExecutionType into ContainerReport. (Botong (arun suresh: rev 56ef5279c1db93d03b2f1e04badbfe804f548918) * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/records/impl/pb/ContainerReportPBImpl.java * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/test/java/org/apache/hadoop/yarn/client/api/impl/TestYarnClient.java * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmcontainer/RMContainerImpl.java * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/proto/yarn_protos.proto * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/ContainerReport.java > Add container ExecutionType into ContainerReport > > > Key: YARN-7203 > URL: https://issues.apache.org/jira/browse/YARN-7203 > Project: Hadoop YARN > Issue Type: Task >Reporter: Botong Huang >Assignee: Botong Huang >Priority: Minor > Fix For: 2.9.0, 3.0.0-beta1 > > Attachments: YARN-7203.v1.patch, YARN-7203.v2.patch, > YARN-7203.v3.patch, YARN-7203.v4.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7203) Add container ExecutionType into ContainerReport
[ https://issues.apache.org/jira/browse/YARN-7203?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16170870#comment-16170870 ] Botong Huang commented on YARN-7203: Thanks [~asuresh] for the review and commit! > Add container ExecutionType into ContainerReport > > > Key: YARN-7203 > URL: https://issues.apache.org/jira/browse/YARN-7203 > Project: Hadoop YARN > Issue Type: Task >Reporter: Botong Huang >Assignee: Botong Huang >Priority: Minor > Fix For: 2.9.0, 3.0.0-beta1 > > Attachments: YARN-7203.v1.patch, YARN-7203.v2.patch, > YARN-7203.v3.patch, YARN-7203.v4.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7203) Add container ExecutionType into ContainerReport
[
https://issues.apache.org/jira/browse/YARN-7203?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16170867#comment-16170867
]
Botong Huang commented on YARN-7203:
{{TestContainerAllocation.testAMContainerAllocationWhenDNSUnavailable}} is
being tracked under YARN-7044
> Add container ExecutionType into ContainerReport
>
>
> Key: YARN-7203
> URL: https://issues.apache.org/jira/browse/YARN-7203
> Project: Hadoop YARN
> Issue Type: Task
>Reporter: Botong Huang
>Assignee: Botong Huang
>Priority: Minor
> Fix For: 2.9.0, 3.0.0-beta1
>
> Attachments: YARN-7203.v1.patch, YARN-7203.v2.patch,
> YARN-7203.v3.patch, YARN-7203.v4.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7210) Some fixes related to Registry DNS
[
https://issues.apache.org/jira/browse/YARN-7210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16170864#comment-16170864
]
Hadoop QA commented on YARN-7210:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
16s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} yarn-native-services Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 3m
37s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 16m
53s{color} | {color:green} yarn-native-services passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 9m
52s{color} | {color:green} yarn-native-services passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
2s{color} | {color:green} yarn-native-services passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
52s{color} | {color:green} yarn-native-services passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Skipped patched modules with no Java source:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
51s{color} | {color:red}
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager
in yarn-native-services has 1 extant Findbugs warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
15s{color} | {color:green} yarn-native-services passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
10s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
13s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 6m
20s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 6m
20s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 59s{color} | {color:orange} hadoop-yarn-project/hadoop-yarn: The patch
generated 5 new + 72 unchanged - 0 fixed = 77 total (was 72) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
47s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Skipped patched modules with no Java source:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
33s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
13s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
52s{color} | {color:green} hadoop-yarn-registry in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 14m 15s{color}
| {color:red} hadoop-yarn-server-nodemanager in the patch failed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 8m 10s{color}
| {color:red} hadoop-yarn-services-core in the patch failed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
13s{color} | {color:green} hadoop-yarn-site in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
30s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 85m 7s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests |
hadoop.yarn.server.nodemanager.containermanager.TestContainerManager |
| | hadoop.yarn.service.TestYarnNativeServices |
| | hadoop.yarn.service.monitor.TestServiceMonitor |
\\
\\
||
[jira] [Commented] (YARN-6570) No logs were found for running application, running container
[
https://issues.apache.org/jira/browse/YARN-6570?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16170860#comment-16170860
]
Hadoop QA commented on YARN-6570:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
21s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 3 new or modified test
files. {color} |
|| || || || {color:brown} branch-2.8 Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
22s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
45s{color} | {color:green} branch-2.8 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
17s{color} | {color:green} branch-2.8 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
35s{color} | {color:green} branch-2.8 passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
59s{color} | {color:green} branch-2.8 passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
0s{color} | {color:green} branch-2.8 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
42s{color} | {color:green} branch-2.8 passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
11s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
49s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
13s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} cc {color} | {color:green} 2m
13s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m
13s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 31s{color} | {color:orange} hadoop-yarn-project/hadoop-yarn: The patch
generated 1 new + 98 unchanged - 0 fixed = 99 total (was 98) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
53s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
39s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
28s{color} | {color:green} hadoop-yarn-api in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 10m 3s{color}
| {color:red} hadoop-yarn-server-nodemanager in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
21s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 41m 57s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.yarn.server.nodemanager.TestNodeManagerResync |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:c2d96dd |
| JIRA Issue | YARN-6570 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12887768/YARN-6570-branch-2.8.001.patch
|
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle cc |
| uname | Linux febddf219d7c 4.4.0-43-generic #63-Ubuntu SMP Wed Oct 12
13:48:03 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | branch-2.8 / b8d86bb |
| Default Java | 1.7.0_151 |
| findbugs | v3.0.0 |
| checkstyle |
https://builds.apache.org/job/PreCommit-YARN-Build/17507/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn.txt
|
| unit |
https://builds.apache.org/job/PreCommit-YARN-Build/17507/artifact/patchprocess/patch-
[jira] [Commented] (YARN-6142) Support rolling upgrade between 2.x and 3.x
[ https://issues.apache.org/jira/browse/YARN-6142?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16170859#comment-16170859 ] Ray Chiang commented on YARN-6142: -- [~andrew.wang], I'll have to do the rest of the .proto files post beta1, but before GA. For JACC output, I'm down to 217 issues out of the 3207 generated by JACC. All the issues I've found so far are already documented as "Incompatible" in existing JIRAs. > Support rolling upgrade between 2.x and 3.x > --- > > Key: YARN-6142 > URL: https://issues.apache.org/jira/browse/YARN-6142 > Project: Hadoop YARN > Issue Type: Task > Components: rolling upgrade >Affects Versions: 3.0.0-alpha2 >Reporter: Andrew Wang >Assignee: Ray Chiang >Priority: Blocker > > Counterpart JIRA to HDFS-11096. We need to: > * examine YARN and MR's JACC report for binary and source incompatibilities > * run the [PB > differ|https://issues.apache.org/jira/browse/HDFS-11096?focusedCommentId=15816405&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15816405] > that Sean wrote for HDFS-11096 for the YARN PBs. > * sanity test some rolling upgrades between 2.x and 3.x. Ideally these are > automated and something we can run upstream. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7203) Add container ExecutionType into ContainerReport
[ https://issues.apache.org/jira/browse/YARN-7203?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16170857#comment-16170857 ] Arun Suresh commented on YARN-7203: --- Committing this shortly. The test failure is unrelated and the checkstyle issue is unavoidable. > Add container ExecutionType into ContainerReport > > > Key: YARN-7203 > URL: https://issues.apache.org/jira/browse/YARN-7203 > Project: Hadoop YARN > Issue Type: Task >Reporter: Botong Huang >Assignee: Botong Huang >Priority: Minor > Attachments: YARN-7203.v1.patch, YARN-7203.v2.patch, > YARN-7203.v3.patch, YARN-7203.v4.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7196) Fix finicky TestContainerManager tests
[
https://issues.apache.org/jira/browse/YARN-7196?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16170815#comment-16170815
]
Hadoop QA commented on YARN-7196:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
30s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 16m
38s{color} | {color:red} root in trunk failed. {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
0s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
27s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
44s{color} | {color:green} trunk passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m
6s{color} | {color:red}
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager
in trunk has 1 extant Findbugs warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
26s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
41s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
0s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 24s{color} | {color:orange}
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager:
The patch generated 2 new + 100 unchanged - 0 fixed = 102 total (was 100)
{color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
46s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
25s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
26s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 16m
26s{color} | {color:green} hadoop-yarn-server-nodemanager in the patch passed.
{color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
32s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 44m 4s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:71bbb86 |
| JIRA Issue | YARN-7196 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12887708/YARN-7196.002.patch |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux 2873251ed106 3.13.0-119-generic #166-Ubuntu SMP Wed May 3
12:18:55 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / 1ee2527 |
| Default Java | 1.8.0_144 |
| mvninstall |
https://builds.apache.org/job/PreCommit-YARN-Build/17505/artifact/patchprocess/branch-mvninstall-root.txt
|
| findbugs | v3.1.0-RC1 |
| findbugs |
https://builds.apache.org/job/PreCommit-YARN-Build/17505/artifact/patchprocess/branch-findbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager-warnings.html
|
| checkstyle |
https://builds.apache.org/job/PreCommit-YARN-Build/17505/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/17505/testReport/ |
| modules | C:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager
U:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nod
[jira] [Updated] (YARN-2497) Changes for fair scheduler to support allocate resource respect labels
[ https://issues.apache.org/jira/browse/YARN-2497?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Templeton updated YARN-2497: --- Attachment: YARN-2497.002.patch Let's see what this breaks. > Changes for fair scheduler to support allocate resource respect labels > -- > > Key: YARN-2497 > URL: https://issues.apache.org/jira/browse/YARN-2497 > Project: Hadoop YARN > Issue Type: Sub-task > Components: fairscheduler >Reporter: Wangda Tan >Assignee: Daniel Templeton > Attachments: YARN-2497.001.patch, YARN-2497.002.patch, > YARN-2499.WIP01.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5534) Allow whitelisted volume mounts
[ https://issues.apache.org/jira/browse/YARN-5534?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16170803#comment-16170803 ] Eric Yang commented on YARN-5534: - [~miklos.szeg...@cloudera.com] I think core-site.xml make most sense to ensure both hdfs and yarn can agree on same security setting even though hdfs service doesn't require knowledge of this today. The idea of global white list and job specific white lists, have their own attractiveness. However, I think having global white list in container-executor.cfg might be risky. If the information is leaked and admin did not secure white list mount point properly, then the system could be vulnerable to attack. For white list, more eye balls can examine the configuration, would make the system more secure. On the other hand, if a black list is to be implemented, then it might have advantage to be in container-executor.cfg and readable by root only. Basic security through obscurity can be performed with some level of effectiveness. > Allow whitelisted volume mounts > > > Key: YARN-5534 > URL: https://issues.apache.org/jira/browse/YARN-5534 > Project: Hadoop YARN > Issue Type: Sub-task > Components: yarn >Reporter: luhuichun >Assignee: Shane Kumpf > Attachments: YARN-5534.001.patch, YARN-5534.002.patch, > YARN-5534.003.patch > > > Introduction > Mounting files or directories from the host is one way of passing > configuration and other information into a docker container. > We could allow the user to set a list of mounts in the environment of > ContainerLaunchContext (e.g. /dir1:/targetdir1,/dir2:/targetdir2). > These would be mounted read-only to the specified target locations. This has > been resolved in YARN-4595 > 2.Problem Definition > Bug mounting arbitrary volumes into a Docker container can be a security risk. > 3.Possible solutions > one approach to provide safe mounts is to allow the cluster administrator to > configure a set of parent directories as white list mounting directories. > Add a property named yarn.nodemanager.volume-mounts.white-list, when > container executor do mount checking, only the allowed directories or > sub-directories can be mounted. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6570) No logs were found for running application, running container
[
https://issues.apache.org/jira/browse/YARN-6570?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16170800#comment-16170800
]
Junping Du commented on YARN-6570:
--
Submit patch to trigger jenkins report.
> No logs were found for running application, running container
> -
>
> Key: YARN-6570
> URL: https://issues.apache.org/jira/browse/YARN-6570
> Project: Hadoop YARN
> Issue Type: Bug
> Components: yarn
>Reporter: Sumana Sathish
>Assignee: Junping Du
>Priority: Critical
> Fix For: 2.9.0, 3.0.0-beta1, 3.1.0
>
> Attachments: YARN-6570-branch-2.8.001.patch, YARN-6570.poc.patch,
> YARN-6570-v2.patch, YARN-6570-v3.patch
>
>
> 1.Obtain running containers from the following CLI for running application:
> yarn container -list appattempt
> 2. Couldnot fetch logs
> {code}
> Can not find any log file matching the pattern: ALL for the container
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-6570) No logs were found for running application, running container
[
https://issues.apache.org/jira/browse/YARN-6570?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Junping Du updated YARN-6570:
-
Attachment: YARN-6570-branch-2.8.001.patch
Upload patch for branch-2.8. Basically, adding SCHEDULED container status
before container get actually running which keep consistent with trunk/branch-2.
> No logs were found for running application, running container
> -
>
> Key: YARN-6570
> URL: https://issues.apache.org/jira/browse/YARN-6570
> Project: Hadoop YARN
> Issue Type: Bug
> Components: yarn
>Reporter: Sumana Sathish
>Assignee: Junping Du
>Priority: Critical
> Fix For: 2.9.0, 3.0.0-beta1, 3.1.0
>
> Attachments: YARN-6570-branch-2.8.001.patch, YARN-6570.poc.patch,
> YARN-6570-v2.patch, YARN-6570-v3.patch
>
>
> 1.Obtain running containers from the following CLI for running application:
> yarn container -list appattempt
> 2. Couldnot fetch logs
> {code}
> Can not find any log file matching the pattern: ALL for the container
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Reopened] (YARN-6570) No logs were found for running application, running container
[
https://issues.apache.org/jira/browse/YARN-6570?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Junping Du reopened YARN-6570:
--
> No logs were found for running application, running container
> -
>
> Key: YARN-6570
> URL: https://issues.apache.org/jira/browse/YARN-6570
> Project: Hadoop YARN
> Issue Type: Bug
> Components: yarn
>Reporter: Sumana Sathish
>Assignee: Junping Du
>Priority: Critical
> Fix For: 2.9.0, 3.0.0-beta1, 3.1.0
>
> Attachments: YARN-6570.poc.patch, YARN-6570-v2.patch,
> YARN-6570-v3.patch
>
>
> 1.Obtain running containers from the following CLI for running application:
> yarn container -list appattempt
> 2. Couldnot fetch logs
> {code}
> Can not find any log file matching the pattern: ALL for the container
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-3661) Basic Federation UI
[
https://issues.apache.org/jira/browse/YARN-3661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16170791#comment-16170791
]
Hadoop QA commented on YARN-3661:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
20s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
45s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 13m
0s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 9m
32s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
58s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
48s{color} | {color:green} trunk passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Skipped patched modules with no Java source:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
33s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
34s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
10s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
26s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 5m
45s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 5m
45s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
48s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
37s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
1s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Skipped patched modules with no Java source:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
42s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
40s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m
11s{color} | {color:green} hadoop-yarn-server-router in the patch passed.
{color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
18s{color} | {color:green} hadoop-yarn-site in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
32s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 46m 51s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:71bbb86 |
| JIRA Issue | YARN-3661 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12887751/YARN-3661-008.patch |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit xml findbugs checkstyle |
| uname | Linux 07c17757c413 4.4.0-43-generic #63-Ubuntu SMP Wed Oct 12
13:48:03 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revisio
[jira] [Commented] (YARN-7196) Fix finicky TestContainerManager tests
[
https://issues.apache.org/jira/browse/YARN-7196?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16170788#comment-16170788
]
Arun Suresh commented on YARN-7196:
---
[~djp], let me know if the latest patch is fine (in which case I will fix the
unused imports while committing).
> Fix finicky TestContainerManager tests
> --
>
> Key: YARN-7196
> URL: https://issues.apache.org/jira/browse/YARN-7196
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Arun Suresh
>Assignee: Arun Suresh
> Attachments: YARN-7196.002.patch, YARN-7196.patch
>
>
> The Testcase {{testContainerUpdateExecTypeGuaranteedToOpportunistic}} seem to
> fail every once in a while. Maybe have to change the way the event is
> triggered.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7196) Fix finicky TestContainerManager tests
[
https://issues.apache.org/jira/browse/YARN-7196?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16170782#comment-16170782
]
Hadoop QA commented on YARN-7196:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
19s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 12m
31s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
40s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
15s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
25s{color} | {color:green} trunk passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
38s{color} | {color:red}
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager
in trunk has 1 extant Findbugs warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
19s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
24s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
37s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
37s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 15s{color} | {color:orange}
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager:
The patch generated 2 new + 100 unchanged - 0 fixed = 102 total (was 100)
{color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
43s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
15s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 14m
21s{color} | {color:green} hadoop-yarn-server-nodemanager in the patch passed.
{color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 33m 46s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:71bbb86 |
| JIRA Issue | YARN-7196 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12887708/YARN-7196.002.patch |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux d7c7934b6970 4.4.0-43-generic #63-Ubuntu SMP Wed Oct 12
13:48:03 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / 7c73292 |
| Default Java | 1.8.0_144 |
| findbugs | v3.1.0-RC1 |
| findbugs |
https://builds.apache.org/job/PreCommit-YARN-Build/17504/artifact/patchprocess/branch-findbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager-warnings.html
|
| checkstyle |
https://builds.apache.org/job/PreCommit-YARN-Build/17504/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/17504/testReport/ |
| modules | C:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager
U:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager
|
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/17504/console |
| Powered by | Apache Yetus 0
[jira] [Commented] (YARN-7203) Add container ExecutionType into ContainerReport
[
https://issues.apache.org/jira/browse/YARN-7203?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16170776#comment-16170776
]
Hadoop QA commented on YARN-7203:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
18s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
42s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 12m
8s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 9m
45s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
50s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 2m
4s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
47s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
41s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
8s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
42s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 5m
48s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} cc {color} | {color:green} 5m
48s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 5m
48s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 51s{color} | {color:orange} hadoop-yarn-project/hadoop-yarn: The patch
generated 1 new + 121 unchanged - 0 fixed = 122 total (was 121) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 2m
12s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
26s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
36s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
30s{color} | {color:green} hadoop-yarn-api in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
37s{color} | {color:green} hadoop-yarn-common in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 45m 52s{color}
| {color:red} hadoop-yarn-server-resourcemanager in the patch failed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 21m
6s{color} | {color:green} hadoop-yarn-client in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
28s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}127m 36s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests |
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestContainerAllocation |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:71bbb86 |
| JIRA Issue | YARN-7203 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12887726/YARN-7203.v4.patch |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle cc |
| uname | Linux 9548e6f150cd 4.4.0-43-generic #63-Ubuntu SMP Wed Oct 12
13:48:03 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / 29dd551 |
| Default Ja
[jira] [Updated] (YARN-7201) Add more sophisticated example YARN service
[ https://issues.apache.org/jira/browse/YARN-7201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Yang updated YARN-7201: Attachment: YARN-7201.yarn-native-services.001.patch Example for deploying Apache, MySQL, PHPMyAdmin docker containers. > Add more sophisticated example YARN service > --- > > Key: YARN-7201 > URL: https://issues.apache.org/jira/browse/YARN-7201 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Jian He > Attachments: YARN-7201.yarn-native-services.001.patch > > > We can show case the following capabilities in the YARN service examples: > # Description of the service > # Component dependencies > # How to mount HDFS volume via NFS Gateway > # Enable privileged container > # Quick link to web application > # Queue to submit > # Use private docker registry -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6570) No logs were found for running application, running container
[
https://issues.apache.org/jira/browse/YARN-6570?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16170763#comment-16170763
]
Junping Du commented on YARN-6570:
--
Thanks [~xgong] for review and commit the patch. I would like to upload a patch
for branch-2.8 as well so will reopen the jira for Jenkins test.
> No logs were found for running application, running container
> -
>
> Key: YARN-6570
> URL: https://issues.apache.org/jira/browse/YARN-6570
> Project: Hadoop YARN
> Issue Type: Bug
> Components: yarn
>Reporter: Sumana Sathish
>Assignee: Junping Du
>Priority: Critical
> Fix For: 2.9.0, 3.0.0-beta1, 3.1.0
>
> Attachments: YARN-6570.poc.patch, YARN-6570-v2.patch,
> YARN-6570-v3.patch
>
>
> 1.Obtain running containers from the following CLI for running application:
> yarn container -list appattempt
> 2. Couldnot fetch logs
> {code}
> Can not find any log file matching the pattern: ALL for the container
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-4266) Allow users to enter containers as UID:GID pair instead of by username
[
https://issues.apache.org/jira/browse/YARN-4266?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16170762#comment-16170762
]
Jason Lowe commented on YARN-4266:
--
Apologies for arriving late on this. Comments on the 003 patch:
These comments don't match the code:
{code}
/** Set user remapping lower uid limit to 1000 by default */
public static final int DEFAULT_NM_DOCKER_USER_REMAPPING_UID_THRESHOLD = 1;
[...]
/** Set user remapping lower gid limit to 100 by default */
public static final int DEFAULT_NM_DOCKER_USER_REMAPPING_GID_THRESHOLD = 1;
{code}
Should we handle ExitCodeException or other types of exceptions that might
happen (e.g.: "no such user" type of errors) explicitly when running the {{id}}
command so we can provide a better debug experience, or is the exception
message enough info to debug issues?
Also I found it odd that getUserIdInfo and getGroupIdInfo take a parameter for
the {{id}} command but these methods are highly dependent upon the "right"
parameter being passed in order to function properly. They are each only
called in one place, and IMHO there's no reason to make this parameterized
given the parsing code _needs_ the corresponding parameter to be correct. We
should just remove the parameter and have it passed directly.
> Allow users to enter containers as UID:GID pair instead of by username
> --
>
> Key: YARN-4266
> URL: https://issues.apache.org/jira/browse/YARN-4266
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: yarn
>Reporter: Sidharta Seethana
>Assignee: luhuichun
> Attachments: YARN-4266.001.patch, YARN-4266.001.patch,
> YARN-4266.002.patch, YARN-4266.003.patch,
> YARN-4266_Allow_whitelisted_users_to_disable_user_re-mapping.pdf,
> YARN-4266_Allow_whitelisted_users_to_disable_user_re-mapping_v2.pdf,
> YARN-4266_Allow_whitelisted_users_to_disable_user_re-mapping_v3.pdf,
> YARN-4266-branch-2.8.001.patch
>
>
> Docker provides a mechanism (the --user switch) that enables us to specify
> the user the container processes should run as. We use this mechanism today
> when launching docker containers . In non-secure mode, we run the docker
> container based on
> `yarn.nodemanager.linux-container-executor.nonsecure-mode.local-user` and in
> secure mode, as the submitting user. However, this mechanism breaks down with
> a large number of 'pre-created' images which don't necessarily have the users
> available within the image. Examples of such images include shared images
> that need to be used by multiple users. We need a way in which we can allow a
> pre-defined set of users to run containers based on existing images, without
> using the --user switch. There are some implications of disabling this user
> squashing that we'll need to work through : log aggregation, artifact
> deletion etc.,
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7210) Some fixes related to Registry DNS
[ https://issues.apache.org/jira/browse/YARN-7210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16170755#comment-16170755 ] Jian He commented on YARN-7210: --- In the new patch, I also included a fix for the CLI to take the --queue parameter > Some fixes related to Registry DNS > -- > > Key: YARN-7210 > URL: https://issues.apache.org/jira/browse/YARN-7210 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Jian He >Assignee: Jian He > Attachments: YARN-7210.yarn-native-services.01.patch, > YARN-7210.yarn-native-services.02.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-7210) Some fixes related to Registry DNS
[ https://issues.apache.org/jira/browse/YARN-7210?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jian He updated YARN-7210: -- Attachment: YARN-7210.yarn-native-services.02.patch > Some fixes related to Registry DNS > -- > > Key: YARN-7210 > URL: https://issues.apache.org/jira/browse/YARN-7210 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Jian He >Assignee: Jian He > Attachments: YARN-7210.yarn-native-services.01.patch, > YARN-7210.yarn-native-services.02.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6570) No logs were found for running application, running container
[
https://issues.apache.org/jira/browse/YARN-6570?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16170744#comment-16170744
]
Hudson commented on YARN-6570:
--
SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #12900 (See
[https://builds.apache.org/job/Hadoop-trunk-Commit/12900/])
YARN-6570. No logs were found for running application, running (xgong: rev
7c732924a889cd280e972882619a1827877fbafa)
* (edit)
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/container/ContainerImpl.java
* (edit)
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/container/TestContainer.java
* (edit)
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/TestEventFlow.java
> No logs were found for running application, running container
> -
>
> Key: YARN-6570
> URL: https://issues.apache.org/jira/browse/YARN-6570
> Project: Hadoop YARN
> Issue Type: Bug
> Components: yarn
>Reporter: Sumana Sathish
>Assignee: Junping Du
>Priority: Critical
> Fix For: 2.9.0, 3.0.0-beta1, 3.1.0
>
> Attachments: YARN-6570.poc.patch, YARN-6570-v2.patch,
> YARN-6570-v3.patch
>
>
> 1.Obtain running containers from the following CLI for running application:
> yarn container -list appattempt
> 2. Couldnot fetch logs
> {code}
> Can not find any log file matching the pattern: ALL for the container
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6626) Embed REST API service into RM
[ https://issues.apache.org/jira/browse/YARN-6626?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16170741#comment-16170741 ] Eric Yang commented on YARN-6626: - Base on new finding that ui2 webapp is highly coupled with RMWebServices.java. However /ui2 and RMWebServices are initialized are two separate web applications. Java Sessions are not synchronized between both web applications. At this time, ui2 is pure javascript based webapp. Java session is not yet a real problem. However, it might be best to manage application oriented REST end points independent from ui2 webapplication to avoid introducing out of sync Java session problems. > Embed REST API service into RM > -- > > Key: YARN-6626 > URL: https://issues.apache.org/jira/browse/YARN-6626 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Gour Saha > Fix For: yarn-native-services > > Attachments: YARN-6626.yarn-native-services.001.patch > > > As of now the deployment model of the Native Services REST API service is > standalone. There are several cross-cutting solutions that can be inherited > for free (kerberos, HA, ACLs, trusted proxy support, etc.) by the REST API > service if it is embedded into the RM process. In fact we can expose the REST > API via the same port as RM UI (8088 default). The URI path > /services/v1/applications will distinguish the REST API calls from other RM > APIs. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6570) No logs were found for running application, running container
[
https://issues.apache.org/jira/browse/YARN-6570?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16170719#comment-16170719
]
Xuan Gong commented on YARN-6570:
-
Committed into trunk, branch-3.0 and branch-2. Thanks for the fix, junping
> No logs were found for running application, running container
> -
>
> Key: YARN-6570
> URL: https://issues.apache.org/jira/browse/YARN-6570
> Project: Hadoop YARN
> Issue Type: Bug
> Components: yarn
>Reporter: Sumana Sathish
>Assignee: Junping Du
>Priority: Critical
> Fix For: 2.9.0, 3.0.0-beta1, 3.1.0
>
> Attachments: YARN-6570.poc.patch, YARN-6570-v2.patch,
> YARN-6570-v3.patch
>
>
> 1.Obtain running containers from the following CLI for running application:
> yarn container -list appattempt
> 2. Couldnot fetch logs
> {code}
> Can not find any log file matching the pattern: ALL for the container
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-3661) Basic Federation UI
[ https://issues.apache.org/jira/browse/YARN-3661?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Íñigo Goiri updated YARN-3661: -- Attachment: YARN-3661-008.patch > Basic Federation UI > > > Key: YARN-3661 > URL: https://issues.apache.org/jira/browse/YARN-3661 > Project: Hadoop YARN > Issue Type: Sub-task > Components: nodemanager, resourcemanager >Reporter: Giovanni Matteo Fumarola >Assignee: Íñigo Goiri > Attachments: YARN-3661-000.patch, YARN-3661-001.patch, > YARN-3661-002.patch, YARN-3661-003.patch, YARN-3661-004.patch, > YARN-3661-005.patch, YARN-3661-006.patch, YARN-3661-007.patch, > YARN-3661-008.patch > > > The UIs provided by each RM, provide a correct "local" view of what is > running in a sub-cluster. In the context of federation we need new > UIs that can track load, jobs, users across sub-clusters. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6570) No logs were found for running application, running container
[
https://issues.apache.org/jira/browse/YARN-6570?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16170713#comment-16170713
]
Xuan Gong commented on YARN-6570:
-
+1 LGTM. Checking this in
> No logs were found for running application, running container
> -
>
> Key: YARN-6570
> URL: https://issues.apache.org/jira/browse/YARN-6570
> Project: Hadoop YARN
> Issue Type: Bug
> Components: yarn
>Reporter: Sumana Sathish
>Assignee: Junping Du
>Priority: Critical
> Attachments: YARN-6570.poc.patch, YARN-6570-v2.patch,
> YARN-6570-v3.patch
>
>
> 1.Obtain running containers from the following CLI for running application:
> yarn container -list appattempt
> 2. Couldnot fetch logs
> {code}
> Can not find any log file matching the pattern: ALL for the container
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5534) Allow whitelisted volume mounts
[ https://issues.apache.org/jira/browse/YARN-5534?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16170610#comment-16170610 ] Miklos Szegedi commented on YARN-5534: -- Thank you, [~eyang] for the comment. Can you please clarify where the user reads the whitelist from? yarn-site.xml? Would it be useful to have the whitelist in both Java and container-executor like [~vinodkv] suggested above? The right location depends also on whether privileged containers are enabled or not, which is the preference of the administrator. > Allow whitelisted volume mounts > > > Key: YARN-5534 > URL: https://issues.apache.org/jira/browse/YARN-5534 > Project: Hadoop YARN > Issue Type: Sub-task > Components: yarn >Reporter: luhuichun >Assignee: Shane Kumpf > Attachments: YARN-5534.001.patch, YARN-5534.002.patch, > YARN-5534.003.patch > > > Introduction > Mounting files or directories from the host is one way of passing > configuration and other information into a docker container. > We could allow the user to set a list of mounts in the environment of > ContainerLaunchContext (e.g. /dir1:/targetdir1,/dir2:/targetdir2). > These would be mounted read-only to the specified target locations. This has > been resolved in YARN-4595 > 2.Problem Definition > Bug mounting arbitrary volumes into a Docker container can be a security risk. > 3.Possible solutions > one approach to provide safe mounts is to allow the cluster administrator to > configure a set of parent directories as white list mounting directories. > Add a property named yarn.nodemanager.volume-mounts.white-list, when > container executor do mount checking, only the allowed directories or > sub-directories can be mounted. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-7149) Cross-queue preemption sometimes starves an underserved queue
[
https://issues.apache.org/jira/browse/YARN-7149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Eric Payne updated YARN-7149:
-
Fix Version/s: 3.1.0
2.9.0
> Cross-queue preemption sometimes starves an underserved queue
> -
>
> Key: YARN-7149
> URL: https://issues.apache.org/jira/browse/YARN-7149
> Project: Hadoop YARN
> Issue Type: Bug
> Components: capacity scheduler
>Affects Versions: 2.9.0, 3.0.0-alpha3
>Reporter: Eric Payne
>Assignee: Eric Payne
> Fix For: 2.9.0, 3.0.0-beta1, 3.1.0
>
> Attachments: YARN-7149.001.patch, YARN-7149.002.patch,
> YARN-7149.demo.unit-test.patch
>
>
> In branch 2 and trunk, I am consistently seeing some use cases where
> cross-queue preemption does not happen when it should. I do not see this in
> branch-2.8.
> Use Case:
> | | *Size* | *Minimum Container Size* |
> |MyCluster | 20 GB | 0.5 GB |
> | *Queue Name* | *Capacity* | *Absolute Capacity* | *Minimum User Limit
> Percent (MULP)* | *User Limit Factor (ULF)* |
> |Q1 | 50% = 10 GB | 100% = 20 GB | 10% = 1 GB | 2.0 |
> |Q2 | 50% = 10 GB | 100% = 20 GB | 10% = 1 GB | 2.0 |
> - {{User1}} launches {{App1}} in {{Q1}} and consumes all resources (20 GB)
> - {{User2}} launches {{App2}} in {{Q2}} and requests 10 GB
> - _Note: containers are 0.5 GB._
> - Preemption monitor kills 2 containers (equals 1 GB) from {{App1}} in {{Q1}}.
> - Capacity Scheduler assigns 2 containers (equals 1 GB) to {{App2}} in {{Q2}}.
> - _No more containers are ever preempted, even though {{Q2}} is far
> underserved_
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7210) Some fixes related to Registry DNS
[
https://issues.apache.org/jira/browse/YARN-7210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16170539#comment-16170539
]
Hadoop QA commented on YARN-7210:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
19s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} yarn-native-services Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
21s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 14m
6s{color} | {color:green} yarn-native-services passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 9m
18s{color} | {color:green} yarn-native-services passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
55s{color} | {color:green} yarn-native-services passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
16s{color} | {color:green} yarn-native-services passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
43s{color} | {color:red}
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager
in yarn-native-services has 1 extant Findbugs warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
53s{color} | {color:green} yarn-native-services passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
9s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
56s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 5m
49s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 5m
49s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 50s{color} | {color:orange} hadoop-yarn-project/hadoop-yarn: The patch
generated 1 new + 48 unchanged - 0 fixed = 49 total (was 48) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
12s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
10s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
47s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 0m 46s{color}
| {color:red} hadoop-yarn-registry in the patch failed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 14m 2s{color}
| {color:red} hadoop-yarn-server-nodemanager in the patch failed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 8m 4s{color}
| {color:red} hadoop-yarn-services-core in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
22s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 75m 3s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.registry.server.dns.TestRegistryDNS |
| | hadoop.registry.server.dns.TestSecureRegistryDNS |
| | hadoop.yarn.server.nodemanager.containermanager.TestContainerManager |
| | hadoop.yarn.service.TestYarnNativeServices |
| | hadoop.yarn.service.monitor.TestServiceMonitor |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:71bbb86 |
| JIRA Issue | YARN-7210 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12887706/YARN-7210.yarn-native-services.01.patch
|
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname
[jira] [Assigned] (YARN-7211) Task in SLS does't work
[
https://issues.apache.org/jira/browse/YARN-7211?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yufei Gu reassigned YARN-7211:
--
Assignee: (was: Yufei Gu)
> Task in SLS does't work
> ---
>
> Key: YARN-7211
> URL: https://issues.apache.org/jira/browse/YARN-7211
> Project: Hadoop YARN
> Issue Type: Bug
> Components: scheduler-load-simulator
>Affects Versions: 3.0.0-beta1, 3.1.0
>Reporter: Yufei Gu
>
> {code}
> java.lang.reflect.UndeclaredThrowableException
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1980)
> at
> org.apache.hadoop.yarn.sls.appmaster.MRAMSimulator.sendContainerRequest(MRAMSimulator.java:339)
> at
> org.apache.hadoop.yarn.sls.appmaster.AMSimulator.middleStep(AMSimulator.java:201)
> at
> org.apache.hadoop.yarn.sls.scheduler.TaskRunner$Task.run(TaskRunner.java:94)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Caused by:
> org.apache.hadoop.yarn.exceptions.InvalidApplicationMasterRequestException:
> Invalid responseId in AllocateRequest from application attempt:
> appattempt_1505762809623_0001_01, expect responseId to be 0, but get 1
> at
> org.apache.hadoop.yarn.server.resourcemanager.ApplicationMasterService.allocate(ApplicationMasterService.java:377)
> at
> org.apache.hadoop.yarn.sls.appmaster.MRAMSimulator$1.run(MRAMSimulator.java:343)
> at
> org.apache.hadoop.yarn.sls.appmaster.MRAMSimulator$1.run(MRAMSimulator.java:340)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1962)
> ... 6 more
> Exception in thread "pool-4-thread-8" java.lang.NullPointerException
> at
> org.apache.hadoop.yarn.sls.scheduler.TaskRunner$Task.run(TaskRunner.java:103)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> {code}
> Seems like it is broken by YARN-6640.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-7211) Task in SLS does't work
Yufei Gu created YARN-7211:
--
Summary: Task in SLS does't work
Key: YARN-7211
URL: https://issues.apache.org/jira/browse/YARN-7211
Project: Hadoop YARN
Issue Type: Bug
Components: scheduler-load-simulator
Affects Versions: 3.0.0-beta1, 3.1.0
Reporter: Yufei Gu
Assignee: Yufei Gu
{code}
java.lang.reflect.UndeclaredThrowableException
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1980)
at
org.apache.hadoop.yarn.sls.appmaster.MRAMSimulator.sendContainerRequest(MRAMSimulator.java:339)
at
org.apache.hadoop.yarn.sls.appmaster.AMSimulator.middleStep(AMSimulator.java:201)
at
org.apache.hadoop.yarn.sls.scheduler.TaskRunner$Task.run(TaskRunner.java:94)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by:
org.apache.hadoop.yarn.exceptions.InvalidApplicationMasterRequestException:
Invalid responseId in AllocateRequest from application attempt:
appattempt_1505762809623_0001_01, expect responseId to be 0, but get 1
at
org.apache.hadoop.yarn.server.resourcemanager.ApplicationMasterService.allocate(ApplicationMasterService.java:377)
at
org.apache.hadoop.yarn.sls.appmaster.MRAMSimulator$1.run(MRAMSimulator.java:343)
at
org.apache.hadoop.yarn.sls.appmaster.MRAMSimulator$1.run(MRAMSimulator.java:340)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1962)
... 6 more
Exception in thread "pool-4-thread-8" java.lang.NullPointerException
at
org.apache.hadoop.yarn.sls.scheduler.TaskRunner$Task.run(TaskRunner.java:103)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
{code}
Seems like it is broken by YARN-6640.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (YARN-5534) Allow whitelisted volume mounts
[ https://issues.apache.org/jira/browse/YARN-5534?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16168728#comment-16168728 ] Miklos Szegedi edited comment on YARN-5534 at 9/18/17 7:25 PM: --- [~eyang] I would approach this from the user point of view. container-executor and container-executor.cfg should govern the rules, how the yarn user can get root access or access it does not have otherwise. If the yarn user cannot access a directory, then mounting it should be whitelisted in container-executor.cfg. was (Author: miklos.szeg...@cloudera.com): [~eyang] I would approach this from the user point of new. container-executor and container-executor.cfg should govern the rules, how the yarn user can get root access or access it does not have otherwise. If the yarn user cannot access a directory, then mounting it should be whitelisted in container-executor.cfg. > Allow whitelisted volume mounts > > > Key: YARN-5534 > URL: https://issues.apache.org/jira/browse/YARN-5534 > Project: Hadoop YARN > Issue Type: Sub-task > Components: yarn >Reporter: luhuichun >Assignee: Shane Kumpf > Attachments: YARN-5534.001.patch, YARN-5534.002.patch, > YARN-5534.003.patch > > > Introduction > Mounting files or directories from the host is one way of passing > configuration and other information into a docker container. > We could allow the user to set a list of mounts in the environment of > ContainerLaunchContext (e.g. /dir1:/targetdir1,/dir2:/targetdir2). > These would be mounted read-only to the specified target locations. This has > been resolved in YARN-4595 > 2.Problem Definition > Bug mounting arbitrary volumes into a Docker container can be a security risk. > 3.Possible solutions > one approach to provide safe mounts is to allow the cluster administrator to > configure a set of parent directories as white list mounting directories. > Add a property named yarn.nodemanager.volume-mounts.white-list, when > container executor do mount checking, only the allowed directories or > sub-directories can be mounted. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7203) Add container ExecutionType into ContainerReport
[ https://issues.apache.org/jira/browse/YARN-7203?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16170529#comment-16170529 ] Arun Suresh commented on YARN-7203: --- Thanks for the update [~botong]. The latest patch lgtm. With commit pending jenkins > Add container ExecutionType into ContainerReport > > > Key: YARN-7203 > URL: https://issues.apache.org/jira/browse/YARN-7203 > Project: Hadoop YARN > Issue Type: Task >Reporter: Botong Huang >Assignee: Botong Huang >Priority: Minor > Attachments: YARN-7203.v1.patch, YARN-7203.v2.patch, > YARN-7203.v3.patch, YARN-7203.v4.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-7203) Add container ExecutionType into ContainerReport
[ https://issues.apache.org/jira/browse/YARN-7203?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Botong Huang updated YARN-7203: --- Attachment: YARN-7203.v4.patch Addressing the comments and add unit test in v4 patch. > Add container ExecutionType into ContainerReport > > > Key: YARN-7203 > URL: https://issues.apache.org/jira/browse/YARN-7203 > Project: Hadoop YARN > Issue Type: Task >Reporter: Botong Huang >Assignee: Botong Huang >Priority: Minor > Attachments: YARN-7203.v1.patch, YARN-7203.v2.patch, > YARN-7203.v3.patch, YARN-7203.v4.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7199) Fix TestAMRMClientContainerRequest.testOpportunisticAndGuaranteedRequests
[ https://issues.apache.org/jira/browse/YARN-7199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16170489#comment-16170489 ] Hudson commented on YARN-7199: -- SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #12899 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/12899/]) YARN-7199. Fix (arun suresh: rev 29dd55153e37471d9c177f4bd173f1d02bc96410) * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/main/java/org/apache/hadoop/yarn/client/api/AMRMClient.java > Fix TestAMRMClientContainerRequest.testOpportunisticAndGuaranteedRequests > - > > Key: YARN-7199 > URL: https://issues.apache.org/jira/browse/YARN-7199 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Botong Huang >Assignee: Botong Huang >Priority: Blocker > Fix For: 3.1.0 > > Attachments: YARN-7199.v1.patch, YARN-7199.v2.patch, > YARN-7199.v3.patch > > > java.lang.IllegalArgumentException: The profile name cannot be null > at > com.google.common.base.Preconditions.checkArgument(Preconditions.java:88) > at > org.apache.hadoop.yarn.api.records.ProfileCapability.newInstance(ProfileCapability.java:68) > at > org.apache.hadoop.yarn.client.api.impl.AMRMClientImpl.addContainerRequest(AMRMClientImpl.java:512) > at > org.apache.hadoop.yarn.client.api.impl.TestAMRMClientContainerRequest.testOpportunisticAndGuaranteedRequests(TestAMRMClientContainerRequest.java:59) -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7203) Add container ExecutionType into ContainerReport
[
https://issues.apache.org/jira/browse/YARN-7203?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16170473#comment-16170473
]
Arun Suresh commented on YARN-7203:
---
Actually, [~botong], just realized that we havn't added any new tests. Can you
update {{TestYarnClient}} ?
> Add container ExecutionType into ContainerReport
>
>
> Key: YARN-7203
> URL: https://issues.apache.org/jira/browse/YARN-7203
> Project: Hadoop YARN
> Issue Type: Task
>Reporter: Botong Huang
>Assignee: Botong Huang
>Priority: Minor
> Attachments: YARN-7203.v1.patch, YARN-7203.v2.patch,
> YARN-7203.v3.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7199) Fix TestAMRMClientContainerRequest.testOpportunisticAndGuaranteedRequests
[ https://issues.apache.org/jira/browse/YARN-7199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16170456#comment-16170456 ] Botong Huang commented on YARN-7199: Cool, thanks [~asuresh] and [~wangda] for the review! > Fix TestAMRMClientContainerRequest.testOpportunisticAndGuaranteedRequests > - > > Key: YARN-7199 > URL: https://issues.apache.org/jira/browse/YARN-7199 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Botong Huang >Assignee: Botong Huang >Priority: Blocker > Fix For: 3.1.0 > > Attachments: YARN-7199.v1.patch, YARN-7199.v2.patch, > YARN-7199.v3.patch > > > java.lang.IllegalArgumentException: The profile name cannot be null > at > com.google.common.base.Preconditions.checkArgument(Preconditions.java:88) > at > org.apache.hadoop.yarn.api.records.ProfileCapability.newInstance(ProfileCapability.java:68) > at > org.apache.hadoop.yarn.client.api.impl.AMRMClientImpl.addContainerRequest(AMRMClientImpl.java:512) > at > org.apache.hadoop.yarn.client.api.impl.TestAMRMClientContainerRequest.testOpportunisticAndGuaranteedRequests(TestAMRMClientContainerRequest.java:59) -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-7199) Fix TestAMRMClientContainerRequest.testOpportunisticAndGuaranteedRequests
[ https://issues.apache.org/jira/browse/YARN-7199?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun Suresh updated YARN-7199: -- Fix Version/s: 3.1.0 > Fix TestAMRMClientContainerRequest.testOpportunisticAndGuaranteedRequests > - > > Key: YARN-7199 > URL: https://issues.apache.org/jira/browse/YARN-7199 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Botong Huang >Assignee: Botong Huang >Priority: Blocker > Fix For: 3.1.0 > > Attachments: YARN-7199.v1.patch, YARN-7199.v2.patch, > YARN-7199.v3.patch > > > java.lang.IllegalArgumentException: The profile name cannot be null > at > com.google.common.base.Preconditions.checkArgument(Preconditions.java:88) > at > org.apache.hadoop.yarn.api.records.ProfileCapability.newInstance(ProfileCapability.java:68) > at > org.apache.hadoop.yarn.client.api.impl.AMRMClientImpl.addContainerRequest(AMRMClientImpl.java:512) > at > org.apache.hadoop.yarn.client.api.impl.TestAMRMClientContainerRequest.testOpportunisticAndGuaranteedRequests(TestAMRMClientContainerRequest.java:59) -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-7199) Fix TestAMRMClientContainerRequest.testOpportunisticAndGuaranteedRequests
[ https://issues.apache.org/jira/browse/YARN-7199?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun Suresh updated YARN-7199: -- Summary: Fix TestAMRMClientContainerRequest.testOpportunisticAndGuaranteedRequests (was: TestAMRMClientContainerRequest.testOpportunisticAndGuaranteedRequests is failing in trunk) > Fix TestAMRMClientContainerRequest.testOpportunisticAndGuaranteedRequests > - > > Key: YARN-7199 > URL: https://issues.apache.org/jira/browse/YARN-7199 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Botong Huang >Assignee: Botong Huang >Priority: Blocker > Attachments: YARN-7199.v1.patch, YARN-7199.v2.patch, > YARN-7199.v3.patch > > > java.lang.IllegalArgumentException: The profile name cannot be null > at > com.google.common.base.Preconditions.checkArgument(Preconditions.java:88) > at > org.apache.hadoop.yarn.api.records.ProfileCapability.newInstance(ProfileCapability.java:68) > at > org.apache.hadoop.yarn.client.api.impl.AMRMClientImpl.addContainerRequest(AMRMClientImpl.java:512) > at > org.apache.hadoop.yarn.client.api.impl.TestAMRMClientContainerRequest.testOpportunisticAndGuaranteedRequests(TestAMRMClientContainerRequest.java:59) -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7199) TestAMRMClientContainerRequest.testOpportunisticAndGuaranteedRequests is failing in trunk
[ https://issues.apache.org/jira/browse/YARN-7199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16170423#comment-16170423 ] Arun Suresh commented on YARN-7199: --- +1 > TestAMRMClientContainerRequest.testOpportunisticAndGuaranteedRequests is > failing in trunk > - > > Key: YARN-7199 > URL: https://issues.apache.org/jira/browse/YARN-7199 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Botong Huang >Assignee: Botong Huang >Priority: Blocker > Attachments: YARN-7199.v1.patch, YARN-7199.v2.patch, > YARN-7199.v3.patch > > > java.lang.IllegalArgumentException: The profile name cannot be null > at > com.google.common.base.Preconditions.checkArgument(Preconditions.java:88) > at > org.apache.hadoop.yarn.api.records.ProfileCapability.newInstance(ProfileCapability.java:68) > at > org.apache.hadoop.yarn.client.api.impl.AMRMClientImpl.addContainerRequest(AMRMClientImpl.java:512) > at > org.apache.hadoop.yarn.client.api.impl.TestAMRMClientContainerRequest.testOpportunisticAndGuaranteedRequests(TestAMRMClientContainerRequest.java:59) -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7203) Add container ExecutionType into ContainerReport
[ https://issues.apache.org/jira/browse/YARN-7203?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16170425#comment-16170425 ] Arun Suresh commented on YARN-7203: --- +1, will commit this shortly > Add container ExecutionType into ContainerReport > > > Key: YARN-7203 > URL: https://issues.apache.org/jira/browse/YARN-7203 > Project: Hadoop YARN > Issue Type: Task >Reporter: Botong Huang >Assignee: Botong Huang >Priority: Minor > Attachments: YARN-7203.v1.patch, YARN-7203.v2.patch, > YARN-7203.v3.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7162) Remove XML excludes file format
[ https://issues.apache.org/jira/browse/YARN-7162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16170415#comment-16170415 ] Hudson commented on YARN-7162: -- SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #12898 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/12898/]) Revert "YARN-7162. Remove XML excludes file format (rkanter)" - wrong (rkanter: rev 5f496683fb00ba26a6bf5a506ae87d4bc4088727) * (edit) hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/resources/mapred-default.xml * (edit) hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/JobResourceUploader.java * (edit) hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/MRJobConfig.java * (edit) hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/pom.xml * (edit) hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapreduce/TestJobResourceUploader.java > Remove XML excludes file format > --- > > Key: YARN-7162 > URL: https://issues.apache.org/jira/browse/YARN-7162 > Project: Hadoop YARN > Issue Type: Sub-task > Components: graceful >Affects Versions: 2.9.0, 3.0.0-beta1 >Reporter: Robert Kanter >Assignee: Robert Kanter >Priority: Blocker > Fix For: 2.9.0, 3.0.0-beta1 > > Attachments: YARN-7162.001.patch, YARN-7162.branch-2.001.patch > > > YARN-5536 aims to replace the XML format for the excludes file with a JSON > format. However, it looks like we won't have time for that for Hadoop 3 Beta > 1. The concern is that if we release it as-is, we'll now have to support the > XML format as-is for all of Hadoop 3.x, which we're either planning on > removing, or rewriting using a pluggable framework. > [This comment in > YARN-5536|https://issues.apache.org/jira/browse/YARN-5536?focusedCommentId=16126194&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16126194] > proposed two quick solutions to prevent this compat issue. In this JIRA, > we're going to remove the XML format. If we later want to add it back in, > YARN-5536 can add it back, rewriting it to be in the pluggable framework. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7199) TestAMRMClientContainerRequest.testOpportunisticAndGuaranteedRequests is failing in trunk
[ https://issues.apache.org/jira/browse/YARN-7199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16170417#comment-16170417 ] Wangda Tan commented on YARN-7199: -- +1 to latest patch, [~asuresh] / [~jianhe], could you review the latest patch? > TestAMRMClientContainerRequest.testOpportunisticAndGuaranteedRequests is > failing in trunk > - > > Key: YARN-7199 > URL: https://issues.apache.org/jira/browse/YARN-7199 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Botong Huang >Assignee: Botong Huang >Priority: Blocker > Attachments: YARN-7199.v1.patch, YARN-7199.v2.patch, > YARN-7199.v3.patch > > > java.lang.IllegalArgumentException: The profile name cannot be null > at > com.google.common.base.Preconditions.checkArgument(Preconditions.java:88) > at > org.apache.hadoop.yarn.api.records.ProfileCapability.newInstance(ProfileCapability.java:68) > at > org.apache.hadoop.yarn.client.api.impl.AMRMClientImpl.addContainerRequest(AMRMClientImpl.java:512) > at > org.apache.hadoop.yarn.client.api.impl.TestAMRMClientContainerRequest.testOpportunisticAndGuaranteedRequests(TestAMRMClientContainerRequest.java:59) -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7196) Fix finicky TestContainerManager tests
[
https://issues.apache.org/jira/browse/YARN-7196?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16170410#comment-16170410
]
Arun Suresh commented on YARN-7196:
---
I noticed that {{testPromotionOfOpportunisticContainers}} does almost exactly
what {{testContainerUpdateExecTypeOpportunisticToGuaranteed}} does, barring an
extra Assert. So I moved the assert over to
{{testPromotionOfOpportunisticContainers}} and removed that test.
> Fix finicky TestContainerManager tests
> --
>
> Key: YARN-7196
> URL: https://issues.apache.org/jira/browse/YARN-7196
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Arun Suresh
>Assignee: Arun Suresh
> Attachments: YARN-7196.002.patch, YARN-7196.patch
>
>
> The Testcase {{testContainerUpdateExecTypeGuaranteedToOpportunistic}} seem to
> fail every once in a while. Maybe have to change the way the event is
> triggered.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-2664) Improve RM webapp to expose info about reservations.
[ https://issues.apache.org/jira/browse/YARN-2664?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Íñigo Goiri updated YARN-2664: -- Attachment: YARN-2664.22.patch Fixed an issue with the JSON generation which was escaping for HTML. > Improve RM webapp to expose info about reservations. > > > Key: YARN-2664 > URL: https://issues.apache.org/jira/browse/YARN-2664 > Project: Hadoop YARN > Issue Type: Sub-task > Components: resourcemanager >Reporter: Carlo Curino >Assignee: Íñigo Goiri > Labels: BB2015-05-TBR > Attachments: legal.patch, PlannerPage_screenshot.pdf, > screenshot_reservation_UI.pdf, YARN-2664.10.patch, YARN-2664.11.patch, > YARN-2664.12.patch, YARN-2664.13.patch, YARN-2664.14.patch, > YARN-2664.15.patch, YARN-2664.16.patch, YARN-2664.17.patch, > YARN-2664.18.patch, YARN-2664.19.patch, YARN-2664.1.patch, > YARN-2664.20.patch, YARN-2664.21.patch, YARN-2664.22.patch, > YARN-2664.2.patch, YARN-2664.3.patch, YARN-2664.4.patch, YARN-2664.5.patch, > YARN-2664.6.patch, YARN-2664.7.patch, YARN-2664.8.patch, YARN-2664.9.patch, > YARN-2664.patch > > > YARN-1051 provides a new functionality in the RM to ask for reservation on > resources. Exposing this through the webapp GUI is important. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-7196) Fix finicky TestContainerManager tests
[
https://issues.apache.org/jira/browse/YARN-7196?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Arun Suresh updated YARN-7196:
--
Attachment: YARN-7196.002.patch
Updating patch.
> Fix finicky TestContainerManager tests
> --
>
> Key: YARN-7196
> URL: https://issues.apache.org/jira/browse/YARN-7196
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Arun Suresh
>Assignee: Arun Suresh
> Attachments: YARN-7196.002.patch, YARN-7196.patch
>
>
> The Testcase {{testContainerUpdateExecTypeGuaranteedToOpportunistic}} seem to
> fail every once in a while. Maybe have to change the way the event is
> triggered.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-7201) Add more sophisticated example YARN service
[ https://issues.apache.org/jira/browse/YARN-7201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Yang updated YARN-7201: Description: We can show case the following capabilities in the YARN service examples: # Description of the service # Component dependencies # How to mount HDFS volume via NFS Gateway # Enable privileged container # Quick link to web application # Queue to submit # Use private docker registry was: We can show case the following capabilities in the YARN service examples: # Description of the service # Component dependencies # How to mount HDFS volume via NFS Gateway # Enable privileged container # Quick link to web application # Queue to submit > Add more sophisticated example YARN service > --- > > Key: YARN-7201 > URL: https://issues.apache.org/jira/browse/YARN-7201 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Jian He > > We can show case the following capabilities in the YARN service examples: > # Description of the service > # Component dependencies > # How to mount HDFS volume via NFS Gateway > # Enable privileged container > # Quick link to web application > # Queue to submit > # Use private docker registry -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Assigned] (YARN-7196) Fix finicky TestContainerManager tests
[
https://issues.apache.org/jira/browse/YARN-7196?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Arun Suresh reassigned YARN-7196:
-
Assignee: Arun Suresh
> Fix finicky TestContainerManager tests
> --
>
> Key: YARN-7196
> URL: https://issues.apache.org/jira/browse/YARN-7196
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Arun Suresh
>Assignee: Arun Suresh
> Attachments: YARN-7196.patch
>
>
> The Testcase {{testContainerUpdateExecTypeGuaranteedToOpportunistic}} seem to
> fail every once in a while. Maybe have to change the way the event is
> triggered.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7210) Some fixes related to Registry DNS
[ https://issues.apache.org/jira/browse/YARN-7210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16170398#comment-16170398 ] Jian He commented on YARN-7210: --- - In componentInstance, it should get a single ip for constructing the ServiceRecord - In ApplicationServiceRecordProcessor, if no external endpoints defined, skip creating the application-level service records. - In BaseServiceRecordProcessor, rename "classpath:org.apache.slider." to "classpath:org.apache.hadoop.yarn.service." - In DockerLinuxContainerRuntime, there's a bug, the quote strip should happen before the 'lastIndexOf' call, otherwise, the lastIndex is not correct and that caused the parsed ip and host incorrect. > Some fixes related to Registry DNS > -- > > Key: YARN-7210 > URL: https://issues.apache.org/jira/browse/YARN-7210 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Jian He >Assignee: Jian He > Attachments: YARN-7210.yarn-native-services.01.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-7201) Add more sophisticated example YARN service
[ https://issues.apache.org/jira/browse/YARN-7201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Yang updated YARN-7201: Description: We can show case the following capabilities in the YARN service examples: # Description of the service # Component dependencies # How to mount HDFS volume via NFS Gateway # Enable privileged container # Quick link to web application # Queue to submit was: We can show case the following capabilities in the YARN service examples: # Description of the service # Component dependencies # How to mount HDFS volume via NFS Gateway # Enforce non-privileged container # Enable privileged container # Quick link to web application # Queue to submit > Add more sophisticated example YARN service > --- > > Key: YARN-7201 > URL: https://issues.apache.org/jira/browse/YARN-7201 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Jian He > > We can show case the following capabilities in the YARN service examples: > # Description of the service > # Component dependencies > # How to mount HDFS volume via NFS Gateway > # Enable privileged container > # Quick link to web application > # Queue to submit -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-7201) Add more sophisticated example YARN service
[ https://issues.apache.org/jira/browse/YARN-7201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Yang updated YARN-7201: Description: We can show case the following capabilities in the YARN service examples: # Description of the service # Component dependencies # How to mount HDFS volume via NFS Gateway # Enforce non-privileged container # Enable privileged container # Quick link to web application # Queue to submit > Add more sophisticated example YARN service > --- > > Key: YARN-7201 > URL: https://issues.apache.org/jira/browse/YARN-7201 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Jian He > > We can show case the following capabilities in the YARN service examples: > # Description of the service > # Component dependencies > # How to mount HDFS volume via NFS Gateway > # Enforce non-privileged container > # Enable privileged container > # Quick link to web application > # Queue to submit -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-7210) Some fixes related to Registry DNS
[ https://issues.apache.org/jira/browse/YARN-7210?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jian He updated YARN-7210: -- Attachment: YARN-7210.yarn-native-services.01.patch > Some fixes related to Registry DNS > -- > > Key: YARN-7210 > URL: https://issues.apache.org/jira/browse/YARN-7210 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Jian He >Assignee: Jian He > Attachments: YARN-7210.yarn-native-services.01.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-7210) Some fixes related to Registry DNS
Jian He created YARN-7210: - Summary: Some fixes related to Registry DNS Key: YARN-7210 URL: https://issues.apache.org/jira/browse/YARN-7210 Project: Hadoop YARN Issue Type: Sub-task Reporter: Jian He Assignee: Jian He -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6840) Implement zookeeper based store for scheduler configuration updates
[ https://issues.apache.org/jira/browse/YARN-6840?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16170354#comment-16170354 ] Jonathan Hung commented on YARN-6840: - Thanks all! > Implement zookeeper based store for scheduler configuration updates > --- > > Key: YARN-6840 > URL: https://issues.apache.org/jira/browse/YARN-6840 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Wangda Tan >Assignee: Jonathan Hung > Fix For: YARN-5734 > > Attachments: YARN-6840-YARN-5734.001.patch, > YARN-6840-YARN-5734.002.patch, YARN-6840-YARN-5734.003.patch, > YARN-6840-YARN-5734.004.patch, YARN-6840-YARN-5734.005.patch, > YARN-6840-YARN-5734.006.patch, YARN-6840-YARN-5734.007.patch > > > Right now there is only in-memory and leveldb based configuration store > supported. Need one which supports RM HA. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7196) Fix finicky TestContainerManager tests
[
https://issues.apache.org/jira/browse/YARN-7196?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16170332#comment-16170332
]
Junping Du commented on YARN-7196:
--
Hi [~asuresh], sure. please take it up. :) Sorry for missing your earlier
comments.
> Fix finicky TestContainerManager tests
> --
>
> Key: YARN-7196
> URL: https://issues.apache.org/jira/browse/YARN-7196
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Arun Suresh
> Attachments: YARN-7196.patch
>
>
> The Testcase {{testContainerUpdateExecTypeGuaranteedToOpportunistic}} seem to
> fail every once in a while. Maybe have to change the way the event is
> triggered.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7196) Fix finicky TestContainerManager tests
[
https://issues.apache.org/jira/browse/YARN-7196?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16170313#comment-16170313
]
Arun Suresh commented on YARN-7196:
---
[~djp], do you mind if I take this up ? are you fine with
[this|https://issues.apache.org/jira/browse/YARN-7196?focusedCommentId=16168638&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16168638]
?
> Fix finicky TestContainerManager tests
> --
>
> Key: YARN-7196
> URL: https://issues.apache.org/jira/browse/YARN-7196
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Arun Suresh
> Attachments: YARN-7196.patch
>
>
> The Testcase {{testContainerUpdateExecTypeGuaranteedToOpportunistic}} seem to
> fail every once in a while. Maybe have to change the way the event is
> triggered.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6620) [YARN-6223] NM Java side code changes to support isolate GPU devices by using CGroups
[ https://issues.apache.org/jira/browse/YARN-6620?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16170294#comment-16170294 ] Wangda Tan commented on YARN-6620: -- [~tangzhankun], Good point, first of all, to me the requirement: bq. ... one physical machine with two different vendor GPU cards .. Is not a real requirement, we may need to spend 20% of the effort to support the 1% requirement. And even if it is true, as you commented, admin may need to register two plugins and handle isolation part separately. I can see some minor code changes needed (For example, make the resource-name inside resource-vector can be configurable, by default it is gpu, and admin can change it to gpu-vendor-a and gpu-vendor-b, but I would prefer to do this once the requirement comes. > [YARN-6223] NM Java side code changes to support isolate GPU devices by using > CGroups > - > > Key: YARN-6620 > URL: https://issues.apache.org/jira/browse/YARN-6620 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Wangda Tan >Assignee: Wangda Tan > Attachments: YARN-6620.001.patch, YARN-6620.002.patch, > YARN-6620.003.patch, YARN-6620.004.patch, YARN-6620.005.patch, > YARN-6620.006-WIP.patch > > > This JIRA plan to add support of: > 1) GPU configuration for NodeManagers > 2) Isolation in CGroups. (Java side). > 3) NM restart and recovery allocated GPU devices -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6623) Add support to turn off launching privileged containers in the container-executor
[ https://issues.apache.org/jira/browse/YARN-6623?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16170280#comment-16170280 ] Varun Vasudev commented on YARN-6623: - Thanks for the feedback [~miklos.szeg...@cloudera.com], [~shaneku...@gmail.com]. I should have a patch addressing you reviews soon. [~miklos.szeg...@cloudera.com] bq. I have some comments but before that, does the jira target 3.0-beta1? I would like to get it into beta1. > Add support to turn off launching privileged containers in the > container-executor > - > > Key: YARN-6623 > URL: https://issues.apache.org/jira/browse/YARN-6623 > Project: Hadoop YARN > Issue Type: Sub-task > Components: nodemanager >Reporter: Varun Vasudev >Assignee: Varun Vasudev >Priority: Blocker > Attachments: YARN-6623.001.patch, YARN-6623.002.patch, > YARN-6623.003.patch, YARN-6623.004.patch, YARN-6623.005.patch, > YARN-6623.006.patch, YARN-6623.007.patch, YARN-6623.008.patch, > YARN-6623.009.patch > > > Currently, launching privileged containers is controlled by the NM. We should > add a flag to the container-executor.cfg allowing admins to disable launching > privileged containers at the container-executor level. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7193) Implement REST API for register Yarnfile in application catalog
[ https://issues.apache.org/jira/browse/YARN-7193?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16170274#comment-16170274 ] Eric Yang commented on YARN-7193: - This modifies dev-support/docker/Dockerfile to include a more recent version of nodejs, and apidoc for documenting REST API. However, JDK 1.7 seems to have disappeared from Oracle website. This broke the pre-commit test for patch 005. Do we need to keep dev-support/docker/Dockerfile and patchprocess/yetus-0.4.0-lib/precommit/test-patch-docker/Dockerfile in sync? Nodejs required dependencies are not described in yetus version of Dockerfile. > Implement REST API for register Yarnfile in application catalog > --- > > Key: YARN-7193 > URL: https://issues.apache.org/jira/browse/YARN-7193 > Project: Hadoop YARN > Issue Type: Sub-task > Components: applications >Affects Versions: 3.1.0 >Reporter: Eric Yang >Assignee: Eric Yang > Attachments: YARN-7193.yarn-native-services.001.patch, > YARN-7193.yarn-native-services.002.patch, > YARN-7193.yarn-native-services.003.patch, > YARN-7193.yarn-native-services.004.patch, > YARN-7193.yarn-native-services.005.patch > > > For support ability to register and index Yarnfile, we need a set of REST API > to register, search, and recommend applications from application catalog. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6771) Use classloader inside configuration class to make new classes
[
https://issues.apache.org/jira/browse/YARN-6771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16170261#comment-16170261
]
Hadoop QA commented on YARN-6771:
-
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
40s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 13m
22s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
33s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
25s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
38s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
11s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
37s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
31s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
28s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
28s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
21s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
32s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
16s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
33s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
28s{color} | {color:green} hadoop-yarn-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 25m 11s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:71bbb86 |
| JIRA Issue | YARN-6771 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12887688/YARN-6771-6.patch |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux 914d4a3a2575 3.13.0-129-generic #178-Ubuntu SMP Fri Aug 11
12:48:20 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / a4f9c7c |
| Default Java | 1.8.0_144 |
| findbugs | v3.1.0-RC1 |
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/17498/testReport/ |
| modules | C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common U:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common |
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/17498/console |
| Powered by | Apache Yetus 0.6.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> Use classloader inside configuration class to make new classes
> ---
>
> Key: YARN-6771
> URL: https://issues.apache.org/jira/browse/YARN-6771
> Project: Hadoop YARN
> Issue Type: Bug
>Affects Versions: 2.8.1, 3.0.0-alpha4
>Reporter: Jongyoul Lee
> Attachments: YARN-6771-1.patch, YARN-6771-2.patch, YARN-6771-3.patch,
> YARN-6771-4.patch, YARN-6771-5.patch, YARN-6771-6.patch, YARN-6771.patch
>
>
> While running {{RpcClientFactoryPBImpl.getClient}},
> {{RpcClientFactoryPBImpl}} uses {{localConf.getCl
[jira] [Commented] (YARN-6840) Implement zookeeper based store for scheduler configuration updates
[ https://issues.apache.org/jira/browse/YARN-6840?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16170223#comment-16170223 ] Wangda Tan commented on YARN-6840: -- +1, will commit today. Thanks [~jhung] > Implement zookeeper based store for scheduler configuration updates > --- > > Key: YARN-6840 > URL: https://issues.apache.org/jira/browse/YARN-6840 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Wangda Tan >Assignee: Jonathan Hung > Attachments: YARN-6840-YARN-5734.001.patch, > YARN-6840-YARN-5734.002.patch, YARN-6840-YARN-5734.003.patch, > YARN-6840-YARN-5734.004.patch, YARN-6840-YARN-5734.005.patch, > YARN-6840-YARN-5734.006.patch, YARN-6840-YARN-5734.007.patch > > > Right now there is only in-memory and leveldb based configuration store > supported. Need one which supports RM HA. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-6771) Use classloader inside configuration class to make new classes
[
https://issues.apache.org/jira/browse/YARN-6771?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jongyoul Lee updated YARN-6771:
---
Attachment: YARN-6771-6.patch
> Use classloader inside configuration class to make new classes
> ---
>
> Key: YARN-6771
> URL: https://issues.apache.org/jira/browse/YARN-6771
> Project: Hadoop YARN
> Issue Type: Bug
>Affects Versions: 2.8.1, 3.0.0-alpha4
>Reporter: Jongyoul Lee
> Attachments: YARN-6771-1.patch, YARN-6771-2.patch, YARN-6771-3.patch,
> YARN-6771-4.patch, YARN-6771-5.patch, YARN-6771-6.patch, YARN-6771.patch
>
>
> While running {{RpcClientFactoryPBImpl.getClient}},
> {{RpcClientFactoryPBImpl}} uses {{localConf.getClassByName}}. But in case of
> using custom classloader, we have to use {{conf.getClassByName}} because
> custom classloader is already stored in {{Configuration}} class.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6771) Use classloader inside configuration class to make new classes
[
https://issues.apache.org/jira/browse/YARN-6771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16170206#comment-16170206
]
Hadoop QA commented on YARN-6771:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
0s{color} | {color:blue} Docker mode activated. {color} |
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 5s{color}
| {color:red} YARN-6771 does not apply to trunk. Rebase required? Wrong Branch?
See https://wiki.apache.org/hadoop/HowToContribute for help. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | YARN-6771 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12887686/YARN-6771-5.patch |
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/17497/console |
| Powered by | Apache Yetus 0.6.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> Use classloader inside configuration class to make new classes
> ---
>
> Key: YARN-6771
> URL: https://issues.apache.org/jira/browse/YARN-6771
> Project: Hadoop YARN
> Issue Type: Bug
>Affects Versions: 2.8.1, 3.0.0-alpha4
>Reporter: Jongyoul Lee
> Attachments: YARN-6771-1.patch, YARN-6771-2.patch, YARN-6771-3.patch,
> YARN-6771-4.patch, YARN-6771-5.patch, YARN-6771.patch
>
>
> While running {{RpcClientFactoryPBImpl.getClient}},
> {{RpcClientFactoryPBImpl}} uses {{localConf.getClassByName}}. But in case of
> using custom classloader, we have to use {{conf.getClassByName}} because
> custom classloader is already stored in {{Configuration}} class.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-6771) Use classloader inside configuration class to make new classes
[
https://issues.apache.org/jira/browse/YARN-6771?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jongyoul Lee updated YARN-6771:
---
Attachment: YARN-6771-5.patch
> Use classloader inside configuration class to make new classes
> ---
>
> Key: YARN-6771
> URL: https://issues.apache.org/jira/browse/YARN-6771
> Project: Hadoop YARN
> Issue Type: Bug
>Affects Versions: 2.8.1, 3.0.0-alpha4
>Reporter: Jongyoul Lee
> Attachments: YARN-6771-1.patch, YARN-6771-2.patch, YARN-6771-3.patch,
> YARN-6771-4.patch, YARN-6771-5.patch, YARN-6771.patch
>
>
> While running {{RpcClientFactoryPBImpl.getClient}},
> {{RpcClientFactoryPBImpl}} uses {{localConf.getClassByName}}. But in case of
> using custom classloader, we have to use {{conf.getClassByName}} because
> custom classloader is already stored in {{Configuration}} class.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-6626) Embed REST API service into RM
[ https://issues.apache.org/jira/browse/YARN-6626?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Yang updated YARN-6626: Attachment: (was: YARN-6626.yarn-native-services.001.patch) > Embed REST API service into RM > -- > > Key: YARN-6626 > URL: https://issues.apache.org/jira/browse/YARN-6626 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Gour Saha > Fix For: yarn-native-services > > Attachments: YARN-6626.yarn-native-services.001.patch > > > As of now the deployment model of the Native Services REST API service is > standalone. There are several cross-cutting solutions that can be inherited > for free (kerberos, HA, ACLs, trusted proxy support, etc.) by the REST API > service if it is embedded into the RM process. In fact we can expose the REST > API via the same port as RM UI (8088 default). The URI path > /services/v1/applications will distinguish the REST API calls from other RM > APIs. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-6626) Embed REST API service into RM
[ https://issues.apache.org/jira/browse/YARN-6626?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Yang updated YARN-6626: Attachment: YARN-6626.yarn-native-services.001.patch > Embed REST API service into RM > -- > > Key: YARN-6626 > URL: https://issues.apache.org/jira/browse/YARN-6626 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Gour Saha > Fix For: yarn-native-services > > Attachments: YARN-6626.yarn-native-services.001.patch > > > As of now the deployment model of the Native Services REST API service is > standalone. There are several cross-cutting solutions that can be inherited > for free (kerberos, HA, ACLs, trusted proxy support, etc.) by the REST API > service if it is embedded into the RM process. In fact we can expose the REST > API via the same port as RM UI (8088 default). The URI path > /services/v1/applications will distinguish the REST API calls from other RM > APIs. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-6771) Use classloader inside configuration class to make new classes
[
https://issues.apache.org/jira/browse/YARN-6771?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jongyoul Lee updated YARN-6771:
---
Attachment: YARN-6771-4.patch
> Use classloader inside configuration class to make new classes
> ---
>
> Key: YARN-6771
> URL: https://issues.apache.org/jira/browse/YARN-6771
> Project: Hadoop YARN
> Issue Type: Bug
>Affects Versions: 2.8.1, 3.0.0-alpha4
>Reporter: Jongyoul Lee
> Attachments: YARN-6771-1.patch, YARN-6771-2.patch, YARN-6771-3.patch,
> YARN-6771-4.patch, YARN-6771.patch
>
>
> While running {{RpcClientFactoryPBImpl.getClient}},
> {{RpcClientFactoryPBImpl}} uses {{localConf.getClassByName}}. But in case of
> using custom classloader, we have to use {{conf.getClassByName}} because
> custom classloader is already stored in {{Configuration}} class.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7192) Add a pluggable StateMachine Listener that is notified of NM Container State changes
[ https://issues.apache.org/jira/browse/YARN-7192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16170165#comment-16170165 ] Hudson commented on YARN-7192: -- SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #12896 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/12896/]) YARN-7192. Add a pluggable StateMachine Listener that is notified of NM (jlowe: rev a4f9c7c9247801dd37beec6fc195622af1b884ad) * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/Context.java * (add) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/state/StateTransitionListener.java * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/amrmproxy/BaseAMRMProxyTest.java * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/NodeManager.java * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/yarn-default.xml * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/state/StateMachineFactory.java * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/TestNodeManager.java * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/container/ContainerImpl.java * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/container/TestContainer.java * (add) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/state/MultiStateTransitionListener.java * (add) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/ContainerStateTransitionListener.java > Add a pluggable StateMachine Listener that is notified of NM Container State > changes > > > Key: YARN-7192 > URL: https://issues.apache.org/jira/browse/YARN-7192 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Arun Suresh >Assignee: Arun Suresh > Fix For: 2.9.0, 3.0.0-beta1 > > Attachments: YARN-7192.001.patch, YARN-7192.002.patch, > YARN-7192.003.patch, YARN-7192.004.patch, YARN-7192.005.patch, > YARN-7192.006.patch > > > This JIRA is to add support for a plugggable class in the NodeManager that is > notified of changes to the Container StateMachine state and the events that > caused the change. > The proposal is to modify the basic StateMachine class add support for a hook > that is called before and after a transition. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7149) Cross-queue preemption sometimes starves an underserved queue
[
https://issues.apache.org/jira/browse/YARN-7149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16170157#comment-16170157
]
Eric Payne commented on YARN-7149:
--
Thanks a lot [~leftnoteasy].
Also, this needs to be pulled back into branch-2. I will do that if there are
no objections.
> Cross-queue preemption sometimes starves an underserved queue
> -
>
> Key: YARN-7149
> URL: https://issues.apache.org/jira/browse/YARN-7149
> Project: Hadoop YARN
> Issue Type: Bug
> Components: capacity scheduler
>Affects Versions: 2.9.0, 3.0.0-alpha3
>Reporter: Eric Payne
>Assignee: Eric Payne
> Fix For: 3.0.0-beta1
>
> Attachments: YARN-7149.001.patch, YARN-7149.002.patch,
> YARN-7149.demo.unit-test.patch
>
>
> In branch 2 and trunk, I am consistently seeing some use cases where
> cross-queue preemption does not happen when it should. I do not see this in
> branch-2.8.
> Use Case:
> | | *Size* | *Minimum Container Size* |
> |MyCluster | 20 GB | 0.5 GB |
> | *Queue Name* | *Capacity* | *Absolute Capacity* | *Minimum User Limit
> Percent (MULP)* | *User Limit Factor (ULF)* |
> |Q1 | 50% = 10 GB | 100% = 20 GB | 10% = 1 GB | 2.0 |
> |Q2 | 50% = 10 GB | 100% = 20 GB | 10% = 1 GB | 2.0 |
> - {{User1}} launches {{App1}} in {{Q1}} and consumes all resources (20 GB)
> - {{User2}} launches {{App2}} in {{Q2}} and requests 10 GB
> - _Note: containers are 0.5 GB._
> - Preemption monitor kills 2 containers (equals 1 GB) from {{App1}} in {{Q1}}.
> - Capacity Scheduler assigns 2 containers (equals 1 GB) to {{App2}} in {{Q2}}.
> - _No more containers are ever preempted, even though {{Q2}} is far
> underserved_
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7192) Add a pluggable StateMachine Listener that is notified of NM Container State changes
[ https://issues.apache.org/jira/browse/YARN-7192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16170141#comment-16170141 ] Jason Lowe commented on YARN-7192: -- Thanks for updating the patch! The test failure is unrelated and tracked by YARN-7196. +1 lgtm. Committing this. > Add a pluggable StateMachine Listener that is notified of NM Container State > changes > > > Key: YARN-7192 > URL: https://issues.apache.org/jira/browse/YARN-7192 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Arun Suresh >Assignee: Arun Suresh > Attachments: YARN-7192.001.patch, YARN-7192.002.patch, > YARN-7192.003.patch, YARN-7192.004.patch, YARN-7192.005.patch, > YARN-7192.006.patch > > > This JIRA is to add support for a plugggable class in the NodeManager that is > notified of changes to the Container StateMachine state and the events that > caused the change. > The proposal is to modify the basic StateMachine class add support for a hook > that is called before and after a transition. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6505) Define the strings used in SLS JSON input file format
[
https://issues.apache.org/jira/browse/YARN-6505?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16170076#comment-16170076
]
Hadoop QA commented on YARN-6505:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
17s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 13m
41s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
17s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
13s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
21s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
30s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
16s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
15s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
15s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
10s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
37s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
11s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 1m 30s{color}
| {color:red} hadoop-sls in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
15s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 20m 27s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.yarn.sls.TestSLSRunner |
| | hadoop.yarn.sls.TestReservationSystemInvariants |
| | hadoop.yarn.sls.nodemanager.TestNMSimulator |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:71bbb86 |
| JIRA Issue | YARN-6505 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12887659/YARN-6505.005.patch |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux c0087897cdef 3.13.0-129-generic #178-Ubuntu SMP Fri Aug 11
12:48:20 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / 0f9af24 |
| Default Java | 1.8.0_144 |
| findbugs | v3.1.0-RC1 |
| unit |
https://builds.apache.org/job/PreCommit-YARN-Build/17496/artifact/patchprocess/patch-unit-hadoop-tools_hadoop-sls.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/17496/testReport/ |
| modules | C: hadoop-tools/hadoop-sls U: hadoop-tools/hadoop-sls |
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/17496/console |
| Powered by | Apache Yetus 0.6.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> Define the strings used in SLS JSON input file format
> -
>
> Key: YARN-6505
> URL: https://issues.apache.org/jira/browse/YARN-6505
> Project: Hadoop YARN
> Is
[jira] [Assigned] (YARN-7133) Clean up lock-try order in fair scheduler
[
https://issues.apache.org/jira/browse/YARN-7133?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Daniel Templeton reassigned YARN-7133:
--
Assignee: Gergo Repas (was: Daniel Templeton)
> Clean up lock-try order in fair scheduler
> -
>
> Key: YARN-7133
> URL: https://issues.apache.org/jira/browse/YARN-7133
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: fairscheduler
>Affects Versions: 3.0.0-alpha4
>Reporter: Daniel Templeton
>Assignee: Gergo Repas
> Labels: newbie
>
> There are many places that follow the pattern:{code}try {
> lock.lock();
> ...
> } finally {
> lock.unlock();
> }{code}
> There are a couple of reasons that's a bad idea. The correct pattern
> is:{code}lock.lock();
> try {
> ...
> } finally {
> lock.unlock();
> }{code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-6505) Define the strings used in SLS JSON input file format
[ https://issues.apache.org/jira/browse/YARN-6505?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gergely Novák updated YARN-6505: Attachment: YARN-6505.005.patch > Define the strings used in SLS JSON input file format > - > > Key: YARN-6505 > URL: https://issues.apache.org/jira/browse/YARN-6505 > Project: Hadoop YARN > Issue Type: Sub-task > Components: scheduler-load-simulator >Reporter: Yufei Gu >Assignee: Gergely Novák > Labels: newbie > Attachments: YARN-6505.001.patch, YARN-6505.002.patch, > YARN-6505.003.patch, YARN-6505.004.patch, YARN-6505.005.patch > > > We could put them in a Java file like what YarnConfiguration does. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6894) RM Apps API returns only active apps when query parameter queue used
[ https://issues.apache.org/jira/browse/YARN-6894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16170026#comment-16170026 ] Gergely Novák commented on YARN-6894: - Any other thoughts? [~sunilg], are you satisfied with the latest patch? > RM Apps API returns only active apps when query parameter queue used > > > Key: YARN-6894 > URL: https://issues.apache.org/jira/browse/YARN-6894 > Project: Hadoop YARN > Issue Type: Bug > Components: resourcemanager, restapi >Reporter: Grant Sohn >Assignee: Gergely Novák >Priority: Minor > Attachments: YARN-6894.001.patch, YARN-6894.002.patch > > > If you run RM's Cluster Applications API with no query parameters, you get a > list of apps. > If you run RM's Cluster Applications API with any query parameters other than > "queue" you get the list of apps with the parameter filters being applied. > However, when you use the "queue" query parameter, you only see the > applications that are active in the cluster (NEW, NEW_SAVING, SUBMITTED, > ACCEPTED, RUNNING). This behavior is inconsistent with the API. If there is > a sound reason behind this, it should be documented and it seems like there > might be as the mapred queue CLI behaves similarly. > http://hadoop.apache.org/docs/stable/hadoop-yarn/hadoop-yarn-site/ResourceManagerRest.html#Cluster_Applications_API -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7157) Add admin configuration to filter per-user's apps in secure cluster
[ https://issues.apache.org/jira/browse/YARN-7157?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16169990#comment-16169990 ] Rohith Sharma K S commented on YARN-7157: - I have committed to branch-3.0 as well. Branch-2 patch has not run properly cc:/ [~sunilg] > Add admin configuration to filter per-user's apps in secure cluster > --- > > Key: YARN-7157 > URL: https://issues.apache.org/jira/browse/YARN-7157 > Project: Hadoop YARN > Issue Type: Bug > Components: webapp >Reporter: Sunil G >Assignee: Sunil G > Fix For: 3.0.0-beta1, 3.1.0 > > Attachments: YARN-7157.001.patch, YARN-7157.002.patch, > YARN-7157.003.patch, YARN-7157.004.patch, YARN-7157.005.patch, > YARN-7157-branch-2.001.patch, YARN-7157-branch-2.002.patch, > YARN-7157-branch-3.0.001.patch, YARN-7157-branch-3.0.002.patch > > > A user who is accessing a secure cluster via a secure UI should be able to > see only his/her own apps. > This feature will not break any compatibility as it will turned off by default -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-7157) Add admin configuration to filter per-user's apps in secure cluster
[ https://issues.apache.org/jira/browse/YARN-7157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rohith Sharma K S updated YARN-7157: Fix Version/s: 3.1.0 3.0.0-beta1 > Add admin configuration to filter per-user's apps in secure cluster > --- > > Key: YARN-7157 > URL: https://issues.apache.org/jira/browse/YARN-7157 > Project: Hadoop YARN > Issue Type: Bug > Components: webapp >Reporter: Sunil G >Assignee: Sunil G > Fix For: 3.0.0-beta1, 3.1.0 > > Attachments: YARN-7157.001.patch, YARN-7157.002.patch, > YARN-7157.003.patch, YARN-7157.004.patch, YARN-7157.005.patch, > YARN-7157-branch-2.001.patch, YARN-7157-branch-2.002.patch, > YARN-7157-branch-3.0.001.patch, YARN-7157-branch-3.0.002.patch > > > A user who is accessing a secure cluster via a secure UI should be able to > see only his/her own apps. > This feature will not break any compatibility as it will turned off by default -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
