[jira] [Commented] (YARN-5168) Add port mapping handling when docker container use bridge network
[ https://issues.apache.org/jira/browse/YARN-5168?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16722058#comment-16722058 ] Xun Liu commented on YARN-5168: --- [~eyang], I haven't done any development on WEB UI2 before. This time, it took 2 days to learn Ember.js, so I missed the time. I will complete this task as soon as possible, please wait a moment. :) > Add port mapping handling when docker container use bridge network > -- > > Key: YARN-5168 > URL: https://issues.apache.org/jira/browse/YARN-5168 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Jun Gong >Assignee: Xun Liu >Priority: Major > Labels: Docker > Attachments: YARN-5168.001.patch, YARN-5168.002.patch, > YARN-5168.003.patch, YARN-5168.004.patch, YARN-5168.005.patch, > YARN-5168.006.patch, YARN-5168.007.patch, YARN-5168.008.patch, > YARN-5168.009.patch, YARN-5168.010.patch > > > YARN-4007 addresses different network setups when launching the docker > container. We need support port mapping when docker container uses bridge > network. > The following problems are what we faced: > 1. Add "-P" to map docker container's exposed ports to automatically. > 2. Add "-p" to let user specify specific ports to map. > 3. Add service registry support for bridge network case, then app could find > each other. It could be done out of YARN, however it might be more convenient > to support it natively in YARN. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Issue Comment Deleted] (YARN-5168) Add port mapping handling when docker container use bridge network
[ https://issues.apache.org/jira/browse/YARN-5168?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xun Liu updated YARN-5168: -- Comment: was deleted (was: [~eyang], Testing, ready to submit patch, please wait a moment. :)) > Add port mapping handling when docker container use bridge network > -- > > Key: YARN-5168 > URL: https://issues.apache.org/jira/browse/YARN-5168 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Jun Gong >Assignee: Xun Liu >Priority: Major > Labels: Docker > Attachments: YARN-5168.001.patch, YARN-5168.002.patch, > YARN-5168.003.patch, YARN-5168.004.patch, YARN-5168.005.patch, > YARN-5168.006.patch, YARN-5168.007.patch, YARN-5168.008.patch, > YARN-5168.009.patch, YARN-5168.010.patch > > > YARN-4007 addresses different network setups when launching the docker > container. We need support port mapping when docker container uses bridge > network. > The following problems are what we faced: > 1. Add "-P" to map docker container's exposed ports to automatically. > 2. Add "-p" to let user specify specific ports to map. > 3. Add service registry support for bridge network case, then app could find > each other. It could be done out of YARN, however it might be more convenient > to support it natively in YARN. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5168) Add port mapping handling when docker container use bridge network
[ https://issues.apache.org/jira/browse/YARN-5168?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16722054#comment-16722054 ] Xun Liu commented on YARN-5168: --- [~eyang], Testing, ready to submit patch, please wait a moment. :) > Add port mapping handling when docker container use bridge network > -- > > Key: YARN-5168 > URL: https://issues.apache.org/jira/browse/YARN-5168 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Jun Gong >Assignee: Xun Liu >Priority: Major > Labels: Docker > Attachments: YARN-5168.001.patch, YARN-5168.002.patch, > YARN-5168.003.patch, YARN-5168.004.patch, YARN-5168.005.patch, > YARN-5168.006.patch, YARN-5168.007.patch, YARN-5168.008.patch, > YARN-5168.009.patch, YARN-5168.010.patch > > > YARN-4007 addresses different network setups when launching the docker > container. We need support port mapping when docker container uses bridge > network. > The following problems are what we faced: > 1. Add "-P" to map docker container's exposed ports to automatically. > 2. Add "-p" to let user specify specific ports to map. > 3. Add service registry support for bridge network case, then app could find > each other. It could be done out of YARN, however it might be more convenient > to support it natively in YARN. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9095) Removed Unused field from Resource: NUM_MANDATORY_RESOURCES
[ https://issues.apache.org/jira/browse/YARN-9095?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16722045#comment-16722045 ] Akira Ajisaka commented on YARN-9095: - +1, this code should have been removed in YARN-6909. > Removed Unused field from Resource: NUM_MANDATORY_RESOURCES > --- > > Key: YARN-9095 > URL: https://issues.apache.org/jira/browse/YARN-9095 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: Szilard Nemeth >Assignee: Vidura Bhathiya Mudalige >Priority: Trivial > Labels: newbie, newbie++ > > I suppose this constant remained in the code from historical reasons, but > this is not used anymore so it could be removed. > This field is especially confusing for new readers, as ResourceInformation > now has a field named MANDATORY_RESOURCES and this map contains not only > memory and vcores but GPU and FPGA as well. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-9095) Removed Unused field from Resource: NUM_MANDATORY_RESOURCES
[ https://issues.apache.org/jira/browse/YARN-9095?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akira Ajisaka updated YARN-9095: Target Version/s: 3.3.0 > Removed Unused field from Resource: NUM_MANDATORY_RESOURCES > --- > > Key: YARN-9095 > URL: https://issues.apache.org/jira/browse/YARN-9095 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: Szilard Nemeth >Assignee: Vidura Bhathiya Mudalige >Priority: Minor > Labels: newbie > Attachments: 445.patch > > > I suppose this constant remained in the code from historical reasons, but > this is not used anymore so it could be removed. > This field is especially confusing for new readers, as ResourceInformation > now has a field named MANDATORY_RESOURCES and this map contains not only > memory and vcores but GPU and FPGA as well. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-9095) Removed Unused field from Resource: NUM_MANDATORY_RESOURCES
[ https://issues.apache.org/jira/browse/YARN-9095?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akira Ajisaka updated YARN-9095: Priority: Minor (was: Trivial) > Removed Unused field from Resource: NUM_MANDATORY_RESOURCES > --- > > Key: YARN-9095 > URL: https://issues.apache.org/jira/browse/YARN-9095 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: Szilard Nemeth >Assignee: Vidura Bhathiya Mudalige >Priority: Minor > Labels: newbie > Attachments: 445.patch > > > I suppose this constant remained in the code from historical reasons, but > this is not used anymore so it could be removed. > This field is especially confusing for new readers, as ResourceInformation > now has a field named MANDATORY_RESOURCES and this map contains not only > memory and vcores but GPU and FPGA as well. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-9095) Removed Unused field from Resource: NUM_MANDATORY_RESOURCES
[ https://issues.apache.org/jira/browse/YARN-9095?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akira Ajisaka updated YARN-9095: Attachment: 445.patch > Removed Unused field from Resource: NUM_MANDATORY_RESOURCES > --- > > Key: YARN-9095 > URL: https://issues.apache.org/jira/browse/YARN-9095 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: Szilard Nemeth >Assignee: Vidura Bhathiya Mudalige >Priority: Trivial > Labels: newbie, newbie++ > Attachments: 445.patch > > > I suppose this constant remained in the code from historical reasons, but > this is not used anymore so it could be removed. > This field is especially confusing for new readers, as ResourceInformation > now has a field named MANDATORY_RESOURCES and this map contains not only > memory and vcores but GPU and FPGA as well. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9072) Web browser close without proper exit can leak shell process
[
https://issues.apache.org/jira/browse/YARN-9072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16722010#comment-16722010
]
Hadoop QA commented on YARN-9072:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
23s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 21m
49s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
6s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
27s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
38s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
13m 20s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
2s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
26s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
37s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
57s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
57s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
22s{color} | {color:green}
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager:
The patch generated 0 new + 0 unchanged - 1 fixed = 0 total (was 1) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
34s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
13m 36s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m
10s{color} | {color:red}
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager
generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
26s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 19m
21s{color} | {color:green} hadoop-yarn-server-nodemanager in the patch passed.
{color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
27s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 76m 41s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs |
module:hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager
|
| | Found reliance on default encoding in
org.apache.hadoop.yarn.server.nodemanager.webapp.ContainerShellWebSocket.onClose(Session,
int, String):in
org.apache.hadoop.yarn.server.nodemanager.webapp.ContainerShellWebSocket.onClose(Session,
int, String): String.getBytes() At ContainerShellWebSocket.java:[line 140] |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:8f97d6f |
| JIRA Issue | YARN-9072 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12951878/YARN-9072.003.patch |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient findbugs checkstyle |
| uname | Linux b29bd6ebda47 4.4.0-138-generic #164~14.04.1-Ubuntu SMP Fri Oct
5 08:56:16 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.
[jira] [Commented] (YARN-9091) Improve terminal message when connection is refused
[ https://issues.apache.org/jira/browse/YARN-9091?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16721999#comment-16721999 ] Hudson commented on YARN-9091: -- SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #15616 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/15616/]) YARN-9091. Improve terminal message when connection is refused. (billie: rev 4f622337eed5d7d956a952d2fa25e78615da1787) * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/resources/TERMINAL/terminal.template > Improve terminal message when connection is refused > --- > > Key: YARN-9091 > URL: https://issues.apache.org/jira/browse/YARN-9091 > Project: Hadoop YARN > Issue Type: Sub-task >Affects Versions: 3.3.0 >Reporter: Eric Yang >Assignee: Eric Yang >Priority: Major > Fix For: 3.3.0 > > Attachments: YARN-9091.001.patch > > > If a user does not have proper access to login to a container. UI2 version > of Terminal will not display any message. It would be nice to report back > the connection has been refused. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9125) Carriage Return character in launch command cause node manager to become unhealthy
[
https://issues.apache.org/jira/browse/YARN-9125?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16721994#comment-16721994
]
Hadoop QA commented on YARN-9125:
-
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 16m
17s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} branch-3.0 Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 22m
26s{color} | {color:green} branch-3.0 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
49s{color} | {color:green} branch-3.0 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
24s{color} | {color:green} branch-3.0 passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
35s{color} | {color:green} branch-3.0 passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
11m 58s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
58s{color} | {color:green} branch-3.0 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
25s{color} | {color:green} branch-3.0 passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
38s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
53s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
53s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 22s{color} | {color:orange}
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager:
The patch generated 1 new + 1 unchanged - 0 fixed = 2 total (was 1) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
34s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
13m 17s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
58s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
20s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 16m
38s{color} | {color:green} hadoop-yarn-server-nodemanager in the patch passed.
{color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
23s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 88m 7s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:e402791 |
| JIRA Issue | YARN-9125 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12951873/YARN-9125-branch-3.0.01.patch
|
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient findbugs checkstyle |
| uname | Linux 4a9890400d64 4.4.0-138-generic #164~14.04.1-Ubuntu SMP Fri Oct
5 08:56:16 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | branch-3.0 / d782342 |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_191 |
| findbugs | v3.1.0-RC1 |
| checkstyle |
https://builds.apache.org/job/PreCommit-YARN-Build/22868/artifact/out/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/22868/testReport/ |
| Max. process+thread count | 303 (vs. ulimit of 1) |
| modules | C:
hadoop-yarn-project/hadoop
[jira] [Commented] (YARN-9072) Web browser close without proper exit can leak shell process
[ https://issues.apache.org/jira/browse/YARN-9072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16721991#comment-16721991 ] Billie Rinaldi commented on YARN-9072: -- I am in favor of the latest patch, pending precommit build. I tested and verified that the shell processes are not leaked on browser close or permission denied. > Web browser close without proper exit can leak shell process > > > Key: YARN-9072 > URL: https://issues.apache.org/jira/browse/YARN-9072 > Project: Hadoop YARN > Issue Type: Sub-task >Affects Versions: 3.3.0 >Reporter: Eric Yang >Assignee: Eric Yang >Priority: Major > Attachments: YARN-9072.001.patch, YARN-9072.002.patch, > YARN-9072.003.patch > > > If web browser is closed without typing exit in container shell, it will > leave bash process in the docker container. It would be nice to detect the > websocket is closed, and terminate the bash process from docker container. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9091) Improve terminal message when connection is refused
[ https://issues.apache.org/jira/browse/YARN-9091?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16721989#comment-16721989 ] Billie Rinaldi commented on YARN-9091: -- Thanks, [~eyang]! > Improve terminal message when connection is refused > --- > > Key: YARN-9091 > URL: https://issues.apache.org/jira/browse/YARN-9091 > Project: Hadoop YARN > Issue Type: Sub-task >Affects Versions: 3.3.0 >Reporter: Eric Yang >Assignee: Eric Yang >Priority: Major > Attachments: YARN-9091.001.patch > > > If a user does not have proper access to login to a container. UI2 version > of Terminal will not display any message. It would be nice to report back > the connection has been refused. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9091) Improve terminal message when connection is refused
[ https://issues.apache.org/jira/browse/YARN-9091?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16721987#comment-16721987 ] Billie Rinaldi commented on YARN-9091: -- +1 for patch 1. I verified that the UI container terminal receives appropriate messages based on the exit codes. > Improve terminal message when connection is refused > --- > > Key: YARN-9091 > URL: https://issues.apache.org/jira/browse/YARN-9091 > Project: Hadoop YARN > Issue Type: Sub-task >Affects Versions: 3.3.0 >Reporter: Eric Yang >Assignee: Eric Yang >Priority: Major > Attachments: YARN-9091.001.patch > > > If a user does not have proper access to login to a container. UI2 version > of Terminal will not display any message. It would be nice to report back > the connection has been refused. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Issue Comment Deleted] (YARN-9091) Improve terminal message when connection is refused
[ https://issues.apache.org/jira/browse/YARN-9091?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Billie Rinaldi updated YARN-9091: - Comment: was deleted (was: Thanks, [~eyang]!) > Improve terminal message when connection is refused > --- > > Key: YARN-9091 > URL: https://issues.apache.org/jira/browse/YARN-9091 > Project: Hadoop YARN > Issue Type: Sub-task >Affects Versions: 3.3.0 >Reporter: Eric Yang >Assignee: Eric Yang >Priority: Major > Attachments: YARN-9091.001.patch > > > If a user does not have proper access to login to a container. UI2 version > of Terminal will not display any message. It would be nice to report back > the connection has been refused. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9091) Improve terminal message when connection is refused
[ https://issues.apache.org/jira/browse/YARN-9091?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16721988#comment-16721988 ] Billie Rinaldi commented on YARN-9091: -- Thanks, [~eyang]! > Improve terminal message when connection is refused > --- > > Key: YARN-9091 > URL: https://issues.apache.org/jira/browse/YARN-9091 > Project: Hadoop YARN > Issue Type: Sub-task >Affects Versions: 3.3.0 >Reporter: Eric Yang >Assignee: Eric Yang >Priority: Major > Attachments: YARN-9091.001.patch > > > If a user does not have proper access to login to a container. UI2 version > of Terminal will not display any message. It would be nice to report back > the connection has been refused. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9072) Web browser close without proper exit can leak shell process
[ https://issues.apache.org/jira/browse/YARN-9072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16721982#comment-16721982 ] Billie Rinaldi commented on YARN-9072: -- Rebased patch 2. > Web browser close without proper exit can leak shell process > > > Key: YARN-9072 > URL: https://issues.apache.org/jira/browse/YARN-9072 > Project: Hadoop YARN > Issue Type: Sub-task >Affects Versions: 3.3.0 >Reporter: Eric Yang >Assignee: Eric Yang >Priority: Major > Attachments: YARN-9072.001.patch, YARN-9072.002.patch, > YARN-9072.003.patch > > > If web browser is closed without typing exit in container shell, it will > leave bash process in the docker container. It would be nice to detect the > websocket is closed, and terminate the bash process from docker container. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9125) Carriage Return character in launch command cause node manager to become unhealthy
[ https://issues.apache.org/jira/browse/YARN-9125?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16721983#comment-16721983 ] Rohith Sharma K S commented on YARN-9125: - Thanks [~billie.rinaldi] for the confirmation. > Carriage Return character in launch command cause node manager to become > unhealthy > -- > > Key: YARN-9125 > URL: https://issues.apache.org/jira/browse/YARN-9125 > Project: Hadoop YARN > Issue Type: Sub-task >Affects Versions: 2.9.0, 3.0.0, 3.1.0, 2.9.1, 3.0.2, 3.1.1, 2.9.2, 3.0.3 >Reporter: Eric Yang >Assignee: Billie Rinaldi >Priority: Major > Attachments: YARN-9125-branch-3.0.01.patch, YARN-9125.1.patch > > > If launch command contains \n character, it can cause node manager to become > unhealthy. There is a protection check against this, but it doesn't seem to > be working as implemented. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-9072) Web browser close without proper exit can leak shell process
[ https://issues.apache.org/jira/browse/YARN-9072?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Billie Rinaldi updated YARN-9072: - Attachment: YARN-9072.003.patch > Web browser close without proper exit can leak shell process > > > Key: YARN-9072 > URL: https://issues.apache.org/jira/browse/YARN-9072 > Project: Hadoop YARN > Issue Type: Sub-task >Affects Versions: 3.3.0 >Reporter: Eric Yang >Assignee: Eric Yang >Priority: Major > Attachments: YARN-9072.001.patch, YARN-9072.002.patch, > YARN-9072.003.patch > > > If web browser is closed without typing exit in container shell, it will > leave bash process in the docker container. It would be nice to detect the > websocket is closed, and terminate the bash process from docker container. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9125) Carriage Return character in launch command cause node manager to become unhealthy
[ https://issues.apache.org/jira/browse/YARN-9125?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16721976#comment-16721976 ] Billie Rinaldi commented on YARN-9125: -- Thanks for reviewing, [~rohithsharma]. This specific problem only occurs in the docker case, related to the docker command file. There does not seem to be an analogous situation for linux containers. > Carriage Return character in launch command cause node manager to become > unhealthy > -- > > Key: YARN-9125 > URL: https://issues.apache.org/jira/browse/YARN-9125 > Project: Hadoop YARN > Issue Type: Sub-task >Affects Versions: 2.9.0, 3.0.0, 3.1.0, 2.9.1, 3.0.2, 3.1.1, 2.9.2, 3.0.3 >Reporter: Eric Yang >Assignee: Billie Rinaldi >Priority: Major > Attachments: YARN-9125-branch-3.0.01.patch, YARN-9125.1.patch > > > If launch command contains \n character, it can cause node manager to become > unhealthy. There is a protection check against this, but it doesn't seem to > be working as implemented. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-8963) Add flag to disable interactive shell
[ https://issues.apache.org/jira/browse/YARN-8963?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16721977#comment-16721977 ] Hudson commented on YARN-8963: -- SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #15615 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/15615/]) YARN-8963. Add flag to disable interactive shell. Contributed by Eric (billie: rev 42297f7d99df2769846a58efcf1b02fadb5ba60b) * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/impl/main.c * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/SecureContainer.md * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/impl/container-executor.c * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/impl/container-executor.h > Add flag to disable interactive shell > - > > Key: YARN-8963 > URL: https://issues.apache.org/jira/browse/YARN-8963 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Eric Yang >Assignee: Eric Yang >Priority: Major > Fix For: 3.3.0 > > Attachments: YARN-8963.001.patch, YARN-8963.002.patch, > YARN-8963.003.patch > > > For some production job, application admin might choose to disable debugging > to production jobs to prevent developer or system admin from accessing the > containers. It would be nice to add an environment variable flag to disable > interactive shell during application submission. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9089) Add Terminal Link to Service component instance page for UI2
[ https://issues.apache.org/jira/browse/YARN-9089?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16721971#comment-16721971 ] Hudson commented on YARN-9089: -- SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #15614 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/15614/]) YARN-9089. Add Terminal Link to Service component instance page for UI2. (billie: rev aab310978f64d57cf414de095e6195cbc4d0ed55) * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-ui/src/main/webapp/app/templates/yarn-component-instance/info.hbs * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-ui/src/main/webapp/app/initializers/loader.js * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-ui/src/main/webapp/app/models/yarn-component-instance.js * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-ui/src/main/webapp/app/routes/yarn-component-instance/info.js * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-ui/src/main/webapp/app/initializers/env.js > Add Terminal Link to Service component instance page for UI2 > > > Key: YARN-9089 > URL: https://issues.apache.org/jira/browse/YARN-9089 > Project: Hadoop YARN > Issue Type: Sub-task > Components: yarn-ui-v2 >Affects Versions: 3.3.0 >Reporter: Eric Yang >Assignee: Eric Yang >Priority: Major > Fix For: 3.3.0 > > Attachments: YARN-9089.001.patch, YARN-9089.002.patch, > YARN-9089.003.patch, YARN-9089.004.patch > > > In UI2, Service > Component > Component Instance uses Timeline server to > aggregate information about Service component instance. Timeline server does > not have the full information like the port number of the node manager, or > the web protocol used by the node manager. It requires some changes to > aggregate node manager information into Timeline server in order to compute > the Terminal link. For reducing the scope of YARN-8914, it is better file > this as a separate task. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9125) Carriage Return character in launch command cause node manager to become unhealthy
[ https://issues.apache.org/jira/browse/YARN-9125?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16721970#comment-16721970 ] Billie Rinaldi commented on YARN-9125: -- The branch-3.0 patch works for branch-2.9 as well. > Carriage Return character in launch command cause node manager to become > unhealthy > -- > > Key: YARN-9125 > URL: https://issues.apache.org/jira/browse/YARN-9125 > Project: Hadoop YARN > Issue Type: Sub-task >Affects Versions: 2.9.0, 3.0.0, 3.1.0, 2.9.1, 3.0.2, 3.1.1, 2.9.2, 3.0.3 >Reporter: Eric Yang >Assignee: Billie Rinaldi >Priority: Major > Attachments: YARN-9125-branch-3.0.01.patch, YARN-9125.1.patch > > > If launch command contains \n character, it can cause node manager to become > unhealthy. There is a protection check against this, but it doesn't seem to > be working as implemented. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-8963) Add flag to disable interactive shell
[ https://issues.apache.org/jira/browse/YARN-8963?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16721969#comment-16721969 ] Billie Rinaldi commented on YARN-8963: -- +1 for patch 3. I verified that the feature.terminal.enabled flag controls whether or not a shell can be opened for a container. In the UI, the feature disabled message is displayed, but at the command line the connection appears to be closed without the message being displayed. I suspect this is due to inconsistent flushing after writing to stderr, which I have opened YARN-9129 to address. Thanks for the patch, [~eyang]! > Add flag to disable interactive shell > - > > Key: YARN-8963 > URL: https://issues.apache.org/jira/browse/YARN-8963 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Eric Yang >Assignee: Eric Yang >Priority: Major > Attachments: YARN-8963.001.patch, YARN-8963.002.patch, > YARN-8963.003.patch > > > For some production job, application admin might choose to disable debugging > to production jobs to prevent developer or system admin from accessing the > containers. It would be nice to add an environment variable flag to disable > interactive shell during application submission. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9125) Carriage Return character in launch command cause node manager to become unhealthy
[ https://issues.apache.org/jira/browse/YARN-9125?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16721968#comment-16721968 ] Rohith Sharma K S commented on YARN-9125: - [~billie.rinaldi] [~eyang] One doubt, fix is done in docker client. Doesn't it applies to non-docker but using LinuxCE? > Carriage Return character in launch command cause node manager to become > unhealthy > -- > > Key: YARN-9125 > URL: https://issues.apache.org/jira/browse/YARN-9125 > Project: Hadoop YARN > Issue Type: Sub-task >Affects Versions: 2.9.0, 3.0.0, 3.1.0, 2.9.1, 3.0.2, 3.1.1, 2.9.2, 3.0.3 >Reporter: Eric Yang >Assignee: Billie Rinaldi >Priority: Major > Attachments: YARN-9125-branch-3.0.01.patch, YARN-9125.1.patch > > > If launch command contains \n character, it can cause node manager to become > unhealthy. There is a protection check against this, but it doesn't seem to > be working as implemented. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9089) Add Terminal Link to Service component instance page for UI2
[ https://issues.apache.org/jira/browse/YARN-9089?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16721965#comment-16721965 ] Billie Rinaldi commented on YARN-9089: -- +1 for patch 4. Thanks for the patch [~eyang] and for the review [~akhilpb]! > Add Terminal Link to Service component instance page for UI2 > > > Key: YARN-9089 > URL: https://issues.apache.org/jira/browse/YARN-9089 > Project: Hadoop YARN > Issue Type: Sub-task > Components: yarn-ui-v2 >Affects Versions: 3.3.0 >Reporter: Eric Yang >Assignee: Eric Yang >Priority: Major > Attachments: YARN-9089.001.patch, YARN-9089.002.patch, > YARN-9089.003.patch, YARN-9089.004.patch > > > In UI2, Service > Component > Component Instance uses Timeline server to > aggregate information about Service component instance. Timeline server does > not have the full information like the port number of the node manager, or > the web protocol used by the node manager. It requires some changes to > aggregate node manager information into Timeline server in order to compute > the Terminal link. For reducing the scope of YARN-8914, it is better file > this as a separate task. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-9125) Carriage Return character in launch command cause node manager to become unhealthy
[ https://issues.apache.org/jira/browse/YARN-9125?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Billie Rinaldi updated YARN-9125: - Attachment: YARN-9125-branch-3.0.01.patch > Carriage Return character in launch command cause node manager to become > unhealthy > -- > > Key: YARN-9125 > URL: https://issues.apache.org/jira/browse/YARN-9125 > Project: Hadoop YARN > Issue Type: Sub-task >Affects Versions: 2.9.0, 3.0.0, 3.1.0, 2.9.1, 3.0.2, 3.1.1, 2.9.2, 3.0.3 >Reporter: Eric Yang >Assignee: Billie Rinaldi >Priority: Major > Attachments: YARN-9125-branch-3.0.01.patch, YARN-9125.1.patch > > > If launch command contains \n character, it can cause node manager to become > unhealthy. There is a protection check against this, but it doesn't seem to > be working as implemented. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-9101) Recovery Contaienr exitCode Not Right
[ https://issues.apache.org/jira/browse/YARN-9101?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SuperbDong updated YARN-9101: - Docs Text: (was: It's correct exitCode when container launch nomally,but it is not correct if the container by recovery. Out of memory exitCode is -104, the exitCode had to be lost when the container was recovered by restart NodeManager.) > Recovery Contaienr exitCode Not Right > - > > Key: YARN-9101 > URL: https://issues.apache.org/jira/browse/YARN-9101 > Project: Hadoop YARN > Issue Type: Bug > Components: nodemanager >Affects Versions: 3.0.0 >Reporter: SuperbDong >Priority: Major > Labels: pull-request-available > Attachments: YARN-9101.patch > > > It's correct exitCode when container launch nomally,but it is not correct if > the container by recovery. > Out of memory exitCode is -104, the exitCode had to be lost when the > container was recovered by restart NodeManager. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-9101) Recovery Contaienr exitCode Not Right
[ https://issues.apache.org/jira/browse/YARN-9101?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SuperbDong updated YARN-9101: - Description: It's correct exitCode when container launch nomally,but it is not correct if the container by recovery. Out of memory exitCode is -104, the exitCode had to be lost when the container was recovered by restart NodeManager. > Recovery Contaienr exitCode Not Right > - > > Key: YARN-9101 > URL: https://issues.apache.org/jira/browse/YARN-9101 > Project: Hadoop YARN > Issue Type: Bug > Components: nodemanager >Affects Versions: 3.0.0 >Reporter: SuperbDong >Priority: Major > Labels: pull-request-available > Attachments: YARN-9101.patch > > > It's correct exitCode when container launch nomally,but it is not correct if > the container by recovery. > Out of memory exitCode is -104, the exitCode had to be lost when the > container was recovered by restart NodeManager. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9130) Add Bind_HOST configuration for Yarn Web Proxy
[
https://issues.apache.org/jira/browse/YARN-9130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16721963#comment-16721963
]
Hadoop QA commented on YARN-9130:
-
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
22s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
45s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 20m
42s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 8m
29s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
31s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 2m
4s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
16m 7s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
31s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
49s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
13s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
40s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 7m
42s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 7m
42s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
1m 30s{color} | {color:orange} hadoop-yarn-project/hadoop-yarn: The patch
generated 1 new + 223 unchanged - 1 fixed = 224 total (was 224) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
59s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
1s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
13m 14s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
44s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
50s{color} | {color:green} hadoop-yarn-api in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 3m
40s{color} | {color:green} hadoop-yarn-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m
5s{color} | {color:green} hadoop-yarn-server-web-proxy in the patch passed.
{color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
46s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 92m 22s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:8f97d6f |
| JIRA Issue | YARN-9130 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12951867/YARN-9130.002.patch |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient findbugs checkstyle xml |
| uname | Linux 900a0bf29b83 4.4.0-138-generic #164~14.04.1-Ubuntu SMP Fri Oct
5 08:56:16 UTC
[jira] [Commented] (YARN-9122) Add table of contents to YARN Service API document
[ https://issues.apache.org/jira/browse/YARN-9122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16721951#comment-16721951 ] Hudson commented on YARN-9122: -- SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #15612 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/15612/]) YARN-9122. Add table of contents to YARN Service API document. (aajisaka: rev 26e55d4ceb691f47801fa5f0ba1147a5bc413f96) * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/yarn-service/YarnServiceAPI.md > Add table of contents to YARN Service API document > -- > > Key: YARN-9122 > URL: https://issues.apache.org/jira/browse/YARN-9122 > Project: Hadoop YARN > Issue Type: Improvement > Components: documentation >Reporter: Akira Ajisaka >Assignee: Zhankun Tang >Priority: Minor > Labels: newbie > Fix For: 3.1.2, 3.3.0, 3.2.1 > > Attachments: YARN-9122-trunk.001.patch, YARN-9122-trunk.001.patch, > s1.png > > > I think it's better to have a table of contents in > https://hadoop.apache.org/docs/r3.1.1/hadoop-yarn/hadoop-yarn-site/yarn-service/YarnServiceAPI.html -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9130) Add Bind_HOST configuration for Yarn Web Proxy
[ https://issues.apache.org/jira/browse/YARN-9130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16721943#comment-16721943 ] Íñigo Goiri commented on YARN-9130: --- Let's also fix the checkstyle, it makes sense to not hide the conf field. > Add Bind_HOST configuration for Yarn Web Proxy > -- > > Key: YARN-9130 > URL: https://issues.apache.org/jira/browse/YARN-9130 > Project: Hadoop YARN > Issue Type: Improvement > Components: yarn >Affects Versions: 3.1.1 >Reporter: Rong Tang >Assignee: Rong Tang >Priority: Major > Attachments: YARN-9130.002.patch, Yarn-9130.001.patch > > > Allow configurable bind-host for Yarn Web Proxy to allow overriding the host > name for which the server accepts connections. > It is similar to what have done in JournalNode and RM. Like > https://issues.apache.org/jira/browse/HDFS-13462 > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9130) Add Bind_HOST configuration for Yarn Web Proxy
[ https://issues.apache.org/jira/browse/YARN-9130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16721908#comment-16721908 ] Rong Tang commented on YARN-9130: - [~elgoiri] Thanks. Send out the second patch with your comment fix. > Add Bind_HOST configuration for Yarn Web Proxy > -- > > Key: YARN-9130 > URL: https://issues.apache.org/jira/browse/YARN-9130 > Project: Hadoop YARN > Issue Type: Improvement > Components: yarn >Affects Versions: 3.1.1 >Reporter: Rong Tang >Assignee: Rong Tang >Priority: Major > Attachments: YARN-9130.002.patch, Yarn-9130.001.patch > > > Allow configurable bind-host for Yarn Web Proxy to allow overriding the host > name for which the server accepts connections. > It is similar to what have done in JournalNode and RM. Like > https://issues.apache.org/jira/browse/HDFS-13462 > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-9130) Add Bind_HOST configuration for Yarn Web Proxy
[ https://issues.apache.org/jira/browse/YARN-9130?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rong Tang updated YARN-9130: Attachment: YARN-9130.002.patch > Add Bind_HOST configuration for Yarn Web Proxy > -- > > Key: YARN-9130 > URL: https://issues.apache.org/jira/browse/YARN-9130 > Project: Hadoop YARN > Issue Type: Improvement > Components: yarn >Affects Versions: 3.1.1 >Reporter: Rong Tang >Assignee: Rong Tang >Priority: Major > Attachments: YARN-9130.002.patch, Yarn-9130.001.patch > > > Allow configurable bind-host for Yarn Web Proxy to allow overriding the host > name for which the server accepts connections. > It is similar to what have done in JournalNode and RM. Like > https://issues.apache.org/jira/browse/HDFS-13462 > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-9132) Add file permission check for auxiliary services manifest file
Eric Yang created YARN-9132: --- Summary: Add file permission check for auxiliary services manifest file Key: YARN-9132 URL: https://issues.apache.org/jira/browse/YARN-9132 Project: Hadoop YARN Issue Type: Bug Reporter: Eric Yang The manifest file in HDFS must be owned by YARN admin or YARN service user only. This check helps to prevent loading of malware into node manager JVM. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-9131) Document usage of Dynamic auxiliary services
Eric Yang created YARN-9131: --- Summary: Document usage of Dynamic auxiliary services Key: YARN-9131 URL: https://issues.apache.org/jira/browse/YARN-9131 Project: Hadoop YARN Issue Type: Bug Reporter: Eric Yang This is a follow up issue to document YARN-9075 for admin to control which aux service to add or remove. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9130) Add Bind_HOST configuration for Yarn Web Proxy
[
https://issues.apache.org/jira/browse/YARN-9130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16721896#comment-16721896
]
Hadoop QA commented on YARN-9130:
-
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
49s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
51s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 26m
55s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 8m
35s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
34s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 2m
15s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
16m 12s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
41s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
49s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
15s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
43s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 9m
4s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 9m
4s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
1m 30s{color} | {color:orange} hadoop-yarn-project/hadoop-yarn: The patch
generated 1 new + 223 unchanged - 1 fixed = 224 total (was 224) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 2m
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
1s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
13m 23s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
53s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
45s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
47s{color} | {color:green} hadoop-yarn-api in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 3m
36s{color} | {color:green} hadoop-yarn-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m
4s{color} | {color:green} hadoop-yarn-server-web-proxy in the patch passed.
{color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
39s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}100m 54s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:8f97d6f |
| JIRA Issue | YARN-9130 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12951858/Yarn-9130.001.patch |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient findbugs checkstyle xml |
| uname | Linux bfdbcd600f61 4.4.0-138-generic #164~14.04.1-Ubuntu SMP Fri Oct
5 08:56:16 UTC
[jira] [Commented] (YARN-8963) Add flag to disable interactive shell
[
https://issues.apache.org/jira/browse/YARN-8963?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16721878#comment-16721878
]
Hadoop QA commented on YARN-8963:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
13s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
48s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 20m
48s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 7m
49s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
13s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
43m 0s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
13s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
50s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 7m
7s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} cc {color} | {color:green} 7m
7s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 7m
7s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
6s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
13m 22s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 18m
59s{color} | {color:green} hadoop-yarn-server-nodemanager in the patch passed.
{color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
21s{color} | {color:green} hadoop-yarn-site in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
40s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 86m 45s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:8f97d6f |
| JIRA Issue | YARN-8963 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12951836/YARN-8963.003.patch |
| Optional Tests | dupname asflicense compile cc mvnsite javac unit |
| uname | Linux d20b4f8d2801 4.4.0-139-generic #165~14.04.1-Ubuntu SMP Wed Oct
31 10:55:11 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / 00d5e63 |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_181 |
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/22864/testReport/ |
| Max. process+thread count | 339 (vs. ulimit of 1) |
| modules | C:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site U:
hadoop-yarn-project/hadoop-yarn |
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/22864/console |
| Powered by | Apache Yetus 0.8.0 http://yetus.apache.org |
This message was automatically generated.
> Add flag to disable interactive shell
> -
>
> Key: YARN-8963
> URL: https://issues.apache.org/jira/browse/YARN-8963
> Project: Hadoop
[jira] [Commented] (YARN-9089) Add Terminal Link to Service component instance page for UI2
[ https://issues.apache.org/jira/browse/YARN-9089?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16721872#comment-16721872 ] Eric Yang commented on YARN-9089: - [~akhilpb] Thanks for the information and review. > Add Terminal Link to Service component instance page for UI2 > > > Key: YARN-9089 > URL: https://issues.apache.org/jira/browse/YARN-9089 > Project: Hadoop YARN > Issue Type: Sub-task > Components: yarn-ui-v2 >Affects Versions: 3.3.0 >Reporter: Eric Yang >Assignee: Eric Yang >Priority: Major > Attachments: YARN-9089.001.patch, YARN-9089.002.patch, > YARN-9089.003.patch, YARN-9089.004.patch > > > In UI2, Service > Component > Component Instance uses Timeline server to > aggregate information about Service component instance. Timeline server does > not have the full information like the port number of the node manager, or > the web protocol used by the node manager. It requires some changes to > aggregate node manager information into Timeline server in order to compute > the Terminal link. For reducing the scope of YARN-8914, it is better file > this as a separate task. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9125) Carriage Return character in launch command cause node manager to become unhealthy
[ https://issues.apache.org/jira/browse/YARN-9125?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16721866#comment-16721866 ] Hudson commented on YARN-9125: -- SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #15611 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/15611/]) YARN-9125. Fixed Carriage Return detection in Docker container launch (eyang: rev b2d7204ed0b64405a0adbe07e3eaf385e046efa1) * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/runtime/docker/DockerClient.java * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/runtime/docker/TestDockerClient.java > Carriage Return character in launch command cause node manager to become > unhealthy > -- > > Key: YARN-9125 > URL: https://issues.apache.org/jira/browse/YARN-9125 > Project: Hadoop YARN > Issue Type: Sub-task >Affects Versions: 2.9.0, 3.0.0, 3.1.0, 2.9.1, 3.0.2, 3.1.1, 2.9.2, 3.0.3 >Reporter: Eric Yang >Assignee: Billie Rinaldi >Priority: Major > Attachments: YARN-9125.1.patch > > > If launch command contains \n character, it can cause node manager to become > unhealthy. There is a protection check against this, but it doesn't seem to > be working as implemented. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9072) Web browser close without proper exit can leak shell process
[
https://issues.apache.org/jira/browse/YARN-9072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16721856#comment-16721856
]
Hadoop QA commented on YARN-9072:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
0s{color} | {color:blue} Docker mode activated. {color} |
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 6s{color}
| {color:red} YARN-9072 does not apply to trunk. Rebase required? Wrong Branch?
See https://wiki.apache.org/hadoop/HowToContribute for help. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | YARN-9072 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12951064/YARN-9072.002.patch |
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/22866/console |
| Powered by | Apache Yetus 0.8.0 http://yetus.apache.org |
This message was automatically generated.
> Web browser close without proper exit can leak shell process
>
>
> Key: YARN-9072
> URL: https://issues.apache.org/jira/browse/YARN-9072
> Project: Hadoop YARN
> Issue Type: Sub-task
>Affects Versions: 3.3.0
>Reporter: Eric Yang
>Assignee: Eric Yang
>Priority: Major
> Attachments: YARN-9072.001.patch, YARN-9072.002.patch
>
>
> If web browser is closed without typing exit in container shell, it will
> leave bash process in the docker container. It would be nice to detect the
> websocket is closed, and terminate the bash process from docker container.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9125) Carriage Return character in launch command cause node manager to become unhealthy
[ https://issues.apache.org/jira/browse/YARN-9125?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16721855#comment-16721855 ] Eric Yang commented on YARN-9125: - Patch 001 committed to branch-3.1, branch-3.2, and trunk. Leaving this issue open for branch-3.0 and branch-2.9. > Carriage Return character in launch command cause node manager to become > unhealthy > -- > > Key: YARN-9125 > URL: https://issues.apache.org/jira/browse/YARN-9125 > Project: Hadoop YARN > Issue Type: Sub-task >Affects Versions: 2.9.0, 3.0.0, 3.1.0, 2.9.1, 3.0.2, 3.1.1, 2.9.2, 3.0.3 >Reporter: Eric Yang >Assignee: Billie Rinaldi >Priority: Major > Attachments: YARN-9125.1.patch > > > If launch command contains \n character, it can cause node manager to become > unhealthy. There is a protection check against this, but it doesn't seem to > be working as implemented. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9032) Support sh shell for interactive container shell at command line
[ https://issues.apache.org/jira/browse/YARN-9032?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16721838#comment-16721838 ] Eric Yang commented on YARN-9032: - [~billie.rinaldi] Thank you for the review and commit. > Support sh shell for interactive container shell at command line > > > Key: YARN-9032 > URL: https://issues.apache.org/jira/browse/YARN-9032 > Project: Hadoop YARN > Issue Type: Sub-task > Components: yarn >Reporter: Eric Yang >Assignee: Eric Yang >Priority: Blocker > Fix For: 3.3.0 > > Attachments: YARN-9032.001.patch, YARN-9032.002.patch > > > CLI can supply second parameter such as: > yarn container -shell [container-id] sh > This is supposed to launch sh instead of bash. This is not fully working yet > in YARN-8778 patch and some organization of code clean up to support > non-docker container in YARN-8962 making this change in YARN-8778 difficult. > Hence, I file this as a separate sub-task to close the gap. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9032) Support sh shell for interactive container shell at command line
[ https://issues.apache.org/jira/browse/YARN-9032?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16721828#comment-16721828 ] Hudson commented on YARN-9032: -- SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #15610 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/15610/]) YARN-9032. Support sh shell for interactive container shell at command (billie: rev ca90b2480ecb9a627914e7a463b6293024ad6e4d) * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/runtime/DefaultLinuxContainerRuntime.java * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/executor/ContainerExecContext.java * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/main/java/org/apache/hadoop/yarn/client/cli/ApplicationCLI.java * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/test/java/org/apache/hadoop/yarn/client/cli/TestYarnCLI.java * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/webapp/ContainerShellWebSocket.java * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/runtime/DockerLinuxContainerRuntime.java * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/main/java/org/apache/hadoop/yarn/client/api/impl/YarnClientImpl.java > Support sh shell for interactive container shell at command line > > > Key: YARN-9032 > URL: https://issues.apache.org/jira/browse/YARN-9032 > Project: Hadoop YARN > Issue Type: Sub-task > Components: yarn >Reporter: Eric Yang >Assignee: Eric Yang >Priority: Blocker > Fix For: 3.3.0 > > Attachments: YARN-9032.001.patch, YARN-9032.002.patch > > > CLI can supply second parameter such as: > yarn container -shell [container-id] sh > This is supposed to launch sh instead of bash. This is not fully working yet > in YARN-8778 patch and some organization of code clean up to support > non-docker container in YARN-8962 making this change in YARN-8778 difficult. > Hence, I file this as a separate sub-task to close the gap. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Assigned] (YARN-9130) Add Bind_HOST configuration for Yarn Web Proxy
[ https://issues.apache.org/jira/browse/YARN-9130?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Íñigo Goiri reassigned YARN-9130: - Assignee: Rong Tang > Add Bind_HOST configuration for Yarn Web Proxy > -- > > Key: YARN-9130 > URL: https://issues.apache.org/jira/browse/YARN-9130 > Project: Hadoop YARN > Issue Type: Improvement > Components: yarn >Affects Versions: 3.1.1 >Reporter: Rong Tang >Assignee: Rong Tang >Priority: Major > Attachments: Yarn-9130.001.patch > > > Allow configurable bind-host for Yarn Web Proxy to allow overriding the host > name for which the server accepts connections. > It is similar to what have done in JournalNode and RM. Like > https://issues.apache.org/jira/browse/HDFS-13462 > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9130) Add Bind_HOST configuration for Yarn Web Proxy
[ https://issues.apache.org/jira/browse/YARN-9130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16721826#comment-16721826 ] Íñigo Goiri commented on YARN-9130: --- Thanks [~trjianjianjiao] for the patch. I'm not sure if the file has to be YARN-9130.001.patch or Yarn-9130.001.patch is fine. A minor comment, in the unit test, the assert for the address should be in the reverse order as the expected is the first parameter. > Add Bind_HOST configuration for Yarn Web Proxy > -- > > Key: YARN-9130 > URL: https://issues.apache.org/jira/browse/YARN-9130 > Project: Hadoop YARN > Issue Type: Improvement > Components: yarn >Affects Versions: 3.1.1 >Reporter: Rong Tang >Priority: Major > Attachments: Yarn-9130.001.patch > > > Allow configurable bind-host for Yarn Web Proxy to allow overriding the host > name for which the server accepts connections. > It is similar to what have done in JournalNode and RM. Like > https://issues.apache.org/jira/browse/HDFS-13462 > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-9032) Support sh shell for interactive container shell at command line
[ https://issues.apache.org/jira/browse/YARN-9032?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Billie Rinaldi updated YARN-9032: - Summary: Support sh shell for interactive container shell at command line (was: Support sh shell for interactive shell) > Support sh shell for interactive container shell at command line > > > Key: YARN-9032 > URL: https://issues.apache.org/jira/browse/YARN-9032 > Project: Hadoop YARN > Issue Type: Sub-task > Components: yarn >Reporter: Eric Yang >Assignee: Eric Yang >Priority: Blocker > Fix For: 3.3.0 > > Attachments: YARN-9032.001.patch, YARN-9032.002.patch > > > CLI can supply second parameter such as: > yarn container -shell [container-id] sh > This is supposed to launch sh instead of bash. This is not fully working yet > in YARN-8778 patch and some organization of code clean up to support > non-docker container in YARN-8962 making this change in YARN-8778 difficult. > Hence, I file this as a separate sub-task to close the gap. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-9130) Add Bind_HOST configuration for Yarn Web Proxy
[ https://issues.apache.org/jira/browse/YARN-9130?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rong Tang updated YARN-9130: Description: Allow configurable bind-host for Yarn Web Proxy to allow overriding the host name for which the server accepts connections. It is similar to what have done in JournalNode and RM. Like https://issues.apache.org/jira/browse/HDFS-13462 was: Allow configurable bind-host for Yarn Web Proxy to allow overriding the host name for which the server accepts connections. It is similar to what have done in JournalNode and RM. > Add Bind_HOST configuration for Yarn Web Proxy > -- > > Key: YARN-9130 > URL: https://issues.apache.org/jira/browse/YARN-9130 > Project: Hadoop YARN > Issue Type: Improvement > Components: yarn >Affects Versions: 3.1.1 >Reporter: Rong Tang >Priority: Major > Attachments: Yarn-9130.001.patch > > > Allow configurable bind-host for Yarn Web Proxy to allow overriding the host > name for which the server accepts connections. > It is similar to what have done in JournalNode and RM. Like > https://issues.apache.org/jira/browse/HDFS-13462 > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-9130) Add Bind_HOST configuration for Yarn Web Proxy
Rong Tang created YARN-9130: --- Summary: Add Bind_HOST configuration for Yarn Web Proxy Key: YARN-9130 URL: https://issues.apache.org/jira/browse/YARN-9130 Project: Hadoop YARN Issue Type: Improvement Components: yarn Affects Versions: 3.1.1 Reporter: Rong Tang Allow configurable bind-host for Yarn Web Proxy to allow overriding the host name for which the server accepts connections. It is similar to what have done in JournalNode and RM. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9032) Support sh shell for interactive shell
[ https://issues.apache.org/jira/browse/YARN-9032?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16721823#comment-16721823 ] Billie Rinaldi commented on YARN-9032: -- +1 for patch 2. Thanks, [~eyang]! > Support sh shell for interactive shell > -- > > Key: YARN-9032 > URL: https://issues.apache.org/jira/browse/YARN-9032 > Project: Hadoop YARN > Issue Type: Sub-task > Components: yarn >Reporter: Eric Yang >Assignee: Eric Yang >Priority: Blocker > Attachments: YARN-9032.001.patch, YARN-9032.002.patch > > > CLI can supply second parameter such as: > yarn container -shell [container-id] sh > This is supposed to launch sh instead of bash. This is not fully working yet > in YARN-8778 patch and some organization of code clean up to support > non-docker container in YARN-8962 making this change in YARN-8778 difficult. > Hence, I file this as a separate sub-task to close the gap. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9127) Create more tests to verify GpuDeviceInformationParser
[
https://issues.apache.org/jira/browse/YARN-9127?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16721810#comment-16721810
]
Hadoop QA commented on YARN-9127:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
15s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 6 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 20m
5s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
4s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
29s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
41s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
11m 10s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
3s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
26s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
39s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
59s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
59s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
21s{color} | {color:green}
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager:
The patch generated 0 new + 9 unchanged - 5 fixed = 9 total (was 14) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
34s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} shellcheck {color} | {color:green} 0m
0s{color} | {color:green} There were no new shellcheck issues. {color} |
| {color:green}+1{color} | {color:green} shelldocs {color} | {color:green} 0m
13s{color} | {color:green} There were no new shelldocs issues. {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch 615 line(s) with tabs. {color} |
| {color:red}-1{color} | {color:red} xml {color} | {color:red} 0m 5s{color} |
{color:red} The patch has 4 ill-formed XML file(s). {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
12m 25s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
4s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
25s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 19m
15s{color} | {color:green} hadoop-yarn-server-nodemanager in the patch passed.
{color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
25s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 72m 50s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| XML | Parsing Error(s): |
| |
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/resources/nvidia-smi-output-excerpt.xml
|
| |
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/resources/nvidia-smi-output-missing-tags.xml
|
| |
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/resources/nvidia-smi-output-missing-tags2.xml
|
| |
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/resources/nvidia-smi-sample-output.xml
|
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hado
[jira] [Updated] (YARN-9129) Ensure flush after printing to stderr plus additional cleanup
[ https://issues.apache.org/jira/browse/YARN-9129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Yang updated YARN-9129: Issue Type: Sub-task (was: Improvement) Parent: YARN-8762 > Ensure flush after printing to stderr plus additional cleanup > - > > Key: YARN-9129 > URL: https://issues.apache.org/jira/browse/YARN-9129 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Billie Rinaldi >Priority: Major > Fix For: 3.3.0 > > > Following up on findings in YARN-8962, I noticed the following issues in > container-executor and main.c: > - There seem to be some vars that are not cleaned up in container_executor: > In run_docker else: free docker_binary > In exec_container: > before return INVALID_COMMAND_FILE: free docker_binary > 3x return DOCKER_EXEC_FAILED: set exit code and goto cleanup instead > cleanup needed before exit calls? > - In YARN-8777 we added several fprintf(stderr calls, but the convention in > container-executor.c appears to be fprintf(ERRORFILE followed by > fflush(ERRORFILE). > - There are leaks in TestDockerUtil_test_add_ports_mapping_to_command_Test. > - There are additional places where flush is not performed after writing to > stderr, including main.c display_feature_disabled_message. This can result in > the client not receiving the error message if the connection is closed too > quickly. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-9129) Ensure flush after printing to stderr plus additional cleanup
Billie Rinaldi created YARN-9129: Summary: Ensure flush after printing to stderr plus additional cleanup Key: YARN-9129 URL: https://issues.apache.org/jira/browse/YARN-9129 Project: Hadoop YARN Issue Type: Improvement Reporter: Billie Rinaldi Fix For: 3.3.0 Following up on findings in YARN-8962, I noticed the following issues in container-executor and main.c: - There seem to be some vars that are not cleaned up in container_executor: In run_docker else: free docker_binary In exec_container: before return INVALID_COMMAND_FILE: free docker_binary 3x return DOCKER_EXEC_FAILED: set exit code and goto cleanup instead cleanup needed before exit calls? - In YARN-8777 we added several fprintf(stderr calls, but the convention in container-executor.c appears to be fprintf(ERRORFILE followed by fflush(ERRORFILE). - There are leaks in TestDockerUtil_test_add_ports_mapping_to_command_Test. - There are additional places where flush is not performed after writing to stderr, including main.c display_feature_disabled_message. This can result in the client not receiving the error message if the connection is closed too quickly. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-9128) Use SerializationUtils from apache commons to serialize / deserialize ResourceMappings
[ https://issues.apache.org/jira/browse/YARN-9128?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Szilard Nemeth updated YARN-9128: - Attachment: YARN-9128.001.patch > Use SerializationUtils from apache commons to serialize / deserialize > ResourceMappings > -- > > Key: YARN-9128 > URL: https://issues.apache.org/jira/browse/YARN-9128 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: Szilard Nemeth >Assignee: Szilard Nemeth >Priority: Major > Attachments: YARN-9128.001.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-9128) Use SerializationUtils from apache commons to serialize / deserialize ResourceMappings
Szilard Nemeth created YARN-9128: Summary: Use SerializationUtils from apache commons to serialize / deserialize ResourceMappings Key: YARN-9128 URL: https://issues.apache.org/jira/browse/YARN-9128 Project: Hadoop YARN Issue Type: Improvement Reporter: Szilard Nemeth Assignee: Szilard Nemeth -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9125) Carriage Return character in launch command cause node manager to become unhealthy
[
https://issues.apache.org/jira/browse/YARN-9125?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16721773#comment-16721773
]
Hadoop QA commented on YARN-9125:
-
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
20s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 20m
35s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
5s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
26s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
40s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
13m 26s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
3s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
25s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
39s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
1s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
1s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 23s{color} | {color:orange}
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager:
The patch generated 1 new + 4 unchanged - 0 fixed = 5 total (was 4) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
35s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
13m 39s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
9s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
23s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 19m
16s{color} | {color:green} hadoop-yarn-server-nodemanager in the patch passed.
{color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
24s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 75m 34s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:8f97d6f |
| JIRA Issue | YARN-9125 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12951848/YARN-9125.1.patch |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient findbugs checkstyle |
| uname | Linux f690a37a8641 4.4.0-138-generic #164~14.04.1-Ubuntu SMP Fri Oct
5 08:56:16 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / 00d5e63 |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_181 |
| findbugs | v3.1.0-RC1 |
| checkstyle |
https://builds.apache.org/job/PreCommit-YARN-Build/22862/artifact/out/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/22862/testReport/ |
| Max. process+thread count | 323 (vs. ulimit of 1) |
| modules | C:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanag
[jira] [Updated] (YARN-9127) Create more tests to verify GpuDeviceInformationParser
[ https://issues.apache.org/jira/browse/YARN-9127?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Szilard Nemeth updated YARN-9127: - Attachment: YARN-9127.001.patch > Create more tests to verify GpuDeviceInformationParser > -- > > Key: YARN-9127 > URL: https://issues.apache.org/jira/browse/YARN-9127 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: Szilard Nemeth >Assignee: Szilard Nemeth >Priority: Major > Attachments: YARN-9127.001.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-9127) Create more tests to verify GpuDeviceInformationParser
Szilard Nemeth created YARN-9127: Summary: Create more tests to verify GpuDeviceInformationParser Key: YARN-9127 URL: https://issues.apache.org/jira/browse/YARN-9127 Project: Hadoop YARN Issue Type: Improvement Reporter: Szilard Nemeth Assignee: Szilard Nemeth -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-8963) Add flag to disable interactive shell
[ https://issues.apache.org/jira/browse/YARN-8963?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16721743#comment-16721743 ] Eric Yang commented on YARN-8963: - YARN UI compilation failure is not related patch 003. > Add flag to disable interactive shell > - > > Key: YARN-8963 > URL: https://issues.apache.org/jira/browse/YARN-8963 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Eric Yang >Assignee: Eric Yang >Priority: Major > Attachments: YARN-8963.001.patch, YARN-8963.002.patch, > YARN-8963.003.patch > > > For some production job, application admin might choose to disable debugging > to production jobs to prevent developer or system admin from accessing the > containers. It would be nice to add an environment variable flag to disable > interactive shell during application submission. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (YARN-9125) Carriage Return character in launch command cause node manager to become unhealthy
[ https://issues.apache.org/jira/browse/YARN-9125?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16721734#comment-16721734 ] Eric Yang edited comment on YARN-9125 at 12/14/18 7:42 PM: --- [~billie.rinaldi] Thank you for the patch. +1 patch looks good, pending precommit build report. Can you also provide patch for branch-3.0, and branch-2.9? Thanks was (Author: eyang): +1 patch looks good, pending precommit build report. > Carriage Return character in launch command cause node manager to become > unhealthy > -- > > Key: YARN-9125 > URL: https://issues.apache.org/jira/browse/YARN-9125 > Project: Hadoop YARN > Issue Type: Sub-task >Affects Versions: 2.9.0, 3.0.0, 3.1.0, 2.9.1, 3.0.2, 3.1.1, 2.9.2, 3.0.3 >Reporter: Eric Yang >Assignee: Billie Rinaldi >Priority: Major > Attachments: YARN-9125.1.patch > > > If launch command contains \n character, it can cause node manager to become > unhealthy. There is a protection check against this, but it doesn't seem to > be working as implemented. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9125) Carriage Return character in launch command cause node manager to become unhealthy
[ https://issues.apache.org/jira/browse/YARN-9125?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16721734#comment-16721734 ] Eric Yang commented on YARN-9125: - +1 patch looks good, pending precommit build report. > Carriage Return character in launch command cause node manager to become > unhealthy > -- > > Key: YARN-9125 > URL: https://issues.apache.org/jira/browse/YARN-9125 > Project: Hadoop YARN > Issue Type: Sub-task >Affects Versions: 2.9.0, 3.0.0, 3.1.0, 2.9.1, 3.0.2, 3.1.1, 2.9.2, 3.0.3 >Reporter: Eric Yang >Assignee: Billie Rinaldi >Priority: Major > Attachments: YARN-9125.1.patch > > > If launch command contains \n character, it can cause node manager to become > unhealthy. There is a protection check against this, but it doesn't seem to > be working as implemented. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-9125) Carriage Return character in launch command cause node manager to become unhealthy
[ https://issues.apache.org/jira/browse/YARN-9125?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Billie Rinaldi updated YARN-9125: - Attachment: YARN-9125.1.patch > Carriage Return character in launch command cause node manager to become > unhealthy > -- > > Key: YARN-9125 > URL: https://issues.apache.org/jira/browse/YARN-9125 > Project: Hadoop YARN > Issue Type: Sub-task >Affects Versions: 2.9.0, 3.0.0, 3.1.0, 2.9.1, 3.0.2, 3.1.1, 2.9.2, 3.0.3 >Reporter: Eric Yang >Assignee: Billie Rinaldi >Priority: Major > Attachments: YARN-9125.1.patch > > > If launch command contains \n character, it can cause node manager to become > unhealthy. There is a protection check against this, but it doesn't seem to > be working as implemented. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-8963) Add flag to disable interactive shell
[
https://issues.apache.org/jira/browse/YARN-8963?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16721697#comment-16721697
]
Hadoop QA commented on YARN-8963:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
23s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
44s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 21m
11s{color} | {color:green} trunk passed {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red} 5m
17s{color} | {color:red} hadoop-yarn in trunk failed. {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
11s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
40m 41s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
13s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
54s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 8m
10s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} cc {color} | {color:green} 8m
10s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 8m
10s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
8s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
13m 8s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 19m
16s{color} | {color:green} hadoop-yarn-server-nodemanager in the patch passed.
{color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
21s{color} | {color:green} hadoop-yarn-site in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
38s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 85m 40s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:8f97d6f |
| JIRA Issue | YARN-8963 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12951836/YARN-8963.003.patch |
| Optional Tests | dupname asflicense compile cc mvnsite javac unit |
| uname | Linux 11ffb194f8dd 4.4.0-138-generic #164~14.04.1-Ubuntu SMP Fri Oct
5 08:56:16 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / aa12859 |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_181 |
| compile |
https://builds.apache.org/job/PreCommit-YARN-Build/22861/artifact/out/branch-compile-hadoop-yarn-project_hadoop-yarn.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/22861/testReport/ |
| Max. process+thread count | 307 (vs. ulimit of 1) |
| modules | C:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site U:
hadoop-yarn-project/hadoop-yarn |
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/22861/console |
| Powered by | Apache Yetus 0.8.0 http://yetus.apache.org |
This message was automatically generated.
> Add flag to disable interactive shell
> -
[jira] [Commented] (YARN-9126) Container reinit always fails in branch-3.2 and trunk
[ https://issues.apache.org/jira/browse/YARN-9126?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16721666#comment-16721666 ] Eric Yang commented on YARN-9126: - We have a couple options here: Option 1 - Force file copy to overwrite existing files in container-executor. Option 2 - Delete the entire working directory, and allow the reinit process to reconstruct the whole thing. Option 3 - Delete the files that will be localized prior to launching container-executor. The only problem is the files are owned by application runner. This approach will require calling container-executor to delete the targeted list of files, then reinit again during relaunch. It's a more expensive process than option 1. The only rationale to go with this solution is to delete files as application runner to protect against overwriting other people's files. > Container reinit always fails in branch-3.2 and trunk > - > > Key: YARN-9126 > URL: https://issues.apache.org/jira/browse/YARN-9126 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Eric Yang >Priority: Major > Labels: docker > > When upgrading container, container reinitialization always failed with code > 33. This error code means the localizing file already exist while copying > resource files. The container will retry with another container ID, hence > the problem is masked. > Hadoop 3.1.x relaunch logic seem to have some way to prevent this bug from > happening. The same logic might be useful in branch 3.2 and trunk. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-9125) Carriage Return character in launch command cause node manager to become unhealthy
[ https://issues.apache.org/jira/browse/YARN-9125?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Yang updated YARN-9125: Affects Version/s: 2.9.0 3.0.0 2.9.1 3.0.2 2.9.2 3.0.3 Target Version/s: 3.2.0, 3.0.4, 3.1.2, 3.3.0, 2.9.3 (was: 3.2.0, 3.1.2, 3.3.0) > Carriage Return character in launch command cause node manager to become > unhealthy > -- > > Key: YARN-9125 > URL: https://issues.apache.org/jira/browse/YARN-9125 > Project: Hadoop YARN > Issue Type: Sub-task >Affects Versions: 2.9.0, 3.0.0, 3.1.0, 2.9.1, 3.0.2, 3.1.1, 2.9.2, 3.0.3 >Reporter: Eric Yang >Assignee: Billie Rinaldi >Priority: Major > > If launch command contains \n character, it can cause node manager to become > unhealthy. There is a protection check against this, but it doesn't seem to > be working as implemented. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9120) Need to have a way to turn off GPU auto-discovery in GpuDiscoverer
[ https://issues.apache.org/jira/browse/YARN-9120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16721650#comment-16721650 ] Wangda Tan commented on YARN-9120: -- [~snemeth] / [~tangzhankun], I prefer to make GPU plugin can be disabled/enabled as a whole. To me adding the new option makes harder to troubleshooting. And I'm not sure if there's any solid requirement to allow enable/disable GPU when the node is running, there're some logics of NM may prevent this as well. Just my $0.02. > Need to have a way to turn off GPU auto-discovery in GpuDiscoverer > -- > > Key: YARN-9120 > URL: https://issues.apache.org/jira/browse/YARN-9120 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: Szilard Nemeth >Assignee: Szilard Nemeth >Priority: Major > > GpuDiscoverer.getGpusUsableByYarn either parses the user-defined GPU devices > or should have the value 'auto' (from property: > yarn.nodemanager.resource-plugins.gpu.allowed-gpu-devices) > In some circumstances, users would want to exclude a node from scheduling, so > they should have an option to turn off auto-discovery. > It's straightforward that this is possible by removing the GPU > resource-plugin from YARN's config along with GPU-related config in > container-executor.cfg, but doing that with a dedicated value for > yarn.nodemanager.resource-plugins.gpu.allowed-gpu-devices is a more > lightweight approach. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-9125) Carriage Return character in launch command cause node manager to become unhealthy
[ https://issues.apache.org/jira/browse/YARN-9125?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Yang updated YARN-9125: Description: If launch command contains \n character, it can cause node manager to become unhealthy. There is a protection check against this, but it doesn't seem to be working as implemented. (was: If launch command contains \n character, it can cause node manager to crash. There is a protection check against this, but it doesn't seem to be working as implemented.) Summary: Carriage Return character in launch command cause node manager to become unhealthy (was: Carriage Return character in launch command cause node manager to crash) > Carriage Return character in launch command cause node manager to become > unhealthy > -- > > Key: YARN-9125 > URL: https://issues.apache.org/jira/browse/YARN-9125 > Project: Hadoop YARN > Issue Type: Sub-task >Affects Versions: 3.1.0, 3.1.1 >Reporter: Eric Yang >Assignee: Billie Rinaldi >Priority: Major > > If launch command contains \n character, it can cause node manager to become > unhealthy. There is a protection check against this, but it doesn't seem to > be working as implemented. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-9126) Container reinit always fails in branch-3.2 and trunk
Eric Yang created YARN-9126: --- Summary: Container reinit always fails in branch-3.2 and trunk Key: YARN-9126 URL: https://issues.apache.org/jira/browse/YARN-9126 Project: Hadoop YARN Issue Type: Sub-task Reporter: Eric Yang When upgrading container, container reinitialization always failed with code 33. This error code means the localizing file already exist while copying resource files. The container will retry with another container ID, hence the problem is masked. Hadoop 3.1.x relaunch logic seem to have some way to prevent this bug from happening. The same logic might be useful in branch 3.2 and trunk. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5168) Add port mapping handling when docker container use bridge network
[ https://issues.apache.org/jira/browse/YARN-5168?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16721637#comment-16721637 ] Eric Yang commented on YARN-5168: - [~liuxun323] Any update? > Add port mapping handling when docker container use bridge network > -- > > Key: YARN-5168 > URL: https://issues.apache.org/jira/browse/YARN-5168 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Jun Gong >Assignee: Xun Liu >Priority: Major > Labels: Docker > Attachments: YARN-5168.001.patch, YARN-5168.002.patch, > YARN-5168.003.patch, YARN-5168.004.patch, YARN-5168.005.patch, > YARN-5168.006.patch, YARN-5168.007.patch, YARN-5168.008.patch, > YARN-5168.009.patch, YARN-5168.010.patch > > > YARN-4007 addresses different network setups when launching the docker > container. We need support port mapping when docker container uses bridge > network. > The following problems are what we faced: > 1. Add "-P" to map docker container's exposed ports to automatically. > 2. Add "-p" to let user specify specific ports to map. > 3. Add service registry support for bridge network case, then app could find > each other. It could be done out of YARN, however it might be more convenient > to support it natively in YARN. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-9117) Container shell does not work when using yarn.nodemanager.linux-container-executor.nonsecure-mode.local-user is set
[ https://issues.apache.org/jira/browse/YARN-9117?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Yang updated YARN-9117: Attachment: YARN-9117.001.patch > Container shell does not work when using > yarn.nodemanager.linux-container-executor.nonsecure-mode.local-user is set > --- > > Key: YARN-9117 > URL: https://issues.apache.org/jira/browse/YARN-9117 > Project: Hadoop YARN > Issue Type: Sub-task >Affects Versions: 3.3.0 >Reporter: Eric Yang >Priority: Major > Attachments: YARN-9117.001.patch > > > If YARN is configured with > yarn.nodemanager.linux-container-executor.nonsecure-mode.local-user to > restrict YARN workload to run as a specific user only. Container shell does > not support this configuration because the workdir directory is owned by > local-user. The container shell is intended to launch a bash process owned > by the application owner. When bash process owner and current working > directory are mismatched. The child process will terminate immediately due > to no permission to WORKDIR. It is probably best to report this > configuration as not supported rather than allowing application owner to gain > all privileges of local-user. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-8963) Add flag to disable interactive shell
[ https://issues.apache.org/jira/browse/YARN-8963?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Yang updated YARN-8963: Attachment: YARN-8963.003.patch > Add flag to disable interactive shell > - > > Key: YARN-8963 > URL: https://issues.apache.org/jira/browse/YARN-8963 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Eric Yang >Assignee: Eric Yang >Priority: Major > Attachments: YARN-8963.001.patch, YARN-8963.002.patch, > YARN-8963.003.patch > > > For some production job, application admin might choose to disable debugging > to production jobs to prevent developer or system admin from accessing the > containers. It would be nice to add an environment variable flag to disable > interactive shell during application submission. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9116) Capacity Scheduler: add the default maximum-allocation-mb and maximum-allocation-vcores for the queues
[ https://issues.apache.org/jira/browse/YARN-9116?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16721603#comment-16721603 ] Aihua Xu commented on YARN-9116: [~cheersyang] Yes. Agree with you. I will implement toward such goal. > Capacity Scheduler: add the default maximum-allocation-mb and > maximum-allocation-vcores for the queues > -- > > Key: YARN-9116 > URL: https://issues.apache.org/jira/browse/YARN-9116 > Project: Hadoop YARN > Issue Type: Sub-task > Components: capacity scheduler >Affects Versions: 2.7.0 >Reporter: Aihua Xu >Assignee: Aihua Xu >Priority: Major > > YARN-1582 adds the support of maximum-allocation-mb configuration per queue > which is targeting to support larger container features on dedicated queues > (larger maximum-allocation-mb/maximum-allocation-vcores for such queue) . > While to achieve larger container configuration, we need to increase the > global maximum-allocation-mb/maximum-allocation-vcores (e.g. 120G/256) and > then override those configurations with desired values on the queues since > queue configuration can't be larger than cluster configuration. There are > many queues in the system and if we forget to configure such values when > adding a new queue, then such queue gets default 120G/256 which typically is > not what we want. > We can come up with a queue-default configuration (set to normal queue > configuration like 16G/8), so the leaf queues gets such values by default. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Assigned] (YARN-9125) Carriage Return character in launch command cause node manager to crash
[ https://issues.apache.org/jira/browse/YARN-9125?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Billie Rinaldi reassigned YARN-9125: Assignee: Billie Rinaldi > Carriage Return character in launch command cause node manager to crash > --- > > Key: YARN-9125 > URL: https://issues.apache.org/jira/browse/YARN-9125 > Project: Hadoop YARN > Issue Type: Sub-task >Affects Versions: 3.1.0, 3.1.1 >Reporter: Eric Yang >Assignee: Billie Rinaldi >Priority: Major > > If launch command contains \n character, it can cause node manager to crash. > There is a protection check against this, but it doesn't seem to be working > as implemented. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-9125) Carriage Return character in launch command cause node manager to crash
Eric Yang created YARN-9125: --- Summary: Carriage Return character in launch command cause node manager to crash Key: YARN-9125 URL: https://issues.apache.org/jira/browse/YARN-9125 Project: Hadoop YARN Issue Type: Sub-task Affects Versions: 3.1.1, 3.1.0 Reporter: Eric Yang If launch command contains \n character, it can cause node manager to crash. There is a protection check against this, but it doesn't seem to be working as implemented. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-8489) Need to support "dominant" component concept inside YARN service
[
https://issues.apache.org/jira/browse/YARN-8489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16721436#comment-16721436
]
Hadoop QA commented on YARN-8489:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
15s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
43s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 19m
3s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 7m
55s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
26s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 2m
20s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
15m 53s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
2s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
38s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
15s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
44s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red} 4m
15s{color} | {color:red} hadoop-yarn in the patch failed. {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 4m 15s{color}
| {color:red} hadoop-yarn in the patch failed. {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
1m 21s{color} | {color:orange} hadoop-yarn-project/hadoop-yarn: The patch
generated 2 new + 48 unchanged - 0 fixed = 50 total (was 48) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 2m
8s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
12m 23s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
18s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
30s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 89m 58s{color}
| {color:red} hadoop-yarn-server-resourcemanager in the patch failed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 15m 48s{color}
| {color:red} hadoop-yarn-services-core in the patch failed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
48s{color} | {color:green} hadoop-yarn-submarine in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
41s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}185m 2s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests |
hadoop.yarn.service.component.instance.TestComponentInstance |
| | hadoop.yarn.service.component.TestComponent |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:8f97d6f |
| JIRA Issue | YARN-8489 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12951801/YARN-8489.001.patch |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient findbugs checkstyle |
| uname | Linux 0dac1ecb31c0 4.4.0-138-generic #164-Ubuntu SMP Tue O
[jira] [Commented] (YARN-9101) Recovery Contaienr exitCode Not Right
[
https://issues.apache.org/jira/browse/YARN-9101?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16721395#comment-16721395
]
Hadoop QA commented on YARN-9101:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
17s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 18m
59s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
56s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
25s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
37s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
11m 57s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
57s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
25s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
36s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
54s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
54s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
20s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
31s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
12m 14s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
7s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
21s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 19m
3s{color} | {color:green} hadoop-yarn-server-nodemanager in the patch passed.
{color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
24s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 70m 2s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:8f97d6f |
| JIRA Issue | YARN-9101 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12951814/YARN-9101.patch |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient findbugs checkstyle |
| uname | Linux be84fb103178 4.4.0-139-generic #165-Ubuntu SMP Wed Oct 24
10:58:50 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / ca379e1 |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_181 |
| findbugs | v3.1.0-RC1 |
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/22860/testReport/ |
| Max. process+thread count | 443 (vs. ulimit of 1) |
| modules | C:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager
U:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager
|
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/22860/console |
| Powered by | Apache Yetus 0.8.0
[jira] [Commented] (YARN-9123) Clean up and split testcases in TestNMWebServices for GPU support
[
https://issues.apache.org/jira/browse/YARN-9123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16721334#comment-16721334
]
Hadoop QA commented on YARN-9123:
-
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
16s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 19m
37s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
2s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
28s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
40s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
12m 36s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
57s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
26s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
37s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
55s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
55s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 22s{color} | {color:orange}
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager:
The patch generated 1 new + 6 unchanged - 0 fixed = 7 total (was 6) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
34s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
12m 46s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
3s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
23s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 19m
32s{color} | {color:green} hadoop-yarn-server-nodemanager in the patch passed.
{color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
27s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 72m 49s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:8f97d6f |
| JIRA Issue | YARN-9123 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12951806/YARN-9123.001.patch |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient findbugs checkstyle |
| uname | Linux b8cf3086f125 4.4.0-138-generic #164-Ubuntu SMP Tue Oct 2
17:16:02 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / ca379e1 |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_191 |
| findbugs | v3.1.0-RC1 |
| checkstyle |
https://builds.apache.org/job/PreCommit-YARN-Build/22859/artifact/out/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/22859/testReport/ |
| Max. process+thread count | 444 (vs. ulimit of 1) |
| modules | C:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager
U:
[jira] [Updated] (YARN-9101) Recovery Contaienr exitCode Not Right
[ https://issues.apache.org/jira/browse/YARN-9101?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SuperbDong updated YARN-9101: - Attachment: (was: YARN-9101.patch) > Recovery Contaienr exitCode Not Right > - > > Key: YARN-9101 > URL: https://issues.apache.org/jira/browse/YARN-9101 > Project: Hadoop YARN > Issue Type: Bug > Components: nodemanager >Affects Versions: 3.0.0 >Reporter: SuperbDong >Priority: Major > Labels: pull-request-available > Attachments: YARN-9101.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-9101) Recovery Contaienr exitCode Not Right
[ https://issues.apache.org/jira/browse/YARN-9101?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SuperbDong updated YARN-9101: - Attachment: YARN-9101.patch > Recovery Contaienr exitCode Not Right > - > > Key: YARN-9101 > URL: https://issues.apache.org/jira/browse/YARN-9101 > Project: Hadoop YARN > Issue Type: Bug > Components: nodemanager >Affects Versions: 3.0.0 >Reporter: SuperbDong >Priority: Major > Labels: pull-request-available > Attachments: YARN-9101.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-9101) Recovery Contaienr exitCode Not Right
[ https://issues.apache.org/jira/browse/YARN-9101?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SuperbDong updated YARN-9101: - Attachment: (was: YARN-9101.patch) > Recovery Contaienr exitCode Not Right > - > > Key: YARN-9101 > URL: https://issues.apache.org/jira/browse/YARN-9101 > Project: Hadoop YARN > Issue Type: Bug > Components: nodemanager >Affects Versions: 3.0.0 >Reporter: SuperbDong >Priority: Major > Labels: pull-request-available > Attachments: YARN-9101.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-9101) Recovery Contaienr exitCode Not Right
[ https://issues.apache.org/jira/browse/YARN-9101?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SuperbDong updated YARN-9101: - Affects Version/s: (was: 2.9.0) (was: 2.8.0) (was: 2.7.0) > Recovery Contaienr exitCode Not Right > - > > Key: YARN-9101 > URL: https://issues.apache.org/jira/browse/YARN-9101 > Project: Hadoop YARN > Issue Type: Bug > Components: nodemanager >Affects Versions: 3.0.0 >Reporter: SuperbDong >Priority: Major > Labels: pull-request-available > Attachments: YARN-9101.patch, YARN-9101.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Assigned] (YARN-9119) Clean up testcase TestGpuDiscoverer.testLinuxGpuResourceDiscoverPluginConfig
[ https://issues.apache.org/jira/browse/YARN-9119?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Szilard Nemeth reassigned YARN-9119: Assignee: Alex Bodo > Clean up testcase TestGpuDiscoverer.testLinuxGpuResourceDiscoverPluginConfig > > > Key: YARN-9119 > URL: https://issues.apache.org/jira/browse/YARN-9119 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: Szilard Nemeth >Assignee: Alex Bodo >Priority: Major > Labels: newbie, newbie++ > > This testcase should be separated to 3 different testcases, as the comment > says in the code. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Assigned] (YARN-9124) Resolve contradiction in ResourceUtils: addMandatoryResources / checkMandatoryResources work differently
[
https://issues.apache.org/jira/browse/YARN-9124?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Szilard Nemeth reassigned YARN-9124:
Assignee: Alex Bodo
> Resolve contradiction in ResourceUtils: addMandatoryResources /
> checkMandatoryResources work differently
>
>
> Key: YARN-9124
> URL: https://issues.apache.org/jira/browse/YARN-9124
> Project: Hadoop YARN
> Issue Type: Improvement
>Reporter: Szilard Nemeth
>Assignee: Alex Bodo
>Priority: Minor
>
> {{ResourceUtils#addMandatoryResources}}: Adds only memory and vcores as
> mandatory resources.
> {{ResourceUtils#checkMandatoryResources}}: YARN-6620 added some code to this.
> This method not only checks memory and vcores, but all the resources referred
> in ResourceInformation#MANDATORY_RESOURCES.
> I think it would be good to call {{MANDATORY_RESOURCES}} as
> {{PREDEFINED_RESOURCES}} or something like that and use a similar name for
> {{checkMandatoryResources}}.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Assigned] (YARN-9094) Remove unused interface method: NodeResourceUpdaterPlugin#handleUpdatedResourceFromRM
[ https://issues.apache.org/jira/browse/YARN-9094?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Szilard Nemeth reassigned YARN-9094: Assignee: Alex Bodo > Remove unused interface method: > NodeResourceUpdaterPlugin#handleUpdatedResourceFromRM > - > > Key: YARN-9094 > URL: https://issues.apache.org/jira/browse/YARN-9094 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: Szilard Nemeth >Assignee: Alex Bodo >Priority: Trivial > Labels: newbie, newbie++ > > Additionally, there's a typo can be fixed in the javadoc of > NodeResourceUpdaterPlugin#updateConfiguredResource: look for "mododule" -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-9124) Resolve contradiction in ResourceUtils: addMandatoryResources / checkMandatoryResources work differently
Szilard Nemeth created YARN-9124:
Summary: Resolve contradiction in ResourceUtils:
addMandatoryResources / checkMandatoryResources work differently
Key: YARN-9124
URL: https://issues.apache.org/jira/browse/YARN-9124
Project: Hadoop YARN
Issue Type: Improvement
Reporter: Szilard Nemeth
{{ResourceUtils#addMandatoryResources}}: Adds only memory and vcores as
mandatory resources.
{{ResourceUtils#checkMandatoryResources}}: YARN-6620 added some code to this.
This method not only checks memory and vcores, but all the resources referred
in ResourceInformation#MANDATORY_RESOURCES.
I think it would be good to call {{MANDATORY_RESOURCES}} as
{{PREDEFINED_RESOURCES}} or something like that and use a similar name for
{{checkMandatoryResources}}.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-9123) Clean up and split testcases in TestNMWebServices for GPU support
[ https://issues.apache.org/jira/browse/YARN-9123?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Szilard Nemeth updated YARN-9123: - Attachment: YARN-9123.001.patch > Clean up and split testcases in TestNMWebServices for GPU support > - > > Key: YARN-9123 > URL: https://issues.apache.org/jira/browse/YARN-9123 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: Szilard Nemeth >Assignee: Szilard Nemeth >Priority: Minor > Attachments: YARN-9123.001.patch > > > The following testcases can be cleaned up a bit: > TestNMWebServices#testGetNMResourceInfo - Can be split up to 3 different cases > TestNMWebServices#testGetYarnGpuResourceInfo -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-9123) Clean up and split testcases in TestNMWebServices for GPU support
Szilard Nemeth created YARN-9123: Summary: Clean up and split testcases in TestNMWebServices for GPU support Key: YARN-9123 URL: https://issues.apache.org/jira/browse/YARN-9123 Project: Hadoop YARN Issue Type: Improvement Reporter: Szilard Nemeth Assignee: Szilard Nemeth The following testcases can be cleaned up a bit: TestNMWebServices#testGetNMResourceInfo - Can be split up to 3 different cases TestNMWebServices#testGetYarnGpuResourceInfo -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9120) Need to have a way to turn off GPU auto-discovery in GpuDiscoverer
[
https://issues.apache.org/jira/browse/YARN-9120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16721237#comment-16721237
]
Zhankun Tang commented on YARN-9120:
{quote}1. Could you please confirm whether only removing the resource plugin
from yarn-site.xml and keeping GPU-related config in container-executor.cfg
would not break anything as a regression on nodes that don't have any GPU?
{quote}
Zhankun => Yeah. The plugin instance won't be added into ResourceHandlerChain
so any GPU related code shouldn't be executed. I'll double-check next week.
{quote}2. Do you think it's still worth to create a jira under YARN-8851 that
handles switching the GPU plugin on/off dynamically (at runtime)? If yes, I'm
fine either creating it or giving this task to you.
{quote}
Zhankun => The YARN-8851 is not mature yet. Once it's muture, we can port
existing GPU plugin to that new framework. In that case, the new GPU plugin
could be more flexible. For instance, report 0 devices based on its own
policy/configuration. And YARN doesn't need to manage specific vendor device
plugin configurations.
For support the disabling GPU devices in a node at runtime, I'm not quit sure
what's the scenarion it fits in. Any idea? [~snemeth], [~leftnoteasy]
> Need to have a way to turn off GPU auto-discovery in GpuDiscoverer
> --
>
> Key: YARN-9120
> URL: https://issues.apache.org/jira/browse/YARN-9120
> Project: Hadoop YARN
> Issue Type: Improvement
>Reporter: Szilard Nemeth
>Assignee: Szilard Nemeth
>Priority: Major
>
> GpuDiscoverer.getGpusUsableByYarn either parses the user-defined GPU devices
> or should have the value 'auto' (from property:
> yarn.nodemanager.resource-plugins.gpu.allowed-gpu-devices)
> In some circumstances, users would want to exclude a node from scheduling, so
> they should have an option to turn off auto-discovery.
> It's straightforward that this is possible by removing the GPU
> resource-plugin from YARN's config along with GPU-related config in
> container-executor.cfg, but doing that with a dedicated value for
> yarn.nodemanager.resource-plugins.gpu.allowed-gpu-devices is a more
> lightweight approach.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-9101) Recovery Contaienr exitCode Not Right
[ https://issues.apache.org/jira/browse/YARN-9101?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SuperbDong updated YARN-9101: - Target Version/s: 3.0.0 > Recovery Contaienr exitCode Not Right > - > > Key: YARN-9101 > URL: https://issues.apache.org/jira/browse/YARN-9101 > Project: Hadoop YARN > Issue Type: Bug > Components: nodemanager >Affects Versions: 2.7.0, 2.8.0, 2.9.0, 3.0.0 >Reporter: SuperbDong >Priority: Major > Labels: pull-request-available > Attachments: YARN-9101.patch, YARN-9101.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9120) Need to have a way to turn off GPU auto-discovery in GpuDiscoverer
[ https://issues.apache.org/jira/browse/YARN-9120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16721194#comment-16721194 ] Szilard Nemeth commented on YARN-9120: -- [~tangzhankun]: I agree that the runtime ability of replace configs would introduce more and more complexity into NM, as we would need to handle if the GPUs are disabled gracefully, when a container already has some mapping for those GPUs. I'm becoming more convinced that this jira should not happen in its described form, so I'm fine not doing that now, given that we already have YARN-8851 as on ongoing change. 1. The property value would have been "off" 2. Fair enough 3. I think it's perfectly valid if users want to have heterogeneous clusters so that the nodes don't have GPUs are not having the GPU support related configurations in yarn-site.xml and in container-executor.cfg. Two questions: 1. Could you please confirm whether only removing the resource plugin from yarn-site.xml and keeping GPU-related config in container-executor.cfg would not break anything as a regression on nodes that don't have any GPU? 2. Do you think it's still worth to create a jira under YARN-8851 that handles switching the GPU plugin on/off dynamically (at runtime)? If yes, I'm fine either creating it or giving this task to you. Thanks! > Need to have a way to turn off GPU auto-discovery in GpuDiscoverer > -- > > Key: YARN-9120 > URL: https://issues.apache.org/jira/browse/YARN-9120 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: Szilard Nemeth >Assignee: Szilard Nemeth >Priority: Major > > GpuDiscoverer.getGpusUsableByYarn either parses the user-defined GPU devices > or should have the value 'auto' (from property: > yarn.nodemanager.resource-plugins.gpu.allowed-gpu-devices) > In some circumstances, users would want to exclude a node from scheduling, so > they should have an option to turn off auto-discovery. > It's straightforward that this is possible by removing the GPU > resource-plugin from YARN's config along with GPU-related config in > container-executor.cfg, but doing that with a dedicated value for > yarn.nodemanager.resource-plugins.gpu.allowed-gpu-devices is a more > lightweight approach. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9122) Add table of contents to YARN Service API document
[
https://issues.apache.org/jira/browse/YARN-9122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16721101#comment-16721101
]
Hadoop QA commented on YARN-9122:
-
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
14s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 23m
34s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
20s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
36m 3s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
14s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
18s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
13m 29s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
25s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 51m 14s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:8f97d6f |
| JIRA Issue | YARN-9122 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12951784/YARN-9122-trunk.001.patch
|
| Optional Tests | dupname asflicense mvnsite |
| uname | Linux 357fd7f22da4 4.4.0-134-generic #160~14.04.1-Ubuntu SMP Fri Aug
17 11:07:07 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / ca379e1 |
| maven | version: Apache Maven 3.3.9 |
| Max. process+thread count | 306 (vs. ulimit of 1) |
| modules | C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site U:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site |
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/22857/console |
| Powered by | Apache Yetus 0.8.0 http://yetus.apache.org |
This message was automatically generated.
> Add table of contents to YARN Service API document
> --
>
> Key: YARN-9122
> URL: https://issues.apache.org/jira/browse/YARN-9122
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: documentation
>Reporter: Akira Ajisaka
>Assignee: Zhankun Tang
>Priority: Minor
> Labels: newbie
> Attachments: YARN-9122-trunk.001.patch, YARN-9122-trunk.001.patch,
> s1.png
>
>
> I think it's better to have a table of contents in
> https://hadoop.apache.org/docs/r3.1.1/hadoop-yarn/hadoop-yarn-site/yarn-service/YarnServiceAPI.html
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (YARN-9036) Escape newlines in health report in YARN UI
[ https://issues.apache.org/jira/browse/YARN-9036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16720898#comment-16720898 ] tianjuan edited comment on YARN-9036 at 12/14/18 9:22 AM: -- Hi, the nodes page failed to show up with this patch(2.9 branch), the error is shown as below, the nodeTableData is just like below. @Jonathan Hung any idea about this? var nodeTableData=[\n[\"\",\"/default-rack\",\"RUNNING\",\"XXX:10025\",\"XXX:8042\",\"Fri Dec 14 11:24:23 +0800 2018\",\"\",\"0\",\"0 B\",\"8 GB\",\"0\",\"8\",\"0\",\"0 B\",\"0\",\"0\",\"2.9.3-SNAPSHOT\"]\n] !image-2018-12-14-11-33-54-361.png! with the given example, nodeTableData = ["","/IP","UNHEALTHY","host:8041","host:8042","Tue Nov 20 01:07:08 + 2018","ERROR -> /dev/sdb - modules.DISK FAILED | OK -> User:yarn,modules.CPU PASS,modules.RAM PASS,modules.PROCESSES PASS,modules.NET PASS,modules.TMP_FULL PASS ;1/2 local-dirs have errors: [ /tmp/yarn : Directory is not writable: /tmp/yarn ] 1/2 log-dirs have errors: [ /tmp/userlogs : Directory is not writable: /tmp/userlogs ] ","0","0 B","0 B","0","0","2.9.0.48"], ...] after StringEscapeUtils.escapeJava, nodeTableDataEscaped = [\"\",\"/IP\",\"UNHEALTHY\",\"host:8041\",\"host:8042\",\"Tue Nov 20 01:07:08 + 2018\",\"ERROR -> /dev/sdb - modules.DISK FAILED | OK -> User:yarn,modules.CPU PASS,modules.RAM PASS,modules.PROCESSES PASS,modules.NET PASS,modules.TMP_FULL PASS\r\n\r\n;1/2 local-dirs have errors: [ /tmp/yarn : Directory is not writable: /tmp/yarn ] 1/2 log-dirs have errors: [ /tmp/userlogs : Directory is not writable: /tmp/userlogs ] \",\"0\",\"0 B\",\"0 B\",\"0\",\"0\",\"2.9.0.48\"],\r\n...] was (Author: jutia): Hi, the nodes page failed to show up with this patch(2.9 branch), the error is shown as below, the nodeTableData is just like below. @Jonathan Hung any idea about this? var nodeTableData=[\n[\"\",\"/default-rack\",\"RUNNING\",\"XXX:10025\",\"XXX:8042\",\"Fri Dec 14 11:24:23 +0800 2018\",\"\",\"0\",\"0 B\",\"8 GB\",\"0\",\"8\",\"0\",\"0 B\",\"0\",\"0\",\"2.9.3-SNAPSHOT\"]\n] !image-2018-12-14-11-33-54-361.png! with the given example, nodeTableData = "[\"\",\"/IP\",\"UNHEALTHY\",\"host:8041\",\"host:8042\",\"Tue Nov 20 01:07:08 + 2018\",\"ERROR -> /dev/sdb - modules.DISK FAILED | OK -> User:yarn,modules.CPU PASS,modules.RAM PASS,modules.PROCESSES PASS,modules.NET PASS,modules.TMP_FULL PASS\r\n" + "\r\n" + ";1/2 local-dirs have errors: [ /tmp/yarn : Directory is not writable: /tmp/yarn ] 1/2 log-dirs have errors: [ /tmp/userlogs : Directory is not writable: /tmp/userlogs ] \",\"0\",\"0 B\",\"0 B\",\"0\",\"0\",\"2.9.0.48\"],\r\n" + "...]"; after StringEscapeUtils.escapeJava, nodeTableDataEscaped = [\"\",\"/IP\",\"UNHEALTHY\",\"host:8041\",\"host:8042\",\"Tue Nov 20 01:07:08 + 2018\",\"ERROR -> /dev/sdb - modules.DISK FAILED | OK -> User:yarn,modules.CPU PASS,modules.RAM PASS,modules.PROCESSES PASS,modules.NET PASS,modules.TMP_FULL PASS\r\n\r\n;1/2 local-dirs have errors: [ /tmp/yarn : Directory is not writable: /tmp/yarn ] 1/2 log-dirs have errors: [ /tmp/userlogs : Directory is not writable: /tmp/userlogs ] \",\"0\",\"0 B\",\"0 B\",\"0\",\"0\",\"2.9.0.48\"],\r\n...] > Escape newlines in health report in YARN UI > --- > > Key: YARN-9036 > URL: https://issues.apache.org/jira/browse/YARN-9036 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: Jonathan Hung >Assignee: Keqiu Hu >Priority: Major > Fix For: 2.10.0, 3.0.4, 3.1.2, 3.3.0, 3.2.1, 2.9.3 > > Attachments: YARN-9036.001.patch, YARN-9036.002.patch, > YARN-9036.003.patch, YARN-9036.003.patch, YARN-9036.004.patch, > image-2018-12-14-11-33-54-361.png > > > NodesPage prints health report info in the UI in a javascript string. If > health report contains newlines it will garble the generated code and the > list of nodes cannot be rendered. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (YARN-9036) Escape newlines in health report in YARN UI
[ https://issues.apache.org/jira/browse/YARN-9036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16720898#comment-16720898 ] tianjuan edited comment on YARN-9036 at 12/14/18 9:21 AM: -- Hi, the nodes page failed to show up with this patch(2.9 branch), the error is shown as below, the nodeTableData is just like below. @Jonathan Hung any idea about this? var nodeTableData=[\n[\"\",\"/default-rack\",\"RUNNING\",\"XXX:10025\",\"XXX:8042\",\"Fri Dec 14 11:24:23 +0800 2018\",\"\",\"0\",\"0 B\",\"8 GB\",\"0\",\"8\",\"0\",\"0 B\",\"0\",\"0\",\"2.9.3-SNAPSHOT\"]\n] !image-2018-12-14-11-33-54-361.png! with the given example, nodeTableData = "[\"\",\"/IP\",\"UNHEALTHY\",\"host:8041\",\"host:8042\",\"Tue Nov 20 01:07:08 + 2018\",\"ERROR -> /dev/sdb - modules.DISK FAILED | OK -> User:yarn,modules.CPU PASS,modules.RAM PASS,modules.PROCESSES PASS,modules.NET PASS,modules.TMP_FULL PASS\r\n" + "\r\n" + ";1/2 local-dirs have errors: [ /tmp/yarn : Directory is not writable: /tmp/yarn ] 1/2 log-dirs have errors: [ /tmp/userlogs : Directory is not writable: /tmp/userlogs ] \",\"0\",\"0 B\",\"0 B\",\"0\",\"0\",\"2.9.0.48\"],\r\n" + "...]"; after StringEscapeUtils.escapeJava, nodeTableDataEscaped = [\"\",\"/IP\",\"UNHEALTHY\",\"host:8041\",\"host:8042\",\"Tue Nov 20 01:07:08 + 2018\",\"ERROR -> /dev/sdb - modules.DISK FAILED | OK -> User:yarn,modules.CPU PASS,modules.RAM PASS,modules.PROCESSES PASS,modules.NET PASS,modules.TMP_FULL PASS\r\n\r\n;1/2 local-dirs have errors: [ /tmp/yarn : Directory is not writable: /tmp/yarn ] 1/2 log-dirs have errors: [ /tmp/userlogs : Directory is not writable: /tmp/userlogs ] \",\"0\",\"0 B\",\"0 B\",\"0\",\"0\",\"2.9.0.48\"],\r\n...] was (Author: jutia): Hi, the nodes page failed to show up with this patch(2.9 branch), the error is shown as below, the nodeTableData is just like below. @Jonathan Hung any idea about this? var nodeTableData=[\n[\"\",\"/default-rack\",\"RUNNING\",\"XXX:10025\",\"XXX:8042\",\"Fri Dec 14 11:24:23 +0800 2018\",\"\",\"0\",\"0 B\",\"8 GB\",\"0\",\"8\",\"0\",\"0 B\",\"0\",\"0\",\"2.9.3-SNAPSHOT\"]\n] !image-2018-12-14-11-33-54-361.png! > Escape newlines in health report in YARN UI > --- > > Key: YARN-9036 > URL: https://issues.apache.org/jira/browse/YARN-9036 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: Jonathan Hung >Assignee: Keqiu Hu >Priority: Major > Fix For: 2.10.0, 3.0.4, 3.1.2, 3.3.0, 3.2.1, 2.9.3 > > Attachments: YARN-9036.001.patch, YARN-9036.002.patch, > YARN-9036.003.patch, YARN-9036.003.patch, YARN-9036.004.patch, > image-2018-12-14-11-33-54-361.png > > > NodesPage prints health report info in the UI in a javascript string. If > health report contains newlines it will garble the generated code and the > list of nodes cannot be rendered. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9116) Capacity Scheduler: add the default maximum-allocation-mb and maximum-allocation-vcores for the queues
[ https://issues.apache.org/jira/browse/YARN-9116?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16721066#comment-16721066 ] Weiwei Yang commented on YARN-9116: --- The logic to inherit should not be a problem. I mean we could still hornor global value if it is set (the old logic not changed). If global value is not set, then we just enforce the check with the queue level configs, and that allows a leaf queue has its own max. Would that make our configuration simpler? I have no problem to add a new config if we run out of options, given we already have a lot of configurations. > Capacity Scheduler: add the default maximum-allocation-mb and > maximum-allocation-vcores for the queues > -- > > Key: YARN-9116 > URL: https://issues.apache.org/jira/browse/YARN-9116 > Project: Hadoop YARN > Issue Type: Sub-task > Components: capacity scheduler >Affects Versions: 2.7.0 >Reporter: Aihua Xu >Assignee: Aihua Xu >Priority: Major > > YARN-1582 adds the support of maximum-allocation-mb configuration per queue > which is targeting to support larger container features on dedicated queues > (larger maximum-allocation-mb/maximum-allocation-vcores for such queue) . > While to achieve larger container configuration, we need to increase the > global maximum-allocation-mb/maximum-allocation-vcores (e.g. 120G/256) and > then override those configurations with desired values on the queues since > queue configuration can't be larger than cluster configuration. There are > many queues in the system and if we forget to configure such values when > adding a new queue, then such queue gets default 120G/256 which typically is > not what we want. > We can come up with a queue-default configuration (set to normal queue > configuration like 16G/8), so the leaf queues gets such values by default. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (YARN-9116) Capacity Scheduler: add the default maximum-allocation-mb and maximum-allocation-vcores for the queues
[ https://issues.apache.org/jira/browse/YARN-9116?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16721066#comment-16721066 ] Weiwei Yang edited comment on YARN-9116 at 12/14/18 8:50 AM: - Hi [~aihuaxu] The logic to inherit should not be a problem. I mean we could still hornor global value if it is set (the old logic not changed). If global value is not set, then we just enforce the check with the queue level configs, and that allows a leaf queue has its own max. Would that make our configuration simpler? I have no problem to add a new config if we run out of options, given we already have a lot of configurations. was (Author: cheersyang): The logic to inherit should not be a problem. I mean we could still hornor global value if it is set (the old logic not changed). If global value is not set, then we just enforce the check with the queue level configs, and that allows a leaf queue has its own max. Would that make our configuration simpler? I have no problem to add a new config if we run out of options, given we already have a lot of configurations. > Capacity Scheduler: add the default maximum-allocation-mb and > maximum-allocation-vcores for the queues > -- > > Key: YARN-9116 > URL: https://issues.apache.org/jira/browse/YARN-9116 > Project: Hadoop YARN > Issue Type: Sub-task > Components: capacity scheduler >Affects Versions: 2.7.0 >Reporter: Aihua Xu >Assignee: Aihua Xu >Priority: Major > > YARN-1582 adds the support of maximum-allocation-mb configuration per queue > which is targeting to support larger container features on dedicated queues > (larger maximum-allocation-mb/maximum-allocation-vcores for such queue) . > While to achieve larger container configuration, we need to increase the > global maximum-allocation-mb/maximum-allocation-vcores (e.g. 120G/256) and > then override those configurations with desired values on the queues since > queue configuration can't be larger than cluster configuration. There are > many queues in the system and if we forget to configure such values when > adding a new queue, then such queue gets default 120G/256 which typically is > not what we want. > We can come up with a queue-default configuration (set to normal queue > configuration like 16G/8), so the leaf queues gets such values by default. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9113) [Submarine] Add proper shutdown hook when user interrupts the client
[
https://issues.apache.org/jira/browse/YARN-9113?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16721056#comment-16721056
]
Hadoop QA commented on YARN-9113:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
22s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 21m
53s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
25s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
16s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
29s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
12m 55s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
35s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
19s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
27s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
12s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
25s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
14m 5s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
50s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
20s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
39s{color} | {color:green} hadoop-yarn-submarine in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
30s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 55m 36s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:8f97d6f |
| JIRA Issue | YARN-9113 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12951782/YARN-9113-trunk.002.patch
|
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient findbugs checkstyle |
| uname | Linux 0cb8db8b6d60 4.4.0-138-generic #164~14.04.1-Ubuntu SMP Fri Oct
5 08:56:16 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / ca379e1 |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_181 |
| findbugs | v3.1.0-RC1 |
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/22856/testReport/ |
| Max. process+thread count | 308 (vs. ulimit of 1) |
| modules | C:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-submarine
U:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-submarine |
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/22856/console |
| Powered by | Apache Yetus 0.8.0
[jira] [Comment Edited] (YARN-9122) Add table of contents to YARN Service API document
[ https://issues.apache.org/jira/browse/YARN-9122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16721053#comment-16721053 ] Akira Ajisaka edited comment on YARN-9122 at 12/14/18 8:31 AM: --- LGTM, +1 pending Jenkins. was (Author: ajisakaa): LGTM > Add table of contents to YARN Service API document > -- > > Key: YARN-9122 > URL: https://issues.apache.org/jira/browse/YARN-9122 > Project: Hadoop YARN > Issue Type: Improvement > Components: documentation >Reporter: Akira Ajisaka >Priority: Minor > Labels: newbie > Attachments: YARN-9122-trunk.001.patch, YARN-9122-trunk.001.patch, > s1.png > > > I think it's better to have a table of contents in > https://hadoop.apache.org/docs/r3.1.1/hadoop-yarn/hadoop-yarn-site/yarn-service/YarnServiceAPI.html -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Assigned] (YARN-9122) Add table of contents to YARN Service API document
[ https://issues.apache.org/jira/browse/YARN-9122?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akira Ajisaka reassigned YARN-9122: --- Assignee: Zhankun Tang > Add table of contents to YARN Service API document > -- > > Key: YARN-9122 > URL: https://issues.apache.org/jira/browse/YARN-9122 > Project: Hadoop YARN > Issue Type: Improvement > Components: documentation >Reporter: Akira Ajisaka >Assignee: Zhankun Tang >Priority: Minor > Labels: newbie > Attachments: YARN-9122-trunk.001.patch, YARN-9122-trunk.001.patch, > s1.png > > > I think it's better to have a table of contents in > https://hadoop.apache.org/docs/r3.1.1/hadoop-yarn/hadoop-yarn-site/yarn-service/YarnServiceAPI.html -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9122) Add table of contents to YARN Service API document
[ https://issues.apache.org/jira/browse/YARN-9122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16721053#comment-16721053 ] Akira Ajisaka commented on YARN-9122: - LGTM > Add table of contents to YARN Service API document > -- > > Key: YARN-9122 > URL: https://issues.apache.org/jira/browse/YARN-9122 > Project: Hadoop YARN > Issue Type: Improvement > Components: documentation >Reporter: Akira Ajisaka >Priority: Minor > Labels: newbie > Attachments: YARN-9122-trunk.001.patch, YARN-9122-trunk.001.patch, > s1.png > > > I think it's better to have a table of contents in > https://hadoop.apache.org/docs/r3.1.1/hadoop-yarn/hadoop-yarn-site/yarn-service/YarnServiceAPI.html -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-9122) Add table of contents to YARN Service API document
[ https://issues.apache.org/jira/browse/YARN-9122?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akira Ajisaka updated YARN-9122: Attachment: YARN-9122-trunk.001.patch > Add table of contents to YARN Service API document > -- > > Key: YARN-9122 > URL: https://issues.apache.org/jira/browse/YARN-9122 > Project: Hadoop YARN > Issue Type: Improvement > Components: documentation >Reporter: Akira Ajisaka >Priority: Minor > Labels: newbie > Attachments: YARN-9122-trunk.001.patch, YARN-9122-trunk.001.patch, > s1.png > > > I think it's better to have a table of contents in > https://hadoop.apache.org/docs/r3.1.1/hadoop-yarn/hadoop-yarn-site/yarn-service/YarnServiceAPI.html -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (YARN-9122) Add table of contents to YARN Service API document
[ https://issues.apache.org/jira/browse/YARN-9122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16721042#comment-16721042 ] Akira Ajisaka edited comment on YARN-9122 at 12/14/18 8:16 AM: --- Attaching the same patch to run the precommit job. The precommit job always picks up the latest attachment, so s1.png had been used. was (Author: ajisakaa): Attaching a patch to run the precommit job. The precommit job always picks up the latest attachment, so s1.png had been used. > Add table of contents to YARN Service API document > -- > > Key: YARN-9122 > URL: https://issues.apache.org/jira/browse/YARN-9122 > Project: Hadoop YARN > Issue Type: Improvement > Components: documentation >Reporter: Akira Ajisaka >Priority: Minor > Labels: newbie > Attachments: YARN-9122-trunk.001.patch, YARN-9122-trunk.001.patch, > s1.png > > > I think it's better to have a table of contents in > https://hadoop.apache.org/docs/r3.1.1/hadoop-yarn/hadoop-yarn-site/yarn-service/YarnServiceAPI.html -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
