[jira] [Updated] (YARN-10160) Add auto queue creation related configs to RMWebService#CapacitySchedulerQueueInfo
[
https://issues.apache.org/jira/browse/YARN-10160?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Prabhu Joseph updated YARN-10160:
-
Attachment: YARN-10160-002.patch
> Add auto queue creation related configs to
> RMWebService#CapacitySchedulerQueueInfo
> --
>
> Key: YARN-10160
> URL: https://issues.apache.org/jira/browse/YARN-10160
> Project: Hadoop YARN
> Issue Type: Improvement
>Affects Versions: 3.3.0
>Reporter: Prabhu Joseph
>Assignee: Prabhu Joseph
>Priority: Major
> Attachments: Screen Shot 2020-02-25 at 9.06.52 PM.png,
> YARN-10160-001.patch, YARN-10160-002.patch
>
>
> Add auto queue creation related configs to
> RMWebService#CapacitySchedulerQueueInfo.
> {code}
> yarn.scheduler.capacity..auto-create-child-queue.enabled
> yarn.scheduler.capacity..leaf-queue-template.
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10160) Add auto queue creation related configs to RMWebService#CapacitySchedulerQueueInfo
[
https://issues.apache.org/jira/browse/YARN-10160?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Prabhu Joseph updated YARN-10160:
-
Attachment: (was: YARN-10124-002.patch)
> Add auto queue creation related configs to
> RMWebService#CapacitySchedulerQueueInfo
> --
>
> Key: YARN-10160
> URL: https://issues.apache.org/jira/browse/YARN-10160
> Project: Hadoop YARN
> Issue Type: Improvement
>Affects Versions: 3.3.0
>Reporter: Prabhu Joseph
>Assignee: Prabhu Joseph
>Priority: Major
> Attachments: Screen Shot 2020-02-25 at 9.06.52 PM.png,
> YARN-10160-001.patch, YARN-10160-002.patch
>
>
> Add auto queue creation related configs to
> RMWebService#CapacitySchedulerQueueInfo.
> {code}
> yarn.scheduler.capacity..auto-create-child-queue.enabled
> yarn.scheduler.capacity..leaf-queue-template.
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10167) FS-CS Converter: Need validate c-s.xml after converting
[ https://issues.apache.org/jira/browse/YARN-10167?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17046238#comment-17046238 ] Sunil G commented on YARN-10167: Trying to understand deeper level here * Submit a new c-s.xml from CLI for validation (may be cli may look like: *yarn queues validate -f c-s.xml*) * CLI uploads the c-s.xml as a config object to RM * In RM, we will be having a method in ClientRMService called *validateCSConfig* * In this method, we will be able to get a new CS object itself, and inject the incoming conf object. ** 1. If CS can be inited, validation seems fine (similar to what [~kmarton] did earlier) ** 2. Can we start CS also ?? I dont think so. In this case, we may need MockRM instead of CS object. But question here is that, what all configs again get validated more when we call start. If this delta is minimum, I would like to pull that out and call explicitly. cc [~kmarton] and [~prabhujoseph] can add more color here. ** If both 1 & 2 can be done, then we can say that the submitted config is good or not. * Propagate success or failure back to CLI [~leftnoteasy] [~kmarton] cud u pls share ur thoughts whether this is what intended? cc. [~pbacsko] [~snemeth] [~prabhujoseph] > FS-CS Converter: Need validate c-s.xml after converting > --- > > Key: YARN-10167 > URL: https://issues.apache.org/jira/browse/YARN-10167 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Wangda Tan >Priority: Major > Labels: fs2cs, newbie > > Currently we just generated c-s.xml, but we didn't validate that. To make > sure the c-s.xml is correct after conversion, it's better to initialize the > CS scheduler using configs. > Also, in the test, we should try to leverage MockRM to validate generated > configs as much as we could. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10167) FS-CS Converter: Need validate c-s.xml after converting
[ https://issues.apache.org/jira/browse/YARN-10167?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17046233#comment-17046233 ] Wangda Tan commented on YARN-10167: --- Kinga: we may not be able to use that since we can not assume there’s a running cluster with CS enabled, we have to do this validation from the CLI itself. > FS-CS Converter: Need validate c-s.xml after converting > --- > > Key: YARN-10167 > URL: https://issues.apache.org/jira/browse/YARN-10167 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Wangda Tan >Priority: Major > Labels: fs2cs, newbie > > Currently we just generated c-s.xml, but we didn't validate that. To make > sure the c-s.xml is correct after conversion, it's better to initialize the > CS scheduler using configs. > Also, in the test, we should try to leverage MockRM to validate generated > configs as much as we could. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10167) FS-CS Converter: Need validate c-s.xml after converting
[ https://issues.apache.org/jira/browse/YARN-10167?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17046229#comment-17046229 ] Kinga Marton commented on YARN-10167: - There is a validation API implemented in YARN-10022. We can use that API to validate the generated configuration. > FS-CS Converter: Need validate c-s.xml after converting > --- > > Key: YARN-10167 > URL: https://issues.apache.org/jira/browse/YARN-10167 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Wangda Tan >Priority: Major > Labels: fs2cs, newbie > > Currently we just generated c-s.xml, but we didn't validate that. To make > sure the c-s.xml is correct after conversion, it's better to initialize the > CS scheduler using configs. > Also, in the test, we should try to leverage MockRM to validate generated > configs as much as we could. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10160) Add auto queue creation related configs to RMWebService#CapacitySchedulerQueueInfo
[
https://issues.apache.org/jira/browse/YARN-10160?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17046197#comment-17046197
]
Prabhu Joseph commented on YARN-10160:
--
[~sunil.gov...@gmail.com] [~snemeth] Can you review this Jira when you get
time. Thanks.
> Add auto queue creation related configs to
> RMWebService#CapacitySchedulerQueueInfo
> --
>
> Key: YARN-10160
> URL: https://issues.apache.org/jira/browse/YARN-10160
> Project: Hadoop YARN
> Issue Type: Improvement
>Affects Versions: 3.3.0
>Reporter: Prabhu Joseph
>Assignee: Prabhu Joseph
>Priority: Major
> Attachments: Screen Shot 2020-02-25 at 9.06.52 PM.png,
> YARN-10124-002.patch, YARN-10160-001.patch
>
>
> Add auto queue creation related configs to
> RMWebService#CapacitySchedulerQueueInfo.
> {code}
> yarn.scheduler.capacity..auto-create-child-queue.enabled
> yarn.scheduler.capacity..leaf-queue-template.
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10170) Should revisit mix-usage of percentage-based and absolute-value-based min/max resource in CapacityScheduler
[ https://issues.apache.org/jira/browse/YARN-10170?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17046076#comment-17046076 ] Wangda Tan commented on YARN-10170: --- cc: [~sunil.gov...@gmail.com] > Should revisit mix-usage of percentage-based and absolute-value-based min/max > resource in CapacityScheduler > --- > > Key: YARN-10170 > URL: https://issues.apache.org/jira/browse/YARN-10170 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Wangda Tan >Priority: Major > > This should be finished after YARN-10169. (If we can get this one easily, we > should do this one instead of YARN-10169). > Absolute resource means mem=x, vcore=y. > Percentage resource means x% > We should not allow percentage-based child, but absolute-based parent. (root > is considered as percentage-based). -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-10170) Should revisit mix-usage of percentage-based and absolute-value-based min/max resource in CapacityScheduler
Wangda Tan created YARN-10170: - Summary: Should revisit mix-usage of percentage-based and absolute-value-based min/max resource in CapacityScheduler Key: YARN-10170 URL: https://issues.apache.org/jira/browse/YARN-10170 Project: Hadoop YARN Issue Type: Bug Reporter: Wangda Tan This should be finished after YARN-10169. (If we can get this one easily, we should do this one instead of YARN-10169). Absolute resource means mem=x, vcore=y. Percentage resource means x% We should not allow percentage-based child, but absolute-based parent. (root is considered as percentage-based). -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10152) Fix findbugs warnings in hadoop-yarn-applications-mawo-core module
[
https://issues.apache.org/jira/browse/YARN-10152?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17046066#comment-17046066
]
Hudson commented on YARN-10152:
---
SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #18001 (See
[https://builds.apache.org/job/Hadoop-trunk-Commit/18001/])
YARN-10152. Fix findbugs warnings in hadoop-yarn-applications-mawo-core
(github: rev 7dfa37e8f0833b04d8e75d9f778b72bf57d4c89e)
* (edit)
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-mawo/hadoop-yarn-applications-mawo-core/src/main/java/org/apache/hadoop/applications/mawo/server/worker/WorkerId.java
* (edit)
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-mawo/hadoop-yarn-applications-mawo-core/src/main/java/org/apache/hadoop/applications/mawo/server/common/TaskStatus.java
> Fix findbugs warnings in hadoop-yarn-applications-mawo-core module
> --
>
> Key: YARN-10152
> URL: https://issues.apache.org/jira/browse/YARN-10152
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Akira Ajisaka
>Assignee: Akira Ajisaka
>Priority: Major
> Fix For: 3.3.0
>
>
> {noformat}
> FindBugs :
>
> module:hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-mawo/hadoop-yarn-applications-mawo-core
> Class org.apache.hadoop.applications.mawo.server.common.TaskStatus
> implements Cloneable but does not define or use clone method At
> TaskStatus.java:does not define or use clone method At TaskStatus.java:[lines
> 39-346]
> Equals method for
> org.apache.hadoop.applications.mawo.server.worker.WorkerId assumes the
> argument is of type WorkerId At WorkerId.java:the argument is of type
> WorkerId At WorkerId.java:[line 114]
>
> org.apache.hadoop.applications.mawo.server.worker.WorkerId.equals(Object)
> does not check for null argument At WorkerId.java:null argument At
> WorkerId.java:[lines 114-115] {noformat}
> Detail:
> [https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/1414/artifact/out/branch-findbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-applications_hadoop-yarn-applications-mawo_hadoop-yarn-applications-mawo-core-warnings.html]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6924) Metrics for Federation AMRMProxy
[
https://issues.apache.org/jira/browse/YARN-6924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17046042#comment-17046042
]
Hadoop QA commented on YARN-6924:
-
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
39s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 29m
43s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
16s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
28s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
42s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
15m 42s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
18s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
26s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
38s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
21s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
38s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
14m 10s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
22s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 21m
32s{color} | {color:green} hadoop-yarn-server-nodemanager in the patch passed.
{color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
33s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 90m 50s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=19.03.6 Server=19.03.6 Image:yetus/hadoop:c44943d1fc3 |
| JIRA Issue | YARN-6924 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12994717/YARN-6924.04.patch |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient findbugs checkstyle |
| uname | Linux 6da19e7b3e39 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19
08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / 033a3d7 |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_242 |
| findbugs | v3.1.0-RC1 |
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/25589/testReport/ |
| Max. process+thread count | 314 (vs. ulimit of 5500) |
| modules | C:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager
U:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager
|
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/25589/console |
| Powered by | Apache Yetus 0.8.0 http://yetus.apache.org |
This message was automatically generated.
> Metrics for Federation AMRMProxy
> ---
[jira] [Commented] (YARN-10169) Mixed absolute resource value and percentage-based resource value in CapacityScheduler should fail
[
https://issues.apache.org/jira/browse/YARN-10169?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17046017#comment-17046017
]
Wangda Tan commented on YARN-10169:
---
cc: [~sunil.gov...@gmail.com]
> Mixed absolute resource value and percentage-based resource value in
> CapacityScheduler should fail
> --
>
> Key: YARN-10169
> URL: https://issues.apache.org/jira/browse/YARN-10169
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Wangda Tan
>Priority: Blocker
>
> To me this is a bug: if there's a queue has capacity set to float, and
> maximum-capacity set to absolute value. Existing logic allows the behavior.
> For example:
> {code:java}
> queue.capacity = 0.8
> queue.maximum-capacity = [mem=x, vcore=y] {code}
> We should throw exception when configured like this.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-10169) Mixed absolute resource value and percentage-based resource value in CapacityScheduler should fail
Wangda Tan created YARN-10169:
-
Summary: Mixed absolute resource value and percentage-based
resource value in CapacityScheduler should fail
Key: YARN-10169
URL: https://issues.apache.org/jira/browse/YARN-10169
Project: Hadoop YARN
Issue Type: Bug
Reporter: Wangda Tan
To me this is a bug: if there's a queue has capacity set to float, and
maximum-capacity set to absolute value. Existing logic allows the behavior.
For example:
{code:java}
queue.capacity = 0.8
queue.maximum-capacity = [mem=x, vcore=y] {code}
We should throw exception when configured like this.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-10168) FS-CS Convert: Converter tool doesn't handle min/max resource conversion correct
Wangda Tan created YARN-10168: - Summary: FS-CS Convert: Converter tool doesn't handle min/max resource conversion correct Key: YARN-10168 URL: https://issues.apache.org/jira/browse/YARN-10168 Project: Hadoop YARN Issue Type: Sub-task Reporter: Wangda Tan Trying to understand logics of convert min and max resource from FS to CS, and found some issues: 1) In FSQueueConverter#emitMaximumCapacity Existing logic in FS is to either specify a maximum percentage for queues against cluster resources. Or, specify an absolute valued maximum resource. In the existing FS2CS converter, when a percentage-based maximum resource is specified, the converter takes a global resource from fs2cs CLI, and applies percentages to that. It is not correct since the percentage-based value will get lost, and in the future when cluster resources go up and down, the maximum resource cannot be changed. 2) The logic to deal with min/weight resource is also questionable: The existing fs2cs tool, it takes precedence of percentage over absoluteResource, and could set both to a queue config. See FSQueueConverter.Capacity#toString However, in CS, comparing to FS, the weights/min resource is quite different: CS use the same queue.capacity to specify both percentage-based or absolute-resource-based configs (Similar to how FS deal with maximum Resource). The capacity defines guaranteed resource, which also impact fairshare of the queue. (The more guaranteed resource a queue has, the larger "pie" the queue can get if there's any additional available resource). In FS, minResource defined the guaranteed resource, and weight defined how much the pie can grow to. So to me, in FS, we should pick and choose either weight or minResource to generate CS. 3) In FS, mix-use of absolute-resource configs (like min/maxResource), and percentage-based (like weight) is allowed. But in CS, it is not allowed. The reason is discussed on YARN-5881, and find [a]Should we support specifying a mix of percentage ... The existing fs2cs doesn't handle the issue, which could set mixed absolute resource and percentage-based resources. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-6924) Metrics for Federation AMRMProxy
[ https://issues.apache.org/jira/browse/YARN-6924?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Young Chen updated YARN-6924: - Attachment: YARN-6924.04.patch > Metrics for Federation AMRMProxy > > > Key: YARN-6924 > URL: https://issues.apache.org/jira/browse/YARN-6924 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Giovanni Matteo Fumarola >Assignee: Young Chen >Priority: Major > Attachments: YARN-6924.01.patch, YARN-6924.01.patch, > YARN-6924.02.patch, YARN-6924.02.patch, YARN-6924.03.patch, YARN-6924.04.patch > > > This JIRA proposes addition of metrics for Federation AMRMProxy -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10167) FS-CS Converter: Need validate c-s.xml after converting
[ https://issues.apache.org/jira/browse/YARN-10167?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated YARN-10167: -- Summary: FS-CS Converter: Need validate c-s.xml after converting (was: Need validate c-s.xml after converting) > FS-CS Converter: Need validate c-s.xml after converting > --- > > Key: YARN-10167 > URL: https://issues.apache.org/jira/browse/YARN-10167 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Wangda Tan >Priority: Major > Labels: fs2cs, newbie > > Currently we just generated c-s.xml, but we didn't validate that. To make > sure the c-s.xml is correct after conversion, it's better to initialize the > CS scheduler using configs. > Also, in the test, we should try to leverage MockRM to validate generated > configs as much as we could. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-10167) Need validate c-s.xml after converting
Wangda Tan created YARN-10167: - Summary: Need validate c-s.xml after converting Key: YARN-10167 URL: https://issues.apache.org/jira/browse/YARN-10167 Project: Hadoop YARN Issue Type: Sub-task Reporter: Wangda Tan Currently we just generated c-s.xml, but we didn't validate that. To make sure the c-s.xml is correct after conversion, it's better to initialize the CS scheduler using configs. Also, in the test, we should try to leverage MockRM to validate generated configs as much as we could. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6924) Metrics for Federation AMRMProxy
[
https://issues.apache.org/jira/browse/YARN-6924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17045971#comment-17045971
]
Hadoop QA commented on YARN-6924:
-
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
43s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 21m
42s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
3s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
25s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
41s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
15m 32s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
19s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
26s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
34s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
55s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
55s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 20s{color} | {color:orange}
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager:
The patch generated 8 new + 3 unchanged - 0 fixed = 11 total (was 3) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
37s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
14m 8s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
22s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 21m
40s{color} | {color:green} hadoop-yarn-server-nodemanager in the patch passed.
{color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
31s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 82m 27s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=19.03.6 Server=19.03.6 Image:yetus/hadoop:c44943d1fc3 |
| JIRA Issue | YARN-6924 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12994712/YARN-6924.03.patch |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient findbugs checkstyle |
| uname | Linux 0aff6ebae347 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19
08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / 2096304 |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_242 |
| findbugs | v3.1.0-RC1 |
| checkstyle |
https://builds.apache.org/job/PreCommit-YARN-Build/25588/artifact/out/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/25588/testReport/ |
| Max. process+thread count | 308 (vs. ulimit of 5500) |
| modules | C:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager
U:
h
[jira] [Updated] (YARN-6924) Metrics for Federation AMRMProxy
[ https://issues.apache.org/jira/browse/YARN-6924?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Young Chen updated YARN-6924: - Attachment: YARN-6924.03.patch > Metrics for Federation AMRMProxy > > > Key: YARN-6924 > URL: https://issues.apache.org/jira/browse/YARN-6924 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Giovanni Matteo Fumarola >Assignee: Young Chen >Priority: Major > Attachments: YARN-6924.01.patch, YARN-6924.01.patch, > YARN-6924.02.patch, YARN-6924.02.patch, YARN-6924.03.patch > > > This JIRA proposes addition of metrics for Federation AMRMProxy -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6924) Metrics for Federation AMRMProxy
[
https://issues.apache.org/jira/browse/YARN-6924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17045907#comment-17045907
]
Hadoop QA commented on YARN-6924:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
0s{color} | {color:blue} Docker mode activated. {color} |
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 9s{color}
| {color:red} YARN-6924 does not apply to trunk. Rebase required? Wrong Branch?
See https://wiki.apache.org/hadoop/HowToContribute for help. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | YARN-6924 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12994708/YARN-6924.02.patch |
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/25587/console |
| Powered by | Apache Yetus 0.8.0 http://yetus.apache.org |
This message was automatically generated.
> Metrics for Federation AMRMProxy
>
>
> Key: YARN-6924
> URL: https://issues.apache.org/jira/browse/YARN-6924
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Giovanni Matteo Fumarola
>Assignee: Young Chen
>Priority: Major
> Attachments: YARN-6924.01.patch, YARN-6924.01.patch,
> YARN-6924.02.patch, YARN-6924.02.patch
>
>
> This JIRA proposes addition of metrics for Federation AMRMProxy
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-6924) Metrics for Federation AMRMProxy
[ https://issues.apache.org/jira/browse/YARN-6924?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Young Chen updated YARN-6924: - Attachment: YARN-6924.02.patch > Metrics for Federation AMRMProxy > > > Key: YARN-6924 > URL: https://issues.apache.org/jira/browse/YARN-6924 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Giovanni Matteo Fumarola >Assignee: Young Chen >Priority: Major > Attachments: YARN-6924.01.patch, YARN-6924.01.patch, > YARN-6924.02.patch, YARN-6924.02.patch > > > This JIRA proposes addition of metrics for Federation AMRMProxy -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-8286) Inform AM of container relaunch
[ https://issues.apache.org/jira/browse/YARN-8286?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Billie Rinaldi updated YARN-8286: - Target Version/s: 3.4.0 (was: 3.3.0) > Inform AM of container relaunch > --- > > Key: YARN-8286 > URL: https://issues.apache.org/jira/browse/YARN-8286 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Billie Rinaldi >Assignee: Adam Antal >Priority: Critical > > The AM may need to perform actions when a container has been relaunched. For > example, the service AM would want to change the state it has recorded for > the container and retrieve new container status for the container, in case > the container IP has changed. (The NM would also need to remove the IP it has > stored for the container, so container status calls don't return an IP for a > container that is not currently running.) -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-8982) [Router] Add locality policy
[ https://issues.apache.org/jira/browse/YARN-8982?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ayush Saxena updated YARN-8982: --- Fix Version/s: (was: 3.4.0) 3.3.0 > [Router] Add locality policy > - > > Key: YARN-8982 > URL: https://issues.apache.org/jira/browse/YARN-8982 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Giovanni Matteo Fumarola >Assignee: Young Chen >Priority: Major > Fix For: 3.3.0 > > Attachments: YARN-8982.v1.patch, YARN-8982.v2.patch, > YARN-8982.v3.patch, YARN-8982.v4.patch, YARN-8982.v5.patch > > > This jira tracks the effort to add a new policy in the Router. > This policy will allow the Router to pick the SubCluster based on the node > that the client requested. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (YARN-8286) Inform AM of container relaunch
[ https://issues.apache.org/jira/browse/YARN-8286?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17045791#comment-17045791 ] Brahma Reddy Battula edited comment on YARN-8286 at 2/26/20 6:25 PM: - [~billie] and [~adam.antal], could we move to 3.4 if this is not critical for 3.3.? will cut branch-3.3 by next week. was (Author: brahmareddy): [~billie] and [~adam.antal], could we move to 3.4 if this is not critical for 3.3.? will cut branch-3.3 by next week. * [|https://issues.apache.org/jira/secure/AddComment!default.jspa?id=13161175] > Inform AM of container relaunch > --- > > Key: YARN-8286 > URL: https://issues.apache.org/jira/browse/YARN-8286 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Billie Rinaldi >Assignee: Adam Antal >Priority: Critical > > The AM may need to perform actions when a container has been relaunched. For > example, the service AM would want to change the state it has recorded for > the container and retrieve new container status for the container, in case > the container IP has changed. (The NM would also need to remove the IP it has > stored for the container, so container status calls don't return an IP for a > container that is not currently running.) -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-8286) Inform AM of container relaunch
[ https://issues.apache.org/jira/browse/YARN-8286?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17045791#comment-17045791 ] Brahma Reddy Battula commented on YARN-8286: [~billie] and [~adam.antal], could we move to 3.4 if this is not critical for 3.3.? will cut branch-3.3 by next week. * [|https://issues.apache.org/jira/secure/AddComment!default.jspa?id=13161175] > Inform AM of container relaunch > --- > > Key: YARN-8286 > URL: https://issues.apache.org/jira/browse/YARN-8286 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Billie Rinaldi >Assignee: Adam Antal >Priority: Critical > > The AM may need to perform actions when a container has been relaunched. For > example, the service AM would want to change the state it has recorded for > the container and retrieve new container status for the container, in case > the container IP has changed. (The NM would also need to remove the IP it has > stored for the container, so container status calls don't return an IP for a > container that is not currently running.) -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10141) Interceptor in FederationInterceptorREST doesnt update on RM switchover
[ https://issues.apache.org/jira/browse/YARN-10141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17045784#comment-17045784 ] Hudson commented on YARN-10141: --- SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #17998 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/17998/]) YARN-10141.Interceptor in FederationInterceptorREST doesnt update on RM (brahma: rev 3a9ccf7f6d91f12ba6db33142cc033e4957e994f) * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router/src/test/java/org/apache/hadoop/yarn/server/router/webapp/TestFederationInterceptorREST.java * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router/src/main/java/org/apache/hadoop/yarn/server/router/webapp/DefaultRequestInterceptorREST.java * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router/src/main/java/org/apache/hadoop/yarn/server/router/webapp/FederationInterceptorREST.java * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router/src/test/java/org/apache/hadoop/yarn/server/router/webapp/TestableFederationInterceptorREST.java > Interceptor in FederationInterceptorREST doesnt update on RM switchover > --- > > Key: YARN-10141 > URL: https://issues.apache.org/jira/browse/YARN-10141 > Project: Hadoop YARN > Issue Type: Bug > Components: federation, restapi >Reporter: D M Murali Krishna Reddy >Assignee: D M Murali Krishna Reddy >Priority: Major > Fix For: 3.3.0 > > Attachments: YARN-10141.001.patch, YARN-10141.002.patch, > YARN-10141.003.patch, YARN-10141.004.patch > > > In Federation Setup, In the event of a RM switchover in a subcluster the > interceptor for that subcluster in FederationInterceptorREST doesnt get > updated. Due to this, Cluster Nodes REST API doesnt return the nodes from the > subcluster in which the RM switchover has occured. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-8340) Capacity Scheduler Intra Queue Preemption Should Work When 3rd or more resources enabled.
[ https://issues.apache.org/jira/browse/YARN-8340?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17045779#comment-17045779 ] Brahma Reddy Battula commented on YARN-8340: [~leftnoteasy] could we move to 3.4 if this is not critical for 3.3.? > Capacity Scheduler Intra Queue Preemption Should Work When 3rd or more > resources enabled. > - > > Key: YARN-8340 > URL: https://issues.apache.org/jira/browse/YARN-8340 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Wangda Tan >Priority: Critical > > Refer to comment from [~eepayne] and discussion below that: > https://issues.apache.org/jira/browse/YARN-8292?focusedCommentId=16482689&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16482689 > for details. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10141) Interceptor in FederationInterceptorREST doesnt update on RM switchover
[ https://issues.apache.org/jira/browse/YARN-10141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17045753#comment-17045753 ] Brahma Reddy Battula commented on YARN-10141: - [~dmmkr] it's nice catch.thanks for reporting. +1, Looks good > Interceptor in FederationInterceptorREST doesnt update on RM switchover > --- > > Key: YARN-10141 > URL: https://issues.apache.org/jira/browse/YARN-10141 > Project: Hadoop YARN > Issue Type: Bug > Components: federation, restapi >Reporter: D M Murali Krishna Reddy >Assignee: D M Murali Krishna Reddy >Priority: Major > Attachments: YARN-10141.001.patch, YARN-10141.002.patch, > YARN-10141.003.patch, YARN-10141.004.patch > > > In Federation Setup, In the event of a RM switchover in a subcluster the > interceptor for that subcluster in FederationInterceptorREST doesnt get > updated. Due to this, Cluster Nodes REST API doesnt return the nodes from the > subcluster in which the RM switchover has occured. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-9893) Capacity scheduler: enhance leaf-queue-template capacity / maximum-capacity setting
[
https://issues.apache.org/jira/browse/YARN-9893?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Peter Bacsko updated YARN-9893:
---
Description:
Capacity Scheduler does not support two percentage values for leaf queue
capacity and maximum-capacity settings. So, you can't do something like this:
{{yarn.scheduler.capacity.root.users.john.leaf-queue-template.capacity=memory-mb=50.0%,
vcores=50.0%}}
Only a single percentage value is accepted.
This makes it nearly impossible to properly convert a similar setting from Fair
Scheduler, where such a configuration is valid and accepted
({{}}).
Note: using absolute resources
({{yarn.scheduler.capacity.root.users.john.leaf-queue-template.capacity=memory-mb=16384,
vcores=8}}) is addressed in YARN-10154.
was:
Capacity Scheduler does not support two percentage values for leaf queue
capacity and maximum-capacity settings. So, you can't do something like this:
{{yarn.scheduler.capacity.root.users.john.leaf-queue-template.capacity=memory-mb=50.0%,
vcores=50.0%}}
On top of that, it's not even possible to define absolute resources:
{{yarn.scheduler.capacity.root.users.john.leaf-queue-template.capacity=memory-mb=16384,
vcores=8}}
Only a single percentage value is accepted.
This makes it nearly impossible to properly convert a similar setting from Fair
Scheduler, where such a configuration is valid and accepted
({{}}).
> Capacity scheduler: enhance leaf-queue-template capacity / maximum-capacity
> setting
> ---
>
> Key: YARN-9893
> URL: https://issues.apache.org/jira/browse/YARN-9893
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: capacity scheduler
>Reporter: Peter Bacsko
>Assignee: Manikandan R
>Priority: Major
>
> Capacity Scheduler does not support two percentage values for leaf queue
> capacity and maximum-capacity settings. So, you can't do something like this:
> {{yarn.scheduler.capacity.root.users.john.leaf-queue-template.capacity=memory-mb=50.0%,
> vcores=50.0%}}
> Only a single percentage value is accepted.
> This makes it nearly impossible to properly convert a similar setting from
> Fair Scheduler, where such a configuration is valid and accepted
> ({{}}).
> Note: using absolute resources
> ({{yarn.scheduler.capacity.root.users.john.leaf-queue-template.capacity=memory-mb=16384,
> vcores=8}}) is addressed in YARN-10154.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9893) Capacity scheduler: enhance leaf-queue-template capacity / maximum-capacity setting
[
https://issues.apache.org/jira/browse/YARN-9893?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17045721#comment-17045721
]
Peter Bacsko commented on YARN-9893:
[~maniraj...@gmail.com] I'll re-phrase this JIRA and remove the absolute
resource part.
> Capacity scheduler: enhance leaf-queue-template capacity / maximum-capacity
> setting
> ---
>
> Key: YARN-9893
> URL: https://issues.apache.org/jira/browse/YARN-9893
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: capacity scheduler
>Reporter: Peter Bacsko
>Assignee: Manikandan R
>Priority: Major
>
> Capacity Scheduler does not support two percentage values for leaf queue
> capacity and maximum-capacity settings. So, you can't do something like this:
> {{yarn.scheduler.capacity.root.users.john.leaf-queue-template.capacity=memory-mb=50.0%,
> vcores=50.0%}}
> On top of that, it's not even possible to define absolute resources:
> {{yarn.scheduler.capacity.root.users.john.leaf-queue-template.capacity=memory-mb=16384,
> vcores=8}}
> Only a single percentage value is accepted.
> This makes it nearly impossible to properly convert a similar setting from
> Fair Scheduler, where such a configuration is valid and accepted
> ({{}}).
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10043) FairOrderingPolicy Improvements
[ https://issues.apache.org/jira/browse/YARN-10043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17045710#comment-17045710 ] Manikandan R commented on YARN-10043: - [~snemeth] Can you please take a final look to commit the code? > FairOrderingPolicy Improvements > --- > > Key: YARN-10043 > URL: https://issues.apache.org/jira/browse/YARN-10043 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Manikandan R >Assignee: Manikandan R >Priority: Major > Attachments: YARN-10043.001.patch, YARN-10043.002.patch, > YARN-10043.003.patch, YARN-10043.004.patch > > > FairOrderingPolicy can be improved by using some of the approaches (only > relevant) implemented in FairSharePolicy of FS. This improvement has > significance in FS to CS migration context. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10155) TestDelegationTokenRenewer.testTokenThreadTimeout fails in trunk
[ https://issues.apache.org/jira/browse/YARN-10155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17045709#comment-17045709 ] Manikandan R commented on YARN-10155: - [~adam.antal] Thanks for your effort. I've been running all my tests through mvn cli with prior clean and build. Can you please try the alternative step? > TestDelegationTokenRenewer.testTokenThreadTimeout fails in trunk > > > Key: YARN-10155 > URL: https://issues.apache.org/jira/browse/YARN-10155 > Project: Hadoop YARN > Issue Type: Bug > Components: yarn >Affects Versions: 3.3.0 >Reporter: Adam Antal >Assignee: Manikandan R >Priority: Major > Attachments: YARN-10155.001.patch, testTokenThreadTimeout.txt, > testTokenThreadTimeout_with_patch.txt > > > The TestDelegationTokenRenewer.testTokenThreadTimeout test committed in > YARN-9768 often fails with timeout. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9893) Capacity scheduler: enhance leaf-queue-template capacity / maximum-capacity setting
[
https://issues.apache.org/jira/browse/YARN-9893?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17045707#comment-17045707
]
Manikandan R commented on YARN-9893:
[~pbacsko] Part of this requirement would be addressed in YARN-10154. We can
handle support for two percentage values in separate jira.
Is that ok?
> Capacity scheduler: enhance leaf-queue-template capacity / maximum-capacity
> setting
> ---
>
> Key: YARN-9893
> URL: https://issues.apache.org/jira/browse/YARN-9893
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: capacity scheduler
>Reporter: Peter Bacsko
>Assignee: Manikandan R
>Priority: Major
>
> Capacity Scheduler does not support two percentage values for leaf queue
> capacity and maximum-capacity settings. So, you can't do something like this:
> {{yarn.scheduler.capacity.root.users.john.leaf-queue-template.capacity=memory-mb=50.0%,
> vcores=50.0%}}
> On top of that, it's not even possible to define absolute resources:
> {{yarn.scheduler.capacity.root.users.john.leaf-queue-template.capacity=memory-mb=16384,
> vcores=8}}
> Only a single percentage value is accepted.
> This makes it nearly impossible to properly convert a similar setting from
> Fair Scheduler, where such a configuration is valid and accepted
> ({{}}).
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Assigned] (YARN-10154) CS Dynamic Queues cannot be configured with absolute resources
[ https://issues.apache.org/jira/browse/YARN-10154?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manikandan R reassigned YARN-10154: --- Assignee: Manikandan R (was: Sunil G) > CS Dynamic Queues cannot be configured with absolute resources > -- > > Key: YARN-10154 > URL: https://issues.apache.org/jira/browse/YARN-10154 > Project: Hadoop YARN > Issue Type: Bug >Affects Versions: 3.1.3 >Reporter: Sunil G >Assignee: Manikandan R >Priority: Major > Attachments: YARN-10154.001.patch > > > In CS, ManagedParent Queue and its template cannot take absolute resource > value like > [memory=8192,vcores=8] > Thsi Jira is to track and improve the configuration reading module of > DynamicQueue to support absolute resource values. > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Assigned] (YARN-9893) Capacity scheduler: enhance leaf-queue-template capacity / maximum-capacity setting
[
https://issues.apache.org/jira/browse/YARN-9893?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Manikandan R reassigned YARN-9893:
--
Assignee: Manikandan R (was: Peter Bacsko)
> Capacity scheduler: enhance leaf-queue-template capacity / maximum-capacity
> setting
> ---
>
> Key: YARN-9893
> URL: https://issues.apache.org/jira/browse/YARN-9893
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: capacity scheduler
>Reporter: Peter Bacsko
>Assignee: Manikandan R
>Priority: Major
>
> Capacity Scheduler does not support two percentage values for leaf queue
> capacity and maximum-capacity settings. So, you can't do something like this:
> {{yarn.scheduler.capacity.root.users.john.leaf-queue-template.capacity=memory-mb=50.0%,
> vcores=50.0%}}
> On top of that, it's not even possible to define absolute resources:
> {{yarn.scheduler.capacity.root.users.john.leaf-queue-template.capacity=memory-mb=16384,
> vcores=8}}
> Only a single percentage value is accepted.
> This makes it nearly impossible to properly convert a similar setting from
> Fair Scheduler, where such a configuration is valid and accepted
> ({{}}).
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10154) CS Dynamic Queues cannot be configured with absolute resources
[ https://issues.apache.org/jira/browse/YARN-10154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17045698#comment-17045698 ] Manikandan R commented on YARN-10154: - Attaching WIP patch with simple test case to validate approach is good enough to proceed further. > CS Dynamic Queues cannot be configured with absolute resources > -- > > Key: YARN-10154 > URL: https://issues.apache.org/jira/browse/YARN-10154 > Project: Hadoop YARN > Issue Type: Bug >Affects Versions: 3.1.3 >Reporter: Sunil G >Assignee: Sunil G >Priority: Major > Attachments: YARN-10154.001.patch > > > In CS, ManagedParent Queue and its template cannot take absolute resource > value like > [memory=8192,vcores=8] > Thsi Jira is to track and improve the configuration reading module of > DynamicQueue to support absolute resource values. > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10154) CS Dynamic Queues cannot be configured with absolute resources
[ https://issues.apache.org/jira/browse/YARN-10154?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manikandan R updated YARN-10154: Attachment: YARN-10154.001.patch > CS Dynamic Queues cannot be configured with absolute resources > -- > > Key: YARN-10154 > URL: https://issues.apache.org/jira/browse/YARN-10154 > Project: Hadoop YARN > Issue Type: Bug >Affects Versions: 3.1.3 >Reporter: Sunil G >Assignee: Sunil G >Priority: Major > Attachments: YARN-10154.001.patch > > > In CS, ManagedParent Queue and its template cannot take absolute resource > value like > [memory=8192,vcores=8] > Thsi Jira is to track and improve the configuration reading module of > DynamicQueue to support absolute resource values. > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10166) Add detail log for ApplicationAttemptNotFoundException
[
https://issues.apache.org/jira/browse/YARN-10166?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17045525#comment-17045525
]
Hadoop QA commented on YARN-10166:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
38s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 5m
18s{color} | {color:red} root in trunk failed. {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red} 0m
40s{color} | {color:red} hadoop-yarn-server-resourcemanager in trunk failed.
{color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
8s{color} | {color:green} trunk passed {color} |
| {color:red}-1{color} | {color:red} mvnsite {color} | {color:red} 0m
30s{color} | {color:red} hadoop-yarn-server-resourcemanager in trunk failed.
{color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
2m 15s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
59s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
29s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
49s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
40s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 0m 40s{color}
| {color:red}
hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager
generated 27 new + 0 unchanged - 0 fixed = 27 total (was 0) {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 28s{color} | {color:orange}
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager:
The patch generated 1 new + 7 unchanged - 0 fixed = 8 total (was 7) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
41s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
14m 56s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
37s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
26s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 44m 19s{color}
| {color:red} hadoop-yarn-server-resourcemanager in the patch passed. {color} |
| {color:blue}0{color} | {color:blue} asflicense {color} | {color:blue} 0m
31s{color} | {color:blue} ASF License check generated no output? {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 76m 59s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests |
hadoop.yarn.server.resourcemanager.applicationsmanager.TestAMRestart |
| |
hadoop.yarn.server.resourcemanager.scheduler.fair.TestFairSchedulerPreemption |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=19.03.6 Server=19.03.6 Image:yetus/hadoop:c44943d1fc3 |
| JIRA Issue | YARN-10166 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12994648/YARN-10166-001.patch |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient findbugs checkstyle |
| uname | Linux d0fd20585db8 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19
08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / b5df1da |
| maven | version: Apache
[jira] [Commented] (YARN-10155) TestDelegationTokenRenewer.testTokenThreadTimeout fails in trunk
[ https://issues.apache.org/jira/browse/YARN-10155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17045509#comment-17045509 ] Adam Antal commented on YARN-10155: --- Uploaded test logs (run by IntelliJ with Repeat: Until failure configuration) - before the patch (current trunk): [^testTokenThreadTimeout.txt] - with the patch applied (the failure happened for the 4th run): [^testTokenThreadTimeout_with_patch.txt] I don't see any ArithmeticException in the logs. If you don't succeed in reproducing it, then maybe it is some setup issue on my local computer/IntelliJ configs, so I'll attempt a git clean+maven clean build to wipe out everything and try again. > TestDelegationTokenRenewer.testTokenThreadTimeout fails in trunk > > > Key: YARN-10155 > URL: https://issues.apache.org/jira/browse/YARN-10155 > Project: Hadoop YARN > Issue Type: Bug > Components: yarn >Affects Versions: 3.3.0 >Reporter: Adam Antal >Assignee: Manikandan R >Priority: Major > Attachments: YARN-10155.001.patch, testTokenThreadTimeout.txt, > testTokenThreadTimeout_with_patch.txt > > > The TestDelegationTokenRenewer.testTokenThreadTimeout test committed in > YARN-9768 often fails with timeout. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10155) TestDelegationTokenRenewer.testTokenThreadTimeout fails in trunk
[ https://issues.apache.org/jira/browse/YARN-10155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Adam Antal updated YARN-10155: -- Attachment: testTokenThreadTimeout_with_patch.txt > TestDelegationTokenRenewer.testTokenThreadTimeout fails in trunk > > > Key: YARN-10155 > URL: https://issues.apache.org/jira/browse/YARN-10155 > Project: Hadoop YARN > Issue Type: Bug > Components: yarn >Affects Versions: 3.3.0 >Reporter: Adam Antal >Assignee: Manikandan R >Priority: Major > Attachments: YARN-10155.001.patch, testTokenThreadTimeout.txt, > testTokenThreadTimeout_with_patch.txt > > > The TestDelegationTokenRenewer.testTokenThreadTimeout test committed in > YARN-9768 often fails with timeout. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10155) TestDelegationTokenRenewer.testTokenThreadTimeout fails in trunk
[ https://issues.apache.org/jira/browse/YARN-10155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Adam Antal updated YARN-10155: -- Attachment: testTokenThreadTimeout.txt > TestDelegationTokenRenewer.testTokenThreadTimeout fails in trunk > > > Key: YARN-10155 > URL: https://issues.apache.org/jira/browse/YARN-10155 > Project: Hadoop YARN > Issue Type: Bug > Components: yarn >Affects Versions: 3.3.0 >Reporter: Adam Antal >Assignee: Manikandan R >Priority: Major > Attachments: YARN-10155.001.patch, testTokenThreadTimeout.txt > > > The TestDelegationTokenRenewer.testTokenThreadTimeout test committed in > YARN-9768 often fails with timeout. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10130) FS-CS converter: Do not allow output dir to be the same as input dir
[
https://issues.apache.org/jira/browse/YARN-10130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17045501#comment-17045501
]
Adam Antal commented on YARN-10130:
---
Thanks [~snemeth]!
> FS-CS converter: Do not allow output dir to be the same as input dir
>
>
> Key: YARN-10130
> URL: https://issues.apache.org/jira/browse/YARN-10130
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Szilard Nemeth
>Assignee: Adam Antal
>Priority: Major
> Fix For: 3.3.0
>
> Attachments: YARN-10130.001.patch, YARN-10130.002.patch,
> YARN-10130.003.patch, YARN-10130.004.patch, YARN-10130.005.patch
>
>
> If the input dir where fair-scheduler.xml / yarn-site.xml sits is the same as
> the output dir (defined by the -o switch), the fs2cs tool overwrites the
> source config files, i.e. yarn-site.xml.
> Reproduce this is easy, just run fs2cs tool with this command:
> {code:java}
> /bin/yarn fs2cs --cluster-resource memory-mb=18044928,vcores=16
> --no-terminal-rule-check -y yarn-site.xml -f fair-scheduler.xml -o .
> {code}
> The following (or similar) is emitted by the tool:
> {code:java}
> WARNING: YARN_OPTS has been replaced by HADOOP_OPTS. Using value of
> YARN_OPTS.WARNING: YARN_OPTS has been replaced by HADOOP_OPTS. Using value of
> YARN_OPTS.20/02/10 12:51:42 INFO converter.FSConfigToCSConfigConverter:
> Output directory for yarn-site.xml and capacity-scheduler.xml is: .20/02/10
> 12:51:42 INFO converter.FSConfigToCSConfigConverter: Conversion rules file is
> not defined, using default conversion config!20/02/10 12:51:42 ERROR
> conf.Configuration: error parsing conf
> yarn-site.xmlcom.ctc.wstx.exc.WstxEOFException: Unexpected EOF in prolog at
> [row,col,system-id]: [1,0,"yarn-site.xml"] at
> com.ctc.wstx.sr.StreamScanner.throwUnexpectedEOF(StreamScanner.java:687) at
> com.ctc.wstx.sr.BasicStreamReader.handleEOF(BasicStreamReader.java:2220) at
> com.ctc.wstx.sr.BasicStreamReader.nextFromProlog(BasicStreamReader.java:2126)
> at com.ctc.wstx.sr.BasicStreamReader.next(BasicStreamReader.java:1181) at
> org.apache.hadoop.conf.Configuration$Parser.parseNext(Configuration.java:3343)
> at
> org.apache.hadoop.conf.Configuration$Parser.parse(Configuration.java:3137) at
> org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:3030) at
> org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:2996)
> at org.apache.hadoop.conf.Configuration.getProps(Configuration.java:2871) at
> org.apache.hadoop.conf.Configuration.set(Configuration.java:1389) at
> org.apache.hadoop.conf.Configuration.set(Configuration.java:1361) at
> org.apache.hadoop.conf.Configuration.setBoolean(Configuration.java:1702) at
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.converter.FSConfigToCSConfigConverter.createConfiguration(FSConfigToCSConfigConverter.java:166)
> at
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.converter.FSConfigToCSConfigConverter.convert(FSConfigToCSConfigConverter.java:98)
> at
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.converter.FSConfigToCSConfigArgumentHandler.parseAndConvert(FSConfigToCSConfigArgumentHandler.java:137)
> at
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.converter.FSConfigToCSConfigConverterMain.main(FSConfigToCSConfigConverterMain.java:40)20/02/10
> 12:51:42 ERROR converter.FSConfigToCSConfigConverterMain: Error while
> starting FS configuration conversion!
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10141) Interceptor in FederationInterceptorREST doesnt update on RM switchover
[ https://issues.apache.org/jira/browse/YARN-10141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17045482#comment-17045482 ] Bilwa S T commented on YARN-10141: -- Patch looks good to me. I have tested it and its working fine. +1 > Interceptor in FederationInterceptorREST doesnt update on RM switchover > --- > > Key: YARN-10141 > URL: https://issues.apache.org/jira/browse/YARN-10141 > Project: Hadoop YARN > Issue Type: Bug > Components: federation, restapi >Reporter: D M Murali Krishna Reddy >Assignee: D M Murali Krishna Reddy >Priority: Major > Attachments: YARN-10141.001.patch, YARN-10141.002.patch, > YARN-10141.003.patch, YARN-10141.004.patch > > > In Federation Setup, In the event of a RM switchover in a subcluster the > interceptor for that subcluster in FederationInterceptorREST doesnt get > updated. Due to this, Cluster Nodes REST API doesnt return the nodes from the > subcluster in which the RM switchover has occured. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10166) Add detail log for ApplicationAttemptNotFoundException
[
https://issues.apache.org/jira/browse/YARN-10166?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Youquan Lin updated YARN-10166:
---
Description:
Suppose user A killed the app, then ApplicationMasterService will call
unregisterAttempt() for this app. Sometimes, app's AM continues to call the
alloate() method and reports an error as follows.
{code:java}
Application attempt appattempt_1582520281010_15271_01 doesn't exist in
ApplicationMasterService cache.
{code}
If user B has been watching the AM log, he will be confused why the attempt
is no longer in the ApplicationMasterService cache. So I think we can add
detail log for ApplicationAttemptNotFoundException as follows.
{code:java}
Application attempt appattempt_1582630210671_14658_01 doesn't exist in
ApplicationMasterService cache.App state: KILLED,finalStatus: KILLED
,diagnostics: App application_1582630210671_14658 killed by userA from 127.0.0.1
{code}
was:
Suppose user A killed the app, then ApplicationMasterService will call
unregisterAttempt() for this app. Sometimes, app's AM continues to call the
alloate() method and reports an error as follows.
{code:java}
Application attempt appattempt_1582520281010_15271_01 doesn't exist in
ApplicationMasterService cache.
{code}
If user B has been watching the AM log, he will be confused why the attempt
is no longer in the ApplicationMasterService cache. So I think we can detail
log for ApplicationAttemptNotFoundException as follows.
{code:java}
Application attempt appattempt_1582630210671_14658_01 doesn't exist in
ApplicationMasterService cache.App state: KILLED,finalStatus: KILLED
,diagnostics: App application_1582630210671_14658 killed by userA from 127.0.0.1
{code}
> Add detail log for ApplicationAttemptNotFoundException
> --
>
> Key: YARN-10166
> URL: https://issues.apache.org/jira/browse/YARN-10166
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: resourcemanager
>Affects Versions: 3.1.3
>Reporter: Youquan Lin
>Priority: Minor
> Labels: patch
> Fix For: 3.1.3
>
> Attachments: YARN-10166-001.patch
>
>
> Suppose user A killed the app, then ApplicationMasterService will call
> unregisterAttempt() for this app. Sometimes, app's AM continues to call the
> alloate() method and reports an error as follows.
> {code:java}
> Application attempt appattempt_1582520281010_15271_01 doesn't exist in
> ApplicationMasterService cache.
> {code}
> If user B has been watching the AM log, he will be confused why the
> attempt is no longer in the ApplicationMasterService cache. So I think we can
> add detail log for ApplicationAttemptNotFoundException as follows.
> {code:java}
> Application attempt appattempt_1582630210671_14658_01 doesn't exist in
> ApplicationMasterService cache.App state: KILLED,finalStatus: KILLED
> ,diagnostics: App application_1582630210671_14658 killed by userA from
> 127.0.0.1
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-10166) Add detail log for ApplicationAttemptNotFoundException
Youquan Lin created YARN-10166:
--
Summary: Add detail log for ApplicationAttemptNotFoundException
Key: YARN-10166
URL: https://issues.apache.org/jira/browse/YARN-10166
Project: Hadoop YARN
Issue Type: Improvement
Components: resourcemanager
Affects Versions: 3.1.3
Reporter: Youquan Lin
Fix For: 3.1.3
Suppose user A killed the app, then ApplicationMasterService will call
unregisterAttempt() for this app. Sometimes, app's AM continues to call the
alloate() method and reports an error as follows.
{code:java}
Application attempt appattempt_1582520281010_15271_01 doesn't exist in
ApplicationMasterService cache.
{code}
If user B has been watching the AM log, he will be confused why the attempt
is no longer in the ApplicationMasterService cache. So I think we can detail
log for ApplicationAttemptNotFoundException as follows.
{code:java}
Application attempt appattempt_1582630210671_14658_01 doesn't exist in
ApplicationMasterService cache.App state: KILLED,finalStatus: KILLED
,diagnostics: App application_1582630210671_14658 killed by userA from 127.0.0.1
{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9593) Updating scheduler conf with comma in config value fails
[
https://issues.apache.org/jira/browse/YARN-9593?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17045326#comment-17045326
]
Hudson commented on YARN-9593:
--
SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #17996 (See
[https://builds.apache.org/job/Hadoop-trunk-Commit/17996/])
YARN-9593. Support Comma in the value of Scheduler Configuration (pjoseph: rev
b5df1da3a12dc70367ecee503e7fb90b4bac3ac3)
* (edit)
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/main/java/org/apache/hadoop/yarn/client/cli/SchedConfCLI.java
* (edit)
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/test/java/org/apache/hadoop/yarn/client/cli/TestSchedConfCLI.java
> Updating scheduler conf with comma in config value fails
>
>
> Key: YARN-9593
> URL: https://issues.apache.org/jira/browse/YARN-9593
> Project: Hadoop YARN
> Issue Type: Bug
>Affects Versions: 2.9.0, 3.0.0, 3.2.0, 3.1.2
>Reporter: Anthony Hsu
>Assignee: Tanu Ajmera
>Priority: Major
> Fix For: 3.3.0
>
> Attachments: YARN-9593-001.patch, YARN-9593-002.patch,
> YARN-9593-003.patch, YARN-9593-004.patch
>
>
> For example:
> {code:java}
> $ yarn schedulerconf -update "root.gridops:acl_administer_queue=user1,user2
> group1,group2"
> Specify configuration key value as confKey=confVal.{code}
> This fails because there is a comma in the config value and the SchedConfCLI
> splits on comma first, expecting each split to a k=v pair.
> {noformat}
> void globalUpdates(String args, SchedConfUpdateInfo updateInfo) {
> if (args == null) {
> return;
> }
> HashMap globalUpdates = new HashMap<>();
> for (String globalUpdate : args.split(",")) {
> putKeyValuePair(globalUpdates, globalUpdate);
> }
> updateInfo.setGlobalParams(globalUpdates);
> }{noformat}
> Cc: [~jhung]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9593) Updating scheduler conf with comma in config value fails
[
https://issues.apache.org/jira/browse/YARN-9593?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17045311#comment-17045311
]
Prabhu Joseph commented on YARN-9593:
-
Have committed to trunk.
> Updating scheduler conf with comma in config value fails
>
>
> Key: YARN-9593
> URL: https://issues.apache.org/jira/browse/YARN-9593
> Project: Hadoop YARN
> Issue Type: Bug
>Affects Versions: 2.9.0, 3.0.0, 3.2.0, 3.1.2
>Reporter: Anthony Hsu
>Assignee: Tanu Ajmera
>Priority: Major
> Attachments: YARN-9593-001.patch, YARN-9593-002.patch,
> YARN-9593-003.patch, YARN-9593-004.patch
>
>
> For example:
> {code:java}
> $ yarn schedulerconf -update "root.gridops:acl_administer_queue=user1,user2
> group1,group2"
> Specify configuration key value as confKey=confVal.{code}
> This fails because there is a comma in the config value and the SchedConfCLI
> splits on comma first, expecting each split to a k=v pair.
> {noformat}
> void globalUpdates(String args, SchedConfUpdateInfo updateInfo) {
> if (args == null) {
> return;
> }
> HashMap globalUpdates = new HashMap<>();
> for (String globalUpdate : args.split(",")) {
> putKeyValuePair(globalUpdates, globalUpdate);
> }
> updateInfo.setGlobalParams(globalUpdates);
> }{noformat}
> Cc: [~jhung]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9593) Updating scheduler conf with comma in config value fails
[
https://issues.apache.org/jira/browse/YARN-9593?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17045310#comment-17045310
]
Prabhu Joseph commented on YARN-9593:
-
Thanks [~tanu.ajmera] for the contribution. Thanks [~adam.antal] for the review.
[^YARN-9593-004.patch] looks good, +1. Will commit it shortly.
> Updating scheduler conf with comma in config value fails
>
>
> Key: YARN-9593
> URL: https://issues.apache.org/jira/browse/YARN-9593
> Project: Hadoop YARN
> Issue Type: Bug
>Affects Versions: 2.9.0, 3.0.0, 3.2.0, 3.1.2
>Reporter: Anthony Hsu
>Assignee: Tanu Ajmera
>Priority: Major
> Attachments: YARN-9593-001.patch, YARN-9593-002.patch,
> YARN-9593-003.patch, YARN-9593-004.patch
>
>
> For example:
> {code:java}
> $ yarn schedulerconf -update "root.gridops:acl_administer_queue=user1,user2
> group1,group2"
> Specify configuration key value as confKey=confVal.{code}
> This fails because there is a comma in the config value and the SchedConfCLI
> splits on comma first, expecting each split to a k=v pair.
> {noformat}
> void globalUpdates(String args, SchedConfUpdateInfo updateInfo) {
> if (args == null) {
> return;
> }
> HashMap globalUpdates = new HashMap<>();
> for (String globalUpdate : args.split(",")) {
> putKeyValuePair(globalUpdates, globalUpdate);
> }
> updateInfo.setGlobalParams(globalUpdates);
> }{noformat}
> Cc: [~jhung]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10141) Interceptor in FederationInterceptorREST doesnt update on RM switchover
[
https://issues.apache.org/jira/browse/YARN-10141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17045309#comment-17045309
]
Hadoop QA commented on YARN-10141:
--
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
37s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 23m
2s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
26s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
25s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
28s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
16m 9s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
33s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
20s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
12s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
14m 18s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
35s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
18s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m
32s{color} | {color:green} hadoop-yarn-server-router in the patch passed.
{color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
26s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 60m 40s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=19.03.6 Server=19.03.6 Image:yetus/hadoop:c44943d1fc3 |
| JIRA Issue | YARN-10141 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12994625/YARN-10141.004.patch |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient findbugs checkstyle |
| uname | Linux a641c479f686 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19
08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / d608e94 |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_242 |
| findbugs | v3.1.0-RC1 |
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/25585/testReport/ |
| Max. process+thread count | 745 (vs. ulimit of 5500) |
| modules | C:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router U:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router |
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/25585/console |
| Powered by | Apache Yetus 0.8.0 http://yetus.apache.org |
This message was automatically generated.
> Interceptor in FederationInterceptorREST doesnt update on RM
[jira] [Commented] (YARN-10160) Add auto queue creation related configs to RMWebService#CapacitySchedulerQueueInfo
[
https://issues.apache.org/jira/browse/YARN-10160?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17045290#comment-17045290
]
Hadoop QA commented on YARN-10160:
--
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
33s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 21m
20s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
43s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
34s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
46s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
15m 38s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
39s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
29s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
46s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
39s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
39s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
29s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
42s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
14m 35s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
45s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
27s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 95m
3s{color} | {color:green} hadoop-yarn-server-resourcemanager in the patch
passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
41s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}156m 37s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=19.03.6 Server=19.03.6 Image:yetus/hadoop:c44943d1fc3 |
| JIRA Issue | YARN-10160 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12994613/YARN-10124-002.patch |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient findbugs checkstyle |
| uname | Linux 4a1ad6315e22 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19
08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / 900430b |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_242 |
| findbugs | v3.1.0-RC1 |
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/25584/testReport/ |
| Max. process+thread count | 853 (vs. ulimit of 5500) |
| modules | C:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
U:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
|
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/25584/console |
| Powered by | Apache Yetus 0.8.0 http://yetus.apache.org |
This message was automatically generated.
> Add auto queue creation related
[jira] [Commented] (YARN-9344) FS should not reserve when container capability is bigger than node total resource
[
https://issues.apache.org/jira/browse/YARN-9344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17045276#comment-17045276
]
Hadoop QA commented on YARN-9344:
-
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 24m
6s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 25m
39s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
46s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
40s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
49s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
14m 25s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
32s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
35s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
44s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
37s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
37s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
31s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
42s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
12m 41s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
34s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
28s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 84m
6s{color} | {color:green} hadoop-yarn-server-resourcemanager in the patch
passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
31s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}170m 18s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=19.03.6 Server=19.03.6 Image:yetus/hadoop:c44943d1fc3 |
| JIRA Issue | YARN-9344 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12962434/YARN-9344.006.patch |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient findbugs checkstyle |
| uname | Linux f7d5ac68b77e 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22
16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / 900430b |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_242 |
| findbugs | v3.1.0-RC1 |
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/25573/testReport/ |
| Max. process+thread count | 877 (vs. ulimit of 5500) |
| modules | C:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
U:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
|
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/25573/console |
| Powered by | Apache Yetus 0.8.0 http://yetus.apache.org |
This message was automatically generated.
> FS should not reserve when container
[jira] [Commented] (YARN-8452) FairScheduler.update can take long time if yarn.scheduler.fair.sizebasedweight is on
[
https://issues.apache.org/jira/browse/YARN-8452?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17045275#comment-17045275
]
Hadoop QA commented on YARN-8452:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
41s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 27m
42s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
42s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
35s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
46s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
15m 30s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
35s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
29s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
41s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
37s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
37s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
27s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
40s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
1s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
14m 12s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
38s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
26s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red}102m 58s{color}
| {color:red} hadoop-yarn-server-resourcemanager in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
27s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}169m 57s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests |
hadoop.yarn.server.resourcemanager.security.TestDelegationTokenRenewer |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=19.03.6 Server=19.03.6 Image:yetus/hadoop:c44943d1fc3 |
| JIRA Issue | YARN-8452 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12929228/YARN-8452.003.patch |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient findbugs checkstyle |
| uname | Linux 7719f6439be2 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19
08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / 900430b |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_242 |
| findbugs | v3.1.0-RC1 |
| unit |
https://builds.apache.org/job/PreCommit-YARN-Build/25579/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/25579/testReport/ |
| Max. process+thread count | 822 (vs. ulimit of 5500) |
| modules | C:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
U:
hadoop-yarn-project/hadoop
[jira] [Commented] (YARN-10124) Remove restriction of ParentQueue capacity zero when childCapacities > 0
[
https://issues.apache.org/jira/browse/YARN-10124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17045262#comment-17045262
]
Hadoop QA commented on YARN-10124:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 1m
13s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 23m
25s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
46s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
38s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
49s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
16m 6s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
46s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
31s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
47s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
45s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
45s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
32s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
42s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
14m 19s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
45s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
29s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 89m 14s{color}
| {color:red} hadoop-yarn-server-resourcemanager in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
31s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}154m 10s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests |
hadoop.yarn.server.resourcemanager.reservation.TestCapacityOverTimePolicy |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=19.03.6 Server=19.03.6 Image:yetus/hadoop:c44943d1fc3 |
| JIRA Issue | YARN-10124 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12994612/YARN-10124-002.patch |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient findbugs checkstyle |
| uname | Linux b1394a15799e 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19
08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / 900430b |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_242 |
| findbugs | v3.1.0-RC1 |
| unit |
https://builds.apache.org/job/PreCommit-YARN-Build/25583/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/25583/testReport/ |
| Max. process+thread count | 822 (vs. ulimit of 5500) |
| modules | C:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
U:
hadoop-yarn-projec
[jira] [Commented] (YARN-10156) Fix typo 'complaint' which means quite different in Federation.md
[
https://issues.apache.org/jira/browse/YARN-10156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17045256#comment-17045256
]
Hudson commented on YARN-10156:
---
SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #17994 (See
[https://builds.apache.org/job/Hadoop-trunk-Commit/17994/])
YARN-10156. Fix typo 'complaint' which means quite different in (github: rev
d608e94f9208354897a59dc8e533535b1b8a30b9)
* (edit)
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/Federation.md
> Fix typo 'complaint' which means quite different in Federation.md
> -
>
> Key: YARN-10156
> URL: https://issues.apache.org/jira/browse/YARN-10156
> Project: Hadoop YARN
> Issue Type: Bug
> Components: documentation, federation
>Reporter: Sungpeo Kook
>Assignee: Sungpeo Kook
>Priority: Minor
> Fix For: 3.3.0, 2.9.3, 3.2.2, 3.1.4, 2.10.1
>
>
> [https://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/Federation.html#Running_Applications_across_Sub-Clusters]
> There is a typo in Fedration.md that could be confusing.
> {quote}{{`be complaint to`}} should be {{`be compliant with`}} on the context.
> {quote}
> I think that it need to be corrected.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10156) Fix typo 'complaint' which means quite different in Federation.md
[
https://issues.apache.org/jira/browse/YARN-10156?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Akira Ajisaka updated YARN-10156:
-
Priority: Minor (was: Major)
> Fix typo 'complaint' which means quite different in Federation.md
> -
>
> Key: YARN-10156
> URL: https://issues.apache.org/jira/browse/YARN-10156
> Project: Hadoop YARN
> Issue Type: Bug
> Components: documentation, federation
>Reporter: Sungpeo Kook
>Assignee: Sungpeo Kook
>Priority: Minor
> Fix For: 3.3.0, 2.9.3, 3.2.2, 3.1.4, 2.10.1
>
>
> [https://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/Federation.html#Running_Applications_across_Sub-Clusters]
> There is a typo in Fedration.md that could be confusing.
> {quote}{{`be complaint to`}} should be {{`be compliant with`}} on the context.
> {quote}
> I think that it need to be corrected.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Resolved] (YARN-10156) Fix typo 'complaint' which means quite different in Federation.md
[
https://issues.apache.org/jira/browse/YARN-10156?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Akira Ajisaka resolved YARN-10156.
--
Fix Version/s: 2.10.1
3.1.4
3.2.2
2.9.3
3.3.0
Hadoop Flags: Reviewed
Resolution: Fixed
Committed this to all the active branches. Thanks [~Elixir Kook] for the
contribution!
> Fix typo 'complaint' which means quite different in Federation.md
> -
>
> Key: YARN-10156
> URL: https://issues.apache.org/jira/browse/YARN-10156
> Project: Hadoop YARN
> Issue Type: Bug
> Components: documentation, federation
>Reporter: Sungpeo Kook
>Assignee: Sungpeo Kook
>Priority: Major
> Fix For: 3.3.0, 2.9.3, 3.2.2, 3.1.4, 2.10.1
>
>
> [https://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/Federation.html#Running_Applications_across_Sub-Clusters]
> There is a typo in Fedration.md that could be confusing.
> {quote}{{`be complaint to`}} should be {{`be compliant with`}} on the context.
> {quote}
> I think that it need to be corrected.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10141) Interceptor in FederationInterceptorREST doesnt update on RM switchover
[ https://issues.apache.org/jira/browse/YARN-10141?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] D M Murali Krishna Reddy updated YARN-10141: Attachment: YARN-10141.004.patch > Interceptor in FederationInterceptorREST doesnt update on RM switchover > --- > > Key: YARN-10141 > URL: https://issues.apache.org/jira/browse/YARN-10141 > Project: Hadoop YARN > Issue Type: Bug > Components: federation, restapi >Reporter: D M Murali Krishna Reddy >Assignee: D M Murali Krishna Reddy >Priority: Major > Attachments: YARN-10141.001.patch, YARN-10141.002.patch, > YARN-10141.003.patch, YARN-10141.004.patch > > > In Federation Setup, In the event of a RM switchover in a subcluster the > interceptor for that subcluster in FederationInterceptorREST doesnt get > updated. Due to this, Cluster Nodes REST API doesnt return the nodes from the > subcluster in which the RM switchover has occured. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Assigned] (YARN-10156) Fix typo 'complaint' which means quite different in Federation.md
[
https://issues.apache.org/jira/browse/YARN-10156?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Akira Ajisaka reassigned YARN-10156:
Assignee: Sungpeo Kook
> Fix typo 'complaint' which means quite different in Federation.md
> -
>
> Key: YARN-10156
> URL: https://issues.apache.org/jira/browse/YARN-10156
> Project: Hadoop YARN
> Issue Type: Bug
> Components: documentation, federation
>Reporter: Sungpeo Kook
>Assignee: Sungpeo Kook
>Priority: Major
>
> [https://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/Federation.html#Running_Applications_across_Sub-Clusters]
> There is a typo in Fedration.md that could be confusing.
> {quote}{{`be complaint to`}} should be {{`be compliant with`}} on the context.
> {quote}
> I think that it need to be corrected.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-8655) FairScheduler: FSStarvedApps is not thread safe
[
https://issues.apache.org/jira/browse/YARN-8655?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17045241#comment-17045241
]
Hadoop QA commented on YARN-8655:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
43s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 21m
32s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
44s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
34s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
47s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
15m 44s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
41s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
30s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
44s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
39s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
39s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
29s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
43s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
14m 22s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
43s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
26s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 75m 35s{color}
| {color:red} hadoop-yarn-server-resourcemanager in the patch passed. {color} |
| {color:blue}0{color} | {color:blue} asflicense {color} | {color:blue} 0m
37s{color} | {color:blue} ASF License check generated no output? {color} |
| {color:black}{color} | {color:black} {color} | {color:black}137m 26s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.yarn.server.resourcemanager.TestRMAdminService |
| | hadoop.yarn.server.resourcemanager.scheduler.fair.TestFSAppStarvation |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=19.03.6 Server=19.03.6 Image:yetus/hadoop:c44943d1fc3 |
| JIRA Issue | YARN-8655 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12958177/YARN-8655.002.patch |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient findbugs checkstyle |
| uname | Linux 223731e8bbb0 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19
08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / 900430b |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_242 |
| findbugs | v3.1.0-RC1 |
| unit |
https://builds.apache.org/job/PreCommit-YARN-Build/25569/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/25569/testReport/ |
| Max. process+thread count | 814 (vs. ulimit of 5500) |
| modules | C:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-reso
[jira] [Commented] (YARN-9198) Corrupted state from a previous version can still cause RM to fail with NPE on FairScheduler
[
https://issues.apache.org/jira/browse/YARN-9198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17045229#comment-17045229
]
Hadoop QA commented on YARN-9198:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
28s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 21m
0s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
42s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
35s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
45s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
15m 27s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
36s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
28s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
42s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
36s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
36s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
28s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
41s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
14m 4s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
36s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
26s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 55m
48s{color} | {color:green} hadoop-yarn-server-resourcemanager in the patch
passed. {color} |
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 1m
13s{color} | {color:red} The patch generated 1 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black}116m 27s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=19.03.6 Server=19.03.6 Image:yetus/hadoop:c44943d1fc3 |
| JIRA Issue | YARN-9198 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12954806/YARN-9198.001.patch |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient findbugs checkstyle |
| uname | Linux 1cf9f4dcdb70 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19
08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / 900430b |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_242 |
| findbugs | v3.1.0-RC1 |
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/25576/testReport/ |
| asflicense |
https://builds.apache.org/job/PreCommit-YARN-Build/25576/artifact/out/patch-asflicense-problems.txt
|
| Max. process+thread count | 819 (vs. ulimit of 5500) |
| modules | C:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
U:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
|
| Con
