[jira] [Resolved] (YARN-10879) Incorrect WARN text in ACL check for application tag based placement
[
https://issues.apache.org/jira/browse/YARN-10879?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Benjamin Teke resolved YARN-10879.
--
Hadoop Flags: Reviewed
Resolution: Fixed
> Incorrect WARN text in ACL check for application tag based placement
>
>
> Key: YARN-10879
> URL: https://issues.apache.org/jira/browse/YARN-10879
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Brian Goerlitz
>Assignee: Susheel Gupta
>Priority: Minor
> Labels: pull-request-available
> Fix For: 3.4.0
>
>
> After YARN-10070 the queue permissions check for application tag based
> placement is performed for the proxy user instead of the end user, but the
> warning message for permissions failure indicates that the end user is
> missing permissions.
> {code:java}
> LOG.warn("User '{}' from application tag does not have access to " +
> " queue '{}'. " + "The placement is done for user '{}'",
> userNameFromAppTag, queue, user);
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10879) Incorrect WARN text in ACL check for application tag based placement
[
https://issues.apache.org/jira/browse/YARN-10879?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Benjamin Teke updated YARN-10879:
-
Fix Version/s: 3.4.0
> Incorrect WARN text in ACL check for application tag based placement
>
>
> Key: YARN-10879
> URL: https://issues.apache.org/jira/browse/YARN-10879
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Brian Goerlitz
>Assignee: Susheel Gupta
>Priority: Minor
> Labels: pull-request-available
> Fix For: 3.4.0
>
>
> After YARN-10070 the queue permissions check for application tag based
> placement is performed for the proxy user instead of the end user, but the
> warning message for permissions failure indicates that the end user is
> missing permissions.
> {code:java}
> LOG.warn("User '{}' from application tag does not have access to " +
> " queue '{}'. " + "The placement is done for user '{}'",
> userNameFromAppTag, queue, user);
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11395) Resource Manager UI, cluster/appattempt/*, can not present FINAL_SAVING state
[
https://issues.apache.org/jira/browse/YARN-11395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17651687#comment-17651687
]
ASF GitHub Bot commented on YARN-11395:
---
brumi1024 closed pull request #5222: YARN-11395. RM UI, RMAttemptBlock can not
render FINAL_SAVING
URL: https://github.com/apache/hadoop/pull/5222
> Resource Manager UI, cluster/appattempt/*, can not present FINAL_SAVING state
> -
>
> Key: YARN-11395
> URL: https://issues.apache.org/jira/browse/YARN-11395
> Project: Hadoop YARN
> Issue Type: Bug
> Components: yarn
>Affects Versions: 3.4.0
>Reporter: Bence Kosztolnik
>Assignee: Bence Kosztolnik
>Priority: Critical
> Labels: pull-request-available
> Fix For: 3.4.0
>
>
> If an attempt is in *FINAL_SAVING* state, the
> *RMAppAttemptBlock#createAttemptHeadRoomTable* method fails with a convert
> error, what will results a
> {code:java}
> RFC6265 Cookie values may not contain character: [ ]{code}
> error in the UI an in the logs as well.
> RM log:
> {code:java}
> ...
> at java.lang.Thread.run(Thread.java:750)
> Caused by: java.lang.IllegalArgumentException: No enum constant
> org.apache.hadoop.yarn.api.records.YarnApplicationAttemptState.FINAL_SAVING
> at java.lang.Enum.valueOf(Enum.java:238)
> at
> org.apache.hadoop.yarn.api.records.YarnApplicationAttemptState.valueOf(YarnApplicationAttemptState.java:27)
> at
> org.apache.hadoop.yarn.server.resourcemanager.webapp.RMAppAttemptBlock.createAttemptHeadRoomTable(RMAppAttemptBlock.java:424)
> at
> org.apache.hadoop.yarn.server.webapp.AppAttemptBlock.render(AppAttemptBlock.java:151)
> at org.apache.hadoop.yarn.webapp.view.HtmlBlock.render(HtmlBlock.java:69)
> at
> org.apache.hadoop.yarn.webapp.view.HtmlBlock.renderPartial(HtmlBlock.java:79)
> at org.apache.hadoop.yarn.webapp.View.render(View.java:243)
> at
> org.apache.hadoop.yarn.webapp.view.HtmlPage$Page.subView(HtmlPage.java:49)
> at
> org.apache.hadoop.yarn.webapp.hamlet2.HamletImpl$EImp._v(HamletImpl.java:117)
> at org.apache.hadoop.yarn.webapp.hamlet2.Hamlet$TD.__(Hamlet.java:848)
> at
> org.apache.hadoop.yarn.webapp.view.TwoColumnLayout.render(TwoColumnLayout.java:71)
> at org.apache.hadoop.yarn.webapp.view.HtmlPage.render(HtmlPage.java:82)
> at org.apache.hadoop.yarn.webapp.Controller.render(Controller.java:216)
> at
> org.apache.hadoop.yarn.server.resourcemanager.webapp.RmController.appattempt(RmController.java:62)
> ... 63 more
> 2022-12-05 04:15:33,029 WARN org.eclipse.jetty.server.HttpChannel:
> /cluster/appattempt/appattempt_1667297151262_0247_01
> java.lang.IllegalArgumentException: RFC6265 Cookie values may not contain
> character: [ ]
> at
> org.eclipse.jetty.http.Syntax.requireValidRFC6265CookieValue(Syntax.java:136)
> ...{code}
> This bug was introduced with the YARN-1345 ticket what also caused a similar
> error called YARN-4411. In case of the YARN-4411 the enum mapping logic from
> RMAppAttemptStates to YarnApplicationAttemptState was modified like this:
> - if the state is FINAL_SAVING we should represent the previous state
> This error can also be occur in case of ALLOCATED_SAVING,
> LAUNCHED_UNMANAGED_SAVING states as well.
> So we should modify the *createAttemptHeadRoomTable* method to be able to
> handle the previously mentioned 3 state just like in case of YARN-4411
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11395) Resource Manager UI, cluster/appattempt/*, can not present FINAL_SAVING state
[
https://issues.apache.org/jira/browse/YARN-11395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17651686#comment-17651686
]
ASF GitHub Bot commented on YARN-11395:
---
brumi1024 commented on PR #5222:
URL: https://github.com/apache/hadoop/pull/5222#issuecomment-1364098900
Merged to trunk, branch-3.3 and branch-3.2.
> Resource Manager UI, cluster/appattempt/*, can not present FINAL_SAVING state
> -
>
> Key: YARN-11395
> URL: https://issues.apache.org/jira/browse/YARN-11395
> Project: Hadoop YARN
> Issue Type: Bug
> Components: yarn
>Affects Versions: 3.4.0
>Reporter: Bence Kosztolnik
>Assignee: Bence Kosztolnik
>Priority: Critical
> Labels: pull-request-available
> Fix For: 3.4.0
>
>
> If an attempt is in *FINAL_SAVING* state, the
> *RMAppAttemptBlock#createAttemptHeadRoomTable* method fails with a convert
> error, what will results a
> {code:java}
> RFC6265 Cookie values may not contain character: [ ]{code}
> error in the UI an in the logs as well.
> RM log:
> {code:java}
> ...
> at java.lang.Thread.run(Thread.java:750)
> Caused by: java.lang.IllegalArgumentException: No enum constant
> org.apache.hadoop.yarn.api.records.YarnApplicationAttemptState.FINAL_SAVING
> at java.lang.Enum.valueOf(Enum.java:238)
> at
> org.apache.hadoop.yarn.api.records.YarnApplicationAttemptState.valueOf(YarnApplicationAttemptState.java:27)
> at
> org.apache.hadoop.yarn.server.resourcemanager.webapp.RMAppAttemptBlock.createAttemptHeadRoomTable(RMAppAttemptBlock.java:424)
> at
> org.apache.hadoop.yarn.server.webapp.AppAttemptBlock.render(AppAttemptBlock.java:151)
> at org.apache.hadoop.yarn.webapp.view.HtmlBlock.render(HtmlBlock.java:69)
> at
> org.apache.hadoop.yarn.webapp.view.HtmlBlock.renderPartial(HtmlBlock.java:79)
> at org.apache.hadoop.yarn.webapp.View.render(View.java:243)
> at
> org.apache.hadoop.yarn.webapp.view.HtmlPage$Page.subView(HtmlPage.java:49)
> at
> org.apache.hadoop.yarn.webapp.hamlet2.HamletImpl$EImp._v(HamletImpl.java:117)
> at org.apache.hadoop.yarn.webapp.hamlet2.Hamlet$TD.__(Hamlet.java:848)
> at
> org.apache.hadoop.yarn.webapp.view.TwoColumnLayout.render(TwoColumnLayout.java:71)
> at org.apache.hadoop.yarn.webapp.view.HtmlPage.render(HtmlPage.java:82)
> at org.apache.hadoop.yarn.webapp.Controller.render(Controller.java:216)
> at
> org.apache.hadoop.yarn.server.resourcemanager.webapp.RmController.appattempt(RmController.java:62)
> ... 63 more
> 2022-12-05 04:15:33,029 WARN org.eclipse.jetty.server.HttpChannel:
> /cluster/appattempt/appattempt_1667297151262_0247_01
> java.lang.IllegalArgumentException: RFC6265 Cookie values may not contain
> character: [ ]
> at
> org.eclipse.jetty.http.Syntax.requireValidRFC6265CookieValue(Syntax.java:136)
> ...{code}
> This bug was introduced with the YARN-1345 ticket what also caused a similar
> error called YARN-4411. In case of the YARN-4411 the enum mapping logic from
> RMAppAttemptStates to YarnApplicationAttemptState was modified like this:
> - if the state is FINAL_SAVING we should represent the previous state
> This error can also be occur in case of ALLOCATED_SAVING,
> LAUNCHED_UNMANAGED_SAVING states as well.
> So we should modify the *createAttemptHeadRoomTable* method to be able to
> handle the previously mentioned 3 state just like in case of YARN-4411
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11320) [Federation] Add getSchedulerInfo REST APIs for Router
[
https://issues.apache.org/jira/browse/YARN-11320?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17651676#comment-17651676
]
ASF GitHub Bot commented on YARN-11320:
---
hadoop-yetus commented on PR #5217:
URL: https://github.com/apache/hadoop/pull/5217#issuecomment-1364041181
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 2m 24s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 2 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 15m 23s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 29m 42s | | trunk passed |
| +1 :green_heart: | compile | 4m 17s | | trunk passed with JDK
Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04 |
| +1 :green_heart: | compile | 3m 31s | | trunk passed with JDK
Private Build-1.8.0_352-8u352-ga-1~20.04-b08 |
| +1 :green_heart: | checkstyle | 1m 21s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 42s | | trunk passed |
| -1 :x: | javadoc | 1m 1s |
[/branch-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdkUbuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5217/10/artifact/out/branch-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdkUbuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04.txt)
| hadoop-yarn-server-resourcemanager in trunk failed with JDK
Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04. |
| +1 :green_heart: | javadoc | 1m 5s | | trunk passed with JDK
Private Build-1.8.0_352-8u352-ga-1~20.04-b08 |
| +1 :green_heart: | spotbugs | 3m 11s | | trunk passed |
| +1 :green_heart: | shadedclient | 23m 41s | | branch has no errors
when building and testing our client artifacts. |
| -0 :warning: | patch | 24m 0s | | Used diff version of patch file.
Binary files and potentially other changes not applied. Please rebase and
squash commits if necessary. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 40s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 1m 26s | | the patch passed |
| +1 :green_heart: | compile | 4m 3s | | the patch passed with JDK
Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04 |
| +1 :green_heart: | javac | 4m 3s | | the patch passed |
| +1 :green_heart: | compile | 3m 20s | | the patch passed with JDK
Private Build-1.8.0_352-8u352-ga-1~20.04-b08 |
| +1 :green_heart: | javac | 3m 20s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 1m 13s | |
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server: The patch generated 0 new +
35 unchanged - 1 fixed = 35 total (was 36) |
| +1 :green_heart: | mvnsite | 1m 25s | | the patch passed |
| -1 :x: | javadoc | 0m 47s |
[/patch-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdkUbuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5217/10/artifact/out/patch-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdkUbuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04.txt)
| hadoop-yarn-server-resourcemanager in the patch failed with JDK
Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04. |
| +1 :green_heart: | javadoc | 1m 1s | | the patch passed with JDK
Private Build-1.8.0_352-8u352-ga-1~20.04-b08 |
| +1 :green_heart: | spotbugs | 3m 1s | | the patch passed |
| +1 :green_heart: | shadedclient | 23m 57s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| -1 :x: | unit | 121m 3s |
[/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5217/10/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt)
| hadoop-yarn-server-resourcemanager in the patch passed. |
| +1 :green_heart: | unit | 0m 34s | | hadoop-yarn-server-router in
the patch passed. |
| +1 :green_hea
[jira] [Commented] (YARN-11393) Fs2cs could be extended to set ULF to -1 upon conversion
[
https://issues.apache.org/jira/browse/YARN-11393?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17651662#comment-17651662
]
ASF GitHub Bot commented on YARN-11393:
---
hadoop-yetus commented on PR #5201:
URL: https://github.com/apache/hadoop/pull/5201#issuecomment-1363955992
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 53s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 1 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 41m 20s | | trunk passed |
| +1 :green_heart: | compile | 1m 6s | | trunk passed with JDK
Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04 |
| +1 :green_heart: | compile | 0m 56s | | trunk passed with JDK
Private Build-1.8.0_352-8u352-ga-1~20.04-b08 |
| +1 :green_heart: | checkstyle | 0m 51s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 3s | | trunk passed |
| -1 :x: | javadoc | 0m 55s |
[/branch-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdkUbuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5201/4/artifact/out/branch-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdkUbuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04.txt)
| hadoop-yarn-server-resourcemanager in trunk failed with JDK
Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04. |
| +1 :green_heart: | javadoc | 0m 41s | | trunk passed with JDK
Private Build-1.8.0_352-8u352-ga-1~20.04-b08 |
| +1 :green_heart: | spotbugs | 2m 9s | | trunk passed |
| +1 :green_heart: | shadedclient | 24m 11s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 51s | | the patch passed |
| +1 :green_heart: | compile | 0m 59s | | the patch passed with JDK
Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04 |
| +1 :green_heart: | javac | 0m 59s | | the patch passed |
| +1 :green_heart: | compile | 0m 50s | | the patch passed with JDK
Private Build-1.8.0_352-8u352-ga-1~20.04-b08 |
| +1 :green_heart: | javac | 0m 50s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| -0 :warning: | checkstyle | 0m 38s |
[/results-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5201/4/artifact/out/results-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt)
|
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager:
The patch generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0) |
| +1 :green_heart: | mvnsite | 0m 54s | | the patch passed |
| -1 :x: | javadoc | 0m 39s |
[/patch-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdkUbuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5201/4/artifact/out/patch-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdkUbuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04.txt)
| hadoop-yarn-server-resourcemanager in the patch failed with JDK
Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04. |
| +1 :green_heart: | javadoc | 0m 36s | | the patch passed with JDK
Private Build-1.8.0_352-8u352-ga-1~20.04-b08 |
| +1 :green_heart: | spotbugs | 2m 0s | | the patch passed |
| +1 :green_heart: | shadedclient | 24m 11s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 117m 27s | |
hadoop-yarn-server-resourcemanager in the patch passed. |
| +1 :green_heart: | asflicense | 0m 33s | | The patch does not
generate ASF License warnings. |
| | | 222m 57s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5201/4/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/5201
[jira] [Commented] (YARN-11225) [Federation] Add postDelegationToken, postDelegationTokenExpiration, cancelDelegationToken REST APIs for Router
[
https://issues.apache.org/jira/browse/YARN-11225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17651656#comment-17651656
]
ASF GitHub Bot commented on YARN-11225:
---
hadoop-yetus commented on PR #5185:
URL: https://github.com/apache/hadoop/pull/5185#issuecomment-1363937688
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 51s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 3 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 41m 12s | | trunk passed |
| +1 :green_heart: | compile | 0m 31s | | trunk passed with JDK
Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04 |
| +1 :green_heart: | compile | 0m 29s | | trunk passed with JDK
Private Build-1.8.0_352-8u352-ga-1~20.04-b08 |
| +1 :green_heart: | checkstyle | 0m 28s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 32s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 36s | | trunk passed with JDK
Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04 |
| +1 :green_heart: | javadoc | 0m 23s | | trunk passed with JDK
Private Build-1.8.0_352-8u352-ga-1~20.04-b08 |
| +1 :green_heart: | spotbugs | 1m 3s | | trunk passed |
| +1 :green_heart: | shadedclient | 22m 58s | | branch has no errors
when building and testing our client artifacts. |
| -0 :warning: | patch | 23m 18s | | Used diff version of patch file.
Binary files and potentially other changes not applied. Please rebase and
squash commits if necessary. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 23s | | the patch passed |
| +1 :green_heart: | compile | 0m 25s | | the patch passed with JDK
Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04 |
| +1 :green_heart: | javac | 0m 25s | | the patch passed |
| +1 :green_heart: | compile | 0m 22s | | the patch passed with JDK
Private Build-1.8.0_352-8u352-ga-1~20.04-b08 |
| +1 :green_heart: | javac | 0m 22s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 0m 14s | | the patch passed |
| +1 :green_heart: | mvnsite | 0m 25s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 19s | | the patch passed with JDK
Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04 |
| +1 :green_heart: | javadoc | 0m 19s | | the patch passed with JDK
Private Build-1.8.0_352-8u352-ga-1~20.04-b08 |
| +1 :green_heart: | spotbugs | 0m 53s | | the patch passed |
| +1 :green_heart: | shadedclient | 22m 42s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 0m 28s | | hadoop-yarn-server-router in
the patch passed. |
| +1 :green_heart: | asflicense | 0m 34s | | The patch does not
generate ASF License warnings. |
| | | 97m 29s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5185/12/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/5185 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux 001c080c0bc3 4.15.0-200-generic #211-Ubuntu SMP Thu Nov 24
18:16:04 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / f79629f0bc90d881c61defcb62272fbe6b47c6ce |
| Default Java | Private Build-1.8.0_352-8u352-ga-1~20.04-b08 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04
/usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_352-8u352-ga-1~20.04-b08 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5185/12/testReport/ |
| Max. process+thread count | 548 (vs. ulimit of 5500) |
| modules | C:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router U:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibr
[jira] [Commented] (YARN-11393) Fs2cs could be extended to set ULF to -1 upon conversion
[
https://issues.apache.org/jira/browse/YARN-11393?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17651573#comment-17651573
]
ASF GitHub Bot commented on YARN-11393:
---
brumi1024 commented on code in PR #5201:
URL: https://github.com/apache/hadoop/pull/5201#discussion_r1056205229
##
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/converter/FSConfigToCSConfigConverter.java:
##
@@ -310,6 +313,7 @@ private void convertCapacitySchedulerXml(FairScheduler fs) {
.withPercentages(usePercentages)
.build();
+queueConverter.emitDefaultUserLimitFactor(fsLeafQueue,
capacitySchedulerConfig);
Review Comment:
This should ideally be called from inside the queueConverter, as this method
operates on queues themselves, not on the whole structure. This way there is no
need to get the leafQueues array.
##
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/converter/FSQueueConverter.java:
##
@@ -215,6 +219,19 @@ private void emitPreemptionDisabled(String queueName,
FSQueue queue) {
}
}
+ public void emitDefaultUserLimitFactor(Collection fsLeafQueue,
CapacitySchedulerConfiguration config) {
+fsLeafQueue
Review Comment:
There is no need to iterate through the queues, as the convertQueueHierarchy
does that. Calling this method there and checking if the queue that we're
currently converting is a leafQueue should achieve what we're trying to do.
> Fs2cs could be extended to set ULF to -1 upon conversion
>
>
> Key: YARN-11393
> URL: https://issues.apache.org/jira/browse/YARN-11393
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: yarn
>Reporter: Susheel Gupta
>Assignee: Susheel Gupta
>Priority: Major
> Labels: pull-request-available
>
> A global configuration to set the default User Limit Factor to -1 on newly
> created queues.
> To solve this is to make fs2cs (Fair Scheduler to Capacity Scheduler tool)
> add the user-limit-factor value -1 to the conversion as default.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11392) ClientRMService implemented getCallerUgi and verifyUserAccessForRMApp methods but forget to use sometimes, caused audit log missing.
[
https://issues.apache.org/jira/browse/YARN-11392?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17651558#comment-17651558
]
ASF GitHub Bot commented on YARN-11392:
---
curie71 commented on PR #5250:
URL: https://github.com/apache/hadoop/pull/5250#issuecomment-1363753000
@cnauroth, thank you for your review. XD
`forceKillApplication` and `submitApplication` has a similar code pattern
but a little different. They log `callerContext` and
`submissionContext.getQueue()` for some reason I don' t know, so I just keep
them as before.
> ClientRMService implemented getCallerUgi and verifyUserAccessForRMApp methods
> but forget to use sometimes, caused audit log missing.
>
>
> Key: YARN-11392
> URL: https://issues.apache.org/jira/browse/YARN-11392
> Project: Hadoop YARN
> Issue Type: Bug
> Components: yarn
>Affects Versions: 3.3.4
>Reporter: Beibei Zhao
>Assignee: Beibei Zhao
>Priority: Major
> Labels: audit, log, pull-request-available, yarn
>
> ClientRMService implemented getCallerUgi and verifyUserAccessForRMApp methods.
> {code:java}
> private UserGroupInformation getCallerUgi(ApplicationId applicationId,
> String operation) throws YarnException {
> UserGroupInformation callerUGI;
> try {
> callerUGI = UserGroupInformation.getCurrentUser();
> } catch (IOException ie) {
> LOG.info("Error getting UGI ", ie);
> RMAuditLogger.logFailure("UNKNOWN", operation, "UNKNOWN",
> "ClientRMService", "Error getting UGI", applicationId);
> throw RPCUtil.getRemoteException(ie);
> }
> return callerUGI;
> }
> {code}
> *Privileged operations* like "getContainerReport" (which called checkAccess
> before op) will call them and *record audit logs* when an *exception*
> happens, but forget to use sometimes, caused audit log {*}missing{*}:
> {code:java}
> // getApplicationReport
> UserGroupInformation callerUGI;
> try {
> callerUGI = UserGroupInformation.getCurrentUser();
> } catch (IOException ie) {
> LOG.info("Error getting UGI ", ie);
> // a logFailure should be called here.
> throw RPCUtil.getRemoteException(ie);
> }
> {code}
> So, I will replace some code blocks like this with getCallerUgi or
> verifyUserAccessForRMApp.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
