[jira] [Commented] (YARN-11303) Upgrade jquery ui to 1.13.2
[ https://issues.apache.org/jira/browse/YARN-11303?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17605623#comment-17605623 ] ASF GitHub Bot commented on YARN-11303: --- ashutoshcipher commented on PR #4895: URL: https://github.com/apache/hadoop/pull/4895#issuecomment-1248944187 I build the with the patch and the UI seems to working fine. Attaching screenshot for reference. https://user-images.githubusercontent.com/17571497/190565691-a3518caf-3caf-4f7f-883d-310f4ec5e6f4.png;> > Upgrade jquery ui to 1.13.2 > --- > > Key: YARN-11303 > URL: https://issues.apache.org/jira/browse/YARN-11303 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: D M Murali Krishna Reddy >Assignee: Ashutosh Gupta >Priority: Major > Labels: pull-request-available > > The current jquery-ui version used(1.13.1) in the trunk has the following > vulnerability > [CVE-2022-31160|https://nvd.nist.gov/vuln/detail/CVE-2022-31160] so we need > to upgrade to at least 1.13.2. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-11303) Upgrade jquery ui to 1.13.2
[ https://issues.apache.org/jira/browse/YARN-11303?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated YARN-11303: -- Labels: pull-request-available (was: ) > Upgrade jquery ui to 1.13.2 > --- > > Key: YARN-11303 > URL: https://issues.apache.org/jira/browse/YARN-11303 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: D M Murali Krishna Reddy >Assignee: Ashutosh Gupta >Priority: Major > Labels: pull-request-available > > The current jquery-ui version used(1.13.1) in the trunk has the following > vulnerability > [CVE-2022-31160|https://nvd.nist.gov/vuln/detail/CVE-2022-31160] so we need > to upgrade to at least 1.13.2. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11303) Upgrade jquery ui to 1.13.2
[ https://issues.apache.org/jira/browse/YARN-11303?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17605622#comment-17605622 ] ASF GitHub Bot commented on YARN-11303: --- ashutoshcipher opened a new pull request, #4895: URL: https://github.com/apache/hadoop/pull/4895 ### Description of PR Upgrade jquery ui to 1.13.2 to mitigate CVE-2022-31160 JIRA - YARN-11303 ### How was this patch tested? Verified on UI ### For code changes: - [X] Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')? - [ ] Object storage: have the integration tests been executed and the endpoint declared according to the connector-specific documentation? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the `LICENSE`, `LICENSE-binary`, `NOTICE-binary` files? > Upgrade jquery ui to 1.13.2 > --- > > Key: YARN-11303 > URL: https://issues.apache.org/jira/browse/YARN-11303 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: D M Murali Krishna Reddy >Assignee: Ashutosh Gupta >Priority: Major > > The current jquery-ui version used(1.13.1) in the trunk has the following > vulnerability > [CVE-2022-31160|https://nvd.nist.gov/vuln/detail/CVE-2022-31160] so we need > to upgrade to at least 1.13.2. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Assigned] (YARN-11303) Upgrade jquery ui to 1.13.2
[ https://issues.apache.org/jira/browse/YARN-11303?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Gupta reassigned YARN-11303: - Assignee: Ashutosh Gupta (was: D M Murali Krishna Reddy) > Upgrade jquery ui to 1.13.2 > --- > > Key: YARN-11303 > URL: https://issues.apache.org/jira/browse/YARN-11303 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: D M Murali Krishna Reddy >Assignee: Ashutosh Gupta >Priority: Major > > The current jquery-ui version used(1.13.1) in the trunk has the following > vulnerability > [CVE-2022-31160|https://nvd.nist.gov/vuln/detail/CVE-2022-31160] so we need > to upgrade to at least 1.13.2. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11207) Add configuration to disable queue's user limit
[
https://issues.apache.org/jira/browse/YARN-11207?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17605620#comment-17605620
]
ASF GitHub Bot commented on YARN-11207:
---
hadoop-yetus commented on PR #4891:
URL: https://github.com/apache/hadoop/pull/4891#issuecomment-1248941261
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 54s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 1 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 42m 1s | | trunk passed |
| +1 :green_heart: | compile | 1m 13s | | trunk passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | compile | 1m 4s | | trunk passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 1m 2s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 11s | | trunk passed |
| +1 :green_heart: | javadoc | 1m 3s | | trunk passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 0m 50s | | trunk passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 2m 17s | | trunk passed |
| +1 :green_heart: | shadedclient | 24m 52s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 56s | | the patch passed |
| +1 :green_heart: | compile | 1m 6s | | the patch passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javac | 1m 6s | | the patch passed |
| +1 :green_heart: | compile | 0m 56s | | the patch passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | javac | 0m 56s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| -0 :warning: | checkstyle | 0m 48s |
[/results-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4891/2/artifact/out/results-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt)
|
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager:
The patch generated 7 new + 374 unchanged - 0 fixed = 381 total (was 374) |
| +1 :green_heart: | mvnsite | 0m 58s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 45s | | the patch passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 0m 39s | | the patch passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 2m 4s | | the patch passed |
| +1 :green_heart: | shadedclient | 24m 14s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 102m 10s | |
hadoop-yarn-server-resourcemanager in the patch passed. |
| +1 :green_heart: | asflicense | 0m 39s | | The patch does not
generate ASF License warnings. |
| | | 211m 13s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4891/2/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/4891 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux 906aa5757d88 4.15.0-191-generic #202-Ubuntu SMP Thu Aug 4
01:49:29 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 67e0d573194383fbcb725decb8862be686028875 |
| Default Java | Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4891/2/testReport/ |
| Max.
[jira] [Commented] (YARN-11226) [Federation] Add createNewReservation, submitReservation, updateReservation, deleteReservation REST APIs for Router
[
https://issues.apache.org/jira/browse/YARN-11226?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17605618#comment-17605618
]
ASF GitHub Bot commented on YARN-11226:
---
hadoop-yetus commented on PR #4892:
URL: https://github.com/apache/hadoop/pull/4892#issuecomment-1248936669
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 53s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 4 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 41m 27s | | trunk passed |
| +1 :green_heart: | compile | 0m 37s | | trunk passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | compile | 0m 34s | | trunk passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 0m 35s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 40s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 44s | | trunk passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 0m 32s | | trunk passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 1m 6s | | trunk passed |
| +1 :green_heart: | shadedclient | 23m 36s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 26s | | the patch passed |
| +1 :green_heart: | compile | 0m 29s | | the patch passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javac | 0m 29s | | the patch passed |
| +1 :green_heart: | compile | 0m 25s | | the patch passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | javac | 0m 25s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| -0 :warning: | checkstyle | 0m 18s |
[/results-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-router.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4892/3/artifact/out/results-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-router.txt)
|
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router:
The patch generated 6 new + 1 unchanged - 0 fixed = 7 total (was 1) |
| +1 :green_heart: | mvnsite | 0m 27s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 23s | | the patch passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 0m 21s | | the patch passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 0m 54s | | the patch passed |
| +1 :green_heart: | shadedclient | 23m 23s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 4m 4s | | hadoop-yarn-server-router in
the patch passed. |
| +1 :green_heart: | asflicense | 0m 40s | | The patch does not
generate ASF License warnings. |
| | | 104m 11s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4892/3/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/4892 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux 5aa66bec1ef7 4.15.0-191-generic #202-Ubuntu SMP Thu Aug 4
01:49:29 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 6e8af507f56cb45f3bc5d6939c34a05cfe4a6984 |
| Default Java | Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4892/3/testReport/ |
| Max. process+thread count | 706 (vs. ulimit of
[jira] [Assigned] (YARN-9049) Add application submit data to state store
[ https://issues.apache.org/jira/browse/YARN-9049?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] fanshilun reassigned YARN-9049: --- Assignee: fanshilun (was: Bibin Chundatt) > Add application submit data to state store > -- > > Key: YARN-9049 > URL: https://issues.apache.org/jira/browse/YARN-9049 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Bibin Chundatt >Assignee: fanshilun >Priority: Major > Attachments: YARN-9049.001.path > > > As per the discussion in YARN-8898 we need to persist trimmend > ApplicationSubmissionContext details to federation State Store. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11290) Improve Query Condition of FederationStateStore#getApplicationsHomeSubCluster
[ https://issues.apache.org/jira/browse/YARN-11290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17605575#comment-17605575 ] ASF GitHub Bot commented on YARN-11290: --- slfan1989 commented on code in PR #4846: URL: https://github.com/apache/hadoop/pull/4846#discussion_r972541463 ## hadoop-yarn-project/hadoop-yarn/bin/FederationStateStore/SQLServer/FederationStateStoreStoreProcs.sql: ## @@ -111,12 +111,24 @@ IF OBJECT_ID ( '[sp_getApplicationsHomeSubCluster]', 'P' ) IS NOT NULL GO CREATE PROCEDURE [dbo].[sp_getApplicationsHomeSubCluster] +@limit int, +@homeSubCluster VARCHAR(256) AS BEGIN DECLARE @errorMessage nvarchar(4000) BEGIN TRY -SELECT [applicationId], [homeSubCluster], [createTime] -FROM [dbo].[applicationsHomeSubCluster] +SELECT +[applicationId], +[homeSubCluster], +[createTime] +FROM(SELECT + [applicationId], + [homeSubCluster], + [createTime], + row_number() over(order by [createTime] desc) AS app_rank + FROM [dbo].[applicationsHomeSubCluster] + WHERE [homeSubCluster] = @homeSubCluster OR @homeSubCluster = '') AS t Review Comment: Thank you very much for helping to review the code, I will modify the code. > Improve Query Condition of FederationStateStore#getApplicationsHomeSubCluster > - > > Key: YARN-11290 > URL: https://issues.apache.org/jira/browse/YARN-11290 > Project: Hadoop YARN > Issue Type: Improvement > Components: federation >Affects Versions: 3.4.0 >Reporter: fanshilun >Assignee: fanshilun >Priority: Minor > Labels: pull-request-available > Attachments: SqlServer-StoredProc.png > > > 1. Currently this interface returns the number of apps in all sub-clusters, > increasing the limit on the number of query apps, limited to 1000 apps. > 2. Allows to query the App based on the specified HomeSubCluster. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11290) Improve Query Condition of FederationStateStore#getApplicationsHomeSubCluster
[ https://issues.apache.org/jira/browse/YARN-11290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17605541#comment-17605541 ] ASF GitHub Bot commented on YARN-11290: --- goiri commented on code in PR #4846: URL: https://github.com/apache/hadoop/pull/4846#discussion_r972467705 ## hadoop-yarn-project/hadoop-yarn/bin/FederationStateStore/SQLServer/FederationStateStoreStoreProcs.sql: ## @@ -111,12 +111,24 @@ IF OBJECT_ID ( '[sp_getApplicationsHomeSubCluster]', 'P' ) IS NOT NULL GO CREATE PROCEDURE [dbo].[sp_getApplicationsHomeSubCluster] +@limit int, +@homeSubCluster VARCHAR(256) AS BEGIN DECLARE @errorMessage nvarchar(4000) BEGIN TRY -SELECT [applicationId], [homeSubCluster], [createTime] -FROM [dbo].[applicationsHomeSubCluster] +SELECT +[applicationId], +[homeSubCluster], +[createTime] +FROM(SELECT + [applicationId], + [homeSubCluster], + [createTime], + row_number() over(order by [createTime] desc) AS app_rank + FROM [dbo].[applicationsHomeSubCluster] + WHERE [homeSubCluster] = @homeSubCluster OR @homeSubCluster = '') AS t Review Comment: In the other one we use AS applicationshomesubcluster; let's try to be consistent with the names as possible. > Improve Query Condition of FederationStateStore#getApplicationsHomeSubCluster > - > > Key: YARN-11290 > URL: https://issues.apache.org/jira/browse/YARN-11290 > Project: Hadoop YARN > Issue Type: Improvement > Components: federation >Affects Versions: 3.4.0 >Reporter: fanshilun >Assignee: fanshilun >Priority: Minor > Labels: pull-request-available > Attachments: SqlServer-StoredProc.png > > > 1. Currently this interface returns the number of apps in all sub-clusters, > increasing the limit on the number of query apps, limited to 1000 apps. > 2. Allows to query the App based on the specified HomeSubCluster. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11255) Support loading alternative docker client config from system environment
[
https://issues.apache.org/jira/browse/YARN-11255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17605539#comment-17605539
]
ASF GitHub Bot commented on YARN-11255:
---
hadoop-yetus commented on PR #4884:
URL: https://github.com/apache/hadoop/pull/4884#issuecomment-1248687610
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 58s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +0 :ok: | markdownlint | 0m 0s | | markdownlint was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 1 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 16m 14s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 28m 37s | | trunk passed |
| +1 :green_heart: | compile | 10m 37s | | trunk passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | compile | 9m 6s | | trunk passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 2m 2s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 57s | | trunk passed |
| +1 :green_heart: | javadoc | 1m 51s | | trunk passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 1m 40s | | trunk passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +0 :ok: | spotbugs | 0m 42s | |
branch/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site no spotbugs output file
(spotbugsXml.xml) |
| +1 :green_heart: | shadedclient | 23m 31s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 25s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 0m 57s | | the patch passed |
| +1 :green_heart: | compile | 9m 56s | | the patch passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javac | 9m 56s | | the patch passed |
| +1 :green_heart: | compile | 9m 3s | | the patch passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | javac | 9m 3s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 1m 51s | | the patch passed |
| +1 :green_heart: | mvnsite | 1m 43s | | the patch passed |
| +1 :green_heart: | javadoc | 1m 30s | | the patch passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 1m 28s | | the patch passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +0 :ok: | spotbugs | 0m 38s | |
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site has no data from spotbugs |
| +1 :green_heart: | shadedclient | 23m 43s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 23m 58s | | hadoop-yarn-server-nodemanager
in the patch passed. |
| +1 :green_heart: | unit | 0m 38s | | hadoop-yarn-site in the patch
passed. |
| +1 :green_heart: | asflicense | 1m 3s | | The patch does not
generate ASF License warnings. |
| | | 182m 8s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4884/4/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/4884 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets markdownlint
|
| uname | Linux 57108abb0d6c 4.15.0-191-generic #202-Ubuntu SMP Thu Aug 4
01:49:29 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / a106f38a92212fb59bf2a8d6ab075e68c698e5d1 |
| Default Java | Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| Test Results |
[jira] [Commented] (YARN-11290) Improve Query Condition of FederationStateStore#getApplicationsHomeSubCluster
[ https://issues.apache.org/jira/browse/YARN-11290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17605523#comment-17605523 ] ASF GitHub Bot commented on YARN-11290: --- slfan1989 commented on PR #4846: URL: https://github.com/apache/hadoop/pull/4846#issuecomment-1248591269 @goiri Please help to review the code again, thank you very much! Error in TestLogAggregationService in unit test, I have submitted pr separately to fix. (https://github.com/apache/hadoop/pull/4893) > Improve Query Condition of FederationStateStore#getApplicationsHomeSubCluster > - > > Key: YARN-11290 > URL: https://issues.apache.org/jira/browse/YARN-11290 > Project: Hadoop YARN > Issue Type: Improvement > Components: federation >Affects Versions: 3.4.0 >Reporter: fanshilun >Assignee: fanshilun >Priority: Minor > Labels: pull-request-available > Attachments: SqlServer-StoredProc.png > > > 1. Currently this interface returns the number of apps in all sub-clusters, > increasing the limit on the number of query apps, limited to 1000 apps. > 2. Allows to query the App based on the specified HomeSubCluster. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11293) [Federation] Router Support DelegationToken storeNewMasterKey/removeStoredMasterKey With MemoryStateStore
[ https://issues.apache.org/jira/browse/YARN-11293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17605522#comment-17605522 ] ASF GitHub Bot commented on YARN-11293: --- slfan1989 commented on PR #4852: URL: https://github.com/apache/hadoop/pull/4852#issuecomment-1248584856 @goiri Please help to review the code again, thank you very much! > [Federation] Router Support DelegationToken > storeNewMasterKey/removeStoredMasterKey With MemoryStateStore > - > > Key: YARN-11293 > URL: https://issues.apache.org/jira/browse/YARN-11293 > Project: Hadoop YARN > Issue Type: Sub-task > Components: federation >Affects Versions: 3.4.0 >Reporter: fanshilun >Assignee: fanshilun >Priority: Major > Labels: pull-request-available > > Support storing storeNewMasterKey/removeStoredMasterKey in MemoryStore mode. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11226) [Federation] Add createNewReservation, submitReservation, updateReservation, deleteReservation REST APIs for Router
[
https://issues.apache.org/jira/browse/YARN-11226?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17605456#comment-17605456
]
ASF GitHub Bot commented on YARN-11226:
---
hadoop-yetus commented on PR #4892:
URL: https://github.com/apache/hadoop/pull/4892#issuecomment-1248384024
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 40s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 2 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 38m 28s | | trunk passed |
| +1 :green_heart: | compile | 0m 44s | | trunk passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | compile | 0m 39s | | trunk passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 0m 43s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 43s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 46s | | trunk passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 0m 36s | | trunk passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 1m 10s | | trunk passed |
| +1 :green_heart: | shadedclient | 20m 32s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 27s | | the patch passed |
| +1 :green_heart: | compile | 0m 29s | | the patch passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javac | 0m 29s | | the patch passed |
| +1 :green_heart: | compile | 0m 26s | | the patch passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | javac | 0m 26s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| -0 :warning: | checkstyle | 0m 19s |
[/results-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-router.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4892/2/artifact/out/results-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-router.txt)
|
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router:
The patch generated 4 new + 1 unchanged - 0 fixed = 5 total (was 1) |
| +1 :green_heart: | mvnsite | 0m 33s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 27s | | the patch passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 0m 27s | | the patch passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 0m 58s | | the patch passed |
| +1 :green_heart: | shadedclient | 20m 6s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| -1 :x: | unit | 4m 5s |
[/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-router.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4892/2/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-router.txt)
| hadoop-yarn-server-router in the patch passed. |
| +1 :green_heart: | asflicense | 0m 48s | | The patch does not
generate ASF License warnings. |
| | | 96m 1s | | |
| Reason | Tests |
|---:|:--|
| Failed junit tests |
hadoop.yarn.server.router.webapp.TestFederationInterceptorREST |
| | hadoop.yarn.server.router.webapp.TestRouterWebServicesREST |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4892/2/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/4892 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux 3a0286ed2b1b 4.15.0-191-generic #202-Ubuntu SMP Thu Aug 4
01:49:29 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk /
[jira] [Commented] (YARN-11305) Fix TestLogAggregationService#testLocalFileDeletionAfterUpload Failed After YARN-11241(#4703)
[
https://issues.apache.org/jira/browse/YARN-11305?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17605420#comment-17605420
]
ASF GitHub Bot commented on YARN-11305:
---
hadoop-yetus commented on PR #4893:
URL: https://github.com/apache/hadoop/pull/4893#issuecomment-1248273239
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 38s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 1 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 40m 27s | | trunk passed |
| +1 :green_heart: | compile | 1m 47s | | trunk passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | compile | 1m 42s | | trunk passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 0m 52s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 5s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 58s | | trunk passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 0m 45s | | trunk passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 1m 45s | | trunk passed |
| +1 :green_heart: | shadedclient | 21m 2s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 40s | | the patch passed |
| +1 :green_heart: | compile | 1m 27s | | the patch passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javac | 1m 27s | | the patch passed |
| +1 :green_heart: | compile | 1m 25s | | the patch passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | javac | 1m 25s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 0m 34s | | the patch passed |
| +1 :green_heart: | mvnsite | 0m 46s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 37s | | the patch passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 0m 37s | | the patch passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 1m 35s | | the patch passed |
| +1 :green_heart: | shadedclient | 20m 39s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 24m 7s | | hadoop-yarn-server-nodemanager
in the patch passed. |
| +1 :green_heart: | asflicense | 0m 44s | | The patch does not
generate ASF License warnings. |
| | | 124m 33s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4893/2/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/4893 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux 53f8eeb216ee 4.15.0-191-generic #202-Ubuntu SMP Thu Aug 4
01:49:29 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 1c89c3cbc5900bd46fad129129debb8e2c0f43d9 |
| Default Java | Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4893/2/testReport/ |
| Max. process+thread count | 633 (vs. ulimit of 5500) |
| modules | C:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager
U:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager
|
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4893/2/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0
[jira] [Commented] (YARN-11255) Support loading alternative docker client config from system environment
[
https://issues.apache.org/jira/browse/YARN-11255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17605405#comment-17605405
]
ASF GitHub Bot commented on YARN-11255:
---
hadoop-yetus commented on PR #4884:
URL: https://github.com/apache/hadoop/pull/4884#issuecomment-1248242670
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 38s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +0 :ok: | markdownlint | 0m 0s | | markdownlint was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 1 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 15m 30s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 25m 39s | | trunk passed |
| +1 :green_heart: | compile | 10m 0s | | trunk passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | compile | 8m 51s | | trunk passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 2m 8s | | trunk passed |
| +1 :green_heart: | mvnsite | 2m 9s | | trunk passed |
| +1 :green_heart: | javadoc | 2m 17s | | trunk passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 1m 55s | | trunk passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +0 :ok: | spotbugs | 0m 55s | |
branch/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site no spotbugs output file
(spotbugsXml.xml) |
| +1 :green_heart: | shadedclient | 21m 1s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 25s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 1m 3s | | the patch passed |
| +1 :green_heart: | compile | 9m 11s | | the patch passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javac | 9m 11s | | the patch passed |
| +1 :green_heart: | compile | 8m 49s | | the patch passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | javac | 8m 49s | | the patch passed |
| -1 :x: | blanks | 0m 0s |
[/blanks-eol.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4884/3/artifact/out/blanks-eol.txt)
| The patch has 1 line(s) that end in blanks. Use git apply --whitespace=fix
<>. Refer https://git-scm.com/docs/git-apply |
| +1 :green_heart: | checkstyle | 2m 0s | | the patch passed |
| +1 :green_heart: | mvnsite | 2m 4s | | the patch passed |
| +1 :green_heart: | javadoc | 1m 56s | | the patch passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 1m 44s | | the patch passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +0 :ok: | spotbugs | 0m 43s | |
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site has no data from spotbugs |
| +1 :green_heart: | shadedclient | 23m 29s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 24m 31s | | hadoop-yarn-server-nodemanager
in the patch passed. |
| +1 :green_heart: | unit | 0m 51s | | hadoop-yarn-site in the patch
passed. |
| +1 :green_heart: | asflicense | 1m 9s | | The patch does not
generate ASF License warnings. |
| | | 177m 18s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4884/3/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/4884 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets markdownlint
|
| uname | Linux 74a4f5d1d577 4.15.0-191-generic #202-Ubuntu SMP Thu Aug 4
01:49:29 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 21aa9ab689a84f28a27313495baeec4492650770 |
| Default Java | Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| Multi-JDK versions |
[jira] [Commented] (YARN-11305) Fix TestLogAggregationService#testLocalFileDeletionAfterUpload Failed After YARN-11241(#4703)
[
https://issues.apache.org/jira/browse/YARN-11305?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17605362#comment-17605362
]
ASF GitHub Bot commented on YARN-11305:
---
hadoop-yetus commented on PR #4893:
URL: https://github.com/apache/hadoop/pull/4893#issuecomment-1248140980
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 53s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 1s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 1 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 38m 49s | | trunk passed |
| +1 :green_heart: | compile | 1m 38s | | trunk passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | compile | 1m 41s | | trunk passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 0m 49s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 56s | | trunk passed |
| +1 :green_heart: | javadoc | 1m 5s | | trunk passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 0m 44s | | trunk passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 1m 47s | | trunk passed |
| +1 :green_heart: | shadedclient | 21m 23s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 41s | | the patch passed |
| +1 :green_heart: | compile | 1m 28s | | the patch passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javac | 1m 28s | | the patch passed |
| +1 :green_heart: | compile | 1m 22s | | the patch passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | javac | 1m 22s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| -0 :warning: | checkstyle | 0m 31s |
[/results-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4893/1/artifact/out/results-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager.txt)
|
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager:
The patch generated 1 new + 112 unchanged - 0 fixed = 113 total (was 112) |
| +1 :green_heart: | mvnsite | 0m 46s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 39s | | the patch passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 0m 34s | | the patch passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 1m 34s | | the patch passed |
| +1 :green_heart: | shadedclient | 20m 59s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 24m 25s | | hadoop-yarn-server-nodemanager
in the patch passed. |
| +1 :green_heart: | asflicense | 0m 49s | | The patch does not
generate ASF License warnings. |
| | | 124m 12s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4893/1/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/4893 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux db6c6bfcbefa 4.15.0-191-generic #202-Ubuntu SMP Thu Aug 4
01:49:29 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 748fa05119f192c6a92794f4b4aeea51aae62520 |
| Default Java | Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4893/1/testReport/ |
| Max. process+thread
[jira] [Commented] (YARN-11226) [Federation] Add createNewReservation, submitReservation, updateReservation, deleteReservation REST APIs for Router
[
https://issues.apache.org/jira/browse/YARN-11226?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17605316#comment-17605316
]
ASF GitHub Bot commented on YARN-11226:
---
hadoop-yetus commented on PR #4892:
URL: https://github.com/apache/hadoop/pull/4892#issuecomment-1248049937
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 37s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 2 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 38m 20s | | trunk passed |
| +1 :green_heart: | compile | 0m 45s | | trunk passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | compile | 0m 44s | | trunk passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 0m 39s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 51s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 46s | | trunk passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 0m 36s | | trunk passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 1m 11s | | trunk passed |
| +1 :green_heart: | shadedclient | 20m 32s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 27s | | the patch passed |
| +1 :green_heart: | compile | 0m 29s | | the patch passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javac | 0m 29s | | the patch passed |
| +1 :green_heart: | compile | 0m 26s | | the patch passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | javac | 0m 26s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| -0 :warning: | checkstyle | 0m 23s |
[/results-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-router.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4892/1/artifact/out/results-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-router.txt)
|
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router:
The patch generated 4 new + 1 unchanged - 0 fixed = 5 total (was 1) |
| +1 :green_heart: | mvnsite | 0m 29s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 25s | | the patch passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 0m 23s | | the patch passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 0m 56s | | the patch passed |
| +1 :green_heart: | shadedclient | 19m 59s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| -1 :x: | unit | 4m 0s |
[/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-router.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4892/1/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-router.txt)
| hadoop-yarn-server-router in the patch passed. |
| +1 :green_heart: | asflicense | 0m 45s | | The patch does not
generate ASF License warnings. |
| | | 95m 33s | | |
| Reason | Tests |
|---:|:--|
| Failed junit tests |
hadoop.yarn.server.router.webapp.TestFederationInterceptorREST |
| | hadoop.yarn.server.router.webapp.TestRouterWebServicesREST |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4892/1/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/4892 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux 184c7aa487d0 4.15.0-191-generic #202-Ubuntu SMP Thu Aug 4
01:49:29 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk /
[jira] [Commented] (YARN-11207) Add configuration to disable queue's user limit
[
https://issues.apache.org/jira/browse/YARN-11207?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17605312#comment-17605312
]
ASF GitHub Bot commented on YARN-11207:
---
hadoop-yetus commented on PR #4891:
URL: https://github.com/apache/hadoop/pull/4891#issuecomment-1248042547
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 40s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 1 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 39m 7s | | trunk passed |
| +1 :green_heart: | compile | 1m 20s | | trunk passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | compile | 1m 12s | | trunk passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 1m 6s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 14s | | trunk passed |
| +1 :green_heart: | javadoc | 1m 14s | | trunk passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 0m 54s | | trunk passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 2m 22s | | trunk passed |
| +1 :green_heart: | shadedclient | 22m 43s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 54s | | the patch passed |
| +1 :green_heart: | compile | 1m 5s | | the patch passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javac | 1m 5s | | the patch passed |
| +1 :green_heart: | compile | 0m 57s | | the patch passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | javac | 0m 57s | | the patch passed |
| -1 :x: | blanks | 0m 0s |
[/blanks-eol.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4891/1/artifact/out/blanks-eol.txt)
| The patch has 1 line(s) that end in blanks. Use git apply --whitespace=fix
<>. Refer https://git-scm.com/docs/git-apply |
| -0 :warning: | checkstyle | 0m 49s |
[/results-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4891/1/artifact/out/results-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt)
|
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager:
The patch generated 7 new + 374 unchanged - 0 fixed = 381 total (was 374) |
| +1 :green_heart: | mvnsite | 0m 58s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 44s | | the patch passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 0m 41s | | the patch passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 1m 58s | | the patch passed |
| +1 :green_heart: | shadedclient | 21m 11s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 99m 2s | |
hadoop-yarn-server-resourcemanager in the patch passed. |
| +1 :green_heart: | asflicense | 0m 44s | | The patch does not
generate ASF License warnings. |
| | | 200m 42s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4891/1/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/4891 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux 7f2072a86c06 4.15.0-191-generic #202-Ubuntu SMP Thu Aug 4
01:49:29 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 5f9471c14ed277f036c064c5629134eed5a0ea02 |
| Default Java | Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04
[jira] [Commented] (YARN-11305) Fix TestLogAggregationService#testLocalFileDeletionAfterUpload Failed After YARN-11241(#4703)
[
https://issues.apache.org/jira/browse/YARN-11305?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17605304#comment-17605304
]
ASF GitHub Bot commented on YARN-11305:
---
ashutoshcipher commented on PR #4893:
URL: https://github.com/apache/hadoop/pull/4893#issuecomment-1248031693
FYI - This failure is observed with parallel mode enabled, hence was
uncaught in
[YARN-11241](https://issues.apache.org/jira/browse/YARN-11241)(https://github.com/apache/hadoop/pull/4703)
> Fix TestLogAggregationService#testLocalFileDeletionAfterUpload Failed After
> YARN-11241(#4703)
> -
>
> Key: YARN-11305
> URL: https://issues.apache.org/jira/browse/YARN-11305
> Project: Hadoop YARN
> Issue Type: Bug
> Components: log-aggregation
>Affects Versions: 3.4.0
>Reporter: fanshilun
>Assignee: fanshilun
>Priority: Major
> Labels: pull-request-available
>
> Yarn's unit tests, with parallel mode enabled
> {code:java}
> /usr/bin/mvn --batch-mode
> -Dmaven.repo.local=/home/jenkins/jenkins-home/workspace/hadoop-multibranch_PR-4846/yetus-m2/hadoop-trunk-patch-0
> -Ptest-patch -Dsurefire.rerunFailingTestsCount=2 -Pparallel-tests
> -P!shelltest -Pnative -Drequire.fuse -Drequire.openssl -Drequire.snappy
> -Drequire.valgrind -Drequire.zstd -Drequire.test.libhadoop -Pyarn-ui clean
> test -fae{code}
> Because the automatically generated application_id is the same,
> *testLocalFileDeletionAfterUpload* is affected by
> *testLocalFileRemainsAfterUploadOnCleanupDisable*
> testLocalFileDeletionAfterUpload needs to check that the log of an
> application is deleted, and testLocalFileRemainsAfterUploadOnCleanupDisable
> regenerates the log of the same application.
> Error Message
> {code:java}
> Directory
> [/home/jenkins/jenkins-home/workspace/hadoop-multibranch_PR-4846/ubuntu-focal/src/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/target/TestLogAggregationService-localLogDir/application_1234_0001]
> was not deleted {code}
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-11255) Support loading alternative docker client config from system environment
[
https://issues.apache.org/jira/browse/YARN-11255?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ashutosh Gupta updated YARN-11255:
--
Description:
When using YARN docker support, although the hadoop shell supported
{code:java}
-docker_client_config{code}
to pass the client config file that contains security token to generate the
docker config for each job as a temporary file.
For other applications that submit jobs to YARN, e.g. Spark, which loads the
docker setting via system environment e.g.
{code:java}
spark.executorEnv.* {code}
will not be able to add those authorization token because this system
environment isn't considered in YARN.
Add genetic solution to handle these kind of cases without making changes in
spark code or others
Eg
When using remote container registry, the
{{YARN_CONTAINER_RUNTIME_DOCKER_CLIENT_CONFIG}} must reference the config.json
file containing the credentials used to authenticate.
{code:java}
DOCKER_IMAGE_NAME=hadoop-docker
DOCKER_CLIENT_CONFIG=hdfs:///user/hadoop/config.json
spark-submit --master yarn \
--deploy-mode cluster \
--conf spark.executorEnv.YARN_CONTAINER_RUNTIME_TYPE=docker \
--conf spark.executorEnv.YARN_CONTAINER_RUNTIME_DOCKER_IMAGE=$DOCKER_IMAGE_NAME
\
--conf
spark.executorEnv.YARN_CONTAINER_RUNTIME_DOCKER_CLIENT_CONFIG=$DOCKER_CLIENT_CONFIG
\
--conf spark.yarn.appMasterEnv.YARN_CONTAINER_RUNTIME_TYPE=docker \
--conf
spark.yarn.appMasterEnv.YARN_CONTAINER_RUNTIME_DOCKER_IMAGE=$DOCKER_IMAGE_NAME \
--conf
spark.yarn.appMasterEnv.YARN_CONTAINER_RUNTIME_DOCKER_CLIENT_CONFIG=$DOCKER_CLIENT_CONFIG
\
sparkR.R{code}
was:
When using YARN docker support, although the hadoop shell supported
{code:java}
-docker_client_config{code}
to pass the client config file that contains security token to generate the
docker config for each job as a temporary file.
For other applications that submit jobs to YARN, e.g. Spark, which loads the
docker setting via system environment e.g.
{code:java}
spark.executorEnv.* {code}
will not be able to add those authorization token because this system
environment isn't considered in YARN.
Add genetic solution to handle these kind of cases without making changes in
spark code or others
Eg
When using remote container registry, the
{{YARN_CONTAINER_RUNTIME_DOCKER_CLIENT_CONFIG}} must reference the config.json
file containing the credentials used to authenticate.
DOCKER_IMAGE_NAME=hadoop-docker
DOCKER_CLIENT_CONFIG=hdfs:///user/hadoop/config.json
spark-submit --master yarn \
--deploy-mode cluster \
--conf spark.executorEnv.YARN_CONTAINER_RUNTIME_TYPE=docker \
--conf spark.executorEnv.YARN_CONTAINER_RUNTIME_DOCKER_IMAGE=$DOCKER_IMAGE_NAME
\
--conf
spark.executorEnv.YARN_CONTAINER_RUNTIME_DOCKER_CLIENT_CONFIG=$DOCKER_CLIENT_CONFIG
\
--conf spark.yarn.appMasterEnv.YARN_CONTAINER_RUNTIME_TYPE=docker \
--conf
spark.yarn.appMasterEnv.YARN_CONTAINER_RUNTIME_DOCKER_IMAGE=$DOCKER_IMAGE_NAME \
--conf
spark.yarn.appMasterEnv.YARN_CONTAINER_RUNTIME_DOCKER_CLIENT_CONFIG=$DOCKER_CLIENT_CONFIG
\
sparkR.R
> Support loading alternative docker client config from system environment
>
>
> Key: YARN-11255
> URL: https://issues.apache.org/jira/browse/YARN-11255
> Project: Hadoop YARN
> Issue Type: New Feature
>Reporter: Ashutosh Gupta
>Assignee: Ashutosh Gupta
>Priority: Major
> Labels: pull-request-available
>
> When using YARN docker support, although the hadoop shell supported
> {code:java}
> -docker_client_config{code}
> to pass the client config file that contains security token to generate the
> docker config for each job as a temporary file.
> For other applications that submit jobs to YARN, e.g. Spark, which loads the
> docker setting via system environment e.g.
> {code:java}
> spark.executorEnv.* {code}
> will not be able to add those authorization token because this system
> environment isn't considered in YARN.
> Add genetic solution to handle these kind of cases without making changes in
> spark code or others
> Eg
> When using remote container registry, the
> {{YARN_CONTAINER_RUNTIME_DOCKER_CLIENT_CONFIG}} must reference the config.json
> file containing the credentials used to authenticate.
> {code:java}
> DOCKER_IMAGE_NAME=hadoop-docker
> DOCKER_CLIENT_CONFIG=hdfs:///user/hadoop/config.json
> spark-submit --master yarn \
> --deploy-mode cluster \
> --conf spark.executorEnv.YARN_CONTAINER_RUNTIME_TYPE=docker \
> --conf
> spark.executorEnv.YARN_CONTAINER_RUNTIME_DOCKER_IMAGE=$DOCKER_IMAGE_NAME \
> --conf
> spark.executorEnv.YARN_CONTAINER_RUNTIME_DOCKER_CLIENT_CONFIG=$DOCKER_CLIENT_CONFIG
> \
> --conf spark.yarn.appMasterEnv.YARN_CONTAINER_RUNTIME_TYPE=docker \
> --conf
> spark.yarn.appMasterEnv.YARN_CONTAINER_RUNTIME_DOCKER_IMAGE=$DOCKER_IMAGE_NAME
> \
> --conf
[jira] [Updated] (YARN-11255) Support loading alternative docker client config from system environment
[
https://issues.apache.org/jira/browse/YARN-11255?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ashutosh Gupta updated YARN-11255:
--
Description:
When using YARN docker support, although the hadoop shell supported
{code:java}
-docker_client_config{code}
to pass the client config file that contains security token to generate the
docker config for each job as a temporary file.
For other applications that submit jobs to YARN, e.g. Spark, which loads the
docker setting via system environment e.g.
{code:java}
spark.executorEnv.* {code}
will not be able to add those authorization token because this system
environment isn't considered in YARN.
Add genetic solution to handle these kind of cases without making changes in
spark code or others
Eg
When using remote container registry, the
{{YARN_CONTAINER_RUNTIME_DOCKER_CLIENT_CONFIG}} must reference the config.json
file containing the credentials used to authenticate.
DOCKER_IMAGE_NAME=hadoop-docker
DOCKER_CLIENT_CONFIG=hdfs:///user/hadoop/config.json
spark-submit --master yarn \
--deploy-mode cluster \
--conf spark.executorEnv.YARN_CONTAINER_RUNTIME_TYPE=docker \
--conf spark.executorEnv.YARN_CONTAINER_RUNTIME_DOCKER_IMAGE=$DOCKER_IMAGE_NAME
\
--conf
spark.executorEnv.YARN_CONTAINER_RUNTIME_DOCKER_CLIENT_CONFIG=$DOCKER_CLIENT_CONFIG
\
--conf spark.yarn.appMasterEnv.YARN_CONTAINER_RUNTIME_TYPE=docker \
--conf
spark.yarn.appMasterEnv.YARN_CONTAINER_RUNTIME_DOCKER_IMAGE=$DOCKER_IMAGE_NAME \
--conf
spark.yarn.appMasterEnv.YARN_CONTAINER_RUNTIME_DOCKER_CLIENT_CONFIG=$DOCKER_CLIENT_CONFIG
\
sparkR.R{{}}
was:
When using YARN docker support, although the hadoop shell supported
{code:java}
-docker_client_config{code}
to pass the client config file that contains security token to generate the
docker config for each job as a temporary file.
For other applications that submit jobs to YARN, e.g. Spark, which loads the
docker setting via system environment e.g.
{code:java}
spark.executorEnv.* {code}
will not be able to add those authorization token because this system
environment isn't considered in YARN.
Add genetic solution to handle these kind of cases without making changes in
spark code or others
> Support loading alternative docker client config from system environment
>
>
> Key: YARN-11255
> URL: https://issues.apache.org/jira/browse/YARN-11255
> Project: Hadoop YARN
> Issue Type: New Feature
>Reporter: Ashutosh Gupta
>Assignee: Ashutosh Gupta
>Priority: Major
> Labels: pull-request-available
>
> When using YARN docker support, although the hadoop shell supported
> {code:java}
> -docker_client_config{code}
> to pass the client config file that contains security token to generate the
> docker config for each job as a temporary file.
> For other applications that submit jobs to YARN, e.g. Spark, which loads the
> docker setting via system environment e.g.
> {code:java}
> spark.executorEnv.* {code}
> will not be able to add those authorization token because this system
> environment isn't considered in YARN.
> Add genetic solution to handle these kind of cases without making changes in
> spark code or others
> Eg
> When using remote container registry, the
> {{YARN_CONTAINER_RUNTIME_DOCKER_CLIENT_CONFIG}} must reference the config.json
> file containing the credentials used to authenticate.
> DOCKER_IMAGE_NAME=hadoop-docker
> DOCKER_CLIENT_CONFIG=hdfs:///user/hadoop/config.json
> spark-submit --master yarn \
> --deploy-mode cluster \
> --conf spark.executorEnv.YARN_CONTAINER_RUNTIME_TYPE=docker \
> --conf
> spark.executorEnv.YARN_CONTAINER_RUNTIME_DOCKER_IMAGE=$DOCKER_IMAGE_NAME \
> --conf
> spark.executorEnv.YARN_CONTAINER_RUNTIME_DOCKER_CLIENT_CONFIG=$DOCKER_CLIENT_CONFIG
> \
> --conf spark.yarn.appMasterEnv.YARN_CONTAINER_RUNTIME_TYPE=docker \
> --conf
> spark.yarn.appMasterEnv.YARN_CONTAINER_RUNTIME_DOCKER_IMAGE=$DOCKER_IMAGE_NAME
> \
> --conf
> spark.yarn.appMasterEnv.YARN_CONTAINER_RUNTIME_DOCKER_CLIENT_CONFIG=$DOCKER_CLIENT_CONFIG
> \
> sparkR.R{{}}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-11255) Support loading alternative docker client config from system environment
[
https://issues.apache.org/jira/browse/YARN-11255?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ashutosh Gupta updated YARN-11255:
--
Description:
When using YARN docker support, although the hadoop shell supported
{code:java}
-docker_client_config{code}
to pass the client config file that contains security token to generate the
docker config for each job as a temporary file.
For other applications that submit jobs to YARN, e.g. Spark, which loads the
docker setting via system environment e.g.
{code:java}
spark.executorEnv.* {code}
will not be able to add those authorization token because this system
environment isn't considered in YARN.
Add genetic solution to handle these kind of cases without making changes in
spark code or others
Eg
When using remote container registry, the
{{YARN_CONTAINER_RUNTIME_DOCKER_CLIENT_CONFIG}} must reference the config.json
file containing the credentials used to authenticate.
DOCKER_IMAGE_NAME=hadoop-docker
DOCKER_CLIENT_CONFIG=hdfs:///user/hadoop/config.json
spark-submit --master yarn \
--deploy-mode cluster \
--conf spark.executorEnv.YARN_CONTAINER_RUNTIME_TYPE=docker \
--conf spark.executorEnv.YARN_CONTAINER_RUNTIME_DOCKER_IMAGE=$DOCKER_IMAGE_NAME
\
--conf
spark.executorEnv.YARN_CONTAINER_RUNTIME_DOCKER_CLIENT_CONFIG=$DOCKER_CLIENT_CONFIG
\
--conf spark.yarn.appMasterEnv.YARN_CONTAINER_RUNTIME_TYPE=docker \
--conf
spark.yarn.appMasterEnv.YARN_CONTAINER_RUNTIME_DOCKER_IMAGE=$DOCKER_IMAGE_NAME \
--conf
spark.yarn.appMasterEnv.YARN_CONTAINER_RUNTIME_DOCKER_CLIENT_CONFIG=$DOCKER_CLIENT_CONFIG
\
sparkR.R
was:
When using YARN docker support, although the hadoop shell supported
{code:java}
-docker_client_config{code}
to pass the client config file that contains security token to generate the
docker config for each job as a temporary file.
For other applications that submit jobs to YARN, e.g. Spark, which loads the
docker setting via system environment e.g.
{code:java}
spark.executorEnv.* {code}
will not be able to add those authorization token because this system
environment isn't considered in YARN.
Add genetic solution to handle these kind of cases without making changes in
spark code or others
Eg
When using remote container registry, the
{{YARN_CONTAINER_RUNTIME_DOCKER_CLIENT_CONFIG}} must reference the config.json
file containing the credentials used to authenticate.
DOCKER_IMAGE_NAME=hadoop-docker
DOCKER_CLIENT_CONFIG=hdfs:///user/hadoop/config.json
spark-submit --master yarn \
--deploy-mode cluster \
--conf spark.executorEnv.YARN_CONTAINER_RUNTIME_TYPE=docker \
--conf spark.executorEnv.YARN_CONTAINER_RUNTIME_DOCKER_IMAGE=$DOCKER_IMAGE_NAME
\
--conf
spark.executorEnv.YARN_CONTAINER_RUNTIME_DOCKER_CLIENT_CONFIG=$DOCKER_CLIENT_CONFIG
\
--conf spark.yarn.appMasterEnv.YARN_CONTAINER_RUNTIME_TYPE=docker \
--conf
spark.yarn.appMasterEnv.YARN_CONTAINER_RUNTIME_DOCKER_IMAGE=$DOCKER_IMAGE_NAME \
--conf
spark.yarn.appMasterEnv.YARN_CONTAINER_RUNTIME_DOCKER_CLIENT_CONFIG=$DOCKER_CLIENT_CONFIG
\
sparkR.R{{}}
> Support loading alternative docker client config from system environment
>
>
> Key: YARN-11255
> URL: https://issues.apache.org/jira/browse/YARN-11255
> Project: Hadoop YARN
> Issue Type: New Feature
>Reporter: Ashutosh Gupta
>Assignee: Ashutosh Gupta
>Priority: Major
> Labels: pull-request-available
>
> When using YARN docker support, although the hadoop shell supported
> {code:java}
> -docker_client_config{code}
> to pass the client config file that contains security token to generate the
> docker config for each job as a temporary file.
> For other applications that submit jobs to YARN, e.g. Spark, which loads the
> docker setting via system environment e.g.
> {code:java}
> spark.executorEnv.* {code}
> will not be able to add those authorization token because this system
> environment isn't considered in YARN.
> Add genetic solution to handle these kind of cases without making changes in
> spark code or others
> Eg
> When using remote container registry, the
> {{YARN_CONTAINER_RUNTIME_DOCKER_CLIENT_CONFIG}} must reference the config.json
> file containing the credentials used to authenticate.
> DOCKER_IMAGE_NAME=hadoop-docker
> DOCKER_CLIENT_CONFIG=hdfs:///user/hadoop/config.json
> spark-submit --master yarn \
> --deploy-mode cluster \
> --conf spark.executorEnv.YARN_CONTAINER_RUNTIME_TYPE=docker \
> --conf
> spark.executorEnv.YARN_CONTAINER_RUNTIME_DOCKER_IMAGE=$DOCKER_IMAGE_NAME \
> --conf
> spark.executorEnv.YARN_CONTAINER_RUNTIME_DOCKER_CLIENT_CONFIG=$DOCKER_CLIENT_CONFIG
> \
> --conf spark.yarn.appMasterEnv.YARN_CONTAINER_RUNTIME_TYPE=docker \
> --conf
> spark.yarn.appMasterEnv.YARN_CONTAINER_RUNTIME_DOCKER_IMAGE=$DOCKER_IMAGE_NAME
> \
> --conf
>
[jira] [Commented] (YARN-11255) Support loading alternative docker client config from system environment
[
https://issues.apache.org/jira/browse/YARN-11255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17605301#comment-17605301
]
ASF GitHub Bot commented on YARN-11255:
---
ashutoshcipher commented on PR #4884:
URL: https://github.com/apache/hadoop/pull/4884#issuecomment-1248022535
> Thanks @ashutoshcipher for the patch. Overall looks good to me except a
minor one.
>
> Could you also document the steps to run an application like Spark using
this new environment variable YARN_CONTAINER_RUNTIME_DOCKER_CLIENT_CONFIG in
https://hadoop.apache.org/docs/r3.0.2/hadoop-yarn/hadoop-yarn-site/DockerContainers.html
>
> For Example:
>
> ```
> DOCKER_IMAGE_NAME=hadoop-docker
> DOCKER_CLIENT_CONFIG=hdfs:///user/hadoop/config.json
> spark-submit --master yarn \
> --deploy-mode cluster \
> --conf spark.executorEnv.YARN_CONTAINER_RUNTIME_TYPE=docker \
> --conf
spark.executorEnv.YARN_CONTAINER_RUNTIME_DOCKER_IMAGE=$DOCKER_IMAGE_NAME \
> --conf
spark.executorEnv.YARN_CONTAINER_RUNTIME_DOCKER_CLIENT_CONFIG=$DOCKER_CLIENT_CONFIG
\
> --conf spark.yarn.appMasterEnv.YARN_CONTAINER_RUNTIME_TYPE=docker \
> --conf
spark.yarn.appMasterEnv.YARN_CONTAINER_RUNTIME_DOCKER_IMAGE=$DOCKER_IMAGE_NAME \
> --conf
spark.yarn.appMasterEnv.YARN_CONTAINER_RUNTIME_DOCKER_CLIENT_CONFIG=$DOCKER_CLIENT_CONFIG
\
> sparkR.R
> ```
Thanks @PrabhuJoseph . Added the documentation
> Support loading alternative docker client config from system environment
>
>
> Key: YARN-11255
> URL: https://issues.apache.org/jira/browse/YARN-11255
> Project: Hadoop YARN
> Issue Type: New Feature
>Reporter: Ashutosh Gupta
>Assignee: Ashutosh Gupta
>Priority: Major
> Labels: pull-request-available
>
> When using YARN docker support, although the hadoop shell supported
> {code:java}
> -docker_client_config{code}
> to pass the client config file that contains security token to generate the
> docker config for each job as a temporary file.
> For other applications that submit jobs to YARN, e.g. Spark, which loads the
> docker setting via system environment e.g.
> {code:java}
> spark.executorEnv.* {code}
> will not be able to add those authorization token because this system
> environment isn't considered in YARN.
> Add genetic solution to handle these kind of cases without making changes in
> spark code or others
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11255) Support loading alternative docker client config from system environment
[
https://issues.apache.org/jira/browse/YARN-11255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17605300#comment-17605300
]
ASF GitHub Bot commented on YARN-11255:
---
ashutoshcipher commented on code in PR #4884:
URL: https://github.com/apache/hadoop/pull/4884#discussion_r971910648
##
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/runtime/DockerLinuxContainerRuntime.java:
##
@@ -215,6 +217,12 @@ public class DockerLinuxContainerRuntime extends
OCIContainerRuntime {
public static final String ENV_DOCKER_CONTAINER_IMAGE =
"YARN_CONTAINER_RUNTIME_DOCKER_IMAGE";
@InterfaceAudience.Private
+ public static final String ENV_DOCKER_CONTAINER_IMAGE_FILE =
Review Comment:
Thanks @PrabhuJoseph. Addressed in my latest commit.
> Support loading alternative docker client config from system environment
>
>
> Key: YARN-11255
> URL: https://issues.apache.org/jira/browse/YARN-11255
> Project: Hadoop YARN
> Issue Type: New Feature
>Reporter: Ashutosh Gupta
>Assignee: Ashutosh Gupta
>Priority: Major
> Labels: pull-request-available
>
> When using YARN docker support, although the hadoop shell supported
> {code:java}
> -docker_client_config{code}
> to pass the client config file that contains security token to generate the
> docker config for each job as a temporary file.
> For other applications that submit jobs to YARN, e.g. Spark, which loads the
> docker setting via system environment e.g.
> {code:java}
> spark.executorEnv.* {code}
> will not be able to add those authorization token because this system
> environment isn't considered in YARN.
> Add genetic solution to handle these kind of cases without making changes in
> spark code or others
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11305) Fix TestLogAggregationService#testLocalFileDeletionAfterUpload Failed After YARN-11241(#4703)
[
https://issues.apache.org/jira/browse/YARN-11305?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17605292#comment-17605292
]
ASF GitHub Bot commented on YARN-11305:
---
slfan1989 commented on PR #4893:
URL: https://github.com/apache/hadoop/pull/4893#issuecomment-1248002545
@ashutoshcipher @aajisaka Please help to review the code, I found that in
parallel test mode, YARN-11241 will cause junit test to
fail.
I let different junit tests use different application_id to avoid mutual
influence.
> Fix TestLogAggregationService#testLocalFileDeletionAfterUpload Failed After
> YARN-11241(#4703)
> -
>
> Key: YARN-11305
> URL: https://issues.apache.org/jira/browse/YARN-11305
> Project: Hadoop YARN
> Issue Type: Bug
> Components: log-aggregation
>Affects Versions: 3.4.0
>Reporter: fanshilun
>Assignee: fanshilun
>Priority: Major
> Labels: pull-request-available
>
> Yarn's unit tests, with parallel mode enabled
> {code:java}
> /usr/bin/mvn --batch-mode
> -Dmaven.repo.local=/home/jenkins/jenkins-home/workspace/hadoop-multibranch_PR-4846/yetus-m2/hadoop-trunk-patch-0
> -Ptest-patch -Dsurefire.rerunFailingTestsCount=2 -Pparallel-tests
> -P!shelltest -Pnative -Drequire.fuse -Drequire.openssl -Drequire.snappy
> -Drequire.valgrind -Drequire.zstd -Drequire.test.libhadoop -Pyarn-ui clean
> test -fae{code}
> Because the automatically generated application_id is the same,
> *testLocalFileDeletionAfterUpload* is affected by
> *testLocalFileRemainsAfterUploadOnCleanupDisable*
> testLocalFileDeletionAfterUpload needs to check that the log of an
> application is deleted, and testLocalFileRemainsAfterUploadOnCleanupDisable
> regenerates the log of the same application.
> Error Message
> {code:java}
> Directory
> [/home/jenkins/jenkins-home/workspace/hadoop-multibranch_PR-4846/ubuntu-focal/src/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/target/TestLogAggregationService-localLogDir/application_1234_0001]
> was not deleted {code}
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11305) Fix TestLogAggregationService#testLocalFileDeletionAfterUpload Failed After YARN-11241(#4703)
[
https://issues.apache.org/jira/browse/YARN-11305?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17605289#comment-17605289
]
ASF GitHub Bot commented on YARN-11305:
---
slfan1989 opened a new pull request, #4893:
URL: https://github.com/apache/hadoop/pull/4893
YARN-11305. Fix TestLogAggregationService#testLocalFileDeletionAfterUpload
Failed After YARN-11241(#4703).
> Fix TestLogAggregationService#testLocalFileDeletionAfterUpload Failed After
> YARN-11241(#4703)
> -
>
> Key: YARN-11305
> URL: https://issues.apache.org/jira/browse/YARN-11305
> Project: Hadoop YARN
> Issue Type: Bug
> Components: log-aggregation
>Affects Versions: 3.4.0
>Reporter: fanshilun
>Assignee: fanshilun
>Priority: Major
>
> Yarn's unit tests, with parallel mode enabled
> {code:java}
> /usr/bin/mvn --batch-mode
> -Dmaven.repo.local=/home/jenkins/jenkins-home/workspace/hadoop-multibranch_PR-4846/yetus-m2/hadoop-trunk-patch-0
> -Ptest-patch -Dsurefire.rerunFailingTestsCount=2 -Pparallel-tests
> -P!shelltest -Pnative -Drequire.fuse -Drequire.openssl -Drequire.snappy
> -Drequire.valgrind -Drequire.zstd -Drequire.test.libhadoop -Pyarn-ui clean
> test -fae{code}
> Because the automatically generated application_id is the same,
> *testLocalFileDeletionAfterUpload* is affected by
> *testLocalFileRemainsAfterUploadOnCleanupDisable*
> testLocalFileDeletionAfterUpload needs to check that the log of an
> application is deleted, and testLocalFileRemainsAfterUploadOnCleanupDisable
> regenerates the log of the same application.
> Error Message
> {code:java}
> Directory
> [/home/jenkins/jenkins-home/workspace/hadoop-multibranch_PR-4846/ubuntu-focal/src/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/target/TestLogAggregationService-localLogDir/application_1234_0001]
> was not deleted {code}
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-11305) Fix TestLogAggregationService#testLocalFileDeletionAfterUpload Failed After YARN-11241(#4703)
[
https://issues.apache.org/jira/browse/YARN-11305?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated YARN-11305:
--
Labels: pull-request-available (was: )
> Fix TestLogAggregationService#testLocalFileDeletionAfterUpload Failed After
> YARN-11241(#4703)
> -
>
> Key: YARN-11305
> URL: https://issues.apache.org/jira/browse/YARN-11305
> Project: Hadoop YARN
> Issue Type: Bug
> Components: log-aggregation
>Affects Versions: 3.4.0
>Reporter: fanshilun
>Assignee: fanshilun
>Priority: Major
> Labels: pull-request-available
>
> Yarn's unit tests, with parallel mode enabled
> {code:java}
> /usr/bin/mvn --batch-mode
> -Dmaven.repo.local=/home/jenkins/jenkins-home/workspace/hadoop-multibranch_PR-4846/yetus-m2/hadoop-trunk-patch-0
> -Ptest-patch -Dsurefire.rerunFailingTestsCount=2 -Pparallel-tests
> -P!shelltest -Pnative -Drequire.fuse -Drequire.openssl -Drequire.snappy
> -Drequire.valgrind -Drequire.zstd -Drequire.test.libhadoop -Pyarn-ui clean
> test -fae{code}
> Because the automatically generated application_id is the same,
> *testLocalFileDeletionAfterUpload* is affected by
> *testLocalFileRemainsAfterUploadOnCleanupDisable*
> testLocalFileDeletionAfterUpload needs to check that the log of an
> application is deleted, and testLocalFileRemainsAfterUploadOnCleanupDisable
> regenerates the log of the same application.
> Error Message
> {code:java}
> Directory
> [/home/jenkins/jenkins-home/workspace/hadoop-multibranch_PR-4846/ubuntu-focal/src/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/target/TestLogAggregationService-localLogDir/application_1234_0001]
> was not deleted {code}
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-11305) Fix TestLogAggregationService#testLocalFileDeletionAfterUpload Failed After YARN-11241(#4703)
fanshilun created YARN-11305:
Summary: Fix
TestLogAggregationService#testLocalFileDeletionAfterUpload Failed After
YARN-11241(#4703)
Key: YARN-11305
URL: https://issues.apache.org/jira/browse/YARN-11305
Project: Hadoop YARN
Issue Type: Bug
Components: log-aggregation
Affects Versions: 3.4.0
Reporter: fanshilun
Assignee: fanshilun
Yarn's unit tests, with parallel mode enabled
{code:java}
/usr/bin/mvn --batch-mode
-Dmaven.repo.local=/home/jenkins/jenkins-home/workspace/hadoop-multibranch_PR-4846/yetus-m2/hadoop-trunk-patch-0
-Ptest-patch -Dsurefire.rerunFailingTestsCount=2 -Pparallel-tests -P!shelltest
-Pnative -Drequire.fuse -Drequire.openssl -Drequire.snappy -Drequire.valgrind
-Drequire.zstd -Drequire.test.libhadoop -Pyarn-ui clean test -fae{code}
Because the automatically generated application_id is the same,
*testLocalFileDeletionAfterUpload* is affected by
*testLocalFileRemainsAfterUploadOnCleanupDisable*
testLocalFileDeletionAfterUpload needs to check that the log of an application
is deleted, and testLocalFileRemainsAfterUploadOnCleanupDisable regenerates the
log of the same application.
Error Message
{code:java}
Directory
[/home/jenkins/jenkins-home/workspace/hadoop-multibranch_PR-4846/ubuntu-focal/src/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/target/TestLogAggregationService-localLogDir/application_1234_0001]
was not deleted {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10846) Add dispatcher metrics to NM
[
https://issues.apache.org/jira/browse/YARN-10846?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17605282#comment-17605282
]
ASF GitHub Bot commented on YARN-10846:
---
hadoop-yetus commented on PR #4687:
URL: https://github.com/apache/hadoop/pull/4687#issuecomment-1247977637
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 52s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 1 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 14m 56s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 25m 47s | | trunk passed |
| +1 :green_heart: | compile | 9m 45s | | trunk passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | compile | 8m 46s | | trunk passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 2m 14s | | trunk passed |
| +1 :green_heart: | mvnsite | 4m 19s | | trunk passed |
| +1 :green_heart: | javadoc | 4m 4s | | trunk passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 3m 46s | | trunk passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 6m 28s | | trunk passed |
| +1 :green_heart: | shadedclient | 22m 1s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 30s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 2m 22s | | the patch passed |
| +1 :green_heart: | compile | 9m 17s | | the patch passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javac | 9m 17s | | the patch passed |
| +1 :green_heart: | compile | 8m 38s | | the patch passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | javac | 8m 38s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| -0 :warning: | checkstyle | 1m 48s |
[/results-checkstyle-hadoop-yarn-project_hadoop-yarn.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4687/11/artifact/out/results-checkstyle-hadoop-yarn-project_hadoop-yarn.txt)
| hadoop-yarn-project/hadoop-yarn: The patch generated 1 new + 53 unchanged -
1 fixed = 54 total (was 54) |
| +1 :green_heart: | mvnsite | 3m 31s | | the patch passed |
| +1 :green_heart: | javadoc | 3m 19s | | the patch passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 3m 16s | | the patch passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 6m 47s | | the patch passed |
| +1 :green_heart: | shadedclient | 21m 57s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 5m 15s | | hadoop-yarn-common in the patch
passed. |
| +1 :green_heart: | unit | 117m 53s | |
hadoop-yarn-server-resourcemanager in the patch passed. |
| +1 :green_heart: | unit | 24m 55s | | hadoop-yarn-server-nodemanager
in the patch passed. |
| +1 :green_heart: | asflicense | 1m 12s | | The patch does not
generate ASF License warnings. |
| | | 316m 45s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4687/11/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/4687 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux 28f6082df0d9 4.15.0-191-generic #202-Ubuntu SMP Thu Aug 4
01:49:29 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 812174324bc0fbafe7503672a22fd905a458062d |
| Default Java | Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04
[jira] [Commented] (YARN-11226) [Federation] Add createNewReservation, submitReservation, updateReservation, deleteReservation REST APIs for Router
[ https://issues.apache.org/jira/browse/YARN-11226?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17605269#comment-17605269 ] ASF GitHub Bot commented on YARN-11226: --- slfan1989 closed pull request #4851: YARN-11226. [Federation] Add createNewReservation, submitReservation, updateReservation, deleteReservation REST APIs for Router URL: https://github.com/apache/hadoop/pull/4851 > [Federation] Add createNewReservation, submitReservation, updateReservation, > deleteReservation REST APIs for Router > --- > > Key: YARN-11226 > URL: https://issues.apache.org/jira/browse/YARN-11226 > Project: Hadoop YARN > Issue Type: Sub-task > Components: federation >Affects Versions: 3.4.0, 3.3.4 >Reporter: fanshilun >Assignee: fanshilun >Priority: Major > Labels: pull-request-available > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11226) [Federation] Add createNewReservation, submitReservation, updateReservation, deleteReservation REST APIs for Router
[ https://issues.apache.org/jira/browse/YARN-11226?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17605268#comment-17605268 ] ASF GitHub Bot commented on YARN-11226: --- slfan1989 opened a new pull request, #4892: URL: https://github.com/apache/hadoop/pull/4892 JIRA: YARN-11226. [Federation] Add createNewReservation, submitReservation, updateReservation, deleteReservation REST APIs for Router. > [Federation] Add createNewReservation, submitReservation, updateReservation, > deleteReservation REST APIs for Router > --- > > Key: YARN-11226 > URL: https://issues.apache.org/jira/browse/YARN-11226 > Project: Hadoop YARN > Issue Type: Sub-task > Components: federation >Affects Versions: 3.4.0, 3.3.4 >Reporter: fanshilun >Assignee: fanshilun >Priority: Major > Labels: pull-request-available > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11296) Fix SQLFederationStateStore#Sql script bug
[
https://issues.apache.org/jira/browse/YARN-11296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17605237#comment-17605237
]
ASF GitHub Bot commented on YARN-11296:
---
hadoop-yetus commented on PR #4858:
URL: https://github.com/apache/hadoop/pull/4858#issuecomment-1247862385
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 49s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +0 :ok: | xmllint | 0m 0s | | xmllint was not available. |
| +1 :green_heart: | @author | 0m 1s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 6 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 17m 1s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 28m 31s | | trunk passed |
| +1 :green_heart: | compile | 25m 36s | | trunk passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | compile | 22m 7s | | trunk passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 4m 32s | | trunk passed |
| +1 :green_heart: | mvnsite | 6m 59s | | trunk passed |
| +1 :green_heart: | javadoc | 4m 45s | | trunk passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 3m 54s | | trunk passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +0 :ok: | spotbugs | 0m 43s | | branch/hadoop-project no spotbugs
output file (spotbugsXml.xml) |
| +1 :green_heart: | shadedclient | 24m 14s | | branch has no errors
when building and testing our client artifacts. |
| -0 :warning: | patch | 24m 37s | | Used diff version of patch file.
Binary files and potentially other changes not applied. Please rebase and
squash commits if necessary. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 29s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 5m 39s | | the patch passed |
| +1 :green_heart: | compile | 24m 51s | | the patch passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javac | 24m 51s | | the patch passed |
| +1 :green_heart: | compile | 22m 12s | | the patch passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | javac | 22m 12s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| -0 :warning: | checkstyle | 4m 16s |
[/results-checkstyle-root.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4858/14/artifact/out/results-checkstyle-root.txt)
| root: The patch generated 1 new + 0 unchanged - 1 fixed = 1 total (was 1) |
| +1 :green_heart: | mvnsite | 6m 59s | | the patch passed |
| +1 :green_heart: | javadoc | 4m 34s | | the patch passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 3m 56s | | the patch passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +0 :ok: | spotbugs | 0m 28s | | hadoop-project has no data from
spotbugs |
| +1 :green_heart: | shadedclient | 24m 42s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 0m 29s | | hadoop-project in the patch
passed. |
| +1 :green_heart: | unit | 243m 29s | | hadoop-yarn in the patch
passed. |
| +1 :green_heart: | unit | 3m 28s | | hadoop-yarn-server-common in
the patch passed. |
| +1 :green_heart: | asflicense | 1m 20s | | The patch does not
generate ASF License warnings. |
| | | 517m 51s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4858/14/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/4858 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient codespell detsecrets xmllint spotbugs checkstyle |
| uname | Linux d798eeab891c 4.15.0-191-generic #202-Ubuntu SMP Thu Aug 4
01:49:29 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk /
[jira] [Commented] (YARN-11290) Improve Query Condition of FederationStateStore#getApplicationsHomeSubCluster
[
https://issues.apache.org/jira/browse/YARN-11290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17605230#comment-17605230
]
ASF GitHub Bot commented on YARN-11290:
---
hadoop-yetus commented on PR #4846:
URL: https://github.com/apache/hadoop/pull/4846#issuecomment-1247843639
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 38s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 1s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available.
|
| +0 :ok: | xmllint | 0m 0s | | xmllint was not available. |
| +0 :ok: | buf | 0m 0s | | buf was not available. |
| +0 :ok: | buf | 0m 0s | | buf was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 5 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 15m 35s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 27m 33s | | trunk passed |
| +1 :green_heart: | compile | 10m 42s | | trunk passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | compile | 9m 17s | | trunk passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 2m 8s | | trunk passed |
| +1 :green_heart: | mvnsite | 8m 31s | | trunk passed |
| +1 :green_heart: | javadoc | 6m 19s | | trunk passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 5m 16s | | trunk passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 19m 15s | | trunk passed |
| +1 :green_heart: | shadedclient | 22m 28s | | branch has no errors
when building and testing our client artifacts. |
| -0 :warning: | patch | 22m 52s | | Used diff version of patch file.
Binary files and potentially other changes not applied. Please rebase and
squash commits if necessary. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 31s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 7m 39s | | the patch passed |
| +1 :green_heart: | compile | 9m 18s | | the patch passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | cc | 9m 18s | | the patch passed |
| +1 :green_heart: | javac | 9m 18s | | the patch passed |
| +1 :green_heart: | compile | 8m 46s | | the patch passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | cc | 8m 46s | | the patch passed |
| +1 :green_heart: | javac | 8m 46s | | the patch passed |
| +1 :green_heart: | blanks | 0m 1s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 1m 53s | | the patch passed |
| +1 :green_heart: | mvnsite | 7m 51s | | the patch passed |
| +1 :green_heart: | javadoc | 5m 11s | | the patch passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 4m 38s | | the patch passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 17m 35s | | the patch passed |
| +1 :green_heart: | shadedclient | 21m 10s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| -1 :x: | unit | 237m 42s |
[/patch-unit-hadoop-yarn-project_hadoop-yarn.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4846/24/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn.txt)
| hadoop-yarn in the patch passed. |
| +1 :green_heart: | unit | 1m 32s | | hadoop-yarn-api in the patch
passed. |
| +1 :green_heart: | unit | 5m 28s | | hadoop-yarn-common in the patch
passed. |
| +1 :green_heart: | unit | 3m 34s | | hadoop-yarn-server-common in
the patch passed. |
| +1 :green_heart: | asflicense | 1m 20s | | The patch does not
generate ASF License warnings. |
| | | 464m 13s | | |
| Reason | Tests |
|---:|:--|
| Failed junit tests |
hadoop.yarn.server.nodemanager.containermanager.logaggregation.TestLogAggregationService
|
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4846/24/artifact/out/Dockerfile
|
| GITHUB PR |
[jira] [Updated] (YARN-11207) Add configuration to disable queue's user limit
[ https://issues.apache.org/jira/browse/YARN-11207?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated YARN-11207: -- Labels: capacityscheduler pull-request-available (was: capacityscheduler) > Add configuration to disable queue's user limit > --- > > Key: YARN-11207 > URL: https://issues.apache.org/jira/browse/YARN-11207 > Project: Hadoop YARN > Issue Type: Improvement > Components: capacityscheduler >Reporter: liu bin >Priority: Major > Labels: capacityscheduler, pull-request-available > > In our cluster, a queue is only used by one user, so there is no need to > limit user resource. > I think we can add a config to disable queue's user limit, and user limit is > enabled by default. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-11207) Add configuration to disable queue's user limit
[ https://issues.apache.org/jira/browse/YARN-11207?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liu bin updated YARN-11207: --- External issue URL: https://github.com/apache/hadoop/pull/4891 > Add configuration to disable queue's user limit > --- > > Key: YARN-11207 > URL: https://issues.apache.org/jira/browse/YARN-11207 > Project: Hadoop YARN > Issue Type: Improvement > Components: capacityscheduler >Reporter: liu bin >Priority: Major > Labels: capacityscheduler, pull-request-available > > In our cluster, a queue is only used by one user, so there is no need to > limit user resource. > I think we can add a config to disable queue's user limit, and user limit is > enabled by default. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11207) Add configuration to disable queue's user limit
[ https://issues.apache.org/jira/browse/YARN-11207?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17605221#comment-17605221 ] ASF GitHub Bot commented on YARN-11207: --- liubin101 opened a new pull request, #4891: URL: https://github.com/apache/hadoop/pull/4891 ### Description of PR In our cluster, a queue is only used by one user, so there is no need to limit user resource. I think we can add a config to disable queue's user limit, and user limit is enabled by default. > Add configuration to disable queue's user limit > --- > > Key: YARN-11207 > URL: https://issues.apache.org/jira/browse/YARN-11207 > Project: Hadoop YARN > Issue Type: Improvement > Components: capacityscheduler >Reporter: liu bin >Priority: Major > Labels: capacityscheduler > > In our cluster, a queue is only used by one user, so there is no need to > limit user resource. > I think we can add a config to disable queue's user limit, and user limit is > enabled by default. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-11207) Add configuration to disable queue's user limit
[ https://issues.apache.org/jira/browse/YARN-11207?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liu bin updated YARN-11207: --- Description: In our cluster, a queue is only used by one user, so there is no need to limit user resource. I think we can add a config to disable queue's user limit, and user limit is enabled by default. was: In our cluster, a queue is only used by one user, so there is no need to limit user resource. I think we can add configuration to disable the user resource limit. And the user limit is enabled by default. Summary: Add configuration to disable queue's user limit (was: Add configuration to disable queue's user resource limit) > Add configuration to disable queue's user limit > --- > > Key: YARN-11207 > URL: https://issues.apache.org/jira/browse/YARN-11207 > Project: Hadoop YARN > Issue Type: Improvement > Components: capacityscheduler >Reporter: liu bin >Priority: Major > Labels: capacityscheduler > > In our cluster, a queue is only used by one user, so there is no need to > limit user resource. > I think we can add a config to disable queue's user limit, and user limit is > enabled by default. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-11207) Add configuration to disable queue's user limit
[ https://issues.apache.org/jira/browse/YARN-11207?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liu bin updated YARN-11207: --- Attachment: (was: YARN-11207.001.patch) > Add configuration to disable queue's user limit > --- > > Key: YARN-11207 > URL: https://issues.apache.org/jira/browse/YARN-11207 > Project: Hadoop YARN > Issue Type: Improvement > Components: capacityscheduler >Reporter: liu bin >Priority: Major > Labels: capacityscheduler > > In our cluster, a queue is only used by one user, so there is no need to > limit user resource. > I think we can add a config to disable queue's user limit, and user limit is > enabled by default. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11247) Remove unused classes introduced by YARN-9615
[ https://issues.apache.org/jira/browse/YARN-11247?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17605130#comment-17605130 ] ASF GitHub Bot commented on YARN-11247: --- slfan1989 commented on PR #4720: URL: https://github.com/apache/hadoop/pull/4720#issuecomment-1247629168 @ayushtkn Can you help review this pr? Thank you very much! > Remove unused classes introduced by YARN-9615 > - > > Key: YARN-11247 > URL: https://issues.apache.org/jira/browse/YARN-11247 > Project: Hadoop YARN > Issue Type: Sub-task > Components: resourcemanager >Affects Versions: 3.4.0 >Reporter: fanshilun >Assignee: fanshilun >Priority: Minor > Labels: pull-request-available > Attachments: DisableEventTypeMetrics-Not used.png > > > YARN-9615 adds Metric to RM's dispatcher, but the patch introduces a class > without any usage > org.apache.hadoop.yarn.metrics#DisableEventTypeMetrics > 1. Without any code references > 2. Without any test code references > 3. Delete this class, the local can be compiled successfully > I think this class can be removed. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10846) Add dispatcher metrics to NM
[ https://issues.apache.org/jira/browse/YARN-10846?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17605129#comment-17605129 ] ASF GitHub Bot commented on YARN-10846: --- slfan1989 commented on PR #4687: URL: https://github.com/apache/hadoop/pull/4687#issuecomment-1247626915 @jojochuang Can you help review the code? Thank you very much! > Add dispatcher metrics to NM > > > Key: YARN-10846 > URL: https://issues.apache.org/jira/browse/YARN-10846 > Project: Hadoop YARN > Issue Type: Sub-task > Components: nodemanager >Reporter: chaosju >Assignee: fanshilun >Priority: Major > Labels: pull-request-available > Attachments: YARN-10846.001.patch, screenshot-1.png > > Time Spent: 1h 20m > Remaining Estimate: 0h > > base [YARN-9615|https://issues.apache.org/jira/browse/YARN-9615], add > dispatcher metrics to NM -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
