[jira] [Created] (YARN-11592) Add timeout to GPGUtils#invokeRMWebService.
Shilun Fan created YARN-11592: - Summary: Add timeout to GPGUtils#invokeRMWebService. Key: YARN-11592 URL: https://issues.apache.org/jira/browse/YARN-11592 Project: Hadoop YARN Issue Type: Improvement Components: federation Affects Versions: 3.4.0 Reporter: Shilun Fan Assignee: Shilun Fan GPGUtils#invokeRMWebService utilizes a Client, and we should set a timeout for the Client. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9013) [GPG] fix order of steps cleaning Registry entries in ApplicationCleaner
[
https://issues.apache.org/jira/browse/YARN-9013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17774790#comment-17774790
]
ASF GitHub Bot commented on YARN-9013:
--
hadoop-yetus commented on PR #6147:
URL: https://github.com/apache/hadoop/pull/6147#issuecomment-1760867408
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 48s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 1 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 47m 58s | | trunk passed |
| +1 :green_heart: | compile | 0m 25s | | trunk passed with JDK
Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04 |
| +1 :green_heart: | compile | 0m 23s | | trunk passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | checkstyle | 0m 24s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 28s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 32s | | trunk passed with JDK
Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04 |
| +1 :green_heart: | javadoc | 0m 25s | | trunk passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | spotbugs | 0m 46s | | trunk passed |
| +1 :green_heart: | shadedclient | 36m 48s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 18s | | the patch passed |
| +1 :green_heart: | compile | 0m 17s | | the patch passed with JDK
Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04 |
| +1 :green_heart: | javac | 0m 17s | | the patch passed |
| +1 :green_heart: | compile | 0m 17s | | the patch passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | javac | 0m 17s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| -0 :warning: | checkstyle | 0m 13s |
[/results-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-globalpolicygenerator.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6147/3/artifact/out/results-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-globalpolicygenerator.txt)
|
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-globalpolicygenerator:
The patch generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0) |
| +1 :green_heart: | mvnsite | 0m 19s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 18s | | the patch passed with JDK
Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04 |
| +1 :green_heart: | javadoc | 0m 17s | | the patch passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | spotbugs | 0m 44s | | the patch passed |
| +1 :green_heart: | shadedclient | 37m 8s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 0m 49s | |
hadoop-yarn-server-globalpolicygenerator in the patch passed. |
| +1 :green_heart: | asflicense | 0m 34s | | The patch does not
generate ASF License warnings. |
| | | 133m 48s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6147/3/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/6147 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux e28c975275a5 5.15.0-78-generic #85-Ubuntu SMP Fri Jul 7
15:25:09 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 8ed421335721a4ac5ace6a771f1f86f2ef12138f |
| Default Java | Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6147/3/testReport/ |
| Max. process+t
[jira] [Commented] (YARN-11571) [GPG] Add Information About YARN GPG in Federation.md
[
https://issues.apache.org/jira/browse/YARN-11571?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17774785#comment-17774785
]
ASF GitHub Bot commented on YARN-11571:
---
hadoop-yetus commented on PR #6158:
URL: https://github.com/apache/hadoop/pull/6158#issuecomment-1760829170
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 56s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +0 :ok: | markdownlint | 0m 0s | | markdownlint was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 44m 42s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 35s | | trunk passed |
| +1 :green_heart: | shadedclient | 78m 29s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 17s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | mvnsite | 0m 21s | | the patch passed |
| +1 :green_heart: | shadedclient | 32m 17s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | asflicense | 0m 41s | | The patch does not
generate ASF License warnings. |
| | | 117m 1s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6158/4/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/6158 |
| Optional Tests | dupname asflicense mvnsite codespell detsecrets
markdownlint |
| uname | Linux 402da78cb998 4.15.0-213-generic #224-Ubuntu SMP Mon Jun 19
13:30:12 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 0400b8bd563f90746edd35dd9c10ee5a4c021525 |
| Max. process+thread count | 731 (vs. ulimit of 5500) |
| modules | C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site U:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6158/4/console |
| versions | git=2.25.1 maven=3.6.3 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was automatically generated.
> [GPG] Add Information About YARN GPG in Federation.md
> -
>
> Key: YARN-11571
> URL: https://issues.apache.org/jira/browse/YARN-11571
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: federation
>Reporter: Shilun Fan
>Assignee: Shilun Fan
>Priority: Major
> Labels: pull-request-available
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11591) Fix some wrong symbols in Federation.md
[ https://issues.apache.org/jira/browse/YARN-11591?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17774784#comment-17774784 ] ASF GitHub Bot commented on YARN-11591: --- slfan1989 commented on PR #6182: URL: https://github.com/apache/hadoop/pull/6182#issuecomment-1760813763 @singer-bin Thank you for your contribution! But the previous symbols displayed well on the page. I believe we don't need this kind of modification. https://github.com/apache/hadoop/assets/55643692/f7e09a16-3188-4582-bf2f-90bbf09fd58b";> > Fix some wrong symbols in Federation.md > --- > > Key: YARN-11591 > URL: https://issues.apache.org/jira/browse/YARN-11591 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: yanbin.zhang >Priority: Trivial > Labels: pull-request-available > > Fix some wrong symbols in Federation.md -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11591) Fix some wrong symbols in Federation.md
[
https://issues.apache.org/jira/browse/YARN-11591?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17774775#comment-17774775
]
ASF GitHub Bot commented on YARN-11591:
---
hadoop-yetus commented on PR #6182:
URL: https://github.com/apache/hadoop/pull/6182#issuecomment-1760707246
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 47s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +0 :ok: | markdownlint | 0m 0s | | markdownlint was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
_ trunk Compile Tests _ |
| -1 :x: | mvninstall | 0m 24s |
[/branch-mvninstall-root.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6182/1/artifact/out/branch-mvninstall-root.txt)
| root in trunk failed. |
| +1 :green_heart: | mvnsite | 1m 19s | | trunk passed |
| -1 :x: | shadedclient | 6m 15s | | branch has errors when building
and testing our client artifacts. |
_ Patch Compile Tests _ |
| -1 :x: | mvninstall | 0m 24s |
[/patch-mvninstall-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-site.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6182/1/artifact/out/patch-mvninstall-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-site.txt)
| hadoop-yarn-site in the patch failed. |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | mvnsite | 0m 30s | | the patch passed |
| -1 :x: | shadedclient | 3m 51s | | patch has errors when building
and testing our client artifacts. |
_ Other Tests _ |
| +0 :ok: | asflicense | 0m 25s | | ASF License check generated no
output? |
| | | 17m 15s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6182/1/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/6182 |
| Optional Tests | dupname asflicense mvnsite codespell detsecrets
markdownlint |
| uname | Linux b27f05e5e724 4.15.0-213-generic #224-Ubuntu SMP Mon Jun 19
13:30:12 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 8e44ed9fd284fd23acc03950e83cb81afc2744c5 |
| Max. process+thread count | 56 (vs. ulimit of 5500) |
| modules | C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site U:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6182/1/console |
| versions | git=2.25.1 maven=3.6.3 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was automatically generated.
> Fix some wrong symbols in Federation.md
> ---
>
> Key: YARN-11591
> URL: https://issues.apache.org/jira/browse/YARN-11591
> Project: Hadoop YARN
> Issue Type: Improvement
>Reporter: yanbin.zhang
>Priority: Trivial
> Labels: pull-request-available
>
> Fix some wrong symbols in Federation.md
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11484) [Federation] Router Supports Yarn Client CLI Cmds.
[
https://issues.apache.org/jira/browse/YARN-11484?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17774771#comment-17774771
]
ASF GitHub Bot commented on YARN-11484:
---
slfan1989 commented on code in PR #6132:
URL: https://github.com/apache/hadoop/pull/6132#discussion_r1357707310
##
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router/src/test/java/org/apache/hadoop/yarn/server/router/clientrm/TestFederationClientInterceptor.java:
##
@@ -1174,6 +1174,22 @@ public void testGetQueueInfo() throws Exception {
Assert.assertEquals(queueInfo.getAccessibleNodeLabels().size(), 1);
}
+ @Test
+ public void testSubClusterGetQueueInfo() throws IOException, YarnException {
+// We have set up a unit test where we access queue information for
subcluster1.
+GetQueueInfoResponse response = interceptor.getQueueInfo(
+GetQueueInfoRequest.newInstance("root", true, true, true, "1"));
+Assert.assertNotNull(response);
+
+QueueInfo queueInfo = response.getQueueInfo();
+Assert.assertNotNull(queueInfo);
+Assert.assertEquals(queueInfo.getQueueName(), "root");
Review Comment:
I will fix it.
> [Federation] Router Supports Yarn Client CLI Cmds.
> --
>
> Key: YARN-11484
> URL: https://issues.apache.org/jira/browse/YARN-11484
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: federation
>Reporter: Shilun Fan
>Assignee: Shilun Fan
>Priority: Major
> Labels: pull-request-available
>
> This Jira ticket aims to enhance the Router command by adding support for all
> Yarn Client CLI options.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-11591) Fix some wrong symbols in Federation.md
[ https://issues.apache.org/jira/browse/YARN-11591?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated YARN-11591: -- Labels: pull-request-available (was: ) > Fix some wrong symbols in Federation.md > --- > > Key: YARN-11591 > URL: https://issues.apache.org/jira/browse/YARN-11591 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: yanbin.zhang >Priority: Trivial > Labels: pull-request-available > > Fix some wrong symbols in Federation.md -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11484) [Federation] Router Supports Yarn Client CLI Cmds.
[
https://issues.apache.org/jira/browse/YARN-11484?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17774770#comment-17774770
]
ASF GitHub Bot commented on YARN-11484:
---
slfan1989 commented on code in PR #6132:
URL: https://github.com/apache/hadoop/pull/6132#discussion_r1357707184
##
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/test/java/org/apache/hadoop/yarn/client/cli/TestYarnCLI.java:
##
@@ -1969,6 +1971,98 @@ public void testGetQueueInfoWithEmptyNodeLabel() throws
Exception {
String queueInfoStr = baos.toString("UTF-8");
Assert.assertEquals(queueInfoStr, sysOutStream.toString());
}
+
+ @Test
+ public void testGetQueueInfoWithFairScheduler() throws Exception {
+// In this test case, we will simulate the queue information of
fairScheduler
+// and check the results of the queue information.
+QueueCLI cli = createAndGetQueueCLI();
+RecordFactory recordFactory = RecordFactoryProvider.getRecordFactory(null);
+QueueInfo queueInfo = recordFactory.newRecordInstance(QueueInfo.class);
+queueInfo.setQueueName("queueA");
+queueInfo.setSchedulerType("FairScheduler");
+queueInfo.setQueueState(QueueState.RUNNING);
+queueInfo.setCapacity(0.3f);
+queueInfo.setCurrentCapacity(0.1f);
+queueInfo.setWeight(0.3f);
+queueInfo.setMinResourceVCore(1);
+queueInfo.setMinResourceMemory(1024);
+queueInfo.setMaxResourceVCore(10);
+queueInfo.setMaxResourceMemory(8192);
+queueInfo.setReservedResourceVCore(0);
+queueInfo.setReservedResourceMemory(0);
+queueInfo.setSteadyFairShareVCore(10);
+queueInfo.setSteadyFairShareMemory(8192);
+queueInfo.setMaxRunningApp(10);
+queueInfo.setPreemptionDisabled(true);
+when(client.getQueueInfo(any(String.class))).thenReturn(queueInfo);
+int result = cli.run(new String[] { "-status", "queueA" });
+assertEquals(0, result);
+verify(client).getQueueInfo("queueA");
+ByteArrayOutputStream baos = new ByteArrayOutputStream();
+PrintWriter pw = new PrintWriter(baos);
+pw.println("Queue Information : ");
+pw.println("Scheduler Name : FairScheduler");
+pw.println("Queue Name : " + "queueA");
+pw.println("\tWeight : " + "0.30");
+pw.println("\tState : " + "RUNNING");
+pw.println("\tMinResource : " + "");
+pw.println("\tMaxResource : " + "");
Review Comment:
Thank you very much for your help in reviewing the code! I wrote this unit
test based on the original unit test. I will improve this part of the code.
> [Federation] Router Supports Yarn Client CLI Cmds.
> --
>
> Key: YARN-11484
> URL: https://issues.apache.org/jira/browse/YARN-11484
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: federation
>Reporter: Shilun Fan
>Assignee: Shilun Fan
>Priority: Major
> Labels: pull-request-available
>
> This Jira ticket aims to enhance the Router command by adding support for all
> Yarn Client CLI options.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11591) Fix some wrong symbols in Federation.md
[ https://issues.apache.org/jira/browse/YARN-11591?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17774772#comment-17774772 ] ASF GitHub Bot commented on YARN-11591: --- singer-bin opened a new pull request, #6182: URL: https://github.com/apache/hadoop/pull/6182 about jira:https://issues.apache.org/jira/browse/YARN-11591 > Fix some wrong symbols in Federation.md > --- > > Key: YARN-11591 > URL: https://issues.apache.org/jira/browse/YARN-11591 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: yanbin.zhang >Priority: Trivial > > Fix some wrong symbols in Federation.md -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-11591) Fix some wrong symbols in Federation.md
yanbin.zhang created YARN-11591: --- Summary: Fix some wrong symbols in Federation.md Key: YARN-11591 URL: https://issues.apache.org/jira/browse/YARN-11591 Project: Hadoop YARN Issue Type: Improvement Reporter: yanbin.zhang Fix some wrong symbols in Federation.md -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11498) Exclude Jettison from jersey-json artifact in hadoop-yarn-common's pom.xml
[
https://issues.apache.org/jira/browse/YARN-11498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17774767#comment-17774767
]
ASF GitHub Bot commented on YARN-11498:
---
hadoop-yetus commented on PR #6063:
URL: https://github.com/apache/hadoop/pull/6063#issuecomment-1760681441
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 53s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +0 :ok: | xmllint | 0m 0s | | xmllint was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| -1 :x: | test4tests | 0m 0s | | The patch doesn't appear to include
any new or modified tests. Please justify why no new tests are needed for this
patch. Also please list what manual steps were performed to verify this patch.
|
_ branch-3.3 Compile Tests _ |
| +0 :ok: | mvndep | 15m 45s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 40m 50s | | branch-3.3 passed |
| +1 :green_heart: | compile | 20m 46s | | branch-3.3 passed |
| +1 :green_heart: | mvnsite | 8m 31s | | branch-3.3 passed |
| +1 :green_heart: | javadoc | 6m 28s | | branch-3.3 passed |
| +1 :green_heart: | shadedclient | 132m 35s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 30s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 25m 50s | | the patch passed |
| +1 :green_heart: | compile | 18m 40s | | the patch passed |
| +1 :green_heart: | javac | 18m 40s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | mvnsite | 8m 27s | | the patch passed |
| +1 :green_heart: | javadoc | 6m 26s | | the patch passed |
| -1 :x: | shadedclient | 56m 32s | | patch has errors when building
and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 0m 29s | | hadoop-project in the patch
passed. |
| +1 :green_heart: | unit | 18m 2s | | hadoop-common in the patch
passed. |
| +1 :green_heart: | unit | 4m 51s | | hadoop-yarn-common in the patch
passed. |
| +1 :green_heart: | unit | 22m 52s | | hadoop-yarn-server-nodemanager
in the patch passed. |
| +1 :green_heart: | unit | 4m 12s | |
hadoop-yarn-server-applicationhistoryservice in the patch passed. |
| +1 :green_heart: | unit | 94m 20s | |
hadoop-yarn-server-resourcemanager in the patch passed. |
| +1 :green_heart: | unit | 1m 10s | |
hadoop-yarn-applications-catalog-webapp in the patch passed. |
| +1 :green_heart: | unit | 0m 53s | | hadoop-resourceestimator in the
patch passed. |
| +1 :green_heart: | unit | 0m 34s | | hadoop-client-runtime in the
patch passed. |
| +1 :green_heart: | unit | 0m 34s | | hadoop-client-minicluster in
the patch passed. |
| +1 :green_heart: | asflicense | 0m 57s | | The patch does not
generate ASF License warnings. |
| | | 389m 5s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6063/7/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/6063 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient codespell detsecrets xmllint |
| uname | Linux 4795f7feffc9 4.15.0-213-generic #224-Ubuntu SMP Mon Jun 19
13:30:12 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | branch-3.3 / 4d6e7806ef9a79231f4cf1d5406f736d45d273b3 |
| Default Java | Private Build-1.8.0_362-8u372-ga~us1-0ubuntu1~18.04-b09 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6063/7/testReport/ |
| Max. process+thread count | 3134 (vs. ulimit of 5500) |
| modules | C: hadoop-project hadoop-common-project/hadoop-common
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
hadoop-yarn-pr
[jira] [Commented] (YARN-11588) Fix uncleaned threads in YARN Federation interceptor threadpool
[ https://issues.apache.org/jira/browse/YARN-11588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17774442#comment-17774442 ] ASF GitHub Bot commented on YARN-11588: --- slfan1989 commented on PR #6159: URL: https://github.com/apache/hadoop/pull/6159#issuecomment-1759410254 @jchanggg Thank you for your contribution! @goiri Thank you for your help reviewing the code! > Fix uncleaned threads in YARN Federation interceptor threadpool > --- > > Key: YARN-11588 > URL: https://issues.apache.org/jira/browse/YARN-11588 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Jeffrey Chang >Priority: Major > Labels: pull-request-available > > In YARN Federation a new interceptor is generated for each new user that > calls YARN Router. This also generates a threadpool which is used to call > YARN RM. However, when the call is finished the threadpool threads never > exits, causing the interceptor object for the user to never be cleaned up. > This causes an issue when the number of users is large in the YARN cluster, > as the number of idle, uncleaned thread can reach # of users calling router > times number of active core threads in thread pool. This causes performance > issues in router due to too many idle threads. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11588) Fix uncleaned threads in YARN Federation interceptor threadpool
[ https://issues.apache.org/jira/browse/YARN-11588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17774441#comment-17774441 ] ASF GitHub Bot commented on YARN-11588: --- slfan1989 merged PR #6159: URL: https://github.com/apache/hadoop/pull/6159 > Fix uncleaned threads in YARN Federation interceptor threadpool > --- > > Key: YARN-11588 > URL: https://issues.apache.org/jira/browse/YARN-11588 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Jeffrey Chang >Priority: Major > Labels: pull-request-available > > In YARN Federation a new interceptor is generated for each new user that > calls YARN Router. This also generates a threadpool which is used to call > YARN RM. However, when the call is finished the threadpool threads never > exits, causing the interceptor object for the user to never be cleaned up. > This causes an issue when the number of users is large in the YARN cluster, > as the number of idle, uncleaned thread can reach # of users calling router > times number of active core threads in thread pool. This causes performance > issues in router due to too many idle threads. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
