[jira] [Commented] (YARN-11414) ClusterMetricsInfo shows wrong availableMB when node labels enabled
[ https://issues.apache.org/jira/browse/YARN-11414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17676495#comment-17676495 ] Prabhu Joseph commented on YARN-11414: -- [~maniraj...@gmail.com] We see ClusterMetricsInfo shows available and allocated only for Default Partition. We have QueueMetrics which already shows for every partition level. This Jira intends to change the ClusterMetrics to show Cluster Wide which will help all the Schedulers. Do you have any comments. > ClusterMetricsInfo shows wrong availableMB when node labels enabled > > > Key: YARN-11414 > URL: https://issues.apache.org/jira/browse/YARN-11414 > Project: Hadoop YARN > Issue Type: Bug >Affects Versions: 3.3.3 >Reporter: Prabhu Joseph >Assignee: Ashutosh Gupta >Priority: Major > Labels: pull-request-available > > ClusterMetricsInfo shows wrong availableMB when node labels enabled. It shows > availableMB of Default Partition alone. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11414) ClusterMetricsInfo shows wrong availableMB when node labels enabled
[ https://issues.apache.org/jira/browse/YARN-11414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17676490#comment-17676490 ] ASF GitHub Bot commented on YARN-11414: --- PrabhuJoseph commented on PR #5293: URL: https://github.com/apache/hadoop/pull/5293#issuecomment-1381382711 @cnauroth Thanks for the review. 1. All those Jiras pointed out above are related to QueueMetrics which is still going to be the same. This patch addresses the issues with ClusterMetricsInfo shown in RMWebService. 2. Currently, ClusterMetricsInfo shows available or allocated memory only for Default Partition. I don't think this is by intention, if so there has to be at least one test case failed with this patch. From the name ClusterMetricsInfo, it is Cluster Wide, showing only for Default Partition does not look right and also does not help clusters where some/all nodes are part of non-default partitions. 3. Without this patch, currently i see there is no way to retrieve the Cluster Wide Available and Allocated Memory. But we have ways to get it for Partition level already through QueueMetrics . Having both ClusterMetrics and QueueMetrics showing for Default Partition seems redundant. > ClusterMetricsInfo shows wrong availableMB when node labels enabled > > > Key: YARN-11414 > URL: https://issues.apache.org/jira/browse/YARN-11414 > Project: Hadoop YARN > Issue Type: Bug >Affects Versions: 3.3.3 >Reporter: Prabhu Joseph >Assignee: Ashutosh Gupta >Priority: Major > Labels: pull-request-available > > ClusterMetricsInfo shows wrong availableMB when node labels enabled. It shows > availableMB of Default Partition alone. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11218) [Federation] Add getActivities, getBulkActivities REST APIs for Router
[
https://issues.apache.org/jira/browse/YARN-11218?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17676471#comment-17676471
]
ASF GitHub Bot commented on YARN-11218:
---
hadoop-yetus commented on PR #5284:
URL: https://github.com/apache/hadoop/pull/5284#issuecomment-1381358119
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 35s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 3 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 16m 0s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 30m 55s | | trunk passed |
| +1 :green_heart: | compile | 3m 51s | | trunk passed with JDK
Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04 |
| +1 :green_heart: | compile | 3m 19s | | trunk passed with JDK
Private Build-1.8.0_352-8u352-ga-1~20.04-b08 |
| +1 :green_heart: | checkstyle | 1m 15s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 41s | | trunk passed |
| -1 :x: | javadoc | 0m 56s |
[/branch-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdkUbuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5284/4/artifact/out/branch-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdkUbuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04.txt)
| hadoop-yarn-server-resourcemanager in trunk failed with JDK
Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04. |
| +1 :green_heart: | javadoc | 1m 14s | | trunk passed with JDK
Private Build-1.8.0_352-8u352-ga-1~20.04-b08 |
| +1 :green_heart: | spotbugs | 2m 59s | | trunk passed |
| +1 :green_heart: | shadedclient | 23m 10s | | branch has no errors
when building and testing our client artifacts. |
| -0 :warning: | patch | 23m 29s | | Used diff version of patch file.
Binary files and potentially other changes not applied. Please rebase and
squash commits if necessary. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 29s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 1m 29s | | the patch passed |
| +1 :green_heart: | compile | 3m 42s | | the patch passed with JDK
Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04 |
| +1 :green_heart: | javac | 3m 42s | | the patch passed |
| +1 :green_heart: | compile | 3m 10s | | the patch passed with JDK
Private Build-1.8.0_352-8u352-ga-1~20.04-b08 |
| +1 :green_heart: | javac | 3m 10s | | the patch passed |
| -1 :x: | blanks | 0m 0s |
[/blanks-eol.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5284/4/artifact/out/blanks-eol.txt)
| The patch has 6 line(s) that end in blanks. Use git apply --whitespace=fix
<>. Refer https://git-scm.com/docs/git-apply |
| +1 :green_heart: | checkstyle | 1m 1s | | the patch passed |
| +1 :green_heart: | mvnsite | 1m 22s | | the patch passed |
| -1 :x: | javadoc | 0m 41s |
[/patch-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdkUbuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5284/4/artifact/out/patch-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdkUbuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04.txt)
| hadoop-yarn-server-resourcemanager in the patch failed with JDK
Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04. |
| +1 :green_heart: | javadoc | 1m 2s | | the patch passed with JDK
Private Build-1.8.0_352-8u352-ga-1~20.04-b08 |
| +1 :green_heart: | spotbugs | 2m 55s | | the patch passed |
| +1 :green_heart: | shadedclient | 23m 0s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 98m 34s | |
hadoop-yarn-server-resourcemanager in the patch passed. |
| +1 :green_heart: | unit | 0m 30s | | hadoop-yarn-server-router in
the patch passed. |
| +1 :green_heart: | asflicense | 0m 35s | | The patch does not
generate ASF License warnings. |
| | | 227m 33s | | |
| Subsystem | Report/Notes |
|--:|:-|
[jira] [Commented] (YARN-11326) [Federation] Add RM FederationStateStoreService Metrics
[
https://issues.apache.org/jira/browse/YARN-11326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17676469#comment-17676469
]
ASF GitHub Bot commented on YARN-11326:
---
hadoop-yetus commented on PR #4963:
URL: https://github.com/apache/hadoop/pull/4963#issuecomment-1381356492
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 53s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 1s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 2 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 46m 31s | | trunk passed |
| +1 :green_heart: | compile | 1m 4s | | trunk passed with JDK
Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04 |
| +1 :green_heart: | compile | 0m 55s | | trunk passed with JDK
Private Build-1.8.0_352-8u352-ga-1~20.04-b08 |
| +1 :green_heart: | checkstyle | 0m 51s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 1s | | trunk passed |
| -1 :x: | javadoc | 0m 54s |
[/branch-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdkUbuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4963/27/artifact/out/branch-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdkUbuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04.txt)
| hadoop-yarn-server-resourcemanager in trunk failed with JDK
Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04. |
| +1 :green_heart: | javadoc | 0m 41s | | trunk passed with JDK
Private Build-1.8.0_352-8u352-ga-1~20.04-b08 |
| +1 :green_heart: | spotbugs | 2m 8s | | trunk passed |
| +1 :green_heart: | shadedclient | 27m 33s | | branch has no errors
when building and testing our client artifacts. |
| -0 :warning: | patch | 27m 50s | | Used diff version of patch file.
Binary files and potentially other changes not applied. Please rebase and
squash commits if necessary. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 59s | | the patch passed |
| +1 :green_heart: | compile | 0m 59s | | the patch passed with JDK
Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04 |
| +1 :green_heart: | javac | 0m 59s | | the patch passed |
| +1 :green_heart: | compile | 0m 51s | | the patch passed with JDK
Private Build-1.8.0_352-8u352-ga-1~20.04-b08 |
| +1 :green_heart: | javac | 0m 51s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 0m 39s | | the patch passed |
| +1 :green_heart: | mvnsite | 0m 52s | | the patch passed |
| -1 :x: | javadoc | 0m 39s |
[/patch-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdkUbuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4963/27/artifact/out/patch-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdkUbuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04.txt)
| hadoop-yarn-server-resourcemanager in the patch failed with JDK
Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04. |
| +1 :green_heart: | javadoc | 0m 36s | | the patch passed with JDK
Private Build-1.8.0_352-8u352-ga-1~20.04-b08 |
| +1 :green_heart: | spotbugs | 2m 1s | | the patch passed |
| +1 :green_heart: | shadedclient | 27m 30s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 102m 34s | |
hadoop-yarn-server-resourcemanager in the patch passed. |
| +1 :green_heart: | asflicense | 0m 34s | | The patch does not
generate ASF License warnings. |
| | | 220m 4s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4963/27/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/4963 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux 2ddc6df70c6c 4.15.0-197-generic #208-Ubuntu SMP Tue Nov 1
17:23:37 UTC 2022 x86_64
[jira] [Commented] (YARN-11217) [Federation] Add dumpSchedulerLogs REST APIs for Router
[
https://issues.apache.org/jira/browse/YARN-11217?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17676459#comment-17676459
]
ASF GitHub Bot commented on YARN-11217:
---
hadoop-yetus commented on PR #5272:
URL: https://github.com/apache/hadoop/pull/5272#issuecomment-1381345154
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 51s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 3 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 46m 6s | | trunk passed |
| +1 :green_heart: | compile | 0m 31s | | trunk passed with JDK
Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04 |
| +1 :green_heart: | compile | 0m 26s | | trunk passed with JDK
Private Build-1.8.0_352-8u352-ga-1~20.04-b08 |
| +1 :green_heart: | checkstyle | 0m 27s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 33s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 35s | | trunk passed with JDK
Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04 |
| +1 :green_heart: | javadoc | 0m 24s | | trunk passed with JDK
Private Build-1.8.0_352-8u352-ga-1~20.04-b08 |
| +1 :green_heart: | spotbugs | 1m 3s | | trunk passed |
| +1 :green_heart: | shadedclient | 26m 26s | | branch has no errors
when building and testing our client artifacts. |
| -0 :warning: | patch | 26m 43s | | Used diff version of patch file.
Binary files and potentially other changes not applied. Please rebase and
squash commits if necessary. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 28s | | the patch passed |
| +1 :green_heart: | compile | 0m 24s | | the patch passed with JDK
Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04 |
| +1 :green_heart: | javac | 0m 24s | | the patch passed |
| +1 :green_heart: | compile | 0m 21s | | the patch passed with JDK
Private Build-1.8.0_352-8u352-ga-1~20.04-b08 |
| +1 :green_heart: | javac | 0m 21s | | the patch passed |
| -1 :x: | blanks | 0m 0s |
[/blanks-eol.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5272/5/artifact/out/blanks-eol.txt)
| The patch has 3 line(s) that end in blanks. Use git apply --whitespace=fix
<>. Refer https://git-scm.com/docs/git-apply |
| +1 :green_heart: | checkstyle | 0m 14s | | the patch passed |
| +1 :green_heart: | mvnsite | 0m 24s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 19s | | the patch passed with JDK
Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04 |
| +1 :green_heart: | javadoc | 0m 17s | | the patch passed with JDK
Private Build-1.8.0_352-8u352-ga-1~20.04-b08 |
| +1 :green_heart: | spotbugs | 0m 54s | | the patch passed |
| +1 :green_heart: | shadedclient | 26m 19s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 0m 28s | | hadoop-yarn-server-router in
the patch passed. |
| +1 :green_heart: | asflicense | 0m 33s | | The patch does not
generate ASF License warnings. |
| | | 109m 28s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5272/5/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/5272 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux faee0be182e4 4.15.0-200-generic #211-Ubuntu SMP Thu Nov 24
18:16:04 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 4b4a57bede3f048da391bfcfcf3d9aa2f9c5041b |
| Default Java | Private Build-1.8.0_352-8u352-ga-1~20.04-b08 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04
/usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_352-8u352-ga-1~20.04-b08 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5272/5/testReport/ |
| Max. process+thread count | 585 (vs. ulimit of 5500) |
| modules | C:
hadoop-yarn-project/hadoop-yarn/hadoop-ya
[jira] [Commented] (YARN-11391) Support using DNS to resolve yarn services to IP addresses
[ https://issues.apache.org/jira/browse/YARN-11391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17676429#comment-17676429 ] ASF GitHub Bot commented on YARN-11391: --- slfan1989 commented on PR #5297: URL: https://github.com/apache/hadoop/pull/5297#issuecomment-1381286425 @lnbest0707 We need to fix the checkstyle issue. > Support using DNS to resolve yarn services to IP addresses > -- > > Key: YARN-11391 > URL: https://issues.apache.org/jira/browse/YARN-11391 > Project: Hadoop YARN > Issue Type: New Feature >Reporter: Shuyi Zhang >Priority: Major > Labels: pull-request-available > > We have been using A-record (hostname) for discovering YARN control > planes(RM, JHS) from both the server and client sides. The components > discover each other using hostnames (DNS A-records) which are preconfigured > in YARN configurations (mapred-site.xml and yarn-site.xml). However, the > hard-coded address mapping is prohibitive to perform appropriate failure > recovery and node replacement. Any host replacement, adding, or removing > requires client integration, which is a quite painful process. > It is essential to find a solution where the discovery configuration of > different components in the cluster stays intact and is able to resolve with > the latest hostname/records. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Resolved] (YARN-11409) Fix Typo of ResourceManager#webapp moudle
[ https://issues.apache.org/jira/browse/YARN-11409?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shilun Fan resolved YARN-11409. --- Target Version/s: 3.4.0 Resolution: Fixed > Fix Typo of ResourceManager#webapp moudle > - > > Key: YARN-11409 > URL: https://issues.apache.org/jira/browse/YARN-11409 > Project: Hadoop YARN > Issue Type: Bug > Components: resourcemanager >Affects Versions: 3.4.0 >Reporter: Shilun Fan >Assignee: Shilun Fan >Priority: Major > Labels: pull-request-available > Fix For: 3.4.0 > > > When finishing YARN-11218, I found some typo problems in RM's RMWebServices. > I checked the java class of the webapp moudle and fixed the typo problems. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11409) Fix Typo of ResourceManager#webapp moudle
[ https://issues.apache.org/jira/browse/YARN-11409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17676427#comment-17676427 ] ASF GitHub Bot commented on YARN-11409: --- slfan1989 merged PR #5285: URL: https://github.com/apache/hadoop/pull/5285 > Fix Typo of ResourceManager#webapp moudle > - > > Key: YARN-11409 > URL: https://issues.apache.org/jira/browse/YARN-11409 > Project: Hadoop YARN > Issue Type: Bug > Components: resourcemanager >Affects Versions: 3.4.0 >Reporter: Shilun Fan >Assignee: Shilun Fan >Priority: Major > Labels: pull-request-available > Fix For: 3.4.0 > > > When finishing YARN-11218, I found some typo problems in RM's RMWebServices. > I checked the java class of the webapp moudle and fixed the typo problems. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11409) Fix Typo of ResourceManager#webapp moudle
[ https://issues.apache.org/jira/browse/YARN-11409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17676425#comment-17676425 ] ASF GitHub Bot commented on YARN-11409: --- slfan1989 commented on PR #5285: URL: https://github.com/apache/hadoop/pull/5285#issuecomment-1381282212 @tomscut Thank you very much for helping to review the code, merge into trunk. > Fix Typo of ResourceManager#webapp moudle > - > > Key: YARN-11409 > URL: https://issues.apache.org/jira/browse/YARN-11409 > Project: Hadoop YARN > Issue Type: Bug > Components: resourcemanager >Affects Versions: 3.4.0 >Reporter: Shilun Fan >Assignee: Shilun Fan >Priority: Major > Labels: pull-request-available > Fix For: 3.4.0 > > > When finishing YARN-11218, I found some typo problems in RM's RMWebServices. > I checked the java class of the webapp moudle and fixed the typo problems. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11391) Support using DNS to resolve yarn services to IP addresses
[
https://issues.apache.org/jira/browse/YARN-11391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17676417#comment-17676417
]
ASF GitHub Bot commented on YARN-11391:
---
hadoop-yetus commented on PR #5297:
URL: https://github.com/apache/hadoop/pull/5297#issuecomment-1381267671
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 57s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +0 :ok: | xmllint | 0m 1s | | xmllint was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 2 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 15m 42s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 31m 12s | | trunk passed |
| +1 :green_heart: | compile | 23m 8s | | trunk passed with JDK
Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04 |
| +1 :green_heart: | compile | 20m 38s | | trunk passed with JDK
Private Build-1.8.0_352-8u352-ga-1~20.04-b08 |
| +1 :green_heart: | checkstyle | 4m 52s | | trunk passed |
| +1 :green_heart: | mvnsite | 3m 57s | | trunk passed |
| -1 :x: | javadoc | 1m 14s |
[/branch-javadoc-hadoop-common-project_hadoop-common-jdkUbuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5297/1/artifact/out/branch-javadoc-hadoop-common-project_hadoop-common-jdkUbuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04.txt)
| hadoop-common in trunk failed with JDK
Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04. |
| +1 :green_heart: | javadoc | 2m 39s | | trunk passed with JDK
Private Build-1.8.0_352-8u352-ga-1~20.04-b08 |
| +1 :green_heart: | spotbugs | 6m 39s | | trunk passed |
| +1 :green_heart: | shadedclient | 24m 39s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 39s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 2m 34s | | the patch passed |
| +1 :green_heart: | compile | 22m 25s | | the patch passed with JDK
Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04 |
| +1 :green_heart: | javac | 22m 25s | | the patch passed |
| +1 :green_heart: | compile | 20m 30s | | the patch passed with JDK
Private Build-1.8.0_352-8u352-ga-1~20.04-b08 |
| +1 :green_heart: | javac | 20m 30s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| -0 :warning: | checkstyle | 3m 37s |
[/results-checkstyle-root.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5297/1/artifact/out/results-checkstyle-root.txt)
| root: The patch generated 2 new + 184 unchanged - 0 fixed = 186 total (was
184) |
| +1 :green_heart: | mvnsite | 3m 46s | | the patch passed |
| -1 :x: | javadoc | 1m 4s |
[/patch-javadoc-hadoop-common-project_hadoop-common-jdkUbuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5297/1/artifact/out/patch-javadoc-hadoop-common-project_hadoop-common-jdkUbuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04.txt)
| hadoop-common in the patch failed with JDK
Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04. |
| +1 :green_heart: | javadoc | 2m 42s | | the patch passed with JDK
Private Build-1.8.0_352-8u352-ga-1~20.04-b08 |
| -1 :x: | spotbugs | 2m 4s |
[/new-spotbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-common.html](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5297/1/artifact/out/new-spotbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-common.html)
| hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common generated 1 new + 0
unchanged - 0 fixed = 1 total (was 0) |
| +1 :green_heart: | shadedclient | 25m 5s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 18m 28s | | hadoop-common in the patch
passed. |
| +1 :green_heart: | unit | 1m 19s | | hadoop-yarn-api in the patch
passed. |
| +1 :green_heart: | unit | 5m 42s | | hadoop-yarn-common in the patch
passed. |
| +1 :green_heart: | asflicense | 1m 2s | | The patch does not
generate ASF License warnings. |
| | | 258m 25s | | |
| Reason | Tests |
|---:|:--|
|
[jira] [Commented] (YARN-11391) Support using DNS to resolve yarn services to IP addresses
[ https://issues.apache.org/jira/browse/YARN-11391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17676416#comment-17676416 ] ASF GitHub Bot commented on YARN-11391: --- slfan1989 commented on PR #5297: URL: https://github.com/apache/hadoop/pull/5297#issuecomment-1381265177 @lnbest0707 Thanks for your contribution, we are waiting for the compilation result of yetus. > Support using DNS to resolve yarn services to IP addresses > -- > > Key: YARN-11391 > URL: https://issues.apache.org/jira/browse/YARN-11391 > Project: Hadoop YARN > Issue Type: New Feature >Reporter: Shuyi Zhang >Priority: Major > Labels: pull-request-available > > We have been using A-record (hostname) for discovering YARN control > planes(RM, JHS) from both the server and client sides. The components > discover each other using hostnames (DNS A-records) which are preconfigured > in YARN configurations (mapred-site.xml and yarn-site.xml). However, the > hard-coded address mapping is prohibitive to perform appropriate failure > recovery and node replacement. Any host replacement, adding, or removing > requires client integration, which is a quite painful process. > It is essential to find a solution where the discovery configuration of > different components in the cluster stays intact and is able to resolve with > the latest hostname/records. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11394) Fix hadoop-yarn-server-resourcemanager module Java Doc Errors.
[ https://issues.apache.org/jira/browse/YARN-11394?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17676415#comment-17676415 ] ASF GitHub Bot commented on YARN-11394: --- slfan1989 commented on PR #5288: URL: https://github.com/apache/hadoop/pull/5288#issuecomment-1381264593 @ayushtkn Thanks for the explanation, I agree with you! > Fix hadoop-yarn-server-resourcemanager module Java Doc Errors. > -- > > Key: YARN-11394 > URL: https://issues.apache.org/jira/browse/YARN-11394 > Project: Hadoop YARN > Issue Type: Improvement > Components: resourcemanager >Affects Versions: 3.4.0 >Reporter: Shilun Fan >Assignee: Shilun Fan >Priority: Major > Labels: pull-request-available > > When I finished the pr of YARN-11350, I encountered java doc error, this jira > will fix the java doc error in hadoop-yarn-server-resourcemanager. > > The java doc error report is as follows: > https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5131/22/artifact/out/branch-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdkUbuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04.txt -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11276) Add lru cache for RMWebServices.getApps
[ https://issues.apache.org/jira/browse/YARN-11276?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17676412#comment-17676412 ] ASF GitHub Bot commented on YARN-11276: --- leixm closed pull request #4793: YARN-11276. Add LRU cache for RMWebServices.getApps URL: https://github.com/apache/hadoop/pull/4793 > Add lru cache for RMWebServices.getApps > --- > > Key: YARN-11276 > URL: https://issues.apache.org/jira/browse/YARN-11276 > Project: Hadoop YARN > Issue Type: Improvement > Components: resourcemanager >Affects Versions: 3.4.0 >Reporter: Xianming Lei >Priority: Minor > Labels: pull-request-available > > In our YARN cluster, there are thousands of apps running at the same time, > the return result of getApps reaches about 10M, and many requests are the > same input parameters, we can add cache for RMWebServices.getApps to reduce > processing delay -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11394) Fix hadoop-yarn-server-resourcemanager module Java Doc Errors.
[ https://issues.apache.org/jira/browse/YARN-11394?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17676399#comment-17676399 ] ASF GitHub Bot commented on YARN-11394: --- ayushtkn commented on PR #5288: URL: https://github.com/apache/hadoop/pull/5288#issuecomment-1381241900 For JDK-11 javadoc there is already a ticket in hadoop common. I think that should fix most of the things. Should wait for it IMO > Fix hadoop-yarn-server-resourcemanager module Java Doc Errors. > -- > > Key: YARN-11394 > URL: https://issues.apache.org/jira/browse/YARN-11394 > Project: Hadoop YARN > Issue Type: Improvement > Components: resourcemanager >Affects Versions: 3.4.0 >Reporter: Shilun Fan >Assignee: Shilun Fan >Priority: Major > Labels: pull-request-available > > When I finished the pr of YARN-11350, I encountered java doc error, this jira > will fix the java doc error in hadoop-yarn-server-resourcemanager. > > The java doc error report is as follows: > https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5131/22/artifact/out/branch-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdkUbuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04.txt -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11394) Fix hadoop-yarn-server-resourcemanager module Java Doc Errors.
[ https://issues.apache.org/jira/browse/YARN-11394?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17676396#comment-17676396 ] ASF GitHub Bot commented on YARN-11394: --- slfan1989 commented on PR #5288: URL: https://github.com/apache/hadoop/pull/5288#issuecomment-1381230036 @ayushtkn @goiri Can you help review this PR? Thank you very much! We use JDK11 to compile RM, java doc error will cause some PRs to fail the yetus test. RM Javadoc Error will appear in the following PRs: [YARN-11218. [Federation] Add getActivities, getBulkActivities REST APIs for Router.](https://github.com/apache/hadoop/pull/5284#top) [YARN-11320. [Federation] Add getSchedulerInfo REST APIs for Router.](https://github.com/apache/hadoop/pull/5217#top) [YARN-11326. [Federation] Add RM FederationStateStoreService Metrics.](https://github.com/apache/hadoop/pull/4963#top) In this pr, I fixed the problem of java doc, which can already be compiled by yetus. The modified compilation results are as follows: https://user-images.githubusercontent.com/55643692/212221189-a29c2c33-f969-468f-a5f7-7e350dd4483b.png";> > Fix hadoop-yarn-server-resourcemanager module Java Doc Errors. > -- > > Key: YARN-11394 > URL: https://issues.apache.org/jira/browse/YARN-11394 > Project: Hadoop YARN > Issue Type: Improvement > Components: resourcemanager >Affects Versions: 3.4.0 >Reporter: Shilun Fan >Assignee: Shilun Fan >Priority: Major > Labels: pull-request-available > > When I finished the pr of YARN-11350, I encountered java doc error, this jira > will fix the java doc error in hadoop-yarn-server-resourcemanager. > > The java doc error report is as follows: > https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5131/22/artifact/out/branch-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdkUbuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04.txt -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11374) [Federation] Support refreshSuperUserGroupsConfiguration、refreshUserToGroupsMappings API's for Federation
[ https://issues.apache.org/jira/browse/YARN-11374?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17676392#comment-17676392 ] ASF GitHub Bot commented on YARN-11374: --- slfan1989 commented on PR #5193: URL: https://github.com/apache/hadoop/pull/5193#issuecomment-1381192365 @goiri Thank you very much for helping to review the code! > [Federation] Support > refreshSuperUserGroupsConfiguration、refreshUserToGroupsMappings API's for > Federation > - > > Key: YARN-11374 > URL: https://issues.apache.org/jira/browse/YARN-11374 > Project: Hadoop YARN > Issue Type: Sub-task > Components: federation, router >Affects Versions: 3.4.0 >Reporter: Shilun Fan >Assignee: Shilun Fan >Priority: Major > Labels: pull-request-available > Fix For: 3.4.0 > > > In this jira, we will complete the refreshSuperUserGroupsConfiguration and > refreshUserToGroupsMappings methods of FederationRMAdminInterceptor. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11374) [Federation] Support refreshSuperUserGroupsConfiguration、refreshUserToGroupsMappings API's for Federation
[ https://issues.apache.org/jira/browse/YARN-11374?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17676365#comment-17676365 ] ASF GitHub Bot commented on YARN-11374: --- goiri merged PR #5193: URL: https://github.com/apache/hadoop/pull/5193 > [Federation] Support > refreshSuperUserGroupsConfiguration、refreshUserToGroupsMappings API's for > Federation > - > > Key: YARN-11374 > URL: https://issues.apache.org/jira/browse/YARN-11374 > Project: Hadoop YARN > Issue Type: Sub-task > Components: federation, router >Affects Versions: 3.4.0 >Reporter: Shilun Fan >Assignee: Shilun Fan >Priority: Major > Labels: pull-request-available > > In this jira, we will complete the refreshSuperUserGroupsConfiguration and > refreshUserToGroupsMappings methods of FederationRMAdminInterceptor. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Resolved] (YARN-11374) [Federation] Support refreshSuperUserGroupsConfiguration、refreshUserToGroupsMappings API's for Federation
[ https://issues.apache.org/jira/browse/YARN-11374?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Íñigo Goiri resolved YARN-11374. Fix Version/s: 3.4.0 Hadoop Flags: Reviewed Resolution: Fixed > [Federation] Support > refreshSuperUserGroupsConfiguration、refreshUserToGroupsMappings API's for > Federation > - > > Key: YARN-11374 > URL: https://issues.apache.org/jira/browse/YARN-11374 > Project: Hadoop YARN > Issue Type: Sub-task > Components: federation, router >Affects Versions: 3.4.0 >Reporter: Shilun Fan >Assignee: Shilun Fan >Priority: Major > Labels: pull-request-available > Fix For: 3.4.0 > > > In this jira, we will complete the refreshSuperUserGroupsConfiguration and > refreshUserToGroupsMappings methods of FederationRMAdminInterceptor. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11404) Add junit5 dependency to hadoop-mapreduce-client-app to fix few unit test failure
[
https://issues.apache.org/jira/browse/YARN-11404?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17676321#comment-17676321
]
ASF GitHub Bot commented on YARN-11404:
---
susheel-gupta commented on code in PR #5295:
URL: https://github.com/apache/hadoop/pull/5295#discussion_r1068597901
##
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapred/TestTaskAttemptListenerImpl.java:
##
@@ -257,12 +260,6 @@ public void testGetMapCompletionEvents() throws
IOException {
createTce(3, false, TaskAttemptCompletionEventStatus.FAILED) };
TaskAttemptCompletionEvent[] mapEvents = { taskEvents[0], taskEvents[2] };
Job mockJob = mock(Job.class);
-when(mockJob.getTaskAttemptCompletionEvents(0, 100))
- .thenReturn(taskEvents);
-when(mockJob.getTaskAttemptCompletionEvents(0, 2))
- .thenReturn(Arrays.copyOfRange(taskEvents, 0, 2));
-when(mockJob.getTaskAttemptCompletionEvents(2, 100))
- .thenReturn(Arrays.copyOfRange(taskEvents, 2, 4));
Review Comment:
@szilard-nemeth Encountered with this error:
```
org.mockito.exceptions.misusing.UnnecessaryStubbingException:
Unnecessary stubbings detected.
Clean & maintainable test code requires zero unnecessary code.
Following stubbings are unnecessary (click to navigate to relevant line of
code):
1. -> at
org.apache.hadoop.mapred.TestTaskAttemptListenerImpl.lambda$testGetMapCompletionEvents$2(TestTaskAttemptListenerImpl.java:267)
2. -> at
org.apache.hadoop.mapred.TestTaskAttemptListenerImpl.lambda$testGetMapCompletionEvents$2(TestTaskAttemptListenerImpl.java:269)
3. -> at
org.apache.hadoop.mapred.TestTaskAttemptListenerImpl.lambda$testGetMapCompletionEvents$2(TestTaskAttemptListenerImpl.java:271)
```
It was a unnecessary stubbed call that was never realized during test
execution.So either I have to remove it or use use 'lenient' strictness like
this:
```
lenient().when(mockJob.getTaskAttemptCompletionEvents(0, 100))
.thenReturn(taskEvents);
lenient().when(mockJob.getTaskAttemptCompletionEvents(0, 2))
.thenReturn(Arrays.copyOfRange(taskEvents, 0, 2));
lenient().when(mockJob.getTaskAttemptCompletionEvents(2, 100))
.thenReturn(Arrays.copyOfRange(taskEvents, 2, 4));
```
I think it is better to use 'lenient' strictness
> Add junit5 dependency to hadoop-mapreduce-client-app to fix few unit test
> failure
> -
>
> Key: YARN-11404
> URL: https://issues.apache.org/jira/browse/YARN-11404
> Project: Hadoop YARN
> Issue Type: Bug
> Components: yarn
>Reporter: Susheel Gupta
>Assignee: Susheel Gupta
>Priority: Major
> Labels: pull-request-available
> Attachments:
> patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app.txt
>
>
> We need to add Junit 5 dependency in
> {code:java}
> /hadoop/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/pom.xml{code}
> as the testcase TestAMWebServicesJobConf, TestAMWebServicesJobs,
> TestAMWebServices, TestAMWebServicesAttempts, TestAMWebServicesTasks were
> passing locally but failed at jenkins build in this
> [link|https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5119/7/artifact/out/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app.txt]
> for YARN-5607
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11404) Add junit5 dependency to hadoop-mapreduce-client-app to fix few unit test failure
[
https://issues.apache.org/jira/browse/YARN-11404?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17676320#comment-17676320
]
ASF GitHub Bot commented on YARN-11404:
---
susheel-gupta commented on code in PR #5295:
URL: https://github.com/apache/hadoop/pull/5295#discussion_r1068597901
##
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapred/TestTaskAttemptListenerImpl.java:
##
@@ -257,12 +260,6 @@ public void testGetMapCompletionEvents() throws
IOException {
createTce(3, false, TaskAttemptCompletionEventStatus.FAILED) };
TaskAttemptCompletionEvent[] mapEvents = { taskEvents[0], taskEvents[2] };
Job mockJob = mock(Job.class);
-when(mockJob.getTaskAttemptCompletionEvents(0, 100))
- .thenReturn(taskEvents);
-when(mockJob.getTaskAttemptCompletionEvents(0, 2))
- .thenReturn(Arrays.copyOfRange(taskEvents, 0, 2));
-when(mockJob.getTaskAttemptCompletionEvents(2, 100))
- .thenReturn(Arrays.copyOfRange(taskEvents, 2, 4));
Review Comment:
Encountered with this error:
```
org.mockito.exceptions.misusing.UnnecessaryStubbingException:
Unnecessary stubbings detected.
Clean & maintainable test code requires zero unnecessary code.
Following stubbings are unnecessary (click to navigate to relevant line of
code):
1. -> at
org.apache.hadoop.mapred.TestTaskAttemptListenerImpl.lambda$testGetMapCompletionEvents$2(TestTaskAttemptListenerImpl.java:267)
2. -> at
org.apache.hadoop.mapred.TestTaskAttemptListenerImpl.lambda$testGetMapCompletionEvents$2(TestTaskAttemptListenerImpl.java:269)
3. -> at
org.apache.hadoop.mapred.TestTaskAttemptListenerImpl.lambda$testGetMapCompletionEvents$2(TestTaskAttemptListenerImpl.java:271)
```
It was a unnecessary stubbed call that was never realized during test
execution.So either I have to remove it or use use 'lenient' strictness like
this:
```
lenient().when(mockJob.getTaskAttemptCompletionEvents(0, 100))
.thenReturn(taskEvents);
lenient().when(mockJob.getTaskAttemptCompletionEvents(0, 2))
.thenReturn(Arrays.copyOfRange(taskEvents, 0, 2));
lenient().when(mockJob.getTaskAttemptCompletionEvents(2, 100))
.thenReturn(Arrays.copyOfRange(taskEvents, 2, 4));
```
I think it is better to use 'lenient' strictness
> Add junit5 dependency to hadoop-mapreduce-client-app to fix few unit test
> failure
> -
>
> Key: YARN-11404
> URL: https://issues.apache.org/jira/browse/YARN-11404
> Project: Hadoop YARN
> Issue Type: Bug
> Components: yarn
>Reporter: Susheel Gupta
>Assignee: Susheel Gupta
>Priority: Major
> Labels: pull-request-available
> Attachments:
> patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app.txt
>
>
> We need to add Junit 5 dependency in
> {code:java}
> /hadoop/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/pom.xml{code}
> as the testcase TestAMWebServicesJobConf, TestAMWebServicesJobs,
> TestAMWebServices, TestAMWebServicesAttempts, TestAMWebServicesTasks were
> passing locally but failed at jenkins build in this
> [link|https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5119/7/artifact/out/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app.txt]
> for YARN-5607
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11374) [Federation] Support refreshSuperUserGroupsConfiguration、refreshUserToGroupsMappings API's for Federation
[
https://issues.apache.org/jira/browse/YARN-11374?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17676251#comment-17676251
]
ASF GitHub Bot commented on YARN-11374:
---
hadoop-yetus commented on PR #5193:
URL: https://github.com/apache/hadoop/pull/5193#issuecomment-1380696109
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 51s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available.
|
| +0 :ok: | buf | 0m 1s | | buf was not available. |
| +0 :ok: | buf | 0m 1s | | buf was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 1 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 15m 23s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 33m 36s | | trunk passed |
| +1 :green_heart: | compile | 10m 32s | | trunk passed with JDK
Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04 |
| +1 :green_heart: | compile | 9m 0s | | trunk passed with JDK
Private Build-1.8.0_352-8u352-ga-1~20.04-b08 |
| +1 :green_heart: | checkstyle | 1m 45s | | trunk passed |
| +1 :green_heart: | mvnsite | 2m 25s | | trunk passed |
| +1 :green_heart: | javadoc | 2m 19s | | trunk passed with JDK
Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04 |
| +1 :green_heart: | javadoc | 2m 3s | | trunk passed with JDK
Private Build-1.8.0_352-8u352-ga-1~20.04-b08 |
| +1 :green_heart: | spotbugs | 4m 53s | | trunk passed |
| +1 :green_heart: | shadedclient | 26m 1s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 23s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 1m 52s | | the patch passed |
| +1 :green_heart: | compile | 9m 50s | | the patch passed with JDK
Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04 |
| +1 :green_heart: | cc | 9m 50s | | the patch passed |
| +1 :green_heart: | javac | 9m 50s | | the patch passed |
| +1 :green_heart: | compile | 8m 54s | | the patch passed with JDK
Private Build-1.8.0_352-8u352-ga-1~20.04-b08 |
| +1 :green_heart: | cc | 8m 54s | | the patch passed |
| +1 :green_heart: | javac | 8m 54s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 1m 35s | |
hadoop-yarn-project/hadoop-yarn: The patch generated 0 new + 0 unchanged - 13
fixed = 0 total (was 13) |
| +1 :green_heart: | mvnsite | 2m 12s | | the patch passed |
| +1 :green_heart: | javadoc | 2m 0s | | the patch passed with JDK
Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04 |
| +1 :green_heart: | javadoc | 1m 54s | | the patch passed with JDK
Private Build-1.8.0_352-8u352-ga-1~20.04-b08 |
| +1 :green_heart: | spotbugs | 5m 10s | | the patch passed |
| +1 :green_heart: | shadedclient | 26m 47s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 1m 3s | | hadoop-yarn-api in the patch
passed. |
| +1 :green_heart: | unit | 5m 23s | | hadoop-yarn-common in the patch
passed. |
| +1 :green_heart: | unit | 0m 34s | | hadoop-yarn-server-router in
the patch passed. |
| +1 :green_heart: | asflicense | 0m 47s | | The patch does not
generate ASF License warnings. |
| | | 180m 34s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5193/16/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/5193 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets cc buflint
bufcompat |
| uname | Linux c7f48f048794 4.15.0-197-generic #208-Ubuntu SMP Tue Nov 1
17:23:37 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 0f8f5fdc6d424ce19e460f834976d5d85ec9aaa4 |
| Default Java | Private Build-1.8.0_352-8u352-ga-1~20.04-b08 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.
[jira] [Updated] (YARN-11404) Add junit5 dependency to hadoop-mapreduce-client-app to fix few unit test failure
[
https://issues.apache.org/jira/browse/YARN-11404?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated YARN-11404:
--
Labels: pull-request-available (was: )
> Add junit5 dependency to hadoop-mapreduce-client-app to fix few unit test
> failure
> -
>
> Key: YARN-11404
> URL: https://issues.apache.org/jira/browse/YARN-11404
> Project: Hadoop YARN
> Issue Type: Bug
> Components: yarn
>Reporter: Susheel Gupta
>Assignee: Susheel Gupta
>Priority: Major
> Labels: pull-request-available
> Attachments:
> patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app.txt
>
>
> We need to add Junit 5 dependency in
> {code:java}
> /hadoop/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/pom.xml{code}
> as the testcase TestAMWebServicesJobConf, TestAMWebServicesJobs,
> TestAMWebServices, TestAMWebServicesAttempts, TestAMWebServicesTasks were
> passing locally but failed at jenkins build in this
> [link|https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5119/7/artifact/out/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app.txt]
> for YARN-5607
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11404) Add junit5 dependency to hadoop-mapreduce-client-app to fix few unit test failure
[
https://issues.apache.org/jira/browse/YARN-11404?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17676043#comment-17676043
]
ASF GitHub Bot commented on YARN-11404:
---
szilard-nemeth commented on code in PR #5295:
URL: https://github.com/apache/hadoop/pull/5295#discussion_r1068133882
##
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapred/TestTaskAttemptListenerImpl.java:
##
@@ -257,12 +260,6 @@ public void testGetMapCompletionEvents() throws
IOException {
createTce(3, false, TaskAttemptCompletionEventStatus.FAILED) };
TaskAttemptCompletionEvent[] mapEvents = { taskEvents[0], taskEvents[2] };
Job mockJob = mock(Job.class);
-when(mockJob.getTaskAttemptCompletionEvents(0, 100))
- .thenReturn(taskEvents);
-when(mockJob.getTaskAttemptCompletionEvents(0, 2))
- .thenReturn(Arrays.copyOfRange(taskEvents, 0, 2));
-when(mockJob.getTaskAttemptCompletionEvents(2, 100))
- .thenReturn(Arrays.copyOfRange(taskEvents, 2, 4));
Review Comment:
Why this code block was removed?
##
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapred/TestTaskAttemptListenerImpl.java:
##
@@ -353,7 +351,7 @@ protected void registerHeartbeatHandler(Configuration conf)
{
}
@Test
- public void testCheckpointIDTracking()
+ void testCheckpointIDTracking()
Review Comment:
What's the point of removing public modifiers from test methods?
This only adds noise and is not strictly necessary for the JUnit upgrade.
Please undo all unnecessary methodmodifier changes.
> Add junit5 dependency to hadoop-mapreduce-client-app to fix few unit test
> failure
> -
>
> Key: YARN-11404
> URL: https://issues.apache.org/jira/browse/YARN-11404
> Project: Hadoop YARN
> Issue Type: Bug
> Components: yarn
>Reporter: Susheel Gupta
>Assignee: Susheel Gupta
>Priority: Major
> Attachments:
> patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app.txt
>
>
> We need to add Junit 5 dependency in
> {code:java}
> /hadoop/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/pom.xml{code}
> as the testcase TestAMWebServicesJobConf, TestAMWebServicesJobs,
> TestAMWebServices, TestAMWebServicesAttempts, TestAMWebServicesTasks were
> passing locally but failed at jenkins build in this
> [link|https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5119/7/artifact/out/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app.txt]
> for YARN-5607
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11413) Fix Junit Test ERROR Introduced By YARN-6412
[ https://issues.apache.org/jira/browse/YARN-11413?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17676022#comment-17676022 ] ASF GitHub Bot commented on YARN-11413: --- slfan1989 commented on PR #5289: URL: https://github.com/apache/hadoop/pull/5289#issuecomment-1380366074 @brumi1024 Thank you very much for helping to review the code! > Fix Junit Test ERROR Introduced By YARN-6412 > > > Key: YARN-11413 > URL: https://issues.apache.org/jira/browse/YARN-11413 > Project: Hadoop YARN > Issue Type: Bug > Components: api >Affects Versions: 3.4.0 >Reporter: Shilun Fan >Assignee: Shilun Fan >Priority: Major > Labels: pull-request-available > > YARN-6412 (#5242) caused > TestCommonConfigurationFields#testCompareXmlAgainstConfigurationClass to be > wrong, I will fix it -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Resolved] (YARN-11413) Fix Junit Test ERROR Introduced By YARN-6412
[ https://issues.apache.org/jira/browse/YARN-11413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Benjamin Teke resolved YARN-11413. -- Hadoop Flags: Reviewed Resolution: Fixed > Fix Junit Test ERROR Introduced By YARN-6412 > > > Key: YARN-11413 > URL: https://issues.apache.org/jira/browse/YARN-11413 > Project: Hadoop YARN > Issue Type: Bug > Components: api >Affects Versions: 3.4.0 >Reporter: Shilun Fan >Assignee: Shilun Fan >Priority: Major > Labels: pull-request-available > > YARN-6412 (#5242) caused > TestCommonConfigurationFields#testCompareXmlAgainstConfigurationClass to be > wrong, I will fix it -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11413) Fix Junit Test ERROR Introduced By YARN-6412
[ https://issues.apache.org/jira/browse/YARN-11413?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17676018#comment-17676018 ] ASF GitHub Bot commented on YARN-11413: --- brumi1024 commented on PR #5289: URL: https://github.com/apache/hadoop/pull/5289#issuecomment-1380358210 @slfan1989 merged to trunk. > Fix Junit Test ERROR Introduced By YARN-6412 > > > Key: YARN-11413 > URL: https://issues.apache.org/jira/browse/YARN-11413 > Project: Hadoop YARN > Issue Type: Bug > Components: api >Affects Versions: 3.4.0 >Reporter: Shilun Fan >Assignee: Shilun Fan >Priority: Major > Labels: pull-request-available > > YARN-6412 (#5242) caused > TestCommonConfigurationFields#testCompareXmlAgainstConfigurationClass to be > wrong, I will fix it -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11413) Fix Junit Test ERROR Introduced By YARN-6412
[ https://issues.apache.org/jira/browse/YARN-11413?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17676016#comment-17676016 ] ASF GitHub Bot commented on YARN-11413: --- brumi1024 merged PR #5289: URL: https://github.com/apache/hadoop/pull/5289 > Fix Junit Test ERROR Introduced By YARN-6412 > > > Key: YARN-11413 > URL: https://issues.apache.org/jira/browse/YARN-11413 > Project: Hadoop YARN > Issue Type: Bug > Components: api >Affects Versions: 3.4.0 >Reporter: Shilun Fan >Assignee: Shilun Fan >Priority: Major > Labels: pull-request-available > > YARN-6412 (#5242) caused > TestCommonConfigurationFields#testCompareXmlAgainstConfigurationClass to be > wrong, I will fix it -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11413) Fix Junit Test ERROR Introduced By YARN-6412
[ https://issues.apache.org/jira/browse/YARN-11413?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17676008#comment-17676008 ] ASF GitHub Bot commented on YARN-11413: --- slfan1989 commented on PR #5289: URL: https://github.com/apache/hadoop/pull/5289#issuecomment-1380350596 @brumi1024 Can you help to review this PR again? Thank you very much! > Fix Junit Test ERROR Introduced By YARN-6412 > > > Key: YARN-11413 > URL: https://issues.apache.org/jira/browse/YARN-11413 > Project: Hadoop YARN > Issue Type: Bug > Components: api >Affects Versions: 3.4.0 >Reporter: Shilun Fan >Assignee: Shilun Fan >Priority: Major > Labels: pull-request-available > > YARN-6412 (#5242) caused > TestCommonConfigurationFields#testCompareXmlAgainstConfigurationClass to be > wrong, I will fix it -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-11404) Add junit5 dependency to hadoop-mapreduce-client-app to fix few unit test failure
[
https://issues.apache.org/jira/browse/YARN-11404?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Szilard Nemeth updated YARN-11404:
--
Attachment:
patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app.txt
> Add junit5 dependency to hadoop-mapreduce-client-app to fix few unit test
> failure
> -
>
> Key: YARN-11404
> URL: https://issues.apache.org/jira/browse/YARN-11404
> Project: Hadoop YARN
> Issue Type: Bug
> Components: yarn
>Reporter: Susheel Gupta
>Assignee: Susheel Gupta
>Priority: Major
> Attachments:
> patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app.txt
>
>
> We need to add Junit 5 dependency in
> {code:java}
> /hadoop/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/pom.xml{code}
> as the testcase TestAMWebServicesJobConf, TestAMWebServicesJobs,
> TestAMWebServices, TestAMWebServicesAttempts, TestAMWebServicesTasks were
> passing locally but failed at jenkins build in this
> [link|https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5119/7/artifact/out/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app.txt]
> for YARN-5607
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11413) Fix Junit Test ERROR Introduced By YARN-6412
[
https://issues.apache.org/jira/browse/YARN-11413?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17675979#comment-17675979
]
ASF GitHub Bot commented on YARN-11413:
---
hadoop-yetus commented on PR #5289:
URL: https://github.com/apache/hadoop/pull/5289#issuecomment-1380247866
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 59s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 3 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 15m 22s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 31m 24s | | trunk passed |
| +1 :green_heart: | compile | 23m 15s | | trunk passed with JDK
Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04 |
| +1 :green_heart: | compile | 20m 25s | | trunk passed with JDK
Private Build-1.8.0_352-8u352-ga-1~20.04-b08 |
| +1 :green_heart: | checkstyle | 3m 47s | | trunk passed |
| +1 :green_heart: | mvnsite | 2m 43s | | trunk passed |
| -1 :x: | javadoc | 1m 15s |
[/branch-javadoc-hadoop-common-project_hadoop-common-jdkUbuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5289/3/artifact/out/branch-javadoc-hadoop-common-project_hadoop-common-jdkUbuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04.txt)
| hadoop-common in trunk failed with JDK
Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04. |
| +1 :green_heart: | javadoc | 1m 41s | | trunk passed with JDK
Private Build-1.8.0_352-8u352-ga-1~20.04-b08 |
| +1 :green_heart: | spotbugs | 4m 46s | | trunk passed |
| +1 :green_heart: | shadedclient | 24m 52s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 29s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 1m 49s | | the patch passed |
| +1 :green_heart: | compile | 22m 28s | | the patch passed with JDK
Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04 |
| +1 :green_heart: | javac | 22m 28s | | the patch passed |
| +1 :green_heart: | compile | 20m 47s | | the patch passed with JDK
Private Build-1.8.0_352-8u352-ga-1~20.04-b08 |
| +1 :green_heart: | javac | 20m 47s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 3m 34s | | the patch passed |
| +1 :green_heart: | mvnsite | 2m 45s | | the patch passed |
| -1 :x: | javadoc | 1m 7s |
[/patch-javadoc-hadoop-common-project_hadoop-common-jdkUbuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5289/3/artifact/out/patch-javadoc-hadoop-common-project_hadoop-common-jdkUbuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04.txt)
| hadoop-common in the patch failed with JDK
Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04. |
| +1 :green_heart: | javadoc | 1m 41s | | the patch passed with JDK
Private Build-1.8.0_352-8u352-ga-1~20.04-b08 |
| +1 :green_heart: | spotbugs | 4m 58s | | the patch passed |
| +1 :green_heart: | shadedclient | 25m 3s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 18m 25s | | hadoop-common in the patch
passed. |
| +1 :green_heart: | unit | 1m 16s | | hadoop-yarn-api in the patch
passed. |
| +1 :green_heart: | asflicense | 1m 2s | | The patch does not
generate ASF License warnings. |
| | | 239m 47s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5289/3/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/5289 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux e9d03bba0c2e 4.15.0-200-generic #211-Ubuntu SMP Thu Nov 24
18:16:04 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / ae84b7567a17fde6cc9943a329ef8bcce09346b6 |
| Default Java | Private Bu
[jira] [Commented] (YARN-11394) Fix hadoop-yarn-server-resourcemanager module Java Doc Errors.
[
https://issues.apache.org/jira/browse/YARN-11394?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17675972#comment-17675972
]
ASF GitHub Bot commented on YARN-11394:
---
hadoop-yetus commented on PR #5288:
URL: https://github.com/apache/hadoop/pull/5288#issuecomment-1380227078
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 51s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 2s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| -1 :x: | test4tests | 0m 0s | | The patch doesn't appear to include
any new or modified tests. Please justify why no new tests are needed for this
patch. Also please list what manual steps were performed to verify this patch.
|
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 47m 0s | | trunk passed |
| +1 :green_heart: | compile | 1m 5s | | trunk passed with JDK
Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04 |
| +1 :green_heart: | compile | 0m 55s | | trunk passed with JDK
Private Build-1.8.0_352-8u352-ga-1~20.04-b08 |
| +1 :green_heart: | checkstyle | 0m 55s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 1s | | trunk passed |
| -1 :x: | javadoc | 0m 54s |
[/branch-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdkUbuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5288/5/artifact/out/branch-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdkUbuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04.txt)
| hadoop-yarn-server-resourcemanager in trunk failed with JDK
Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04. |
| +1 :green_heart: | javadoc | 0m 42s | | trunk passed with JDK
Private Build-1.8.0_352-8u352-ga-1~20.04-b08 |
| +1 :green_heart: | spotbugs | 2m 9s | | trunk passed |
| +1 :green_heart: | shadedclient | 27m 35s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 59s | | the patch passed |
| +1 :green_heart: | compile | 1m 0s | | the patch passed with JDK
Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04 |
| +1 :green_heart: | javac | 1m 1s | | the patch passed |
| +1 :green_heart: | compile | 0m 50s | | the patch passed with JDK
Private Build-1.8.0_352-8u352-ga-1~20.04-b08 |
| +1 :green_heart: | javac | 0m 50s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 0m 44s | |
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager:
The patch generated 0 new + 702 unchanged - 51 fixed = 702 total (was 753) |
| +1 :green_heart: | mvnsite | 0m 52s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 39s | |
hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdkUbuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04
with JDK Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04 generated 0 new + 1
unchanged - 100 fixed = 1 total (was 101) |
| +1 :green_heart: | javadoc | 0m 34s | |
hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdkPrivateBuild-1.8.0_352-8u352-ga-1~20.04-b08
with JDK Private Build-1.8.0_352-8u352-ga-1~20.04-b08 generated 0 new + 1
unchanged - 342 fixed = 1 total (was 343) |
| +1 :green_heart: | spotbugs | 2m 2s | | the patch passed |
| +1 :green_heart: | shadedclient | 27m 25s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 102m 48s | |
hadoop-yarn-server-resourcemanager in the patch passed. |
| +1 :green_heart: | asflicense | 0m 33s | | The patch does not
generate ASF License warnings. |
| | | 220m 50s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5288/5/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/5288 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux bd5a1592cc27 4.15.0-200-generic #211-Ubuntu SMP Thu
[jira] [Resolved] (YARN-11190) CS Mapping rule bug: User matcher does not work correctly for usernames with dot
[
https://issues.apache.org/jira/browse/YARN-11190?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Benjamin Teke resolved YARN-11190.
--
Fix Version/s: 3.4.0
Hadoop Flags: Reviewed
Resolution: Fixed
> CS Mapping rule bug: User matcher does not work correctly for usernames with
> dot
>
>
> Key: YARN-11190
> URL: https://issues.apache.org/jira/browse/YARN-11190
> Project: Hadoop YARN
> Issue Type: Bug
> Components: capacity scheduler
>Reporter: Szilard Nemeth
>Assignee: Szilard Nemeth
>Priority: Major
> Labels: pull-request-available
> Fix For: 3.4.0
>
> Attachments: testUserNameSetDefaultAndPlaceWith2Rules.log,
> testUserNameSetDefaultAndPlaceWith2RulesUsernameReplacedWithDot.log,
> testcases.patch
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Given the following scenario, the placement engine does not work as expected.
> A user with a '.' (dot) inside his/her username submits a job.
> Let the username be "test.user"
> There are 2 mapping rules:
> 1. The matcher matches the user with name "test.user" and has an associated
> mapping rule action that sets the default queue to "root.user".
> 2. The second mapping rule matches the same user ("test.user") and places the
> application to the default queue.
> *Expactation:*
> When the user with username "root.user" submits a job, the application will
> be placed to queue "root.user".
> *Observed behaviour:*
> The application is placed to test_dot_user.
> This means that the dot is replaced to "{_}dot{_}" too early so that the
> default queue is set incorrectly.
>
> I have attached a patch file that demonstrates this behaviour with 2 new
> testcases along with the logs of these testcases.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11190) CS Mapping rule bug: User matcher does not work correctly for usernames with dot
[
https://issues.apache.org/jira/browse/YARN-11190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17675971#comment-17675971
]
ASF GitHub Bot commented on YARN-11190:
---
brumi1024 commented on PR #4471:
URL: https://github.com/apache/hadoop/pull/4471#issuecomment-1380225930
Nvm, YARN-10376 was only introduced in 3.4.0, so no backport is needed.
> CS Mapping rule bug: User matcher does not work correctly for usernames with
> dot
>
>
> Key: YARN-11190
> URL: https://issues.apache.org/jira/browse/YARN-11190
> Project: Hadoop YARN
> Issue Type: Bug
> Components: capacity scheduler
>Reporter: Szilard Nemeth
>Assignee: Szilard Nemeth
>Priority: Major
> Labels: pull-request-available
> Attachments: testUserNameSetDefaultAndPlaceWith2Rules.log,
> testUserNameSetDefaultAndPlaceWith2RulesUsernameReplacedWithDot.log,
> testcases.patch
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Given the following scenario, the placement engine does not work as expected.
> A user with a '.' (dot) inside his/her username submits a job.
> Let the username be "test.user"
> There are 2 mapping rules:
> 1. The matcher matches the user with name "test.user" and has an associated
> mapping rule action that sets the default queue to "root.user".
> 2. The second mapping rule matches the same user ("test.user") and places the
> application to the default queue.
> *Expactation:*
> When the user with username "root.user" submits a job, the application will
> be placed to queue "root.user".
> *Observed behaviour:*
> The application is placed to test_dot_user.
> This means that the dot is replaced to "{_}dot{_}" too early so that the
> default queue is set incorrectly.
>
> I have attached a patch file that demonstrates this behaviour with 2 new
> testcases along with the logs of these testcases.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
