[GitHub] [hadoop] hadoop-yetus commented on pull request #6116: YARN-11548. [Federation] Router Supports Format FederationStateStore.
hadoop-yetus commented on PR #6116:
URL: https://github.com/apache/hadoop/pull/6116#issuecomment-1734796810
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 40s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| -1 :x: | test4tests | 0m 0s | | The patch doesn't appear to include
any new or modified tests. Please justify why no new tests are needed for this
patch. Also please list what manual steps were performed to verify this patch.
|
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 15m 41s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 33m 57s | | trunk passed |
| +1 :green_heart: | compile | 2m 35s | | trunk passed with JDK
Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04 |
| +1 :green_heart: | compile | 2m 32s | | trunk passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | checkstyle | 1m 28s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 17s | | trunk passed |
| +1 :green_heart: | javadoc | 1m 19s | | trunk passed with JDK
Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04 |
| +1 :green_heart: | javadoc | 1m 9s | | trunk passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | spotbugs | 2m 28s | | trunk passed |
| +1 :green_heart: | shadedclient | 36m 23s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 31s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 0m 57s | | the patch passed |
| -1 :x: | compile | 2m 4s |
[/patch-compile-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server-jdkUbuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6116/1/artifact/out/patch-compile-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server-jdkUbuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04.txt)
| hadoop-yarn-server in the patch failed with JDK
Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04. |
| -1 :x: | javac | 2m 4s |
[/patch-compile-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server-jdkUbuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6116/1/artifact/out/patch-compile-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server-jdkUbuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04.txt)
| hadoop-yarn-server in the patch failed with JDK
Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04. |
| -1 :x: | compile | 1m 53s |
[/patch-compile-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server-jdkPrivateBuild-1.8.0_382-8u382-ga-1~20.04.1-b05.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6116/1/artifact/out/patch-compile-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server-jdkPrivateBuild-1.8.0_382-8u382-ga-1~20.04.1-b05.txt)
| hadoop-yarn-server in the patch failed with JDK Private
Build-1.8.0_382-8u382-ga-1~20.04.1-b05. |
| -1 :x: | javac | 1m 53s |
[/patch-compile-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server-jdkPrivateBuild-1.8.0_382-8u382-ga-1~20.04.1-b05.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6116/1/artifact/out/patch-compile-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server-jdkPrivateBuild-1.8.0_382-8u382-ga-1~20.04.1-b05.txt)
| hadoop-yarn-server in the patch failed with JDK Private
Build-1.8.0_382-8u382-ga-1~20.04.1-b05. |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 1m 13s | | the patch passed |
| +1 :green_heart: | mvnsite | 1m 1s | | the patch passed |
| +1 :green_heart: | javadoc | 1m 0s | | the patch passed with JDK
Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04 |
| +1 :green_heart: | javadoc | 0m 55s | | the patch passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | spotbugs | 2m 36s | | the patch passed |
| -1 :x: | shadedclient | 9m 53s | | patch has errors when building
and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 3m 30s | | hadoop-yarn-server-common in
the patch passed. |
| +1 :green_heart: | unit | 0m 31s | | hadoop-yarn-server-router in
the patch passed. |
| +1 :green_heart: | asflicense | 0m 35s | | The patch does not
generate ASF License warnings. |
| | | 131m 19s
[GitHub] [hadoop] hadoop-yetus commented on pull request #6117: YARN-11576. Improve FederationInterceptorREST AuditLog.
hadoop-yetus commented on PR #6117:
URL: https://github.com/apache/hadoop/pull/6117#issuecomment-1734796210
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 52s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| -1 :x: | test4tests | 0m 0s | | The patch doesn't appear to include
any new or modified tests. Please justify why no new tests are needed for this
patch. Also please list what manual steps were performed to verify this patch.
|
_ trunk Compile Tests _ |
| -1 :x: | mvninstall | 17m 17s |
[/branch-mvninstall-root.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6117/1/artifact/out/branch-mvninstall-root.txt)
| root in trunk failed. |
| +1 :green_heart: | compile | 0m 32s | | trunk passed with JDK
Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04 |
| +1 :green_heart: | compile | 0m 25s | | trunk passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | checkstyle | 0m 25s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 30s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 34s | | trunk passed with JDK
Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04 |
| +1 :green_heart: | javadoc | 0m 25s | | trunk passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | spotbugs | 0m 57s | | trunk passed |
| +1 :green_heart: | shadedclient | 38m 33s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 23s | | the patch passed |
| +1 :green_heart: | compile | 0m 23s | | the patch passed with JDK
Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04 |
| +1 :green_heart: | javac | 0m 23s | | the patch passed |
| +1 :green_heart: | compile | 0m 20s | | the patch passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | javac | 0m 20s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 0m 15s | | the patch passed |
| +1 :green_heart: | mvnsite | 0m 23s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 22s | | the patch passed with JDK
Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04 |
| +1 :green_heart: | javadoc | 0m 20s | | the patch passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | spotbugs | 0m 55s | | the patch passed |
| +1 :green_heart: | shadedclient | 38m 8s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 0m 27s | | hadoop-yarn-server-router in
the patch passed. |
| +1 :green_heart: | asflicense | 0m 35s | | The patch does not
generate ASF License warnings. |
| | | 107m 29s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6117/1/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/6117 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux 3898848f8194 4.15.0-213-generic #224-Ubuntu SMP Mon Jun 19
13:30:12 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 84d89feee2c4437bd0ae1ee807273b1d704a54d3 |
| Default Java | Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6117/1/testReport/ |
| Max. process+thread count | 631 (vs. ulimit of 5500) |
| modules | C:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router U:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6117/1/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This
[GitHub] [hadoop] hadoop-yetus commented on pull request #6083: YARN-8862. [BackPort] [GPG] Add Yarn Registry cleanup in ApplicationCleaner.
hadoop-yetus commented on PR #6083:
URL: https://github.com/apache/hadoop/pull/6083#issuecomment-1734775757
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 38s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 1s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +0 :ok: | xmllint | 0m 0s | | xmllint was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 3 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 15m 5s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 31m 33s | | trunk passed |
| +1 :green_heart: | compile | 2m 39s | | trunk passed with JDK
Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04 |
| +1 :green_heart: | compile | 2m 20s | | trunk passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | checkstyle | 1m 25s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 55s | | trunk passed |
| +1 :green_heart: | javadoc | 1m 56s | | trunk passed with JDK
Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04 |
| +1 :green_heart: | javadoc | 1m 47s | | trunk passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | spotbugs | 3m 12s | | trunk passed |
| +1 :green_heart: | shadedclient | 32m 51s | | branch has no errors
when building and testing our client artifacts. |
| -0 :warning: | patch | 33m 15s | | Used diff version of patch file.
Binary files and potentially other changes not applied. Please rebase and
squash commits if necessary. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 33s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 1m 20s | | the patch passed |
| +1 :green_heart: | compile | 2m 22s | | the patch passed with JDK
Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04 |
| +1 :green_heart: | javac | 2m 22s | | the patch passed |
| +1 :green_heart: | compile | 2m 15s | | the patch passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | javac | 2m 15s | | the patch passed |
| -1 :x: | blanks | 0m 0s |
[/blanks-eol.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6083/6/artifact/out/blanks-eol.txt)
| The patch has 1 line(s) that end in blanks. Use git apply --whitespace=fix
<>. Refer https://git-scm.com/docs/git-apply |
| +1 :green_heart: | checkstyle | 1m 13s | | the patch passed |
| +1 :green_heart: | mvnsite | 1m 30s | | the patch passed |
| +1 :green_heart: | javadoc | 1m 27s | | the patch passed with JDK
Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04 |
| +1 :green_heart: | javadoc | 1m 17s | | the patch passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | spotbugs | 3m 14s | | the patch passed |
| +1 :green_heart: | shadedclient | 33m 57s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 3m 25s | | hadoop-yarn-server-common in
the patch passed. |
| +1 :green_heart: | unit | 0m 33s | | hadoop-yarn-server-router in
the patch passed. |
| +1 :green_heart: | unit | 0m 55s | |
hadoop-yarn-server-globalpolicygenerator in the patch passed. |
| +1 :green_heart: | asflicense | 0m 42s | | The patch does not
generate ASF License warnings. |
| | | 156m 11s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6083/6/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/6083 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets xmllint |
| uname | Linux 8d02d85000d0 4.15.0-213-generic #224-Ubuntu SMP Mon Jun 19
13:30:12 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / be8816f0958ecef7e6b36c371c9f25f3b45ba631 |
| Default Java | Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
|
[GitHub] [hadoop] slfan1989 opened a new pull request, #6117: YARN-11576. Improve FederationInterceptorREST AuditLog.
slfan1989 opened a new pull request, #6117: URL: https://github.com/apache/hadoop/pull/6117 ### Description of PR JIRA: YARN-11576. Improve FederationInterceptorREST AuditLog. ### How was this patch tested? ### For code changes: - [ ] Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')? - [ ] Object storage: have the integration tests been executed and the endpoint declared according to the connector-specific documentation? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the `LICENSE`, `LICENSE-binary`, `NOTICE-binary` files? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] slfan1989 opened a new pull request, #6116: YARN-11548. [Federation] Router Supports Format FederationStateStore.
slfan1989 opened a new pull request, #6116: URL: https://github.com/apache/hadoop/pull/6116 ### Description of PR JIRA: YARN-11548. [Federation] Router Supports Format FederationStateStore. ### How was this patch tested? Add Junit Test. ### For code changes: - [ ] Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')? - [ ] Object storage: have the integration tests been executed and the endpoint declared according to the connector-specific documentation? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the `LICENSE`, `LICENSE-binary`, `NOTICE-binary` files? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] slfan1989 commented on pull request #6055: YARN-11547. [Federation] Router Supports Remove individual application records from FederationStateStore.
slfan1989 commented on PR #6055: URL: https://github.com/apache/hadoop/pull/6055#issuecomment-1734671947 @goiri Thank you very much for your help in reviewing the code! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #6114: YARN-11468. Zookeeper SSL/TLS support
hadoop-yetus commented on PR #6114:
URL: https://github.com/apache/hadoop/pull/6114#issuecomment-1734625684
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 28s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +0 :ok: | xmllint | 0m 0s | | xmllint was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| -1 :x: | test4tests | 0m 0s | | The patch doesn't appear to include
any new or modified tests. Please justify why no new tests are needed for this
patch. Also please list what manual steps were performed to verify this patch.
|
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 13m 57s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 21m 43s | | trunk passed |
| +1 :green_heart: | compile | 5m 41s | | trunk passed with JDK
Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04 |
| +1 :green_heart: | compile | 4m 48s | | trunk passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | checkstyle | 1m 25s | | trunk passed |
| +1 :green_heart: | mvnsite | 2m 30s | | trunk passed |
| +1 :green_heart: | javadoc | 2m 32s | | trunk passed with JDK
Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04 |
| +1 :green_heart: | javadoc | 2m 21s | | trunk passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | spotbugs | 4m 36s | | trunk passed |
| +1 :green_heart: | shadedclient | 23m 26s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 23s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 1m 26s | | the patch passed |
| +1 :green_heart: | compile | 4m 49s | | the patch passed with JDK
Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04 |
| +1 :green_heart: | javac | 4m 49s | | the patch passed |
| +1 :green_heart: | compile | 4m 46s | | the patch passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | javac | 4m 46s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 1m 17s | | the patch passed |
| +1 :green_heart: | mvnsite | 2m 17s | | the patch passed |
| +1 :green_heart: | javadoc | 2m 14s | | the patch passed with JDK
Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04 |
| +1 :green_heart: | javadoc | 2m 6s | | the patch passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | spotbugs | 4m 48s | | the patch passed |
| +1 :green_heart: | shadedclient | 23m 40s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 0m 56s | | hadoop-yarn-api in the patch
passed. |
| +1 :green_heart: | unit | 4m 53s | | hadoop-yarn-common in the patch
passed. |
| +1 :green_heart: | unit | 86m 47s | |
hadoop-yarn-server-resourcemanager in the patch passed. |
| +1 :green_heart: | asflicense | 0m 45s | | The patch does not
generate ASF License warnings. |
| | | 228m 20s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6114/3/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/6114 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets xmllint |
| uname | Linux 73a894bc5865 4.15.0-213-generic #224-Ubuntu SMP Mon Jun 19
13:30:12 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 82d0a1ceca94bcd053382361603d3b3b2e803de8 |
| Default Java | Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6114/3/testReport/ |
| Max. process+thread count | 943 (vs. ulimit of 5500) |
| modules | C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common
[GitHub] [hadoop] hadoop-yetus commented on pull request #6114: YARN-11468. Zookeeper SSL/TLS support
hadoop-yetus commented on PR #6114:
URL: https://github.com/apache/hadoop/pull/6114#issuecomment-1734624458
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 29s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +0 :ok: | xmllint | 0m 0s | | xmllint was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| -1 :x: | test4tests | 0m 0s | | The patch doesn't appear to include
any new or modified tests. Please justify why no new tests are needed for this
patch. Also please list what manual steps were performed to verify this patch.
|
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 14m 5s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 21m 33s | | trunk passed |
| +1 :green_heart: | compile | 5m 32s | | trunk passed with JDK
Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04 |
| +1 :green_heart: | compile | 4m 52s | | trunk passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | checkstyle | 1m 14s | | trunk passed |
| +1 :green_heart: | mvnsite | 2m 30s | | trunk passed |
| +1 :green_heart: | javadoc | 2m 29s | | trunk passed with JDK
Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04 |
| +1 :green_heart: | javadoc | 2m 24s | | trunk passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | spotbugs | 4m 42s | | trunk passed |
| +1 :green_heart: | shadedclient | 23m 22s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 24s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 1m 31s | | the patch passed |
| +1 :green_heart: | compile | 4m 53s | | the patch passed with JDK
Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04 |
| +1 :green_heart: | javac | 4m 53s | | the patch passed |
| +1 :green_heart: | compile | 4m 47s | | the patch passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | javac | 4m 47s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 1m 11s | | the patch passed |
| +1 :green_heart: | mvnsite | 2m 16s | | the patch passed |
| +1 :green_heart: | javadoc | 2m 15s | | the patch passed with JDK
Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04 |
| +1 :green_heart: | javadoc | 2m 9s | | the patch passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | spotbugs | 4m 54s | | the patch passed |
| +1 :green_heart: | shadedclient | 23m 29s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 0m 56s | | hadoop-yarn-api in the patch
passed. |
| +1 :green_heart: | unit | 4m 51s | | hadoop-yarn-common in the patch
passed. |
| +1 :green_heart: | unit | 87m 1s | |
hadoop-yarn-server-resourcemanager in the patch passed. |
| +1 :green_heart: | asflicense | 0m 46s | | The patch does not
generate ASF License warnings. |
| | | 228m 46s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6114/2/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/6114 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets xmllint |
| uname | Linux 624756fd29ad 4.15.0-213-generic #224-Ubuntu SMP Mon Jun 19
13:30:12 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 82d0a1ceca94bcd053382361603d3b3b2e803de8 |
| Default Java | Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6114/2/testReport/ |
| Max. process+thread count | 942 (vs. ulimit of 5500) |
| modules | C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common
[GitHub] [hadoop] goiri commented on a diff in pull request #6083: YARN-8862. [BackPort] [GPG] Add Yarn Registry cleanup in ApplicationCleaner.
goiri commented on code in PR #6083:
URL: https://github.com/apache/hadoop/pull/6083#discussion_r1336380046
##
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router/src/main/java/org/apache/hadoop/yarn/server/router/webapp/FederationInterceptorREST.java:
##
@@ -761,7 +761,7 @@ public AppsInfo getApps(HttpServletRequest hsr, String
stateQuery,
});
if (apps.getApps().isEmpty()) {
- return null;
+ return apps;
Review Comment:
Is this what we want?
##
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router/src/test/java/org/apache/hadoop/yarn/server/router/webapp/TestRouterWebServicesREST.java:
##
@@ -373,6 +373,8 @@ public void testClusterInfoXML() throws Exception {
assertEquals(
rmResponse.getRMVersion(),
routerResponse.getRMVersion());
+
+Thread.sleep(1000);
Review Comment:
Debugging?
##
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router/src/test/resources/yarn-site.xml:
##
@@ -39,7 +39,7 @@
yarn.resourcemanager.resource-profiles.source-file
profiles/sample-profiles-1.json
-
+
Review Comment:
Avoid
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] goiri merged pull request #6055: YARN-11547. [Federation] Router Supports Remove individual application records from FederationStateStore.
goiri merged PR #6055: URL: https://github.com/apache/hadoop/pull/6055 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18868) Optimize the configuration and use of callqueue overflow trigger failover
[
https://issues.apache.org/jira/browse/HADOOP-18868?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1776#comment-1776
]
ASF GitHub Bot commented on HADOOP-18868:
-
hadoop-yetus commented on PR #5998:
URL: https://github.com/apache/hadoop/pull/5998#issuecomment-1734364197
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 47s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available.
|
| +0 :ok: | xmllint | 0m 1s | | xmllint was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 4 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 14m 40s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 32m 28s | | trunk passed |
| +1 :green_heart: | compile | 17m 32s | | trunk passed with JDK
Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04 |
| +1 :green_heart: | compile | 15m 57s | | trunk passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | checkstyle | 4m 22s | | trunk passed |
| +1 :green_heart: | mvnsite | 3m 37s | | trunk passed |
| +1 :green_heart: | javadoc | 2m 46s | | trunk passed with JDK
Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04 |
| +1 :green_heart: | javadoc | 2m 51s | | trunk passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | spotbugs | 6m 8s | | trunk passed |
| +1 :green_heart: | shadedclient | 36m 38s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 32s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 2m 15s | | the patch passed |
| +1 :green_heart: | compile | 16m 36s | | the patch passed with JDK
Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04 |
| +1 :green_heart: | javac | 16m 36s | | the patch passed |
| +1 :green_heart: | compile | 16m 21s | | the patch passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | javac | 16m 21s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 4m 21s | | root: The patch generated
0 new + 208 unchanged - 1 fixed = 208 total (was 209) |
| +1 :green_heart: | mvnsite | 3m 31s | | the patch passed |
| +1 :green_heart: | javadoc | 2m 40s | | the patch passed with JDK
Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04 |
| +1 :green_heart: | javadoc | 2m 54s | | the patch passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | spotbugs | 6m 28s | | the patch passed |
| +1 :green_heart: | shadedclient | 37m 16s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 19m 18s | | hadoop-common in the patch
passed. |
| +1 :green_heart: | unit | 223m 51s | | hadoop-hdfs in the patch
passed. |
| +1 :green_heart: | asflicense | 1m 24s | | The patch does not
generate ASF License warnings. |
| | | 479m 31s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5998/10/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/5998 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets xmllint |
| uname | Linux f943cf6b9741 4.15.0-213-generic #224-Ubuntu SMP Mon Jun 19
13:30:12 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / c607adb3a3674ba024c60b956044ddd1f3648c63 |
| Default Java | Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5998/10/testReport/ |
| Max. process+thread count | 3420 (vs. ulimit of 5500) |
| modules | C: hadoop-common-project/hadoop-common
[GitHub] [hadoop] hadoop-yetus commented on pull request #5998: HADOOP-18868. Optimize the configuration and use of callqueue overflow trigger failover
hadoop-yetus commented on PR #5998:
URL: https://github.com/apache/hadoop/pull/5998#issuecomment-1734364197
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 47s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available.
|
| +0 :ok: | xmllint | 0m 1s | | xmllint was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 4 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 14m 40s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 32m 28s | | trunk passed |
| +1 :green_heart: | compile | 17m 32s | | trunk passed with JDK
Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04 |
| +1 :green_heart: | compile | 15m 57s | | trunk passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | checkstyle | 4m 22s | | trunk passed |
| +1 :green_heart: | mvnsite | 3m 37s | | trunk passed |
| +1 :green_heart: | javadoc | 2m 46s | | trunk passed with JDK
Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04 |
| +1 :green_heart: | javadoc | 2m 51s | | trunk passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | spotbugs | 6m 8s | | trunk passed |
| +1 :green_heart: | shadedclient | 36m 38s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 32s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 2m 15s | | the patch passed |
| +1 :green_heart: | compile | 16m 36s | | the patch passed with JDK
Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04 |
| +1 :green_heart: | javac | 16m 36s | | the patch passed |
| +1 :green_heart: | compile | 16m 21s | | the patch passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | javac | 16m 21s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 4m 21s | | root: The patch generated
0 new + 208 unchanged - 1 fixed = 208 total (was 209) |
| +1 :green_heart: | mvnsite | 3m 31s | | the patch passed |
| +1 :green_heart: | javadoc | 2m 40s | | the patch passed with JDK
Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04 |
| +1 :green_heart: | javadoc | 2m 54s | | the patch passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | spotbugs | 6m 28s | | the patch passed |
| +1 :green_heart: | shadedclient | 37m 16s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 19m 18s | | hadoop-common in the patch
passed. |
| +1 :green_heart: | unit | 223m 51s | | hadoop-hdfs in the patch
passed. |
| +1 :green_heart: | asflicense | 1m 24s | | The patch does not
generate ASF License warnings. |
| | | 479m 31s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5998/10/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/5998 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets xmllint |
| uname | Linux f943cf6b9741 4.15.0-213-generic #224-Ubuntu SMP Mon Jun 19
13:30:12 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / c607adb3a3674ba024c60b956044ddd1f3648c63 |
| Default Java | Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5998/10/testReport/ |
| Max. process+thread count | 3420 (vs. ulimit of 5500) |
| modules | C: hadoop-common-project/hadoop-common
hadoop-hdfs-project/hadoop-hdfs U: . |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5998/10/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0
[GitHub] [hadoop] hadoop-yetus commented on pull request #6113: HDFS-17210.Optimize AvailableSpaceBlockPlacementPolicy
hadoop-yetus commented on PR #6113:
URL: https://github.com/apache/hadoop/pull/6113#issuecomment-1734337567
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 56s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available.
|
| +0 :ok: | xmllint | 0m 1s | | xmllint was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| -1 :x: | test4tests | 0m 0s | | The patch doesn't appear to include
any new or modified tests. Please justify why no new tests are needed for this
patch. Also please list what manual steps were performed to verify this patch.
|
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 47m 45s | | trunk passed |
| +1 :green_heart: | compile | 1m 23s | | trunk passed with JDK
Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04 |
| +1 :green_heart: | compile | 1m 17s | | trunk passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | checkstyle | 1m 10s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 24s | | trunk passed |
| +1 :green_heart: | javadoc | 1m 9s | | trunk passed with JDK
Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04 |
| +1 :green_heart: | javadoc | 1m 38s | | trunk passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | spotbugs | 3m 27s | | trunk passed |
| +1 :green_heart: | shadedclient | 40m 14s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 1m 14s | | the patch passed |
| +1 :green_heart: | compile | 1m 17s | | the patch passed with JDK
Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04 |
| +1 :green_heart: | javac | 1m 17s | | the patch passed |
| +1 :green_heart: | compile | 1m 9s | | the patch passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | javac | 1m 9s | | the patch passed |
| +1 :green_heart: | blanks | 0m 1s | | The patch has no blanks
issues. |
| -0 :warning: | checkstyle | 1m 2s |
[/results-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6113/1/artifact/out/results-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt)
| hadoop-hdfs-project/hadoop-hdfs: The patch generated 5 new + 205 unchanged
- 0 fixed = 210 total (was 205) |
| +1 :green_heart: | mvnsite | 1m 16s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 56s | | the patch passed with JDK
Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04 |
| +1 :green_heart: | javadoc | 1m 32s | | the patch passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | spotbugs | 3m 21s | | the patch passed |
| +1 :green_heart: | shadedclient | 40m 28s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 236m 20s | | hadoop-hdfs in the patch
passed. |
| +1 :green_heart: | asflicense | 0m 45s | | The patch does not
generate ASF License warnings. |
| | | 393m 0s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6113/1/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/6113 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets xmllint |
| uname | Linux d30544716667 4.15.0-213-generic #224-Ubuntu SMP Mon Jun 19
13:30:12 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / bccc4c4714de1b45058e790a304f2f81cc459cf3 |
| Default Java | Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6113/1/testReport/ |
| Max. process+thread count | 2362 (vs. ulimit of 5500) |
| modules | C: hadoop-hdfs-project/hadoop-hdfs U:
hadoop-hdfs-project/hadoop-hdfs |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6113/1/console |
| versions |
[GitHub] [hadoop] hadoop-yetus commented on pull request #6083: YARN-8862. [BackPort] [GPG] Add Yarn Registry cleanup in ApplicationCleaner.
hadoop-yetus commented on PR #6083:
URL: https://github.com/apache/hadoop/pull/6083#issuecomment-1734285313
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 56s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +0 :ok: | xmllint | 0m 0s | | xmllint was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 4 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 14m 6s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 36m 46s | | trunk passed |
| +1 :green_heart: | compile | 2m 36s | | trunk passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | compile | 2m 21s | | trunk passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | checkstyle | 1m 22s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 37s | | trunk passed |
| +1 :green_heart: | javadoc | 1m 35s | | trunk passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 1m 23s | | trunk passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | spotbugs | 2m 54s | | trunk passed |
| +1 :green_heart: | shadedclient | 38m 5s | | branch has no errors
when building and testing our client artifacts. |
| -0 :warning: | patch | 38m 25s | | Used diff version of patch file.
Binary files and potentially other changes not applied. Please rebase and
squash commits if necessary. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 30s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 1m 12s | | the patch passed |
| +1 :green_heart: | compile | 2m 29s | | the patch passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javac | 2m 29s | | the patch passed |
| +1 :green_heart: | compile | 2m 12s | | the patch passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | javac | 2m 12s | | the patch passed |
| -1 :x: | blanks | 0m 0s |
[/blanks-eol.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6083/5/artifact/out/blanks-eol.txt)
| The patch has 1 line(s) that end in blanks. Use git apply --whitespace=fix
<>. Refer https://git-scm.com/docs/git-apply |
| +1 :green_heart: | checkstyle | 1m 14s | | the patch passed |
| +1 :green_heart: | mvnsite | 1m 17s | | the patch passed |
| +1 :green_heart: | javadoc | 1m 17s | | the patch passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 1m 10s | | the patch passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | spotbugs | 3m 11s | | the patch passed |
| +1 :green_heart: | shadedclient | 39m 10s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 3m 24s | | hadoop-yarn-server-common in
the patch passed. |
| +1 :green_heart: | unit | 0m 27s | | hadoop-yarn-server-router in
the patch passed. |
| +1 :green_heart: | unit | 0m 48s | |
hadoop-yarn-server-globalpolicygenerator in the patch passed. |
| +1 :green_heart: | asflicense | 0m 35s | | The patch does not
generate ASF License warnings. |
| | | 168m 44s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6083/5/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/6083 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets xmllint |
| uname | Linux 9fdaf2e6ad43 4.15.0-213-generic #224-Ubuntu SMP Mon Jun 19
13:30:12 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / affb7d57493f5371403c71fea467c8f1e8a01146 |
| Default Java | Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
[jira] [Commented] (HADOOP-18912) upgrade snappy-java to 1.1.10.4 due to CVE
[ https://issues.apache.org/jira/browse/HADOOP-18912?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17768785#comment-17768785 ] ASF GitHub Bot commented on HADOOP-18912: - pjfanning opened a new pull request, #6115: URL: https://github.com/apache/hadoop/pull/6115 ### Description of PR HADOOP-18912 ### How was this patch tested? CI build ### For code changes: - [x] Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')? - [ ] Object storage: have the integration tests been executed and the endpoint declared according to the connector-specific documentation? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [x] If applicable, have you updated the `LICENSE`, `LICENSE-binary`, `NOTICE-binary` files? > upgrade snappy-java to 1.1.10.4 due to CVE > -- > > Key: HADOOP-18912 > URL: https://issues.apache.org/jira/browse/HADOOP-18912 > Project: Hadoop Common > Issue Type: Bug > Components: build >Reporter: PJ Fanning >Priority: Major > > follow up to HADOOP-18782 > https://github.com/xerial/snappy-java/security/advisories/GHSA-55g7-9cwv-5qfv -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-18912) upgrade snappy-java to 1.1.10.4 due to CVE
[ https://issues.apache.org/jira/browse/HADOOP-18912?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HADOOP-18912: Labels: pull-request-available (was: ) > upgrade snappy-java to 1.1.10.4 due to CVE > -- > > Key: HADOOP-18912 > URL: https://issues.apache.org/jira/browse/HADOOP-18912 > Project: Hadoop Common > Issue Type: Bug > Components: build >Reporter: PJ Fanning >Priority: Major > Labels: pull-request-available > > follow up to HADOOP-18782 > https://github.com/xerial/snappy-java/security/advisories/GHSA-55g7-9cwv-5qfv -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #6114: YARN-11468. Zookeeper SSL/TLS support
hadoop-yetus commented on PR #6114:
URL: https://github.com/apache/hadoop/pull/6114#issuecomment-1734138046
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 33s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +0 :ok: | xmllint | 0m 0s | | xmllint was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| -1 :x: | test4tests | 0m 0s | | The patch doesn't appear to include
any new or modified tests. Please justify why no new tests are needed for this
patch. Also please list what manual steps were performed to verify this patch.
|
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 14m 10s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 19m 59s | | trunk passed |
| +1 :green_heart: | compile | 4m 57s | | trunk passed with JDK
Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04 |
| +1 :green_heart: | compile | 4m 21s | | trunk passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | checkstyle | 1m 13s | | trunk passed |
| +1 :green_heart: | mvnsite | 2m 27s | | trunk passed |
| +1 :green_heart: | javadoc | 2m 25s | | trunk passed with JDK
Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04 |
| +1 :green_heart: | javadoc | 2m 17s | | trunk passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | spotbugs | 4m 12s | | trunk passed |
| +1 :green_heart: | shadedclient | 21m 57s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 26s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 1m 24s | | the patch passed |
| +1 :green_heart: | compile | 4m 14s | | the patch passed with JDK
Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04 |
| +1 :green_heart: | javac | 4m 14s | | the patch passed |
| +1 :green_heart: | compile | 4m 16s | | the patch passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | javac | 4m 16s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| -0 :warning: | checkstyle | 1m 7s |
[/results-checkstyle-hadoop-yarn-project_hadoop-yarn.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6114/1/artifact/out/results-checkstyle-hadoop-yarn-project_hadoop-yarn.txt)
| hadoop-yarn-project/hadoop-yarn: The patch generated 1 new + 204 unchanged
- 0 fixed = 205 total (was 204) |
| +1 :green_heart: | mvnsite | 2m 10s | | the patch passed |
| +1 :green_heart: | javadoc | 2m 8s | | the patch passed with JDK
Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04 |
| +1 :green_heart: | javadoc | 2m 3s | | the patch passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | spotbugs | 4m 24s | | the patch passed |
| +1 :green_heart: | shadedclient | 22m 15s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 0m 55s | | hadoop-yarn-api in the patch
passed. |
| +1 :green_heart: | unit | 4m 48s | | hadoop-yarn-common in the patch
passed. |
| -1 :x: | unit | 77m 7s |
[/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6114/1/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt)
| hadoop-yarn-server-resourcemanager in the patch passed. |
| +0 :ok: | asflicense | 0m 45s | | ASF License check generated no
output? |
| | | 210m 42s | | |
| Reason | Tests |
|---:|:--|
| Failed junit tests |
hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesReservation |
| |
hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesForCSWithPartitions |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6114/1/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/6114 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets xmllint |
| uname | Linux 54589d8dd46e
[GitHub] [hadoop] pjfanning opened a new pull request, #6115: HADOOP-18912. upgrade snappy-java to 1.1.10.4
pjfanning opened a new pull request, #6115: URL: https://github.com/apache/hadoop/pull/6115 ### Description of PR HADOOP-18912 ### How was this patch tested? CI build ### For code changes: - [x] Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')? - [ ] Object storage: have the integration tests been executed and the endpoint declared according to the connector-specific documentation? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [x] If applicable, have you updated the `LICENSE`, `LICENSE-binary`, `NOTICE-binary` files? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Created] (HADOOP-18912) upgrade snappy-java to 1.1.10.4 due to CVE
PJ Fanning created HADOOP-18912: --- Summary: upgrade snappy-java to 1.1.10.4 due to CVE Key: HADOOP-18912 URL: https://issues.apache.org/jira/browse/HADOOP-18912 Project: Hadoop Common Issue Type: Bug Components: build Reporter: PJ Fanning follow up to HADOOP-18782 https://github.com/xerial/snappy-java/security/advisories/GHSA-55g7-9cwv-5qfv -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #6112: HDFS-17205. HdfsServerConstants.MIN_BLOCKS_FOR_WRITE should be configurable
hadoop-yetus commented on PR #6112:
URL: https://github.com/apache/hadoop/pull/6112#issuecomment-1734090460
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 39s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 1s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +0 :ok: | xmllint | 0m 0s | | xmllint was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 1s | | The patch appears to
include 3 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 32m 29s | | trunk passed |
| +1 :green_heart: | compile | 0m 56s | | trunk passed with JDK
Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04 |
| +1 :green_heart: | compile | 0m 50s | | trunk passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | checkstyle | 0m 48s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 58s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 52s | | trunk passed with JDK
Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04 |
| +1 :green_heart: | javadoc | 1m 11s | | trunk passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | spotbugs | 2m 1s | | trunk passed |
| +1 :green_heart: | shadedclient | 22m 5s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 45s | | the patch passed |
| +1 :green_heart: | compile | 0m 48s | | the patch passed with JDK
Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04 |
| +1 :green_heart: | javac | 0m 48s | | the patch passed |
| +1 :green_heart: | compile | 0m 42s | | the patch passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | javac | 0m 42s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| -0 :warning: | checkstyle | 0m 35s |
[/results-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6112/1/artifact/out/results-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt)
| hadoop-hdfs-project/hadoop-hdfs: The patch generated 1 new + 460 unchanged
- 0 fixed = 461 total (was 460) |
| +1 :green_heart: | mvnsite | 0m 47s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 39s | | the patch passed with JDK
Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04 |
| +1 :green_heart: | javadoc | 1m 5s | | the patch passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | spotbugs | 1m 55s | | the patch passed |
| +1 :green_heart: | shadedclient | 22m 5s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| -1 :x: | unit | 197m 2s |
[/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6112/1/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt)
| hadoop-hdfs in the patch passed. |
| +1 :green_heart: | asflicense | 0m 39s | | The patch does not
generate ASF License warnings. |
| | | 291m 34s | | |
| Reason | Tests |
|---:|:--|
| Failed junit tests | hadoop.hdfs.server.datanode.TestDirectoryScanner |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6112/1/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/6112 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets xmllint |
| uname | Linux 47f0753c31bc 4.15.0-213-generic #224-Ubuntu SMP Mon Jun 19
13:30:12 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 96e75e65cc89baaf425073307702ceec04de9a79 |
| Default Java | Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6112/1/testReport/ |
| Max. process+thread count | 3481 (vs. ulimit of 5500) |
| modules | C:
[GitHub] [hadoop] slfan1989 commented on a diff in pull request #6083: YARN-8862. [BackPort] [GPG] Add Yarn Registry cleanup in ApplicationCleaner.
slfan1989 commented on code in PR #6083:
URL: https://github.com/apache/hadoop/pull/6083#discussion_r1336093087
##
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router/src/test/java/org/apache/hadoop/yarn/server/router/subcluster/fair/TestYarnFederationWithFairScheduler.java:
##
@@ -64,6 +64,7 @@ public void testGetClusterInfo() throws InterruptedException,
IOException {
FederationClusterInfo federationClusterInfo =
TestFederationSubCluster.performGetCalls(ROUTER_WEB_ADDRESS,
RM_WEB_SERVICE_PATH,
FederationClusterInfo.class, null, null);
+Thread.sleep(200);
Review Comment:
I will fix it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] slfan1989 commented on a diff in pull request #6083: YARN-8862. [BackPort] [GPG] Add Yarn Registry cleanup in ApplicationCleaner.
slfan1989 commented on code in PR #6083:
URL: https://github.com/apache/hadoop/pull/6083#discussion_r1336092755
##
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-globalpolicygenerator/src/test/java/org/apache/hadoop/yarn/server/globalpolicygenerator/applicationcleaner/TestDefaultApplicationCleaner.java:
##
@@ -96,6 +119,13 @@ public void breakDown() {
stateStore.close();
stateStore = null;
}
+if (registryClient != null) {
+ registryClient.cleanAllApplications();
Review Comment:
Thank you for your help in reviewing the code! I will improve this part of
the code.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #6027: YARN-11468. Zookeeper SSL/TLS support
hadoop-yetus commented on PR #6027:
URL: https://github.com/apache/hadoop/pull/6027#issuecomment-1733954497
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 45s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available.
|
| +0 :ok: | xmllint | 0m 1s | | xmllint was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 1 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 14m 24s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 34m 33s | | trunk passed |
| +1 :green_heart: | compile | 8m 0s | | trunk passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | compile | 8m 12s | | trunk passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | checkstyle | 2m 4s | | trunk passed |
| +1 :green_heart: | mvnsite | 3m 29s | | trunk passed |
| +1 :green_heart: | javadoc | 3m 32s | | trunk passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 3m 17s | | trunk passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | spotbugs | 8m 9s | | trunk passed |
| -1 :x: | shadedclient | 48m 25s | | branch has errors when building
and testing our client artifacts. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 7s | | Maven dependency ordering for patch |
| -1 :x: | mvninstall | 0m 41s |
[/patch-mvninstall-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-common.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6027/2/artifact/out/patch-mvninstall-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-common.txt)
| hadoop-yarn-common in the patch failed. |
| -1 :x: | mvninstall | 0m 56s |
[/patch-mvninstall-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6027/2/artifact/out/patch-mvninstall-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt)
| hadoop-yarn-server-resourcemanager in the patch failed. |
| -1 :x: | compile | 2m 40s |
[/patch-compile-hadoop-yarn-project_hadoop-yarn-jdkUbuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6027/2/artifact/out/patch-compile-hadoop-yarn-project_hadoop-yarn-jdkUbuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04.txt)
| hadoop-yarn in the patch failed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04. |
| -1 :x: | javac | 2m 40s |
[/patch-compile-hadoop-yarn-project_hadoop-yarn-jdkUbuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6027/2/artifact/out/patch-compile-hadoop-yarn-project_hadoop-yarn-jdkUbuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04.txt)
| hadoop-yarn in the patch failed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04. |
| -1 :x: | compile | 0m 25s |
[/patch-compile-hadoop-yarn-project_hadoop-yarn-jdkPrivateBuild-1.8.0_382-8u382-ga-1~20.04.1-b05.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6027/2/artifact/out/patch-compile-hadoop-yarn-project_hadoop-yarn-jdkPrivateBuild-1.8.0_382-8u382-ga-1~20.04.1-b05.txt)
| hadoop-yarn in the patch failed with JDK Private
Build-1.8.0_382-8u382-ga-1~20.04.1-b05. |
| -1 :x: | javac | 0m 25s |
[/patch-compile-hadoop-yarn-project_hadoop-yarn-jdkPrivateBuild-1.8.0_382-8u382-ga-1~20.04.1-b05.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6027/2/artifact/out/patch-compile-hadoop-yarn-project_hadoop-yarn-jdkPrivateBuild-1.8.0_382-8u382-ga-1~20.04.1-b05.txt)
| hadoop-yarn in the patch failed with JDK Private
Build-1.8.0_382-8u382-ga-1~20.04.1-b05. |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| -0 :warning: | checkstyle | 0m 24s |
[/buildtool-patch-checkstyle-hadoop-yarn-project_hadoop-yarn.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6027/2/artifact/out/buildtool-patch-checkstyle-hadoop-yarn-project_hadoop-yarn.txt)
| The patch fails to run checkstyle in hadoop-yarn |
| -1 :x: | mvnsite | 0m 25s |
[jira] [Commented] (HADOOP-18257) Analyzing S3A Audit Logs
[
https://issues.apache.org/jira/browse/HADOOP-18257?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17768740#comment-17768740
]
ASF GitHub Bot commented on HADOOP-18257:
-
hadoop-yetus commented on PR #6000:
URL: https://github.com/apache/hadoop/pull/6000#issuecomment-1733914140
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 1m 35s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +0 :ok: | xmllint | 0m 0s | | xmllint was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 2 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 56m 27s | | trunk passed |
| +1 :green_heart: | compile | 1m 15s | | trunk passed with JDK
Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04 |
| +1 :green_heart: | compile | 0m 44s | | trunk passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | checkstyle | 0m 52s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 6s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 38s | | trunk passed with JDK
Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04 |
| +1 :green_heart: | javadoc | 0m 51s | | trunk passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| -1 :x: | spotbugs | 0m 53s |
[/branch-spotbugs-hadoop-tools_hadoop-aws.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6000/2/artifact/out/branch-spotbugs-hadoop-tools_hadoop-aws.txt)
| hadoop-aws in trunk failed. |
| +1 :green_heart: | shadedclient | 5m 17s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| -1 :x: | mvninstall | 0m 40s |
[/patch-mvninstall-hadoop-tools_hadoop-aws.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6000/2/artifact/out/patch-mvninstall-hadoop-tools_hadoop-aws.txt)
| hadoop-aws in the patch failed. |
| -1 :x: | compile | 0m 26s |
[/patch-compile-hadoop-tools_hadoop-aws-jdkUbuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6000/2/artifact/out/patch-compile-hadoop-tools_hadoop-aws-jdkUbuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04.txt)
| hadoop-aws in the patch failed with JDK
Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04. |
| -1 :x: | javac | 0m 26s |
[/patch-compile-hadoop-tools_hadoop-aws-jdkUbuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6000/2/artifact/out/patch-compile-hadoop-tools_hadoop-aws-jdkUbuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04.txt)
| hadoop-aws in the patch failed with JDK
Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04. |
| -1 :x: | compile | 0m 24s |
[/patch-compile-hadoop-tools_hadoop-aws-jdkPrivateBuild-1.8.0_382-8u382-ga-1~20.04.1-b05.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6000/2/artifact/out/patch-compile-hadoop-tools_hadoop-aws-jdkPrivateBuild-1.8.0_382-8u382-ga-1~20.04.1-b05.txt)
| hadoop-aws in the patch failed with JDK Private
Build-1.8.0_382-8u382-ga-1~20.04.1-b05. |
| -1 :x: | javac | 0m 24s |
[/patch-compile-hadoop-tools_hadoop-aws-jdkPrivateBuild-1.8.0_382-8u382-ga-1~20.04.1-b05.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6000/2/artifact/out/patch-compile-hadoop-tools_hadoop-aws-jdkPrivateBuild-1.8.0_382-8u382-ga-1~20.04.1-b05.txt)
| hadoop-aws in the patch failed with JDK Private
Build-1.8.0_382-8u382-ga-1~20.04.1-b05. |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| -0 :warning: | checkstyle | 0m 22s |
[/buildtool-patch-checkstyle-hadoop-tools_hadoop-aws.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6000/2/artifact/out/buildtool-patch-checkstyle-hadoop-tools_hadoop-aws.txt)
| The patch fails to run checkstyle in hadoop-aws |
| -1 :x: | mvnsite | 0m 22s |
[/patch-mvnsite-hadoop-tools_hadoop-aws.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6000/2/artifact/out/patch-mvnsite-hadoop-tools_hadoop-aws.txt)
| hadoop-aws in the patch failed. |
| -1 :x: | javadoc | 0m 24s |
[GitHub] [hadoop] hadoop-yetus commented on pull request #6000: HADOOP-18257. Merging and Parsing S3A audit logs into Avro format for analysis.
hadoop-yetus commented on PR #6000:
URL: https://github.com/apache/hadoop/pull/6000#issuecomment-1733914140
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 1m 35s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +0 :ok: | xmllint | 0m 0s | | xmllint was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 2 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 56m 27s | | trunk passed |
| +1 :green_heart: | compile | 1m 15s | | trunk passed with JDK
Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04 |
| +1 :green_heart: | compile | 0m 44s | | trunk passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | checkstyle | 0m 52s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 6s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 38s | | trunk passed with JDK
Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04 |
| +1 :green_heart: | javadoc | 0m 51s | | trunk passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| -1 :x: | spotbugs | 0m 53s |
[/branch-spotbugs-hadoop-tools_hadoop-aws.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6000/2/artifact/out/branch-spotbugs-hadoop-tools_hadoop-aws.txt)
| hadoop-aws in trunk failed. |
| +1 :green_heart: | shadedclient | 5m 17s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| -1 :x: | mvninstall | 0m 40s |
[/patch-mvninstall-hadoop-tools_hadoop-aws.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6000/2/artifact/out/patch-mvninstall-hadoop-tools_hadoop-aws.txt)
| hadoop-aws in the patch failed. |
| -1 :x: | compile | 0m 26s |
[/patch-compile-hadoop-tools_hadoop-aws-jdkUbuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6000/2/artifact/out/patch-compile-hadoop-tools_hadoop-aws-jdkUbuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04.txt)
| hadoop-aws in the patch failed with JDK
Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04. |
| -1 :x: | javac | 0m 26s |
[/patch-compile-hadoop-tools_hadoop-aws-jdkUbuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6000/2/artifact/out/patch-compile-hadoop-tools_hadoop-aws-jdkUbuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04.txt)
| hadoop-aws in the patch failed with JDK
Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04. |
| -1 :x: | compile | 0m 24s |
[/patch-compile-hadoop-tools_hadoop-aws-jdkPrivateBuild-1.8.0_382-8u382-ga-1~20.04.1-b05.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6000/2/artifact/out/patch-compile-hadoop-tools_hadoop-aws-jdkPrivateBuild-1.8.0_382-8u382-ga-1~20.04.1-b05.txt)
| hadoop-aws in the patch failed with JDK Private
Build-1.8.0_382-8u382-ga-1~20.04.1-b05. |
| -1 :x: | javac | 0m 24s |
[/patch-compile-hadoop-tools_hadoop-aws-jdkPrivateBuild-1.8.0_382-8u382-ga-1~20.04.1-b05.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6000/2/artifact/out/patch-compile-hadoop-tools_hadoop-aws-jdkPrivateBuild-1.8.0_382-8u382-ga-1~20.04.1-b05.txt)
| hadoop-aws in the patch failed with JDK Private
Build-1.8.0_382-8u382-ga-1~20.04.1-b05. |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| -0 :warning: | checkstyle | 0m 22s |
[/buildtool-patch-checkstyle-hadoop-tools_hadoop-aws.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6000/2/artifact/out/buildtool-patch-checkstyle-hadoop-tools_hadoop-aws.txt)
| The patch fails to run checkstyle in hadoop-aws |
| -1 :x: | mvnsite | 0m 22s |
[/patch-mvnsite-hadoop-tools_hadoop-aws.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6000/2/artifact/out/patch-mvnsite-hadoop-tools_hadoop-aws.txt)
| hadoop-aws in the patch failed. |
| -1 :x: | javadoc | 0m 24s |
[/patch-javadoc-hadoop-tools_hadoop-aws-jdkUbuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6000/2/artifact/out/patch-javadoc-hadoop-tools_hadoop-aws-jdkUbuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04.txt)
| hadoop-aws in the patch failed with JDK
Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04. |
| -1 :x: | javadoc | 0m 25s |
[GitHub] [hadoop] brumi1024 commented on pull request #5989: YARN-11514. Extend SchedulerResponse with capacityVector
brumi1024 commented on PR #5989: URL: https://github.com/apache/hadoop/pull/5989#issuecomment-1733817064 Thanks @tomicooler for the review, merging to trunk. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] brumi1024 merged pull request #5989: YARN-11514. Extend SchedulerResponse with capacityVector
brumi1024 merged PR #5989: URL: https://github.com/apache/hadoop/pull/5989 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18908) Improve s3a region handling, including determining from endpoint
[
https://issues.apache.org/jira/browse/HADOOP-18908?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17768698#comment-17768698
]
ASF GitHub Bot commented on HADOOP-18908:
-
hadoop-yetus commented on PR #6106:
URL: https://github.com/apache/hadoop/pull/6106#issuecomment-1733731771
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 42s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 2s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 2s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 2 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 14m 19s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 31m 19s | | trunk passed |
| +1 :green_heart: | compile | 17m 18s | | trunk passed with JDK
Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04 |
| +1 :green_heart: | compile | 15m 55s | | trunk passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | checkstyle | 4m 28s | | trunk passed |
| +1 :green_heart: | mvnsite | 2m 47s | | trunk passed |
| +1 :green_heart: | javadoc | 2m 9s | | trunk passed with JDK
Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04 |
| +1 :green_heart: | javadoc | 1m 53s | | trunk passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | spotbugs | 4m 7s | | trunk passed |
| +1 :green_heart: | shadedclient | 33m 57s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 33s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 1m 30s | | the patch passed |
| +1 :green_heart: | compile | 16m 23s | | the patch passed with JDK
Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04 |
| +1 :green_heart: | javac | 16m 23s | | the patch passed |
| +1 :green_heart: | compile | 15m 57s | | the patch passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | javac | 15m 57s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| -0 :warning: | checkstyle | 4m 18s |
[/results-checkstyle-root.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6106/4/artifact/out/results-checkstyle-root.txt)
| root: The patch generated 8 new + 4 unchanged - 0 fixed = 12 total (was 4)
|
| +1 :green_heart: | mvnsite | 2m 47s | | the patch passed |
| +1 :green_heart: | javadoc | 2m 3s | | the patch passed with JDK
Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04 |
| +1 :green_heart: | javadoc | 1m 54s | | the patch passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| -1 :x: | spotbugs | 1m 37s |
[/new-spotbugs-hadoop-tools_hadoop-aws.html](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6106/4/artifact/out/new-spotbugs-hadoop-tools_hadoop-aws.html)
| hadoop-tools/hadoop-aws generated 1 new + 0 unchanged - 0 fixed = 1 total
(was 0) |
| +1 :green_heart: | shadedclient | 34m 41s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 19m 29s | | hadoop-common in the patch
passed. |
| +1 :green_heart: | unit | 3m 4s | | hadoop-aws in the patch passed.
|
| +1 :green_heart: | asflicense | 1m 14s | | The patch does not
generate ASF License warnings. |
| | | 246m 11s | | |
| Reason | Tests |
|---:|:--|
| SpotBugs | module:hadoop-tools/hadoop-aws |
| | Unused field:S3ClientFactory.java |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6106/4/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/6106 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux 3f52098cab9a 4.15.0-213-generic #224-Ubuntu SMP Mon Jun 19
13:30:12 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / f7ad38001b7334c0b886c20308c2c3095614274a |
| Default Java | Private
[GitHub] [hadoop] hadoop-yetus commented on pull request #6106: HADOOP-18908. Improve S3A region handling.
hadoop-yetus commented on PR #6106:
URL: https://github.com/apache/hadoop/pull/6106#issuecomment-1733731771
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 42s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 2s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 2s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 2 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 14m 19s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 31m 19s | | trunk passed |
| +1 :green_heart: | compile | 17m 18s | | trunk passed with JDK
Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04 |
| +1 :green_heart: | compile | 15m 55s | | trunk passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | checkstyle | 4m 28s | | trunk passed |
| +1 :green_heart: | mvnsite | 2m 47s | | trunk passed |
| +1 :green_heart: | javadoc | 2m 9s | | trunk passed with JDK
Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04 |
| +1 :green_heart: | javadoc | 1m 53s | | trunk passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | spotbugs | 4m 7s | | trunk passed |
| +1 :green_heart: | shadedclient | 33m 57s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 33s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 1m 30s | | the patch passed |
| +1 :green_heart: | compile | 16m 23s | | the patch passed with JDK
Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04 |
| +1 :green_heart: | javac | 16m 23s | | the patch passed |
| +1 :green_heart: | compile | 15m 57s | | the patch passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | javac | 15m 57s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| -0 :warning: | checkstyle | 4m 18s |
[/results-checkstyle-root.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6106/4/artifact/out/results-checkstyle-root.txt)
| root: The patch generated 8 new + 4 unchanged - 0 fixed = 12 total (was 4)
|
| +1 :green_heart: | mvnsite | 2m 47s | | the patch passed |
| +1 :green_heart: | javadoc | 2m 3s | | the patch passed with JDK
Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04 |
| +1 :green_heart: | javadoc | 1m 54s | | the patch passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| -1 :x: | spotbugs | 1m 37s |
[/new-spotbugs-hadoop-tools_hadoop-aws.html](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6106/4/artifact/out/new-spotbugs-hadoop-tools_hadoop-aws.html)
| hadoop-tools/hadoop-aws generated 1 new + 0 unchanged - 0 fixed = 1 total
(was 0) |
| +1 :green_heart: | shadedclient | 34m 41s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 19m 29s | | hadoop-common in the patch
passed. |
| +1 :green_heart: | unit | 3m 4s | | hadoop-aws in the patch passed.
|
| +1 :green_heart: | asflicense | 1m 14s | | The patch does not
generate ASF License warnings. |
| | | 246m 11s | | |
| Reason | Tests |
|---:|:--|
| SpotBugs | module:hadoop-tools/hadoop-aws |
| | Unused field:S3ClientFactory.java |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6106/4/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/6106 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux 3f52098cab9a 4.15.0-213-generic #224-Ubuntu SMP Mon Jun 19
13:30:12 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / f7ad38001b7334c0b886c20308c2c3095614274a |
| Default Java | Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| Test Results |
[GitHub] [hadoop] ferdelyi commented on pull request #6027: YARN-11468. Zookeeper SSL/TLS support
ferdelyi commented on PR #6027: URL: https://github.com/apache/hadoop/pull/6027#issuecomment-1733714660 A new pull request was created for the same change, as broke the chain: https://github.com/apache/hadoop/pull/6114 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] ferdelyi opened a new pull request, #6114: YARN-11468. Zookeeper SSL/TLS support
ferdelyi opened a new pull request, #6114: URL: https://github.com/apache/hadoop/pull/6114 ### Description of PR New parameter introduced to enable SSL/TLS for ZK Client for YARN HA, which takes effect when Curator is used (when yarn.resourcemanager.ha.curator-leader-elector.enabled is enabled). There is no unit test for this change, as the Curator client setup is tested under the Hadoop Common code change HADOOP-18709. This is the continuation of https://github.com/apache/hadoop/pull/6027 ### How was this patch tested? Via an integration test on my cluster and also created a simple unit test to show that the new config setting works. ### For code changes: - [x] Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')? - [ ] Object storage: have the integration tests been executed and the endpoint declared according to the connector-specific documentation? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the `LICENSE`, `LICENSE-binary`, `NOTICE-binary` files? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-18911) DfsClientShmManager allocSlot hang on awaitUninterruptibly itself
[
https://issues.apache.org/jira/browse/HADOOP-18911?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
konwu updated HADOOP-18911:
---
Environment:
hadoop 3.3.1
yarn 3.3.1
spark 3.1.3
was:
hadoop 3.3.1
yarn 3.3.1
> DfsClientShmManager allocSlot hang on awaitUninterruptibly itself
> -
>
> Key: HADOOP-18911
> URL: https://issues.apache.org/jira/browse/HADOOP-18911
> Project: Hadoop Common
> Issue Type: Bug
> Components: hdfs-client
>Affects Versions: 3.3.1
> Environment: hadoop 3.3.1
> yarn 3.3.1
> spark 3.1.3
>
>Reporter: konwu
>Priority: Major
>
> Spark task Driver Thread hang on awaitUninterruptibly
> thread stack like below:
> {code:java}
> sun.misc.Unsafe.park(Native Method)
> java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitUninterruptibly(AbstractQueuedSynchronizer.java:1976)
> org.apache.hadoop.hdfs.shortcircuit.DfsClientShmManager$EndpointShmManager.allocSlot(DfsClientShmManager.java:244)
> org.apache.hadoop.hdfs.shortcircuit.DfsClientShmManager.allocSlot(DfsClientShmManager.java:417)
> org.apache.hadoop.hdfs.shortcircuit.ShortCircuitCache.allocShmSlot(ShortCircuitCache.java:1006)
> org.apache.hadoop.hdfs.client.impl.BlockReaderFactory.createShortCircuitReplicaInfo(BlockReaderFactory.java:535)
> org.apache.hadoop.hdfs.shortcircuit.ShortCircuitCache.create(ShortCircuitCache.java:786)
> org.apache.hadoop.hdfs.shortcircuit.ShortCircuitCache.fetchOrCreate(ShortCircuitCache.java:723)
> org.apache.hadoop.hdfs.client.impl.BlockReaderFactory.getBlockReaderLocal(BlockReaderFactory.java:483)
> org.apache.hadoop.hdfs.client.impl.BlockReaderFactory.build(BlockReaderFactory.java:360)
> org.apache.hadoop.hdfs.DFSInputStream.getBlockReader(DFSInputStream.java:755)
> org.apache.hadoop.hdfs.DFSInputStream.blockSeekTo(DFSInputStream.java:685) =>
> holding Monitor(org.apache.hadoop.hdfs.DFSInputStream@59925309})
> org.apache.hadoop.hdfs.DFSInputStream.readWithStrategy(DFSInputStream.java:884)
> => holding Monitor(org.apache.hadoop.hdfs.DFSInputStream@59925309})
> org.apache.hadoop.hdfs.DFSInputStream.read(DFSInputStream.java:957) =>
> holding Monitor(org.apache.hadoop.hdfs.DFSInputStream@59925309})
> java.io.DataInputStream.read(DataInputStream.java:100)
> java.nio.file.Files.copy(Files.java:2908)
> java.nio.file.Files.copy(Files.java:3027)
> sun.net.www.protocol.jar.URLJarFile$1.run(URLJarFile.java:220)
> sun.net.www.protocol.jar.URLJarFile$1.run(URLJarFile.java:216)
> java.security.AccessController.doPrivileged(Native Method)
> sun.net.www.protocol.jar.URLJarFile.retrieve(URLJarFile.java:215)
> sun.net.www.protocol.jar.URLJarFile.getJarFile(URLJarFile.java:71)
> sun.net.www.protocol.jar.JarFileFactory.get(JarFileFactory.java:84)
> sun.net.www.protocol.jar.JarURLConnection.connect(JarURLConnection.java:122)
> sun.net.www.protocol.jar.JarURLConnection.getJarFile(JarURLConnection.java:89)
> sun.misc.URLClassPath$JarLoader.getJarFile(URLClassPath.java:944)
> sun.misc.URLClassPath$JarLoader.access$800(URLClassPath.java:801)
> sun.misc.URLClassPath$JarLoader$1.run(URLClassPath.java:886)
> sun.misc.URLClassPath$JarLoader$1.run(URLClassPath.java:879)
> java.security.AccessController.doPrivileged(Native Method)
> sun.misc.URLClassPath$JarLoader.ensureOpen(URLClassPath.java:878)
> sun.misc.URLClassPath$JarLoader.(URLClassPath.java:829)
> sun.misc.URLClassPath$3.run(URLClassPath.java:575)
> sun.misc.URLClassPath$3.run(URLClassPath.java:565)
> java.security.AccessController.doPrivileged(Native Method)
> sun.misc.URLClassPath.getLoader(URLClassPath.java:564)
> sun.misc.URLClassPath.getLoader(URLClassPath.java:529)
> sun.misc.URLClassPath.getNextLoader(URLClassPath.java:494) => holding
> Monitor(sun.misc.URLClassPath@929546251})
> sun.misc.URLClassPath.findResource(URLClassPath.java:224)
> java.net.URLClassLoader$2.run(URLClassLoader.java:572)
> java.net.URLClassLoader$2.run(URLClassLoader.java:570)
> java.security.AccessController.doPrivileged(Native Method)
> java.net.URLClassLoader.findResource(URLClassLoader.java:569)
> java.lang.ClassLoader.getResource(ClassLoader.java:1096)
> java.lang.ClassLoader.getResource(ClassLoader.java:1091)
> org.apache.hadoop.conf.Configuration.getResource(Configuration.java:2809)
> org.apache.hadoop.conf.Configuration.getStreamReader(Configuration.java:3081)
> org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:3040)
> org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:3013)
> org.apache.hadoop.conf.Configuration.loadProps(Configuration.java:2893) =>
> holding Monitor(org.apache.hadoop.conf.Configuration@1190238736})
> org.apache.hadoop.conf.Configuration.getProps(Configuration.java:2875) =>
> holding
[jira] [Updated] (HADOOP-18911) DfsClientShmManager allocSlot hang on awaitUninterruptibly itself
[
https://issues.apache.org/jira/browse/HADOOP-18911?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
konwu updated HADOOP-18911:
---
Description:
Spark task Driver Thread hang on awaitUninterruptibly
thread stack like below:
{code:java}
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitUninterruptibly(AbstractQueuedSynchronizer.java:1976)
org.apache.hadoop.hdfs.shortcircuit.DfsClientShmManager$EndpointShmManager.allocSlot(DfsClientShmManager.java:244)
org.apache.hadoop.hdfs.shortcircuit.DfsClientShmManager.allocSlot(DfsClientShmManager.java:417)
org.apache.hadoop.hdfs.shortcircuit.ShortCircuitCache.allocShmSlot(ShortCircuitCache.java:1006)
org.apache.hadoop.hdfs.client.impl.BlockReaderFactory.createShortCircuitReplicaInfo(BlockReaderFactory.java:535)
org.apache.hadoop.hdfs.shortcircuit.ShortCircuitCache.create(ShortCircuitCache.java:786)
org.apache.hadoop.hdfs.shortcircuit.ShortCircuitCache.fetchOrCreate(ShortCircuitCache.java:723)
org.apache.hadoop.hdfs.client.impl.BlockReaderFactory.getBlockReaderLocal(BlockReaderFactory.java:483)
org.apache.hadoop.hdfs.client.impl.BlockReaderFactory.build(BlockReaderFactory.java:360)
org.apache.hadoop.hdfs.DFSInputStream.getBlockReader(DFSInputStream.java:755)
org.apache.hadoop.hdfs.DFSInputStream.blockSeekTo(DFSInputStream.java:685) =>
holding Monitor(org.apache.hadoop.hdfs.DFSInputStream@59925309})
org.apache.hadoop.hdfs.DFSInputStream.readWithStrategy(DFSInputStream.java:884)
=> holding Monitor(org.apache.hadoop.hdfs.DFSInputStream@59925309})
org.apache.hadoop.hdfs.DFSInputStream.read(DFSInputStream.java:957) => holding
Monitor(org.apache.hadoop.hdfs.DFSInputStream@59925309})
java.io.DataInputStream.read(DataInputStream.java:100)
java.nio.file.Files.copy(Files.java:2908)
java.nio.file.Files.copy(Files.java:3027)
sun.net.www.protocol.jar.URLJarFile$1.run(URLJarFile.java:220)
sun.net.www.protocol.jar.URLJarFile$1.run(URLJarFile.java:216)
java.security.AccessController.doPrivileged(Native Method)
sun.net.www.protocol.jar.URLJarFile.retrieve(URLJarFile.java:215)
sun.net.www.protocol.jar.URLJarFile.getJarFile(URLJarFile.java:71)
sun.net.www.protocol.jar.JarFileFactory.get(JarFileFactory.java:84)
sun.net.www.protocol.jar.JarURLConnection.connect(JarURLConnection.java:122)
sun.net.www.protocol.jar.JarURLConnection.getJarFile(JarURLConnection.java:89)
sun.misc.URLClassPath$JarLoader.getJarFile(URLClassPath.java:944)
sun.misc.URLClassPath$JarLoader.access$800(URLClassPath.java:801)
sun.misc.URLClassPath$JarLoader$1.run(URLClassPath.java:886)
sun.misc.URLClassPath$JarLoader$1.run(URLClassPath.java:879)
java.security.AccessController.doPrivileged(Native Method)
sun.misc.URLClassPath$JarLoader.ensureOpen(URLClassPath.java:878)
sun.misc.URLClassPath$JarLoader.(URLClassPath.java:829)
sun.misc.URLClassPath$3.run(URLClassPath.java:575)
sun.misc.URLClassPath$3.run(URLClassPath.java:565)
java.security.AccessController.doPrivileged(Native Method)
sun.misc.URLClassPath.getLoader(URLClassPath.java:564)
sun.misc.URLClassPath.getLoader(URLClassPath.java:529)
sun.misc.URLClassPath.getNextLoader(URLClassPath.java:494) => holding
Monitor(sun.misc.URLClassPath@929546251})
sun.misc.URLClassPath.findResource(URLClassPath.java:224)
java.net.URLClassLoader$2.run(URLClassLoader.java:572)
java.net.URLClassLoader$2.run(URLClassLoader.java:570)
java.security.AccessController.doPrivileged(Native Method)
java.net.URLClassLoader.findResource(URLClassLoader.java:569)
java.lang.ClassLoader.getResource(ClassLoader.java:1096)
java.lang.ClassLoader.getResource(ClassLoader.java:1091)
org.apache.hadoop.conf.Configuration.getResource(Configuration.java:2809)
org.apache.hadoop.conf.Configuration.getStreamReader(Configuration.java:3081)
org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:3040)
org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:3013)
org.apache.hadoop.conf.Configuration.loadProps(Configuration.java:2893) =>
holding Monitor(org.apache.hadoop.conf.Configuration@1190238736})
org.apache.hadoop.conf.Configuration.getProps(Configuration.java:2875) =>
holding Monitor(org.apache.hadoop.conf.Configuration@1190238736})
org.apache.hadoop.conf.Configuration.get(Configuration.java:1225)
org.apache.hadoop.conf.Configuration.getTrimmed(Configuration.java:1279)
org.apache.hadoop.conf.Configuration.getBoolean(Configuration.java:1686)
org.apache.hadoop.io.nativeio.NativeIO$POSIX.(NativeIO.java:334)
org.apache.hadoop.io.nativeio.NativeIO.initNative(Native Method)
org.apache.hadoop.io.nativeio.NativeIO.(NativeIO.java:831)
org.apache.hadoop.hdfs.shortcircuit.ShortCircuitShm.(ShortCircuitShm.java:469)
org.apache.hadoop.hdfs.shortcircuit.DfsClientShm.(DfsClientShm.java:70)
org.apache.hadoop.hdfs.shortcircuit.DfsClientShmManager$EndpointShmManager.requestNewShm(DfsClientShmManager.java:181)
[jira] [Created] (HADOOP-18911) DfsClientShmManager allocSlot hang on awaitUninterruptibly itself
konwu created HADOOP-18911: -- Summary: DfsClientShmManager allocSlot hang on awaitUninterruptibly itself Key: HADOOP-18911 URL: https://issues.apache.org/jira/browse/HADOOP-18911 Project: Hadoop Common Issue Type: Bug Components: hdfs-client Affects Versions: 3.3.1 Environment: hadoop 3.3.1 yarn 3.3.1 Reporter: konwu Spark task Driver Thread hang on awaitUninterruptibly thread stack like below: ```java sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitUninterruptibly(AbstractQueuedSynchronizer.java:1976) org.apache.hadoop.hdfs.shortcircuit.DfsClientShmManager$EndpointShmManager.allocSlot(DfsClientShmManager.java:244) org.apache.hadoop.hdfs.shortcircuit.DfsClientShmManager.allocSlot(DfsClientShmManager.java:417) org.apache.hadoop.hdfs.shortcircuit.ShortCircuitCache.allocShmSlot(ShortCircuitCache.java:1006) org.apache.hadoop.hdfs.client.impl.BlockReaderFactory.createShortCircuitReplicaInfo(BlockReaderFactory.java:535) org.apache.hadoop.hdfs.shortcircuit.ShortCircuitCache.create(ShortCircuitCache.java:786) org.apache.hadoop.hdfs.shortcircuit.ShortCircuitCache.fetchOrCreate(ShortCircuitCache.java:723) org.apache.hadoop.hdfs.client.impl.BlockReaderFactory.getBlockReaderLocal(BlockReaderFactory.java:483) org.apache.hadoop.hdfs.client.impl.BlockReaderFactory.build(BlockReaderFactory.java:360) org.apache.hadoop.hdfs.DFSInputStream.getBlockReader(DFSInputStream.java:755) org.apache.hadoop.hdfs.DFSInputStream.blockSeekTo(DFSInputStream.java:685) => holding Monitor(org.apache.hadoop.hdfs.DFSInputStream@59925309}) org.apache.hadoop.hdfs.DFSInputStream.readWithStrategy(DFSInputStream.java:884) => holding Monitor(org.apache.hadoop.hdfs.DFSInputStream@59925309}) org.apache.hadoop.hdfs.DFSInputStream.read(DFSInputStream.java:957) => holding Monitor(org.apache.hadoop.hdfs.DFSInputStream@59925309}) java.io.DataInputStream.read(DataInputStream.java:100) java.nio.file.Files.copy(Files.java:2908) java.nio.file.Files.copy(Files.java:3027) sun.net.www.protocol.jar.URLJarFile$1.run(URLJarFile.java:220) sun.net.www.protocol.jar.URLJarFile$1.run(URLJarFile.java:216) java.security.AccessController.doPrivileged(Native Method) sun.net.www.protocol.jar.URLJarFile.retrieve(URLJarFile.java:215) sun.net.www.protocol.jar.URLJarFile.getJarFile(URLJarFile.java:71) sun.net.www.protocol.jar.JarFileFactory.get(JarFileFactory.java:84) sun.net.www.protocol.jar.JarURLConnection.connect(JarURLConnection.java:122) sun.net.www.protocol.jar.JarURLConnection.getJarFile(JarURLConnection.java:89) sun.misc.URLClassPath$JarLoader.getJarFile(URLClassPath.java:944) sun.misc.URLClassPath$JarLoader.access$800(URLClassPath.java:801) sun.misc.URLClassPath$JarLoader$1.run(URLClassPath.java:886) sun.misc.URLClassPath$JarLoader$1.run(URLClassPath.java:879) java.security.AccessController.doPrivileged(Native Method) sun.misc.URLClassPath$JarLoader.ensureOpen(URLClassPath.java:878) sun.misc.URLClassPath$JarLoader.(URLClassPath.java:829) sun.misc.URLClassPath$3.run(URLClassPath.java:575) sun.misc.URLClassPath$3.run(URLClassPath.java:565) java.security.AccessController.doPrivileged(Native Method) sun.misc.URLClassPath.getLoader(URLClassPath.java:564) sun.misc.URLClassPath.getLoader(URLClassPath.java:529) sun.misc.URLClassPath.getNextLoader(URLClassPath.java:494) => holding Monitor(sun.misc.URLClassPath@929546251}) sun.misc.URLClassPath.findResource(URLClassPath.java:224) java.net.URLClassLoader$2.run(URLClassLoader.java:572) java.net.URLClassLoader$2.run(URLClassLoader.java:570) java.security.AccessController.doPrivileged(Native Method) java.net.URLClassLoader.findResource(URLClassLoader.java:569) java.lang.ClassLoader.getResource(ClassLoader.java:1096) java.lang.ClassLoader.getResource(ClassLoader.java:1091) org.apache.hadoop.conf.Configuration.getResource(Configuration.java:2809) org.apache.hadoop.conf.Configuration.getStreamReader(Configuration.java:3081) org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:3040) org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:3013) org.apache.hadoop.conf.Configuration.loadProps(Configuration.java:2893) => holding Monitor(org.apache.hadoop.conf.Configuration@1190238736}) org.apache.hadoop.conf.Configuration.getProps(Configuration.java:2875) => holding Monitor(org.apache.hadoop.conf.Configuration@1190238736}) org.apache.hadoop.conf.Configuration.get(Configuration.java:1225) org.apache.hadoop.conf.Configuration.getTrimmed(Configuration.java:1279) org.apache.hadoop.conf.Configuration.getBoolean(Configuration.java:1686) org.apache.hadoop.io.nativeio.NativeIO$POSIX.(NativeIO.java:334) org.apache.hadoop.io.nativeio.NativeIO.initNative(Native Method) org.apache.hadoop.io.nativeio.NativeIO.(NativeIO.java:831)
[GitHub] [hadoop] ferdelyi closed pull request #6027: YARN-11468. Zookeeper SSL/TLS support
ferdelyi closed pull request #6027: YARN-11468. Zookeeper SSL/TLS support URL: https://github.com/apache/hadoop/pull/6027 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18257) Analyzing S3A Audit Logs
[ https://issues.apache.org/jira/browse/HADOOP-18257?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17768659#comment-17768659 ] ASF GitHub Bot commented on HADOOP-18257: - mehakmeet commented on PR #6000: URL: https://github.com/apache/hadoop/pull/6000#issuecomment-1733516788 > Have you tested this on actual files? And specifically so many files... total size in GB's kind of scale testing? Have tested this but not at scale. Will do that. Example test: ``` ❯ bin/hadoop org.apache.hadoop.fs.s3a.audit.AuditTool s3a://mehakmeet-singh-data/logdir2/ s3a://mehakmeet-singh-data/logsdir/ 16:48:41,319 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 16:49:58,339 INFO mapreduce.S3AAuditLogMergerAndParser: Successfully generated avro data 16:49:58,839 INFO mapreduce.S3AAuditLogMergerAndParser: Successfully parsed :7547 audit logs and 6718 referrer headers logs in the logs 16:49:58,854 INFO impl.MetricsSystemImpl: Stopping s3a-file-system metrics system... 16:49:58,854 INFO impl.MetricsSystemImpl: s3a-file-system metrics system stopped. 16:49:58,854 INFO impl.MetricsSystemImpl: s3a-file-system metrics system shutdown complete. ``` Since we're reading each file serially per line, I would assume this would be alot slower in that scenario. Optimisation can be a follow-up patch. > If there are so many files for ex 1000, does it launch multiple mappers to process x files ny each mapper based on the splits? Not currently. Is that something we would have to write the logic off of, I'll have to check the code for it? Specifically for number of mappers maybe we could have a threshold of number of files and then paginate them based on that. > Analyzing S3A Audit Logs > - > > Key: HADOOP-18257 > URL: https://issues.apache.org/jira/browse/HADOOP-18257 > Project: Hadoop Common > Issue Type: Task > Components: fs/s3 >Reporter: Sravani Gadey >Assignee: Mehakmeet Singh >Priority: Major > Labels: pull-request-available > > The main aim is to analyze S3A Audit logs to give better insights in Hive and > Spark jobs. > Steps involved are: > * Merging audit log files containing huge number of audit logs collected > from a job containing various S3 requests. > * Parsing audit logs using regular expressions i.e., dividing them into key > value pairs. > * Converting the key value pairs into CSV file and AVRO file formats. > * Querying on data which would give better insights for different jobs. > * Visualizing the audit logs on Zeppelin or Jupyter notebook with graphs. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] mehakmeet commented on pull request #6000: HADOOP-18257. Merging and Parsing S3A audit logs into Avro format for analysis.
mehakmeet commented on PR #6000: URL: https://github.com/apache/hadoop/pull/6000#issuecomment-1733516788 > Have you tested this on actual files? And specifically so many files... total size in GB's kind of scale testing? Have tested this but not at scale. Will do that. Example test: ``` ❯ bin/hadoop org.apache.hadoop.fs.s3a.audit.AuditTool s3a://mehakmeet-singh-data/logdir2/ s3a://mehakmeet-singh-data/logsdir/ 16:48:41,319 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 16:49:58,339 INFO mapreduce.S3AAuditLogMergerAndParser: Successfully generated avro data 16:49:58,839 INFO mapreduce.S3AAuditLogMergerAndParser: Successfully parsed :7547 audit logs and 6718 referrer headers logs in the logs 16:49:58,854 INFO impl.MetricsSystemImpl: Stopping s3a-file-system metrics system... 16:49:58,854 INFO impl.MetricsSystemImpl: s3a-file-system metrics system stopped. 16:49:58,854 INFO impl.MetricsSystemImpl: s3a-file-system metrics system shutdown complete. ``` Since we're reading each file serially per line, I would assume this would be alot slower in that scenario. Optimisation can be a follow-up patch. > If there are so many files for ex 1000, does it launch multiple mappers to process x files ny each mapper based on the splits? Not currently. Is that something we would have to write the logic off of, I'll have to check the code for it? Specifically for number of mappers maybe we could have a threshold of number of files and then paginate them based on that. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18257) Analyzing S3A Audit Logs
[
https://issues.apache.org/jira/browse/HADOOP-18257?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17768655#comment-17768655
]
ASF GitHub Bot commented on HADOOP-18257:
-
mehakmeet commented on code in PR #6000:
URL: https://github.com/apache/hadoop/pull/6000#discussion_r1335767883
##
hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/audit/AuditTool.java:
##
@@ -0,0 +1,258 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.fs.s3a.audit;
+
+import java.io.Closeable;
+import java.io.IOException;
+import java.io.PrintWriter;
+import java.net.URI;
+import java.net.URISyntaxException;
+import java.util.Arrays;
+import java.util.List;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.conf.Configured;
+import org.apache.hadoop.fs.FileStatus;
+import org.apache.hadoop.fs.FileSystem;
+import org.apache.hadoop.fs.Path;
+import org.apache.hadoop.fs.s3a.audit.mapreduce.S3AAuditLogMergerAndParser;
+import org.apache.hadoop.util.ExitUtil;
+import org.apache.hadoop.util.Tool;
+import org.apache.hadoop.util.ToolRunner;
+
+import static
org.apache.hadoop.service.launcher.LauncherExitCodes.EXIT_COMMAND_ARGUMENT_ERROR;
+import static org.apache.hadoop.service.launcher.LauncherExitCodes.EXIT_FAIL;
+import static
org.apache.hadoop.service.launcher.LauncherExitCodes.EXIT_SUCCESS;
+
+/**
+ * AuditTool is a Command Line Interface.
+ * Its functionality is to parse the audit log files
+ * and generate avro file.
+ */
+public class AuditTool extends Configured implements Tool, Closeable {
+
+ private static final Logger LOG = LoggerFactory.getLogger(AuditTool.class);
+
+ private final S3AAuditLogMergerAndParser s3AAuditLogMergerAndParser =
+ new S3AAuditLogMergerAndParser();
+
+ /**
+ * Name of this tool: {@value}.
+ */
+ public static final String AUDIT_TOOL =
+ "org.apache.hadoop.fs.s3a.audit.AuditTool";
+
+ /**
+ * Purpose of this tool: {@value}.
+ */
+ public static final String PURPOSE =
+ "\n\nUSAGE:\nMerge, parse audit log files and convert into avro file "
+ + "for "
+ + "better "
+ + "visualization";
+
+ // Exit codes
+ private static final int SUCCESS = EXIT_SUCCESS;
+ private static final int FAILURE = EXIT_FAIL;
+ private static final int INVALID_ARGUMENT = EXIT_COMMAND_ARGUMENT_ERROR;
+
+ private static final String USAGE =
+ "bin/hadoop " + "Class" + " DestinationPath" + " SourcePath" + "\n" +
+ "bin/hadoop " + AUDIT_TOOL + " s3a://BUCKET" + " s3a://BUCKET" +
"\n";
+
+ private PrintWriter out;
+
+ public AuditTool() {
+super();
+ }
+
+ /**
+ * Tells us the usage of the AuditTool by commands.
+ *
+ * @return the string USAGE
+ */
+ public String getUsage() {
+return USAGE + PURPOSE;
+ }
+
+ public String getName() {
+return AUDIT_TOOL;
+ }
+
+ /**
+ * This run method in AuditTool takes source and destination path of bucket,
+ * and check if there are directories and pass these paths to merge and
+ * parse audit log files.
+ *
+ * @param args argument list
+ * @return SUCCESS i.e, '0', which is an exit code
+ * @throws Exception on any failure.
+ */
+ @Override
+ public int run(String[] args) throws Exception {
+List paths = Arrays.asList(args);
+if(paths.size() == 2) {
+ // Path of audit log files
+ Path logsPath = new Path(paths.get(1));
+ // Path of destination directory
+ Path destPath = new Path(paths.get(0));
+
+ // Setting the file system
+ URI fsURI = new URI(logsPath.toString());
+ FileSystem fileSystem = FileSystem.get(fsURI, new Configuration());
+
+ FileStatus fileStatus = fileSystem.getFileStatus(logsPath);
+ if (fileStatus.isFile()) {
+errorln("Expecting a directory, but " + logsPath.getName() + " is a"
++ " file which was passed as an argument");
+throw invalidArgs(
+"Expecting a directory, but " + logsPath.getName() + " is a"
++ " file which was passed as an argument");
+ }
+
[GitHub] [hadoop] mehakmeet commented on a diff in pull request #6000: HADOOP-18257. Merging and Parsing S3A audit logs into Avro format for analysis.
mehakmeet commented on code in PR #6000:
URL: https://github.com/apache/hadoop/pull/6000#discussion_r1335767883
##
hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/audit/AuditTool.java:
##
@@ -0,0 +1,258 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.fs.s3a.audit;
+
+import java.io.Closeable;
+import java.io.IOException;
+import java.io.PrintWriter;
+import java.net.URI;
+import java.net.URISyntaxException;
+import java.util.Arrays;
+import java.util.List;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.conf.Configured;
+import org.apache.hadoop.fs.FileStatus;
+import org.apache.hadoop.fs.FileSystem;
+import org.apache.hadoop.fs.Path;
+import org.apache.hadoop.fs.s3a.audit.mapreduce.S3AAuditLogMergerAndParser;
+import org.apache.hadoop.util.ExitUtil;
+import org.apache.hadoop.util.Tool;
+import org.apache.hadoop.util.ToolRunner;
+
+import static
org.apache.hadoop.service.launcher.LauncherExitCodes.EXIT_COMMAND_ARGUMENT_ERROR;
+import static org.apache.hadoop.service.launcher.LauncherExitCodes.EXIT_FAIL;
+import static
org.apache.hadoop.service.launcher.LauncherExitCodes.EXIT_SUCCESS;
+
+/**
+ * AuditTool is a Command Line Interface.
+ * Its functionality is to parse the audit log files
+ * and generate avro file.
+ */
+public class AuditTool extends Configured implements Tool, Closeable {
+
+ private static final Logger LOG = LoggerFactory.getLogger(AuditTool.class);
+
+ private final S3AAuditLogMergerAndParser s3AAuditLogMergerAndParser =
+ new S3AAuditLogMergerAndParser();
+
+ /**
+ * Name of this tool: {@value}.
+ */
+ public static final String AUDIT_TOOL =
+ "org.apache.hadoop.fs.s3a.audit.AuditTool";
+
+ /**
+ * Purpose of this tool: {@value}.
+ */
+ public static final String PURPOSE =
+ "\n\nUSAGE:\nMerge, parse audit log files and convert into avro file "
+ + "for "
+ + "better "
+ + "visualization";
+
+ // Exit codes
+ private static final int SUCCESS = EXIT_SUCCESS;
+ private static final int FAILURE = EXIT_FAIL;
+ private static final int INVALID_ARGUMENT = EXIT_COMMAND_ARGUMENT_ERROR;
+
+ private static final String USAGE =
+ "bin/hadoop " + "Class" + " DestinationPath" + " SourcePath" + "\n" +
+ "bin/hadoop " + AUDIT_TOOL + " s3a://BUCKET" + " s3a://BUCKET" +
"\n";
+
+ private PrintWriter out;
+
+ public AuditTool() {
+super();
+ }
+
+ /**
+ * Tells us the usage of the AuditTool by commands.
+ *
+ * @return the string USAGE
+ */
+ public String getUsage() {
+return USAGE + PURPOSE;
+ }
+
+ public String getName() {
+return AUDIT_TOOL;
+ }
+
+ /**
+ * This run method in AuditTool takes source and destination path of bucket,
+ * and check if there are directories and pass these paths to merge and
+ * parse audit log files.
+ *
+ * @param args argument list
+ * @return SUCCESS i.e, '0', which is an exit code
+ * @throws Exception on any failure.
+ */
+ @Override
+ public int run(String[] args) throws Exception {
+List paths = Arrays.asList(args);
+if(paths.size() == 2) {
+ // Path of audit log files
+ Path logsPath = new Path(paths.get(1));
+ // Path of destination directory
+ Path destPath = new Path(paths.get(0));
+
+ // Setting the file system
+ URI fsURI = new URI(logsPath.toString());
+ FileSystem fileSystem = FileSystem.get(fsURI, new Configuration());
+
+ FileStatus fileStatus = fileSystem.getFileStatus(logsPath);
+ if (fileStatus.isFile()) {
+errorln("Expecting a directory, but " + logsPath.getName() + " is a"
++ " file which was passed as an argument");
+throw invalidArgs(
+"Expecting a directory, but " + logsPath.getName() + " is a"
++ " file which was passed as an argument");
+ }
+ FileStatus fileStatus1 = fileSystem.getFileStatus(destPath);
+ if (fileStatus1.isFile()) {
+errorln("Expecting a directory, but " + destPath.getName() + " is a"
++ " file which was passed as an argument");
+
[jira] [Commented] (HADOOP-18868) Optimize the configuration and use of callqueue overflow trigger failover
[ https://issues.apache.org/jira/browse/HADOOP-18868?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17768653#comment-17768653 ] ASF GitHub Bot commented on HADOOP-18868: - haiyang1987 commented on PR #5998: URL: https://github.com/apache/hadoop/pull/5998#issuecomment-1733494535 Hi Sirs @Hexiaoqiao @goiri can you help me take a look at this PR when you have time? Thank you very much~ > Optimize the configuration and use of callqueue overflow trigger failover > - > > Key: HADOOP-18868 > URL: https://issues.apache.org/jira/browse/HADOOP-18868 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Haiyang Hu >Assignee: Haiyang Hu >Priority: Major > Labels: pull-request-available > > In HADOOP-16268 implemented enable callqueue overflow trigger failover when > RPC call queue is filled for hdfs router servers . > the ticket will do the following things： > * provide default properties for callqueue.overflow.trigger.failover such > that if properties with port is not configured, we can fallback to default > property (port-less) > * support refreshCallQueue to update callqueue.overflow.trigger.failover > * update callqueue.overflow.trigger.failover description information in > core-site.xml -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] haiyang1987 commented on pull request #5998: HADOOP-18868. Optimize the configuration and use of callqueue overflow trigger failover
haiyang1987 commented on PR #5998: URL: https://github.com/apache/hadoop/pull/5998#issuecomment-1733494535 Hi Sirs @Hexiaoqiao @goiri can you help me take a look at this PR when you have time? Thank you very much~ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] haiyang1987 opened a new pull request, #6112: HDFS-17205. HdfsServerConstants.MIN_BLOCKS_FOR_WRITE should be configurable
haiyang1987 opened a new pull request, #6112:
URL: https://github.com/apache/hadoop/pull/6112
### Description of PR
https://issues.apache.org/jira/browse/HDFS-17205.
Current allocate new block , the NameNode will choose datanode and choose a
good storage of given storage type from datanode, the specific calling code is
DatanodeDescriptor#chooseStorage4Block, here will calculate the space required
for write operations,
requiredSize = blockSize * HdfsServerConstants.MIN_BLOCKS_FOR_WRITE(default
is 1).
```
public DatanodeStorageInfo chooseStorage4Block(StorageType t,
long blockSize) {
final long requiredSize =
blockSize * HdfsServerConstants.MIN_BLOCKS_FOR_WRITE;
final long scheduledSize = blockSize * getBlocksScheduled(t);
long remaining = 0;
DatanodeStorageInfo storage = null;
for (DatanodeStorageInfo s : getStorageInfos()) {
if (s.getState() == State.NORMAL && s.getStorageType() == t) {
if (storage == null) {
storage = s;
}
long r = s.getRemaining();
if (r >= requiredSize) {
remaining += r;
}
}
}
if (requiredSize > remaining - scheduledSize) {
BlockPlacementPolicy.LOG.debug(
"The node {} does not have enough {} space (required={},"
+ " scheduled={}, remaining={}).",
this, t, requiredSize, scheduledSize, remaining);
return null;
}
return storage;
}
```
But when multiple NameSpaces select the storage of the same datanode to
write blocks at the same time.
In extreme cases, if there is only one block size left in the current
storage, there will be a situation where there is not enough free space for the
writer to write data.
log similar to the following appears:
`The volume [file:/disk1/] with the available space (=21129618 B) is less
than the block size (=268435456 B). `
In order to avoid this case, consider
HdfsServerConstants.MIN_BLOCKS_FOR_WRITE should be configurable, and the
parameters can be adjusted in larger clusters.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18257) Analyzing S3A Audit Logs
[
https://issues.apache.org/jira/browse/HADOOP-18257?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17768647#comment-17768647
]
ASF GitHub Bot commented on HADOOP-18257:
-
mehakmeet commented on code in PR #6000:
URL: https://github.com/apache/hadoop/pull/6000#discussion_r1335750686
##
hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/audit/AuditTool.java:
##
@@ -0,0 +1,258 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.fs.s3a.audit;
+
+import java.io.Closeable;
+import java.io.IOException;
+import java.io.PrintWriter;
+import java.net.URI;
+import java.net.URISyntaxException;
+import java.util.Arrays;
+import java.util.List;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.conf.Configured;
+import org.apache.hadoop.fs.FileStatus;
+import org.apache.hadoop.fs.FileSystem;
+import org.apache.hadoop.fs.Path;
+import org.apache.hadoop.fs.s3a.audit.mapreduce.S3AAuditLogMergerAndParser;
+import org.apache.hadoop.util.ExitUtil;
+import org.apache.hadoop.util.Tool;
+import org.apache.hadoop.util.ToolRunner;
+
+import static
org.apache.hadoop.service.launcher.LauncherExitCodes.EXIT_COMMAND_ARGUMENT_ERROR;
+import static org.apache.hadoop.service.launcher.LauncherExitCodes.EXIT_FAIL;
+import static
org.apache.hadoop.service.launcher.LauncherExitCodes.EXIT_SUCCESS;
+
+/**
+ * AuditTool is a Command Line Interface.
+ * Its functionality is to parse the audit log files
+ * and generate avro file.
+ */
+public class AuditTool extends Configured implements Tool, Closeable {
+
+ private static final Logger LOG = LoggerFactory.getLogger(AuditTool.class);
+
+ private final S3AAuditLogMergerAndParser s3AAuditLogMergerAndParser =
+ new S3AAuditLogMergerAndParser();
+
+ /**
+ * Name of this tool: {@value}.
+ */
+ public static final String AUDIT_TOOL =
+ "org.apache.hadoop.fs.s3a.audit.AuditTool";
+
+ /**
+ * Purpose of this tool: {@value}.
+ */
+ public static final String PURPOSE =
+ "\n\nUSAGE:\nMerge, parse audit log files and convert into avro file "
+ + "for "
+ + "better "
+ + "visualization";
+
+ // Exit codes
+ private static final int SUCCESS = EXIT_SUCCESS;
+ private static final int FAILURE = EXIT_FAIL;
+ private static final int INVALID_ARGUMENT = EXIT_COMMAND_ARGUMENT_ERROR;
+
+ private static final String USAGE =
+ "bin/hadoop " + "Class" + " DestinationPath" + " SourcePath" + "\n" +
Review Comment:
this is to essentially define the command in verbose vs an example. This is
what we'll see
```
❯ bin/hadoop org.apache.hadoop.fs.s3a.audit.AuditTool
bin/hadoop Class DestinationPath SourcePath
bin/hadoop org.apache.hadoop.fs.s3a.audit.AuditTool s3a://BUCKET s3a://BUCKET
```
I'll simplify this, it may be confusing
##
hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/audit/AuditTool.java:
##
@@ -0,0 +1,258 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.fs.s3a.audit;
+
+import java.io.Closeable;
+import java.io.IOException;
+import java.io.PrintWriter;
+import java.net.URI;
+import java.net.URISyntaxException;
+import java.util.Arrays;
+import java.util.List;
+
+import org.slf4j.Logger;
+import
[GitHub] [hadoop] mehakmeet commented on a diff in pull request #6000: HADOOP-18257. Merging and Parsing S3A audit logs into Avro format for analysis.
mehakmeet commented on code in PR #6000:
URL: https://github.com/apache/hadoop/pull/6000#discussion_r1335750686
##
hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/audit/AuditTool.java:
##
@@ -0,0 +1,258 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.fs.s3a.audit;
+
+import java.io.Closeable;
+import java.io.IOException;
+import java.io.PrintWriter;
+import java.net.URI;
+import java.net.URISyntaxException;
+import java.util.Arrays;
+import java.util.List;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.conf.Configured;
+import org.apache.hadoop.fs.FileStatus;
+import org.apache.hadoop.fs.FileSystem;
+import org.apache.hadoop.fs.Path;
+import org.apache.hadoop.fs.s3a.audit.mapreduce.S3AAuditLogMergerAndParser;
+import org.apache.hadoop.util.ExitUtil;
+import org.apache.hadoop.util.Tool;
+import org.apache.hadoop.util.ToolRunner;
+
+import static
org.apache.hadoop.service.launcher.LauncherExitCodes.EXIT_COMMAND_ARGUMENT_ERROR;
+import static org.apache.hadoop.service.launcher.LauncherExitCodes.EXIT_FAIL;
+import static
org.apache.hadoop.service.launcher.LauncherExitCodes.EXIT_SUCCESS;
+
+/**
+ * AuditTool is a Command Line Interface.
+ * Its functionality is to parse the audit log files
+ * and generate avro file.
+ */
+public class AuditTool extends Configured implements Tool, Closeable {
+
+ private static final Logger LOG = LoggerFactory.getLogger(AuditTool.class);
+
+ private final S3AAuditLogMergerAndParser s3AAuditLogMergerAndParser =
+ new S3AAuditLogMergerAndParser();
+
+ /**
+ * Name of this tool: {@value}.
+ */
+ public static final String AUDIT_TOOL =
+ "org.apache.hadoop.fs.s3a.audit.AuditTool";
+
+ /**
+ * Purpose of this tool: {@value}.
+ */
+ public static final String PURPOSE =
+ "\n\nUSAGE:\nMerge, parse audit log files and convert into avro file "
+ + "for "
+ + "better "
+ + "visualization";
+
+ // Exit codes
+ private static final int SUCCESS = EXIT_SUCCESS;
+ private static final int FAILURE = EXIT_FAIL;
+ private static final int INVALID_ARGUMENT = EXIT_COMMAND_ARGUMENT_ERROR;
+
+ private static final String USAGE =
+ "bin/hadoop " + "Class" + " DestinationPath" + " SourcePath" + "\n" +
Review Comment:
this is to essentially define the command in verbose vs an example. This is
what we'll see
```
❯ bin/hadoop org.apache.hadoop.fs.s3a.audit.AuditTool
bin/hadoop Class DestinationPath SourcePath
bin/hadoop org.apache.hadoop.fs.s3a.audit.AuditTool s3a://BUCKET s3a://BUCKET
```
I'll simplify this, it may be confusing
##
hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/audit/AuditTool.java:
##
@@ -0,0 +1,258 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.fs.s3a.audit;
+
+import java.io.Closeable;
+import java.io.IOException;
+import java.io.PrintWriter;
+import java.net.URI;
+import java.net.URISyntaxException;
+import java.util.Arrays;
+import java.util.List;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.conf.Configured;
+import org.apache.hadoop.fs.FileStatus;
+import org.apache.hadoop.fs.FileSystem;
+import org.apache.hadoop.fs.Path;
+import
[GitHub] [hadoop] hadoop-yetus commented on pull request #6016: YARN-9048. Add znode hierarchy in Federation ZK State Store.
hadoop-yetus commented on PR #6016:
URL: https://github.com/apache/hadoop/pull/6016#issuecomment-1733325918
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 41s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available.
|
| +0 :ok: | xmllint | 0m 1s | | xmllint was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 3 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 14m 22s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 31m 43s | | trunk passed |
| +1 :green_heart: | compile | 2m 36s | | trunk passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | compile | 2m 26s | | trunk passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | checkstyle | 1m 24s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 46s | | trunk passed |
| +1 :green_heart: | javadoc | 1m 45s | | trunk passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 1m 32s | | trunk passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | spotbugs | 3m 26s | | trunk passed |
| +1 :green_heart: | shadedclient | 34m 20s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 34s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 1m 26s | | the patch passed |
| +1 :green_heart: | compile | 2m 28s | | the patch passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javac | 2m 28s | | the patch passed |
| +1 :green_heart: | compile | 2m 16s | | the patch passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | javac | 2m 16s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| -0 :warning: | checkstyle | 1m 10s |
[/results-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6016/12/artifact/out/results-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server.txt)
| hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server: The patch generated 1
new + 23 unchanged - 0 fixed = 24 total (was 23) |
| +1 :green_heart: | mvnsite | 1m 30s | | the patch passed |
| +1 :green_heart: | javadoc | 1m 23s | | the patch passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 1m 16s | | the patch passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | spotbugs | 3m 31s | | the patch passed |
| +1 :green_heart: | shadedclient | 34m 36s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 3m 25s | | hadoop-yarn-server-common in
the patch passed. |
| +1 :green_heart: | unit | 100m 53s | |
hadoop-yarn-server-resourcemanager in the patch passed. |
| +1 :green_heart: | asflicense | 0m 41s | | The patch does not
generate ASF License warnings. |
| | | 254m 58s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6016/12/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/6016 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets xmllint |
| uname | Linux 8efd9e15d0c5 4.15.0-213-generic #224-Ubuntu SMP Mon Jun 19
13:30:12 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 753e444c762a52ff4a9d00ab02a55460c58bc07a |
| Default Java | Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6016/12/testReport/ |
| Max. process+thread count | 981 (vs.
[jira] [Commented] (HADOOP-18829) s3a prefetch LRU cache eviction metric
[
https://issues.apache.org/jira/browse/HADOOP-18829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17768605#comment-17768605
]
ASF GitHub Bot commented on HADOOP-18829:
-
hadoop-yetus commented on PR #6111:
URL: https://github.com/apache/hadoop/pull/6111#issuecomment-1733240527
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 7m 32s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 1s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 4 new or modified test files. |
_ branch-3.3 Compile Tests _ |
| +0 :ok: | mvndep | 13m 43s | | Maven dependency ordering for branch |
| -1 :x: | mvninstall | 35m 32s |
[/branch-mvninstall-root.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6111/1/artifact/out/branch-mvninstall-root.txt)
| root in branch-3.3 failed. |
| +1 :green_heart: | compile | 19m 4s | | branch-3.3 passed |
| +1 :green_heart: | checkstyle | 2m 53s | | branch-3.3 passed |
| +1 :green_heart: | mvnsite | 2m 52s | | branch-3.3 passed |
| +1 :green_heart: | javadoc | 1m 52s | | branch-3.3 passed |
| +1 :green_heart: | spotbugs | 4m 9s | | branch-3.3 passed |
| -1 :x: | shadedclient | 37m 36s | | branch has errors when building
and testing our client artifacts. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 33s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 1m 43s | | the patch passed |
| +1 :green_heart: | compile | 18m 37s | | the patch passed |
| +1 :green_heart: | javac | 18m 37s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 2m 42s | | the patch passed |
| +1 :green_heart: | mvnsite | 2m 52s | | the patch passed |
| +1 :green_heart: | javadoc | 1m 45s | | the patch passed |
| +1 :green_heart: | spotbugs | 4m 23s | | the patch passed |
| -1 :x: | shadedclient | 38m 40s | | patch has errors when building
and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 18m 31s | | hadoop-common in the patch
passed. |
| +1 :green_heart: | unit | 2m 55s | | hadoop-aws in the patch passed.
|
| +1 :green_heart: | asflicense | 1m 8s | | The patch does not
generate ASF License warnings. |
| | | 225m 38s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6111/1/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/6111 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux 3acb0ff4d744 4.15.0-213-generic #224-Ubuntu SMP Mon Jun 19
13:30:12 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | branch-3.3 / e9679a2161cfb9d327254555850828f9cb6fe5b2 |
| Default Java | Private Build-1.8.0_362-8u372-ga~us1-0ubuntu1~18.04-b09 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6111/1/testReport/ |
| Max. process+thread count | 3137 (vs. ulimit of 5500) |
| modules | C: hadoop-common-project/hadoop-common hadoop-tools/hadoop-aws
U: . |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6111/1/console |
| versions | git=2.17.1 maven=3.6.0 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was automatically generated.
> s3a prefetch LRU cache eviction metric
> --
>
> Key: HADOOP-18829
> URL: https://issues.apache.org/jira/browse/HADOOP-18829
> Project: Hadoop Common
> Issue Type: Sub-task
>Reporter: Viraj Jasani
>Assignee: Viraj Jasani
>Priority: Major
> Labels: pull-request-available
>
> Follow-up from HADOOP-18291:
> Add new IO statistics metric to capture s3a prefetch LRU cache eviction.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[GitHub] [hadoop] hadoop-yetus commented on pull request #6111: HADOOP-18829. S3A prefetch LRU cache eviction metrics (#5893)
hadoop-yetus commented on PR #6111:
URL: https://github.com/apache/hadoop/pull/6111#issuecomment-1733240527
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 7m 32s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 1s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 4 new or modified test files. |
_ branch-3.3 Compile Tests _ |
| +0 :ok: | mvndep | 13m 43s | | Maven dependency ordering for branch |
| -1 :x: | mvninstall | 35m 32s |
[/branch-mvninstall-root.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6111/1/artifact/out/branch-mvninstall-root.txt)
| root in branch-3.3 failed. |
| +1 :green_heart: | compile | 19m 4s | | branch-3.3 passed |
| +1 :green_heart: | checkstyle | 2m 53s | | branch-3.3 passed |
| +1 :green_heart: | mvnsite | 2m 52s | | branch-3.3 passed |
| +1 :green_heart: | javadoc | 1m 52s | | branch-3.3 passed |
| +1 :green_heart: | spotbugs | 4m 9s | | branch-3.3 passed |
| -1 :x: | shadedclient | 37m 36s | | branch has errors when building
and testing our client artifacts. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 33s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 1m 43s | | the patch passed |
| +1 :green_heart: | compile | 18m 37s | | the patch passed |
| +1 :green_heart: | javac | 18m 37s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 2m 42s | | the patch passed |
| +1 :green_heart: | mvnsite | 2m 52s | | the patch passed |
| +1 :green_heart: | javadoc | 1m 45s | | the patch passed |
| +1 :green_heart: | spotbugs | 4m 23s | | the patch passed |
| -1 :x: | shadedclient | 38m 40s | | patch has errors when building
and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 18m 31s | | hadoop-common in the patch
passed. |
| +1 :green_heart: | unit | 2m 55s | | hadoop-aws in the patch passed.
|
| +1 :green_heart: | asflicense | 1m 8s | | The patch does not
generate ASF License warnings. |
| | | 225m 38s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6111/1/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/6111 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux 3acb0ff4d744 4.15.0-213-generic #224-Ubuntu SMP Mon Jun 19
13:30:12 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | branch-3.3 / e9679a2161cfb9d327254555850828f9cb6fe5b2 |
| Default Java | Private Build-1.8.0_362-8u372-ga~us1-0ubuntu1~18.04-b09 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6111/1/testReport/ |
| Max. process+thread count | 3137 (vs. ulimit of 5500) |
| modules | C: hadoop-common-project/hadoop-common hadoop-tools/hadoop-aws
U: . |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6111/1/console |
| versions | git=2.17.1 maven=3.6.0 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18873) ABFS: AbfsOutputStream doesnt close DataBlocks object.
[ https://issues.apache.org/jira/browse/HADOOP-18873?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17768594#comment-17768594 ] ASF GitHub Bot commented on HADOOP-18873: - mehakmeet merged PR #6105: URL: https://github.com/apache/hadoop/pull/6105 > ABFS: AbfsOutputStream doesnt close DataBlocks object. > -- > > Key: HADOOP-18873 > URL: https://issues.apache.org/jira/browse/HADOOP-18873 > Project: Hadoop Common > Issue Type: Sub-task >Affects Versions: 3.3.4 >Reporter: Pranav Saxena >Assignee: Pranav Saxena >Priority: Major > Labels: pull-request-available > Fix For: 3.3.4 > > > AbfsOutputStream doesnt close the dataBlock object created for the upload. > What is the implication of not doing that: > DataBlocks has three implementations: > # ByteArrayBlock > ## This creates an object of DataBlockByteArrayOutputStream (child of > ByteArrayOutputStream: wrapper arround byte-arrray for populating, reading > the array. > ## This gets GCed. > # ByteBufferBlock: > ## There is a defined *DirectBufferPool* from which it tries to request the > directBuffer. > ## If nothing in the pool, a new directBuffer is created. > ## the `close` method on the this object has the responsiblity of returning > back the buffer to pool so it can be reused. > ## Since we are not calling the `close`: > ### The pool is rendered of less use, since each request creates a new > directBuffer from memory. > ### All the object can be GCed and the direct-memory allocated may be > returned on the GC. What if the process crashes, the memory never goes back > and cause memory issue on the machine. > # DiskBlock: > ## This creates a file on disk on which the data-to-upload is written. This > file gets deleted in startUpload().close(). > > startUpload() gives an object of BlockUploadData which gives method of > `toByteArray()` which is used in abfsOutputStream to get the byteArray in the > dataBlock. > > Method which uses the DataBlock object: > https://github.com/apache/hadoop/blob/fac7d26c5d7f791565cc3ab45d079e2cca725f95/hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsOutputStream.java#L298 -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] mehakmeet merged pull request #6105: HADOOP-18873. ABFS: AbfsOutputStream doesnt close DataBlocks object. [branch-3.3]
mehakmeet merged PR #6105: URL: https://github.com/apache/hadoop/pull/6105 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #6083: YARN-8862. [BackPort] [GPG] Add Yarn Registry cleanup in ApplicationCleaner.
hadoop-yetus commented on PR #6083:
URL: https://github.com/apache/hadoop/pull/6083#issuecomment-1733147498
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 1m 7s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available.
|
| +0 :ok: | xmllint | 0m 1s | | xmllint was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 5 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 14m 5s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 36m 28s | | trunk passed |
| +1 :green_heart: | compile | 2m 42s | | trunk passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | compile | 2m 20s | | trunk passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | checkstyle | 1m 22s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 35s | | trunk passed |
| +1 :green_heart: | javadoc | 1m 37s | | trunk passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 1m 25s | | trunk passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | spotbugs | 2m 57s | | trunk passed |
| +1 :green_heart: | shadedclient | 38m 0s | | branch has no errors
when building and testing our client artifacts. |
| -0 :warning: | patch | 38m 21s | | Used diff version of patch file.
Binary files and potentially other changes not applied. Please rebase and
squash commits if necessary. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 28s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 1m 13s | | the patch passed |
| +1 :green_heart: | compile | 2m 29s | | the patch passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javac | 2m 29s | | the patch passed |
| +1 :green_heart: | compile | 2m 15s | | the patch passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | javac | 2m 15s | | the patch passed |
| -1 :x: | blanks | 0m 0s |
[/blanks-eol.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6083/4/artifact/out/blanks-eol.txt)
| The patch has 1 line(s) that end in blanks. Use git apply --whitespace=fix
<>. Refer https://git-scm.com/docs/git-apply |
| +1 :green_heart: | checkstyle | 1m 13s | | the patch passed |
| +1 :green_heart: | mvnsite | 1m 19s | | the patch passed |
| +1 :green_heart: | javadoc | 1m 14s | | the patch passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 1m 8s | | the patch passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | spotbugs | 3m 9s | | the patch passed |
| +1 :green_heart: | shadedclient | 38m 3s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 3m 18s | | hadoop-yarn-server-common in
the patch passed. |
| +1 :green_heart: | unit | 0m 27s | | hadoop-yarn-server-router in
the patch passed. |
| +1 :green_heart: | unit | 0m 49s | |
hadoop-yarn-server-globalpolicygenerator in the patch passed. |
| +1 :green_heart: | asflicense | 0m 34s | | The patch does not
generate ASF License warnings. |
| | | 167m 30s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6083/4/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/6083 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets xmllint |
| uname | Linux 1e219823dc45 4.15.0-213-generic #224-Ubuntu SMP Mon Jun 19
13:30:12 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / a02824169e54e87859c6a2a7e572763555c977d5 |
| Default Java | Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
[GitHub] [hadoop] lfrancke commented on pull request #6108: HDFS-17181 WebHDFS: Route all CREATE requests to the BlockManager
lfrancke commented on PR #6108: URL: https://github.com/apache/hadoop/pull/6108#issuecomment-1733108376 All test failures seem unrelated and I didn't add any new tests because the code seems to be covered by tests already. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #6055: YARN-11547. [Federation] Router Supports Remove individual application records from FederationStateStore.
hadoop-yetus commented on PR #6055:
URL: https://github.com/apache/hadoop/pull/6055#issuecomment-1733030176
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 30s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 1 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 32m 11s | | trunk passed |
| +1 :green_heart: | compile | 0m 26s | | trunk passed with JDK
Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04 |
| +1 :green_heart: | compile | 0m 24s | | trunk passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | checkstyle | 0m 25s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 27s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 30s | | trunk passed with JDK
Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04 |
| +1 :green_heart: | javadoc | 0m 25s | | trunk passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | spotbugs | 0m 46s | | trunk passed |
| +1 :green_heart: | shadedclient | 20m 53s | | branch has no errors
when building and testing our client artifacts. |
| -0 :warning: | patch | 21m 11s | | Used diff version of patch file.
Binary files and potentially other changes not applied. Please rebase and
squash commits if necessary. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 18s | | the patch passed |
| +1 :green_heart: | compile | 0m 18s | | the patch passed with JDK
Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04 |
| +1 :green_heart: | javac | 0m 18s | | the patch passed |
| +1 :green_heart: | compile | 0m 17s | | the patch passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | javac | 0m 17s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 0m 13s | | the patch passed |
| +1 :green_heart: | mvnsite | 0m 18s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 18s | | the patch passed with JDK
Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04 |
| +1 :green_heart: | javadoc | 0m 17s | | the patch passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | spotbugs | 0m 40s | | the patch passed |
| +1 :green_heart: | shadedclient | 20m 44s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 0m 23s | | hadoop-yarn-server-router in
the patch passed. |
| +1 :green_heart: | asflicense | 0m 29s | | The patch does not
generate ASF License warnings. |
| | | 84m 27s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6055/12/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/6055 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux 2b3fcb3dd830 4.15.0-213-generic #224-Ubuntu SMP Mon Jun 19
13:30:12 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 14675c309afb1cd79a16c0e260237cdd0bce0c6a |
| Default Java | Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6055/12/testReport/ |
| Max. process+thread count | 676 (vs. ulimit of 5500) |
| modules | C:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router U:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6055/12/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated
[jira] [Commented] (HADOOP-18883) Expect-100 JDK bug resolution: prevent multiple server calls
[ https://issues.apache.org/jira/browse/HADOOP-18883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17768542#comment-17768542 ] ASF GitHub Bot commented on HADOOP-18883: - saxenapranav commented on PR #6022: URL: https://github.com/apache/hadoop/pull/6022#issuecomment-1732987230 @steveloughran @mehakmeet , requesting your review please. Thank you so much. > Expect-100 JDK bug resolution: prevent multiple server calls > > > Key: HADOOP-18883 > URL: https://issues.apache.org/jira/browse/HADOOP-18883 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/azure >Reporter: Pranav Saxena >Assignee: Pranav Saxena >Priority: Major > Fix For: 3.4.0 > > > This is inline to JDK bug: [https://bugs.openjdk.org/browse/JDK-8314978]. > > With the current implementation of HttpURLConnection if server rejects the > “Expect 100-continue” then there will be ‘java.net.ProtocolException’ will be > thrown from 'expect100Continue()' method. > After the exception thrown, If we call any other method on the same instance > (ex getHeaderField(), or getHeaderFields()). They will internally call > getOuputStream() which invokes writeRequests(), which make the actual server > call. > In the AbfsHttpOperation, after sendRequest() we call processResponse() > method from AbfsRestOperation. Even if the conn.getOutputStream() fails due > to expect-100 error, we consume the exception and let the code go ahead. So, > we can have getHeaderField() / getHeaderFields() / getHeaderFieldLong() which > will be triggered after getOutputStream is failed. These invocation will lead > to server calls. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-18883) Expect-100 JDK bug resolution: prevent multiple server calls
[ https://issues.apache.org/jira/browse/HADOOP-18883?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HADOOP-18883: Labels: pull-request-available (was: ) > Expect-100 JDK bug resolution: prevent multiple server calls > > > Key: HADOOP-18883 > URL: https://issues.apache.org/jira/browse/HADOOP-18883 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/azure >Reporter: Pranav Saxena >Assignee: Pranav Saxena >Priority: Major > Labels: pull-request-available > Fix For: 3.4.0 > > > This is inline to JDK bug: [https://bugs.openjdk.org/browse/JDK-8314978]. > > With the current implementation of HttpURLConnection if server rejects the > “Expect 100-continue” then there will be ‘java.net.ProtocolException’ will be > thrown from 'expect100Continue()' method. > After the exception thrown, If we call any other method on the same instance > (ex getHeaderField(), or getHeaderFields()). They will internally call > getOuputStream() which invokes writeRequests(), which make the actual server > call. > In the AbfsHttpOperation, after sendRequest() we call processResponse() > method from AbfsRestOperation. Even if the conn.getOutputStream() fails due > to expect-100 error, we consume the exception and let the code go ahead. So, > we can have getHeaderField() / getHeaderFields() / getHeaderFieldLong() which > will be triggered after getOutputStream is failed. These invocation will lead > to server calls. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] saxenapranav commented on pull request #6022: HADOOP-18883. [ABFS]: Expect-100 JDK bug resolution: prevent multiple server calls
saxenapranav commented on PR #6022: URL: https://github.com/apache/hadoop/pull/6022#issuecomment-1732987230 @steveloughran @mehakmeet , requesting your review please. Thank you so much. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
