[GitHub] [hadoop] hadoop-yetus commented on pull request #5546: HDFS-16975. FileWithSnapshotFeature.isCurrentFileDeleted is not reloaded from FSImage.
hadoop-yetus commented on PR #5546:
URL: https://github.com/apache/hadoop/pull/5546#issuecomment-1506335328
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 38s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 1 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 41m 2s | | trunk passed |
| +1 :green_heart: | compile | 2m 1s | | trunk passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | compile | 1m 28s | | trunk passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | checkstyle | 1m 7s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 29s | | trunk passed |
| +1 :green_heart: | javadoc | 1m 9s | | trunk passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javadoc | 1m 35s | | trunk passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | spotbugs | 3m 32s | | trunk passed |
| +1 :green_heart: | shadedclient | 23m 48s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 1m 23s | | the patch passed |
| +1 :green_heart: | compile | 1m 23s | | the patch passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javac | 1m 23s | | the patch passed |
| +1 :green_heart: | compile | 1m 17s | | the patch passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | javac | 1m 17s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| -0 :warning: | checkstyle | 0m 58s |
[/results-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5546/3/artifact/out/results-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt)
| hadoop-hdfs-project/hadoop-hdfs: The patch generated 1 new + 273 unchanged
- 1 fixed = 274 total (was 274) |
| +1 :green_heart: | mvnsite | 1m 24s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 53s | | the patch passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javadoc | 1m 26s | | the patch passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | spotbugs | 3m 32s | | the patch passed |
| +1 :green_heart: | shadedclient | 20m 18s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| -1 :x: | unit | 211m 32s |
[/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5546/3/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt)
| hadoop-hdfs in the patch passed. |
| +1 :green_heart: | asflicense | 0m 47s | | The patch does not
generate ASF License warnings. |
| | | 320m 54s | | |
| Reason | Tests |
|---:|:--|
| Failed junit tests | hadoop.hdfs.server.namenode.ha.TestObserverNode |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.42 ServerAPI=1.42 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5546/3/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/5546 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux 6e535f6d5cd6 4.15.0-206-generic #217-Ubuntu SMP Fri Feb 3
19:10:13 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 84b21c12220234761b43a023f4550db8a94bd6c6 |
| Default Java | Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5546/3/testReport/ |
| Max. process+thread count | 3203 (vs. ulimit of 5500) |
| modules | C: hadoop-hdfs-project/hadoop-hdfs U:
[GitHub] [hadoop] hadoop-yetus commented on pull request #5550: YARN-11465. Improved YarnClient Log Format
hadoop-yetus commented on PR #5550:
URL: https://github.com/apache/hadoop/pull/5550#issuecomment-1506312104
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 53s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| -1 :x: | test4tests | 0m 0s | | The patch doesn't appear to include
any new or modified tests. Please justify why no new tests are needed for this
patch. Also please list what manual steps were performed to verify this patch.
|
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 39m 29s | | trunk passed |
| +1 :green_heart: | compile | 0m 37s | | trunk passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | compile | 0m 34s | | trunk passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | checkstyle | 0m 34s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 39s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 40s | | trunk passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javadoc | 0m 28s | | trunk passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | spotbugs | 1m 10s | | trunk passed |
| +1 :green_heart: | shadedclient | 20m 21s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 27s | | the patch passed |
| +1 :green_heart: | compile | 0m 28s | | the patch passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javac | 0m 28s | | the patch passed |
| +1 :green_heart: | compile | 0m 26s | | the patch passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | javac | 0m 27s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| -0 :warning: | checkstyle | 0m 19s |
[/results-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5550/1/artifact/out/results-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client.txt)
| hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client: The patch generated 91
new + 56 unchanged - 0 fixed = 147 total (was 56) |
| +1 :green_heart: | mvnsite | 0m 27s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 22s | | the patch passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javadoc | 0m 21s | | the patch passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | spotbugs | 0m 56s | | the patch passed |
| +1 :green_heart: | shadedclient | 20m 5s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 28m 16s | | hadoop-yarn-client in the patch
passed. |
| +1 :green_heart: | asflicense | 0m 40s | | The patch does not
generate ASF License warnings. |
| | | 119m 55s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.42 ServerAPI=1.42 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5550/1/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/5550 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux 941b17f8f968 4.15.0-206-generic #217-Ubuntu SMP Fri Feb 3
19:10:13 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 8363709d805df0b782dcbf294b3e312a01007f01 |
| Default Java | Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5550/1/testReport/ |
| Max. process+thread count | 677 (vs. ulimit of 5500) |
| modules | C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client U:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client |
| Console output |
[GitHub] [hadoop] slfan1989 commented on pull request #5550: YARN-11465. Improved YarnClient Log Format
slfan1989 commented on PR #5550: URL: https://github.com/apache/hadoop/pull/5550#issuecomment-1506296837 LGTM. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] YuanbenWang commented on a diff in pull request #5547: HDFS-16977. Forbid assigned characters in pathname.
YuanbenWang commented on code in PR #5547: URL: https://github.com/apache/hadoop/pull/5547#discussion_r1164952080 ## hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml: ## @@ -2888,6 +2888,16 @@ + + dfs.pathname.forbidden.characters + Review Comment: Thank you for the review ! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] YuanbenWang commented on a diff in pull request #5547: HDFS-16977. Forbid assigned characters in pathname.
YuanbenWang commented on code in PR #5547: URL: https://github.com/apache/hadoop/pull/5547#discussion_r1164951032 ## hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml: ## @@ -2888,6 +2888,16 @@ + + dfs.pathname.forbidden.characters + Review Comment: I thought some users might persist in using some special characters to mark path, so I leaf it empty. Maybe it's not a good idea, and I will update it in the next commit. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18399) SingleFilePerBlockCache to use LocalDirAllocator for file allocation
[
https://issues.apache.org/jira/browse/HADOOP-18399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17711645#comment-17711645
]
ASF GitHub Bot commented on HADOOP-18399:
-
hadoop-yetus commented on PR #5054:
URL: https://github.com/apache/hadoop/pull/5054#issuecomment-1506255713
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 48s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +0 :ok: | xmllint | 0m 0s | | xmllint was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 5 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 15m 52s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 28m 57s | | trunk passed |
| +1 :green_heart: | compile | 25m 28s | | trunk passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | compile | 21m 45s | | trunk passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | checkstyle | 4m 26s | | trunk passed |
| +1 :green_heart: | mvnsite | 2m 31s | | trunk passed |
| +1 :green_heart: | javadoc | 1m 38s | | trunk passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javadoc | 1m 19s | | trunk passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | spotbugs | 4m 0s | | trunk passed |
| +1 :green_heart: | shadedclient | 23m 45s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 23s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 1m 32s | | the patch passed |
| +1 :green_heart: | compile | 24m 27s | | the patch passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javac | 24m 27s | | the patch passed |
| +1 :green_heart: | compile | 21m 34s | | the patch passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | javac | 21m 34s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 3m 49s | | the patch passed |
| +1 :green_heart: | mvnsite | 2m 24s | | the patch passed |
| +1 :green_heart: | javadoc | 1m 26s | | the patch passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javadoc | 1m 18s | | the patch passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | spotbugs | 4m 5s | | the patch passed |
| +1 :green_heart: | shadedclient | 23m 46s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 18m 13s | | hadoop-common in the patch
passed. |
| +1 :green_heart: | unit | 2m 42s | | hadoop-aws in the patch passed.
|
| +1 :green_heart: | asflicense | 0m 52s | | The patch does not
generate ASF License warnings. |
| | | 240m 46s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.42 ServerAPI=1.42 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5054/23/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/5054 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets xmllint |
| uname | Linux 7b53be31aaf3 4.15.0-206-generic #217-Ubuntu SMP Fri Feb 3
19:10:13 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 38050c772778c720b92df38bca8d5faa62e041f6 |
| Default Java | Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5054/23/testReport/ |
| Max. process+thread count | 2383 (vs. ulimit of 5500) |
| modules | C: hadoop-common-project/hadoop-common
[GitHub] [hadoop] hadoop-yetus commented on pull request #5054: HADOOP-18399 Prefetch - SingleFilePerBlockCache to use LocalDirAllocator for file allocation
hadoop-yetus commented on PR #5054:
URL: https://github.com/apache/hadoop/pull/5054#issuecomment-1506255713
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 48s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +0 :ok: | xmllint | 0m 0s | | xmllint was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 5 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 15m 52s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 28m 57s | | trunk passed |
| +1 :green_heart: | compile | 25m 28s | | trunk passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | compile | 21m 45s | | trunk passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | checkstyle | 4m 26s | | trunk passed |
| +1 :green_heart: | mvnsite | 2m 31s | | trunk passed |
| +1 :green_heart: | javadoc | 1m 38s | | trunk passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javadoc | 1m 19s | | trunk passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | spotbugs | 4m 0s | | trunk passed |
| +1 :green_heart: | shadedclient | 23m 45s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 23s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 1m 32s | | the patch passed |
| +1 :green_heart: | compile | 24m 27s | | the patch passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javac | 24m 27s | | the patch passed |
| +1 :green_heart: | compile | 21m 34s | | the patch passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | javac | 21m 34s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 3m 49s | | the patch passed |
| +1 :green_heart: | mvnsite | 2m 24s | | the patch passed |
| +1 :green_heart: | javadoc | 1m 26s | | the patch passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javadoc | 1m 18s | | the patch passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | spotbugs | 4m 5s | | the patch passed |
| +1 :green_heart: | shadedclient | 23m 46s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 18m 13s | | hadoop-common in the patch
passed. |
| +1 :green_heart: | unit | 2m 42s | | hadoop-aws in the patch passed.
|
| +1 :green_heart: | asflicense | 0m 52s | | The patch does not
generate ASF License warnings. |
| | | 240m 46s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.42 ServerAPI=1.42 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5054/23/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/5054 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets xmllint |
| uname | Linux 7b53be31aaf3 4.15.0-206-generic #217-Ubuntu SMP Fri Feb 3
19:10:13 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 38050c772778c720b92df38bca8d5faa62e041f6 |
| Default Java | Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5054/23/testReport/ |
| Max. process+thread count | 2383 (vs. ulimit of 5500) |
| modules | C: hadoop-common-project/hadoop-common hadoop-tools/hadoop-aws
U: . |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5054/23/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
[GitHub] [hadoop] hadoop-yetus commented on pull request #5532: HDFS-16972. Delete a snapshot may deleteCurrentFile.
hadoop-yetus commented on PR #5532:
URL: https://github.com/apache/hadoop/pull/5532#issuecomment-1506227426
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 1m 1s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| -1 :x: | test4tests | 0m 0s | | The patch doesn't appear to include
any new or modified tests. Please justify why no new tests are needed for this
patch. Also please list what manual steps were performed to verify this patch.
|
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 47m 9s | | trunk passed |
| +1 :green_heart: | compile | 1m 47s | | trunk passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | compile | 1m 28s | | trunk passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | checkstyle | 1m 15s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 52s | | trunk passed |
| +1 :green_heart: | javadoc | 1m 19s | | trunk passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javadoc | 1m 43s | | trunk passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | spotbugs | 3m 59s | | trunk passed |
| +1 :green_heart: | shadedclient | 28m 51s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 1m 27s | | the patch passed |
| +1 :green_heart: | compile | 1m 26s | | the patch passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javac | 1m 26s | | the patch passed |
| +1 :green_heart: | compile | 1m 17s | | the patch passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | javac | 1m 17s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 0m 56s | | the patch passed |
| +1 :green_heart: | mvnsite | 1m 40s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 57s | | the patch passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javadoc | 1m 28s | | the patch passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | spotbugs | 3m 57s | | the patch passed |
| +1 :green_heart: | shadedclient | 28m 39s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| -1 :x: | unit | 268m 43s |
[/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5532/5/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt)
| hadoop-hdfs in the patch passed. |
| +1 :green_heart: | asflicense | 0m 54s | | The patch does not
generate ASF License warnings. |
| | | 398m 59s | | |
| Reason | Tests |
|---:|:--|
| Failed junit tests | hadoop.hdfs.server.namenode.ha.TestObserverNode |
| | hadoop.hdfs.server.datanode.TestDirectoryScanner |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.42 ServerAPI=1.42 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5532/5/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/5532 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux ed0dcd6a4723 4.15.0-206-generic #217-Ubuntu SMP Fri Feb 3
19:10:13 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / acef9ae87202564db1cac870642b1614924779f4 |
| Default Java | Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5532/5/testReport/ |
| Max. process+thread count | 2366 (vs. ulimit of 5500) |
| modules | C: hadoop-hdfs-project/hadoop-hdfs U:
hadoop-hdfs-project/hadoop-hdfs |
| Console output |
[jira] [Reopened] (HADOOP-18700) add rpc metrics for response time
[ https://issues.apache.org/jira/browse/HADOOP-18700?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liuguanghua reopened HADOOP-18700: -- > add rpc metrics for response time > - > > Key: HADOOP-18700 > URL: https://issues.apache.org/jira/browse/HADOOP-18700 > Project: Hadoop Common > Issue Type: Improvement >Reporter: liuguanghua >Priority: Minor > > Rpc ResponseTime record the time to encode and send response. This should be > added in the rpc metrics. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Resolved] (HADOOP-18700) add rpc metrics for response time
[ https://issues.apache.org/jira/browse/HADOOP-18700?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liuguanghua resolved HADOOP-18700. -- Resolution: Fixed > add rpc metrics for response time > - > > Key: HADOOP-18700 > URL: https://issues.apache.org/jira/browse/HADOOP-18700 > Project: Hadoop Common > Issue Type: Improvement >Reporter: liuguanghua >Priority: Minor > > Rpc ResponseTime record the time to encode and send response. This should be > added in the rpc metrics. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Created] (HADOOP-18700) add rpc metrics for response time
liuguanghua created HADOOP-18700: Summary: add rpc metrics for response time Key: HADOOP-18700 URL: https://issues.apache.org/jira/browse/HADOOP-18700 Project: Hadoop Common Issue Type: Improvement Reporter: liuguanghua Rpc ResponseTime record the time to encode and send response. This should be added in the rpc metrics. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] yl09099 opened a new pull request, #5550: YARN-11465. Improved YarnClient Log Format
yl09099 opened a new pull request, #5550: URL: https://github.com/apache/hadoop/pull/5550 JIRA: YARN-11465. Improved YarnClient Log Format -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] slfan1989 commented on pull request #4687: YARN-10846. Add dispatcher metrics to NM.
slfan1989 commented on PR #4687: URL: https://github.com/apache/hadoop/pull/4687#issuecomment-1506184067 @goiri Thank you very much for helping to review the code! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] slfan1989 commented on a diff in pull request #5547: HDFS-16977. Forbid assigned characters in pathname when new file or directory is created.
slfan1989 commented on code in PR #5547: URL: https://github.com/apache/hadoop/pull/5547#discussion_r1164829717 ## hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml: ## @@ -2888,6 +2888,16 @@ + + dfs.pathname.forbidden.characters + Review Comment: value is empty? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] slfan1989 commented on pull request #5547: HDFS-16977. Forbid assigned characters in pathname when new file or directory is created.
slfan1989 commented on PR #5547: URL: https://github.com/apache/hadoop/pull/5547#issuecomment-1506178316 @YuanbenWang Thanks for the contribution, need to fix checkstyle, and pay attention to junit test error. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] goiri commented on pull request #5536: [HDFS-16971] Add read metrics for remote reads in FileSystem Statistics #5534
goiri commented on PR #5536: URL: https://github.com/apache/hadoop/pull/5536#issuecomment-1506159914 @simbadzina any further comments? Can you take a final look? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #5532: HDFS-16972. Delete a snapshot may deleteCurrentFile.
hadoop-yetus commented on PR #5532:
URL: https://github.com/apache/hadoop/pull/5532#issuecomment-1506136128
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 1m 5s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| -1 :x: | test4tests | 0m 0s | | The patch doesn't appear to include
any new or modified tests. Please justify why no new tests are needed for this
patch. Also please list what manual steps were performed to verify this patch.
|
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 43m 28s | | trunk passed |
| +1 :green_heart: | compile | 1m 28s | | trunk passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | compile | 1m 20s | | trunk passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | checkstyle | 1m 3s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 30s | | trunk passed |
| +1 :green_heart: | javadoc | 1m 7s | | trunk passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javadoc | 1m 30s | | trunk passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | spotbugs | 3m 40s | | trunk passed |
| +1 :green_heart: | shadedclient | 25m 30s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 1m 20s | | the patch passed |
| +1 :green_heart: | compile | 1m 20s | | the patch passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javac | 1m 20s | | the patch passed |
| +1 :green_heart: | compile | 1m 15s | | the patch passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | javac | 1m 16s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 0m 53s | | the patch passed |
| +1 :green_heart: | mvnsite | 1m 23s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 54s | | the patch passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javadoc | 1m 27s | | the patch passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | spotbugs | 3m 31s | | the patch passed |
| +1 :green_heart: | shadedclient | 27m 23s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| -1 :x: | unit | 262m 39s |
[/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5532/4/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt)
| hadoop-hdfs in the patch passed. |
| +1 :green_heart: | asflicense | 1m 27s | | The patch does not
generate ASF License warnings. |
| | | 382m 51s | | |
| Reason | Tests |
|---:|:--|
| Failed junit tests | hadoop.hdfs.server.namenode.ha.TestObserverNode |
| | hadoop.hdfs.server.datanode.TestDirectoryScanner |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.42 ServerAPI=1.42 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5532/4/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/5532 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux 3165179b8b0d 4.15.0-206-generic #217-Ubuntu SMP Fri Feb 3
19:10:13 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 5b7ff7e1fb2919c5c9f998cbdc4ec8f9b682d8b9 |
| Default Java | Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5532/4/testReport/ |
| Max. process+thread count | 2305 (vs. ulimit of 5500) |
| modules | C: hadoop-hdfs-project/hadoop-hdfs U:
hadoop-hdfs-project/hadoop-hdfs |
| Console output |
[GitHub] [hadoop] melissayou commented on pull request #5536: [HDFS-16971] Add read metrics for remote reads in FileSystem Statistics #5534
melissayou commented on PR #5536: URL: https://github.com/apache/hadoop/pull/5536#issuecomment-1506064911 Hi @goiri , I addressed your comments(1 open). Can you review again? Thank you! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #5536: [HDFS-16971] Add read metrics for remote reads in FileSystem Statistics #5534
hadoop-yetus commented on PR #5536:
URL: https://github.com/apache/hadoop/pull/5536#issuecomment-1506037561
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 53s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 1 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 15m 30s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 30m 28s | | trunk passed |
| +1 :green_heart: | compile | 25m 16s | | trunk passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | compile | 21m 44s | | trunk passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | checkstyle | 3m 59s | | trunk passed |
| +1 :green_heart: | mvnsite | 2m 44s | | trunk passed |
| +1 :green_heart: | javadoc | 1m 58s | | trunk passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javadoc | 1m 27s | | trunk passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | spotbugs | 5m 26s | | trunk passed |
| +1 :green_heart: | shadedclient | 25m 7s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 25s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 1m 51s | | the patch passed |
| +1 :green_heart: | compile | 24m 27s | | the patch passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javac | 24m 27s | | the patch passed |
| +1 :green_heart: | compile | 21m 49s | | the patch passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | javac | 21m 49s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 3m 51s | | the patch passed |
| +1 :green_heart: | mvnsite | 2m 42s | | the patch passed |
| +1 :green_heart: | javadoc | 1m 47s | | the patch passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javadoc | 1m 25s | | the patch passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | spotbugs | 5m 39s | | the patch passed |
| +1 :green_heart: | shadedclient | 25m 40s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 18m 14s | | hadoop-common in the patch
passed. |
| +1 :green_heart: | unit | 2m 32s | | hadoop-hdfs-client in the patch
passed. |
| +1 :green_heart: | asflicense | 0m 52s | | The patch does not
generate ASF License warnings. |
| | | 246m 13s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.42 ServerAPI=1.42 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5536/8/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/5536 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux 337098edbfa9 4.15.0-206-generic #217-Ubuntu SMP Fri Feb 3
19:10:13 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 5d3873e3ed0b265840ac8d719b7686d94d0be3c9 |
| Default Java | Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5536/8/testReport/ |
| Max. process+thread count | 1375 (vs. ulimit of 5500) |
| modules | C: hadoop-common-project/hadoop-common
hadoop-hdfs-project/hadoop-hdfs-client U: . |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5536/8/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was automatically generated.
[GitHub] [hadoop] hadoop-yetus commented on pull request #5547: HDFS-16977. Forbid assigned characters in pathname when new file or directory is created.
hadoop-yetus commented on PR #5547:
URL: https://github.com/apache/hadoop/pull/5547#issuecomment-1506014678
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 39s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available.
|
| +0 :ok: | xmllint | 0m 1s | | xmllint was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 2 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 15m 40s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 29m 0s | | trunk passed |
| +1 :green_heart: | compile | 6m 32s | | trunk passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | compile | 6m 14s | | trunk passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | checkstyle | 1m 26s | | trunk passed |
| +1 :green_heart: | mvnsite | 2m 38s | | trunk passed |
| +1 :green_heart: | javadoc | 1m 50s | | trunk passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javadoc | 2m 20s | | trunk passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | spotbugs | 6m 18s | | trunk passed |
| +1 :green_heart: | shadedclient | 23m 0s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 28s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 2m 18s | | the patch passed |
| +1 :green_heart: | compile | 6m 11s | | the patch passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javac | 6m 11s | | the patch passed |
| +1 :green_heart: | compile | 5m 56s | | the patch passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | javac | 5m 56s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| -0 :warning: | checkstyle | 1m 7s |
[/results-checkstyle-hadoop-hdfs-project.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5547/1/artifact/out/results-checkstyle-hadoop-hdfs-project.txt)
| hadoop-hdfs-project: The patch generated 2 new + 394 unchanged - 0 fixed =
396 total (was 394) |
| +1 :green_heart: | mvnsite | 2m 16s | | the patch passed |
| +1 :green_heart: | javadoc | 1m 31s | | the patch passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javadoc | 1m 59s | | the patch passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| -1 :x: | spotbugs | 2m 45s |
[/new-spotbugs-hadoop-hdfs-project_hadoop-hdfs-client.html](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5547/1/artifact/out/new-spotbugs-hadoop-hdfs-project_hadoop-hdfs-client.html)
| hadoop-hdfs-project/hadoop-hdfs-client generated 1 new + 0 unchanged - 0
fixed = 1 total (was 0) |
| +1 :green_heart: | shadedclient | 23m 14s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 2m 28s | | hadoop-hdfs-client in the patch
passed. |
| -1 :x: | unit | 203m 33s |
[/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5547/1/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt)
| hadoop-hdfs in the patch passed. |
| +1 :green_heart: | asflicense | 0m 52s | | The patch does not
generate ASF License warnings. |
| | | 352m 52s | | |
| Reason | Tests |
|---:|:--|
| SpotBugs | module:hadoop-hdfs-project/hadoop-hdfs-client |
| | org.apache.hadoop.hdfs.DFSClient.pathnameForbiddenCharacters is a
volatile reference to an array; the array elements are non-volatile In
DFSClient.java:to an array; the array elements are non-volatile In
DFSClient.java |
| Failed junit tests | hadoop.hdfs.tools.TestDFSAdmin |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.42 ServerAPI=1.42 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5547/1/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/5547 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
[jira] [Commented] (HADOOP-18695) S3A: reject multipart copy requests initiated by transfer manager when disabled
[
https://issues.apache.org/jira/browse/HADOOP-18695?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17711575#comment-17711575
]
ASF GitHub Bot commented on HADOOP-18695:
-
hadoop-yetus commented on PR #5548:
URL: https://github.com/apache/hadoop/pull/5548#issuecomment-1505969728
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 46s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 5 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 41m 46s | | trunk passed |
| +1 :green_heart: | compile | 0m 40s | | trunk passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | compile | 0m 33s | | trunk passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | checkstyle | 0m 32s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 40s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 26s | | trunk passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javadoc | 0m 29s | | trunk passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | spotbugs | 1m 18s | | trunk passed |
| +1 :green_heart: | shadedclient | 23m 52s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 31s | | the patch passed |
| +1 :green_heart: | compile | 0m 36s | | the patch passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javac | 0m 36s | | the patch passed |
| +1 :green_heart: | compile | 0m 28s | | the patch passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | javac | 0m 28s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| -0 :warning: | checkstyle | 0m 18s |
[/results-checkstyle-hadoop-tools_hadoop-aws.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5548/1/artifact/out/results-checkstyle-hadoop-tools_hadoop-aws.txt)
| hadoop-tools/hadoop-aws: The patch generated 2 new + 21 unchanged - 0 fixed
= 23 total (was 21) |
| +1 :green_heart: | mvnsite | 0m 33s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 14s | | the patch passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| -1 :x: | javadoc | 0m 21s |
[/results-javadoc-javadoc-hadoop-tools_hadoop-aws-jdkPrivateBuild-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5548/1/artifact/out/results-javadoc-javadoc-hadoop-tools_hadoop-aws-jdkPrivateBuild-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09.txt)
|
hadoop-tools_hadoop-aws-jdkPrivateBuild-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09
with JDK Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 generated 1 new
+ 0 unchanged - 0 fixed = 1 total (was 0) |
| +1 :green_heart: | spotbugs | 1m 7s | | the patch passed |
| +1 :green_heart: | shadedclient | 23m 52s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 2m 26s | | hadoop-aws in the patch passed.
|
| +1 :green_heart: | asflicense | 0m 33s | | The patch does not
generate ASF License warnings. |
| | | 103m 18s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.42 ServerAPI=1.42 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5548/1/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/5548 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux 5a2d7e2c0653 4.15.0-206-generic #217-Ubuntu SMP Fri Feb 3
19:10:13 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 8136c9922a04effd65ba00b7f33d778a4ce69847 |
| Default Java | Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| Multi-JDK versions |
[GitHub] [hadoop] hadoop-yetus commented on pull request #5548: HADOOP-18695. S3A: reject multipart copy requests initiated by tranfser manager when disabled
hadoop-yetus commented on PR #5548:
URL: https://github.com/apache/hadoop/pull/5548#issuecomment-1505969728
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 46s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 5 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 41m 46s | | trunk passed |
| +1 :green_heart: | compile | 0m 40s | | trunk passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | compile | 0m 33s | | trunk passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | checkstyle | 0m 32s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 40s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 26s | | trunk passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javadoc | 0m 29s | | trunk passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | spotbugs | 1m 18s | | trunk passed |
| +1 :green_heart: | shadedclient | 23m 52s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 31s | | the patch passed |
| +1 :green_heart: | compile | 0m 36s | | the patch passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javac | 0m 36s | | the patch passed |
| +1 :green_heart: | compile | 0m 28s | | the patch passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | javac | 0m 28s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| -0 :warning: | checkstyle | 0m 18s |
[/results-checkstyle-hadoop-tools_hadoop-aws.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5548/1/artifact/out/results-checkstyle-hadoop-tools_hadoop-aws.txt)
| hadoop-tools/hadoop-aws: The patch generated 2 new + 21 unchanged - 0 fixed
= 23 total (was 21) |
| +1 :green_heart: | mvnsite | 0m 33s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 14s | | the patch passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| -1 :x: | javadoc | 0m 21s |
[/results-javadoc-javadoc-hadoop-tools_hadoop-aws-jdkPrivateBuild-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5548/1/artifact/out/results-javadoc-javadoc-hadoop-tools_hadoop-aws-jdkPrivateBuild-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09.txt)
|
hadoop-tools_hadoop-aws-jdkPrivateBuild-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09
with JDK Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 generated 1 new
+ 0 unchanged - 0 fixed = 1 total (was 0) |
| +1 :green_heart: | spotbugs | 1m 7s | | the patch passed |
| +1 :green_heart: | shadedclient | 23m 52s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 2m 26s | | hadoop-aws in the patch passed.
|
| +1 :green_heart: | asflicense | 0m 33s | | The patch does not
generate ASF License warnings. |
| | | 103m 18s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.42 ServerAPI=1.42 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5548/1/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/5548 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux 5a2d7e2c0653 4.15.0-206-generic #217-Ubuntu SMP Fri Feb 3
19:10:13 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 8136c9922a04effd65ba00b7f33d778a4ce69847 |
| Default Java | Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5548/1/testReport/ |
| Max. process+thread count
[jira] [Commented] (HADOOP-18687) Remove unnecessary dependency on json-smart
[ https://issues.apache.org/jira/browse/HADOOP-18687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17711570#comment-17711570 ] ASF GitHub Bot commented on HADOOP-18687: - pjfanning opened a new pull request, #5549: URL: https://github.com/apache/hadoop/pull/5549 complete removal of json-smart jar > Remove unnecessary dependency on json-smart > --- > > Key: HADOOP-18687 > URL: https://issues.apache.org/jira/browse/HADOOP-18687 > Project: Hadoop Common > Issue Type: Improvement > Components: auth >Affects Versions: 3.4.0, 3.3.5 >Reporter: Michiel de Jong >Assignee: Michiel de Jong >Priority: Major > Labels: pull-request-available > Fix For: 3.4.0 > > > hadoop-auth has a dependency on net.minidev:json-smart 2.4.7, but this > dependency is never used. > This dependency was originally included because the transitive dependency > that nimbus-jose-jwt had did not work properly (see > https://issues.apache.org/jira/browse/HADOOP-14903). Since version 9.* > nimbus-jose-jwt is using its own shaded version of json-smart, so the version > declared in hadoop-auth is never actually used. > json-smart 2.4.7 shows up in CVE scans for CVE-2023-1370. It is still used as > a transitive dependency in hadoop-hdfs -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] pjfanning opened a new pull request, #5549: HADOOP-18687: remove json-smart dependency
pjfanning opened a new pull request, #5549: URL: https://github.com/apache/hadoop/pull/5549 complete removal of json-smart jar -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #5546: HDFS-16975. FileWithSnapshotFeature.isCurrentFileDeleted is not reloaded from FSImage.
hadoop-yetus commented on PR #5546:
URL: https://github.com/apache/hadoop/pull/5546#issuecomment-1505921927
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 33s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 1 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 39m 18s | | trunk passed |
| +1 :green_heart: | compile | 1m 27s | | trunk passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | compile | 1m 18s | | trunk passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | checkstyle | 1m 7s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 35s | | trunk passed |
| +1 :green_heart: | javadoc | 1m 8s | | trunk passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javadoc | 1m 37s | | trunk passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | spotbugs | 3m 34s | | trunk passed |
| +1 :green_heart: | shadedclient | 22m 56s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 1m 15s | | the patch passed |
| +1 :green_heart: | compile | 1m 28s | | the patch passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javac | 1m 28s | | the patch passed |
| +1 :green_heart: | compile | 1m 16s | | the patch passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | javac | 1m 16s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| -0 :warning: | checkstyle | 0m 59s |
[/results-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5546/1/artifact/out/results-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt)
| hadoop-hdfs-project/hadoop-hdfs: The patch generated 9 new + 273 unchanged
- 1 fixed = 282 total (was 274) |
| +1 :green_heart: | mvnsite | 1m 24s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 52s | | the patch passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javadoc | 1m 25s | | the patch passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | spotbugs | 3m 27s | | the patch passed |
| +1 :green_heart: | shadedclient | 19m 23s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| -1 :x: | unit | 216m 7s |
[/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5546/1/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt)
| hadoop-hdfs in the patch passed. |
| +1 :green_heart: | asflicense | 0m 49s | | The patch does not
generate ASF License warnings. |
| | | 320m 49s | | |
| Reason | Tests |
|---:|:--|
| Failed junit tests |
hadoop.hdfs.server.namenode.snapshot.TestRenameWithSnapshots |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.42 ServerAPI=1.42 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5546/1/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/5546 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux 414667cdf682 4.15.0-206-generic #217-Ubuntu SMP Fri Feb 3
19:10:13 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 0a0bf14fe9b5c1a7b41e3fbf1a70c07cd4336754 |
| Default Java | Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5546/1/testReport/ |
| Max. process+thread count | 3560 (vs. ulimit of 5500) |
| modules | C:
[GitHub] [hadoop] hadoop-yetus commented on pull request #5546: HDFS-16975. FileWithSnapshotFeature.isCurrentFileDeleted is not reloaded from FSImage.
hadoop-yetus commented on PR #5546:
URL: https://github.com/apache/hadoop/pull/5546#issuecomment-1505918561
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 37s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 1 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 39m 9s | | trunk passed |
| +1 :green_heart: | compile | 1m 27s | | trunk passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | compile | 1m 24s | | trunk passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | checkstyle | 1m 12s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 33s | | trunk passed |
| +1 :green_heart: | javadoc | 1m 11s | | trunk passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javadoc | 1m 38s | | trunk passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | spotbugs | 3m 35s | | trunk passed |
| +1 :green_heart: | shadedclient | 22m 55s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 1m 25s | | the patch passed |
| +1 :green_heart: | compile | 1m 25s | | the patch passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javac | 1m 25s | | the patch passed |
| +1 :green_heart: | compile | 1m 14s | | the patch passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | javac | 1m 14s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| -0 :warning: | checkstyle | 0m 54s |
[/results-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5546/2/artifact/out/results-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt)
| hadoop-hdfs-project/hadoop-hdfs: The patch generated 9 new + 273 unchanged
- 1 fixed = 282 total (was 274) |
| +1 :green_heart: | mvnsite | 1m 26s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 53s | | the patch passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javadoc | 1m 25s | | the patch passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | spotbugs | 3m 25s | | the patch passed |
| +1 :green_heart: | shadedclient | 22m 51s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| -1 :x: | unit | 208m 53s |
[/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5546/2/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt)
| hadoop-hdfs in the patch passed. |
| +1 :green_heart: | asflicense | 0m 48s | | The patch does not
generate ASF License warnings. |
| | | 317m 14s | | |
| Reason | Tests |
|---:|:--|
| Failed junit tests |
hadoop.hdfs.server.namenode.snapshot.TestRenameWithSnapshots |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.42 ServerAPI=1.42 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5546/2/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/5546 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux a0100fd134e8 4.15.0-206-generic #217-Ubuntu SMP Fri Feb 3
19:10:13 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 0a0bf14fe9b5c1a7b41e3fbf1a70c07cd4336754 |
| Default Java | Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5546/2/testReport/ |
| Max. process+thread count | 3263 (vs. ulimit of 5500) |
| modules | C:
[jira] [Updated] (HADOOP-18699) InvalidProtocolBufferException caused by JDK 11 < 11.0.18 AES-CTR cipher state corruption with AVX-512 bug
[
https://issues.apache.org/jira/browse/HADOOP-18699?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Siyao Meng updated HADOOP-18699:
Description:
This serves as a PSA for a JDK bug. Not really a bug in Hadoop / HDFS.
Symptom/Workaround/Solution detailed below.
[~relek] identified [JDK-8292158|https://bugs.openjdk.org/browse/JDK-8292158]
(backported to JDK 11 in
[JDK-8295297|https://bugs.openjdk.org/browse/JDK-8295297]) causes HDFS clients
to fail with InvalidProtocolBufferException due to corrupted protobuf message
in Hadoop RPC request when all of the below conditions are met:
1. The host is capable of AVX-512 instruction sets
2. AVX-512 is enabled in JVM. This should be enabled by default on AVX-512
capable hosts, equivalent to specifying JVM arg {{-XX:UseAVX=3}}
3. Hadoop native library (e.g. libhadoop.so) is not available. So the HDFS
client falls back using Hotspot JVM's {{aesctr_encrypt}} implementation for
AES/CTR/NoPadding.
4. Client uses JDK 11. And OpenJDK version < 11.0.18
As a result, the client could print messages like these:
{code:title=Symptoms on the HDFS client}
2023-02-21 15:21:44,380 WARN org.apache.hadoop.hdfs.DFSClient: Connection
failure: Failed to connect to for file
/tmp/.cloudera_health_monitoring_canary_files/.canary_file_2023_02_21-15_21_25.b6788e89894a61b5
for block
BP-1836197545-10.125.248.11-1672668423261:blk_1073935111_194857:com.google.protobuf.InvalidProtocolBufferException:
Protocol message tag had invalid wire type.
com.google.protobuf.InvalidProtocolBufferException: Protocol message tag had
invalid wire type.
2023-02-21 15:21:44,378 WARN org.apache.hadoop.hdfs.DFSClient: Connection
failure: Failed to connect to for file
/tmp/.cloudera_health_monitoring_canary_files/.canary_file_2023_02_21-15_21_25.b6788e89894a61b5
for block
BP-1836197545--1672668423261:blk_1073935111_194857:com.google.protobuf.InvalidProtocolBufferException:
Protocol message end-group tag did not match expected tag.
com.google.protobuf.InvalidProtocolBufferException: Protocol message end-group
tag did not match expected tag.
2023-02-21 15:06:55,530 WARN org.apache.hadoop.hdfs.DFSClient: Connection
failure: Failed to connect to for file
/tmp/.cloudera_health_monitoring_canary_files/.canary_file_2023_02_21-15_06_55.b4a633a8bde014aa
for block
BP-1836197545--1672668423261:blk_1073935025_194771:com.google.protobuf.InvalidProtocolBufferException:
While parsing a protocol message, the input ended unexpectedly in the middle
of a field. This could mean either than the input has been truncated or that an
embedded message misreported its own length.
com.google.protobuf.InvalidProtocolBufferException: While parsing a protocol
message, the input ended unexpectedly in the middle of a field. This could mean
either than the input has been truncated or that an embedded message
misreported its own length.
{code}
The error message might mislead devs/users into thinking this is a Hadoop
Common or HDFS bug (while it is a JDK bug in this case).
{color:red}Solutions:{color}
1. As a workaround, append {{-XX:UseAVX=2}} to client JVM args; or
2. Upgrade to OpenJDK >= 11.0.18.
I might post a repro test case for this, or find a way in the code to prompt
the user that this could be the potential issue (need to upgrade JDK 11) when
it occurs.
was:
This serves as a PSA for a JDK bug. Not really a bug in Hadoop / HDFS.
Symptom/Workaround/Solution detailed below.
[~relek] identified [JDK-8292158|https://bugs.openjdk.org/browse/JDK-8292158]
(backported to JDK 11 in
[JDK-8295297|https://bugs.openjdk.org/browse/JDK-8295297]) causes HDFS clients
to fail with InvalidProtocolBufferException due to corrupted protobuf message
in Hadoop RPC request when all of the below conditions are met:
1. The host is capable of AVX-512 instruction sets
2. AVX-512 is enabled in JVM. This should be enabled by default on AVX-512
capable hosts, equivalent to specifying JVM arg {{-XX:UseAVX=3}}
3. Hadoop native library (e.g. libhadoop.so) is not available. So the HDFS
client falls back to AES/CTR/NoPadding and thus uses Hotspot JVM's
{{aesctr_encrypt}} implementation.
4. Client uses JDK 11. And OpenJDK version < 11.0.18
As a result, the client could print messages like these:
{code:title=Symptoms on the HDFS client}
2023-02-21 15:21:44,380 WARN org.apache.hadoop.hdfs.DFSClient: Connection
failure: Failed to connect to for file
/tmp/.cloudera_health_monitoring_canary_files/.canary_file_2023_02_21-15_21_25.b6788e89894a61b5
for block
BP-1836197545-10.125.248.11-1672668423261:blk_1073935111_194857:com.google.protobuf.InvalidProtocolBufferException:
Protocol message tag had invalid wire type.
com.google.protobuf.InvalidProtocolBufferException: Protocol message tag had
invalid wire type.
2023-02-21 15:21:44,378 WARN org.apache.hadoop.hdfs.DFSClient: Connection
failure: Failed to connect to for file
[jira] [Updated] (HADOOP-18699) InvalidProtocolBufferException caused by JDK 11 < 11.0.18 AES-CTR cipher state corruption with AVX-512 bug
[
https://issues.apache.org/jira/browse/HADOOP-18699?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Siyao Meng updated HADOOP-18699:
Description:
This serves as a PSA for a JDK bug. Not really a bug in Hadoop / HDFS.
Symptom/Workaround/Solution detailed below.
[~relek] identified [JDK-8292158|https://bugs.openjdk.org/browse/JDK-8292158]
(backported to JDK 11 in
[JDK-8295297|https://bugs.openjdk.org/browse/JDK-8295297]) causes HDFS clients
to fail with InvalidProtocolBufferException due to corrupted protobuf message
in Hadoop RPC request when all of the below conditions are met:
1. The host is capable of AVX-512 instruction sets
2. AVX-512 is enabled in JVM. This should be enabled by default on AVX-512
capable hosts, equivalent to specifying JVM arg {{-XX:UseAVX=3}}
3. Hadoop native library (e.g. libhadoop.so) is not available. So the HDFS
client falls back to AES/CTR/NoPadding and thus uses Hotspot JVM's
{{aesctr_encrypt}} implementation.
4. Client uses JDK 11. And OpenJDK version < 11.0.18
As a result, the client could print messages like these:
{code:title=Symptoms on the HDFS client}
2023-02-21 15:21:44,380 WARN org.apache.hadoop.hdfs.DFSClient: Connection
failure: Failed to connect to for file
/tmp/.cloudera_health_monitoring_canary_files/.canary_file_2023_02_21-15_21_25.b6788e89894a61b5
for block
BP-1836197545-10.125.248.11-1672668423261:blk_1073935111_194857:com.google.protobuf.InvalidProtocolBufferException:
Protocol message tag had invalid wire type.
com.google.protobuf.InvalidProtocolBufferException: Protocol message tag had
invalid wire type.
2023-02-21 15:21:44,378 WARN org.apache.hadoop.hdfs.DFSClient: Connection
failure: Failed to connect to for file
/tmp/.cloudera_health_monitoring_canary_files/.canary_file_2023_02_21-15_21_25.b6788e89894a61b5
for block
BP-1836197545--1672668423261:blk_1073935111_194857:com.google.protobuf.InvalidProtocolBufferException:
Protocol message end-group tag did not match expected tag.
com.google.protobuf.InvalidProtocolBufferException: Protocol message end-group
tag did not match expected tag.
2023-02-21 15:06:55,530 WARN org.apache.hadoop.hdfs.DFSClient: Connection
failure: Failed to connect to for file
/tmp/.cloudera_health_monitoring_canary_files/.canary_file_2023_02_21-15_06_55.b4a633a8bde014aa
for block
BP-1836197545--1672668423261:blk_1073935025_194771:com.google.protobuf.InvalidProtocolBufferException:
While parsing a protocol message, the input ended unexpectedly in the middle
of a field. This could mean either than the input has been truncated or that an
embedded message misreported its own length.
com.google.protobuf.InvalidProtocolBufferException: While parsing a protocol
message, the input ended unexpectedly in the middle of a field. This could mean
either than the input has been truncated or that an embedded message
misreported its own length.
{code}
The error message might mislead devs/users into thinking this is a Hadoop
Common or HDFS bug (while it is a JDK bug in this case).
{color:red}Solutions:{color}
1. As a workaround, append {{-XX:UseAVX=2}} to client JVM args; or
2. Upgrade to OpenJDK >= 11.0.18.
I might post a repro test case for this, or find a way in the code to prompt
the user that this could be the potential issue (need to upgrade JDK 11) when
it occurs.
was:
This serves as a PSA for a JDK bug. Not really a bug in Hadoop / HDFS.
Symptom/Workaround/Solution detailed below.
[~relek] identified [JDK-8292158|https://bugs.openjdk.org/browse/JDK-8292158]
(backported to JDK 11 in
[JDK-8295297|https://bugs.openjdk.org/browse/JDK-8295297]) causes HDFS clients
to fail with InvalidProtocolBufferException due to corrupted protobuf message
in Hadoop RPC request when all of the below conditions are met:
1. The host is capable of AVX-512 instruction sets
2. AVX-512 is enabled in JVM. This should be enabled by default on AVX-512
capable hosts, equivalent to specifying JVM arg {{-XX:UseAVX=3}}
3. Hadoop native library (e.g. libhadoop.so) is not available
4. Client uses JDK 11. And OpenJDK version < 11.0.18
As a result, the client could print messages like these:
{code:title=Symptoms on the HDFS client}
2023-02-21 15:21:44,380 WARN org.apache.hadoop.hdfs.DFSClient: Connection
failure: Failed to connect to for file
/tmp/.cloudera_health_monitoring_canary_files/.canary_file_2023_02_21-15_21_25.b6788e89894a61b5
for block
BP-1836197545-10.125.248.11-1672668423261:blk_1073935111_194857:com.google.protobuf.InvalidProtocolBufferException:
Protocol message tag had invalid wire type.
com.google.protobuf.InvalidProtocolBufferException: Protocol message tag had
invalid wire type.
2023-02-21 15:21:44,378 WARN org.apache.hadoop.hdfs.DFSClient: Connection
failure: Failed to connect to for file
/tmp/.cloudera_health_monitoring_canary_files/.canary_file_2023_02_21-15_21_25.b6788e89894a61b5
for block
[jira] [Updated] (HADOOP-18699) InvalidProtocolBufferException caused by JDK 11 < 11.0.18 AES-CTR cipher state corruption with AVX-512 bug
[
https://issues.apache.org/jira/browse/HADOOP-18699?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Siyao Meng updated HADOOP-18699:
Description:
This serves as a PSA for a JDK bug. Not really a bug in Hadoop / HDFS.
Symptom/Workaround/Solution detailed below.
[~relek] identified [JDK-8292158|https://bugs.openjdk.org/browse/JDK-8292158]
(backported to JDK 11 in
[JDK-8295297|https://bugs.openjdk.org/browse/JDK-8295297]) causes HDFS clients
to fail with InvalidProtocolBufferException due to corrupted protobuf message
in Hadoop RPC request when all of the below conditions are met:
1. The host is capable of AVX-512 instruction sets
2. AVX-512 is enabled in JVM. This should be enabled by default on AVX-512
capable hosts, equivalent to specifying JVM arg {{-XX:UseAVX=3}}
3. Hadoop native library (e.g. libhadoop.so) is not available
4. Client uses JDK 11. And OpenJDK version < 11.0.18
As a result, the client could print messages like these:
{code:title=Symptoms on the HDFS client}
2023-02-21 15:21:44,380 WARN org.apache.hadoop.hdfs.DFSClient: Connection
failure: Failed to connect to for file
/tmp/.cloudera_health_monitoring_canary_files/.canary_file_2023_02_21-15_21_25.b6788e89894a61b5
for block
BP-1836197545-10.125.248.11-1672668423261:blk_1073935111_194857:com.google.protobuf.InvalidProtocolBufferException:
Protocol message tag had invalid wire type.
com.google.protobuf.InvalidProtocolBufferException: Protocol message tag had
invalid wire type.
2023-02-21 15:21:44,378 WARN org.apache.hadoop.hdfs.DFSClient: Connection
failure: Failed to connect to for file
/tmp/.cloudera_health_monitoring_canary_files/.canary_file_2023_02_21-15_21_25.b6788e89894a61b5
for block
BP-1836197545--1672668423261:blk_1073935111_194857:com.google.protobuf.InvalidProtocolBufferException:
Protocol message end-group tag did not match expected tag.
com.google.protobuf.InvalidProtocolBufferException: Protocol message end-group
tag did not match expected tag.
2023-02-21 15:06:55,530 WARN org.apache.hadoop.hdfs.DFSClient: Connection
failure: Failed to connect to for file
/tmp/.cloudera_health_monitoring_canary_files/.canary_file_2023_02_21-15_06_55.b4a633a8bde014aa
for block
BP-1836197545--1672668423261:blk_1073935025_194771:com.google.protobuf.InvalidProtocolBufferException:
While parsing a protocol message, the input ended unexpectedly in the middle
of a field. This could mean either than the input has been truncated or that an
embedded message misreported its own length.
com.google.protobuf.InvalidProtocolBufferException: While parsing a protocol
message, the input ended unexpectedly in the middle of a field. This could mean
either than the input has been truncated or that an embedded message
misreported its own length.
{code}
The error message might mislead devs/users into thinking this is a Hadoop
Common or HDFS bug (while it is a JDK bug in this case).
{color:red}Solutions:{color}
1. As a workaround, append {{-XX:UseAVX=2}} to client JVM args; or
2. Upgrade to OpenJDK >= 11.0.18.
I might post a repro test case for this, or find a way in the code to prompt
the user that this could be the potential issue (need to upgrade JDK 11) when
it occurs.
was:
This serves as a PSA for a JDK bug. Not really a bug in Hadoop / HDFS itself.
[~relek] identified [JDK-8292158|https://bugs.openjdk.org/browse/JDK-8292158]
(backported to JDK 11 in
[JDK-8295297|https://bugs.openjdk.org/browse/JDK-8295297]) causes HDFS clients
to fail with InvalidProtocolBufferException due to corrupted protobuf message
in Hadoop RPC request when all of the below conditions are met:
1. The host is capable of AVX-512 instruction sets
2. AVX-512 is enabled in JVM. This should be enabled by default on AVX-512
capable hosts, equivalent to specifying JVM arg {{-XX:UseAVX=3}}
As a result, the client could print messages like these:
{code:title=Symptoms on the HDFS client}
2023-02-21 15:21:44,380 WARN org.apache.hadoop.hdfs.DFSClient: Connection
failure: Failed to connect to for file
/tmp/.cloudera_health_monitoring_canary_files/.canary_file_2023_02_21-15_21_25.b6788e89894a61b5
for block
BP-1836197545-10.125.248.11-1672668423261:blk_1073935111_194857:com.google.protobuf.InvalidProtocolBufferException:
Protocol message tag had invalid wire type.
com.google.protobuf.InvalidProtocolBufferException: Protocol message tag had
invalid wire type.
2023-02-21 15:21:44,378 WARN org.apache.hadoop.hdfs.DFSClient: Connection
failure: Failed to connect to for file
/tmp/.cloudera_health_monitoring_canary_files/.canary_file_2023_02_21-15_21_25.b6788e89894a61b5
for block
BP-1836197545--1672668423261:blk_1073935111_194857:com.google.protobuf.InvalidProtocolBufferException:
Protocol message end-group tag did not match expected tag.
com.google.protobuf.InvalidProtocolBufferException: Protocol message end-group
tag did not match expected tag.
2023-02-21 15:06:55,530
[jira] [Created] (HADOOP-18699) InvalidProtocolBufferException caused by JDK 11 < 11.0.18 AES-CTR cipher state corruption with AVX-512 bug
Siyao Meng created HADOOP-18699:
---
Summary: InvalidProtocolBufferException caused by JDK 11 < 11.0.18
AES-CTR cipher state corruption with AVX-512 bug
Key: HADOOP-18699
URL: https://issues.apache.org/jira/browse/HADOOP-18699
Project: Hadoop Common
Issue Type: Bug
Components: hdfs-client
Reporter: Siyao Meng
This serves as a PSA for a JDK bug. Not really a bug in Hadoop / HDFS itself.
[~relek] identified [JDK-8292158|https://bugs.openjdk.org/browse/JDK-8292158]
(backported to JDK 11 in
[JDK-8295297|https://bugs.openjdk.org/browse/JDK-8295297]) causes HDFS clients
to fail with InvalidProtocolBufferException due to corrupted protobuf message
in Hadoop RPC request when all of the below conditions are met:
1. The host is capable of AVX-512 instruction sets
2. AVX-512 is enabled in JVM. This should be enabled by default on AVX-512
capable hosts, equivalent to specifying JVM arg {{-XX:UseAVX=3}}
As a result, the client could print messages like these:
{code:title=Symptoms on the HDFS client}
2023-02-21 15:21:44,380 WARN org.apache.hadoop.hdfs.DFSClient: Connection
failure: Failed to connect to for file
/tmp/.cloudera_health_monitoring_canary_files/.canary_file_2023_02_21-15_21_25.b6788e89894a61b5
for block
BP-1836197545-10.125.248.11-1672668423261:blk_1073935111_194857:com.google.protobuf.InvalidProtocolBufferException:
Protocol message tag had invalid wire type.
com.google.protobuf.InvalidProtocolBufferException: Protocol message tag had
invalid wire type.
2023-02-21 15:21:44,378 WARN org.apache.hadoop.hdfs.DFSClient: Connection
failure: Failed to connect to for file
/tmp/.cloudera_health_monitoring_canary_files/.canary_file_2023_02_21-15_21_25.b6788e89894a61b5
for block
BP-1836197545--1672668423261:blk_1073935111_194857:com.google.protobuf.InvalidProtocolBufferException:
Protocol message end-group tag did not match expected tag.
com.google.protobuf.InvalidProtocolBufferException: Protocol message end-group
tag did not match expected tag.
2023-02-21 15:06:55,530 WARN org.apache.hadoop.hdfs.DFSClient: Connection
failure: Failed to connect to for file
/tmp/.cloudera_health_monitoring_canary_files/.canary_file_2023_02_21-15_06_55.b4a633a8bde014aa
for block
BP-1836197545--1672668423261:blk_1073935025_194771:com.google.protobuf.InvalidProtocolBufferException:
While parsing a protocol message, the input ended unexpectedly in the middle
of a field. This could mean either than the input has been truncated or that an
embedded message misreported its own length.
com.google.protobuf.InvalidProtocolBufferException: While parsing a protocol
message, the input ended unexpectedly in the middle of a field. This could mean
either than the input has been truncated or that an embedded message
misreported its own length.
{code}
Solutions:
1. As a workaround, append {{-XX:UseAVX=2}} to client JVM args; or
2. Upgrade to OpenJDK >= 11.0.18.
I might post a repro test case for this, or find a way in the code to prompt
the user that this could be the potential issue (need to upgrade JDK 11) when
it occurs.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-18695) S3A: reject multipart copy requests initiated by transfer manager when disabled
[ https://issues.apache.org/jira/browse/HADOOP-18695?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HADOOP-18695: Labels: pull-request-available (was: ) > S3A: reject multipart copy requests initiated by transfer manager when > disabled > --- > > Key: HADOOP-18695 > URL: https://issues.apache.org/jira/browse/HADOOP-18695 > Project: Hadoop Common > Issue Type: Improvement > Components: fs/s3 >Affects Versions: 3.4.0 >Reporter: Steve Loughran >Assignee: Steve Loughran >Priority: Minor > Labels: pull-request-available > > follow-on to HADOOP-18637 and support for huge file uploads with stores which > don't support MPU. > * prevent use of API against any s3 store when disabled, using logging > auditor to reject it > * tests to verify rename of huge files still works (by setting large part > size) -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18695) S3A: reject multipart copy requests initiated by transfer manager when disabled
[ https://issues.apache.org/jira/browse/HADOOP-18695?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17711534#comment-17711534 ] ASF GitHub Bot commented on HADOOP-18695: - steveloughran opened a new pull request, #5548: URL: https://github.com/apache/hadoop/pull/5548 Adds a new subclass of AuditFailureException, AuditOperationRejectedException to indicate audit rejection of the operation. This is mapped to a new PathIOE subclass, UnsupportedRequestException, in the method AuditIntegration.translateAuditException(), so avoiding making S3AUtils.translateException even more complicated. New test ITestS3AHugeFilesNoMultipart extends the huge file test to verify rename worked, and that it is correctly rejected if the part size is too small. Contains HADOOP-18698 S3ABlockOutputStream doesn't collect stats on multipart block uploads This adds a new DurationTrackerFactory parameter to WriteOperation.uploadPart() and the S3AFileSystem equivalent; all callers updated to collect duration of this in their IOStatistics. ### How was this patch tested? new unit tests + new ITest ### For code changes: - [X] Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')? - [ ] Object storage: have the integration tests been executed and the endpoint declared according to the connector-specific documentation? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the `LICENSE`, `LICENSE-binary`, `NOTICE-binary` files? > S3A: reject multipart copy requests initiated by transfer manager when > disabled > --- > > Key: HADOOP-18695 > URL: https://issues.apache.org/jira/browse/HADOOP-18695 > Project: Hadoop Common > Issue Type: Improvement > Components: fs/s3 >Affects Versions: 3.4.0 >Reporter: Steve Loughran >Assignee: Steve Loughran >Priority: Minor > > follow-on to HADOOP-18637 and support for huge file uploads with stores which > don't support MPU. > * prevent use of API against any s3 store when disabled, using logging > auditor to reject it > * tests to verify rename of huge files still works (by setting large part > size) -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18695) S3A: reject multipart copy requests initiated by transfer manager when disabled
[ https://issues.apache.org/jira/browse/HADOOP-18695?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17711535#comment-17711535 ] ASF GitHub Bot commented on HADOOP-18695: - steveloughran commented on PR #5548: URL: https://github.com/apache/hadoop/pull/5548#issuecomment-1505826149 tests in progress after final changes all test failures have separate jiras created * [HADOOP-18697](https://issues.apache.org/jira/browse/HADOOP-18697) transient failure of ITestS3APrefetchingInputStream.testRandomReadLargeFile * [HADOOP-18696](https://issues.apache.org/jira/browse/HADOOP-18696) S3A ITestS3ABucketExistence access point test failure > S3A: reject multipart copy requests initiated by transfer manager when > disabled > --- > > Key: HADOOP-18695 > URL: https://issues.apache.org/jira/browse/HADOOP-18695 > Project: Hadoop Common > Issue Type: Improvement > Components: fs/s3 >Affects Versions: 3.4.0 >Reporter: Steve Loughran >Assignee: Steve Loughran >Priority: Minor > Labels: pull-request-available > > follow-on to HADOOP-18637 and support for huge file uploads with stores which > don't support MPU. > * prevent use of API against any s3 store when disabled, using logging > auditor to reject it > * tests to verify rename of huge files still works (by setting large part > size) -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] steveloughran commented on pull request #5548: HADOOP-18695. S3A: reject multipart copy requests initiated by tranfser manager when disabled
steveloughran commented on PR #5548: URL: https://github.com/apache/hadoop/pull/5548#issuecomment-1505826149 tests in progress after final changes all test failures have separate jiras created * [HADOOP-18697](https://issues.apache.org/jira/browse/HADOOP-18697) transient failure of ITestS3APrefetchingInputStream.testRandomReadLargeFile * [HADOOP-18696](https://issues.apache.org/jira/browse/HADOOP-18696) S3A ITestS3ABucketExistence access point test failure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] steveloughran opened a new pull request, #5548: HADOOP-18695. S3A: reject multipart copy requests initiated by tranfser manager when disabled
steveloughran opened a new pull request, #5548: URL: https://github.com/apache/hadoop/pull/5548 Adds a new subclass of AuditFailureException, AuditOperationRejectedException to indicate audit rejection of the operation. This is mapped to a new PathIOE subclass, UnsupportedRequestException, in the method AuditIntegration.translateAuditException(), so avoiding making S3AUtils.translateException even more complicated. New test ITestS3AHugeFilesNoMultipart extends the huge file test to verify rename worked, and that it is correctly rejected if the part size is too small. Contains HADOOP-18698 S3ABlockOutputStream doesn't collect stats on multipart block uploads This adds a new DurationTrackerFactory parameter to WriteOperation.uploadPart() and the S3AFileSystem equivalent; all callers updated to collect duration of this in their IOStatistics. ### How was this patch tested? new unit tests + new ITest ### For code changes: - [X] Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')? - [ ] Object storage: have the integration tests been executed and the endpoint declared according to the connector-specific documentation? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the `LICENSE`, `LICENSE-binary`, `NOTICE-binary` files? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #5544: hdfs auditlog add remote port info
hadoop-yetus commented on PR #5544:
URL: https://github.com/apache/hadoop/pull/5544#issuecomment-1505817155
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 34s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 3 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 15m 27s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 25m 35s | | trunk passed |
| +1 :green_heart: | compile | 23m 12s | | trunk passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | compile | 20m 40s | | trunk passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | checkstyle | 3m 47s | | trunk passed |
| +1 :green_heart: | mvnsite | 3m 25s | | trunk passed |
| +1 :green_heart: | javadoc | 2m 27s | | trunk passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javadoc | 2m 40s | | trunk passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | spotbugs | 6m 10s | | trunk passed |
| +1 :green_heart: | shadedclient | 22m 58s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 28s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 2m 20s | | the patch passed |
| +1 :green_heart: | compile | 22m 38s | | the patch passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javac | 22m 38s | | the patch passed |
| +1 :green_heart: | compile | 20m 39s | | the patch passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | javac | 20m 39s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| -0 :warning: | checkstyle | 3m 40s |
[/results-checkstyle-root.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5544/2/artifact/out/results-checkstyle-root.txt)
| root: The patch generated 9 new + 335 unchanged - 0 fixed = 344 total (was
335) |
| +1 :green_heart: | mvnsite | 3m 23s | | the patch passed |
| +1 :green_heart: | javadoc | 2m 18s | | the patch passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javadoc | 2m 42s | | the patch passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | spotbugs | 6m 15s | | the patch passed |
| +1 :green_heart: | shadedclient | 23m 19s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 18m 19s | | hadoop-common in the patch
passed. |
| -1 :x: | unit | 204m 24s |
[/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5544/2/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt)
| hadoop-hdfs in the patch passed. |
| +1 :green_heart: | asflicense | 1m 13s | | The patch does not
generate ASF License warnings. |
| | | 438m 35s | | |
| Reason | Tests |
|---:|:--|
| Failed junit tests | hadoop.hdfs.server.namenode.TestFsck |
| | hadoop.hdfs.server.namenode.TestAuditLogger |
| | hadoop.hdfs.server.namenode.TestAuditLogs |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.42 ServerAPI=1.42 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5544/2/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/5544 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux 070c94a22087 4.15.0-206-generic #217-Ubuntu SMP Fri Feb 3
19:10:13 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 0e475b9d7e3c14b7c0ea16d8b96328ec7a92fbae |
| Default Java | Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1
/usr/lib/jvm/java-8-openjdk-amd64:Private
[jira] [Updated] (HADOOP-18695) S3A: reject multipart copy requests initiated by transfer manager when disabled
[ https://issues.apache.org/jira/browse/HADOOP-18695?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-18695: Summary: S3A: reject multipart copy requests initiated by transfer manager when disabled (was: S3A: reject multipart copy requests initiated by transfer manager when multipart api disabled) > S3A: reject multipart copy requests initiated by transfer manager when > disabled > --- > > Key: HADOOP-18695 > URL: https://issues.apache.org/jira/browse/HADOOP-18695 > Project: Hadoop Common > Issue Type: Improvement > Components: fs/s3 >Affects Versions: 3.4.0 >Reporter: Steve Loughran >Assignee: Steve Loughran >Priority: Minor > > follow-on to HADOOP-18637 and support for huge file uploads with stores which > don't support MPU. > * prevent use of API against any s3 store when disabled, using logging > auditor to reject it > * tests to verify rename of huge files still works (by setting large part > size) -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-17195) Intermittent OutOfMemory error while performing hdfs CopyFromLocal to abfs
[ https://issues.apache.org/jira/browse/HADOOP-17195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17711527#comment-17711527 ] ASF GitHub Bot commented on HADOOP-17195: - ivanthewebber commented on PR #3467: URL: https://github.com/apache/hadoop/pull/3467#issuecomment-1505798380 I couldn't find anything about this in public documentation. It would be nice if it could be added there. > Intermittent OutOfMemory error while performing hdfs CopyFromLocal to abfs > --- > > Key: HADOOP-17195 > URL: https://issues.apache.org/jira/browse/HADOOP-17195 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/azure >Affects Versions: 3.3.0 >Reporter: Mehakmeet Singh >Assignee: Mehakmeet Singh >Priority: Major > Labels: abfsactive, pull-request-available > Fix For: 3.3.2 > > Time Spent: 7h 50m > Remaining Estimate: 0h > > OutOfMemory error due to new ThreadPools being made each time > AbfsOutputStream is created. Since threadPool aren't limited a lot of data is > loaded in buffer and thus it causes OutOfMemory error. > Possible fixes: > - Limit the number of ThreadCounts while performing hdfs copyFromLocal (Using > -t property). > - Reducing OUTPUT_BUFFER_SIZE significantly which would limit the amount of > buffer to be loaded in threads. > - Don't create new ThreadPools each time AbfsOutputStream is created and > limit the number of ThreadPools each AbfsOutputStream could create. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] ivanthewebber commented on pull request #3467: HADOOP-17195. ABFS: OutOfMemory error while uploading huge files (#3446)
ivanthewebber commented on PR #3467: URL: https://github.com/apache/hadoop/pull/3467#issuecomment-1505798380 I couldn't find anything about this in public documentation. It would be nice if it could be added there. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #5544: hdfs auditlog add remote port info
hadoop-yetus commented on PR #5544:
URL: https://github.com/apache/hadoop/pull/5544#issuecomment-1505778316
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 49s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 4 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 40m 48s | | trunk passed |
| +1 :green_heart: | compile | 1m 29s | | trunk passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | compile | 1m 22s | | trunk passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | checkstyle | 1m 5s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 30s | | trunk passed |
| +1 :green_heart: | javadoc | 1m 8s | | trunk passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javadoc | 1m 31s | | trunk passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | spotbugs | 3m 37s | | trunk passed |
| +1 :green_heart: | shadedclient | 25m 35s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 1m 19s | | the patch passed |
| +1 :green_heart: | compile | 1m 22s | | the patch passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javac | 1m 22s | | the patch passed |
| +1 :green_heart: | compile | 1m 14s | | the patch passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | javac | 1m 14s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| -0 :warning: | checkstyle | 0m 54s |
[/results-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5544/4/artifact/out/results-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt)
| hadoop-hdfs-project/hadoop-hdfs: The patch generated 7 new + 168 unchanged
- 0 fixed = 175 total (was 168) |
| +1 :green_heart: | mvnsite | 1m 22s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 52s | | the patch passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javadoc | 1m 27s | | the patch passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | spotbugs | 3m 31s | | the patch passed |
| +1 :green_heart: | shadedclient | 25m 47s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| -1 :x: | unit | 221m 58s |
[/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5544/4/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt)
| hadoop-hdfs in the patch passed. |
| +1 :green_heart: | asflicense | 0m 41s | | The patch does not
generate ASF License warnings. |
| | | 336m 50s | | |
| Reason | Tests |
|---:|:--|
| Failed junit tests | hadoop.hdfs.server.namenode.TestAuditLogger |
| | hadoop.hdfs.server.namenode.TestFsck |
| | hadoop.hdfs.server.datanode.TestDirectoryScanner |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.42 ServerAPI=1.42 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5544/4/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/5544 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux 10e62e4415f5 4.15.0-206-generic #217-Ubuntu SMP Fri Feb 3
19:10:13 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 3d404024085b41edcfb830f193d132220ba49937 |
| Default Java | Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5544/4/testReport/ |
| Max.
[jira] [Created] (HADOOP-18698) S3ABlockOutputStream doesn't collect stats on multipart block uploads
Steve Loughran created HADOOP-18698: --- Summary: S3ABlockOutputStream doesn't collect stats on multipart block uploads Key: HADOOP-18698 URL: https://issues.apache.org/jira/browse/HADOOP-18698 Project: Hadoop Common Issue Type: Sub-task Components: fs/s3 Affects Versions: 3.3.5 Reporter: Steve Loughran Assignee: Steve Loughran assertion failures in HADOOP-18695 show that S3ABlockOutputStream isn't collecting stats on how long the individual post calls of multipart uploads: duration or number. that is despite some counters in the IOStatistics of the stream set up to do this and merge to the FS in close() Looks like the simplest fix is to instrument the FS to do collect these, rather than in-stream. in stream would be best for context stats -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18696) S3A ITestS3ABucketExistence access point test failure
[
https://issues.apache.org/jira/browse/HADOOP-18696?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17711523#comment-17711523
]
Steve Loughran commented on HADOOP-18696:
-
oh, i understand this. the test config is created from the working fs
{code}
Configuration conf = new Configuration(getFileSystem().getConf());
{code}
which has its per-bucket settings updated.
need to do either of
* point at us-east-1 with appropriate region
* point at eu-west-1 with appropriate region
option #2 seems better.
thoughts?
> S3A ITestS3ABucketExistence access point test failure
> -
>
> Key: HADOOP-18696
> URL: https://issues.apache.org/jira/browse/HADOOP-18696
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
>Affects Versions: 3.4.0
>Reporter: Steve Loughran
>Priority: Major
>
> this is inevitably me having some config option in the way; just need to find
> out what and clear it for the test case so the probes are to the same
> region/endpont as the mock bucket
> {code}
> [ERROR]
> testAccessPointProbingV2(org.apache.hadoop.fs.s3a.ITestS3ABucketExistence)
> Time elapsed: 1.748 s <<< ERROR!
> java.lang.IllegalArgumentException: The region field of the ARN being passed
> as a bucket parameter to an S3 operation does not match the region the client
> was configured with. Provided region: 'eu-west-1'; client region: 'eu-west-2'.
> at
> com.amazonaws.services.s3.AmazonS3Client.validateIsTrue(AmazonS3Client.java:6588)
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Created] (HADOOP-18697) transient failure of ITestS3APrefetchingInputStream.testRandomReadLargeFile
Steve Loughran created HADOOP-18697:
---
Summary: transient failure of
ITestS3APrefetchingInputStream.testRandomReadLargeFile
Key: HADOOP-18697
URL: https://issues.apache.org/jira/browse/HADOOP-18697
Project: Hadoop Common
Issue Type: Sub-task
Components: fs/s3, test
Affects Versions: 3.4.0
Reporter: Steve Loughran
intermittent failure of test; caller in uk, no VPN involved. does not always
surface
{code}
testRandomReadLargeFile(org.apache.hadoop.fs.s3a.ITestS3APrefetchingInputStream)
Time elapsed: 19.913 s <<< FAILURE!
org.junit.ComparisonFailure: [Gauge named stream_read_blocks_in_cache with
expected value 3] expected:<[3]L> but was:<[2]L>
{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18696) S3A ITestS3ABucketExistence access point test failure
[
https://issues.apache.org/jira/browse/HADOOP-18696?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17711521#comment-17711521
]
Steve Loughran commented on HADOOP-18696:
-
and storediag list of properties with eu-west in them
{code}
[009] fs.s3a.endpoint = "s3.eu-west-2.amazonaws.com"
[fs.s3a.bucket.stevel-london.endpoint via [core-site.xml]]
[010] fs.s3a.endpoint.region = "eu-west-2"
[fs.s3a.bucket.stevel-london.endpoint.region via [core-site.xml]]
[073] fs.s3a.assumed.role.sts.endpoint = "sts.eu-west-2.amazonaws.com"
[core-site.xml]
[074] fs.s3a.assumed.role.sts.endpoint.region = "eu-west-2" [core-site.xml]
[087] fs.s3a.s3guard.ddb.region = "eu-west-2"
[fs.s3a.bucket.stevel-london.s3guard.ddb.region via [core-site.xml]]
{code}
aws config says us-west 2 (don't remember why)
{code}
[default]
#region = eu-west-2
region = us-west-2
output = text
{code}
> S3A ITestS3ABucketExistence access point test failure
> -
>
> Key: HADOOP-18696
> URL: https://issues.apache.org/jira/browse/HADOOP-18696
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
>Affects Versions: 3.4.0
>Reporter: Steve Loughran
>Priority: Major
>
> this is inevitably me having some config option in the way; just need to find
> out what and clear it for the test case so the probes are to the same
> region/endpont as the mock bucket
> {code}
> [ERROR]
> testAccessPointProbingV2(org.apache.hadoop.fs.s3a.ITestS3ABucketExistence)
> Time elapsed: 1.748 s <<< ERROR!
> java.lang.IllegalArgumentException: The region field of the ARN being passed
> as a bucket parameter to an S3 operation does not match the region the client
> was configured with. Provided region: 'eu-west-1'; client region: 'eu-west-2'.
> at
> com.amazonaws.services.s3.AmazonS3Client.validateIsTrue(AmazonS3Client.java:6588)
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Created] (HADOOP-18696) S3A ITestS3ABucketExistence access point test failure
Steve Loughran created HADOOP-18696:
---
Summary: S3A ITestS3ABucketExistence access point test failure
Key: HADOOP-18696
URL: https://issues.apache.org/jira/browse/HADOOP-18696
Project: Hadoop Common
Issue Type: Sub-task
Components: fs/s3
Affects Versions: 3.4.0
Reporter: Steve Loughran
this is inevitably me having some config option in the way; just need to find
out what and clear it for the test case so the probes are to the same
region/endpont as the mock bucket
{code}
[ERROR]
testAccessPointProbingV2(org.apache.hadoop.fs.s3a.ITestS3ABucketExistence)
Time elapsed: 1.748 s <<< ERROR!
java.lang.IllegalArgumentException: The region field of the ARN being passed as
a bucket parameter to an S3 operation does not match the region the client was
configured with. Provided region: 'eu-west-1'; client region: 'eu-west-2'.
at
com.amazonaws.services.s3.AmazonS3Client.validateIsTrue(AmazonS3Client.java:6588)
{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18696) S3A ITestS3ABucketExistence access point test failure
[
https://issues.apache.org/jira/browse/HADOOP-18696?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17711520#comment-17711520
]
Steve Loughran commented on HADOOP-18696:
-
full stack
{code}
[ERROR] Tests run: 6, Failures: 0, Errors: 2, Skipped: 0, Time elapsed: 7.299 s
<<< FAILURE! - in org.apache.hadoop.fs.s3a.ITestS3ABucketExistence
[ERROR]
testAccessPointProbingV2(org.apache.hadoop.fs.s3a.ITestS3ABucketExistence)
Time elapsed: 1.748 s <<< ERROR!
java.lang.IllegalArgumentException: The region field of the ARN being passed as
a bucket parameter to an S3 operation does not match the region the client was
configured with. Provided region: 'eu-west-1'; client region: 'eu-west-2'.
at
com.amazonaws.services.s3.AmazonS3Client.validateIsTrue(AmazonS3Client.java:6588)
at
com.amazonaws.services.s3.AmazonS3Client.validateS3ResourceArn(AmazonS3Client.java:5158)
at
com.amazonaws.services.s3.AmazonS3Client.createRequest(AmazonS3Client.java:4959)
at
com.amazonaws.services.s3.AmazonS3Client.createRequest(AmazonS3Client.java:4923)
at
com.amazonaws.services.s3.AmazonS3Client.getAcl(AmazonS3Client.java:4043)
at
com.amazonaws.services.s3.AmazonS3Client.getBucketAcl(AmazonS3Client.java:1278)
at
com.amazonaws.services.s3.AmazonS3Client.getBucketAcl(AmazonS3Client.java:1268)
at
org.apache.hadoop.fs.s3a.S3AFileSystem.lambda$verifyBucketExistsV2$2(S3AFileSystem.java:864)
at
org.apache.hadoop.fs.statistics.impl.IOStatisticsBinding.invokeTrackingDuration(IOStatisticsBinding.java:547)
at
org.apache.hadoop.fs.statistics.impl.IOStatisticsBinding.lambda$trackDurationOfOperation$5(IOStatisticsBinding.java:528)
at org.apache.hadoop.fs.s3a.Invoker.once(Invoker.java:122)
at org.apache.hadoop.fs.s3a.Invoker.lambda$retry$4(Invoker.java:376)
at org.apache.hadoop.fs.s3a.Invoker.retryUntranslated(Invoker.java:468)
at org.apache.hadoop.fs.s3a.Invoker.retry(Invoker.java:372)
at org.apache.hadoop.fs.s3a.Invoker.retry(Invoker.java:347)
at
org.apache.hadoop.fs.s3a.S3AFileSystem.verifyBucketExistsV2(S3AFileSystem.java:857)
at
org.apache.hadoop.fs.s3a.S3AFileSystem.doBucketProbing(S3AFileSystem.java:730)
at
org.apache.hadoop.fs.s3a.S3AFileSystem.initialize(S3AFileSystem.java:590)
at
org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:3601)
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:552)
at
org.apache.hadoop.fs.s3a.ITestS3ABucketExistence.lambda$testAccessPointProbingV2$12(ITestS3ABucketExistence.java:172)
at
org.apache.hadoop.test.LambdaTestUtils.intercept(LambdaTestUtils.java:498)
at
org.apache.hadoop.test.LambdaTestUtils.intercept(LambdaTestUtils.java:384)
at
org.apache.hadoop.fs.s3a.ITestS3ABucketExistence.expectUnknownStore(ITestS3ABucketExistence.java:103)
at
org.apache.hadoop.fs.s3a.ITestS3ABucketExistence.testAccessPointProbingV2(ITestS3ABucketExistence.java:171)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
at
org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:61)
at
org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299)
at
org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.lang.Thread.run(Thread.java:750)
[ERROR]
testAccessPointRequired(org.apache.hadoop.fs.s3a.ITestS3ABucketExistence) Time
elapsed: 0.581 s <<< ERROR!
java.lang.IllegalArgumentException: The region field of the ARN being passed as
a bucket parameter to an S3 operation does not match the region the client was
configured with. Provided region: 'eu-west-1'; client region: 'eu-west-2'.
at
com.amazonaws.services.s3.AmazonS3Client.validateIsTrue(AmazonS3Client.java:6588)
at
com.amazonaws.services.s3.AmazonS3Client.validateS3ResourceArn(AmazonS3Client.java:5158)
at
[jira] [Updated] (HADOOP-18695) S3A: reject multipart copy requests initiated by transfer manager when multipart api disabled
[ https://issues.apache.org/jira/browse/HADOOP-18695?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-18695: Summary: S3A: reject multipart copy requests initiated by transfer manager when multipart api disabled (was: s3a fs to reject multipart copy-in-rename when multipart api disabled) > S3A: reject multipart copy requests initiated by transfer manager when > multipart api disabled > - > > Key: HADOOP-18695 > URL: https://issues.apache.org/jira/browse/HADOOP-18695 > Project: Hadoop Common > Issue Type: Improvement > Components: fs/s3 >Affects Versions: 3.4.0 >Reporter: Steve Loughran >Assignee: Steve Loughran >Priority: Minor > > follow-on to HADOOP-18637 and support for huge file uploads with stores which > don't support MPU. > * prevent use of API against any s3 store when disabled, using logging > auditor to reject it > * tests to verify rename of huge files still works (by setting large part > size) -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #5544: hdfs auditlog add remote port info
hadoop-yetus commented on PR #5544:
URL: https://github.com/apache/hadoop/pull/5544#issuecomment-1505724775
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 55s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 3 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 38m 3s | | trunk passed |
| +1 :green_heart: | compile | 1m 25s | | trunk passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | compile | 1m 22s | | trunk passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | checkstyle | 1m 6s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 30s | | trunk passed |
| +1 :green_heart: | javadoc | 1m 7s | | trunk passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javadoc | 1m 36s | | trunk passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | spotbugs | 3m 25s | | trunk passed |
| +1 :green_heart: | shadedclient | 22m 39s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 1m 18s | | the patch passed |
| +1 :green_heart: | compile | 1m 19s | | the patch passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javac | 1m 19s | | the patch passed |
| +1 :green_heart: | compile | 1m 11s | | the patch passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | javac | 1m 11s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| -0 :warning: | checkstyle | 0m 52s |
[/results-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5544/3/artifact/out/results-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt)
| hadoop-hdfs-project/hadoop-hdfs: The patch generated 7 new + 143 unchanged
- 0 fixed = 150 total (was 143) |
| +1 :green_heart: | mvnsite | 1m 20s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 50s | | the patch passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javadoc | 1m 24s | | the patch passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | spotbugs | 3m 18s | | the patch passed |
| +1 :green_heart: | shadedclient | 22m 32s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| -1 :x: | unit | 246m 56s |
[/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5544/3/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt)
| hadoop-hdfs in the patch passed. |
| +1 :green_heart: | asflicense | 0m 49s | | The patch does not
generate ASF License warnings. |
| | | 353m 4s | | |
| Reason | Tests |
|---:|:--|
| Failed junit tests | hadoop.hdfs.server.namenode.TestFsck |
| | hadoop.hdfs.server.namenode.TestAuditLogger |
| | hadoop.hdfs.TestRollingUpgrade |
| | hadoop.hdfs.server.datanode.TestDirectoryScanner |
| | hadoop.hdfs.server.namenode.TestAuditLogs |
| | hadoop.hdfs.server.namenode.ha.TestObserverNode |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.42 ServerAPI=1.42 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5544/3/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/5544 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux f27bf9a14596 4.15.0-206-generic #217-Ubuntu SMP Fri Feb 3
19:10:13 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 1c6403afe93d99f4935375fe5fb32f8767fb9541 |
| Default Java | Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1
/usr/lib/jvm/java-8-openjdk-amd64:Private
[GitHub] [hadoop] melissayou commented on a diff in pull request #5536: [HDFS-16971] Add read metrics for remote reads in FileSystem Statistics #5534
melissayou commented on code in PR #5536:
URL: https://github.com/apache/hadoop/pull/5536#discussion_r1164487062
##
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileSystem.java:
##
@@ -3942,6 +3942,7 @@ public static class StatisticsData {
private volatile long bytesReadDistanceOfThreeOrFour;
private volatile long bytesReadDistanceOfFiveOrLarger;
private volatile long bytesReadErasureCoded;
+ private volatile long remoteBytesReadTimeMS;
Review Comment:
renamed all variables to get rid of Bytes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] melissayou commented on a diff in pull request #5536: [HDFS-16971] Add read metrics for remote reads in FileSystem Statistics #5534
melissayou commented on code in PR #5536:
URL: https://github.com/apache/hadoop/pull/5536#discussion_r1164453134
##
hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestFileSystemStorageStatistics.java:
##
@@ -74,6 +75,7 @@ public void setup() {
statistics.incrementBytesReadByDistance(1, RandomUtils.nextInt(0, 100));
statistics.incrementBytesReadByDistance(3, RandomUtils.nextInt(0, 100));
statistics.incrementBytesReadErasureCoded(RandomUtils.nextInt(0, 100));
+statistics.increaseRemoteBytesReadTime(RandomUtils.nextInt(0, 100));
Review Comment:
The existing test method `testGetLongStatistics` and `testGetLong` applies
to any newly added field as well. While running this test locally, I can see
assertion results like below
```
2023-04-12 10:37:10,362 INFO fs.TestFileSystemStorageStatistics
(TestFileSystemStorageStatistics.java:testGetLong(99)) - remoteReadTimeMS:
FileSystem.Statistics=75, FileSystemStorageStatistics=75
2023-04-12 10:37:10,353 INFO fs.TestFileSystemStorageStatistics
(TestFileSystemStorageStatistics.java:testGetLongStatistics(88)) -
remoteReadTimeMS: FileSystem.Statistics=10, FileSystemStorageStatistics=10
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] goiri merged pull request #4687: YARN-10846. Add dispatcher metrics to NM.
goiri merged PR #4687: URL: https://github.com/apache/hadoop/pull/4687 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #5545: add rpc metrics for response time
hadoop-yetus commented on PR #5545:
URL: https://github.com/apache/hadoop/pull/5545#issuecomment-1505562751
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 46s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| -1 :x: | test4tests | 0m 0s | | The patch doesn't appear to include
any new or modified tests. Please justify why no new tests are needed for this
patch. Also please list what manual steps were performed to verify this patch.
|
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 41m 25s | | trunk passed |
| +1 :green_heart: | compile | 25m 9s | | trunk passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | compile | 21m 43s | | trunk passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | checkstyle | 1m 6s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 39s | | trunk passed |
| +1 :green_heart: | javadoc | 1m 8s | | trunk passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javadoc | 0m 43s | | trunk passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | spotbugs | 2m 44s | | trunk passed |
| +1 :green_heart: | shadedclient | 25m 23s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 1m 1s | | the patch passed |
| +1 :green_heart: | compile | 24m 26s | | the patch passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javac | 24m 26s | | the patch passed |
| +1 :green_heart: | compile | 21m 42s | | the patch passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | javac | 21m 42s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| -0 :warning: | checkstyle | 0m 59s |
[/results-checkstyle-hadoop-common-project_hadoop-common.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5545/1/artifact/out/results-checkstyle-hadoop-common-project_hadoop-common.txt)
| hadoop-common-project/hadoop-common: The patch generated 2 new + 192
unchanged - 0 fixed = 194 total (was 192) |
| +1 :green_heart: | mvnsite | 1m 36s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 59s | | the patch passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javadoc | 0m 41s | | the patch passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | spotbugs | 2m 41s | | the patch passed |
| +1 :green_heart: | shadedclient | 25m 36s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 18m 12s | | hadoop-common in the patch
passed. |
| +1 :green_heart: | asflicense | 0m 52s | | The patch does not
generate ASF License warnings. |
| | | 220m 23s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.42 ServerAPI=1.42 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5545/1/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/5545 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux d05eb22b5266 4.15.0-206-generic #217-Ubuntu SMP Fri Feb 3
19:10:13 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 50e9110954da85dd9337d4f5ebe899b77cc8bf5f |
| Default Java | Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5545/1/testReport/ |
| Max. process+thread count | 1353 (vs. ulimit of 5500) |
| modules | C: hadoop-common-project/hadoop-common U:
hadoop-common-project/hadoop-common |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5545/1/console |
| versions |
[GitHub] [hadoop] hadoop-yetus commented on pull request #4687: YARN-10846. Add dispatcher metrics to NM.
hadoop-yetus commented on PR #4687:
URL: https://github.com/apache/hadoop/pull/4687#issuecomment-1505553707
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 48s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available.
|
| +0 :ok: | xmllint | 0m 1s | | xmllint was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 5 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 15m 6s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 27m 36s | | trunk passed |
| +1 :green_heart: | compile | 10m 34s | | trunk passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | compile | 9m 14s | | trunk passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | checkstyle | 1m 46s | | trunk passed |
| +1 :green_heart: | mvnsite | 4m 43s | | trunk passed |
| +1 :green_heart: | javadoc | 4m 13s | | trunk passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javadoc | 4m 1s | | trunk passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +0 :ok: | spotbugs | 0m 39s | |
branch/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests
no spotbugs output file (spotbugsXml.xml) |
| +1 :green_heart: | shadedclient | 21m 45s | | branch has no errors
when building and testing our client artifacts. |
| -0 :warning: | patch | 22m 6s | | Used diff version of patch file.
Binary files and potentially other changes not applied. Please rebase and
squash commits if necessary. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 30s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 3m 19s | | the patch passed |
| +1 :green_heart: | compile | 9m 54s | | the patch passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javac | 9m 54s | | the patch passed |
| +1 :green_heart: | compile | 9m 1s | | the patch passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | javac | 9m 1s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 1m 38s | |
hadoop-yarn-project/hadoop-yarn: The patch generated 0 new + 253 unchanged - 1
fixed = 253 total (was 254) |
| +1 :green_heart: | mvnsite | 4m 42s | | the patch passed |
| +1 :green_heart: | javadoc | 3m 54s | | the patch passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javadoc | 3m 47s | | the patch passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +0 :ok: | spotbugs | 0m 38s | |
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests has
no data from spotbugs |
| +1 :green_heart: | shadedclient | 20m 34s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 1m 13s | | hadoop-yarn-api in the patch
passed. |
| +1 :green_heart: | unit | 5m 38s | | hadoop-yarn-common in the patch
passed. |
| +1 :green_heart: | unit | 101m 45s | |
hadoop-yarn-server-resourcemanager in the patch passed. |
| +1 :green_heart: | unit | 24m 17s | | hadoop-yarn-server-nodemanager
in the patch passed. |
| +1 :green_heart: | unit | 3m 32s | | hadoop-yarn-server-tests in the
patch passed. |
| +1 :green_heart: | asflicense | 0m 56s | | The patch does not
generate ASF License warnings. |
| | | 317m 5s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.42 ServerAPI=1.42 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4687/22/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/4687 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets xmllint |
| uname | Linux 1a1e584fb237 4.15.0-206-generic #217-Ubuntu SMP Fri Feb 3
19:10:13 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk /
[GitHub] [hadoop] YuanbenWang opened a new pull request, #5547: HDFS-16977. Forbid assigned characters in pathname when new file or directory is created.
YuanbenWang opened a new pull request, #5547: URL: https://github.com/apache/hadoop/pull/5547 JIRA: [HDFS-16977](https://issues.apache.org/jira/browse/HDFS-16977). Forbid assigned characters in pathname when new file or directory is created. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] szetszwo opened a new pull request, #5546: HDFS-16975. FileWithSnapshotFeature.isCurrentFileDeleted is not reloaded from FSImage.
szetszwo opened a new pull request, #5546: URL: https://github.com/apache/hadoop/pull/5546 ### Description of PR FileWithSnapshotFeature.isCurrentFileDeleted is not serialized to FSImage. If it is true for any files, it will becomes false after saving to a FSImage and loading it back. We will fix it by checking `isInCurrentState()` when loading FSImage. https://issues.apache.org/jira/browse/HDFS-16975 ### How was this patch tested? Added "isCurrentFileDeleted" to `FileWithSnapshotFeature.toString()`. Then, `TestFSImageWithSnapshot.testSaveLoadImage()` will fail without the fix. ### For code changes: - [X] Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')? - [NA] Object storage: have the integration tests been executed and the endpoint declared according to the connector-specific documentation? - [NA] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [NA] If applicable, have you updated the `LICENSE`, `LICENSE-binary`, `NOTICE-binary` files? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18637) S3A to support upload of files greater than 2 GB using DiskBlocks
[ https://issues.apache.org/jira/browse/HADOOP-18637?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17711434#comment-17711434 ] Harshit Gupta commented on HADOOP-18637: Sure, I will get on it. > S3A to support upload of files greater than 2 GB using DiskBlocks > - > > Key: HADOOP-18637 > URL: https://issues.apache.org/jira/browse/HADOOP-18637 > Project: Hadoop Common > Issue Type: Improvement > Components: fs/s3 >Reporter: Harshit Gupta >Assignee: Harshit Gupta >Priority: Major > Labels: pull-request-available > > Use S3A Diskblocks to support the upload of files greater than 2 GB using > DiskBlocks. Currently, the max upload size of a single block is ~2GB. > cc: [~mthakur] [~ste...@apache.org] [~mehakmeet] -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18695) s3a fs to reject multipart copy-in-rename when multipart api disabled
[ https://issues.apache.org/jira/browse/HADOOP-18695?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17711424#comment-17711424 ] Steve Loughran commented on HADOOP-18695: - auditor will raise access denied exception, ``` [ERROR] Tests run: 10, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 21.882 s <<< FAILURE! - in org.apache.hadoop.fs.s3a.scale.ITestS3AHugeFilesSinglePut [ERROR] test_100_renameHugeFile(org.apache.hadoop.fs.s3a.scale.ITestS3AHugeFilesSinglePut) Time elapsed: 1.363 s <<< ERROR! java.nio.file.AccessDeniedException: tests3ascale/disk/hugefile: org.apache.hadoop.fs.s3a.audit.AuditFailureException: Multipart IO request com.amazonaws.services.s3.model.EncryptedInitiateMultipartUploadRequest@6781cfbe rejected https://audit.example.org/hadoop/1/op_rename/458f1c6e-aa1f-46da-a46b-9c03c30d2c7f-0049/?op=op_rename=s3a://stevel-london/tests3ascale/disk/hugefile=stevel=s3a://stevel-london/tests3ascale/disk/hugefileRenamed=7d5e3e98-6b66-49de-a7d2-e6b9281fa410=458f1c6e-aa1f-46da-a46b-9c03c30d2c7f-0049=94=458f1c6e-aa1f-46da-a46b-9c03c30d2c7f=102=1681306681816 at org.apache.hadoop.fs.s3a.S3AUtils.translateException(S3AUtils.java:211) at org.apache.hadoop.fs.s3a.Invoker.once(Invoker.java:124) at org.apache.hadoop.fs.s3a.Invoker.lambda$retry$4(Invoker.java:376) at org.apache.hadoop.fs.s3a.Invoker.retryUntranslated(Invoker.java:468) at org.apache.hadoop.fs.s3a.Invoker.retry(Invoker.java:372) at org.apache.hadoop.fs.s3a.Invoker.retry(Invoker.java:347) at org.apache.hadoop.fs.s3a.S3AFileSystem.copyFile(S3AFileSystem.java:4254) at org.apache.hadoop.fs.s3a.S3AFileSystem.access$1300(S3AFileSystem.java:266) at org.apache.hadoop.fs.s3a.S3AFileSystem$OperationCallbacksImpl.copyFile(S3AFileSystem.java:2275) at org.apache.hadoop.fs.s3a.impl.RenameOperation.copySource(RenameOperation.java:562) at org.apache.hadoop.fs.s3a.impl.RenameOperation.renameFileToDest(RenameOperation.java:313) at org.apache.hadoop.fs.s3a.impl.RenameOperation.execute(RenameOperation.java:267) at org.apache.hadoop.fs.s3a.S3AFileSystem.innerRename(S3AFileSystem.java:2194) at org.apache.hadoop.fs.s3a.S3AFileSystem.lambda$rename$8(S3AFileSystem.java:2045) at org.apache.hadoop.fs.statistics.impl.IOStatisticsBinding.invokeTrackingDuration(IOStatisticsBinding.java:547) at org.apache.hadoop.fs.statistics.impl.IOStatisticsBinding.lambda$trackDurationOfOperation$5(IOStatisticsBinding.java:528) at org.apache.hadoop.fs.statistics.impl.IOStatisticsBinding.trackDuration(IOStatisticsBinding.java:449) at org.apache.hadoop.fs.s3a.S3AFileSystem.trackDurationAndSpan(S3AFileSystem.java:2470) at org.apache.hadoop.fs.s3a.S3AFileSystem.rename(S3AFileSystem.java:2043) at org.apache.hadoop.fs.s3a.scale.AbstractSTestS3AHugeFiles.test_100_renameHugeFile(AbstractSTestS3AHugeFiles.java:552) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:61) at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299) at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.lang.Thread.run(Thread.java:750) Caused by: org.apache.hadoop.fs.s3a.audit.AuditFailureException: Multipart IO request com.amazonaws.services.s3.model.EncryptedInitiateMultipartUploadRequest@6781cfbe rejected https://audit.example.org/hadoop/1/op_rename/458f1c6e-aa1f-46da-a46b-9c03c30d2c7f-0049/?op=op_rename=s3a://stevel-london/tests3ascale/disk/hugefile=stevel=s3a://stevel-london/tests3ascale/disk/hugefileRenamed=7d5e3e98-6b66-49de-a7d2-e6b9281fa410=458f1c6e-aa1f-46da-a46b-9c03c30d2c7f-0049=94=458f1c6e-aa1f-46da-a46b-9c03c30d2c7f=102=1681306681816 at org.apache.hadoop.fs.s3a.audit.impl.LoggingAuditor$LoggingAuditSpan.beforeExecution(LoggingAuditor.java:380) at
[jira] [Commented] (HADOOP-18689) Bump jettison from 1.5.3 to 1.5.4 in /hadoop-project
[ https://issues.apache.org/jira/browse/HADOOP-18689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17711396#comment-17711396 ] ASF GitHub Bot commented on HADOOP-18689: - ayushtkn commented on PR #5502: URL: https://github.com/apache/hadoop/pull/5502#issuecomment-1505308898 I tried ``TestRuntimeEstimators`` locally and it passed. ``` [INFO] --- [INFO] T E S T S [INFO] --- [INFO] Running org.apache.hadoop.mapreduce.v2.app.TestRuntimeEstimators [INFO] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 28.899 s - in org.apache.hadoop.mapreduce.v2.app.TestRuntimeEstimators [INFO] [INFO] Results: [INFO] [INFO] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0 ``` Other ones I didn't try because they are already failing in the daily build, so not related. https://ci-hadoop.apache.org/view/Hadoop/job/hadoop-qbt-trunk-java8-linux-x86_64/1192/testReport/ & https://ci-hadoop.apache.org/view/Hadoop/job/hadoop-qbt-trunk-java8-linux-x86_64/1193/testReport/ > Bump jettison from 1.5.3 to 1.5.4 in /hadoop-project > > > Key: HADOOP-18689 > URL: https://issues.apache.org/jira/browse/HADOOP-18689 > Project: Hadoop Common > Issue Type: Improvement > Components: common >Affects Versions: 3.4.0, 3.3.9 >Reporter: Ayush Saxena >Priority: Major > Labels: pull-request-available > > PR from github depandabot > https://github.com/apache/hadoop/pull/5502 > Mentions CVE: https://nvd.nist.gov/vuln/detail/CVE-2023-1436 > Creating ticket for tracking. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] ayushtkn commented on pull request #5502: HADOOP-18689. Bump jettison from 1.5.3 to 1.5.4 in /hadoop-project
ayushtkn commented on PR #5502: URL: https://github.com/apache/hadoop/pull/5502#issuecomment-1505308898 I tried ``TestRuntimeEstimators`` locally and it passed. ``` [INFO] --- [INFO] T E S T S [INFO] --- [INFO] Running org.apache.hadoop.mapreduce.v2.app.TestRuntimeEstimators [INFO] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 28.899 s - in org.apache.hadoop.mapreduce.v2.app.TestRuntimeEstimators [INFO] [INFO] Results: [INFO] [INFO] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0 ``` Other ones I didn't try because they are already failing in the daily build, so not related. https://ci-hadoop.apache.org/view/Hadoop/job/hadoop-qbt-trunk-java8-linux-x86_64/1192/testReport/ & https://ci-hadoop.apache.org/view/Hadoop/job/hadoop-qbt-trunk-java8-linux-x86_64/1193/testReport/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-18695) s3a fs to reject multipart copy-in-rename when multipart api disabled
[ https://issues.apache.org/jira/browse/HADOOP-18695?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-18695: Priority: Minor (was: Major) > s3a fs to reject multipart copy-in-rename when multipart api disabled > - > > Key: HADOOP-18695 > URL: https://issues.apache.org/jira/browse/HADOOP-18695 > Project: Hadoop Common > Issue Type: Improvement > Components: fs/s3 >Affects Versions: 3.4.0 >Reporter: Steve Loughran >Assignee: Steve Loughran >Priority: Minor > > follow-on to HADOOP-18637 and support for huge file uploads with stores which > don't support MPU. > * prevent use of API against any s3 store when disabled, using logging > auditor to reject it > * tests to verify rename of huge files still works (by setting large part > size) -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Created] (HADOOP-18695) s3a fs to reject multipart copy-in-rename when multipart api disabled
Steve Loughran created HADOOP-18695: --- Summary: s3a fs to reject multipart copy-in-rename when multipart api disabled Key: HADOOP-18695 URL: https://issues.apache.org/jira/browse/HADOOP-18695 Project: Hadoop Common Issue Type: Improvement Components: fs/s3 Affects Versions: 3.4.0 Reporter: Steve Loughran Assignee: Steve Loughran follow-on to HADOOP-18637 and support for huge file uploads with stores which don't support MPU. * prevent use of API against any s3 store when disabled, using logging auditor to reject it * tests to verify rename of huge files still works (by setting large part size) -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18637) S3A to support upload of files greater than 2 GB using DiskBlocks
[ https://issues.apache.org/jira/browse/HADOOP-18637?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17711390#comment-17711390 ] Steve Loughran commented on HADOOP-18637: - [~harshit.gupta] can you do a PR of the trunk commit against branch-3.3 while I do a followup? > S3A to support upload of files greater than 2 GB using DiskBlocks > - > > Key: HADOOP-18637 > URL: https://issues.apache.org/jira/browse/HADOOP-18637 > Project: Hadoop Common > Issue Type: Improvement > Components: fs/s3 >Reporter: Harshit Gupta >Assignee: Harshit Gupta >Priority: Major > Labels: pull-request-available > > Use S3A Diskblocks to support the upload of files greater than 2 GB using > DiskBlocks. Currently, the max upload size of a single block is ~2GB. > cc: [~mthakur] [~ste...@apache.org] [~mehakmeet] -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18637) S3A to support upload of files greater than 2 GB using DiskBlocks
[ https://issues.apache.org/jira/browse/HADOOP-18637?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17711389#comment-17711389 ] ASF GitHub Bot commented on HADOOP-18637: - steveloughran commented on PR #5543: URL: https://github.com/apache/hadoop/pull/5543#issuecomment-1505272396 I have a followup for this feature, primarily to reject multipart copy requests when disabled, test to verify that for a large enough threshold, calls don't get rejected. > S3A to support upload of files greater than 2 GB using DiskBlocks > - > > Key: HADOOP-18637 > URL: https://issues.apache.org/jira/browse/HADOOP-18637 > Project: Hadoop Common > Issue Type: Improvement > Components: fs/s3 >Reporter: Harshit Gupta >Assignee: Harshit Gupta >Priority: Major > Labels: pull-request-available > > Use S3A Diskblocks to support the upload of files greater than 2 GB using > DiskBlocks. Currently, the max upload size of a single block is ~2GB. > cc: [~mthakur] [~ste...@apache.org] [~mehakmeet] -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] steveloughran commented on pull request #5543: HADOOP-18637. S3A to support upload of files greater than 2 GB using DiskBlocks
steveloughran commented on PR #5543: URL: https://github.com/apache/hadoop/pull/5543#issuecomment-1505272396 I have a followup for this feature, primarily to reject multipart copy requests when disabled, test to verify that for a large enough threshold, calls don't get rejected. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-17377) ABFS: MsiTokenProvider doesn't retry HTTP 429 from the Instance Metadata Service
[
https://issues.apache.org/jira/browse/HADOOP-17377?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17711386#comment-17711386
]
ASF GitHub Bot commented on HADOOP-17377:
-
steveloughran commented on code in PR #5273:
URL: https://github.com/apache/hadoop/pull/5273#discussion_r1164119014
##
hadoop-tools/hadoop-azure/pom.xml:
##
@@ -321,8 +321,23 @@
org.mockito
mockito-core
+ 4.11.0
Review Comment:
sorry, hadoop-project defines the version, and through properties. revert
this
##
hadoop-tools/hadoop-azure/pom.xml:
##
@@ -321,8 +321,23 @@
org.mockito
mockito-core
+ 4.11.0
test
+
+
+ org.mockito
+ mockito-inline
+ 4.11.0
Review Comment:
if this is new to hadoop, declare it in hadoop-project/pom.xml, with
versions and exclusions, then declare here without those
##
hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/ITestAbfsMsiTokenProvider.java:
##
@@ -40,13 +47,16 @@
import static
org.apache.hadoop.fs.azurebfs.constants.ConfigurationKeys.FS_AZURE_ACCOUNT_OAUTH_MSI_AUTHORITY;
import static
org.apache.hadoop.fs.azurebfs.constants.ConfigurationKeys.FS_AZURE_ACCOUNT_OAUTH_MSI_ENDPOINT;
import static
org.apache.hadoop.fs.azurebfs.constants.ConfigurationKeys.FS_AZURE_ACCOUNT_OAUTH_MSI_TENANT;
+import static org.mockito.Mockito.times;
/**
* Test MsiTokenProvider.
*/
public final class ITestAbfsMsiTokenProvider
extends AbstractAbfsIntegrationTest {
+ private static final int HTTP_TOO_MANY_REQUESTS = 429;
Review Comment:
refer to the value in the src/main code
##
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/ExponentialRetryPolicy.java:
##
@@ -58,6 +58,13 @@ public class ExponentialRetryPolicy {
*/
private static final double MAX_RANDOM_RATIO = 1.2;
+ /**
+ * Qualifies for retry based on
Review Comment:
needs a . in it, maybe split into
"qualifies for retry."
and "see..."
##
hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/ITestAbfsMsiTokenProvider.java:
##
@@ -90,4 +100,109 @@ private String getTrimmedPasswordString(AbfsConfiguration
conf, String key,
return value.trim();
}
+ /**
+ * Test to verify that token fetch is retried for throttling errors (too
many requests 429).
+ * @throws Exception
+ */
+ @Test
+ public void testRetryForThrottling() throws Exception {
+AbfsConfiguration conf = getConfiguration();
+
+// Exception to be thrown with throttling error code 429.
+AzureADAuthenticator.HttpException httpException
+= new AzureADAuthenticator.HttpException(HTTP_TOO_MANY_REQUESTS,
+"abc", "abc", "abc", "abc", "abc");
+
+String tenantGuid = "abcd";
+String clientId = "abcd";
+String authEndpoint = getTrimmedPasswordString(conf,
Review Comment:
what if these are undefined? skip the test?
##
hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/ITestAbfsMsiTokenProvider.java:
##
@@ -90,4 +100,109 @@ private String getTrimmedPasswordString(AbfsConfiguration
conf, String key,
return value.trim();
}
+ /**
+ * Test to verify that token fetch is retried for throttling errors (too
many requests 429).
+ * @throws Exception
Review Comment:
cut this @throws
##
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/oauth2/AzureADAuthenticator.java:
##
@@ -341,7 +341,7 @@ private static boolean isRecoverableFailure(IOException e) {
|| e instanceof FileNotFoundException);
}
- private static AzureADToken getTokenSingleCall(String authEndpoint,
+ public static AzureADToken getTokenSingleCall(String authEndpoint,
Review Comment:
now it is public, add a javadoc
##
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/ExponentialRetryPolicy.java:
##
@@ -58,6 +58,13 @@ public class ExponentialRetryPolicy {
*/
private static final double MAX_RANDOM_RATIO = 1.2;
+ /**
+ * Qualifies for retry based on
+ * https://learn.microsoft.com/en-us/azure/active-directory/
+ * managed-identities-azure-resources/how-to-use-vm-token#error-handling
+ */
+ private static final int HTTP_TOO_MANY_REQUESTS = 429;
Review Comment:
make public and refer from tests, maybe put in a different file for this
> ABFS: MsiTokenProvider doesn't retry HTTP 429 from the Instance Metadata
> Service
>
>
> Key: HADOOP-17377
> URL: https://issues.apache.org/jira/browse/HADOOP-17377
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs/azure
>Affects Versions: 3.2.1
>Reporter: Brandon
>
[GitHub] [hadoop] steveloughran commented on a diff in pull request #5273: HADOOP-17377: AABFS: MsiTokenProvider doesn't retry HTTP 429/410 from the Instance Metadata Service
steveloughran commented on code in PR #5273:
URL: https://github.com/apache/hadoop/pull/5273#discussion_r1164119014
##
hadoop-tools/hadoop-azure/pom.xml:
##
@@ -321,8 +321,23 @@
org.mockito
mockito-core
+ 4.11.0
Review Comment:
sorry, hadoop-project defines the version, and through properties. revert
this
##
hadoop-tools/hadoop-azure/pom.xml:
##
@@ -321,8 +321,23 @@
org.mockito
mockito-core
+ 4.11.0
test
+
+
+ org.mockito
+ mockito-inline
+ 4.11.0
Review Comment:
if this is new to hadoop, declare it in hadoop-project/pom.xml, with
versions and exclusions, then declare here without those
##
hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/ITestAbfsMsiTokenProvider.java:
##
@@ -40,13 +47,16 @@
import static
org.apache.hadoop.fs.azurebfs.constants.ConfigurationKeys.FS_AZURE_ACCOUNT_OAUTH_MSI_AUTHORITY;
import static
org.apache.hadoop.fs.azurebfs.constants.ConfigurationKeys.FS_AZURE_ACCOUNT_OAUTH_MSI_ENDPOINT;
import static
org.apache.hadoop.fs.azurebfs.constants.ConfigurationKeys.FS_AZURE_ACCOUNT_OAUTH_MSI_TENANT;
+import static org.mockito.Mockito.times;
/**
* Test MsiTokenProvider.
*/
public final class ITestAbfsMsiTokenProvider
extends AbstractAbfsIntegrationTest {
+ private static final int HTTP_TOO_MANY_REQUESTS = 429;
Review Comment:
refer to the value in the src/main code
##
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/ExponentialRetryPolicy.java:
##
@@ -58,6 +58,13 @@ public class ExponentialRetryPolicy {
*/
private static final double MAX_RANDOM_RATIO = 1.2;
+ /**
+ * Qualifies for retry based on
Review Comment:
needs a . in it, maybe split into
"qualifies for retry."
and "see..."
##
hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/ITestAbfsMsiTokenProvider.java:
##
@@ -90,4 +100,109 @@ private String getTrimmedPasswordString(AbfsConfiguration
conf, String key,
return value.trim();
}
+ /**
+ * Test to verify that token fetch is retried for throttling errors (too
many requests 429).
+ * @throws Exception
+ */
+ @Test
+ public void testRetryForThrottling() throws Exception {
+AbfsConfiguration conf = getConfiguration();
+
+// Exception to be thrown with throttling error code 429.
+AzureADAuthenticator.HttpException httpException
+= new AzureADAuthenticator.HttpException(HTTP_TOO_MANY_REQUESTS,
+"abc", "abc", "abc", "abc", "abc");
+
+String tenantGuid = "abcd";
+String clientId = "abcd";
+String authEndpoint = getTrimmedPasswordString(conf,
Review Comment:
what if these are undefined? skip the test?
##
hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/ITestAbfsMsiTokenProvider.java:
##
@@ -90,4 +100,109 @@ private String getTrimmedPasswordString(AbfsConfiguration
conf, String key,
return value.trim();
}
+ /**
+ * Test to verify that token fetch is retried for throttling errors (too
many requests 429).
+ * @throws Exception
Review Comment:
cut this @throws
##
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/oauth2/AzureADAuthenticator.java:
##
@@ -341,7 +341,7 @@ private static boolean isRecoverableFailure(IOException e) {
|| e instanceof FileNotFoundException);
}
- private static AzureADToken getTokenSingleCall(String authEndpoint,
+ public static AzureADToken getTokenSingleCall(String authEndpoint,
Review Comment:
now it is public, add a javadoc
##
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/ExponentialRetryPolicy.java:
##
@@ -58,6 +58,13 @@ public class ExponentialRetryPolicy {
*/
private static final double MAX_RANDOM_RATIO = 1.2;
+ /**
+ * Qualifies for retry based on
+ * https://learn.microsoft.com/en-us/azure/active-directory/
+ * managed-identities-azure-resources/how-to-use-vm-token#error-handling
+ */
+ private static final int HTTP_TOO_MANY_REQUESTS = 429;
Review Comment:
make public and refer from tests, maybe put in a different file for this
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18154) S3A Authentication to support WebIdentity
[ https://issues.apache.org/jira/browse/HADOOP-18154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17711381#comment-17711381 ] ASF GitHub Bot commented on HADOOP-18154: - steveloughran commented on PR #4070: URL: https://github.com/apache/hadoop/pull/4070#issuecomment-1505242360 @insider89 not been any changes on this PR since it was last reviewed. if the author takes up the pr, addresses those issues etc then we can work on getting it in to hadoop. I would suggest asking the author what the status is, and then working with them and other interested parties to get it into state where the reviewers are happy. I am not set up to test this, so a key role of those who need this is to verify the patch works. > S3A Authentication to support WebIdentity > - > > Key: HADOOP-18154 > URL: https://issues.apache.org/jira/browse/HADOOP-18154 > Project: Hadoop Common > Issue Type: Improvement > Components: fs/s3 >Affects Versions: 2.10.1 >Reporter: Ju Clarysse >Assignee: Ju Clarysse >Priority: Major > Labels: pull-request-available > Time Spent: 3h 20m > Remaining Estimate: 0h > > We are using the latest version of > [delta-sharing|https://github.com/delta-io/delta-sharing] which takes > advantage of > [hadoop-aws|https://hadoop.apache.org/docs/current/hadoop-aws/tools/hadoop-aws/index.html] > (S3A) connector in [Hadoop release version > 2.10.1|https://github.com/apache/hadoop/tree/rel/release-2.10.1] to mount an > AWS S3 File System. In our particular setup, all services are operated in > Amazon Elastic Kubernetes Service (EKS) and need to comply to the AWS > security concept [IAM roles for service > accounts|https://docs.aws.amazon.com/eks/latest/userguide/iam-roles-for-service-accounts.html] > (IRSA). > As [Delta sharing S3 connection|https://github.com/delta-io/delta-sharing#s3] > doesn't offer any corresponding support, we patched hadoop-aws-2.10.1 to > address this need via a new credentials provider class > org.apache.hadoop.fs.s3a.OIDCTokenCredentialsProvider. We also upgraded > dependency aws-java-sdk-bundle to its latest version 1.12.167 as [AWS > WebIdentityTokenCredentialsProvider > class|https://docs.aws.amazon.com/AWSJavaSDK/latest/javadoc/com/amazonaws/auth/WebIdentityTokenCredentialsProvider.html%E2%80%A6] > was not yet available in original version 1.11.271. > We believe that other delta-sharing users could benefit from this short-term > contribution. Then sooner or later, delta-sharing owners will have to upgrade > their project to a more recent version of hadoop-aws that is probably more > widely used. The effort to promote this change is probably low. > Additional note: AWS WebIdentityTokenCredentialsProvider class is directly > supported by Spark applications submitted with configuration properties > `spark.hadoop.fs.s3a.aws.credentials.provider`and > `spark.kubernetes.authenticate.submission.oauthToken` > ([doc|https://spark.apache.org/docs/latest/running-on-kubernetes.html#spark-properties]). > So bringing this support to Hadoop will primarily be interesting for > non-Spark users. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] steveloughran commented on pull request #4070: HADOOP-18154. S3A Authentication to support WebIdentity
steveloughran commented on PR #4070: URL: https://github.com/apache/hadoop/pull/4070#issuecomment-1505242360 @insider89 not been any changes on this PR since it was last reviewed. if the author takes up the pr, addresses those issues etc then we can work on getting it in to hadoop. I would suggest asking the author what the status is, and then working with them and other interested parties to get it into state where the reviewers are happy. I am not set up to test this, so a key role of those who need this is to verify the patch works. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18689) Bump jettison from 1.5.3 to 1.5.4 in /hadoop-project
[ https://issues.apache.org/jira/browse/HADOOP-18689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17711372#comment-17711372 ] ASF GitHub Bot commented on HADOOP-18689: - steveloughran commented on PR #5502: URL: https://github.com/apache/hadoop/pull/5502#issuecomment-1505233783 you think those test failures are related? (unlikely, i know...) > Bump jettison from 1.5.3 to 1.5.4 in /hadoop-project > > > Key: HADOOP-18689 > URL: https://issues.apache.org/jira/browse/HADOOP-18689 > Project: Hadoop Common > Issue Type: Improvement > Components: common >Affects Versions: 3.4.0, 3.3.9 >Reporter: Ayush Saxena >Priority: Major > Labels: pull-request-available > > PR from github depandabot > https://github.com/apache/hadoop/pull/5502 > Mentions CVE: https://nvd.nist.gov/vuln/detail/CVE-2023-1436 > Creating ticket for tracking. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] steveloughran commented on pull request #5502: HADOOP-18689. Bump jettison from 1.5.3 to 1.5.4 in /hadoop-project
steveloughran commented on PR #5502: URL: https://github.com/apache/hadoop/pull/5502#issuecomment-1505233783 you think those test failures are related? (unlikely, i know...) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] lgh-cn opened a new pull request, #5545: add rpc metrics for response time
lgh-cn opened a new pull request, #5545: URL: https://github.com/apache/hadoop/pull/5545 ### Description of PR Rpc ResponseTime record the time to encode and send response. This should be added in the rpc metrics. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #4687: YARN-10846. Add dispatcher metrics to NM.
hadoop-yetus commented on PR #4687:
URL: https://github.com/apache/hadoop/pull/4687#issuecomment-1504918972
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 45s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 1s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +0 :ok: | xmllint | 0m 0s | | xmllint was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 5 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 16m 23s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 25m 57s | | trunk passed |
| +1 :green_heart: | compile | 9m 41s | | trunk passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | compile | 8m 30s | | trunk passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | checkstyle | 1m 48s | | trunk passed |
| +1 :green_heart: | mvnsite | 4m 54s | | trunk passed |
| +1 :green_heart: | javadoc | 4m 24s | | trunk passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javadoc | 4m 10s | | trunk passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +0 :ok: | spotbugs | 0m 42s | |
branch/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests
no spotbugs output file (spotbugsXml.xml) |
| +1 :green_heart: | shadedclient | 20m 23s | | branch has no errors
when building and testing our client artifacts. |
| -0 :warning: | patch | 20m 45s | | Used diff version of patch file.
Binary files and potentially other changes not applied. Please rebase and
squash commits if necessary. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 28s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 3m 9s | | the patch passed |
| +1 :green_heart: | compile | 9m 5s | | the patch passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javac | 9m 5s | | the patch passed |
| +1 :green_heart: | compile | 8m 30s | | the patch passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | javac | 8m 30s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| -0 :warning: | checkstyle | 1m 38s |
[/results-checkstyle-hadoop-yarn-project_hadoop-yarn.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4687/21/artifact/out/results-checkstyle-hadoop-yarn-project_hadoop-yarn.txt)
| hadoop-yarn-project/hadoop-yarn: The patch generated 1 new + 253 unchanged
- 1 fixed = 254 total (was 254) |
| +1 :green_heart: | mvnsite | 4m 34s | | the patch passed |
| +1 :green_heart: | javadoc | 3m 55s | | the patch passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javadoc | 3m 49s | | the patch passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +0 :ok: | spotbugs | 0m 38s | |
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests has
no data from spotbugs |
| +1 :green_heart: | shadedclient | 20m 28s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 1m 14s | | hadoop-yarn-api in the patch
passed. |
| +1 :green_heart: | unit | 5m 41s | | hadoop-yarn-common in the patch
passed. |
| +1 :green_heart: | unit | 102m 10s | |
hadoop-yarn-server-resourcemanager in the patch passed. |
| +1 :green_heart: | unit | 25m 32s | | hadoop-yarn-server-nodemanager
in the patch passed. |
| +1 :green_heart: | unit | 3m 32s | | hadoop-yarn-server-tests in the
patch passed. |
| +1 :green_heart: | asflicense | 0m 56s | | The patch does not
generate ASF License warnings. |
| | | 314m 13s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.42 ServerAPI=1.42 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4687/21/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/4687 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets xmllint |
| uname | Linux 3e4fe6cd6f73
[GitHub] [hadoop] hadoop-yetus commented on pull request #5544: hdfs auditlog add remote port info
hadoop-yetus commented on PR #5544:
URL: https://github.com/apache/hadoop/pull/5544#issuecomment-1504819624
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 37s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 3 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 39m 17s | | trunk passed |
| +1 :green_heart: | compile | 1m 27s | | trunk passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | compile | 1m 20s | | trunk passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | checkstyle | 1m 8s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 32s | | trunk passed |
| +1 :green_heart: | javadoc | 1m 8s | | trunk passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javadoc | 1m 33s | | trunk passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | spotbugs | 3m 30s | | trunk passed |
| +1 :green_heart: | shadedclient | 22m 48s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 1m 16s | | the patch passed |
| +1 :green_heart: | compile | 1m 17s | | the patch passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javac | 1m 17s | | the patch passed |
| +1 :green_heart: | compile | 1m 14s | | the patch passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | javac | 1m 14s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| -0 :warning: | checkstyle | 0m 52s |
[/results-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5544/1/artifact/out/results-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt)
| hadoop-hdfs-project/hadoop-hdfs: The patch generated 7 new + 143 unchanged
- 0 fixed = 150 total (was 143) |
| +1 :green_heart: | mvnsite | 1m 21s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 51s | | the patch passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javadoc | 1m 30s | | the patch passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | spotbugs | 3m 14s | | the patch passed |
| +1 :green_heart: | shadedclient | 22m 19s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| -1 :x: | unit | 206m 49s |
[/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5544/1/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt)
| hadoop-hdfs in the patch passed. |
| +1 :green_heart: | asflicense | 0m 51s | | The patch does not
generate ASF License warnings. |
| | | 314m 17s | | |
| Reason | Tests |
|---:|:--|
| Failed junit tests | hadoop.hdfs.server.namenode.TestFsck |
| | hadoop.hdfs.server.datanode.TestDirectoryScanner |
| | hadoop.hdfs.server.namenode.TestAuditLogger |
| | hadoop.hdfs.server.namenode.TestAuditLogs |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.42 ServerAPI=1.42 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5544/1/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/5544 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux c9a7af6faf4e 4.15.0-206-generic #217-Ubuntu SMP Fri Feb 3
19:10:13 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 1c6403afe93d99f4935375fe5fb32f8767fb9541 |
| Default Java | Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| Test Results |
[jira] [Commented] (HADOOP-18694) Client.Connection#updateAddress needs to ensure that address is resolved before updating
[
https://issues.apache.org/jira/browse/HADOOP-18694?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17711225#comment-17711225
]
ASF GitHub Bot commented on HADOOP-18694:
-
hadoop-yetus commented on PR #5542:
URL: https://github.com/apache/hadoop/pull/5542#issuecomment-1504769966
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 48s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| -1 :x: | test4tests | 0m 0s | | The patch doesn't appear to include
any new or modified tests. Please justify why no new tests are needed for this
patch. Also please list what manual steps were performed to verify this patch.
|
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 43m 50s | | trunk passed |
| +1 :green_heart: | compile | 25m 5s | | trunk passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | compile | 21m 42s | | trunk passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | checkstyle | 1m 5s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 39s | | trunk passed |
| +1 :green_heart: | javadoc | 1m 9s | | trunk passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javadoc | 0m 42s | | trunk passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | spotbugs | 2m 43s | | trunk passed |
| +1 :green_heart: | shadedclient | 25m 37s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 1m 0s | | the patch passed |
| +1 :green_heart: | compile | 24m 37s | | the patch passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javac | 24m 37s | | the patch passed |
| +1 :green_heart: | compile | 21m 43s | | the patch passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | javac | 21m 43s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 0m 59s | | the patch passed |
| +1 :green_heart: | mvnsite | 1m 35s | | the patch passed |
| +1 :green_heart: | javadoc | 1m 1s | | the patch passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javadoc | 0m 41s | | the patch passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | spotbugs | 2m 40s | | the patch passed |
| +1 :green_heart: | shadedclient | 25m 14s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 18m 9s | | hadoop-common in the patch
passed. |
| +1 :green_heart: | asflicense | 0m 53s | | The patch does not
generate ASF License warnings. |
| | | 223m 6s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.42 ServerAPI=1.42 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5542/2/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/5542 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux b1511504b561 4.15.0-206-generic #217-Ubuntu SMP Fri Feb 3
19:10:13 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 596bb839cdbdf52a34f324c8313e5777aa604894 |
| Default Java | Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5542/2/testReport/ |
| Max. process+thread count | 1375 (vs. ulimit of 5500) |
| modules | C: hadoop-common-project/hadoop-common U:
hadoop-common-project/hadoop-common |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5542/2/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered
[jira] [Updated] (HADOOP-18694) Client.Connection#updateAddress needs to ensure that address is resolved before updating
[
https://issues.apache.org/jira/browse/HADOOP-18694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated HADOOP-18694:
Labels: pull-request-available (was: )
> Client.Connection#updateAddress needs to ensure that address is resolved
> before updating
>
>
> Key: HADOOP-18694
> URL: https://issues.apache.org/jira/browse/HADOOP-18694
> Project: Hadoop Common
> Issue Type: Improvement
> Components: common
>Affects Versions: 3.4.0, 3.3.5
>Reporter: dzcxzl
>Priority: Major
> Labels: pull-request-available
>
> When Client.Connection#setupConnection encounters an IOException, it will try
> to update the server address.
> ([HADOOP-18365|https://issues.apache.org/jira/browse/HADOOP-18365])
> When the address is re-parsed, it may be an unresolved address
> (UnknownHostException), which causes Client.Connection#setupConnection to
> fail to reconnect.
> {code:java}
> while (true) {
> try {
> if (server.isUnresolved()) { {code}
> Especially when DN is connected to NN, BPServiceActor#bpNamenode is only
> initialized once, which causes DN to never connect to NN before restarting.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #5542: HADOOP-18694. Client.Connection#updateAddress needs to ensure that address is resolved before updating
hadoop-yetus commented on PR #5542:
URL: https://github.com/apache/hadoop/pull/5542#issuecomment-1504769966
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 48s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| -1 :x: | test4tests | 0m 0s | | The patch doesn't appear to include
any new or modified tests. Please justify why no new tests are needed for this
patch. Also please list what manual steps were performed to verify this patch.
|
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 43m 50s | | trunk passed |
| +1 :green_heart: | compile | 25m 5s | | trunk passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | compile | 21m 42s | | trunk passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | checkstyle | 1m 5s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 39s | | trunk passed |
| +1 :green_heart: | javadoc | 1m 9s | | trunk passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javadoc | 0m 42s | | trunk passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | spotbugs | 2m 43s | | trunk passed |
| +1 :green_heart: | shadedclient | 25m 37s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 1m 0s | | the patch passed |
| +1 :green_heart: | compile | 24m 37s | | the patch passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javac | 24m 37s | | the patch passed |
| +1 :green_heart: | compile | 21m 43s | | the patch passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | javac | 21m 43s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 0m 59s | | the patch passed |
| +1 :green_heart: | mvnsite | 1m 35s | | the patch passed |
| +1 :green_heart: | javadoc | 1m 1s | | the patch passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javadoc | 0m 41s | | the patch passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | spotbugs | 2m 40s | | the patch passed |
| +1 :green_heart: | shadedclient | 25m 14s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 18m 9s | | hadoop-common in the patch
passed. |
| +1 :green_heart: | asflicense | 0m 53s | | The patch does not
generate ASF License warnings. |
| | | 223m 6s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.42 ServerAPI=1.42 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5542/2/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/5542 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux b1511504b561 4.15.0-206-generic #217-Ubuntu SMP Fri Feb 3
19:10:13 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 596bb839cdbdf52a34f324c8313e5777aa604894 |
| Default Java | Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5542/2/testReport/ |
| Max. process+thread count | 1375 (vs. ulimit of 5500) |
| modules | C: hadoop-common-project/hadoop-common U:
hadoop-common-project/hadoop-common |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5542/2/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL
