[GitHub] [hadoop] anmolanmol1234 commented on pull request #6025: Pass down eTag as part of accessCondition to SDK
anmolanmol1234 commented on PR #6025: URL: https://github.com/apache/hadoop/pull/6025#issuecomment-1718804733 > Can we also handle deletes as well? They exhibit the same problem as uploads. Hi Alkis, can you please confirm is this issue is still been seen ? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18883) Expect-100 JDK bug resolution: prevent multiple server calls
[ https://issues.apache.org/jira/browse/HADOOP-18883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17764986#comment-17764986 ] Pranav Saxena commented on HADOOP-18883: Hi [~ste...@apache.org], This Jira is not related to HADOOP-18865. This Jira is a resolution on ABFS side for the JDK bug. This JDK bug has always been there. Since, it is discovered now, we want to have a resolution on our side. Thank you. > Expect-100 JDK bug resolution: prevent multiple server calls > > > Key: HADOOP-18883 > URL: https://issues.apache.org/jira/browse/HADOOP-18883 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/azure >Reporter: Pranav Saxena >Assignee: Pranav Saxena >Priority: Major > Fix For: 3.4.0 > > > This is inline to JDK bug: [https://bugs.openjdk.org/browse/JDK-8314978]. > > With the current implementation of HttpURLConnection if server rejects the > “Expect 100-continue” then there will be ‘java.net.ProtocolException’ will be > thrown from 'expect100Continue()' method. > After the exception thrown, If we call any other method on the same instance > (ex getHeaderField(), or getHeaderFields()). They will internally call > getOuputStream() which invokes writeRequests(), which make the actual server > call. > In the AbfsHttpOperation, after sendRequest() we call processResponse() > method from AbfsRestOperation. Even if the conn.getOutputStream() fails due > to expect-100 error, we consume the exception and let the code go ahead. So, > we can have getHeaderField() / getHeaderFields() / getHeaderFieldLong() which > will be triggered after getOutputStream is failed. These invocation will lead > to server calls. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18890) remove okhttp usage
[ https://issues.apache.org/jira/browse/HADOOP-18890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17764982#comment-17764982 ] Cheng Pan commented on HADOOP-18890: Related to HADOOP-18503, stuck because I don't have an OBS account. > remove okhttp usage > --- > > Key: HADOOP-18890 > URL: https://issues.apache.org/jira/browse/HADOOP-18890 > Project: Hadoop Common > Issue Type: Bug > Components: common >Reporter: PJ Fanning >Priority: Major > > * relates to HADOOP-18496 > * simplifies the dependencies if hadoop doesn't use multiple 3rd party libs > to make http calls > * okhttp brings in other dependencies like the kotlin runtime > * hadoop already uses apache httpclient in some places -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18896) NegativeArraySizeException thrown in FSOutputSummer.java given large file.bytes-per-checksum
[
https://issues.apache.org/jira/browse/HADOOP-18896?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17764972#comment-17764972
]
ASF GitHub Bot commented on HADOOP-18896:
-
hadoop-yetus commented on PR #6064:
URL: https://github.com/apache/hadoop/pull/6064#issuecomment-1718760771
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 46s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| -1 :x: | test4tests | 0m 0s | | The patch doesn't appear to include
any new or modified tests. Please justify why no new tests are needed for this
patch. Also please list what manual steps were performed to verify this patch.
|
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 49m 44s | | trunk passed |
| +1 :green_heart: | compile | 18m 16s | | trunk passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | compile | 17m 12s | | trunk passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | checkstyle | 1m 11s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 37s | | trunk passed |
| +1 :green_heart: | javadoc | 1m 10s | | trunk passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 0m 45s | | trunk passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | spotbugs | 2m 34s | | trunk passed |
| +1 :green_heart: | shadedclient | 41m 34s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 57s | | the patch passed |
| +1 :green_heart: | compile | 18m 29s | | the patch passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javac | 18m 29s | | the patch passed |
| +1 :green_heart: | compile | 17m 25s | | the patch passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | javac | 17m 25s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 1m 9s | | the patch passed |
| +1 :green_heart: | mvnsite | 1m 33s | | the patch passed |
| +1 :green_heart: | javadoc | 1m 4s | | the patch passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 0m 45s | | the patch passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | spotbugs | 2m 43s | | the patch passed |
| +1 :green_heart: | shadedclient | 41m 39s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 20m 28s | | hadoop-common in the patch
passed. |
| +1 :green_heart: | asflicense | 0m 54s | | The patch does not
generate ASF License warnings. |
| | | 244m 53s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6064/1/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/6064 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux c128f29a75ca 4.15.0-212-generic #223-Ubuntu SMP Tue May 23
13:09:22 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / c8b6207eb7e02ae56e62d8d2009cf0bf5352cca6 |
| Default Java | Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6064/1/testReport/ |
| Max. process+thread count | 1261 (vs. ulimit of 5500) |
| modules | C: hadoop-common-project/hadoop-common U:
hadoop-common-project/hadoop-common |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6064/1/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0
[GitHub] [hadoop] hadoop-yetus commented on pull request #6064: HADOOP-18896. NegativeArraySizeException thrown in FSOutputSummer.java given large file.bytes-per-checksum
hadoop-yetus commented on PR #6064:
URL: https://github.com/apache/hadoop/pull/6064#issuecomment-1718760771
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 46s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| -1 :x: | test4tests | 0m 0s | | The patch doesn't appear to include
any new or modified tests. Please justify why no new tests are needed for this
patch. Also please list what manual steps were performed to verify this patch.
|
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 49m 44s | | trunk passed |
| +1 :green_heart: | compile | 18m 16s | | trunk passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | compile | 17m 12s | | trunk passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | checkstyle | 1m 11s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 37s | | trunk passed |
| +1 :green_heart: | javadoc | 1m 10s | | trunk passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 0m 45s | | trunk passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | spotbugs | 2m 34s | | trunk passed |
| +1 :green_heart: | shadedclient | 41m 34s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 57s | | the patch passed |
| +1 :green_heart: | compile | 18m 29s | | the patch passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javac | 18m 29s | | the patch passed |
| +1 :green_heart: | compile | 17m 25s | | the patch passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | javac | 17m 25s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 1m 9s | | the patch passed |
| +1 :green_heart: | mvnsite | 1m 33s | | the patch passed |
| +1 :green_heart: | javadoc | 1m 4s | | the patch passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 0m 45s | | the patch passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | spotbugs | 2m 43s | | the patch passed |
| +1 :green_heart: | shadedclient | 41m 39s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 20m 28s | | hadoop-common in the patch
passed. |
| +1 :green_heart: | asflicense | 0m 54s | | The patch does not
generate ASF License warnings. |
| | | 244m 53s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6064/1/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/6064 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux c128f29a75ca 4.15.0-212-generic #223-Ubuntu SMP Tue May 23
13:09:22 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / c8b6207eb7e02ae56e62d8d2009cf0bf5352cca6 |
| Default Java | Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6064/1/testReport/ |
| Max. process+thread count | 1261 (vs. ulimit of 5500) |
| modules | C: hadoop-common-project/hadoop-common U:
hadoop-common-project/hadoop-common |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6064/1/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To
[jira] [Commented] (HADOOP-18821) LdapGroupsMapping crashes with NullPointerException while going up the group hierarchy
[
https://issues.apache.org/jira/browse/HADOOP-18821?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17764967#comment-17764967
]
ASF GitHub Bot commented on HADOOP-18821:
-
hadoop-yetus commented on PR #6066:
URL: https://github.com/apache/hadoop/pull/6066#issuecomment-1718760217
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 45s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| -1 :x: | test4tests | 0m 0s | | The patch doesn't appear to include
any new or modified tests. Please justify why no new tests are needed for this
patch. Also please list what manual steps were performed to verify this patch.
|
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 44m 58s | | trunk passed |
| +1 :green_heart: | compile | 17m 18s | | trunk passed with JDK
Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04 |
| +1 :green_heart: | compile | 16m 6s | | trunk passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | checkstyle | 1m 24s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 47s | | trunk passed |
| +1 :green_heart: | javadoc | 1m 24s | | trunk passed with JDK
Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04 |
| +1 :green_heart: | javadoc | 0m 59s | | trunk passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | spotbugs | 2m 42s | | trunk passed |
| +1 :green_heart: | shadedclient | 35m 43s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 56s | | the patch passed |
| +1 :green_heart: | compile | 16m 32s | | the patch passed with JDK
Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04 |
| +1 :green_heart: | javac | 16m 33s | | the patch passed |
| +1 :green_heart: | compile | 16m 10s | | the patch passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | javac | 16m 10s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 1m 20s | | the patch passed |
| +1 :green_heart: | mvnsite | 1m 45s | | the patch passed |
| +1 :green_heart: | javadoc | 1m 18s | | the patch passed with JDK
Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04 |
| +1 :green_heart: | javadoc | 0m 59s | | the patch passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | spotbugs | 2m 49s | | the patch passed |
| +1 :green_heart: | shadedclient | 35m 46s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 19m 14s | | hadoop-common in the patch
passed. |
| +1 :green_heart: | asflicense | 1m 11s | | The patch does not
generate ASF License warnings. |
| | | 226m 8s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6066/1/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/6066 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux c220fdf3fb47 4.15.0-212-generic #223-Ubuntu SMP Tue May 23
13:09:22 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / c8b6207eb7e02ae56e62d8d2009cf0bf5352cca6 |
| Default Java | Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6066/1/testReport/ |
| Max. process+thread count | 2650 (vs. ulimit of 5500) |
| modules | C: hadoop-common-project/hadoop-common U:
hadoop-common-project/hadoop-common |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6066/1/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
[GitHub] [hadoop] hadoop-yetus commented on pull request #6066: HADOOP-18821. LdapGroupsMapping crashes with NullPointerException while going up the group hierarchy
hadoop-yetus commented on PR #6066:
URL: https://github.com/apache/hadoop/pull/6066#issuecomment-1718760217
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 45s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| -1 :x: | test4tests | 0m 0s | | The patch doesn't appear to include
any new or modified tests. Please justify why no new tests are needed for this
patch. Also please list what manual steps were performed to verify this patch.
|
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 44m 58s | | trunk passed |
| +1 :green_heart: | compile | 17m 18s | | trunk passed with JDK
Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04 |
| +1 :green_heart: | compile | 16m 6s | | trunk passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | checkstyle | 1m 24s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 47s | | trunk passed |
| +1 :green_heart: | javadoc | 1m 24s | | trunk passed with JDK
Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04 |
| +1 :green_heart: | javadoc | 0m 59s | | trunk passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | spotbugs | 2m 42s | | trunk passed |
| +1 :green_heart: | shadedclient | 35m 43s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 56s | | the patch passed |
| +1 :green_heart: | compile | 16m 32s | | the patch passed with JDK
Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04 |
| +1 :green_heart: | javac | 16m 33s | | the patch passed |
| +1 :green_heart: | compile | 16m 10s | | the patch passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | javac | 16m 10s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 1m 20s | | the patch passed |
| +1 :green_heart: | mvnsite | 1m 45s | | the patch passed |
| +1 :green_heart: | javadoc | 1m 18s | | the patch passed with JDK
Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04 |
| +1 :green_heart: | javadoc | 0m 59s | | the patch passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | spotbugs | 2m 49s | | the patch passed |
| +1 :green_heart: | shadedclient | 35m 46s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 19m 14s | | hadoop-common in the patch
passed. |
| +1 :green_heart: | asflicense | 1m 11s | | The patch does not
generate ASF License warnings. |
| | | 226m 8s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6066/1/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/6066 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux c220fdf3fb47 4.15.0-212-generic #223-Ubuntu SMP Tue May 23
13:09:22 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / c8b6207eb7e02ae56e62d8d2009cf0bf5352cca6 |
| Default Java | Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6066/1/testReport/ |
| Max. process+thread count | 2650 (vs. ulimit of 5500) |
| modules | C: hadoop-common-project/hadoop-common U:
hadoop-common-project/hadoop-common |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6066/1/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe,
[GitHub] [hadoop] hadoop-yetus commented on pull request #6065: HDFS-17189. GcTimeMonitor crashes with NegativeArraySizeException during initialization
hadoop-yetus commented on PR #6065:
URL: https://github.com/apache/hadoop/pull/6065#issuecomment-1718708169
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 31s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| -1 :x: | test4tests | 0m 0s | | The patch doesn't appear to include
any new or modified tests. Please justify why no new tests are needed for this
patch. Also please list what manual steps were performed to verify this patch.
|
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 33m 36s | | trunk passed |
| +1 :green_heart: | compile | 10m 30s | | trunk passed with JDK
Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04 |
| +1 :green_heart: | compile | 9m 36s | | trunk passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | checkstyle | 0m 53s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 10s | | trunk passed |
| +1 :green_heart: | javadoc | 1m 0s | | trunk passed with JDK
Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04 |
| +1 :green_heart: | javadoc | 0m 42s | | trunk passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | spotbugs | 1m 44s | | trunk passed |
| +1 :green_heart: | shadedclient | 22m 16s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 39s | | the patch passed |
| +1 :green_heart: | compile | 9m 54s | | the patch passed with JDK
Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04 |
| +1 :green_heart: | javac | 9m 54s | | the patch passed |
| +1 :green_heart: | compile | 9m 34s | | the patch passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | javac | 9m 34s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 0m 50s | | the patch passed |
| +1 :green_heart: | mvnsite | 1m 10s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 55s | | the patch passed with JDK
Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04 |
| +1 :green_heart: | javadoc | 0m 42s | | the patch passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | spotbugs | 1m 46s | | the patch passed |
| +1 :green_heart: | shadedclient | 22m 25s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 16m 34s | | hadoop-common in the patch
passed. |
| +1 :green_heart: | asflicense | 0m 50s | | The patch does not
generate ASF License warnings. |
| | | 150m 24s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6065/1/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/6065 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux dc962f78411f 4.15.0-213-generic #224-Ubuntu SMP Mon Jun 19
13:30:12 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / c8b6207eb7e02ae56e62d8d2009cf0bf5352cca6 |
| Default Java | Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6065/1/testReport/ |
| Max. process+thread count | 2160 (vs. ulimit of 5500) |
| modules | C: hadoop-common-project/hadoop-common U:
hadoop-common-project/hadoop-common |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6065/1/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe,
[GitHub] [hadoop] AuthurWang2009 closed pull request #6059: HADOOP-18892 fix Catalog Webapp module compile errors due to low nodejs version
AuthurWang2009 closed pull request #6059: HADOOP-18892 fix Catalog Webapp module compile errors due to low nodejs version URL: https://github.com/apache/hadoop/pull/6059 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] AuthurWang2009 commented on pull request #6059: HADOOP-18892 fix Catalog Webapp module compile errors due to low nodejs version
AuthurWang2009 commented on PR #6059: URL: https://github.com/apache/hadoop/pull/6059#issuecomment-1718634751 @slfan1989 Thank you for your answer. Indeed, the [YARN-11528](https://issues.apache.org/jira/browse/YARN-11528) ,PR: (https://github.com/apache/hadoop/pull/5827) seems to be a better solution. I will close it later. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] slfan1989 commented on a diff in pull request #6055: YARN-11547. [Federation] Router Supports Remove individual application records from FederationStateStore.
slfan1989 commented on code in PR #6055:
URL: https://github.com/apache/hadoop/pull/6055#discussion_r1325251620
##
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router/src/main/java/org/apache/hadoop/yarn/server/router/Router.java:
##
@@ -291,22 +296,30 @@ public static String getProxyHostAndPort(Configuration
conf) {
public static void main(String[] argv) {
Configuration conf = new YarnConfiguration();
Thread
-.setDefaultUncaughtExceptionHandler(new
YarnUncaughtExceptionHandler());
+.setDefaultUncaughtExceptionHandler(new
YarnUncaughtExceptionHandler());
StringUtils.startupShutdownMessage(Router.class, argv, LOG);
Router router = new Router();
try {
-
- // Remove the old hook if we are rebooting.
- if (null != routerShutdownHook) {
-ShutdownHookManager.get().removeShutdownHook(routerShutdownHook);
+ GenericOptionsParser hParser = new GenericOptionsParser(conf, argv);
+ argv = hParser.getRemainingArgs();
+ if (argv.length > 1) {
Review Comment:
Thanks for your suggestion! I will improve this part of the code.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] goiri commented on a diff in pull request #6055: YARN-11547. [Federation] Router Supports Remove individual application records from FederationStateStore.
goiri commented on code in PR #6055:
URL: https://github.com/apache/hadoop/pull/6055#discussion_r1325219102
##
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/federation/utils/FederationStateStoreFacade.java:
##
@@ -1088,4 +1089,21 @@ public ApplicationSubmissionContext
getApplicationSubmissionContext(ApplicationI
public FederationCache getFederationCache() {
return federationCache;
}
+
+ /**
+ * Delete the mapping of home {@code SubClusterId} of a previously submitted
+ * {@code ApplicationId}. Currently response is empty if the operation was
+ * successful, if not an exception reporting reason for a failure.
+ *
+ * @param appId ApplicationId.
+ */
+ public void deleteApplicationHomeSubCluster(ApplicationId appId) {
+try {
+ DeleteApplicationHomeSubClusterRequest request =
+ DeleteApplicationHomeSubClusterRequest.newInstance(appId);
+ stateStore.deleteApplicationHomeSubCluster(request);
+} catch (Exception e) {
+ LOG.error("deleteApplicationHomeSubCluster error, applicationId = {}.",
appId, e);
Review Comment:
Shouldn't we surface this somehow?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18821) LdapGroupsMapping crashes with NullPointerException while going up the group hierarchy
[
https://issues.apache.org/jira/browse/HADOOP-18821?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17764927#comment-17764927
]
ASF GitHub Bot commented on HADOOP-18821:
-
teamconfx opened a new pull request, #6066:
URL: https://github.com/apache/hadoop/pull/6066
### Description of PR
https://issues.apache.org/jira/browse/HADOOP-18821
This PR provides a fix by checking the groupResults is not Null before it is
accessed, similar to what's done in
`org.apache.hadoop.security.LdapGroupsMapping#lookupGroup`
### How was this patch tested?
Unit test
### For code changes:
- [x] Does the title or this PR starts with the corresponding JIRA issue id
(e.g. 'HADOOP-17799. Your PR title ...')?
- [ ] Object storage: have the integration tests been executed and the
endpoint declared according to the connector-specific documentation?
- [ ] If adding new dependencies to the code, are these dependencies
licensed in a way that is compatible for inclusion under [ASF
2.0](http://www.apache.org/legal/resolved.html#category-a)?
- [ ] If applicable, have you updated the `LICENSE`, `LICENSE-binary`,
`NOTICE-binary` files?
> LdapGroupsMapping crashes with NullPointerException while going up the group
> hierarchy
> ---
>
> Key: HADOOP-18821
> URL: https://issues.apache.org/jira/browse/HADOOP-18821
> Project: Hadoop Common
> Issue Type: Bug
>Affects Versions: 3.3.6
>Reporter: ConfX
>Priority: Critical
> Attachments: reproduce.sh
>
>
> h2. What happened:
> When set {{hadoop.security.group.mapping.ldap.search.group.hierarchy.levels}}
> to a value larger than 0, {{goUpGroupHierarchy}} in
> {{org/apache/hadoop/security/LdapGroupsMapping.java}} may return a null
> {{{}groupResults{}}}and use it without checking null.
> h2. Buggy code:
> {noformat}
> void goUpGroupHierarchy(Set groupDNs, int goUpHierarchy,
> Set groups) throws NamingException {
> if (goUpHierarchy <= 0 || groups.isEmpty()) {
> return;
> }
> ...
> NamingEnumeration groupResults =
> context.search(groupbaseDN, filter.toString(), SEARCH_CONTROLS);
> while (groupResults.hasMoreElements()) { // <--- Here
> groupResults may be null
> ...
> }
> ...
> }{noformat}
> h2. How to reproduce:
> (1) Set {{hadoop.security.group.mapping.ldap.search.group.hierarchy.levels}}
> to 1
> (2) Run test
> {{org.apache.hadoop.security.TestLdapGroupsMapping#testGetGroupsWithConnectionClosed}}
> h2. Stack trace:
> {noformat}
> java.lang.NullPointerException
> at
> org.apache.hadoop.security.LdapGroupsMapping.goUpGroupHierarchy(LdapGroupsMapping.java:612)
> at
> org.apache.hadoop.security.LdapGroupsMapping.lookupGroup(LdapGroupsMapping.java:489)
> at
> org.apache.hadoop.security.LdapGroupsMapping.doGetGroups(LdapGroupsMapping.java:552)
> at
> org.apache.hadoop.security.LdapGroupsMapping.getGroups(LdapGroupsMapping.java:365){noformat}
> For an easy reproduction, run the reproduce.sh in the attachment.
> We also create a PR provides a fix by checking the groupResults is not Null
> before it is accessed, similar to what's done in
> `org.apache.hadoop.security.LdapGroupsMapping#lookupGroup`
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-18821) LdapGroupsMapping crashes with NullPointerException while going up the group hierarchy
[
https://issues.apache.org/jira/browse/HADOOP-18821?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated HADOOP-18821:
Labels: pull-request-available (was: )
> LdapGroupsMapping crashes with NullPointerException while going up the group
> hierarchy
> ---
>
> Key: HADOOP-18821
> URL: https://issues.apache.org/jira/browse/HADOOP-18821
> Project: Hadoop Common
> Issue Type: Bug
>Affects Versions: 3.3.6
>Reporter: ConfX
>Priority: Critical
> Labels: pull-request-available
> Attachments: reproduce.sh
>
>
> h2. What happened:
> When set {{hadoop.security.group.mapping.ldap.search.group.hierarchy.levels}}
> to a value larger than 0, {{goUpGroupHierarchy}} in
> {{org/apache/hadoop/security/LdapGroupsMapping.java}} may return a null
> {{{}groupResults{}}}and use it without checking null.
> h2. Buggy code:
> {noformat}
> void goUpGroupHierarchy(Set groupDNs, int goUpHierarchy,
> Set groups) throws NamingException {
> if (goUpHierarchy <= 0 || groups.isEmpty()) {
> return;
> }
> ...
> NamingEnumeration groupResults =
> context.search(groupbaseDN, filter.toString(), SEARCH_CONTROLS);
> while (groupResults.hasMoreElements()) { // <--- Here
> groupResults may be null
> ...
> }
> ...
> }{noformat}
> h2. How to reproduce:
> (1) Set {{hadoop.security.group.mapping.ldap.search.group.hierarchy.levels}}
> to 1
> (2) Run test
> {{org.apache.hadoop.security.TestLdapGroupsMapping#testGetGroupsWithConnectionClosed}}
> h2. Stack trace:
> {noformat}
> java.lang.NullPointerException
> at
> org.apache.hadoop.security.LdapGroupsMapping.goUpGroupHierarchy(LdapGroupsMapping.java:612)
> at
> org.apache.hadoop.security.LdapGroupsMapping.lookupGroup(LdapGroupsMapping.java:489)
> at
> org.apache.hadoop.security.LdapGroupsMapping.doGetGroups(LdapGroupsMapping.java:552)
> at
> org.apache.hadoop.security.LdapGroupsMapping.getGroups(LdapGroupsMapping.java:365){noformat}
> For an easy reproduction, run the reproduce.sh in the attachment.
> We also create a PR provides a fix by checking the groupResults is not Null
> before it is accessed, similar to what's done in
> `org.apache.hadoop.security.LdapGroupsMapping#lookupGroup`
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] teamconfx opened a new pull request, #6066: HADOOP-18821: LdapGroupsMapping crashes with NullPointerException while going up the group hierarchy
teamconfx opened a new pull request, #6066: URL: https://github.com/apache/hadoop/pull/6066 ### Description of PR https://issues.apache.org/jira/browse/HADOOP-18821 This PR provides a fix by checking the groupResults is not Null before it is accessed, similar to what's done in `org.apache.hadoop.security.LdapGroupsMapping#lookupGroup` ### How was this patch tested? Unit test ### For code changes: - [x] Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')? - [ ] Object storage: have the integration tests been executed and the endpoint declared according to the connector-specific documentation? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the `LICENSE`, `LICENSE-binary`, `NOTICE-binary` files? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-18821) LdapGroupsMapping crashes with NullPointerException while going up the group hierarchy
[
https://issues.apache.org/jira/browse/HADOOP-18821?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ConfX updated HADOOP-18821:
---
Description:
h2. What happened:
When set {{hadoop.security.group.mapping.ldap.search.group.hierarchy.levels}}
to a value larger than 0, {{goUpGroupHierarchy}} in
{{org/apache/hadoop/security/LdapGroupsMapping.java}} may return a null
{{{}groupResults{}}}and use it without checking null.
h2. Buggy code:
{noformat}
void goUpGroupHierarchy(Set groupDNs, int goUpHierarchy, Set
groups) throws NamingException {
if (goUpHierarchy <= 0 || groups.isEmpty()) {
return;
}
...
NamingEnumeration groupResults = context.search(groupbaseDN,
filter.toString(), SEARCH_CONTROLS);
while (groupResults.hasMoreElements()) { // <--- Here groupResults
may be null
...
}
...
}{noformat}
h2. How to reproduce:
(1) Set {{hadoop.security.group.mapping.ldap.search.group.hierarchy.levels}} to
1
(2) Run test
{{org.apache.hadoop.security.TestLdapGroupsMapping#testGetGroupsWithConnectionClosed}}
h2. Stack trace:
{noformat}
java.lang.NullPointerException
at
org.apache.hadoop.security.LdapGroupsMapping.goUpGroupHierarchy(LdapGroupsMapping.java:612)
at
org.apache.hadoop.security.LdapGroupsMapping.lookupGroup(LdapGroupsMapping.java:489)
at
org.apache.hadoop.security.LdapGroupsMapping.doGetGroups(LdapGroupsMapping.java:552)
at
org.apache.hadoop.security.LdapGroupsMapping.getGroups(LdapGroupsMapping.java:365){noformat}
For an easy reproduction, run the reproduce.sh in the attachment.
We also create a PR provides a fix by checking the groupResults is not Null
before it is accessed, similar to what's done in
`org.apache.hadoop.security.LdapGroupsMapping#lookupGroup`
was:
h2. What happened:
When set {{hadoop.security.group.mapping.ldap.search.group.hierarchy.levels}}
to a value larger than 0, {{goUpGroupHierarchy}} in
{{org/apache/hadoop/security/LdapGroupsMapping.java}} may return a null
{{{}groupResults{}}}and use it without checking null.
h2. Buggy code:
{noformat}
void goUpGroupHierarchy(Set groupDNs, int goUpHierarchy, Set
groups) throws NamingException {
if (goUpHierarchy <= 0 || groups.isEmpty()) {
return;
}
...
NamingEnumeration groupResults = context.search(groupbaseDN,
filter.toString(), SEARCH_CONTROLS);
while (groupResults.hasMoreElements()) { // <--- Here groupResults
may be null
...
}
...
}{noformat}
h2. How to reproduce:
(1) Set {{hadoop.security.group.mapping.ldap.search.group.hierarchy.levels}} to
1
(2) Run test
{{org.apache.hadoop.security.TestLdapGroupsMapping#testGetGroupsWithConnectionClosed}}
h2. Stack trace:
{noformat}
java.lang.NullPointerException
at
org.apache.hadoop.security.LdapGroupsMapping.goUpGroupHierarchy(LdapGroupsMapping.java:612)
at
org.apache.hadoop.security.LdapGroupsMapping.lookupGroup(LdapGroupsMapping.java:489)
at
org.apache.hadoop.security.LdapGroupsMapping.doGetGroups(LdapGroupsMapping.java:552)
at
org.apache.hadoop.security.LdapGroupsMapping.getGroups(LdapGroupsMapping.java:365){noformat}
For an easy reproduction, run the reproduce.sh in the attachment.
We are happy to provide a patch if this issue is confirmed.
> LdapGroupsMapping crashes with NullPointerException while going up the group
> hierarchy
> ---
>
> Key: HADOOP-18821
> URL: https://issues.apache.org/jira/browse/HADOOP-18821
> Project: Hadoop Common
> Issue Type: Bug
>Affects Versions: 3.3.6
>Reporter: ConfX
>Priority: Critical
> Attachments: reproduce.sh
>
>
> h2. What happened:
> When set {{hadoop.security.group.mapping.ldap.search.group.hierarchy.levels}}
> to a value larger than 0, {{goUpGroupHierarchy}} in
> {{org/apache/hadoop/security/LdapGroupsMapping.java}} may return a null
> {{{}groupResults{}}}and use it without checking null.
> h2. Buggy code:
> {noformat}
> void goUpGroupHierarchy(Set groupDNs, int goUpHierarchy,
> Set groups) throws NamingException {
> if (goUpHierarchy <= 0 || groups.isEmpty()) {
> return;
> }
> ...
> NamingEnumeration groupResults =
> context.search(groupbaseDN, filter.toString(), SEARCH_CONTROLS);
> while (groupResults.hasMoreElements()) { // <--- Here
> groupResults may be null
> ...
> }
> ...
> }{noformat}
> h2. How to reproduce:
> (1) Set {{hadoop.security.group.mapping.ldap.search.group.hierarchy.levels}}
> to 1
> (2) Run test
> {{org.apache.hadoop.security.TestLdapGroupsMapping#testGetGroupsWithConnectionClosed}}
> h2. Stack trace:
> {noformat}
> java.lang.NullPointerException
> at
> org.apache.hadoop.security.LdapGroupsMapping.goUpGroupHierarchy(LdapGroupsMapping.java:612)
>
[jira] [Updated] (HADOOP-18821) LdapGroupsMapping crashes with NullPointerException while going up the group hierarchy
[
https://issues.apache.org/jira/browse/HADOOP-18821?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ConfX updated HADOOP-18821:
---
Affects Version/s: 3.3.6
> LdapGroupsMapping crashes with NullPointerException while going up the group
> hierarchy
> ---
>
> Key: HADOOP-18821
> URL: https://issues.apache.org/jira/browse/HADOOP-18821
> Project: Hadoop Common
> Issue Type: Bug
>Affects Versions: 3.3.6
>Reporter: ConfX
>Priority: Critical
> Attachments: reproduce.sh
>
>
> h2. What happened:
> When set {{hadoop.security.group.mapping.ldap.search.group.hierarchy.levels}}
> to a value larger than 0, {{goUpGroupHierarchy}} in
> {{org/apache/hadoop/security/LdapGroupsMapping.java}} may return a null
> {{{}groupResults{}}}and use it without checking null.
> h2. Buggy code:
> {noformat}
> void goUpGroupHierarchy(Set groupDNs, int goUpHierarchy,
> Set groups) throws NamingException {
> if (goUpHierarchy <= 0 || groups.isEmpty()) {
> return;
> }
> ...
> NamingEnumeration groupResults =
> context.search(groupbaseDN, filter.toString(), SEARCH_CONTROLS);
> while (groupResults.hasMoreElements()) { // <--- Here
> groupResults may be null
> ...
> }
> ...
> }{noformat}
> h2. How to reproduce:
> (1) Set {{hadoop.security.group.mapping.ldap.search.group.hierarchy.levels}}
> to 1
> (2) Run test
> {{org.apache.hadoop.security.TestLdapGroupsMapping#testGetGroupsWithConnectionClosed}}
> h2. Stack trace:
> {noformat}
> java.lang.NullPointerException
> at
> org.apache.hadoop.security.LdapGroupsMapping.goUpGroupHierarchy(LdapGroupsMapping.java:612)
> at
> org.apache.hadoop.security.LdapGroupsMapping.lookupGroup(LdapGroupsMapping.java:489)
> at
> org.apache.hadoop.security.LdapGroupsMapping.doGetGroups(LdapGroupsMapping.java:552)
> at
> org.apache.hadoop.security.LdapGroupsMapping.getGroups(LdapGroupsMapping.java:365){noformat}
> For an easy reproduction, run the reproduce.sh in the attachment.
> We are happy to provide a patch if this issue is confirmed.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-18821) LdapGroupsMapping crashes with NullPointerException while going up the group hierarchy
[
https://issues.apache.org/jira/browse/HADOOP-18821?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ConfX updated HADOOP-18821:
---
Summary: LdapGroupsMapping crashes with NullPointerException while going up
the group hierarchy (was: Walking up the ldap hierarchy may trigger null
pointer exception)
> LdapGroupsMapping crashes with NullPointerException while going up the group
> hierarchy
> ---
>
> Key: HADOOP-18821
> URL: https://issues.apache.org/jira/browse/HADOOP-18821
> Project: Hadoop Common
> Issue Type: Bug
>Reporter: ConfX
>Priority: Critical
> Attachments: reproduce.sh
>
>
> h2. What happened:
> When set {{hadoop.security.group.mapping.ldap.search.group.hierarchy.levels}}
> to a value larger than 0, {{goUpGroupHierarchy}} in
> {{org/apache/hadoop/security/LdapGroupsMapping.java}} may return a null
> {{{}groupResults{}}}and use it without checking null.
> h2. Buggy code:
> {noformat}
> void goUpGroupHierarchy(Set groupDNs, int goUpHierarchy,
> Set groups) throws NamingException {
> if (goUpHierarchy <= 0 || groups.isEmpty()) {
> return;
> }
> ...
> NamingEnumeration groupResults =
> context.search(groupbaseDN, filter.toString(), SEARCH_CONTROLS);
> while (groupResults.hasMoreElements()) { // <--- Here
> groupResults may be null
> ...
> }
> ...
> }{noformat}
> h2. How to reproduce:
> (1) Set {{hadoop.security.group.mapping.ldap.search.group.hierarchy.levels}}
> to 1
> (2) Run test
> {{org.apache.hadoop.security.TestLdapGroupsMapping#testGetGroupsWithConnectionClosed}}
> h2. Stack trace:
> {noformat}
> java.lang.NullPointerException
> at
> org.apache.hadoop.security.LdapGroupsMapping.goUpGroupHierarchy(LdapGroupsMapping.java:612)
> at
> org.apache.hadoop.security.LdapGroupsMapping.lookupGroup(LdapGroupsMapping.java:489)
> at
> org.apache.hadoop.security.LdapGroupsMapping.doGetGroups(LdapGroupsMapping.java:552)
> at
> org.apache.hadoop.security.LdapGroupsMapping.getGroups(LdapGroupsMapping.java:365){noformat}
> For an easy reproduction, run the reproduce.sh in the attachment.
> We are happy to provide a patch if this issue is confirmed.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] teamconfx opened a new pull request, #6065: HDFS-17189: GcTimeMonitor crashes with NegativeArraySizeException during initialization
teamconfx opened a new pull request, #6065: URL: https://github.com/apache/hadoop/pull/6065 ### Description of PR The garbage collection data buffer size equals to `dfs.namenode.gc.time.monitor.observation.window.ms` divided by `dfs.namenode.gc.time.monitor.sleep.interval.ms` plus 2. When the observation window is a large value, the caculation overflows, resulting in a negative buffer size. To reproduce: 1. set `dfs.namenode.gc.time.monitor.observation.window.ms` to 945099495m 2. run `mvn surefire:test -Dtest=org.apache.hadoop.hdfs.TestHDFSFileSystemContract#testWriteReadAndDeleteHalfABlock` This PR provides a fix by checking the computed buffer size is not only within `128 * 1024` but also greater than 0. ### How was this patch tested? Unit test ### For code changes: - [x] Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')? - [ ] Object storage: have the integration tests been executed and the endpoint declared according to the connector-specific documentation? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0? - [ ] If applicable, have you updated the LICENSE, LICENSE-binary, NOTICE-binary files? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-18896) NegativeArraySizeException thrown in FSOutputSummer.java given large file.bytes-per-checksum
[ https://issues.apache.org/jira/browse/HADOOP-18896?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ConfX updated HADOOP-18896: --- Description: Buffer size of FSOutputSummer equals to `file.bytes-per-checksum` times `BUFFER_NUM_CHUNKS`. A large `file.bytes-per-checksum` causes buffer size to overflow and crash with NegativeArraySizeException. To reproduce: 1. set `file.bytes-per-checksum` to 238609295 2. `mvn surefire:test -Dtest=org.apache.hadoop.hdfs.TestDecommissionWithStriped#testFileSmallerThanOneStripe` We created a PR that provides a fix which checks the buffer size is positive after multiplying `file.bytes-per-checksum` with `BUFFER_NUM_CHUNKS` was: Buffer size of FSOutputSummer equals to `file.bytes-per-checksum` times `BUFFER_NUM_CHUNKS`. A large `file.bytes-per-checksum` causes buffer size to overflow and crash with NegativeArraySizeException. To reproduce: 1. set `file.bytes-per-checksum` to 238609295 2. `mvn surefire:test -Dtest=org.apache.hadoop.hdfs.TestDecommissionWithStriped#testFileSmallerThanOneStripe` We linked this issue to a PR that provides a fix which checks the buffer size is positive after multiplying `file.bytes-per-checksum` with `BUFFER_NUM_CHUNKS` > NegativeArraySizeException thrown in FSOutputSummer.java given large > file.bytes-per-checksum > > > Key: HADOOP-18896 > URL: https://issues.apache.org/jira/browse/HADOOP-18896 > Project: Hadoop Common > Issue Type: Bug >Affects Versions: 3.3.6 >Reporter: ConfX >Priority: Critical > Labels: pull-request-available > > Buffer size of FSOutputSummer equals to `file.bytes-per-checksum` times > `BUFFER_NUM_CHUNKS`. A large `file.bytes-per-checksum` causes buffer size to > overflow and crash with NegativeArraySizeException. > To reproduce: > 1. set `file.bytes-per-checksum` to 238609295 > 2. `mvn surefire:test > -Dtest=org.apache.hadoop.hdfs.TestDecommissionWithStriped#testFileSmallerThanOneStripe` > We created a PR that provides a fix which checks the buffer size is positive > after multiplying `file.bytes-per-checksum` with `BUFFER_NUM_CHUNKS` -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18896) NegativeArraySizeException thrown in FSOutputSummer.java given large file.bytes-per-checksum
[ https://issues.apache.org/jira/browse/HADOOP-18896?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17764925#comment-17764925 ] ASF GitHub Bot commented on HADOOP-18896: - teamconfx opened a new pull request, #6064: URL: https://github.com/apache/hadoop/pull/6064 ### Description of PR Buffer size of FSOutputSummer equals to `file.bytes-per-checksum` times `BUFFER_NUM_CHUNKS`. A large `file.bytes-per-checksum` causes buffer size to overflow and crash with NegativeArraySizeException. To reproduce: 1. set `file.bytes-per-checksum` to 238609295 2. run `mvn surefire:test -Dtest=org.apache.hadoop.hdfs.TestDecommissionWithStriped#testFileSmallerThanOneStripe` This PR provides a fix which checks the buffer size is positive after multiplying `file.bytes-per-checksum` with `BUFFER_NUM_CHUNKS` ### How was this patch tested? Unit test ### For code changes: - [x] Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')? - [ ] Object storage: have the integration tests been executed and the endpoint declared according to the connector-specific documentation? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0? - [ ] If applicable, have you updated the LICENSE, LICENSE-binary, NOTICE-binary files? > NegativeArraySizeException thrown in FSOutputSummer.java given large > file.bytes-per-checksum > > > Key: HADOOP-18896 > URL: https://issues.apache.org/jira/browse/HADOOP-18896 > Project: Hadoop Common > Issue Type: Bug >Affects Versions: 3.3.6 >Reporter: ConfX >Priority: Critical > > Buffer size of FSOutputSummer equals to `file.bytes-per-checksum` times > `BUFFER_NUM_CHUNKS`. A large `file.bytes-per-checksum` causes buffer size to > overflow and crash with NegativeArraySizeException. > To reproduce: > 1. set `file.bytes-per-checksum` to 238609295 > 2. `mvn surefire:test > -Dtest=org.apache.hadoop.hdfs.TestDecommissionWithStriped#testFileSmallerThanOneStripe` > We linked this issue to a PR that provides a fix which checks the buffer size > is positive after multiplying `file.bytes-per-checksum` with > `BUFFER_NUM_CHUNKS` -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-18896) NegativeArraySizeException thrown in FSOutputSummer.java given large file.bytes-per-checksum
[ https://issues.apache.org/jira/browse/HADOOP-18896?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HADOOP-18896: Labels: pull-request-available (was: ) > NegativeArraySizeException thrown in FSOutputSummer.java given large > file.bytes-per-checksum > > > Key: HADOOP-18896 > URL: https://issues.apache.org/jira/browse/HADOOP-18896 > Project: Hadoop Common > Issue Type: Bug >Affects Versions: 3.3.6 >Reporter: ConfX >Priority: Critical > Labels: pull-request-available > > Buffer size of FSOutputSummer equals to `file.bytes-per-checksum` times > `BUFFER_NUM_CHUNKS`. A large `file.bytes-per-checksum` causes buffer size to > overflow and crash with NegativeArraySizeException. > To reproduce: > 1. set `file.bytes-per-checksum` to 238609295 > 2. `mvn surefire:test > -Dtest=org.apache.hadoop.hdfs.TestDecommissionWithStriped#testFileSmallerThanOneStripe` > We linked this issue to a PR that provides a fix which checks the buffer size > is positive after multiplying `file.bytes-per-checksum` with > `BUFFER_NUM_CHUNKS` -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] teamconfx opened a new pull request, #6064: [[HADOOP-18896](https://issues.apache.org/jira/browse/HADOOP-18896)] NegativeArraySizeException thrown in FSOutputSummer.java given large
teamconfx opened a new pull request, #6064: URL: https://github.com/apache/hadoop/pull/6064 ### Description of PR Buffer size of FSOutputSummer equals to `file.bytes-per-checksum` times `BUFFER_NUM_CHUNKS`. A large `file.bytes-per-checksum` causes buffer size to overflow and crash with NegativeArraySizeException. To reproduce: 1. set `file.bytes-per-checksum` to 238609295 2. run `mvn surefire:test -Dtest=org.apache.hadoop.hdfs.TestDecommissionWithStriped#testFileSmallerThanOneStripe` This PR provides a fix which checks the buffer size is positive after multiplying `file.bytes-per-checksum` with `BUFFER_NUM_CHUNKS` ### How was this patch tested? Unit test ### For code changes: - [x] Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')? - [ ] Object storage: have the integration tests been executed and the endpoint declared according to the connector-specific documentation? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0? - [ ] If applicable, have you updated the LICENSE, LICENSE-binary, NOTICE-binary files? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Created] (HADOOP-18896) NegativeArraySizeException thrown in FSOutputSummer.java given large file.bytes-per-checksum
ConfX created HADOOP-18896: -- Summary: NegativeArraySizeException thrown in FSOutputSummer.java given large file.bytes-per-checksum Key: HADOOP-18896 URL: https://issues.apache.org/jira/browse/HADOOP-18896 Project: Hadoop Common Issue Type: Bug Affects Versions: 3.3.6 Reporter: ConfX Buffer size of FSOutputSummer equals to `file.bytes-per-checksum` times `BUFFER_NUM_CHUNKS`. A large `file.bytes-per-checksum` causes buffer size to overflow and crash with NegativeArraySizeException. To reproduce: 1. set `file.bytes-per-checksum` to 238609295 2. `mvn surefire:test -Dtest=org.apache.hadoop.hdfs.TestDecommissionWithStriped#testFileSmallerThanOneStripe` We linked this issue to a PR that provides a fix which checks the buffer size is positive after multiplying `file.bytes-per-checksum` with `BUFFER_NUM_CHUNKS` -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #6063: YARN-11498 backport jettison exclusions
hadoop-yetus commented on PR #6063:
URL: https://github.com/apache/hadoop/pull/6063#issuecomment-1718449114
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 11m 1s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +0 :ok: | xmllint | 0m 0s | | xmllint was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| -1 :x: | test4tests | 0m 0s | | The patch doesn't appear to include
any new or modified tests. Please justify why no new tests are needed for this
patch. Also please list what manual steps were performed to verify this patch.
|
_ branch-3.3 Compile Tests _ |
| +0 :ok: | mvndep | 14m 20s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 39m 53s | | branch-3.3 passed |
| +1 :green_heart: | compile | 19m 17s | | branch-3.3 passed |
| +1 :green_heart: | mvnsite | 7m 44s | | branch-3.3 passed |
| +1 :green_heart: | javadoc | 5m 46s | | branch-3.3 passed |
| +1 :green_heart: | shadedclient | 127m 39s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 27s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 17m 49s | | the patch passed |
| -1 :x: | compile | 3m 19s |
[/patch-compile-root.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6063/1/artifact/out/patch-compile-root.txt)
| root in the patch failed. |
| -1 :x: | javac | 3m 19s |
[/patch-compile-root.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6063/1/artifact/out/patch-compile-root.txt)
| root in the patch failed. |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | mvnsite | 5m 37s | | the patch passed |
| +1 :green_heart: | javadoc | 3m 39s | | the patch passed |
| -1 :x: | shadedclient | 19m 38s | | patch has errors when building
and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 0m 15s | | hadoop-project in the patch
passed. |
| +1 :green_heart: | unit | 17m 41s | | hadoop-common in the patch
passed. |
| +1 :green_heart: | unit | 4m 34s | | hadoop-yarn-common in the patch
passed. |
| +1 :green_heart: | unit | 22m 30s | | hadoop-yarn-server-nodemanager
in the patch passed. |
| +1 :green_heart: | unit | 3m 53s | |
hadoop-yarn-server-applicationhistoryservice in the patch passed. |
| -1 :x: | unit | 93m 55s |
[/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6063/1/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt)
| hadoop-yarn-server-resourcemanager in the patch passed. |
| +1 :green_heart: | unit | 0m 51s | |
hadoop-yarn-applications-catalog-webapp in the patch passed. |
| +1 :green_heart: | unit | 0m 36s | | hadoop-resourceestimator in the
patch passed. |
| +1 :green_heart: | unit | 0m 17s | | hadoop-client-minicluster in
the patch passed. |
| +1 :green_heart: | asflicense | 0m 33s | | The patch does not
generate ASF License warnings. |
| | | 329m 27s | | |
| Reason | Tests |
|---:|:--|
| Failed junit tests |
hadoop.yarn.server.resourcemanager.reservation.TestCapacityOverTimePolicy |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6063/1/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/6063 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient codespell detsecrets xmllint |
| uname | Linux 2edf82886038 4.15.0-212-generic #223-Ubuntu SMP Tue May 23
13:09:22 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | branch-3.3 / bfcc491f6838ee66ab1761a454cfac7d7060c1b4 |
| Default Java | Private Build-1.8.0_362-8u372-ga~us1-0ubuntu1~18.04-b09 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6063/1/testReport/ |
| Max. process+thread count | 1234 (vs. ulimit of 5500) |
| modules | C: hadoop-project
[jira] [Commented] (HADOOP-18894) upgrade sshd-core due to CVEs
[
https://issues.apache.org/jira/browse/HADOOP-18894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17764865#comment-17764865
]
ASF GitHub Bot commented on HADOOP-18894:
-
hadoop-yetus commented on PR #6060:
URL: https://github.com/apache/hadoop/pull/6060#issuecomment-1718264413
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 42s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 1s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +0 :ok: | xmllint | 0m 0s | | xmllint was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 2 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 14m 57s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 32m 24s | | trunk passed |
| +1 :green_heart: | compile | 17m 17s | | trunk passed with JDK
Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04 |
| +1 :green_heart: | compile | 16m 6s | | trunk passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | checkstyle | 4m 26s | | trunk passed |
| +1 :green_heart: | mvnsite | 2m 36s | | trunk passed |
| +1 :green_heart: | javadoc | 2m 11s | | trunk passed with JDK
Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04 |
| +1 :green_heart: | javadoc | 1m 45s | | trunk passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +0 :ok: | spotbugs | 0m 53s | | branch/hadoop-project no spotbugs
output file (spotbugsXml.xml) |
| +1 :green_heart: | shadedclient | 35m 38s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 41s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 1m 15s | | the patch passed |
| +1 :green_heart: | compile | 16m 23s | | the patch passed with JDK
Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04 |
| +1 :green_heart: | javac | 16m 23s | | the patch passed |
| +1 :green_heart: | compile | 16m 11s | | the patch passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | javac | 16m 11s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 4m 24s | | the patch passed |
| +1 :green_heart: | mvnsite | 2m 33s | | the patch passed |
| +1 :green_heart: | javadoc | 2m 5s | | the patch passed with JDK
Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04 |
| +1 :green_heart: | javadoc | 1m 45s | | the patch passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +0 :ok: | spotbugs | 0m 45s | | hadoop-project has no data from
spotbugs |
| +1 :green_heart: | shadedclient | 36m 20s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 0m 44s | | hadoop-project in the patch
passed. |
| +1 :green_heart: | unit | 19m 18s | | hadoop-common in the patch
passed. |
| +1 :green_heart: | asflicense | 1m 11s | | The patch does not
generate ASF License warnings. |
| | | 243m 11s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6060/3/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/6060 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient codespell detsecrets xmllint spotbugs checkstyle |
| uname | Linux e9e2bac40623 4.15.0-212-generic #223-Ubuntu SMP Tue May 23
13:09:22 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 09e73a3be636354918ec5162364d115bd404545b |
| Default Java | Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6060/3/testReport/ |
| Max. process+thread count | 3159 (vs. ulimit of 5500) |
| modules | C: hadoop-project
[GitHub] [hadoop] hadoop-yetus commented on pull request #6060: HADOOP-18894: upgrade sshd-core due to CVEs
hadoop-yetus commented on PR #6060:
URL: https://github.com/apache/hadoop/pull/6060#issuecomment-1718264413
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 42s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 1s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +0 :ok: | xmllint | 0m 0s | | xmllint was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 2 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 14m 57s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 32m 24s | | trunk passed |
| +1 :green_heart: | compile | 17m 17s | | trunk passed with JDK
Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04 |
| +1 :green_heart: | compile | 16m 6s | | trunk passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | checkstyle | 4m 26s | | trunk passed |
| +1 :green_heart: | mvnsite | 2m 36s | | trunk passed |
| +1 :green_heart: | javadoc | 2m 11s | | trunk passed with JDK
Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04 |
| +1 :green_heart: | javadoc | 1m 45s | | trunk passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +0 :ok: | spotbugs | 0m 53s | | branch/hadoop-project no spotbugs
output file (spotbugsXml.xml) |
| +1 :green_heart: | shadedclient | 35m 38s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 41s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 1m 15s | | the patch passed |
| +1 :green_heart: | compile | 16m 23s | | the patch passed with JDK
Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04 |
| +1 :green_heart: | javac | 16m 23s | | the patch passed |
| +1 :green_heart: | compile | 16m 11s | | the patch passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | javac | 16m 11s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 4m 24s | | the patch passed |
| +1 :green_heart: | mvnsite | 2m 33s | | the patch passed |
| +1 :green_heart: | javadoc | 2m 5s | | the patch passed with JDK
Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04 |
| +1 :green_heart: | javadoc | 1m 45s | | the patch passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +0 :ok: | spotbugs | 0m 45s | | hadoop-project has no data from
spotbugs |
| +1 :green_heart: | shadedclient | 36m 20s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 0m 44s | | hadoop-project in the patch
passed. |
| +1 :green_heart: | unit | 19m 18s | | hadoop-common in the patch
passed. |
| +1 :green_heart: | asflicense | 1m 11s | | The patch does not
generate ASF License warnings. |
| | | 243m 11s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6060/3/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/6060 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient codespell detsecrets xmllint spotbugs checkstyle |
| uname | Linux e9e2bac40623 4.15.0-212-generic #223-Ubuntu SMP Tue May 23
13:09:22 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 09e73a3be636354918ec5162364d115bd404545b |
| Default Java | Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6060/3/testReport/ |
| Max. process+thread count | 3159 (vs. ulimit of 5500) |
| modules | C: hadoop-project hadoop-common-project/hadoop-common U: . |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6060/3/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
[GitHub] [hadoop] goiri commented on a diff in pull request #6016: YARN-9048. Add znode hierarchy in Federation ZK State Store.
goiri commented on code in PR #6016:
URL: https://github.com/apache/hadoop/pull/6016#discussion_r1324968497
##
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/federation/store/impl/ZookeeperFederationStateStore.java:
##
@@ -123,16 +126,24 @@
import static org.apache.hadoop.util.curator.ZKCuratorManager.getNodePath;
/**
- * ZooKeeper implementation of {@link FederationStateStore}.
+ * ZooKeeper's implementation of {@link FederationStateStore}.
Review Comment:
I would leave it as it was.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] goiri commented on pull request #5921: HDFS-17138 RBF: We changed the hadoop.security.auth_to_local configur…
goiri commented on PR #5921: URL: https://github.com/apache/hadoop/pull/5921#issuecomment-1718187234 This should be good to go. Can you rebase to trigger a new build? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18894) upgrade sshd-core due to CVEs
[
https://issues.apache.org/jira/browse/HADOOP-18894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17764825#comment-17764825
]
ASF GitHub Bot commented on HADOOP-18894:
-
hadoop-yetus commented on PR #6060:
URL: https://github.com/apache/hadoop/pull/6060#issuecomment-1718131338
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 1m 23s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available.
|
| +0 :ok: | xmllint | 0m 1s | | xmllint was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 2 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 15m 3s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 36m 30s | | trunk passed |
| +1 :green_heart: | compile | 18m 22s | | trunk passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | compile | 16m 27s | | trunk passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | checkstyle | 4m 43s | | trunk passed |
| +1 :green_heart: | mvnsite | 2m 8s | | trunk passed |
| +1 :green_heart: | javadoc | 1m 40s | | trunk passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 1m 17s | | trunk passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +0 :ok: | spotbugs | 0m 38s | | branch/hadoop-project no spotbugs
output file (spotbugsXml.xml) |
| +1 :green_heart: | shadedclient | 40m 9s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 34s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 1m 7s | | the patch passed |
| +1 :green_heart: | compile | 17m 31s | | the patch passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javac | 17m 31s | | the patch passed |
| +1 :green_heart: | compile | 16m 23s | | the patch passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | javac | 16m 23s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| -0 :warning: | checkstyle | 4m 36s |
[/results-checkstyle-root.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6060/2/artifact/out/results-checkstyle-root.txt)
| root: The patch generated 1 new + 3 unchanged - 0 fixed = 4 total (was 3) |
| +1 :green_heart: | mvnsite | 2m 7s | | the patch passed |
| +1 :green_heart: | javadoc | 1m 35s | | the patch passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 1m 18s | | the patch passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +0 :ok: | spotbugs | 0m 31s | | hadoop-project has no data from
spotbugs |
| +1 :green_heart: | shadedclient | 40m 3s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 0m 29s | | hadoop-project in the patch
passed. |
| +1 :green_heart: | unit | 19m 0s | | hadoop-common in the patch
passed. |
| +1 :green_heart: | asflicense | 0m 57s | | The patch does not
generate ASF License warnings. |
| | | 254m 5s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6060/2/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/6060 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient codespell detsecrets xmllint spotbugs checkstyle |
| uname | Linux 2156a6889233 4.15.0-212-generic #223-Ubuntu SMP Tue May 23
13:09:22 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 125ef111a2118654b2503f68208f3d3226fb8439 |
| Default Java | Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| Test
[GitHub] [hadoop] hadoop-yetus commented on pull request #6060: HADOOP-18894: upgrade sshd-core due to CVEs
hadoop-yetus commented on PR #6060:
URL: https://github.com/apache/hadoop/pull/6060#issuecomment-1718131338
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 1m 23s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available.
|
| +0 :ok: | xmllint | 0m 1s | | xmllint was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 2 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 15m 3s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 36m 30s | | trunk passed |
| +1 :green_heart: | compile | 18m 22s | | trunk passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | compile | 16m 27s | | trunk passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | checkstyle | 4m 43s | | trunk passed |
| +1 :green_heart: | mvnsite | 2m 8s | | trunk passed |
| +1 :green_heart: | javadoc | 1m 40s | | trunk passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 1m 17s | | trunk passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +0 :ok: | spotbugs | 0m 38s | | branch/hadoop-project no spotbugs
output file (spotbugsXml.xml) |
| +1 :green_heart: | shadedclient | 40m 9s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 34s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 1m 7s | | the patch passed |
| +1 :green_heart: | compile | 17m 31s | | the patch passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javac | 17m 31s | | the patch passed |
| +1 :green_heart: | compile | 16m 23s | | the patch passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | javac | 16m 23s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| -0 :warning: | checkstyle | 4m 36s |
[/results-checkstyle-root.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6060/2/artifact/out/results-checkstyle-root.txt)
| root: The patch generated 1 new + 3 unchanged - 0 fixed = 4 total (was 3) |
| +1 :green_heart: | mvnsite | 2m 7s | | the patch passed |
| +1 :green_heart: | javadoc | 1m 35s | | the patch passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 1m 18s | | the patch passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +0 :ok: | spotbugs | 0m 31s | | hadoop-project has no data from
spotbugs |
| +1 :green_heart: | shadedclient | 40m 3s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 0m 29s | | hadoop-project in the patch
passed. |
| +1 :green_heart: | unit | 19m 0s | | hadoop-common in the patch
passed. |
| +1 :green_heart: | asflicense | 0m 57s | | The patch does not
generate ASF License warnings. |
| | | 254m 5s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6060/2/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/6060 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient codespell detsecrets xmllint spotbugs checkstyle |
| uname | Linux 2156a6889233 4.15.0-212-generic #223-Ubuntu SMP Tue May 23
13:09:22 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 125ef111a2118654b2503f68208f3d3226fb8439 |
| Default Java | Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6060/2/testReport/ |
| Max. process+thread count | 3010 (vs. ulimit of 5500) |
| modules | C: hadoop-project hadoop-common-project/hadoop-common U: . |
| Console output |
[GitHub] [hadoop] pjfanning commented on pull request #5786: YARN-11498 add exclusion for jettison everywhere jersey-json is loaded
pjfanning commented on PR #5786: URL: https://github.com/apache/hadoop/pull/5786#issuecomment-1718096102 @steveloughran I've created https://github.com/apache/hadoop/pull/6063 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] pjfanning opened a new pull request, #6063: YARN-11498 backport jettison exclusions
pjfanning opened a new pull request, #6063: URL: https://github.com/apache/hadoop/pull/6063 ### Description of PR backport https://github.com/apache/hadoop/pull/5623 and https://github.com/apache/hadoop/pull/5786 ### How was this patch tested? CI build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18851) AbstractDelegationTokenSecretManager- Performance improvement by optimising the synchronization context
[
https://issues.apache.org/jira/browse/HADOOP-18851?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17764809#comment-17764809
]
ASF GitHub Bot commented on HADOOP-18851:
-
hadoop-yetus commented on PR #6001:
URL: https://github.com/apache/hadoop/pull/6001#issuecomment-1718056474
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 58s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| -1 :x: | test4tests | 0m 0s | | The patch doesn't appear to include
any new or modified tests. Please justify why no new tests are needed for this
patch. Also please list what manual steps were performed to verify this patch.
|
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 44m 28s | | trunk passed |
| +1 :green_heart: | compile | 17m 7s | | trunk passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | compile | 15m 59s | | trunk passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | checkstyle | 1m 23s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 47s | | trunk passed |
| +1 :green_heart: | javadoc | 1m 22s | | trunk passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 0m 59s | | trunk passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | spotbugs | 2m 50s | | trunk passed |
| +1 :green_heart: | shadedclient | 37m 42s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 1m 0s | | the patch passed |
| +1 :green_heart: | compile | 17m 21s | | the patch passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javac | 17m 21s | | the patch passed |
| +1 :green_heart: | compile | 16m 16s | | the patch passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | javac | 16m 16s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 1m 13s | | the patch passed |
| +1 :green_heart: | mvnsite | 1m 41s | | the patch passed |
| +1 :green_heart: | javadoc | 1m 13s | | the patch passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 0m 53s | | the patch passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | spotbugs | 2m 44s | | the patch passed |
| +1 :green_heart: | shadedclient | 38m 39s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 19m 38s | | hadoop-common in the patch
passed. |
| +1 :green_heart: | asflicense | 1m 5s | | The patch does not
generate ASF License warnings. |
| | | 230m 30s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6001/7/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/6001 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux a2e40121feb1 4.15.0-212-generic #223-Ubuntu SMP Tue May 23
13:09:22 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / d5eded867b72eb6d5018ea63ee20bde882f21b29 |
| Default Java | Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6001/7/testReport/ |
| Max. process+thread count | 1278 (vs. ulimit of 5500) |
| modules | C: hadoop-common-project/hadoop-common U:
hadoop-common-project/hadoop-common |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6001/7/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0
[GitHub] [hadoop] hadoop-yetus commented on pull request #6001: HADOOP-18851: Performance improvement for DelegationTokenSecretManager.
hadoop-yetus commented on PR #6001:
URL: https://github.com/apache/hadoop/pull/6001#issuecomment-1718056474
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 58s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| -1 :x: | test4tests | 0m 0s | | The patch doesn't appear to include
any new or modified tests. Please justify why no new tests are needed for this
patch. Also please list what manual steps were performed to verify this patch.
|
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 44m 28s | | trunk passed |
| +1 :green_heart: | compile | 17m 7s | | trunk passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | compile | 15m 59s | | trunk passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | checkstyle | 1m 23s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 47s | | trunk passed |
| +1 :green_heart: | javadoc | 1m 22s | | trunk passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 0m 59s | | trunk passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | spotbugs | 2m 50s | | trunk passed |
| +1 :green_heart: | shadedclient | 37m 42s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 1m 0s | | the patch passed |
| +1 :green_heart: | compile | 17m 21s | | the patch passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javac | 17m 21s | | the patch passed |
| +1 :green_heart: | compile | 16m 16s | | the patch passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | javac | 16m 16s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 1m 13s | | the patch passed |
| +1 :green_heart: | mvnsite | 1m 41s | | the patch passed |
| +1 :green_heart: | javadoc | 1m 13s | | the patch passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 0m 53s | | the patch passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | spotbugs | 2m 44s | | the patch passed |
| +1 :green_heart: | shadedclient | 38m 39s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 19m 38s | | hadoop-common in the patch
passed. |
| +1 :green_heart: | asflicense | 1m 5s | | The patch does not
generate ASF License warnings. |
| | | 230m 30s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6001/7/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/6001 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux a2e40121feb1 4.15.0-212-generic #223-Ubuntu SMP Tue May 23
13:09:22 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / d5eded867b72eb6d5018ea63ee20bde882f21b29 |
| Default Java | Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6001/7/testReport/ |
| Max. process+thread count | 1278 (vs. ulimit of 5500) |
| modules | C: hadoop-common-project/hadoop-common U:
hadoop-common-project/hadoop-common |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6001/7/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To
[jira] [Commented] (HADOOP-18888) S3A. createS3AsyncClient() always enables multipart
[ https://issues.apache.org/jira/browse/HADOOP-1?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17764801#comment-17764801 ] ASF GitHub Bot commented on HADOOP-1: - steveloughran commented on PR #6056: URL: https://github.com/apache/hadoop/pull/6056#issuecomment-1718037318 thanks; will make sure I've run the full test suite and merge > S3A. createS3AsyncClient() always enables multipart > --- > > Key: HADOOP-1 > URL: https://issues.apache.org/jira/browse/HADOOP-1 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/s3 >Affects Versions: 3.4.0 >Reporter: Steve Loughran >Assignee: Steve Loughran >Priority: Major > Labels: pull-request-available > > DefaultS3ClientFactory.createS3AsyncClient() always creates clients with > multipart enabled; if it is disabled in s3a config it should be disabled here > and in the transfer manager -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] steveloughran commented on pull request #6056: HADOOP-18888. S3A. createS3AsyncClient() always enables multipart
steveloughran commented on PR #6056: URL: https://github.com/apache/hadoop/pull/6056#issuecomment-1718037318 thanks; will make sure I've run the full test suite and merge -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] steveloughran commented on a diff in pull request #6038: MAPREDUCE-7448. Inconsistent Behavior for FileOutputCommitter V1 to commit successfully many times
steveloughran commented on code in PR #6038:
URL: https://github.com/apache/hadoop/pull/6038#discussion_r1324843146
##
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/lib/output/FileOutputCommitter.java:
##
@@ -158,6 +158,11 @@ public FileOutputCommitter(Path outputPath,
"output directory:" + skipCleanup + ", ignore cleanup failures: " +
ignoreCleanupFailures);
+if (algorithmVersion == 1 && skipCleanup) {
+LOG.warn("Skip cleaning up when using FileOutputCommitter V1 can lead
to unexpected behaviors. " +
+"For example, committing several times may be allowed
falsely.");
Review Comment:
"Skip cleaning up when using FileOutputCommitter V1 may corrupt the output".
there's another option here: we just ignore the setting on v1 jobs?
it's only there because directory deletion is so O(files) on GCS, and it
targets v2 because that same file-by-file operation means that directory rename
is never atomic; you may as well use the already unsafe v2 algorithm.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] steveloughran commented on pull request #6059: HADOOP-18892 fix Catalog Webapp module compile errors due to low nodejs version
steveloughran commented on PR #6059: URL: https://github.com/apache/hadoop/pull/6059#issuecomment-1718027775 - moved to be a yarn pr; please change title and commit jira to YARN-11569 - the yarn team will need to review (not me) - all PRs need to be against hadoop `trunk` branch, then backported to branch-3.3, branch-3.2 as appropriate. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] steveloughran commented on pull request #5786: YARN-11498 add exclusion for jettison everywhere jersey-json is loaded
steveloughran commented on PR #5786: URL: https://github.com/apache/hadoop/pull/5786#issuecomment-1718011406 ok, in trunk. does it need to go into 3.3.x too? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] steveloughran merged pull request #5786: YARN-11498 add exclusion for jettison everywhere jersey-json is loaded
steveloughran merged PR #5786: URL: https://github.com/apache/hadoop/pull/5786 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18873) ABFS: AbfsOutputStream doesnt close DataBlocks object.
[ https://issues.apache.org/jira/browse/HADOOP-18873?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17764793#comment-17764793 ] ASF GitHub Bot commented on HADOOP-18873: - steveloughran commented on PR #6010: URL: https://github.com/apache/hadoop/pull/6010#issuecomment-1717997799 ...handing off review to @mehakmeet; if he's happy it's good > ABFS: AbfsOutputStream doesnt close DataBlocks object. > -- > > Key: HADOOP-18873 > URL: https://issues.apache.org/jira/browse/HADOOP-18873 > Project: Hadoop Common > Issue Type: Sub-task >Affects Versions: 3.3.4 >Reporter: Pranav Saxena >Assignee: Pranav Saxena >Priority: Major > Labels: pull-request-available > Fix For: 3.3.4 > > > AbfsOutputStream doesnt close the dataBlock object created for the upload. > What is the implication of not doing that: > DataBlocks has three implementations: > # ByteArrayBlock > ## This creates an object of DataBlockByteArrayOutputStream (child of > ByteArrayOutputStream: wrapper arround byte-arrray for populating, reading > the array. > ## This gets GCed. > # ByteBufferBlock: > ## There is a defined *DirectBufferPool* from which it tries to request the > directBuffer. > ## If nothing in the pool, a new directBuffer is created. > ## the `close` method on the this object has the responsiblity of returning > back the buffer to pool so it can be reused. > ## Since we are not calling the `close`: > ### The pool is rendered of less use, since each request creates a new > directBuffer from memory. > ### All the object can be GCed and the direct-memory allocated may be > returned on the GC. What if the process crashes, the memory never goes back > and cause memory issue on the machine. > # DiskBlock: > ## This creates a file on disk on which the data-to-upload is written. This > file gets deleted in startUpload().close(). > > startUpload() gives an object of BlockUploadData which gives method of > `toByteArray()` which is used in abfsOutputStream to get the byteArray in the > dataBlock. > > Method which uses the DataBlock object: > https://github.com/apache/hadoop/blob/fac7d26c5d7f791565cc3ab45d079e2cca725f95/hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsOutputStream.java#L298 -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] steveloughran commented on pull request #6010: HADOOP-18873. ABFS: AbfsOutputStream doesnt close DataBlocks object.
steveloughran commented on PR #6010: URL: https://github.com/apache/hadoop/pull/6010#issuecomment-1717997799 ...handing off review to @mehakmeet; if he's happy it's good -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18887) Java 17 Runtime Support
[ https://issues.apache.org/jira/browse/HADOOP-18887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17764788#comment-17764788 ] Steve Loughran commented on HADOOP-18887: - # happy to watch this, but not got time to review anything # changes like jar updates are best isolated into their own jiras -maybe create them under the big "java17" jira but declare as a requirement for here. just knowing what needs fixing is a good first step; launching mr jobs with the right --add-opens stuff for every process launched. # some of my colleagues have been busy with java17; they may have some input here. > Java 17 Runtime Support > --- > > Key: HADOOP-18887 > URL: https://issues.apache.org/jira/browse/HADOOP-18887 > Project: Hadoop Common > Issue Type: New Feature > Components: build >Reporter: Bruno Ramirez >Priority: Major > Original Estimate: 336h > Remaining Estimate: 336h > > This JIRA feature aims to extend the Java runtime support for Hadoop to > include Java 17 in addition to Java 8 and Java 11. Currently Hadoop version > 3.3.6 supports Java 8 compile and Java 8/11 runtime. The goal is to make > Hadoop compatible with Java 17 runtime as well. > The plan for this release is to allow Hadoop to default to Java 11/Java 17, > while still providing the flexibility for customers to configure Hadoop to > use Java 8, Java 11, or Java 17 based on their specific needs. This project's > objectives include: > # Certifying that Hadoop works seamlessly on Java 8/11/17 for common use > cases. > # Ensuring that running Hadoop on Java 11/17 does not disrupt other > applications and libraries such as Spark, Hive, Flink, Presto/Trino, and > HBase. > The decision to support Java 17 runtime is motivated by customer requests and > significant performance improvements observed in downstream applications like > Apache Hive and Apache Spark. The testing process encompasses unit tests, > integration tests, and performance tests, as well as verifying the proper > functioning of all Hadoop daemons with Java 17. > The project will address compile time issues across various Hadoop > components, ensuring that Hadoop remains compatible with Java 17 throughout > the entire codebase. > This ticket serves as a vital step in enhancing Hadoop's capabilities, > providing customers with more choices and improved performance for their big > data processing needs. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18860) Upgrade mockito to 4.11.0
[ https://issues.apache.org/jira/browse/HADOOP-18860?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17764785#comment-17764785 ] Steve Loughran commented on HADOOP-18860: - [~asrani_anmol] have you got another iteration of this PR which includes the client lib changes needed? > Upgrade mockito to 4.11.0 > - > > Key: HADOOP-18860 > URL: https://issues.apache.org/jira/browse/HADOOP-18860 > Project: Hadoop Common > Issue Type: Improvement > Components: build, test >Affects Versions: 3.3.6 >Reporter: Anmol Asrani >Assignee: Anmol Asrani >Priority: Major > Labels: pull-request-available > > Upgrading mockito in hadoop-project -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] steveloughran commented on pull request #6020: Oss/hadoop java17 runtime feature branch prerequisite
steveloughran commented on PR #6020: URL: https://github.com/apache/hadoop/pull/6020#issuecomment-1717974563 hi, welcome to contributing to hadoop. every jar update should go in as its own pr, own tests, own update to LICENSE-binary (which does need updating, as we log versions in there). that makes it easier to isolate updates. Mockito has an active update; your help getting that in should avoid some of the changes here. (this is an example of how we roll back a change...) HADOOP-18860. Upgrade mockito version to 4.11.0 (#5977) the graal stuff is its own issue too: how does that work with older releases? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18496) upgrade kotlin-stdlib due to CVEs
[
https://issues.apache.org/jira/browse/HADOOP-18496?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17764784#comment-17764784
]
ASF GitHub Bot commented on HADOOP-18496:
-
pjfanning commented on code in PR #6057:
URL: https://github.com/apache/hadoop/pull/6057#discussion_r1324791220
##
hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/oauth2/CredentialBasedAccessTokenProvider.java:
##
@@ -97,38 +104,37 @@ public synchronized String getAccessToken() throws
IOException {
}
void refresh() throws IOException {
-OkHttpClient client = new OkHttpClient.Builder()
-.connectTimeout(URLConnectionFactory.DEFAULT_SOCKET_TIMEOUT,
TimeUnit.MILLISECONDS)
-.readTimeout(URLConnectionFactory.DEFAULT_SOCKET_TIMEOUT,
TimeUnit.MILLISECONDS)
-.build();
-
-String bodyString = Utils.postBody(CLIENT_SECRET, getCredential(),
-GRANT_TYPE, CLIENT_CREDENTIALS,
-CLIENT_ID, clientId);
-
-RequestBody body = RequestBody.create(bodyString, URLENCODED);
-
-Request request = new Request.Builder()
-.url(refreshURL)
-.post(body)
+final List pairs = new ArrayList<>();
+pairs.add(new BasicNameValuePair(CLIENT_SECRET, getCredential()));
+pairs.add(new BasicNameValuePair(GRANT_TYPE, CLIENT_CREDENTIALS));
+pairs.add(new BasicNameValuePair(CLIENT_ID, clientId));
+final RequestConfig config = RequestConfig.custom()
+.setConnectTimeout(URLConnectionFactory.DEFAULT_SOCKET_TIMEOUT)
+
.setConnectionRequestTimeout(URLConnectionFactory.DEFAULT_SOCKET_TIMEOUT)
+.setSocketTimeout(URLConnectionFactory.DEFAULT_SOCKET_TIMEOUT)
.build();
-try (Response response = client.newCall(request).execute()) {
- if (!response.isSuccessful()) {
-throw new IOException("Unexpected code " + response);
- }
-
- if (response.code() != HttpStatus.SC_OK) {
-throw new IllegalArgumentException("Received invalid http response: "
-+ response.code() + ", text = " + response.toString());
+try (CloseableHttpClient client =
+
HttpClientBuilder.create().setDefaultRequestConfig(config).build()) {
+ final HttpPost httpPost = new HttpPost(refreshURL);
+ httpPost.setEntity(new UrlEncodedFormEntity(pairs,
StandardCharsets.UTF_8));
+ httpPost.setHeader(HttpHeaders.CONTENT_TYPE, URLENCODED);
+ try (CloseableHttpResponse response = client.execute(httpPost)) {
+final int statusCode = response.getStatusLine().getStatusCode();
+if (statusCode != HttpStatus.SC_OK) {
+ throw new IllegalArgumentException(
+ "Received invalid http response: " + statusCode + ", text = " +
+ EntityUtils.toString(response.getEntity()));
+}
+Map responseBody = JsonSerialization.mapReader().readValue(
Review Comment:
The pre-existing code did not check the content-type of the return. It is
not uncommon for an API to return JSON but not to have `application/json`
Content-Type. I can make the change but I'm worried that the API we call may
not set the expected content-type on the response.
> upgrade kotlin-stdlib due to CVEs
> -
>
> Key: HADOOP-18496
> URL: https://issues.apache.org/jira/browse/HADOOP-18496
> Project: Hadoop Common
> Issue Type: Improvement
>Reporter: PJ Fanning
>Priority: Major
> Labels: pull-request-available
>
> I'm not an expert on Kotlin but dependabot show these 2 CVEs with the version
> of kotlin-stdlib used in Hadoop.
> * [https://github.com/advisories/GHSA-cqj8-47ch-rvvq]
> * [https://github.com/advisories/GHSA-2qp4-g3q3-f92w]
> kotlin-stlib 1.6.0 is the minimum version needed to fix both. It might be
> better to use latest v1.6 jar (currently 1.6.21) or even use latest jar
> altogether (currently 1.7.20).
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] pjfanning commented on a diff in pull request #6057: HADOOP-18496: remove use of okhttp in runtime code
pjfanning commented on code in PR #6057:
URL: https://github.com/apache/hadoop/pull/6057#discussion_r1324791220
##
hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/oauth2/CredentialBasedAccessTokenProvider.java:
##
@@ -97,38 +104,37 @@ public synchronized String getAccessToken() throws
IOException {
}
void refresh() throws IOException {
-OkHttpClient client = new OkHttpClient.Builder()
-.connectTimeout(URLConnectionFactory.DEFAULT_SOCKET_TIMEOUT,
TimeUnit.MILLISECONDS)
-.readTimeout(URLConnectionFactory.DEFAULT_SOCKET_TIMEOUT,
TimeUnit.MILLISECONDS)
-.build();
-
-String bodyString = Utils.postBody(CLIENT_SECRET, getCredential(),
-GRANT_TYPE, CLIENT_CREDENTIALS,
-CLIENT_ID, clientId);
-
-RequestBody body = RequestBody.create(bodyString, URLENCODED);
-
-Request request = new Request.Builder()
-.url(refreshURL)
-.post(body)
+final List pairs = new ArrayList<>();
+pairs.add(new BasicNameValuePair(CLIENT_SECRET, getCredential()));
+pairs.add(new BasicNameValuePair(GRANT_TYPE, CLIENT_CREDENTIALS));
+pairs.add(new BasicNameValuePair(CLIENT_ID, clientId));
+final RequestConfig config = RequestConfig.custom()
+.setConnectTimeout(URLConnectionFactory.DEFAULT_SOCKET_TIMEOUT)
+
.setConnectionRequestTimeout(URLConnectionFactory.DEFAULT_SOCKET_TIMEOUT)
+.setSocketTimeout(URLConnectionFactory.DEFAULT_SOCKET_TIMEOUT)
.build();
-try (Response response = client.newCall(request).execute()) {
- if (!response.isSuccessful()) {
-throw new IOException("Unexpected code " + response);
- }
-
- if (response.code() != HttpStatus.SC_OK) {
-throw new IllegalArgumentException("Received invalid http response: "
-+ response.code() + ", text = " + response.toString());
+try (CloseableHttpClient client =
+
HttpClientBuilder.create().setDefaultRequestConfig(config).build()) {
+ final HttpPost httpPost = new HttpPost(refreshURL);
+ httpPost.setEntity(new UrlEncodedFormEntity(pairs,
StandardCharsets.UTF_8));
+ httpPost.setHeader(HttpHeaders.CONTENT_TYPE, URLENCODED);
+ try (CloseableHttpResponse response = client.execute(httpPost)) {
+final int statusCode = response.getStatusLine().getStatusCode();
+if (statusCode != HttpStatus.SC_OK) {
+ throw new IllegalArgumentException(
+ "Received invalid http response: " + statusCode + ", text = " +
+ EntityUtils.toString(response.getEntity()));
+}
+Map responseBody = JsonSerialization.mapReader().readValue(
Review Comment:
The pre-existing code did not check the content-type of the return. It is
not uncommon for an API to return JSON but not to have `application/json`
Content-Type. I can make the change but I'm worried that the API we call may
not set the expected content-type on the response.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18496) upgrade kotlin-stdlib due to CVEs
[
https://issues.apache.org/jira/browse/HADOOP-18496?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17764781#comment-17764781
]
ASF GitHub Bot commented on HADOOP-18496:
-
pjfanning commented on code in PR #6057:
URL: https://github.com/apache/hadoop/pull/6057#discussion_r1324786164
##
hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/oauth2/CredentialBasedAccessTokenProvider.java:
##
@@ -97,38 +104,37 @@ public synchronized String getAccessToken() throws
IOException {
}
void refresh() throws IOException {
-OkHttpClient client = new OkHttpClient.Builder()
-.connectTimeout(URLConnectionFactory.DEFAULT_SOCKET_TIMEOUT,
TimeUnit.MILLISECONDS)
-.readTimeout(URLConnectionFactory.DEFAULT_SOCKET_TIMEOUT,
TimeUnit.MILLISECONDS)
-.build();
-
-String bodyString = Utils.postBody(CLIENT_SECRET, getCredential(),
-GRANT_TYPE, CLIENT_CREDENTIALS,
-CLIENT_ID, clientId);
-
-RequestBody body = RequestBody.create(bodyString, URLENCODED);
-
-Request request = new Request.Builder()
-.url(refreshURL)
-.post(body)
+final List pairs = new ArrayList<>();
+pairs.add(new BasicNameValuePair(CLIENT_SECRET, getCredential()));
+pairs.add(new BasicNameValuePair(GRANT_TYPE, CLIENT_CREDENTIALS));
+pairs.add(new BasicNameValuePair(CLIENT_ID, clientId));
+final RequestConfig config = RequestConfig.custom()
+.setConnectTimeout(URLConnectionFactory.DEFAULT_SOCKET_TIMEOUT)
+
.setConnectionRequestTimeout(URLConnectionFactory.DEFAULT_SOCKET_TIMEOUT)
+.setSocketTimeout(URLConnectionFactory.DEFAULT_SOCKET_TIMEOUT)
.build();
-try (Response response = client.newCall(request).execute()) {
- if (!response.isSuccessful()) {
-throw new IOException("Unexpected code " + response);
- }
-
- if (response.code() != HttpStatus.SC_OK) {
-throw new IllegalArgumentException("Received invalid http response: "
-+ response.code() + ", text = " + response.toString());
+try (CloseableHttpClient client =
+
HttpClientBuilder.create().setDefaultRequestConfig(config).build()) {
+ final HttpPost httpPost = new HttpPost(refreshURL);
+ httpPost.setEntity(new UrlEncodedFormEntity(pairs,
StandardCharsets.UTF_8));
+ httpPost.setHeader(HttpHeaders.CONTENT_TYPE, URLENCODED);
+ try (CloseableHttpResponse response = client.execute(httpPost)) {
+final int statusCode = response.getStatusLine().getStatusCode();
+if (statusCode != HttpStatus.SC_OK) {
+ throw new IllegalArgumentException(
+ "Received invalid http response: " + statusCode + ", text = " +
+ EntityUtils.toString(response.getEntity()));
+}
+Map responseBody = JsonSerialization.mapReader().readValue(
+EntityUtils.toString(response.getEntity()));
+
+String newExpiresIn = responseBody.get(EXPIRES_IN).toString();
+timer.setExpiresIn(newExpiresIn);
+
+accessToken = responseBody.get(ACCESS_TOKEN).toString();
}
-
- Map responseBody = JsonSerialization.mapReader().readValue(
- response.body().string());
-
- String newExpiresIn = responseBody.get(EXPIRES_IN).toString();
- timer.setExpiresIn(newExpiresIn);
-
- accessToken = responseBody.get(ACCESS_TOKEN).toString();
+} catch (RuntimeException e) {
+ throw new IOException("Unable to obtain access token from credential",
e);
Review Comment:
I'm duplicating this because Spotbugs has its weird rules. The pre-existing
catch already does this - do I change all the catches?
> upgrade kotlin-stdlib due to CVEs
> -
>
> Key: HADOOP-18496
> URL: https://issues.apache.org/jira/browse/HADOOP-18496
> Project: Hadoop Common
> Issue Type: Improvement
>Reporter: PJ Fanning
>Priority: Major
> Labels: pull-request-available
>
> I'm not an expert on Kotlin but dependabot show these 2 CVEs with the version
> of kotlin-stdlib used in Hadoop.
> * [https://github.com/advisories/GHSA-cqj8-47ch-rvvq]
> * [https://github.com/advisories/GHSA-2qp4-g3q3-f92w]
> kotlin-stlib 1.6.0 is the minimum version needed to fix both. It might be
> better to use latest v1.6 jar (currently 1.6.21) or even use latest jar
> altogether (currently 1.7.20).
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] pjfanning commented on a diff in pull request #6057: HADOOP-18496: remove use of okhttp in runtime code
pjfanning commented on code in PR #6057:
URL: https://github.com/apache/hadoop/pull/6057#discussion_r1324786164
##
hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/oauth2/CredentialBasedAccessTokenProvider.java:
##
@@ -97,38 +104,37 @@ public synchronized String getAccessToken() throws
IOException {
}
void refresh() throws IOException {
-OkHttpClient client = new OkHttpClient.Builder()
-.connectTimeout(URLConnectionFactory.DEFAULT_SOCKET_TIMEOUT,
TimeUnit.MILLISECONDS)
-.readTimeout(URLConnectionFactory.DEFAULT_SOCKET_TIMEOUT,
TimeUnit.MILLISECONDS)
-.build();
-
-String bodyString = Utils.postBody(CLIENT_SECRET, getCredential(),
-GRANT_TYPE, CLIENT_CREDENTIALS,
-CLIENT_ID, clientId);
-
-RequestBody body = RequestBody.create(bodyString, URLENCODED);
-
-Request request = new Request.Builder()
-.url(refreshURL)
-.post(body)
+final List pairs = new ArrayList<>();
+pairs.add(new BasicNameValuePair(CLIENT_SECRET, getCredential()));
+pairs.add(new BasicNameValuePair(GRANT_TYPE, CLIENT_CREDENTIALS));
+pairs.add(new BasicNameValuePair(CLIENT_ID, clientId));
+final RequestConfig config = RequestConfig.custom()
+.setConnectTimeout(URLConnectionFactory.DEFAULT_SOCKET_TIMEOUT)
+
.setConnectionRequestTimeout(URLConnectionFactory.DEFAULT_SOCKET_TIMEOUT)
+.setSocketTimeout(URLConnectionFactory.DEFAULT_SOCKET_TIMEOUT)
.build();
-try (Response response = client.newCall(request).execute()) {
- if (!response.isSuccessful()) {
-throw new IOException("Unexpected code " + response);
- }
-
- if (response.code() != HttpStatus.SC_OK) {
-throw new IllegalArgumentException("Received invalid http response: "
-+ response.code() + ", text = " + response.toString());
+try (CloseableHttpClient client =
+
HttpClientBuilder.create().setDefaultRequestConfig(config).build()) {
+ final HttpPost httpPost = new HttpPost(refreshURL);
+ httpPost.setEntity(new UrlEncodedFormEntity(pairs,
StandardCharsets.UTF_8));
+ httpPost.setHeader(HttpHeaders.CONTENT_TYPE, URLENCODED);
+ try (CloseableHttpResponse response = client.execute(httpPost)) {
+final int statusCode = response.getStatusLine().getStatusCode();
+if (statusCode != HttpStatus.SC_OK) {
+ throw new IllegalArgumentException(
+ "Received invalid http response: " + statusCode + ", text = " +
+ EntityUtils.toString(response.getEntity()));
+}
+Map responseBody = JsonSerialization.mapReader().readValue(
+EntityUtils.toString(response.getEntity()));
+
+String newExpiresIn = responseBody.get(EXPIRES_IN).toString();
+timer.setExpiresIn(newExpiresIn);
+
+accessToken = responseBody.get(ACCESS_TOKEN).toString();
}
-
- Map responseBody = JsonSerialization.mapReader().readValue(
- response.body().string());
-
- String newExpiresIn = responseBody.get(EXPIRES_IN).toString();
- timer.setExpiresIn(newExpiresIn);
-
- accessToken = responseBody.get(ACCESS_TOKEN).toString();
+} catch (RuntimeException e) {
+ throw new IOException("Unable to obtain access token from credential",
e);
Review Comment:
I'm duplicating this because Spotbugs has its weird rules. The pre-existing
catch already does this - do I change all the catches?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18496) upgrade kotlin-stdlib due to CVEs
[
https://issues.apache.org/jira/browse/HADOOP-18496?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17764780#comment-17764780
]
ASF GitHub Bot commented on HADOOP-18496:
-
pjfanning commented on code in PR #6057:
URL: https://github.com/apache/hadoop/pull/6057#discussion_r1324784674
##
hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/oauth2/CredentialBasedAccessTokenProvider.java:
##
@@ -97,38 +104,37 @@ public synchronized String getAccessToken() throws
IOException {
}
void refresh() throws IOException {
-OkHttpClient client = new OkHttpClient.Builder()
-.connectTimeout(URLConnectionFactory.DEFAULT_SOCKET_TIMEOUT,
TimeUnit.MILLISECONDS)
-.readTimeout(URLConnectionFactory.DEFAULT_SOCKET_TIMEOUT,
TimeUnit.MILLISECONDS)
-.build();
-
-String bodyString = Utils.postBody(CLIENT_SECRET, getCredential(),
-GRANT_TYPE, CLIENT_CREDENTIALS,
-CLIENT_ID, clientId);
-
-RequestBody body = RequestBody.create(bodyString, URLENCODED);
-
-Request request = new Request.Builder()
-.url(refreshURL)
-.post(body)
+final List pairs = new ArrayList<>();
+pairs.add(new BasicNameValuePair(CLIENT_SECRET, getCredential()));
+pairs.add(new BasicNameValuePair(GRANT_TYPE, CLIENT_CREDENTIALS));
+pairs.add(new BasicNameValuePair(CLIENT_ID, clientId));
+final RequestConfig config = RequestConfig.custom()
+.setConnectTimeout(URLConnectionFactory.DEFAULT_SOCKET_TIMEOUT)
+
.setConnectionRequestTimeout(URLConnectionFactory.DEFAULT_SOCKET_TIMEOUT)
+.setSocketTimeout(URLConnectionFactory.DEFAULT_SOCKET_TIMEOUT)
.build();
-try (Response response = client.newCall(request).execute()) {
- if (!response.isSuccessful()) {
-throw new IOException("Unexpected code " + response);
- }
-
- if (response.code() != HttpStatus.SC_OK) {
-throw new IllegalArgumentException("Received invalid http response: "
-+ response.code() + ", text = " + response.toString());
+try (CloseableHttpClient client =
+
HttpClientBuilder.create().setDefaultRequestConfig(config).build()) {
+ final HttpPost httpPost = new HttpPost(refreshURL);
+ httpPost.setEntity(new UrlEncodedFormEntity(pairs,
StandardCharsets.UTF_8));
+ httpPost.setHeader(HttpHeaders.CONTENT_TYPE, URLENCODED);
+ try (CloseableHttpResponse response = client.execute(httpPost)) {
+final int statusCode = response.getStatusLine().getStatusCode();
+if (statusCode != HttpStatus.SC_OK) {
+ throw new IllegalArgumentException(
+ "Received invalid http response: " + statusCode + ", text = " +
+ EntityUtils.toString(response.getEntity()));
+}
+Map responseBody = JsonSerialization.mapReader().readValue(
+EntityUtils.toString(response.getEntity()));
+
+String newExpiresIn = responseBody.get(EXPIRES_IN).toString();
+timer.setExpiresIn(newExpiresIn);
+
+accessToken = responseBody.get(ACCESS_TOKEN).toString();
}
-
- Map responseBody = JsonSerialization.mapReader().readValue(
- response.body().string());
-
- String newExpiresIn = responseBody.get(EXPIRES_IN).toString();
- timer.setExpiresIn(newExpiresIn);
-
- accessToken = responseBody.get(ACCESS_TOKEN).toString();
+} catch (RuntimeException e) {
+ throw new IOException("Unable to obtain access token from credential",
e);
} catch (Exception e) {
Review Comment:
Spotbugs is what is making me duplicate the catch. It doesn't like catches
of Exception and this is one workaround.
> upgrade kotlin-stdlib due to CVEs
> -
>
> Key: HADOOP-18496
> URL: https://issues.apache.org/jira/browse/HADOOP-18496
> Project: Hadoop Common
> Issue Type: Improvement
>Reporter: PJ Fanning
>Priority: Major
> Labels: pull-request-available
>
> I'm not an expert on Kotlin but dependabot show these 2 CVEs with the version
> of kotlin-stdlib used in Hadoop.
> * [https://github.com/advisories/GHSA-cqj8-47ch-rvvq]
> * [https://github.com/advisories/GHSA-2qp4-g3q3-f92w]
> kotlin-stlib 1.6.0 is the minimum version needed to fix both. It might be
> better to use latest v1.6 jar (currently 1.6.21) or even use latest jar
> altogether (currently 1.7.20).
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] pjfanning commented on a diff in pull request #6057: HADOOP-18496: remove use of okhttp in runtime code
pjfanning commented on code in PR #6057:
URL: https://github.com/apache/hadoop/pull/6057#discussion_r1324784674
##
hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/oauth2/CredentialBasedAccessTokenProvider.java:
##
@@ -97,38 +104,37 @@ public synchronized String getAccessToken() throws
IOException {
}
void refresh() throws IOException {
-OkHttpClient client = new OkHttpClient.Builder()
-.connectTimeout(URLConnectionFactory.DEFAULT_SOCKET_TIMEOUT,
TimeUnit.MILLISECONDS)
-.readTimeout(URLConnectionFactory.DEFAULT_SOCKET_TIMEOUT,
TimeUnit.MILLISECONDS)
-.build();
-
-String bodyString = Utils.postBody(CLIENT_SECRET, getCredential(),
-GRANT_TYPE, CLIENT_CREDENTIALS,
-CLIENT_ID, clientId);
-
-RequestBody body = RequestBody.create(bodyString, URLENCODED);
-
-Request request = new Request.Builder()
-.url(refreshURL)
-.post(body)
+final List pairs = new ArrayList<>();
+pairs.add(new BasicNameValuePair(CLIENT_SECRET, getCredential()));
+pairs.add(new BasicNameValuePair(GRANT_TYPE, CLIENT_CREDENTIALS));
+pairs.add(new BasicNameValuePair(CLIENT_ID, clientId));
+final RequestConfig config = RequestConfig.custom()
+.setConnectTimeout(URLConnectionFactory.DEFAULT_SOCKET_TIMEOUT)
+
.setConnectionRequestTimeout(URLConnectionFactory.DEFAULT_SOCKET_TIMEOUT)
+.setSocketTimeout(URLConnectionFactory.DEFAULT_SOCKET_TIMEOUT)
.build();
-try (Response response = client.newCall(request).execute()) {
- if (!response.isSuccessful()) {
-throw new IOException("Unexpected code " + response);
- }
-
- if (response.code() != HttpStatus.SC_OK) {
-throw new IllegalArgumentException("Received invalid http response: "
-+ response.code() + ", text = " + response.toString());
+try (CloseableHttpClient client =
+
HttpClientBuilder.create().setDefaultRequestConfig(config).build()) {
+ final HttpPost httpPost = new HttpPost(refreshURL);
+ httpPost.setEntity(new UrlEncodedFormEntity(pairs,
StandardCharsets.UTF_8));
+ httpPost.setHeader(HttpHeaders.CONTENT_TYPE, URLENCODED);
+ try (CloseableHttpResponse response = client.execute(httpPost)) {
+final int statusCode = response.getStatusLine().getStatusCode();
+if (statusCode != HttpStatus.SC_OK) {
+ throw new IllegalArgumentException(
+ "Received invalid http response: " + statusCode + ", text = " +
+ EntityUtils.toString(response.getEntity()));
+}
+Map responseBody = JsonSerialization.mapReader().readValue(
+EntityUtils.toString(response.getEntity()));
+
+String newExpiresIn = responseBody.get(EXPIRES_IN).toString();
+timer.setExpiresIn(newExpiresIn);
+
+accessToken = responseBody.get(ACCESS_TOKEN).toString();
}
-
- Map responseBody = JsonSerialization.mapReader().readValue(
- response.body().string());
-
- String newExpiresIn = responseBody.get(EXPIRES_IN).toString();
- timer.setExpiresIn(newExpiresIn);
-
- accessToken = responseBody.get(ACCESS_TOKEN).toString();
+} catch (RuntimeException e) {
+ throw new IOException("Unable to obtain access token from credential",
e);
} catch (Exception e) {
Review Comment:
Spotbugs is what is making me duplicate the catch. It doesn't like catches
of Exception and this is one workaround.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18496) upgrade kotlin-stdlib due to CVEs
[
https://issues.apache.org/jira/browse/HADOOP-18496?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17764776#comment-17764776
]
ASF GitHub Bot commented on HADOOP-18496:
-
steveloughran commented on code in PR #6057:
URL: https://github.com/apache/hadoop/pull/6057#discussion_r1324778866
##
hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/oauth2/CredentialBasedAccessTokenProvider.java:
##
@@ -97,38 +104,37 @@ public synchronized String getAccessToken() throws
IOException {
}
void refresh() throws IOException {
-OkHttpClient client = new OkHttpClient.Builder()
-.connectTimeout(URLConnectionFactory.DEFAULT_SOCKET_TIMEOUT,
TimeUnit.MILLISECONDS)
-.readTimeout(URLConnectionFactory.DEFAULT_SOCKET_TIMEOUT,
TimeUnit.MILLISECONDS)
-.build();
-
-String bodyString = Utils.postBody(CLIENT_SECRET, getCredential(),
-GRANT_TYPE, CLIENT_CREDENTIALS,
-CLIENT_ID, clientId);
-
-RequestBody body = RequestBody.create(bodyString, URLENCODED);
-
-Request request = new Request.Builder()
-.url(refreshURL)
-.post(body)
+final List pairs = new ArrayList<>();
+pairs.add(new BasicNameValuePair(CLIENT_SECRET, getCredential()));
+pairs.add(new BasicNameValuePair(GRANT_TYPE, CLIENT_CREDENTIALS));
+pairs.add(new BasicNameValuePair(CLIENT_ID, clientId));
+final RequestConfig config = RequestConfig.custom()
+.setConnectTimeout(URLConnectionFactory.DEFAULT_SOCKET_TIMEOUT)
+
.setConnectionRequestTimeout(URLConnectionFactory.DEFAULT_SOCKET_TIMEOUT)
+.setSocketTimeout(URLConnectionFactory.DEFAULT_SOCKET_TIMEOUT)
.build();
-try (Response response = client.newCall(request).execute()) {
- if (!response.isSuccessful()) {
-throw new IOException("Unexpected code " + response);
- }
-
- if (response.code() != HttpStatus.SC_OK) {
-throw new IllegalArgumentException("Received invalid http response: "
-+ response.code() + ", text = " + response.toString());
+try (CloseableHttpClient client =
+
HttpClientBuilder.create().setDefaultRequestConfig(config).build()) {
+ final HttpPost httpPost = new HttpPost(refreshURL);
+ httpPost.setEntity(new UrlEncodedFormEntity(pairs,
StandardCharsets.UTF_8));
+ httpPost.setHeader(HttpHeaders.CONTENT_TYPE, URLENCODED);
+ try (CloseableHttpResponse response = client.execute(httpPost)) {
+final int statusCode = response.getStatusLine().getStatusCode();
+if (statusCode != HttpStatus.SC_OK) {
+ throw new IllegalArgumentException(
+ "Received invalid http response: " + statusCode + ", text = " +
+ EntityUtils.toString(response.getEntity()));
+}
+Map responseBody = JsonSerialization.mapReader().readValue(
Review Comment:
let's check the return content type too; been burned by both proxies and how
the abfs oauth failure is 200 + text/html for users.
##
hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/oauth2/CredentialBasedAccessTokenProvider.java:
##
@@ -97,38 +104,37 @@ public synchronized String getAccessToken() throws
IOException {
}
void refresh() throws IOException {
-OkHttpClient client = new OkHttpClient.Builder()
-.connectTimeout(URLConnectionFactory.DEFAULT_SOCKET_TIMEOUT,
TimeUnit.MILLISECONDS)
-.readTimeout(URLConnectionFactory.DEFAULT_SOCKET_TIMEOUT,
TimeUnit.MILLISECONDS)
-.build();
-
-String bodyString = Utils.postBody(CLIENT_SECRET, getCredential(),
-GRANT_TYPE, CLIENT_CREDENTIALS,
-CLIENT_ID, clientId);
-
-RequestBody body = RequestBody.create(bodyString, URLENCODED);
-
-Request request = new Request.Builder()
-.url(refreshURL)
-.post(body)
+final List pairs = new ArrayList<>();
+pairs.add(new BasicNameValuePair(CLIENT_SECRET, getCredential()));
+pairs.add(new BasicNameValuePair(GRANT_TYPE, CLIENT_CREDENTIALS));
+pairs.add(new BasicNameValuePair(CLIENT_ID, clientId));
+final RequestConfig config = RequestConfig.custom()
+.setConnectTimeout(URLConnectionFactory.DEFAULT_SOCKET_TIMEOUT)
+
.setConnectionRequestTimeout(URLConnectionFactory.DEFAULT_SOCKET_TIMEOUT)
+.setSocketTimeout(URLConnectionFactory.DEFAULT_SOCKET_TIMEOUT)
.build();
-try (Response response = client.newCall(request).execute()) {
- if (!response.isSuccessful()) {
-throw new IOException("Unexpected code " + response);
- }
-
- if (response.code() != HttpStatus.SC_OK) {
-throw new IllegalArgumentException("Received invalid http response: "
-+ response.code() + ", text = " + response.toString());
+try (CloseableHttpClient client =
+
[GitHub] [hadoop] steveloughran commented on a diff in pull request #6057: HADOOP-18496: remove use of okhttp in runtime code
steveloughran commented on code in PR #6057:
URL: https://github.com/apache/hadoop/pull/6057#discussion_r1324778866
##
hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/oauth2/CredentialBasedAccessTokenProvider.java:
##
@@ -97,38 +104,37 @@ public synchronized String getAccessToken() throws
IOException {
}
void refresh() throws IOException {
-OkHttpClient client = new OkHttpClient.Builder()
-.connectTimeout(URLConnectionFactory.DEFAULT_SOCKET_TIMEOUT,
TimeUnit.MILLISECONDS)
-.readTimeout(URLConnectionFactory.DEFAULT_SOCKET_TIMEOUT,
TimeUnit.MILLISECONDS)
-.build();
-
-String bodyString = Utils.postBody(CLIENT_SECRET, getCredential(),
-GRANT_TYPE, CLIENT_CREDENTIALS,
-CLIENT_ID, clientId);
-
-RequestBody body = RequestBody.create(bodyString, URLENCODED);
-
-Request request = new Request.Builder()
-.url(refreshURL)
-.post(body)
+final List pairs = new ArrayList<>();
+pairs.add(new BasicNameValuePair(CLIENT_SECRET, getCredential()));
+pairs.add(new BasicNameValuePair(GRANT_TYPE, CLIENT_CREDENTIALS));
+pairs.add(new BasicNameValuePair(CLIENT_ID, clientId));
+final RequestConfig config = RequestConfig.custom()
+.setConnectTimeout(URLConnectionFactory.DEFAULT_SOCKET_TIMEOUT)
+
.setConnectionRequestTimeout(URLConnectionFactory.DEFAULT_SOCKET_TIMEOUT)
+.setSocketTimeout(URLConnectionFactory.DEFAULT_SOCKET_TIMEOUT)
.build();
-try (Response response = client.newCall(request).execute()) {
- if (!response.isSuccessful()) {
-throw new IOException("Unexpected code " + response);
- }
-
- if (response.code() != HttpStatus.SC_OK) {
-throw new IllegalArgumentException("Received invalid http response: "
-+ response.code() + ", text = " + response.toString());
+try (CloseableHttpClient client =
+
HttpClientBuilder.create().setDefaultRequestConfig(config).build()) {
+ final HttpPost httpPost = new HttpPost(refreshURL);
+ httpPost.setEntity(new UrlEncodedFormEntity(pairs,
StandardCharsets.UTF_8));
+ httpPost.setHeader(HttpHeaders.CONTENT_TYPE, URLENCODED);
+ try (CloseableHttpResponse response = client.execute(httpPost)) {
+final int statusCode = response.getStatusLine().getStatusCode();
+if (statusCode != HttpStatus.SC_OK) {
+ throw new IllegalArgumentException(
+ "Received invalid http response: " + statusCode + ", text = " +
+ EntityUtils.toString(response.getEntity()));
+}
+Map responseBody = JsonSerialization.mapReader().readValue(
Review Comment:
let's check the return content type too; been burned by both proxies and how
the abfs oauth failure is 200 + text/html for users.
##
hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/oauth2/CredentialBasedAccessTokenProvider.java:
##
@@ -97,38 +104,37 @@ public synchronized String getAccessToken() throws
IOException {
}
void refresh() throws IOException {
-OkHttpClient client = new OkHttpClient.Builder()
-.connectTimeout(URLConnectionFactory.DEFAULT_SOCKET_TIMEOUT,
TimeUnit.MILLISECONDS)
-.readTimeout(URLConnectionFactory.DEFAULT_SOCKET_TIMEOUT,
TimeUnit.MILLISECONDS)
-.build();
-
-String bodyString = Utils.postBody(CLIENT_SECRET, getCredential(),
-GRANT_TYPE, CLIENT_CREDENTIALS,
-CLIENT_ID, clientId);
-
-RequestBody body = RequestBody.create(bodyString, URLENCODED);
-
-Request request = new Request.Builder()
-.url(refreshURL)
-.post(body)
+final List pairs = new ArrayList<>();
+pairs.add(new BasicNameValuePair(CLIENT_SECRET, getCredential()));
+pairs.add(new BasicNameValuePair(GRANT_TYPE, CLIENT_CREDENTIALS));
+pairs.add(new BasicNameValuePair(CLIENT_ID, clientId));
+final RequestConfig config = RequestConfig.custom()
+.setConnectTimeout(URLConnectionFactory.DEFAULT_SOCKET_TIMEOUT)
+
.setConnectionRequestTimeout(URLConnectionFactory.DEFAULT_SOCKET_TIMEOUT)
+.setSocketTimeout(URLConnectionFactory.DEFAULT_SOCKET_TIMEOUT)
.build();
-try (Response response = client.newCall(request).execute()) {
- if (!response.isSuccessful()) {
-throw new IOException("Unexpected code " + response);
- }
-
- if (response.code() != HttpStatus.SC_OK) {
-throw new IllegalArgumentException("Received invalid http response: "
-+ response.code() + ", text = " + response.toString());
+try (CloseableHttpClient client =
+
HttpClientBuilder.create().setDefaultRequestConfig(config).build()) {
+ final HttpPost httpPost = new HttpPost(refreshURL);
+ httpPost.setEntity(new UrlEncodedFormEntity(pairs,
StandardCharsets.UTF_8));
+ httpPost.setHeader(HttpHeaders.CONTENT_TYPE, URLENCODED);
+
[jira] [Commented] (HADOOP-18496) upgrade kotlin-stdlib due to CVEs
[ https://issues.apache.org/jira/browse/HADOOP-18496?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17764775#comment-17764775 ] ASF GitHub Bot commented on HADOOP-18496: - steveloughran commented on PR #6057: URL: https://github.com/apache/hadoop/pull/6057#issuecomment-1717950864 @rohit-kb what do you think of this? would it suit? > upgrade kotlin-stdlib due to CVEs > - > > Key: HADOOP-18496 > URL: https://issues.apache.org/jira/browse/HADOOP-18496 > Project: Hadoop Common > Issue Type: Improvement >Reporter: PJ Fanning >Priority: Major > Labels: pull-request-available > > I'm not an expert on Kotlin but dependabot show these 2 CVEs with the version > of kotlin-stdlib used in Hadoop. > * [https://github.com/advisories/GHSA-cqj8-47ch-rvvq] > * [https://github.com/advisories/GHSA-2qp4-g3q3-f92w] > kotlin-stlib 1.6.0 is the minimum version needed to fix both. It might be > better to use latest v1.6 jar (currently 1.6.21) or even use latest jar > altogether (currently 1.7.20). > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] steveloughran commented on pull request #6057: HADOOP-18496: remove use of okhttp in runtime code
steveloughran commented on PR #6057: URL: https://github.com/apache/hadoop/pull/6057#issuecomment-1717950864 @rohit-kb what do you think of this? would it suit? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18893) Make Trash Policy pluggable for different FileSystems
[
https://issues.apache.org/jira/browse/HADOOP-18893?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17764763#comment-17764763
]
ASF GitHub Bot commented on HADOOP-18893:
-
hadoop-yetus commented on PR #6061:
URL: https://github.com/apache/hadoop/pull/6061#issuecomment-1717907292
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 29s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +0 :ok: | xmllint | 0m 0s | | xmllint was not available. |
| +1 :green_heart: | @author | 0m 1s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 2 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 14m 44s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 20m 0s | | trunk passed |
| +1 :green_heart: | compile | 10m 40s | | trunk passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | compile | 9m 25s | | trunk passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | checkstyle | 2m 22s | | trunk passed |
| +1 :green_heart: | mvnsite | 2m 39s | | trunk passed |
| +1 :green_heart: | javadoc | 2m 10s | | trunk passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 2m 0s | | trunk passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | spotbugs | 3m 32s | | trunk passed |
| +1 :green_heart: | shadedclient | 23m 22s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 23s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 1m 26s | | the patch passed |
| +1 :green_heart: | compile | 10m 36s | | the patch passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javac | 10m 36s | | the patch passed |
| +1 :green_heart: | compile | 9m 34s | | the patch passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | javac | 9m 34s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| -0 :warning: | checkstyle | 2m 18s |
[/results-checkstyle-root.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6061/1/artifact/out/results-checkstyle-root.txt)
| root: The patch generated 5 new + 73 unchanged - 5 fixed = 78 total (was
78) |
| +1 :green_heart: | mvnsite | 2m 37s | | the patch passed |
| +1 :green_heart: | javadoc | 2m 6s | | the patch passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 2m 0s | | the patch passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| -1 :x: | spotbugs | 1m 45s |
[/new-spotbugs-hadoop-common-project_hadoop-common.html](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6061/1/artifact/out/new-spotbugs-hadoop-common-project_hadoop-common.html)
| hadoop-common-project/hadoop-common generated 1 new + 0 unchanged - 0 fixed
= 1 total (was 0) |
| +1 :green_heart: | shadedclient | 21m 30s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| -1 :x: | unit | 16m 29s |
[/patch-unit-hadoop-common-project_hadoop-common.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6061/1/artifact/out/patch-unit-hadoop-common-project_hadoop-common.txt)
| hadoop-common in the patch passed. |
| +1 :green_heart: | unit | 2m 39s | | hadoop-aws in the patch passed.
|
| +1 :green_heart: | unit | 2m 6s | | hadoop-azure in the patch
passed. |
| +1 :green_heart: | asflicense | 0m 49s | | The patch does not
generate ASF License warnings. |
| | | 175m 18s | | |
| Reason | Tests |
|---:|:--|
| SpotBugs | module:hadoop-common-project/hadoop-common |
| | Dead store to dir in
org.apache.hadoop.fs.TrashPolicyDefault.deleteCheckpoint(Path) At
TrashPolicyDefault.java:org.apache.hadoop.fs.TrashPolicyDefault.deleteCheckpoint(Path)
At TrashPolicyDefault.java:[line 426] |
| Failed junit tests | hadoop.conf.TestCommonConfigurationFields |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.43
[GitHub] [hadoop] hadoop-yetus commented on pull request #6061: HADOOP-18893. Make Trash Policy pluggable for different FileSystems
hadoop-yetus commented on PR #6061:
URL: https://github.com/apache/hadoop/pull/6061#issuecomment-1717907292
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 29s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +0 :ok: | xmllint | 0m 0s | | xmllint was not available. |
| +1 :green_heart: | @author | 0m 1s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 2 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 14m 44s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 20m 0s | | trunk passed |
| +1 :green_heart: | compile | 10m 40s | | trunk passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | compile | 9m 25s | | trunk passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | checkstyle | 2m 22s | | trunk passed |
| +1 :green_heart: | mvnsite | 2m 39s | | trunk passed |
| +1 :green_heart: | javadoc | 2m 10s | | trunk passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 2m 0s | | trunk passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | spotbugs | 3m 32s | | trunk passed |
| +1 :green_heart: | shadedclient | 23m 22s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 23s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 1m 26s | | the patch passed |
| +1 :green_heart: | compile | 10m 36s | | the patch passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javac | 10m 36s | | the patch passed |
| +1 :green_heart: | compile | 9m 34s | | the patch passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | javac | 9m 34s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| -0 :warning: | checkstyle | 2m 18s |
[/results-checkstyle-root.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6061/1/artifact/out/results-checkstyle-root.txt)
| root: The patch generated 5 new + 73 unchanged - 5 fixed = 78 total (was
78) |
| +1 :green_heart: | mvnsite | 2m 37s | | the patch passed |
| +1 :green_heart: | javadoc | 2m 6s | | the patch passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 2m 0s | | the patch passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| -1 :x: | spotbugs | 1m 45s |
[/new-spotbugs-hadoop-common-project_hadoop-common.html](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6061/1/artifact/out/new-spotbugs-hadoop-common-project_hadoop-common.html)
| hadoop-common-project/hadoop-common generated 1 new + 0 unchanged - 0 fixed
= 1 total (was 0) |
| +1 :green_heart: | shadedclient | 21m 30s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| -1 :x: | unit | 16m 29s |
[/patch-unit-hadoop-common-project_hadoop-common.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6061/1/artifact/out/patch-unit-hadoop-common-project_hadoop-common.txt)
| hadoop-common in the patch passed. |
| +1 :green_heart: | unit | 2m 39s | | hadoop-aws in the patch passed.
|
| +1 :green_heart: | unit | 2m 6s | | hadoop-azure in the patch
passed. |
| +1 :green_heart: | asflicense | 0m 49s | | The patch does not
generate ASF License warnings. |
| | | 175m 18s | | |
| Reason | Tests |
|---:|:--|
| SpotBugs | module:hadoop-common-project/hadoop-common |
| | Dead store to dir in
org.apache.hadoop.fs.TrashPolicyDefault.deleteCheckpoint(Path) At
TrashPolicyDefault.java:org.apache.hadoop.fs.TrashPolicyDefault.deleteCheckpoint(Path)
At TrashPolicyDefault.java:[line 426] |
| Failed junit tests | hadoop.conf.TestCommonConfigurationFields |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6061/1/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/6061 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
[jira] [Commented] (HADOOP-18873) ABFS: AbfsOutputStream doesnt close DataBlocks object.
[
https://issues.apache.org/jira/browse/HADOOP-18873?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17764761#comment-17764761
]
ASF GitHub Bot commented on HADOOP-18873:
-
hadoop-yetus commented on PR #6010:
URL: https://github.com/apache/hadoop/pull/6010#issuecomment-1717900485
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 29s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 1 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 14m 34s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 21m 0s | | trunk passed |
| +1 :green_heart: | compile | 11m 2s | | trunk passed with JDK
Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04 |
| +1 :green_heart: | compile | 9m 33s | | trunk passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | checkstyle | 2m 26s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 52s | | trunk passed |
| +1 :green_heart: | javadoc | 1m 37s | | trunk passed with JDK
Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04 |
| +1 :green_heart: | javadoc | 1m 19s | | trunk passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | spotbugs | 2m 30s | | trunk passed |
| +1 :green_heart: | shadedclient | 23m 21s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 26s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 0m 56s | | the patch passed |
| +1 :green_heart: | compile | 11m 18s | | the patch passed with JDK
Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04 |
| +1 :green_heart: | javac | 11m 18s | | the patch passed |
| +1 :green_heart: | compile | 9m 46s | | the patch passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | javac | 9m 46s | | the patch passed |
| +1 :green_heart: | blanks | 0m 1s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 2m 24s | | the patch passed |
| +1 :green_heart: | mvnsite | 1m 49s | | the patch passed |
| +1 :green_heart: | javadoc | 1m 30s | | the patch passed with JDK
Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04 |
| +1 :green_heart: | javadoc | 1m 21s | | the patch passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | spotbugs | 2m 55s | | the patch passed |
| +1 :green_heart: | shadedclient | 22m 27s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 16m 34s | | hadoop-common in the patch
passed. |
| +1 :green_heart: | unit | 2m 11s | | hadoop-azure in the patch
passed. |
| +1 :green_heart: | asflicense | 0m 49s | | The patch does not
generate ASF License warnings. |
| | | 169m 27s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6010/6/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/6010 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux 4090ed52a0ae 4.15.0-213-generic #224-Ubuntu SMP Mon Jun 19
13:30:12 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / d9da57b16ea00f4e79444a5109807ac6fef26e6c |
| Default Java | Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6010/6/testReport/ |
| Max. process+thread count | 1738 (vs. ulimit of 5500) |
| modules | C: hadoop-common-project/hadoop-common hadoop-tools/hadoop-azure
U: . |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6010/6/console |
|
[GitHub] [hadoop] hadoop-yetus commented on pull request #6010: HADOOP-18873. ABFS: AbfsOutputStream doesnt close DataBlocks object.
hadoop-yetus commented on PR #6010:
URL: https://github.com/apache/hadoop/pull/6010#issuecomment-1717900485
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 29s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 1 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 14m 34s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 21m 0s | | trunk passed |
| +1 :green_heart: | compile | 11m 2s | | trunk passed with JDK
Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04 |
| +1 :green_heart: | compile | 9m 33s | | trunk passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | checkstyle | 2m 26s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 52s | | trunk passed |
| +1 :green_heart: | javadoc | 1m 37s | | trunk passed with JDK
Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04 |
| +1 :green_heart: | javadoc | 1m 19s | | trunk passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | spotbugs | 2m 30s | | trunk passed |
| +1 :green_heart: | shadedclient | 23m 21s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 26s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 0m 56s | | the patch passed |
| +1 :green_heart: | compile | 11m 18s | | the patch passed with JDK
Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04 |
| +1 :green_heart: | javac | 11m 18s | | the patch passed |
| +1 :green_heart: | compile | 9m 46s | | the patch passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | javac | 9m 46s | | the patch passed |
| +1 :green_heart: | blanks | 0m 1s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 2m 24s | | the patch passed |
| +1 :green_heart: | mvnsite | 1m 49s | | the patch passed |
| +1 :green_heart: | javadoc | 1m 30s | | the patch passed with JDK
Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04 |
| +1 :green_heart: | javadoc | 1m 21s | | the patch passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | spotbugs | 2m 55s | | the patch passed |
| +1 :green_heart: | shadedclient | 22m 27s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 16m 34s | | hadoop-common in the patch
passed. |
| +1 :green_heart: | unit | 2m 11s | | hadoop-azure in the patch
passed. |
| +1 :green_heart: | asflicense | 0m 49s | | The patch does not
generate ASF License warnings. |
| | | 169m 27s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6010/6/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/6010 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux 4090ed52a0ae 4.15.0-213-generic #224-Ubuntu SMP Mon Jun 19
13:30:12 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / d9da57b16ea00f4e79444a5109807ac6fef26e6c |
| Default Java | Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6010/6/testReport/ |
| Max. process+thread count | 1738 (vs. ulimit of 5500) |
| modules | C: hadoop-common-project/hadoop-common hadoop-tools/hadoop-azure
U: . |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6010/6/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To
[GitHub] [hadoop] alkis commented on pull request #6025: Pass down eTag as part of accessCondition to SDK
alkis commented on PR #6025: URL: https://github.com/apache/hadoop/pull/6025#issuecomment-1717873630 Can we also handle deletes as well? They exhibit the same problem as uploads. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #6016: YARN-9048. Add znode hierarchy in Federation ZK State Store.
hadoop-yetus commented on PR #6016:
URL: https://github.com/apache/hadoop/pull/6016#issuecomment-1717871021
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 39s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 2 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 14m 23s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 30m 58s | | trunk passed |
| +1 :green_heart: | compile | 2m 33s | | trunk passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | compile | 2m 23s | | trunk passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | checkstyle | 1m 25s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 52s | | trunk passed |
| +1 :green_heart: | javadoc | 1m 46s | | trunk passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 1m 34s | | trunk passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | spotbugs | 3m 33s | | trunk passed |
| +1 :green_heart: | shadedclient | 34m 18s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 33s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 1m 26s | | the patch passed |
| +1 :green_heart: | compile | 2m 23s | | the patch passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javac | 2m 23s | | the patch passed |
| +1 :green_heart: | compile | 2m 17s | | the patch passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | javac | 2m 17s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 1m 12s | | the patch passed |
| +1 :green_heart: | mvnsite | 1m 34s | | the patch passed |
| +1 :green_heart: | javadoc | 1m 23s | | the patch passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 1m 18s | | the patch passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | spotbugs | 3m 26s | | the patch passed |
| +1 :green_heart: | shadedclient | 34m 54s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 3m 33s | | hadoop-yarn-server-common in
the patch passed. |
| +1 :green_heart: | unit | 100m 55s | |
hadoop-yarn-server-resourcemanager in the patch passed. |
| +1 :green_heart: | asflicense | 0m 39s | | The patch does not
generate ASF License warnings. |
| | | 254m 19s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6016/7/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/6016 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux 9ac195819be2 4.15.0-212-generic #223-Ubuntu SMP Tue May 23
13:09:22 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 64c76b4651bc4c4e318ee24937aa80afbd7ff7e8 |
| Default Java | Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6016/7/testReport/ |
| Max. process+thread count | 948 (vs. ulimit of 5500) |
| modules | C:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6016/7/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
|
[jira] [Commented] (HADOOP-18496) upgrade kotlin-stdlib due to CVEs
[
https://issues.apache.org/jira/browse/HADOOP-18496?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17764743#comment-17764743
]
ASF GitHub Bot commented on HADOOP-18496:
-
hadoop-yetus commented on PR #6057:
URL: https://github.com/apache/hadoop/pull/6057#issuecomment-1717842308
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 59s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available.
|
| +0 :ok: | xmllint | 0m 1s | | xmllint was not available. |
| +0 :ok: | shelldocs | 0m 1s | | Shelldocs was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| -1 :x: | test4tests | 0m 0s | | The patch doesn't appear to include
any new or modified tests. Please justify why no new tests are needed for this
patch. Also please list what manual steps were performed to verify this patch.
|
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 14m 37s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 36m 42s | | trunk passed |
| +1 :green_heart: | compile | 18m 26s | | trunk passed with JDK
Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04 |
| +1 :green_heart: | compile | 16m 41s | | trunk passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | checkstyle | 4m 40s | | trunk passed |
| +1 :green_heart: | mvnsite | 20m 13s | | trunk passed |
| +1 :green_heart: | javadoc | 8m 48s | | trunk passed with JDK
Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04 |
| +1 :green_heart: | javadoc | 7m 26s | | trunk passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +0 :ok: | spotbugs | 0m 17s | | branch/hadoop-project no spotbugs
output file (spotbugsXml.xml) |
| +0 :ok: | spotbugs | 0m 19s | |
branch/hadoop-client-modules/hadoop-client no spotbugs output file
(spotbugsXml.xml) |
| +1 :green_heart: | shadedclient | 68m 50s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 39s | | Maven dependency ordering for patch |
| -1 :x: | mvninstall | 0m 54s |
[/patch-mvninstall-hadoop-common-project_hadoop-common.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6057/2/artifact/out/patch-mvninstall-hadoop-common-project_hadoop-common.txt)
| hadoop-common in the patch failed. |
| -1 :x: | mvninstall | 0m 24s |
[/patch-mvninstall-hadoop-hdfs-project_hadoop-hdfs-client.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6057/2/artifact/out/patch-mvninstall-hadoop-hdfs-project_hadoop-hdfs-client.txt)
| hadoop-hdfs-client in the patch failed. |
| -1 :x: | mvninstall | 0m 17s |
[/patch-mvninstall-hadoop-hdfs-project_hadoop-hdfs-httpfs.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6057/2/artifact/out/patch-mvninstall-hadoop-hdfs-project_hadoop-hdfs-httpfs.txt)
| hadoop-hdfs-httpfs in the patch failed. |
| -1 :x: | mvninstall | 0m 16s |
[/patch-mvninstall-hadoop-cloud-storage-project_hadoop-huaweicloud.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6057/2/artifact/out/patch-mvninstall-hadoop-cloud-storage-project_hadoop-huaweicloud.txt)
| hadoop-huaweicloud in the patch failed. |
| -1 :x: | mvninstall | 1m 12s |
[/patch-mvninstall-root.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6057/2/artifact/out/patch-mvninstall-root.txt)
| root in the patch failed. |
| +1 :green_heart: | compile | 17m 35s | | the patch passed with JDK
Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04 |
| +1 :green_heart: | javac | 17m 35s | | the patch passed |
| +1 :green_heart: | compile | 16m 38s | | the patch passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | javac | 16m 38s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| -0 :warning: | checkstyle | 5m 49s |
[/results-checkstyle-root.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6057/2/artifact/out/results-checkstyle-root.txt)
| root: The patch generated 2 new + 0 unchanged - 0 fixed = 2 total (was 0) |
| +1 :green_heart: | mvnsite | 15m 8s | | the patch passed |
| +1 :green_heart: | shellcheck | 0m 0s | | No new issues. |
| +1 :green_heart: | javadoc | 8m 44s | | the patch passed with JDK
[GitHub] [hadoop] hadoop-yetus commented on pull request #6057: HADOOP-18496: remove use of okhttp in runtime code
hadoop-yetus commented on PR #6057:
URL: https://github.com/apache/hadoop/pull/6057#issuecomment-1717842308
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 59s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available.
|
| +0 :ok: | xmllint | 0m 1s | | xmllint was not available. |
| +0 :ok: | shelldocs | 0m 1s | | Shelldocs was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| -1 :x: | test4tests | 0m 0s | | The patch doesn't appear to include
any new or modified tests. Please justify why no new tests are needed for this
patch. Also please list what manual steps were performed to verify this patch.
|
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 14m 37s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 36m 42s | | trunk passed |
| +1 :green_heart: | compile | 18m 26s | | trunk passed with JDK
Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04 |
| +1 :green_heart: | compile | 16m 41s | | trunk passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | checkstyle | 4m 40s | | trunk passed |
| +1 :green_heart: | mvnsite | 20m 13s | | trunk passed |
| +1 :green_heart: | javadoc | 8m 48s | | trunk passed with JDK
Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04 |
| +1 :green_heart: | javadoc | 7m 26s | | trunk passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +0 :ok: | spotbugs | 0m 17s | | branch/hadoop-project no spotbugs
output file (spotbugsXml.xml) |
| +0 :ok: | spotbugs | 0m 19s | |
branch/hadoop-client-modules/hadoop-client no spotbugs output file
(spotbugsXml.xml) |
| +1 :green_heart: | shadedclient | 68m 50s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 39s | | Maven dependency ordering for patch |
| -1 :x: | mvninstall | 0m 54s |
[/patch-mvninstall-hadoop-common-project_hadoop-common.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6057/2/artifact/out/patch-mvninstall-hadoop-common-project_hadoop-common.txt)
| hadoop-common in the patch failed. |
| -1 :x: | mvninstall | 0m 24s |
[/patch-mvninstall-hadoop-hdfs-project_hadoop-hdfs-client.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6057/2/artifact/out/patch-mvninstall-hadoop-hdfs-project_hadoop-hdfs-client.txt)
| hadoop-hdfs-client in the patch failed. |
| -1 :x: | mvninstall | 0m 17s |
[/patch-mvninstall-hadoop-hdfs-project_hadoop-hdfs-httpfs.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6057/2/artifact/out/patch-mvninstall-hadoop-hdfs-project_hadoop-hdfs-httpfs.txt)
| hadoop-hdfs-httpfs in the patch failed. |
| -1 :x: | mvninstall | 0m 16s |
[/patch-mvninstall-hadoop-cloud-storage-project_hadoop-huaweicloud.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6057/2/artifact/out/patch-mvninstall-hadoop-cloud-storage-project_hadoop-huaweicloud.txt)
| hadoop-huaweicloud in the patch failed. |
| -1 :x: | mvninstall | 1m 12s |
[/patch-mvninstall-root.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6057/2/artifact/out/patch-mvninstall-root.txt)
| root in the patch failed. |
| +1 :green_heart: | compile | 17m 35s | | the patch passed with JDK
Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04 |
| +1 :green_heart: | javac | 17m 35s | | the patch passed |
| +1 :green_heart: | compile | 16m 38s | | the patch passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | javac | 16m 38s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| -0 :warning: | checkstyle | 5m 49s |
[/results-checkstyle-root.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6057/2/artifact/out/results-checkstyle-root.txt)
| root: The patch generated 2 new + 0 unchanged - 0 fixed = 2 total (was 0) |
| +1 :green_heart: | mvnsite | 15m 8s | | the patch passed |
| +1 :green_heart: | shellcheck | 0m 0s | | No new issues. |
| +1 :green_heart: | javadoc | 8m 44s | | the patch passed with JDK
Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04 |
| +1 :green_heart: | javadoc | 7m 24s | | the patch passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +0 :ok: | spotbugs | 0m 17s | | hadoop-project has no data from
[GitHub] [hadoop] hadoop-yetus commented on pull request #5943: [Testing] Fix shaded client for building Hadoop on Windows
hadoop-yetus commented on PR #5943:
URL: https://github.com/apache/hadoop/pull/5943#issuecomment-1717841305
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 29s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +0 :ok: | shelldocs | 0m 0s | | Shelldocs was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| -1 :x: | test4tests | 0m 0s | | The patch doesn't appear to include
any new or modified tests. Please justify why no new tests are needed for this
patch. Also please list what manual steps were performed to verify this patch.
|
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 37m 30s | | trunk passed |
| +1 :green_heart: | compile | 0m 44s | | trunk passed with JDK
Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04 |
| +1 :green_heart: | compile | 0m 39s | | trunk passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | checkstyle | 0m 26s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 43s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 38s | | trunk passed with JDK
Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04 |
| +1 :green_heart: | javadoc | 0m 30s | | trunk passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | spotbugs | 1m 38s | | trunk passed |
| +1 :green_heart: | shadedclient | 23m 29s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 35s | | the patch passed |
| +1 :green_heart: | compile | 0m 35s | | the patch passed with JDK
Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04 |
| +1 :green_heart: | javac | 0m 35s | | the patch passed |
| +1 :green_heart: | compile | 0m 31s | | the patch passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | javac | 0m 31s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 0m 15s | | the patch passed |
| +1 :green_heart: | mvnsite | 0m 33s | | the patch passed |
| -1 :x: | shellcheck | 0m 0s |
[/results-shellcheck.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5943/3/artifact/out/results-shellcheck.txt)
| The patch generated 13 new + 0 unchanged - 0 fixed = 13 total (was 0) |
| +1 :green_heart: | javadoc | 0m 29s | | the patch passed with JDK
Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04 |
| +1 :green_heart: | javadoc | 0m 25s | | the patch passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | spotbugs | 1m 38s | | the patch passed |
| +1 :green_heart: | shadedclient | 26m 47s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 1m 56s | | hadoop-hdfs-client in the patch
passed. |
| +1 :green_heart: | asflicense | 0m 27s | | The patch does not
generate ASF License warnings. |
| | | 103m 0s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5943/3/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/5943 |
| Optional Tests | dupname asflicense mvnsite unit codespell detsecrets
shellcheck shelldocs compile javac javadoc mvninstall shadedclient spotbugs
checkstyle |
| uname | Linux e5b0a648e4bc 4.15.0-213-generic #224-Ubuntu SMP Mon Jun 19
13:30:12 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 2b71c9883210287a0a54d89d75def39a62625764 |
| Default Java | Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5943/3/testReport/ |
| Max. process+thread count | 726 (vs. ulimit of 5500) |
| modules | C: hadoop-hdfs-project/hadoop-hdfs-client U:
hadoop-hdfs-project/hadoop-hdfs-client |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5943/3/console |
| versions
[jira] [Updated] (HADOOP-18895) upgrade to commons-compress 1.24.0
[ https://issues.apache.org/jira/browse/HADOOP-18895?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HADOOP-18895: Labels: pull-request-available (was: ) > upgrade to commons-compress 1.24.0 > -- > > Key: HADOOP-18895 > URL: https://issues.apache.org/jira/browse/HADOOP-18895 > Project: Hadoop Common > Issue Type: Improvement >Reporter: PJ Fanning >Priority: Major > Labels: pull-request-available > > Includes some important bug fixes -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] pjfanning opened a new pull request, #6062: HADOOP-18895: upgrade to commons-compress 1.24.0
pjfanning opened a new pull request, #6062: URL: https://github.com/apache/hadoop/pull/6062 ### Description of PR https://issues.apache.org/jira/browse/HADOOP-18895 ### How was this patch tested? CI Build ### For code changes: - [x] Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')? - [ ] Object storage: have the integration tests been executed and the endpoint declared according to the connector-specific documentation? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [x] If applicable, have you updated the `LICENSE`, `LICENSE-binary`, `NOTICE-binary` files? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18895) upgrade to commons-compress 1.24.0
[ https://issues.apache.org/jira/browse/HADOOP-18895?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17764739#comment-17764739 ] ASF GitHub Bot commented on HADOOP-18895: - pjfanning opened a new pull request, #6062: URL: https://github.com/apache/hadoop/pull/6062 ### Description of PR https://issues.apache.org/jira/browse/HADOOP-18895 ### How was this patch tested? CI Build ### For code changes: - [x] Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')? - [ ] Object storage: have the integration tests been executed and the endpoint declared according to the connector-specific documentation? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [x] If applicable, have you updated the `LICENSE`, `LICENSE-binary`, `NOTICE-binary` files? > upgrade to commons-compress 1.24.0 > -- > > Key: HADOOP-18895 > URL: https://issues.apache.org/jira/browse/HADOOP-18895 > Project: Hadoop Common > Issue Type: Improvement >Reporter: PJ Fanning >Priority: Major > > Includes some important bug fixes -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18496) upgrade kotlin-stdlib due to CVEs
[
https://issues.apache.org/jira/browse/HADOOP-18496?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17764736#comment-17764736
]
ASF GitHub Bot commented on HADOOP-18496:
-
hadoop-yetus commented on PR #6057:
URL: https://github.com/apache/hadoop/pull/6057#issuecomment-1717813326
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 51s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available.
|
| +0 :ok: | xmllint | 0m 1s | | xmllint was not available. |
| +0 :ok: | shelldocs | 0m 1s | | Shelldocs was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| -1 :x: | test4tests | 0m 0s | | The patch doesn't appear to include
any new or modified tests. Please justify why no new tests are needed for this
patch. Also please list what manual steps were performed to verify this patch.
|
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 14m 38s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 36m 11s | | trunk passed |
| +1 :green_heart: | compile | 18m 48s | | trunk passed with JDK
Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04 |
| +1 :green_heart: | compile | 16m 55s | | trunk passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | checkstyle | 4m 36s | | trunk passed |
| +1 :green_heart: | mvnsite | 20m 47s | | trunk passed |
| +1 :green_heart: | javadoc | 8m 55s | | trunk passed with JDK
Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04 |
| +1 :green_heart: | javadoc | 7m 39s | | trunk passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +0 :ok: | spotbugs | 0m 17s | | branch/hadoop-project no spotbugs
output file (spotbugsXml.xml) |
| +0 :ok: | spotbugs | 0m 19s | |
branch/hadoop-client-modules/hadoop-client no spotbugs output file
(spotbugsXml.xml) |
| +1 :green_heart: | shadedclient | 68m 48s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 41s | | Maven dependency ordering for patch |
| -1 :x: | mvninstall | 0m 57s |
[/patch-mvninstall-hadoop-common-project_hadoop-common.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6057/1/artifact/out/patch-mvninstall-hadoop-common-project_hadoop-common.txt)
| hadoop-common in the patch failed. |
| -1 :x: | mvninstall | 0m 24s |
[/patch-mvninstall-hadoop-hdfs-project_hadoop-hdfs-client.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6057/1/artifact/out/patch-mvninstall-hadoop-hdfs-project_hadoop-hdfs-client.txt)
| hadoop-hdfs-client in the patch failed. |
| -1 :x: | mvninstall | 0m 16s |
[/patch-mvninstall-hadoop-hdfs-project_hadoop-hdfs-httpfs.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6057/1/artifact/out/patch-mvninstall-hadoop-hdfs-project_hadoop-hdfs-httpfs.txt)
| hadoop-hdfs-httpfs in the patch failed. |
| -1 :x: | mvninstall | 0m 17s |
[/patch-mvninstall-hadoop-cloud-storage-project_hadoop-huaweicloud.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6057/1/artifact/out/patch-mvninstall-hadoop-cloud-storage-project_hadoop-huaweicloud.txt)
| hadoop-huaweicloud in the patch failed. |
| -1 :x: | mvninstall | 1m 17s |
[/patch-mvninstall-root.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6057/1/artifact/out/patch-mvninstall-root.txt)
| root in the patch failed. |
| +1 :green_heart: | compile | 17m 57s | | the patch passed with JDK
Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04 |
| +1 :green_heart: | javac | 17m 57s | | the patch passed |
| +1 :green_heart: | compile | 16m 53s | | the patch passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | javac | 16m 53s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| -0 :warning: | checkstyle | 5m 56s |
[/results-checkstyle-root.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6057/1/artifact/out/results-checkstyle-root.txt)
| root: The patch generated 2 new + 0 unchanged - 0 fixed = 2 total (was 0) |
| +1 :green_heart: | mvnsite | 15m 50s | | the patch passed |
| +1 :green_heart: | shellcheck | 0m 0s | | No new issues. |
| +1 :green_heart: | javadoc | 8m 52s | | the patch passed with JDK
[jira] [Created] (HADOOP-18895) upgrade to commons-compress 1.24.0
PJ Fanning created HADOOP-18895: --- Summary: upgrade to commons-compress 1.24.0 Key: HADOOP-18895 URL: https://issues.apache.org/jira/browse/HADOOP-18895 Project: Hadoop Common Issue Type: Improvement Reporter: PJ Fanning Includes some important bug fixes -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #6057: HADOOP-18496: remove use of okhttp in runtime code
hadoop-yetus commented on PR #6057:
URL: https://github.com/apache/hadoop/pull/6057#issuecomment-1717813326
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 51s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available.
|
| +0 :ok: | xmllint | 0m 1s | | xmllint was not available. |
| +0 :ok: | shelldocs | 0m 1s | | Shelldocs was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| -1 :x: | test4tests | 0m 0s | | The patch doesn't appear to include
any new or modified tests. Please justify why no new tests are needed for this
patch. Also please list what manual steps were performed to verify this patch.
|
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 14m 38s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 36m 11s | | trunk passed |
| +1 :green_heart: | compile | 18m 48s | | trunk passed with JDK
Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04 |
| +1 :green_heart: | compile | 16m 55s | | trunk passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | checkstyle | 4m 36s | | trunk passed |
| +1 :green_heart: | mvnsite | 20m 47s | | trunk passed |
| +1 :green_heart: | javadoc | 8m 55s | | trunk passed with JDK
Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04 |
| +1 :green_heart: | javadoc | 7m 39s | | trunk passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +0 :ok: | spotbugs | 0m 17s | | branch/hadoop-project no spotbugs
output file (spotbugsXml.xml) |
| +0 :ok: | spotbugs | 0m 19s | |
branch/hadoop-client-modules/hadoop-client no spotbugs output file
(spotbugsXml.xml) |
| +1 :green_heart: | shadedclient | 68m 48s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 41s | | Maven dependency ordering for patch |
| -1 :x: | mvninstall | 0m 57s |
[/patch-mvninstall-hadoop-common-project_hadoop-common.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6057/1/artifact/out/patch-mvninstall-hadoop-common-project_hadoop-common.txt)
| hadoop-common in the patch failed. |
| -1 :x: | mvninstall | 0m 24s |
[/patch-mvninstall-hadoop-hdfs-project_hadoop-hdfs-client.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6057/1/artifact/out/patch-mvninstall-hadoop-hdfs-project_hadoop-hdfs-client.txt)
| hadoop-hdfs-client in the patch failed. |
| -1 :x: | mvninstall | 0m 16s |
[/patch-mvninstall-hadoop-hdfs-project_hadoop-hdfs-httpfs.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6057/1/artifact/out/patch-mvninstall-hadoop-hdfs-project_hadoop-hdfs-httpfs.txt)
| hadoop-hdfs-httpfs in the patch failed. |
| -1 :x: | mvninstall | 0m 17s |
[/patch-mvninstall-hadoop-cloud-storage-project_hadoop-huaweicloud.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6057/1/artifact/out/patch-mvninstall-hadoop-cloud-storage-project_hadoop-huaweicloud.txt)
| hadoop-huaweicloud in the patch failed. |
| -1 :x: | mvninstall | 1m 17s |
[/patch-mvninstall-root.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6057/1/artifact/out/patch-mvninstall-root.txt)
| root in the patch failed. |
| +1 :green_heart: | compile | 17m 57s | | the patch passed with JDK
Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04 |
| +1 :green_heart: | javac | 17m 57s | | the patch passed |
| +1 :green_heart: | compile | 16m 53s | | the patch passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | javac | 16m 53s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| -0 :warning: | checkstyle | 5m 56s |
[/results-checkstyle-root.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6057/1/artifact/out/results-checkstyle-root.txt)
| root: The patch generated 2 new + 0 unchanged - 0 fixed = 2 total (was 0) |
| +1 :green_heart: | mvnsite | 15m 50s | | the patch passed |
| +1 :green_heart: | shellcheck | 0m 0s | | No new issues. |
| +1 :green_heart: | javadoc | 8m 52s | | the patch passed with JDK
Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04 |
| +1 :green_heart: | javadoc | 7m 31s | | the patch passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +0 :ok: | spotbugs | 0m 17s | | hadoop-project has no data from
[GitHub] [hadoop] pjfanning commented on pull request #5786: YARN-11498 add exclusion for jettison everywhere jersey-json is loaded
pjfanning commented on PR #5786: URL: https://github.com/apache/hadoop/pull/5786#issuecomment-1717752218 @slfan1989 @steveloughran Would it be possible to merge this? If #5623 is useful, it seems that we should be consistent and apply the exclusion everywhere jersey-json is used. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #6055: YARN-11547. [Federation] Router Supports Remove individual application records from FederationStateStore.
hadoop-yetus commented on PR #6055:
URL: https://github.com/apache/hadoop/pull/6055#issuecomment-1717716152
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 29s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 1 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 14m 25s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 20m 17s | | trunk passed |
| +1 :green_heart: | compile | 1m 50s | | trunk passed with JDK
Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04 |
| +1 :green_heart: | compile | 1m 31s | | trunk passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | checkstyle | 0m 51s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 0s | | trunk passed |
| +1 :green_heart: | javadoc | 1m 1s | | trunk passed with JDK
Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04 |
| +1 :green_heart: | javadoc | 0m 54s | | trunk passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | spotbugs | 1m 42s | | trunk passed |
| +1 :green_heart: | shadedclient | 20m 54s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 25s | | Maven dependency ordering for patch |
| -1 :x: | mvninstall | 0m 18s |
[/patch-mvninstall-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-router.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6055/6/artifact/out/patch-mvninstall-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-router.txt)
| hadoop-yarn-server-router in the patch failed. |
| -1 :x: | compile | 1m 34s |
[/patch-compile-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server-jdkUbuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6055/6/artifact/out/patch-compile-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server-jdkUbuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04.txt)
| hadoop-yarn-server in the patch failed with JDK
Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04. |
| -1 :x: | javac | 1m 34s |
[/patch-compile-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server-jdkUbuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6055/6/artifact/out/patch-compile-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server-jdkUbuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04.txt)
| hadoop-yarn-server in the patch failed with JDK
Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04. |
| -1 :x: | compile | 1m 23s |
[/patch-compile-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server-jdkPrivateBuild-1.8.0_382-8u382-ga-1~20.04.1-b05.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6055/6/artifact/out/patch-compile-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server-jdkPrivateBuild-1.8.0_382-8u382-ga-1~20.04.1-b05.txt)
| hadoop-yarn-server in the patch failed with JDK Private
Build-1.8.0_382-8u382-ga-1~20.04.1-b05. |
| -1 :x: | javac | 1m 23s |
[/patch-compile-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server-jdkPrivateBuild-1.8.0_382-8u382-ga-1~20.04.1-b05.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6055/6/artifact/out/patch-compile-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server-jdkPrivateBuild-1.8.0_382-8u382-ga-1~20.04.1-b05.txt)
| hadoop-yarn-server in the patch failed with JDK Private
Build-1.8.0_382-8u382-ga-1~20.04.1-b05. |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 0m 42s | | the patch passed |
| -1 :x: | mvnsite | 0m 20s |
[/patch-mvnsite-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-router.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6055/6/artifact/out/patch-mvnsite-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-router.txt)
| hadoop-yarn-server-router in the patch failed. |
| +1 :green_heart: | javadoc | 0m 45s | | the patch passed with JDK
Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04 |
| +1 :green_heart: | javadoc | 0m 42s | | the patch passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| -1 :x: | spotbugs | 0m 19s |
[jira] [Commented] (HADOOP-18873) ABFS: AbfsOutputStream doesnt close DataBlocks object.
[ https://issues.apache.org/jira/browse/HADOOP-18873?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17764693#comment-17764693 ] ASF GitHub Bot commented on HADOOP-18873: - saxenapranav commented on PR #6010: URL: https://github.com/apache/hadoop/pull/6010#issuecomment-1717686794 Thanks @mehakmeet for the review. I have taken the comments. Thank you. > ABFS: AbfsOutputStream doesnt close DataBlocks object. > -- > > Key: HADOOP-18873 > URL: https://issues.apache.org/jira/browse/HADOOP-18873 > Project: Hadoop Common > Issue Type: Sub-task >Affects Versions: 3.3.4 >Reporter: Pranav Saxena >Assignee: Pranav Saxena >Priority: Major > Labels: pull-request-available > Fix For: 3.3.4 > > > AbfsOutputStream doesnt close the dataBlock object created for the upload. > What is the implication of not doing that: > DataBlocks has three implementations: > # ByteArrayBlock > ## This creates an object of DataBlockByteArrayOutputStream (child of > ByteArrayOutputStream: wrapper arround byte-arrray for populating, reading > the array. > ## This gets GCed. > # ByteBufferBlock: > ## There is a defined *DirectBufferPool* from which it tries to request the > directBuffer. > ## If nothing in the pool, a new directBuffer is created. > ## the `close` method on the this object has the responsiblity of returning > back the buffer to pool so it can be reused. > ## Since we are not calling the `close`: > ### The pool is rendered of less use, since each request creates a new > directBuffer from memory. > ### All the object can be GCed and the direct-memory allocated may be > returned on the GC. What if the process crashes, the memory never goes back > and cause memory issue on the machine. > # DiskBlock: > ## This creates a file on disk on which the data-to-upload is written. This > file gets deleted in startUpload().close(). > > startUpload() gives an object of BlockUploadData which gives method of > `toByteArray()` which is used in abfsOutputStream to get the byteArray in the > dataBlock. > > Method which uses the DataBlock object: > https://github.com/apache/hadoop/blob/fac7d26c5d7f791565cc3ab45d079e2cca725f95/hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsOutputStream.java#L298 -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] saxenapranav commented on pull request #6010: HADOOP-18873. ABFS: AbfsOutputStream doesnt close DataBlocks object.
saxenapranav commented on PR #6010: URL: https://github.com/apache/hadoop/pull/6010#issuecomment-1717686794 Thanks @mehakmeet for the review. I have taken the comments. Thank you. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-18873) ABFS: AbfsOutputStream doesnt close DataBlocks object.
[ https://issues.apache.org/jira/browse/HADOOP-18873?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HADOOP-18873: Labels: pull-request-available (was: ) > ABFS: AbfsOutputStream doesnt close DataBlocks object. > -- > > Key: HADOOP-18873 > URL: https://issues.apache.org/jira/browse/HADOOP-18873 > Project: Hadoop Common > Issue Type: Sub-task >Affects Versions: 3.3.4 >Reporter: Pranav Saxena >Assignee: Pranav Saxena >Priority: Major > Labels: pull-request-available > Fix For: 3.3.4 > > > AbfsOutputStream doesnt close the dataBlock object created for the upload. > What is the implication of not doing that: > DataBlocks has three implementations: > # ByteArrayBlock > ## This creates an object of DataBlockByteArrayOutputStream (child of > ByteArrayOutputStream: wrapper arround byte-arrray for populating, reading > the array. > ## This gets GCed. > # ByteBufferBlock: > ## There is a defined *DirectBufferPool* from which it tries to request the > directBuffer. > ## If nothing in the pool, a new directBuffer is created. > ## the `close` method on the this object has the responsiblity of returning > back the buffer to pool so it can be reused. > ## Since we are not calling the `close`: > ### The pool is rendered of less use, since each request creates a new > directBuffer from memory. > ### All the object can be GCed and the direct-memory allocated may be > returned on the GC. What if the process crashes, the memory never goes back > and cause memory issue on the machine. > # DiskBlock: > ## This creates a file on disk on which the data-to-upload is written. This > file gets deleted in startUpload().close(). > > startUpload() gives an object of BlockUploadData which gives method of > `toByteArray()` which is used in abfsOutputStream to get the byteArray in the > dataBlock. > > Method which uses the DataBlock object: > https://github.com/apache/hadoop/blob/fac7d26c5d7f791565cc3ab45d079e2cca725f95/hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsOutputStream.java#L298 -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18873) ABFS: AbfsOutputStream doesnt close DataBlocks object.
[
https://issues.apache.org/jira/browse/HADOOP-18873?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17764691#comment-17764691
]
ASF GitHub Bot commented on HADOOP-18873:
-
saxenapranav commented on PR #6010:
URL: https://github.com/apache/hadoop/pull/6010#issuecomment-1717685620
AGGREGATED TEST RESULT
HNS-OAuth
[INFO] Results:
[INFO]
[ERROR] Failures:
[ERROR]
TestAccountConfiguration.testConfigPropNotFound:386->testMissingConfigKey:399
Expected a
org.apache.hadoop.fs.azurebfs.contracts.exceptions.TokenAccessProviderException
to be thrown, but got the result: :
"org.apache.hadoop.fs.azurebfs.oauth2.ClientCredsTokenProvider"
[INFO]
[ERROR] Tests run: 141, Failures: 1, Errors: 0, Skipped: 5
[INFO] Results:
[INFO]
[ERROR] Errors:
[ERROR] ITestAzureBlobFileSystemLease.testAcquireRetry:329 » TestTimedOut
test timed o...
[INFO]
[ERROR] Tests run: 589, Failures: 0, Errors: 1, Skipped: 54
[INFO] Results:
[INFO]
[WARNING] Tests run: 339, Failures: 0, Errors: 0, Skipped: 41
HNS-SharedKey
[INFO] Results:
[INFO]
[ERROR] Failures:
[ERROR]
TestAccountConfiguration.testConfigPropNotFound:386->testMissingConfigKey:399
Expected a
org.apache.hadoop.fs.azurebfs.contracts.exceptions.TokenAccessProviderException
to be thrown, but got the result: :
"org.apache.hadoop.fs.azurebfs.oauth2.ClientCredsTokenProvider"
[INFO]
[ERROR] Tests run: 141, Failures: 1, Errors: 0, Skipped: 5
[INFO] Results:
[INFO]
[ERROR] Failures:
[ERROR]
ITestAzureBlobFileSystemRandomRead.testSkipBounds:218->Assert.assertTrue:42->Assert.fail:89
There should not be any network I/O (elapsedTimeMs=24).
[ERROR] Errors:
[ERROR] ITestAzureBlobFileSystemLease.testAcquireRetry:336 » TestTimedOut
test timed o...
[INFO]
[ERROR] Tests run: 589, Failures: 1, Errors: 1, Skipped: 54
[INFO] Results:
[INFO]
[WARNING] Tests run: 339, Failures: 0, Errors: 0, Skipped: 41
NonHNS-SharedKey
[INFO] Results:
[INFO]
[ERROR] Failures:
[ERROR]
TestAccountConfiguration.testConfigPropNotFound:386->testMissingConfigKey:399
Expected a
org.apache.hadoop.fs.azurebfs.contracts.exceptions.TokenAccessProviderException
to be thrown, but got the result: :
"org.apache.hadoop.fs.azurebfs.oauth2.ClientCredsTokenProvider"
[INFO]
[ERROR] Tests run: 141, Failures: 1, Errors: 0, Skipped: 11
[INFO] Results:
[INFO]
[ERROR] Errors:
[ERROR]
ITestAzureBlobFileSystemLease.testAcquireRetry:344->lambda$testAcquireRetry$6:345
» TestTimedOut
[INFO]
[ERROR] Tests run: 589, Failures: 0, Errors: 1, Skipped: 277
[INFO] Results:
[INFO]
[WARNING] Tests run: 339, Failures: 0, Errors: 0, Skipped: 44
AppendBlob-HNS-OAuth
[INFO] Results:
[INFO]
[ERROR] Failures:
[ERROR]
TestAccountConfiguration.testConfigPropNotFound:386->testMissingConfigKey:399
Expected a
org.apache.hadoop.fs.azurebfs.contracts.exceptions.TokenAccessProviderException
to be thrown, but got the result: :
"org.apache.hadoop.fs.azurebfs.oauth2.ClientCredsTokenProvider"
[INFO]
[ERROR] Tests run: 141, Failures: 1, Errors: 0, Skipped: 5
[INFO] Results:
[INFO]
[ERROR] Errors:
[ERROR]
ITestAzureBlobFileSystemLease.testAcquireRetry:344->lambda$testAcquireRetry$6:345
» TestTimedOut
[INFO]
[ERROR] Tests run: 589, Failures: 0, Errors: 1, Skipped: 54
[INFO] Results:
[INFO]
[WARNING] Tests run: 339, Failures: 0, Errors: 0, Skipped: 41
Time taken: 48 mins 26 secs.
azureuser@Hadoop-VM-EAST2:~/hadoop/hadoop-tools/hadoop-azure$ git log
commit d9da57b16ea00f4e79444a5109807ac6fef26e6c (HEAD -> HADOOP-18873,
origin/HADOOP-18873)
Author: Pranav Saxena <>
Date: Wed Sep 13 06:02:03 2023 -0700
close with IoUtils; test refactors;
> ABFS: AbfsOutputStream doesnt close DataBlocks object.
> --
>
> Key: HADOOP-18873
> URL: https://issues.apache.org/jira/browse/HADOOP-18873
> Project: Hadoop Common
> Issue Type: Sub-task
>Affects Versions: 3.3.4
>Reporter: Pranav Saxena
>Assignee: Pranav Saxena
>Priority: Major
> Fix For: 3.3.4
>
>
> AbfsOutputStream doesnt close the dataBlock object created for the upload.
> What is the implication of not doing that:
> DataBlocks has three implementations:
> # ByteArrayBlock
> ## This creates an object of DataBlockByteArrayOutputStream (child of
> ByteArrayOutputStream: wrapper arround byte-arrray for populating, reading
> the array.
> ## This gets GCed.
> # ByteBufferBlock:
> ## There is a defined *DirectBufferPool* from which it tries to request the
> directBuffer.
> ##
[GitHub] [hadoop] saxenapranav commented on pull request #6010: HADOOP-18873. ABFS: AbfsOutputStream doesnt close DataBlocks object.
saxenapranav commented on PR #6010:
URL: https://github.com/apache/hadoop/pull/6010#issuecomment-1717685620
AGGREGATED TEST RESULT
HNS-OAuth
[INFO] Results:
[INFO]
[ERROR] Failures:
[ERROR]
TestAccountConfiguration.testConfigPropNotFound:386->testMissingConfigKey:399
Expected a
org.apache.hadoop.fs.azurebfs.contracts.exceptions.TokenAccessProviderException
to be thrown, but got the result: :
"org.apache.hadoop.fs.azurebfs.oauth2.ClientCredsTokenProvider"
[INFO]
[ERROR] Tests run: 141, Failures: 1, Errors: 0, Skipped: 5
[INFO] Results:
[INFO]
[ERROR] Errors:
[ERROR] ITestAzureBlobFileSystemLease.testAcquireRetry:329 » TestTimedOut
test timed o...
[INFO]
[ERROR] Tests run: 589, Failures: 0, Errors: 1, Skipped: 54
[INFO] Results:
[INFO]
[WARNING] Tests run: 339, Failures: 0, Errors: 0, Skipped: 41
HNS-SharedKey
[INFO] Results:
[INFO]
[ERROR] Failures:
[ERROR]
TestAccountConfiguration.testConfigPropNotFound:386->testMissingConfigKey:399
Expected a
org.apache.hadoop.fs.azurebfs.contracts.exceptions.TokenAccessProviderException
to be thrown, but got the result: :
"org.apache.hadoop.fs.azurebfs.oauth2.ClientCredsTokenProvider"
[INFO]
[ERROR] Tests run: 141, Failures: 1, Errors: 0, Skipped: 5
[INFO] Results:
[INFO]
[ERROR] Failures:
[ERROR]
ITestAzureBlobFileSystemRandomRead.testSkipBounds:218->Assert.assertTrue:42->Assert.fail:89
There should not be any network I/O (elapsedTimeMs=24).
[ERROR] Errors:
[ERROR] ITestAzureBlobFileSystemLease.testAcquireRetry:336 » TestTimedOut
test timed o...
[INFO]
[ERROR] Tests run: 589, Failures: 1, Errors: 1, Skipped: 54
[INFO] Results:
[INFO]
[WARNING] Tests run: 339, Failures: 0, Errors: 0, Skipped: 41
NonHNS-SharedKey
[INFO] Results:
[INFO]
[ERROR] Failures:
[ERROR]
TestAccountConfiguration.testConfigPropNotFound:386->testMissingConfigKey:399
Expected a
org.apache.hadoop.fs.azurebfs.contracts.exceptions.TokenAccessProviderException
to be thrown, but got the result: :
"org.apache.hadoop.fs.azurebfs.oauth2.ClientCredsTokenProvider"
[INFO]
[ERROR] Tests run: 141, Failures: 1, Errors: 0, Skipped: 11
[INFO] Results:
[INFO]
[ERROR] Errors:
[ERROR]
ITestAzureBlobFileSystemLease.testAcquireRetry:344->lambda$testAcquireRetry$6:345
» TestTimedOut
[INFO]
[ERROR] Tests run: 589, Failures: 0, Errors: 1, Skipped: 277
[INFO] Results:
[INFO]
[WARNING] Tests run: 339, Failures: 0, Errors: 0, Skipped: 44
AppendBlob-HNS-OAuth
[INFO] Results:
[INFO]
[ERROR] Failures:
[ERROR]
TestAccountConfiguration.testConfigPropNotFound:386->testMissingConfigKey:399
Expected a
org.apache.hadoop.fs.azurebfs.contracts.exceptions.TokenAccessProviderException
to be thrown, but got the result: :
"org.apache.hadoop.fs.azurebfs.oauth2.ClientCredsTokenProvider"
[INFO]
[ERROR] Tests run: 141, Failures: 1, Errors: 0, Skipped: 5
[INFO] Results:
[INFO]
[ERROR] Errors:
[ERROR]
ITestAzureBlobFileSystemLease.testAcquireRetry:344->lambda$testAcquireRetry$6:345
» TestTimedOut
[INFO]
[ERROR] Tests run: 589, Failures: 0, Errors: 1, Skipped: 54
[INFO] Results:
[INFO]
[WARNING] Tests run: 339, Failures: 0, Errors: 0, Skipped: 41
Time taken: 48 mins 26 secs.
azureuser@Hadoop-VM-EAST2:~/hadoop/hadoop-tools/hadoop-azure$ git log
commit d9da57b16ea00f4e79444a5109807ac6fef26e6c (HEAD -> HADOOP-18873,
origin/HADOOP-18873)
Author: Pranav Saxena <>
Date: Wed Sep 13 06:02:03 2023 -0700
close with IoUtils; test refactors;
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #6055: YARN-11547. [Federation] Router Supports Remove individual application records from FederationStateStore.
hadoop-yetus commented on PR #6055:
URL: https://github.com/apache/hadoop/pull/6055#issuecomment-1717684333
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 28s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 1 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 14m 18s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 20m 18s | | trunk passed |
| +1 :green_heart: | compile | 1m 41s | | trunk passed with JDK
Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04 |
| +1 :green_heart: | compile | 1m 32s | | trunk passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | checkstyle | 0m 50s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 0s | | trunk passed |
| +1 :green_heart: | javadoc | 1m 1s | | trunk passed with JDK
Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04 |
| +1 :green_heart: | javadoc | 0m 54s | | trunk passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | spotbugs | 1m 41s | | trunk passed |
| +1 :green_heart: | shadedclient | 20m 58s | | branch has no errors
when building and testing our client artifacts. |
| -0 :warning: | patch | 21m 16s | | Used diff version of patch file.
Binary files and potentially other changes not applied. Please rebase and
squash commits if necessary. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 26s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 0m 42s | | the patch passed |
| +1 :green_heart: | compile | 1m 31s | | the patch passed with JDK
Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04 |
| +1 :green_heart: | javac | 1m 31s | | the patch passed |
| +1 :green_heart: | compile | 1m 25s | | the patch passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | javac | 1m 25s | | the patch passed |
| -1 :x: | blanks | 0m 0s |
[/blanks-eol.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6055/5/artifact/out/blanks-eol.txt)
| The patch has 1 line(s) that end in blanks. Use git apply --whitespace=fix
<>. Refer https://git-scm.com/docs/git-apply |
| -0 :warning: | checkstyle | 0m 42s |
[/results-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6055/5/artifact/out/results-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server.txt)
| hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server: The patch generated 6
new + 3 unchanged - 0 fixed = 9 total (was 3) |
| +1 :green_heart: | mvnsite | 0m 46s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 45s | | the patch passed with JDK
Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04 |
| +1 :green_heart: | javadoc | 0m 42s | | the patch passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | spotbugs | 1m 40s | | the patch passed |
| +1 :green_heart: | shadedclient | 21m 8s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 2m 36s | | hadoop-yarn-server-common in
the patch passed. |
| +1 :green_heart: | unit | 0m 23s | | hadoop-yarn-server-router in
the patch passed. |
| +1 :green_heart: | asflicense | 0m 29s | | The patch does not
generate ASF License warnings. |
| | | 101m 23s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6055/5/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/6055 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux fc2f0fd8842b 4.15.0-213-generic #224-Ubuntu SMP Mon Jun 19
13:30:12 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 010ca8009cc8e29891451ad8b60fb4af8ce3319c |
| Default Java | Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| Multi-JDK versions |
[GitHub] [hadoop] hadoop-yetus commented on pull request #6060: HADOOP-18894: upgrade sshd-core due to CVEs
hadoop-yetus commented on PR #6060:
URL: https://github.com/apache/hadoop/pull/6060#issuecomment-1717653894
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 47s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +0 :ok: | xmllint | 0m 0s | | xmllint was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| -1 :x: | test4tests | 0m 0s | | The patch doesn't appear to include
any new or modified tests. Please justify why no new tests are needed for this
patch. Also please list what manual steps were performed to verify this patch.
|
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 48m 23s | | trunk passed |
| +1 :green_heart: | compile | 0m 18s | | trunk passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | compile | 0m 19s | | trunk passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | mvnsite | 0m 23s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 23s | | trunk passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 0m 20s | | trunk passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | shadedclient | 87m 39s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 13s | | the patch passed |
| +1 :green_heart: | compile | 0m 13s | | the patch passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javac | 0m 13s | | the patch passed |
| +1 :green_heart: | compile | 0m 12s | | the patch passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | javac | 0m 12s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | mvnsite | 0m 15s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 13s | | the patch passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 0m 12s | | the patch passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| -1 :x: | shadedclient | 3m 41s | | patch has errors when building
and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 0m 13s | | hadoop-project in the patch
passed. |
| +1 :green_heart: | asflicense | 0m 28s | | The patch does not
generate ASF License warnings. |
| | | 97m 39s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6060/1/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/6060 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient codespell detsecrets xmllint |
| uname | Linux 793b25e51f87 4.15.0-212-generic #223-Ubuntu SMP Tue May 23
13:09:22 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / ee3284cee15a6aad5cb1c1f2579d3add169e08a8 |
| Default Java | Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6060/1/testReport/ |
| Max. process+thread count | 585 (vs. ulimit of 5500) |
| modules | C: hadoop-project U: hadoop-project |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6060/1/console |
| versions | git=2.25.1 maven=3.6.3 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For
[jira] [Commented] (HADOOP-18894) upgrade sshd-core due to CVEs
[
https://issues.apache.org/jira/browse/HADOOP-18894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17764681#comment-17764681
]
ASF GitHub Bot commented on HADOOP-18894:
-
hadoop-yetus commented on PR #6060:
URL: https://github.com/apache/hadoop/pull/6060#issuecomment-1717653894
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 47s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +0 :ok: | xmllint | 0m 0s | | xmllint was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| -1 :x: | test4tests | 0m 0s | | The patch doesn't appear to include
any new or modified tests. Please justify why no new tests are needed for this
patch. Also please list what manual steps were performed to verify this patch.
|
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 48m 23s | | trunk passed |
| +1 :green_heart: | compile | 0m 18s | | trunk passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | compile | 0m 19s | | trunk passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | mvnsite | 0m 23s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 23s | | trunk passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 0m 20s | | trunk passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | shadedclient | 87m 39s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 13s | | the patch passed |
| +1 :green_heart: | compile | 0m 13s | | the patch passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javac | 0m 13s | | the patch passed |
| +1 :green_heart: | compile | 0m 12s | | the patch passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | javac | 0m 12s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | mvnsite | 0m 15s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 13s | | the patch passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 0m 12s | | the patch passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| -1 :x: | shadedclient | 3m 41s | | patch has errors when building
and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 0m 13s | | hadoop-project in the patch
passed. |
| +1 :green_heart: | asflicense | 0m 28s | | The patch does not
generate ASF License warnings. |
| | | 97m 39s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6060/1/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/6060 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient codespell detsecrets xmllint |
| uname | Linux 793b25e51f87 4.15.0-212-generic #223-Ubuntu SMP Tue May 23
13:09:22 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / ee3284cee15a6aad5cb1c1f2579d3add169e08a8 |
| Default Java | Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6060/1/testReport/ |
| Max. process+thread count | 585 (vs. ulimit of 5500) |
| modules | C: hadoop-project U: hadoop-project |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6060/1/console |
| versions | git=2.25.1 maven=3.6.3 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was automatically generated.
> upgrade sshd-core due to CVEs
> -
>
> Key: HADOOP-18894
> URL: https://issues.apache.org/jira/browse/HADOOP-18894
> Project: Hadoop Common
>
[GitHub] [hadoop] hadoop-yetus commented on pull request #5943: [Testing] Fix shaded client for building Hadoop on Windows
hadoop-yetus commented on PR #5943: URL: https://github.com/apache/hadoop/pull/5943#issuecomment-1717653735 (!) A patch to the testing environment has been detected. Re-executing against the patched versions to perform further tests. The console is at https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5943/3/console in case of problems. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #6055: YARN-11547. [Federation] Router Supports Remove individual application records from FederationStateStore.
hadoop-yetus commented on PR #6055:
URL: https://github.com/apache/hadoop/pull/6055#issuecomment-1717652629
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 37s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 1 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 14m 26s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 31m 27s | | trunk passed |
| +1 :green_heart: | compile | 2m 35s | | trunk passed with JDK
Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04 |
| +1 :green_heart: | compile | 2m 21s | | trunk passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | checkstyle | 1m 24s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 25s | | trunk passed |
| +1 :green_heart: | javadoc | 1m 25s | | trunk passed with JDK
Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04 |
| +1 :green_heart: | javadoc | 1m 15s | | trunk passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | spotbugs | 2m 29s | | trunk passed |
| +1 :green_heart: | shadedclient | 33m 27s | | branch has no errors
when building and testing our client artifacts. |
| -0 :warning: | patch | 33m 51s | | Used diff version of patch file.
Binary files and potentially other changes not applied. Please rebase and
squash commits if necessary. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 32s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 0m 58s | | the patch passed |
| +1 :green_heart: | compile | 2m 22s | | the patch passed with JDK
Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04 |
| +1 :green_heart: | javac | 2m 22s | | the patch passed |
| +1 :green_heart: | compile | 2m 15s | | the patch passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | javac | 2m 15s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| -0 :warning: | checkstyle | 1m 13s |
[/results-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6055/3/artifact/out/results-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server.txt)
| hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server: The patch generated 6
new + 3 unchanged - 0 fixed = 9 total (was 3) |
| +1 :green_heart: | mvnsite | 1m 6s | | the patch passed |
| +1 :green_heart: | javadoc | 1m 3s | | the patch passed with JDK
Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04 |
| +1 :green_heart: | javadoc | 0m 59s | | the patch passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | spotbugs | 2m 29s | | the patch passed |
| +1 :green_heart: | shadedclient | 33m 35s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 3m 27s | | hadoop-yarn-server-common in
the patch passed. |
| +1 :green_heart: | unit | 0m 32s | | hadoop-yarn-server-router in
the patch passed. |
| +1 :green_heart: | asflicense | 0m 43s | | The patch does not
generate ASF License warnings. |
| | | 149m 29s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6055/3/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/6055 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux 270e16f707ac 4.15.0-212-generic #223-Ubuntu SMP Tue May 23
13:09:22 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 339aa7c85f692336cc7e51051e6070a470106a59 |
| Default Java | Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| Test Results |
[GitHub] [hadoop] Hexiaoqiao commented on pull request #6001: HADOOP-18851: Performance improvement for DelegationTokenSecretManager.
Hexiaoqiao commented on PR #6001: URL: https://github.com/apache/hadoop/pull/6001#issuecomment-1717644318 It is OK for me. Please fix the checkstyle as Yetus reported before approve and checkin. Thanks. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18851) AbstractDelegationTokenSecretManager- Performance improvement by optimising the synchronization context
[
https://issues.apache.org/jira/browse/HADOOP-18851?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17764678#comment-17764678
]
ASF GitHub Bot commented on HADOOP-18851:
-
Hexiaoqiao commented on PR #6001:
URL: https://github.com/apache/hadoop/pull/6001#issuecomment-1717644318
It is OK for me. Please fix the checkstyle as Yetus reported before approve
and checkin. Thanks.
> AbstractDelegationTokenSecretManager- Performance improvement by optimising
> the synchronization context
> ---
>
> Key: HADOOP-18851
> URL: https://issues.apache.org/jira/browse/HADOOP-18851
> Project: Hadoop Common
> Issue Type: Task
>Reporter: Vikas Kumar
>Assignee: Vikas Kumar
>Priority: Major
> Labels: pull-request-available
> Attachments:
> 0001-HADOOP-18851-Perfm-improvement-for-ZKDT-management.patch, Screenshot
> 2023-08-16 at 5.36.57 PM.png
>
>
> *Context:*
> KMS depends on hadoop-common for DT management. Recently we were analysing
> one performance issue and following is out findings:
> # Around 96% (196 out of 200) KMS container threads were in BLOCKED state at
> following:
> ## *AbstractDelegationTokenSecretManager.verifyToken()*
> ## *AbstractDelegationTokenSecretManager.createPassword()*
> # And then process crashed.
>
> {code:java}
> http-nio-9292-exec-200PRIORITY : 5THREAD ID : 0X7F075C157800NATIVE ID :
> 0X2C87FNATIVE ID (DECIMAL) : 182399STATE : BLOCKED
> stackTrace:
> java.lang.Thread.State: BLOCKED (on object monitor)
> at
> org.apache.hadoop.security.token.delegation.AbstractDelegationTokenSecretManager.verifyToken(AbstractDelegationTokenSecretManager.java:474)
> - waiting to lock <0x0005f2f545e8> (a
> org.apache.hadoop.security.token.delegation.web.DelegationTokenManager$ZKSecretManager)
> at
> org.apache.hadoop.security.token.delegation.web.DelegationTokenManager.verifyToken(DelegationTokenManager.java:213)
> at
> org.apache.hadoop.security.token.delegation.web.DelegationTokenAuthenticationHandler.authenticate(DelegationTokenAuthenticationHandler.java:396)
> at {code}
> All the 199 out of 200 were blocked at above point.
> And the lock they are waiting for is acquired by a thread that was trying to
> createPassword and publishing the same on ZK.
>
> {code:java}
> stackTrace:
> java.lang.Thread.State: WAITING (on object monitor)
> at java.lang.Object.wait(Native Method)
> at java.lang.Object.wait(Object.java:502)
> at org.apache.zookeeper.ClientCnxn.submitRequest(ClientCnxn.java:1598)
> - locked <0x000749263ec0> (a org.apache.zookeeper.ClientCnxn$Packet)
> at org.apache.zookeeper.ClientCnxn.submitRequest(ClientCnxn.java:1570)
> at org.apache.zookeeper.ZooKeeper.setData(ZooKeeper.java:2235)
> at
> org.apache.curator.framework.imps.SetDataBuilderImpl$7.call(SetDataBuilderImpl.java:398)
> at
> org.apache.curator.framework.imps.SetDataBuilderImpl$7.call(SetDataBuilderImpl.java:385)
> at org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:93)
> at
> org.apache.curator.framework.imps.SetDataBuilderImpl.pathInForeground(SetDataBuilderImpl.java:382)
> at
> org.apache.curator.framework.imps.SetDataBuilderImpl.forPath(SetDataBuilderImpl.java:358)
> at
> org.apache.curator.framework.imps.SetDataBuilderImpl.forPath(SetDataBuilderImpl.java:36)
> at
> org.apache.curator.framework.recipes.shared.SharedValue.trySetValue(SharedValue.java:201)
> at
> org.apache.curator.framework.recipes.shared.SharedCount.trySetCount(SharedCount.java:116)
> at
> org.apache.hadoop.security.token.delegation.ZKDelegationTokenSecretManager.incrSharedCount(ZKDelegationTokenSecretManager.java:586)
> at
> org.apache.hadoop.security.token.delegation.ZKDelegationTokenSecretManager.incrementDelegationTokenSeqNum(ZKDelegationTokenSecretManager.java:601)
> at
> org.apache.hadoop.security.token.delegation.AbstractDelegationTokenSecretManager.createPassword(AbstractDelegationTokenSecretManager.java:402)
> - locked <0x0005f2f545e8> (a
> org.apache.hadoop.security.token.delegation.web.DelegationTokenManager$ZKSecretManager)
> at
> org.apache.hadoop.security.token.delegation.AbstractDelegationTokenSecretManager.createPassword(AbstractDelegationTokenSecretManager.java:48)
> at org.apache.hadoop.security.token.Token.(Token.java:67)
> at
> org.apache.hadoop.security.token.delegation.web.DelegationTokenManager.createToken(DelegationTokenManager.java:183)
> {code}
> We can say that this thread is slow and has blocked remaining all. But
> following is my observation:
>
> # verifyToken() and createPaswword() has been synchronized because one is
> reading the tokenMap and another is updating the map. If it's only to protect
> the map, then can't we simply use ConcurrentHashMap and remove the
> "synchronized" keyword.
[jira] [Assigned] (HADOOP-18889) S3A: V2 SDK client does not work with third-party store
[ https://issues.apache.org/jira/browse/HADOOP-18889?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ahmar Suhail reassigned HADOOP-18889: - Assignee: Ahmar Suhail > S3A: V2 SDK client does not work with third-party store > --- > > Key: HADOOP-18889 > URL: https://issues.apache.org/jira/browse/HADOOP-18889 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/s3 >Affects Versions: 3.4.0 >Reporter: Steve Loughran >Assignee: Ahmar Suhail >Priority: Critical > > testing against an external store without specifying region now blows up > because the region is queried off eu-west-1. > What are we do to here? require the region setting *which wasn't needed > before? what even region do we provide for third party stores? -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18889) S3A: V2 SDK client does not work with third-party store
[ https://issues.apache.org/jira/browse/HADOOP-18889?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17764664#comment-17764664 ] Ahmar Suhail commented on HADOOP-18889: --- we can probably get rid of the region query now..as SDK V2 has cross region support as of v2.20.99, it wasn't there when we started this work. > S3A: V2 SDK client does not work with third-party store > --- > > Key: HADOOP-18889 > URL: https://issues.apache.org/jira/browse/HADOOP-18889 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/s3 >Affects Versions: 3.4.0 >Reporter: Steve Loughran >Priority: Critical > > testing against an external store without specifying region now blows up > because the region is queried off eu-west-1. > What are we do to here? require the region setting *which wasn't needed > before? what even region do we provide for third party stores? -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18893) Make Trash Policy pluggable for different FileSystems
[ https://issues.apache.org/jira/browse/HADOOP-18893?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17764659#comment-17764659 ] ASF GitHub Bot commented on HADOOP-18893: - mehakmeet commented on PR #6061: URL: https://github.com/apache/hadoop/pull/6061#issuecomment-1717593662 Putting up an initial PR for Trash policy separation. Need to verify this in a cluster still, but have added a small test for S3A and ABFS. HDFS has some good tests for Trash too. > Make Trash Policy pluggable for different FileSystems > - > > Key: HADOOP-18893 > URL: https://issues.apache.org/jira/browse/HADOOP-18893 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs, fs/azure, fs/s3 >Reporter: Mehakmeet Singh >Assignee: Mehakmeet Singh >Priority: Major > Labels: pull-request-available > > Add capability for Trash Policy to be pluggable using a property leveraging > the schema of different filesystems. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] mehakmeet commented on pull request #6061: HADOOP-18893. Make Trash Policy pluggable for different FileSystems
mehakmeet commented on PR #6061: URL: https://github.com/apache/hadoop/pull/6061#issuecomment-1717593662 Putting up an initial PR for Trash policy separation. Need to verify this in a cluster still, but have added a small test for S3A and ABFS. HDFS has some good tests for Trash too. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-18893) Make Trash Policy pluggable for different FileSystems
[ https://issues.apache.org/jira/browse/HADOOP-18893?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HADOOP-18893: Labels: pull-request-available (was: ) > Make Trash Policy pluggable for different FileSystems > - > > Key: HADOOP-18893 > URL: https://issues.apache.org/jira/browse/HADOOP-18893 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs, fs/azure, fs/s3 >Reporter: Mehakmeet Singh >Assignee: Mehakmeet Singh >Priority: Major > Labels: pull-request-available > > Add capability for Trash Policy to be pluggable using a property leveraging > the schema of different filesystems. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18893) Make Trash Policy pluggable for different FileSystems
[ https://issues.apache.org/jira/browse/HADOOP-18893?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17764657#comment-17764657 ] ASF GitHub Bot commented on HADOOP-18893: - mehakmeet opened a new pull request, #6061: URL: https://github.com/apache/hadoop/pull/6061 ### Description of PR Follow up from #4729 ### How was this patch tested? ### For code changes: - [ ] Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')? - [ ] Object storage: have the integration tests been executed and the endpoint declared according to the connector-specific documentation? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the `LICENSE`, `LICENSE-binary`, `NOTICE-binary` files? > Make Trash Policy pluggable for different FileSystems > - > > Key: HADOOP-18893 > URL: https://issues.apache.org/jira/browse/HADOOP-18893 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs, fs/azure, fs/s3 >Reporter: Mehakmeet Singh >Assignee: Mehakmeet Singh >Priority: Major > > Add capability for Trash Policy to be pluggable using a property leveraging > the schema of different filesystems. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] mehakmeet opened a new pull request, #6061: HADOOP-18893. Make Trash Policy pluggable for different FileSystems
mehakmeet opened a new pull request, #6061: URL: https://github.com/apache/hadoop/pull/6061 ### Description of PR Follow up from #4729 ### How was this patch tested? ### For code changes: - [ ] Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')? - [ ] Object storage: have the integration tests been executed and the endpoint declared according to the connector-specific documentation? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the `LICENSE`, `LICENSE-binary`, `NOTICE-binary` files? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18892) compile failure due to nodejs is incompatible with module
[
https://issues.apache.org/jira/browse/HADOOP-18892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17764651#comment-17764651
]
ASF GitHub Bot commented on HADOOP-18892:
-
hadoop-yetus commented on PR #6059:
URL: https://github.com/apache/hadoop/pull/6059#issuecomment-1717571950
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 9m 41s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +0 :ok: | xmllint | 0m 0s | | xmllint was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| -1 :x: | test4tests | 0m 0s | | The patch doesn't appear to include
any new or modified tests. Please justify why no new tests are needed for this
patch. Also please list what manual steps were performed to verify this patch.
|
_ branch-3.3.6 Compile Tests _ |
| -1 :x: | mvninstall | 35m 19s |
[/branch-mvninstall-root.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6059/1/artifact/out/branch-mvninstall-root.txt)
| root in branch-3.3.6 failed. |
| +1 :green_heart: | compile | 0m 24s | | branch-3.3.6 passed |
| +1 :green_heart: | mvnsite | 0m 35s | | branch-3.3.6 passed |
| +1 :green_heart: | javadoc | 0m 29s | | branch-3.3.6 passed |
| -1 :x: | shadedclient | 38m 49s | | branch has errors when building
and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 17s | | the patch passed |
| -1 :x: | compile | 0m 10s |
[/patch-compile-hadoop-project.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6059/1/artifact/out/patch-compile-hadoop-project.txt)
| hadoop-project in the patch failed. |
| -1 :x: | javac | 0m 10s |
[/patch-compile-hadoop-project.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6059/1/artifact/out/patch-compile-hadoop-project.txt)
| hadoop-project in the patch failed. |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | mvnsite | 0m 20s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 17s | | the patch passed |
| -1 :x: | shadedclient | 10m 36s | | patch has errors when building
and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 0m 17s | | hadoop-project in the patch
passed. |
| +1 :green_heart: | asflicense | 0m 36s | | The patch does not
generate ASF License warnings. |
| | | 65m 42s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6059/1/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/6059 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient codespell detsecrets xmllint |
| uname | Linux 90272e67f1a2 4.15.0-212-generic #223-Ubuntu SMP Tue May 23
13:09:22 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | branch-3.3.6 / 06dbcb9f6e7cd1796195ed44a2b7b0818b94a420 |
| Default Java | Private Build-1.8.0_362-8u372-ga~us1-0ubuntu1~18.04-b09 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6059/1/testReport/ |
| Max. process+thread count | 94 (vs. ulimit of 5500) |
| modules | C: hadoop-project U: hadoop-project |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6059/1/console |
| versions | git=2.17.1 maven=3.6.0 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was automatically generated.
> compile failure due to nodejs is incompatible with module
> -
>
> Key: HADOOP-18892
> URL: https://issues.apache.org/jira/browse/HADOOP-18892
> Project: Hadoop Common
> Issue Type: Bug
> Components: build
>Affects Versions: 3.3.6
> Environment: hadoop 3.3.6
>Reporter: Authur Wang
>Priority: Critical
> Labels: pull-request-available
>
> hadoop compile error due to the following error:
> [INFO] Running 'yarn ' in
>
[GitHub] [hadoop] hadoop-yetus commented on pull request #6059: HADOOP-18892 fix Catalog Webapp module compile errors due to low nodejs version
hadoop-yetus commented on PR #6059:
URL: https://github.com/apache/hadoop/pull/6059#issuecomment-1717571950
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 9m 41s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +0 :ok: | xmllint | 0m 0s | | xmllint was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| -1 :x: | test4tests | 0m 0s | | The patch doesn't appear to include
any new or modified tests. Please justify why no new tests are needed for this
patch. Also please list what manual steps were performed to verify this patch.
|
_ branch-3.3.6 Compile Tests _ |
| -1 :x: | mvninstall | 35m 19s |
[/branch-mvninstall-root.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6059/1/artifact/out/branch-mvninstall-root.txt)
| root in branch-3.3.6 failed. |
| +1 :green_heart: | compile | 0m 24s | | branch-3.3.6 passed |
| +1 :green_heart: | mvnsite | 0m 35s | | branch-3.3.6 passed |
| +1 :green_heart: | javadoc | 0m 29s | | branch-3.3.6 passed |
| -1 :x: | shadedclient | 38m 49s | | branch has errors when building
and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 17s | | the patch passed |
| -1 :x: | compile | 0m 10s |
[/patch-compile-hadoop-project.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6059/1/artifact/out/patch-compile-hadoop-project.txt)
| hadoop-project in the patch failed. |
| -1 :x: | javac | 0m 10s |
[/patch-compile-hadoop-project.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6059/1/artifact/out/patch-compile-hadoop-project.txt)
| hadoop-project in the patch failed. |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | mvnsite | 0m 20s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 17s | | the patch passed |
| -1 :x: | shadedclient | 10m 36s | | patch has errors when building
and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 0m 17s | | hadoop-project in the patch
passed. |
| +1 :green_heart: | asflicense | 0m 36s | | The patch does not
generate ASF License warnings. |
| | | 65m 42s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6059/1/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/6059 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient codespell detsecrets xmllint |
| uname | Linux 90272e67f1a2 4.15.0-212-generic #223-Ubuntu SMP Tue May 23
13:09:22 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | branch-3.3.6 / 06dbcb9f6e7cd1796195ed44a2b7b0818b94a420 |
| Default Java | Private Build-1.8.0_362-8u372-ga~us1-0ubuntu1~18.04-b09 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6059/1/testReport/ |
| Max. process+thread count | 94 (vs. ulimit of 5500) |
| modules | C: hadoop-project U: hadoop-project |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6059/1/console |
| versions | git=2.17.1 maven=3.6.0 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] slfan1989 commented on a diff in pull request #6055: YARN-11547. [Federation] Router Supports Remove individual application records from FederationStateStore.
slfan1989 commented on code in PR #6055: URL: https://github.com/apache/hadoop/pull/6055#discussion_r1324441005 ## hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router/src/main/java/org/apache/hadoop/yarn/server/router/Router.java: ## @@ -36,13 +37,17 @@ import org.apache.hadoop.security.SecurityUtil; import org.apache.hadoop.security.UserGroupInformation; import org.apache.hadoop.service.CompositeService; -import org.apache.hadoop.util.JvmPauseMonitor; -import org.apache.hadoop.util.ShutdownHookManager; -import org.apache.hadoop.util.StringUtils; -import org.apache.hadoop.util.VersionInfo; +import org.apache.hadoop.util.*; Review Comment: Thank you very much for your help in reviewing the code! I will improve this part of the code. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] slfan1989 commented on a diff in pull request #6055: YARN-11547. [Federation] Router Supports Remove individual application records from FederationStateStore.
slfan1989 commented on code in PR #6055:
URL: https://github.com/apache/hadoop/pull/6055#discussion_r1324441622
##
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router/src/main/java/org/apache/hadoop/yarn/server/router/Router.java:
##
@@ -291,22 +296,30 @@ public static String getProxyHostAndPort(Configuration
conf) {
public static void main(String[] argv) {
Configuration conf = new YarnConfiguration();
Thread
Review Comment:
I will fix it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] GauthamBanasandra commented on a diff in pull request #6032: Removing Trailing Spaces in Java Files
GauthamBanasandra commented on code in PR #6032: URL: https://github.com/apache/hadoop/pull/6032#discussion_r1324423739 ## Blanks_eol.py: ## @@ -0,0 +1,23 @@ +import os +import re + +def trim_right_spaces(text): +return text.rstrip() + +def main(): Review Comment: You need to add a docstring for every function that you create. You may follow [this](https://github.com/apache/hadoop/blob/9c8cdbe28e317e703560c3eefcde2f6eebcabb17/dev-support/docker/pkg-resolver/check_platform.py#L28-L30). ## Blanks_eol.py: ## @@ -0,0 +1,23 @@ +import os Review Comment: @Prajwal-Surendra could you please add the license header as done [here](https://github.com/apache/hadoop/blob/9c8cdbe28e317e703560c3eefcde2f6eebcabb17/dev-support/docker/pkg-resolver/check_platform.py#L1-L17)? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] GauthamBanasandra commented on pull request #6032: Removing Trailing Spaces in Java Files
GauthamBanasandra commented on PR #6032: URL: https://github.com/apache/hadoop/pull/6032#issuecomment-1717519766 Hi @Prajwal-Surendra , Welcome to the Apache Hadoop community and kudos for raising your first PR. Please resolve the linting errors as indicated by `ASF Cloudbees Jenkins` on the `Blanks_eol.py` and ping me. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-18894) upgrade sshd-core due to CVEs
[ https://issues.apache.org/jira/browse/HADOOP-18894?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] PJ Fanning updated HADOOP-18894: Description: https://mvnrepository.com/artifact/org.apache.sshd/sshd-core hadoop currently uses v1.6.0 was: https://mvnrepository.com/artifact/org.apache.sshd/sshd-core hadoop currently uses v1.7.0 > upgrade sshd-core due to CVEs > - > > Key: HADOOP-18894 > URL: https://issues.apache.org/jira/browse/HADOOP-18894 > Project: Hadoop Common > Issue Type: Bug >Reporter: PJ Fanning >Priority: Major > Labels: pull-request-available > > https://mvnrepository.com/artifact/org.apache.sshd/sshd-core > hadoop currently uses v1.6.0 -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18894) upgrade sshd-core due to CVEs
[ https://issues.apache.org/jira/browse/HADOOP-18894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17764621#comment-17764621 ] ASF GitHub Bot commented on HADOOP-18894: - pjfanning opened a new pull request, #6060: URL: https://github.com/apache/hadoop/pull/6060 ### Description of PR https://issues.apache.org/jira/browse/HADOOP-18894 ### How was this patch tested? CI build ### For code changes: - [X ] Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')? - [ ] Object storage: have the integration tests been executed and the endpoint declared according to the connector-specific documentation? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the `LICENSE`, `LICENSE-binary`, `NOTICE-binary` files? > upgrade sshd-core due to CVEs > - > > Key: HADOOP-18894 > URL: https://issues.apache.org/jira/browse/HADOOP-18894 > Project: Hadoop Common > Issue Type: Bug >Reporter: PJ Fanning >Priority: Major > > https://mvnrepository.com/artifact/org.apache.sshd/sshd-core > hadoop currently uses v1.7.0 -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
