[jira] [Commented] (HADOOP-18329) Add support for IBM Semeru OE JRE 11.0.15.0 and greater
[ https://issues.apache.org/jira/browse/HADOOP-18329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17643661#comment-17643661 ] ASF GitHub Bot commented on HADOOP-18329: - microeastcowboy commented on PR #4537: URL: https://github.com/apache/hadoop/pull/4537#issuecomment-1338771626 Has there been any movement on the pr? > Add support for IBM Semeru OE JRE 11.0.15.0 and greater > --- > > Key: HADOOP-18329 > URL: https://issues.apache.org/jira/browse/HADOOP-18329 > Project: Hadoop Common > Issue Type: Bug > Components: auth, common >Affects Versions: 3.0.0, 3.1.0, 3.0.1, 3.2.0, 3.0.2, 3.1.1, 3.0.3, 3.3.0, > 3.1.2, 3.2.1, 3.1.3, 3.1.4, 3.2.2, 3.3.1, 3.2.3, 3.3.2, 3.3.3 > Environment: Running Hadoop (or Apache Spark 3.2.1 or above) on IBM > Semeru runtimes open edition 11.0.15.0 or greater. >Reporter: Jack >Priority: Major > Labels: pull-request-available > Original Estimate: 1h > Time Spent: 2.5h > Remaining Estimate: 0h > > There are checks within the PlatformName class that use the Vendor property > of the provided runtime JVM specifically looking for `IBM` within the name. > Whilst this check worked for IBM's [java technology > edition|https://www.ibm.com/docs/en/sdk-java-technology] it fails to work on > [Semeru|https://developer.ibm.com/languages/java/semeru-runtimes/] since > 11.0.15.0 due to the following change: > h4. java.vendor system property > In this release, the {{java.vendor}} system property has been changed from > "International Business Machines Corporation" to "IBM Corporation". > Modules such as the below are not provided in these runtimes. > com.ibm.security.auth.module.JAASLoginModule -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18329) Add support for IBM Semeru OE JRE 11.0.15.0 and greater
[ https://issues.apache.org/jira/browse/HADOOP-18329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17643951#comment-17643951 ] ASF GitHub Bot commented on HADOOP-18329: - JackBuggins commented on PR #4537: URL: https://github.com/apache/hadoop/pull/4537#issuecomment-1339652525 > Has there been any movement on this pr? I'll try and carve out a few hours this week 👍 some slight enhancement I think could be made in the class loader method I originally came up with (I want to test my concerns). > Add support for IBM Semeru OE JRE 11.0.15.0 and greater > --- > > Key: HADOOP-18329 > URL: https://issues.apache.org/jira/browse/HADOOP-18329 > Project: Hadoop Common > Issue Type: Bug > Components: auth, common >Affects Versions: 3.0.0, 3.1.0, 3.0.1, 3.2.0, 3.0.2, 3.1.1, 3.0.3, 3.3.0, > 3.1.2, 3.2.1, 3.1.3, 3.1.4, 3.2.2, 3.3.1, 3.2.3, 3.3.2, 3.3.3 > Environment: Running Hadoop (or Apache Spark 3.2.1 or above) on IBM > Semeru runtimes open edition 11.0.15.0 or greater. >Reporter: Jack >Priority: Major > Labels: pull-request-available > Original Estimate: 1h > Time Spent: 2.5h > Remaining Estimate: 0h > > There are checks within the PlatformName class that use the Vendor property > of the provided runtime JVM specifically looking for `IBM` within the name. > Whilst this check worked for IBM's [java technology > edition|https://www.ibm.com/docs/en/sdk-java-technology] it fails to work on > [Semeru|https://developer.ibm.com/languages/java/semeru-runtimes/] since > 11.0.15.0 due to the following change: > h4. java.vendor system property > In this release, the {{java.vendor}} system property has been changed from > "International Business Machines Corporation" to "IBM Corporation". > Modules such as the below are not provided in these runtimes. > com.ibm.security.auth.module.JAASLoginModule -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18329) Add support for IBM Semeru OE JRE 11.0.15.0 and greater
[ https://issues.apache.org/jira/browse/HADOOP-18329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17644002#comment-17644002 ] ASF GitHub Bot commented on HADOOP-18329: - JackBuggins commented on PR #4537: URL: https://github.com/apache/hadoop/pull/4537#issuecomment-1339816599 This should be a bit more robust to extension as well as handle the concerns I had about the class loader from before. Is there any consensus/ruling around adding a test against the IBM JREs? I appreciate it would take a bit of time on CI and this is a once in a blue moon activity, but it could be a single suite of integration tests against auth that execute to verify the result against latest semeru is not IBM, and vice versa. > Add support for IBM Semeru OE JRE 11.0.15.0 and greater > --- > > Key: HADOOP-18329 > URL: https://issues.apache.org/jira/browse/HADOOP-18329 > Project: Hadoop Common > Issue Type: Bug > Components: auth, common >Affects Versions: 3.0.0, 3.1.0, 3.0.1, 3.2.0, 3.0.2, 3.1.1, 3.0.3, 3.3.0, > 3.1.2, 3.2.1, 3.1.3, 3.1.4, 3.2.2, 3.3.1, 3.2.3, 3.3.2, 3.3.3 > Environment: Running Hadoop (or Apache Spark 3.2.1 or above) on IBM > Semeru runtimes open edition 11.0.15.0 or greater. >Reporter: Jack >Priority: Major > Labels: pull-request-available > Original Estimate: 1h > Time Spent: 2.5h > Remaining Estimate: 0h > > There are checks within the PlatformName class that use the Vendor property > of the provided runtime JVM specifically looking for `IBM` within the name. > Whilst this check worked for IBM's [java technology > edition|https://www.ibm.com/docs/en/sdk-java-technology] it fails to work on > [Semeru|https://developer.ibm.com/languages/java/semeru-runtimes/] since > 11.0.15.0 due to the following change: > h4. java.vendor system property > In this release, the {{java.vendor}} system property has been changed from > "International Business Machines Corporation" to "IBM Corporation". > Modules such as the below are not provided in these runtimes. > com.ibm.security.auth.module.JAASLoginModule -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18329) Add support for IBM Semeru OE JRE 11.0.15.0 and greater
[ https://issues.apache.org/jira/browse/HADOOP-18329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17644048#comment-17644048 ] ASF GitHub Bot commented on HADOOP-18329: - hadoop-yetus commented on PR #4537: URL: https://github.com/apache/hadoop/pull/4537#issuecomment-1340106496 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reexec | 0m 54s | | Docker mode activated. | _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | | No case conflicting files found. | | +0 :ok: | codespell | 0m 2s | | codespell was not available. | | +0 :ok: | detsecrets | 0m 2s | | detect-secrets was not available. | | +0 :ok: | xmllint | 0m 2s | | xmllint was not available. | | +1 :green_heart: | @author | 0m 0s | | The patch does not contain any @author tags. | | +1 :green_heart: | test4tests | 0m 0s | | The patch appears to include 5 new or modified test files. | _ trunk Compile Tests _ | | +0 :ok: | mvndep | 20m 19s | | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 29m 15s | | trunk passed | | +1 :green_heart: | compile | 25m 25s | | trunk passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 | | +1 :green_heart: | compile | 21m 41s | | trunk passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 | | +1 :green_heart: | checkstyle | 1m 14s | | trunk passed | | +1 :green_heart: | mvnsite | 3m 18s | | trunk passed | | +1 :green_heart: | javadoc | 2m 46s | | trunk passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 | | +1 :green_heart: | javadoc | 2m 12s | | trunk passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 | | +1 :green_heart: | spotbugs | 5m 0s | | trunk passed | | +1 :green_heart: | shadedclient | 23m 18s | | branch has no errors when building and testing our client artifacts. | _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 23s | | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 1m 54s | | the patch passed | | +1 :green_heart: | compile | 24m 29s | | the patch passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 | | +1 :green_heart: | javac | 24m 29s | | the patch passed | | +1 :green_heart: | compile | 21m 41s | | the patch passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 | | +1 :green_heart: | javac | 21m 41s | | the patch passed | | +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks issues. | | +1 :green_heart: | checkstyle | 1m 9s | | the patch passed | | +1 :green_heart: | mvnsite | 3m 15s | | the patch passed | | +1 :green_heart: | javadoc | 2m 34s | | the patch passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 | | +1 :green_heart: | javadoc | 2m 12s | | the patch passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 | | -1 :x: | spotbugs | 1m 0s | [/new-spotbugs-hadoop-common-project_hadoop-auth.html](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4537/6/artifact/out/new-spotbugs-hadoop-common-project_hadoop-auth.html) | hadoop-common-project/hadoop-auth generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0) | | +1 :green_heart: | shadedclient | 24m 9s | | patch has no errors when building and testing our client artifacts. | _ Other Tests _ | | +1 :green_heart: | unit | 0m 38s | | hadoop-minikdc in the patch passed. | | +1 :green_heart: | unit | 3m 18s | | hadoop-auth in the patch passed. | | +1 :green_heart: | unit | 18m 13s | | hadoop-common in the patch passed. | | +1 :green_heart: | unit | 1m 17s | | hadoop-registry in the patch passed. | | +1 :green_heart: | asflicense | 0m 52s | | The patch does not generate ASF License warnings. | | | | 250m 48s | | | | Reason | Tests | |---:|:--| | SpotBugs | module:hadoop-common-project/hadoop-auth | | | org.apache.hadoop.util.PlatformName.() creates a org.apache.hadoop.util.PlatformName$SystemClassAccessor classloader, which should be performed within a doPrivileged block At PlatformName.java:a org.apache.hadoop.util.PlatformName$SystemClassAccessor classloader, which should be performed within a doPrivileged block At PlatformName.java:[line 61] | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4537/6/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hadoop/pull/4537 | | Optional Tests
[jira] [Commented] (HADOOP-18329) Add support for IBM Semeru OE JRE 11.0.15.0 and greater
[ https://issues.apache.org/jira/browse/HADOOP-18329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17644481#comment-17644481 ] ASF GitHub Bot commented on HADOOP-18329: - hadoop-yetus commented on PR #4537: URL: https://github.com/apache/hadoop/pull/4537#issuecomment-1341464321 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reexec | 0m 51s | | Docker mode activated. | _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | | No case conflicting files found. | | +0 :ok: | codespell | 0m 0s | | codespell was not available. | | +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available. | | +0 :ok: | xmllint | 0m 0s | | xmllint was not available. | | +1 :green_heart: | @author | 0m 0s | | The patch does not contain any @author tags. | | +1 :green_heart: | test4tests | 0m 0s | | The patch appears to include 5 new or modified test files. | _ trunk Compile Tests _ | | +0 :ok: | mvndep | 17m 14s | | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 28m 42s | | trunk passed | | +1 :green_heart: | compile | 24m 48s | | trunk passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 | | +1 :green_heart: | compile | 21m 38s | | trunk passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 | | +1 :green_heart: | checkstyle | 1m 13s | | trunk passed | | +1 :green_heart: | mvnsite | 3m 18s | | trunk passed | | +1 :green_heart: | javadoc | 2m 42s | | trunk passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 | | +1 :green_heart: | javadoc | 2m 11s | | trunk passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 | | +1 :green_heart: | spotbugs | 4m 58s | | trunk passed | | +1 :green_heart: | shadedclient | 23m 15s | | branch has no errors when building and testing our client artifacts. | _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 23s | | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 1m 53s | | the patch passed | | +1 :green_heart: | compile | 24m 28s | | the patch passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 | | +1 :green_heart: | javac | 24m 28s | | the patch passed | | +1 :green_heart: | compile | 21m 39s | | the patch passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 | | +1 :green_heart: | javac | 21m 39s | | the patch passed | | +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks issues. | | +1 :green_heart: | checkstyle | 1m 8s | | the patch passed | | +1 :green_heart: | mvnsite | 3m 14s | | the patch passed | | +1 :green_heart: | javadoc | 2m 34s | | the patch passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 | | +1 :green_heart: | javadoc | 2m 9s | | the patch passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 | | +1 :green_heart: | spotbugs | 5m 30s | | the patch passed | | +1 :green_heart: | shadedclient | 23m 24s | | patch has no errors when building and testing our client artifacts. | _ Other Tests _ | | +1 :green_heart: | unit | 0m 38s | | hadoop-minikdc in the patch passed. | | +1 :green_heart: | unit | 3m 17s | | hadoop-auth in the patch passed. | | +1 :green_heart: | unit | 18m 9s | | hadoop-common in the patch passed. | | +1 :green_heart: | unit | 1m 15s | | hadoop-registry in the patch passed. | | +1 :green_heart: | asflicense | 0m 51s | | The patch does not generate ASF License warnings. | | | | 245m 9s | | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4537/7/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hadoop/pull/4537 | | Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets xmllint | | uname | Linux bd1ee2ae4f51 4.15.0-200-generic #211-Ubuntu SMP Thu Nov 24 18:16:04 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/bin/hadoop.sh | | git revision | trunk / 2f21b8b7512ac914b73e0f44967cc119de34482c | | Default Java | Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 | | Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 | | Test Results | https://ci-hadoop.apache.org/jo
[jira] [Commented] (HADOOP-18329) Add support for IBM Semeru OE JRE 11.0.15.0 and greater
[ https://issues.apache.org/jira/browse/HADOOP-18329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17644756#comment-17644756 ] ASF GitHub Bot commented on HADOOP-18329: - JackBuggins commented on PR #4537: URL: https://github.com/apache/hadoop/pull/4537#issuecomment-1342580453 Few changes made here based on checks against other architectures/platforms. > Add support for IBM Semeru OE JRE 11.0.15.0 and greater > --- > > Key: HADOOP-18329 > URL: https://issues.apache.org/jira/browse/HADOOP-18329 > Project: Hadoop Common > Issue Type: Bug > Components: auth, common >Affects Versions: 3.0.0, 3.1.0, 3.0.1, 3.2.0, 3.0.2, 3.1.1, 3.0.3, 3.3.0, > 3.1.2, 3.2.1, 3.1.3, 3.1.4, 3.2.2, 3.3.1, 3.2.3, 3.3.2, 3.3.3 > Environment: Running Hadoop (or Apache Spark 3.2.1 or above) on IBM > Semeru runtimes open edition 11.0.15.0 or greater. >Reporter: Jack >Priority: Major > Labels: pull-request-available > Original Estimate: 1h > Time Spent: 2.5h > Remaining Estimate: 0h > > There are checks within the PlatformName class that use the Vendor property > of the provided runtime JVM specifically looking for `IBM` within the name. > Whilst this check worked for IBM's [java technology > edition|https://www.ibm.com/docs/en/sdk-java-technology] it fails to work on > [Semeru|https://developer.ibm.com/languages/java/semeru-runtimes/] since > 11.0.15.0 due to the following change: > h4. java.vendor system property > In this release, the {{java.vendor}} system property has been changed from > "International Business Machines Corporation" to "IBM Corporation". > Modules such as the below are not provided in these runtimes. > com.ibm.security.auth.module.JAASLoginModule -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18329) Add support for IBM Semeru OE JRE 11.0.15.0 and greater
[ https://issues.apache.org/jira/browse/HADOOP-18329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17644845#comment-17644845 ] ASF GitHub Bot commented on HADOOP-18329: - hadoop-yetus commented on PR #4537: URL: https://github.com/apache/hadoop/pull/4537#issuecomment-1342833600 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reexec | 1m 22s | | Docker mode activated. | _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | | No case conflicting files found. | | +0 :ok: | codespell | 0m 1s | | codespell was not available. | | +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available. | | +0 :ok: | xmllint | 0m 1s | | xmllint was not available. | | +1 :green_heart: | @author | 0m 0s | | The patch does not contain any @author tags. | | +1 :green_heart: | test4tests | 0m 0s | | The patch appears to include 5 new or modified test files. | _ trunk Compile Tests _ | | +0 :ok: | mvndep | 15m 43s | | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 29m 4s | | trunk passed | | +1 :green_heart: | compile | 28m 0s | | trunk passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 | | +1 :green_heart: | compile | 25m 59s | | trunk passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 | | +1 :green_heart: | checkstyle | 1m 17s | | trunk passed | | +1 :green_heart: | mvnsite | 3m 37s | | trunk passed | | +1 :green_heart: | javadoc | 2m 50s | | trunk passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 | | +1 :green_heart: | javadoc | 2m 28s | | trunk passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 | | +1 :green_heart: | spotbugs | 6m 18s | | trunk passed | | +1 :green_heart: | shadedclient | 24m 19s | | branch has no errors when building and testing our client artifacts. | _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 26s | | Maven dependency ordering for patch | | -1 :x: | mvninstall | 0m 15s | [/patch-mvninstall-hadoop-common-project_hadoop-auth.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4537/8/artifact/out/patch-mvninstall-hadoop-common-project_hadoop-auth.txt) | hadoop-auth in the patch failed. | | +1 :green_heart: | compile | 25m 59s | | the patch passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 | | +1 :green_heart: | javac | 25m 59s | | the patch passed | | -1 :x: | compile | 0m 44s | [/patch-compile-root-jdkPrivateBuild-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4537/8/artifact/out/patch-compile-root-jdkPrivateBuild-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07.txt) | root in the patch failed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07. | | -1 :x: | javac | 0m 44s | [/patch-compile-root-jdkPrivateBuild-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4537/8/artifact/out/patch-compile-root-jdkPrivateBuild-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07.txt) | root in the patch failed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07. | | +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks issues. | | +1 :green_heart: | checkstyle | 1m 15s | | the patch passed | | -1 :x: | mvnsite | 0m 31s | [/patch-mvnsite-hadoop-common-project_hadoop-auth.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4537/8/artifact/out/patch-mvnsite-hadoop-common-project_hadoop-auth.txt) | hadoop-auth in the patch failed. | | +1 :green_heart: | javadoc | 3m 22s | | the patch passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 | | -1 :x: | javadoc | 0m 30s | [/patch-javadoc-hadoop-common-project_hadoop-auth-jdkPrivateBuild-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4537/8/artifact/out/patch-javadoc-hadoop-common-project_hadoop-auth-jdkPrivateBuild-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07.txt) | hadoop-auth in the patch failed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07. | | -1 :x: | spotbugs | 0m 30s | [/patch-spotbugs-hadoop-common-project_hadoop-auth.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4537/8/artifact/out/patch-spotbugs-hadoop-common-project_hadoop-auth.txt) | hadoop-auth in the patch failed. | | -1 :x: | shadedclient | 2m 31s | | patch has errors when building and testing our client artifacts. | _ Other Tests _ | | +1 :green_heart: | unit | 0m 37s | | hadoop-minikdc in the patch passed. | |
[jira] [Commented] (HADOOP-18329) Add support for IBM Semeru OE JRE 11.0.15.0 and greater
[ https://issues.apache.org/jira/browse/HADOOP-18329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17644857#comment-17644857 ] ASF GitHub Bot commented on HADOOP-18329: - hadoop-yetus commented on PR #4537: URL: https://github.com/apache/hadoop/pull/4537#issuecomment-1342868131 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reexec | 0m 51s | | Docker mode activated. | _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | | No case conflicting files found. | | +0 :ok: | codespell | 0m 0s | | codespell was not available. | | +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available. | | +0 :ok: | xmllint | 0m 0s | | xmllint was not available. | | +1 :green_heart: | @author | 0m 0s | | The patch does not contain any @author tags. | | +1 :green_heart: | test4tests | 0m 0s | | The patch appears to include 5 new or modified test files. | _ trunk Compile Tests _ | | +0 :ok: | mvndep | 14m 20s | | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 33m 7s | | trunk passed | | +1 :green_heart: | compile | 29m 41s | | trunk passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 | | +1 :green_heart: | compile | 23m 27s | | trunk passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 | | +1 :green_heart: | checkstyle | 1m 13s | | trunk passed | | +1 :green_heart: | mvnsite | 3m 18s | | trunk passed | | +1 :green_heart: | javadoc | 2m 44s | | trunk passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 | | +1 :green_heart: | javadoc | 2m 9s | | trunk passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 | | +1 :green_heart: | spotbugs | 5m 22s | | trunk passed | | +1 :green_heart: | shadedclient | 24m 42s | | branch has no errors when building and testing our client artifacts. | _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 26s | | Maven dependency ordering for patch | | -1 :x: | mvninstall | 0m 15s | [/patch-mvninstall-hadoop-common-project_hadoop-auth.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4537/9/artifact/out/patch-mvninstall-hadoop-common-project_hadoop-auth.txt) | hadoop-auth in the patch failed. | | +1 :green_heart: | compile | 27m 5s | | the patch passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 | | +1 :green_heart: | javac | 27m 5s | | the patch passed | | -1 :x: | compile | 0m 42s | [/patch-compile-root-jdkPrivateBuild-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4537/9/artifact/out/patch-compile-root-jdkPrivateBuild-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07.txt) | root in the patch failed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07. | | -1 :x: | javac | 0m 42s | [/patch-compile-root-jdkPrivateBuild-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4537/9/artifact/out/patch-compile-root-jdkPrivateBuild-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07.txt) | root in the patch failed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07. | | +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks issues. | | +1 :green_heart: | checkstyle | 1m 11s | | hadoop-common-project: The patch generated 0 new + 310 unchanged - 1 fixed = 310 total (was 311) | | -1 :x: | mvnsite | 0m 31s | [/patch-mvnsite-hadoop-common-project_hadoop-auth.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4537/9/artifact/out/patch-mvnsite-hadoop-common-project_hadoop-auth.txt) | hadoop-auth in the patch failed. | | +1 :green_heart: | javadoc | 2m 35s | | the patch passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 | | -1 :x: | javadoc | 0m 32s | [/patch-javadoc-hadoop-common-project_hadoop-auth-jdkPrivateBuild-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4537/9/artifact/out/patch-javadoc-hadoop-common-project_hadoop-auth-jdkPrivateBuild-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07.txt) | hadoop-auth in the patch failed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07. | | -1 :x: | spotbugs | 0m 29s | [/patch-spotbugs-hadoop-common-project_hadoop-auth.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4537/9/artifact/out/patch-spotbugs-hadoop-common-project_hadoop-auth.txt) | hadoop-auth in the patch failed. | | -1 :x: | shadedclient | 2m 16s | | patch has errors when building and testing our client artifacts. | _ Other Tests _ | | +1 :gre
[jira] [Commented] (HADOOP-18329) Add support for IBM Semeru OE JRE 11.0.15.0 and greater
[ https://issues.apache.org/jira/browse/HADOOP-18329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17644901#comment-17644901 ] ASF GitHub Bot commented on HADOOP-18329: - JackBuggins commented on PR #4537: URL: https://github.com/apache/hadoop/pull/4537#issuecomment-1343032982 Javadoc seemed to stackoverflow, that should be correct now. > Add support for IBM Semeru OE JRE 11.0.15.0 and greater > --- > > Key: HADOOP-18329 > URL: https://issues.apache.org/jira/browse/HADOOP-18329 > Project: Hadoop Common > Issue Type: Bug > Components: auth, common >Affects Versions: 3.0.0, 3.1.0, 3.0.1, 3.2.0, 3.0.2, 3.1.1, 3.0.3, 3.3.0, > 3.1.2, 3.2.1, 3.1.3, 3.1.4, 3.2.2, 3.3.1, 3.2.3, 3.3.2, 3.3.3 > Environment: Running Hadoop (or Apache Spark 3.2.1 or above) on IBM > Semeru runtimes open edition 11.0.15.0 or greater. >Reporter: Jack >Priority: Major > Labels: pull-request-available > Original Estimate: 1h > Time Spent: 2.5h > Remaining Estimate: 0h > > There are checks within the PlatformName class that use the Vendor property > of the provided runtime JVM specifically looking for `IBM` within the name. > Whilst this check worked for IBM's [java technology > edition|https://www.ibm.com/docs/en/sdk-java-technology] it fails to work on > [Semeru|https://developer.ibm.com/languages/java/semeru-runtimes/] since > 11.0.15.0 due to the following change: > h4. java.vendor system property > In this release, the {{java.vendor}} system property has been changed from > "International Business Machines Corporation" to "IBM Corporation". > Modules such as the below are not provided in these runtimes. > com.ibm.security.auth.module.JAASLoginModule -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18329) Add support for IBM Semeru OE JRE 11.0.15.0 and greater
[ https://issues.apache.org/jira/browse/HADOOP-18329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17644946#comment-17644946 ] ASF GitHub Bot commented on HADOOP-18329: - hadoop-yetus commented on PR #4537: URL: https://github.com/apache/hadoop/pull/4537#issuecomment-1343200499 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reexec | 0m 47s | | Docker mode activated. | _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | | No case conflicting files found. | | +0 :ok: | codespell | 0m 1s | | codespell was not available. | | +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available. | | +0 :ok: | xmllint | 0m 1s | | xmllint was not available. | | +1 :green_heart: | @author | 0m 0s | | The patch does not contain any @author tags. | | +1 :green_heart: | test4tests | 0m 0s | | The patch appears to include 5 new or modified test files. | _ trunk Compile Tests _ | | +0 :ok: | mvndep | 15m 26s | | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 28m 57s | | trunk passed | | +1 :green_heart: | compile | 25m 0s | | trunk passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 | | +1 :green_heart: | compile | 21m 44s | | trunk passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 | | +1 :green_heart: | checkstyle | 1m 14s | | trunk passed | | +1 :green_heart: | mvnsite | 3m 17s | | trunk passed | | +1 :green_heart: | javadoc | 2m 45s | | trunk passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 | | +1 :green_heart: | javadoc | 2m 11s | | trunk passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 | | +1 :green_heart: | spotbugs | 5m 1s | | trunk passed | | +1 :green_heart: | shadedclient | 24m 1s | | branch has no errors when building and testing our client artifacts. | _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 24s | | Maven dependency ordering for patch | | -1 :x: | mvninstall | 0m 14s | [/patch-mvninstall-hadoop-common-project_hadoop-auth.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4537/10/artifact/out/patch-mvninstall-hadoop-common-project_hadoop-auth.txt) | hadoop-auth in the patch failed. | | +1 :green_heart: | compile | 28m 52s | | the patch passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 | | +1 :green_heart: | javac | 28m 52s | | the patch passed | | -1 :x: | compile | 0m 45s | [/patch-compile-root-jdkPrivateBuild-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4537/10/artifact/out/patch-compile-root-jdkPrivateBuild-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07.txt) | root in the patch failed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07. | | -1 :x: | javac | 0m 45s | [/patch-compile-root-jdkPrivateBuild-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4537/10/artifact/out/patch-compile-root-jdkPrivateBuild-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07.txt) | root in the patch failed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07. | | +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks issues. | | +1 :green_heart: | checkstyle | 1m 8s | | hadoop-common-project: The patch generated 0 new + 310 unchanged - 1 fixed = 310 total (was 311) | | -1 :x: | mvnsite | 0m 29s | [/patch-mvnsite-hadoop-common-project_hadoop-auth.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4537/10/artifact/out/patch-mvnsite-hadoop-common-project_hadoop-auth.txt) | hadoop-auth in the patch failed. | | +1 :green_heart: | javadoc | 2m 40s | | the patch passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 | | -1 :x: | javadoc | 0m 30s | [/patch-javadoc-hadoop-common-project_hadoop-auth-jdkPrivateBuild-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4537/10/artifact/out/patch-javadoc-hadoop-common-project_hadoop-auth-jdkPrivateBuild-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07.txt) | hadoop-auth in the patch failed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07. | | -1 :x: | spotbugs | 0m 35s | [/patch-spotbugs-hadoop-common-project_hadoop-auth.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4537/10/artifact/out/patch-spotbugs-hadoop-common-project_hadoop-auth.txt) | hadoop-auth in the patch failed. | | -1 :x: | shadedclient | 2m 29s | | patch has errors when building and testing our client artifacts. | _ Other Tests _ | | +
[jira] [Commented] (HADOOP-18329) Add support for IBM Semeru OE JRE 11.0.15.0 and greater
[ https://issues.apache.org/jira/browse/HADOOP-18329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17644997#comment-17644997 ] ASF GitHub Bot commented on HADOOP-18329: - hadoop-yetus commented on PR #4537: URL: https://github.com/apache/hadoop/pull/4537#issuecomment-1343410646 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reexec | 1m 3s | | Docker mode activated. | _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | | No case conflicting files found. | | +0 :ok: | codespell | 0m 0s | | codespell was not available. | | +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available. | | +0 :ok: | xmllint | 0m 0s | | xmllint was not available. | | +1 :green_heart: | @author | 0m 0s | | The patch does not contain any @author tags. | | +1 :green_heart: | test4tests | 0m 0s | | The patch appears to include 5 new or modified test files. | _ trunk Compile Tests _ | | +0 :ok: | mvndep | 15m 14s | | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 36m 30s | | trunk passed | | +1 :green_heart: | compile | 28m 52s | | trunk passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 | | +1 :green_heart: | compile | 24m 23s | | trunk passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 | | +1 :green_heart: | checkstyle | 1m 18s | | trunk passed | | +1 :green_heart: | mvnsite | 3m 22s | | trunk passed | | +1 :green_heart: | javadoc | 2m 53s | | trunk passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 | | +1 :green_heart: | javadoc | 2m 13s | | trunk passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 | | +1 :green_heart: | spotbugs | 5m 0s | | trunk passed | | +1 :green_heart: | shadedclient | 24m 39s | | branch has no errors when building and testing our client artifacts. | _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 30s | | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 2m 45s | | the patch passed | | +1 :green_heart: | compile | 41m 59s | | the patch passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 | | +1 :green_heart: | javac | 41m 59s | | the patch passed | | -1 :x: | compile | 27m 24s | [/patch-compile-root-jdkPrivateBuild-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4537/11/artifact/out/patch-compile-root-jdkPrivateBuild-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07.txt) | root in the patch failed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07. | | -1 :x: | javac | 27m 24s | [/patch-compile-root-jdkPrivateBuild-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4537/11/artifact/out/patch-compile-root-jdkPrivateBuild-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07.txt) | root in the patch failed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07. | | +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks issues. | | -0 :warning: | checkstyle | 0m 36s | [/buildtool-patch-checkstyle-hadoop-common-project.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4537/11/artifact/out/buildtool-patch-checkstyle-hadoop-common-project.txt) | The patch fails to run checkstyle in hadoop-common-project | | -1 :x: | mvnsite | 0m 38s | [/patch-mvnsite-hadoop-common-project_hadoop-minikdc.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4537/11/artifact/out/patch-mvnsite-hadoop-common-project_hadoop-minikdc.txt) | hadoop-minikdc in the patch failed. | | -1 :x: | mvnsite | 0m 39s | [/patch-mvnsite-hadoop-common-project_hadoop-auth.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4537/11/artifact/out/patch-mvnsite-hadoop-common-project_hadoop-auth.txt) | hadoop-auth in the patch failed. | | -1 :x: | mvnsite | 0m 39s | [/patch-mvnsite-hadoop-common-project_hadoop-common.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4537/11/artifact/out/patch-mvnsite-hadoop-common-project_hadoop-common.txt) | hadoop-common in the patch failed. | | -1 :x: | mvnsite | 0m 41s | [/patch-mvnsite-hadoop-common-project_hadoop-registry.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4537/11/artifact/out/patch-mvnsite-hadoop-common-project_hadoop-registry.txt) | hadoop-registry in the patch failed. | | -1 :x: | javadoc | 0m 40s | [/patch-javadoc-hadoop-common-project_hadoop-minikdc-jdkUbuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4537/1
[jira] [Commented] (HADOOP-18329) Add support for IBM Semeru OE JRE 11.0.15.0 and greater
[ https://issues.apache.org/jira/browse/HADOOP-18329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17645274#comment-17645274 ] ASF GitHub Bot commented on HADOOP-18329: - JackBuggins commented on PR #4537: URL: https://github.com/apache/hadoop/pull/4537#issuecomment-1344249638 @steveloughran is there a recommended way to kick CI different from pushing empty commits? I'm fairly confident the failures I'm observing here have been pulled from upstream based on what I'm seeing locally, if that's the case and I can hold off until a fix comes in... if it's not I'm a little baffled and may need to investigate deeper. > Add support for IBM Semeru OE JRE 11.0.15.0 and greater > --- > > Key: HADOOP-18329 > URL: https://issues.apache.org/jira/browse/HADOOP-18329 > Project: Hadoop Common > Issue Type: Bug > Components: auth, common >Affects Versions: 3.0.0, 3.1.0, 3.0.1, 3.2.0, 3.0.2, 3.1.1, 3.0.3, 3.3.0, > 3.1.2, 3.2.1, 3.1.3, 3.1.4, 3.2.2, 3.3.1, 3.2.3, 3.3.2, 3.3.3 > Environment: Running Hadoop (or Apache Spark 3.2.1 or above) on IBM > Semeru runtimes open edition 11.0.15.0 or greater. >Reporter: Jack >Priority: Major > Labels: pull-request-available > Original Estimate: 1h > Time Spent: 2.5h > Remaining Estimate: 0h > > There are checks within the PlatformName class that use the Vendor property > of the provided runtime JVM specifically looking for `IBM` within the name. > Whilst this check worked for IBM's [java technology > edition|https://www.ibm.com/docs/en/sdk-java-technology] it fails to work on > [Semeru|https://developer.ibm.com/languages/java/semeru-runtimes/] since > 11.0.15.0 due to the following change: > h4. java.vendor system property > In this release, the {{java.vendor}} system property has been changed from > "International Business Machines Corporation" to "IBM Corporation". > Modules such as the below are not provided in these runtimes. > com.ibm.security.auth.module.JAASLoginModule -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18329) Add support for IBM Semeru OE JRE 11.0.15.0 and greater
[ https://issues.apache.org/jira/browse/HADOOP-18329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17645357#comment-17645357 ] ASF GitHub Bot commented on HADOOP-18329: - steveloughran commented on PR #4537: URL: https://github.com/apache/hadoop/pull/4537#issuecomment-1344466958 a big everything failure is the container runtime playing up; CI tooling rather than the source treee > Add support for IBM Semeru OE JRE 11.0.15.0 and greater > --- > > Key: HADOOP-18329 > URL: https://issues.apache.org/jira/browse/HADOOP-18329 > Project: Hadoop Common > Issue Type: Bug > Components: auth, common >Affects Versions: 3.0.0, 3.1.0, 3.0.1, 3.2.0, 3.0.2, 3.1.1, 3.0.3, 3.3.0, > 3.1.2, 3.2.1, 3.1.3, 3.1.4, 3.2.2, 3.3.1, 3.2.3, 3.3.2, 3.3.3 > Environment: Running Hadoop (or Apache Spark 3.2.1 or above) on IBM > Semeru runtimes open edition 11.0.15.0 or greater. >Reporter: Jack >Priority: Major > Labels: pull-request-available > Original Estimate: 1h > Time Spent: 2.5h > Remaining Estimate: 0h > > There are checks within the PlatformName class that use the Vendor property > of the provided runtime JVM specifically looking for `IBM` within the name. > Whilst this check worked for IBM's [java technology > edition|https://www.ibm.com/docs/en/sdk-java-technology] it fails to work on > [Semeru|https://developer.ibm.com/languages/java/semeru-runtimes/] since > 11.0.15.0 due to the following change: > h4. java.vendor system property > In this release, the {{java.vendor}} system property has been changed from > "International Business Machines Corporation" to "IBM Corporation". > Modules such as the below are not provided in these runtimes. > com.ibm.security.auth.module.JAASLoginModule -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18329) Add support for IBM Semeru OE JRE 11.0.15.0 and greater
[ https://issues.apache.org/jira/browse/HADOOP-18329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17645369#comment-17645369 ] ASF GitHub Bot commented on HADOOP-18329: - steveloughran commented on code in PR #4537: URL: https://github.com/apache/hadoop/pull/4537#discussion_r1044590462 ## hadoop-common-project/hadoop-auth/src/main/java/org/apache/hadoop/util/PlatformName.java: ## @@ -33,16 +39,33 @@ public class PlatformName { * per the java-vm. */ public static final String PLATFORM_NAME = - (System.getProperty("os.name").startsWith("Windows") - ? System.getenv("os") : System.getProperty("os.name")) - + "-" + System.getProperty("os.arch") - + "-" + System.getProperty("sun.arch.data.model"); + (System.getProperty("os.name").startsWith("Windows") ? + System.getenv("os") : System.getProperty("os.name")) + + "-" + System.getProperty("os.arch") + "-" + + System.getProperty("sun.arch.data.model"); /** * The java vendor name used in this platform. */ public static final String JAVA_VENDOR_NAME = System.getProperty("java.vendor"); + /** + * A concurrently accessible hashmap that saves re-computation of vendor checks. + */ + private static final Map SYSTEM_CLASS_AVAILABILITY = new ConcurrentHashMap<>(); Review Comment: so this a static but mutable map. good to see it is concurrent *and* not visible outside the class. ## hadoop-common-project/hadoop-minikdc/src/test/java/org/apache/hadoop/minikdc/TestMiniKdc.java: ## @@ -38,8 +38,35 @@ import java.util.Arrays; public class TestMiniKdc extends KerberosSecurityTestcase { - private static final boolean IBM_JAVA = System.getProperty("java.vendor") - .contains("IBM"); + private static final boolean IBM_JAVA = shouldUseIbmPackages(); + // duplicated to avoid cycles in the build + private static boolean shouldUseIbmPackages() { +final List ibmTechnologyEditionSecurityModules = Arrays.asList( +"com.ibm.security.auth.module.JAASLoginModule", +"com.ibm.security.auth.module.Win64LoginModule", +"com.ibm.security.auth.module.NTLoginModule", +"com.ibm.security.auth.module.AIX64LoginModule", +"com.ibm.security.auth.module.LinuxLoginModule", +"com.ibm.security.auth.module.Krb5LoginModule" +); + +if (System.getProperty("java.vendor").contains("IBM")) { + return ibmTechnologyEditionSecurityModules + .stream().anyMatch((module) -> isSystemClassAvailable(module)); +} + +return false; + } + + private static boolean isSystemClassAvailable(String className) { Review Comment: any reason for not using a copy of isSystemClassAvailable() here ... are you just waiting for feedback on the hadoop auth section? ## hadoop-common-project/hadoop-auth/src/main/java/org/apache/hadoop/util/PlatformName.java: ## @@ -33,21 +37,72 @@ public class PlatformName { * per the java-vm. */ public static final String PLATFORM_NAME = - (System.getProperty("os.name").startsWith("Windows") - ? System.getenv("os") : System.getProperty("os.name")) - + "-" + System.getProperty("os.arch") - + "-" + System.getProperty("sun.arch.data.model"); + (System.getProperty("os.name").startsWith("Windows") ? + System.getenv("os") : System.getProperty("os.name")) + + "-" + System.getProperty("os.arch") + "-" + + System.getProperty("sun.arch.data.model"); /** * The java vendor name used in this platform. */ public static final String JAVA_VENDOR_NAME = System.getProperty("java.vendor"); + /** + * Define a system class accessor that is open to changes in underlying implementations + * of the system class loader modules. + */ + private static final class SystemClassAccessor extends ClassLoader { +public Class getSystemClass(String className) throws ClassNotFoundException { + return findSystemClass(className); +} + } + /** * A public static variable to indicate the current java vendor is - * IBM java or not. + * IBM and the type is Java Technology Edition which provides its + * own implementations of many security packages and Cipher suites. + * Note that these are not provided in Semeru runtimes: + * See https://developer.ibm.com/languages/java/semeru-runtimes/ + * The class used is present in any supported IBM JTE Runtimes. Review Comment: not sure what this sentence means. easiest to cut it. > Add support for IBM Semeru OE JRE 11.0.15.0 and greater > --- > > Key: HADOOP-18329 > URL: https://issues.apache.org/jira/browse/HADOOP-18329 > Project: Hadoop Common > Issue Type: Bug > Components: auth, common >Affects Versions: 3.0.0, 3.1.0, 3.0.1, 3.2.0, 3.0.2, 3.1.1, 3.0.3, 3.
[jira] [Commented] (HADOOP-18329) Add support for IBM Semeru OE JRE 11.0.15.0 and greater
[ https://issues.apache.org/jira/browse/HADOOP-18329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17645448#comment-17645448 ] ASF GitHub Bot commented on HADOOP-18329: - JackBuggins commented on code in PR #4537: URL: https://github.com/apache/hadoop/pull/4537#discussion_r1044746318 ## hadoop-common-project/hadoop-auth/src/main/java/org/apache/hadoop/util/PlatformName.java: ## @@ -33,16 +39,33 @@ public class PlatformName { * per the java-vm. */ public static final String PLATFORM_NAME = - (System.getProperty("os.name").startsWith("Windows") - ? System.getenv("os") : System.getProperty("os.name")) - + "-" + System.getProperty("os.arch") - + "-" + System.getProperty("sun.arch.data.model"); + (System.getProperty("os.name").startsWith("Windows") ? + System.getenv("os") : System.getProperty("os.name")) + + "-" + System.getProperty("os.arch") + "-" + + System.getProperty("sun.arch.data.model"); /** * The java vendor name used in this platform. */ public static final String JAVA_VENDOR_NAME = System.getProperty("java.vendor"); + /** + * A concurrently accessible hashmap that saves re-computation of vendor checks. + */ + private static final Map SYSTEM_CLASS_AVAILABILITY = new ConcurrentHashMap<>(); Review Comment: Findbugs was warning about initiating an instance outside of a doPrivileged call despite that being private, so keeping this private but static was the compromise made to avoid adding exceptions. Originally I would've been tempted to keep this non static and use the instance each time, as well as writing the result to a concurrent map so we could only run the computation if it did not exist. > Add support for IBM Semeru OE JRE 11.0.15.0 and greater > --- > > Key: HADOOP-18329 > URL: https://issues.apache.org/jira/browse/HADOOP-18329 > Project: Hadoop Common > Issue Type: Bug > Components: auth, common >Affects Versions: 3.0.0, 3.1.0, 3.0.1, 3.2.0, 3.0.2, 3.1.1, 3.0.3, 3.3.0, > 3.1.2, 3.2.1, 3.1.3, 3.1.4, 3.2.2, 3.3.1, 3.2.3, 3.3.2, 3.3.3 > Environment: Running Hadoop (or Apache Spark 3.2.1 or above) on IBM > Semeru runtimes open edition 11.0.15.0 or greater. >Reporter: Jack >Priority: Major > Labels: pull-request-available > Original Estimate: 1h > Time Spent: 2.5h > Remaining Estimate: 0h > > There are checks within the PlatformName class that use the Vendor property > of the provided runtime JVM specifically looking for `IBM` within the name. > Whilst this check worked for IBM's [java technology > edition|https://www.ibm.com/docs/en/sdk-java-technology] it fails to work on > [Semeru|https://developer.ibm.com/languages/java/semeru-runtimes/] since > 11.0.15.0 due to the following change: > h4. java.vendor system property > In this release, the {{java.vendor}} system property has been changed from > "International Business Machines Corporation" to "IBM Corporation". > Modules such as the below are not provided in these runtimes. > com.ibm.security.auth.module.JAASLoginModule -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18329) Add support for IBM Semeru OE JRE 11.0.15.0 and greater
[ https://issues.apache.org/jira/browse/HADOOP-18329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17645455#comment-17645455 ] ASF GitHub Bot commented on HADOOP-18329: - JackBuggins commented on code in PR #4537: URL: https://github.com/apache/hadoop/pull/4537#discussion_r1044749307 ## hadoop-common-project/hadoop-minikdc/src/test/java/org/apache/hadoop/minikdc/TestMiniKdc.java: ## @@ -38,8 +38,35 @@ import java.util.Arrays; public class TestMiniKdc extends KerberosSecurityTestcase { - private static final boolean IBM_JAVA = System.getProperty("java.vendor") - .contains("IBM"); + private static final boolean IBM_JAVA = shouldUseIbmPackages(); + // duplicated to avoid cycles in the build + private static boolean shouldUseIbmPackages() { +final List ibmTechnologyEditionSecurityModules = Arrays.asList( +"com.ibm.security.auth.module.JAASLoginModule", +"com.ibm.security.auth.module.Win64LoginModule", +"com.ibm.security.auth.module.NTLoginModule", +"com.ibm.security.auth.module.AIX64LoginModule", +"com.ibm.security.auth.module.LinuxLoginModule", +"com.ibm.security.auth.module.Krb5LoginModule" +); + +if (System.getProperty("java.vendor").contains("IBM")) { + return ibmTechnologyEditionSecurityModules + .stream().anyMatch((module) -> isSystemClassAvailable(module)); +} + +return false; + } + + private static boolean isSystemClassAvailable(String className) { Review Comment: No strong reasons, other than we shouldn't need the do privileged calls here in unit tests. I'm happy to make that change for consistency if preferred 👍🏼 > Add support for IBM Semeru OE JRE 11.0.15.0 and greater > --- > > Key: HADOOP-18329 > URL: https://issues.apache.org/jira/browse/HADOOP-18329 > Project: Hadoop Common > Issue Type: Bug > Components: auth, common >Affects Versions: 3.0.0, 3.1.0, 3.0.1, 3.2.0, 3.0.2, 3.1.1, 3.0.3, 3.3.0, > 3.1.2, 3.2.1, 3.1.3, 3.1.4, 3.2.2, 3.3.1, 3.2.3, 3.3.2, 3.3.3 > Environment: Running Hadoop (or Apache Spark 3.2.1 or above) on IBM > Semeru runtimes open edition 11.0.15.0 or greater. >Reporter: Jack >Priority: Major > Labels: pull-request-available > Original Estimate: 1h > Time Spent: 2.5h > Remaining Estimate: 0h > > There are checks within the PlatformName class that use the Vendor property > of the provided runtime JVM specifically looking for `IBM` within the name. > Whilst this check worked for IBM's [java technology > edition|https://www.ibm.com/docs/en/sdk-java-technology] it fails to work on > [Semeru|https://developer.ibm.com/languages/java/semeru-runtimes/] since > 11.0.15.0 due to the following change: > h4. java.vendor system property > In this release, the {{java.vendor}} system property has been changed from > "International Business Machines Corporation" to "IBM Corporation". > Modules such as the below are not provided in these runtimes. > com.ibm.security.auth.module.JAASLoginModule -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18329) Add support for IBM Semeru OE JRE 11.0.15.0 and greater
[ https://issues.apache.org/jira/browse/HADOOP-18329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17645460#comment-17645460 ] ASF GitHub Bot commented on HADOOP-18329: - JackBuggins commented on code in PR #4537: URL: https://github.com/apache/hadoop/pull/4537#discussion_r1044753951 ## hadoop-common-project/hadoop-auth/src/main/java/org/apache/hadoop/util/PlatformName.java: ## @@ -33,21 +37,72 @@ public class PlatformName { * per the java-vm. */ public static final String PLATFORM_NAME = - (System.getProperty("os.name").startsWith("Windows") - ? System.getenv("os") : System.getProperty("os.name")) - + "-" + System.getProperty("os.arch") - + "-" + System.getProperty("sun.arch.data.model"); + (System.getProperty("os.name").startsWith("Windows") ? + System.getenv("os") : System.getProperty("os.name")) + + "-" + System.getProperty("os.arch") + "-" + + System.getProperty("sun.arch.data.model"); /** * The java vendor name used in this platform. */ public static final String JAVA_VENDOR_NAME = System.getProperty("java.vendor"); + /** + * Define a system class accessor that is open to changes in underlying implementations + * of the system class loader modules. + */ + private static final class SystemClassAccessor extends ClassLoader { +public Class getSystemClass(String className) throws ClassNotFoundException { + return findSystemClass(className); +} + } + /** * A public static variable to indicate the current java vendor is - * IBM java or not. + * IBM and the type is Java Technology Edition which provides its + * own implementations of many security packages and Cipher suites. + * Note that these are not provided in Semeru runtimes: + * See https://developer.ibm.com/languages/java/semeru-runtimes/ + * The class used is present in any supported IBM JTE Runtimes. Review Comment: ```suggestion ``` > Add support for IBM Semeru OE JRE 11.0.15.0 and greater > --- > > Key: HADOOP-18329 > URL: https://issues.apache.org/jira/browse/HADOOP-18329 > Project: Hadoop Common > Issue Type: Bug > Components: auth, common >Affects Versions: 3.0.0, 3.1.0, 3.0.1, 3.2.0, 3.0.2, 3.1.1, 3.0.3, 3.3.0, > 3.1.2, 3.2.1, 3.1.3, 3.1.4, 3.2.2, 3.3.1, 3.2.3, 3.3.2, 3.3.3 > Environment: Running Hadoop (or Apache Spark 3.2.1 or above) on IBM > Semeru runtimes open edition 11.0.15.0 or greater. >Reporter: Jack >Priority: Major > Labels: pull-request-available > Original Estimate: 1h > Time Spent: 2.5h > Remaining Estimate: 0h > > There are checks within the PlatformName class that use the Vendor property > of the provided runtime JVM specifically looking for `IBM` within the name. > Whilst this check worked for IBM's [java technology > edition|https://www.ibm.com/docs/en/sdk-java-technology] it fails to work on > [Semeru|https://developer.ibm.com/languages/java/semeru-runtimes/] since > 11.0.15.0 due to the following change: > h4. java.vendor system property > In this release, the {{java.vendor}} system property has been changed from > "International Business Machines Corporation" to "IBM Corporation". > Modules such as the below are not provided in these runtimes. > com.ibm.security.auth.module.JAASLoginModule -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18329) Add support for IBM Semeru OE JRE 11.0.15.0 and greater
[ https://issues.apache.org/jira/browse/HADOOP-18329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17645462#comment-17645462 ] ASF GitHub Bot commented on HADOOP-18329: - JackBuggins commented on code in PR #4537: URL: https://github.com/apache/hadoop/pull/4537#discussion_r1044756952 ## hadoop-common-project/hadoop-auth/src/main/java/org/apache/hadoop/util/PlatformName.java: ## @@ -33,21 +37,71 @@ public class PlatformName { * per the java-vm. */ public static final String PLATFORM_NAME = - (System.getProperty("os.name").startsWith("Windows") - ? System.getenv("os") : System.getProperty("os.name")) - + "-" + System.getProperty("os.arch") - + "-" + System.getProperty("sun.arch.data.model"); + (System.getProperty("os.name").startsWith("Windows") ? + System.getenv("os") : System.getProperty("os.name")) + + "-" + System.getProperty("os.arch") + "-" + + System.getProperty("sun.arch.data.model"); /** * The java vendor name used in this platform. */ public static final String JAVA_VENDOR_NAME = System.getProperty("java.vendor"); + /** + * Define a system class accessor that is open to changes in underlying implementations + * of the system class loader modules. + */ + private static final class SystemClassAccessor extends ClassLoader { +public Class getSystemClass(String className) throws ClassNotFoundException { + return findSystemClass(className); +} + } + /** * A public static variable to indicate the current java vendor is - * IBM java or not. + * IBM and the type is Java Technology Edition which provides its + * own implementations of many security packages and Cipher suites. + * Note that these are not provided in Semeru runtimes: + * See https://developer.ibm.com/languages/java/semeru-runtimes/ Review Comment: ```suggestion * See https://developer.ibm.com/languages/java/semeru-runtimes for details. ``` > Add support for IBM Semeru OE JRE 11.0.15.0 and greater > --- > > Key: HADOOP-18329 > URL: https://issues.apache.org/jira/browse/HADOOP-18329 > Project: Hadoop Common > Issue Type: Bug > Components: auth, common >Affects Versions: 3.0.0, 3.1.0, 3.0.1, 3.2.0, 3.0.2, 3.1.1, 3.0.3, 3.3.0, > 3.1.2, 3.2.1, 3.1.3, 3.1.4, 3.2.2, 3.3.1, 3.2.3, 3.3.2, 3.3.3 > Environment: Running Hadoop (or Apache Spark 3.2.1 or above) on IBM > Semeru runtimes open edition 11.0.15.0 or greater. >Reporter: Jack >Priority: Major > Labels: pull-request-available > Original Estimate: 1h > Time Spent: 2.5h > Remaining Estimate: 0h > > There are checks within the PlatformName class that use the Vendor property > of the provided runtime JVM specifically looking for `IBM` within the name. > Whilst this check worked for IBM's [java technology > edition|https://www.ibm.com/docs/en/sdk-java-technology] it fails to work on > [Semeru|https://developer.ibm.com/languages/java/semeru-runtimes/] since > 11.0.15.0 due to the following change: > h4. java.vendor system property > In this release, the {{java.vendor}} system property has been changed from > "International Business Machines Corporation" to "IBM Corporation". > Modules such as the below are not provided in these runtimes. > com.ibm.security.auth.module.JAASLoginModule -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18329) Add support for IBM Semeru OE JRE 11.0.15.0 and greater
[ https://issues.apache.org/jira/browse/HADOOP-18329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17645513#comment-17645513 ] ASF GitHub Bot commented on HADOOP-18329: - hadoop-yetus commented on PR #4537: URL: https://github.com/apache/hadoop/pull/4537#issuecomment-1344888018 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reexec | 0m 55s | | Docker mode activated. | _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | | No case conflicting files found. | | +0 :ok: | codespell | 0m 0s | | codespell was not available. | | +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available. | | +0 :ok: | xmllint | 0m 0s | | xmllint was not available. | | +1 :green_heart: | @author | 0m 0s | | The patch does not contain any @author tags. | | +1 :green_heart: | test4tests | 0m 0s | | The patch appears to include 5 new or modified test files. | _ trunk Compile Tests _ | | +0 :ok: | mvndep | 15m 22s | | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 30m 16s | | trunk passed | | -1 :x: | compile | 22m 26s | [/branch-compile-root-jdkUbuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4537/13/artifact/out/branch-compile-root-jdkUbuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04.txt) | root in trunk failed with JDK Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04. | | +1 :green_heart: | compile | 26m 21s | | trunk passed with JDK Private Build-1.8.0_352-8u352-ga-1~20.04-b08 | | +1 :green_heart: | checkstyle | 1m 20s | | trunk passed | | +1 :green_heart: | mvnsite | 3m 25s | | trunk passed | | -1 :x: | javadoc | 1m 4s | [/branch-javadoc-hadoop-common-project_hadoop-common-jdkUbuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4537/13/artifact/out/branch-javadoc-hadoop-common-project_hadoop-common-jdkUbuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04.txt) | hadoop-common in trunk failed with JDK Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04. | | +1 :green_heart: | javadoc | 2m 15s | | trunk passed with JDK Private Build-1.8.0_352-8u352-ga-1~20.04-b08 | | +1 :green_heart: | spotbugs | 5m 23s | | trunk passed | | +1 :green_heart: | shadedclient | 23m 34s | | branch has no errors when building and testing our client artifacts. | _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 26s | | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 2m 2s | | the patch passed | | +1 :green_heart: | compile | 30m 10s | | the patch passed with JDK Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04 | | -1 :x: | javac | 30m 10s | [/results-compile-javac-root-jdkUbuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4537/13/artifact/out/results-compile-javac-root-jdkUbuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04.txt) | root-jdkUbuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04 with JDK Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04 generated 308 new + 2506 unchanged - 0 fixed = 2814 total (was 2506) | | +1 :green_heart: | compile | 25m 57s | | the patch passed with JDK Private Build-1.8.0_352-8u352-ga-1~20.04-b08 | | +1 :green_heart: | javac | 25m 57s | | the patch passed | | +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks issues. | | +1 :green_heart: | checkstyle | 1m 13s | | hadoop-common-project: The patch generated 0 new + 310 unchanged - 1 fixed = 310 total (was 311) | | +1 :green_heart: | mvnsite | 3m 24s | | the patch passed | | -1 :x: | javadoc | 1m 4s | [/patch-javadoc-hadoop-common-project_hadoop-common-jdkUbuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4537/13/artifact/out/patch-javadoc-hadoop-common-project_hadoop-common-jdkUbuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04.txt) | hadoop-common in the patch failed with JDK Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04. | | +1 :green_heart: | javadoc | 2m 17s | | the patch passed with JDK Private Build-1.8.0_352-8u352-ga-1~20.04-b08 | | +1 :green_heart: | spotbugs | 6m 8s | | the patch passed | | +1 :green_heart: | shadedclient | 23m 47s | | patch has no errors when building and testing our client artifacts. | _ Other Tests _ | | +1 :green_heart: | unit | 0m 37s | | hadoop-minikdc in the patch passed. | | +1 :green_heart: | unit | 3m 21s | | hadoop-auth in the patch passed. | | +1 :green_heart: | unit | 18m 46s | | hadoop-common in the patch passed. | | +1 :
[jira] [Commented] (HADOOP-18329) Add support for IBM Semeru OE JRE 11.0.15.0 and greater
[ https://issues.apache.org/jira/browse/HADOOP-18329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17645514#comment-17645514 ] ASF GitHub Bot commented on HADOOP-18329: - hadoop-yetus commented on PR #4537: URL: https://github.com/apache/hadoop/pull/4537#issuecomment-1344888606 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reexec | 0m 58s | | Docker mode activated. | _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | | No case conflicting files found. | | +0 :ok: | codespell | 0m 1s | | codespell was not available. | | +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available. | | +0 :ok: | xmllint | 0m 1s | | xmllint was not available. | | +1 :green_heart: | @author | 0m 0s | | The patch does not contain any @author tags. | | +1 :green_heart: | test4tests | 0m 0s | | The patch appears to include 5 new or modified test files. | _ trunk Compile Tests _ | | +0 :ok: | mvndep | 15m 23s | | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 30m 27s | | trunk passed | | +1 :green_heart: | compile | 30m 21s | | trunk passed with JDK Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04 | | +1 :green_heart: | compile | 25m 20s | | trunk passed with JDK Private Build-1.8.0_352-8u352-ga-1~20.04-b08 | | +1 :green_heart: | checkstyle | 1m 20s | | trunk passed | | +1 :green_heart: | mvnsite | 3m 24s | | trunk passed | | -1 :x: | javadoc | 1m 6s | [/branch-javadoc-hadoop-common-project_hadoop-common-jdkUbuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4537/12/artifact/out/branch-javadoc-hadoop-common-project_hadoop-common-jdkUbuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04.txt) | hadoop-common in trunk failed with JDK Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04. | | +1 :green_heart: | javadoc | 2m 14s | | trunk passed with JDK Private Build-1.8.0_352-8u352-ga-1~20.04-b08 | | +1 :green_heart: | spotbugs | 5m 21s | | trunk passed | | +1 :green_heart: | shadedclient | 23m 18s | | branch has no errors when building and testing our client artifacts. | _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 28s | | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 2m 12s | | the patch passed | | +1 :green_heart: | compile | 29m 31s | | the patch passed with JDK Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04 | | +1 :green_heart: | javac | 29m 31s | | the patch passed | | +1 :green_heart: | compile | 25m 31s | | the patch passed with JDK Private Build-1.8.0_352-8u352-ga-1~20.04-b08 | | +1 :green_heart: | javac | 25m 31s | | the patch passed | | +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks issues. | | +1 :green_heart: | checkstyle | 1m 11s | | hadoop-common-project: The patch generated 0 new + 310 unchanged - 1 fixed = 310 total (was 311) | | +1 :green_heart: | mvnsite | 3m 25s | | the patch passed | | -1 :x: | javadoc | 1m 3s | [/patch-javadoc-hadoop-common-project_hadoop-common-jdkUbuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4537/12/artifact/out/patch-javadoc-hadoop-common-project_hadoop-common-jdkUbuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04.txt) | hadoop-common in the patch failed with JDK Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04. | | +1 :green_heart: | javadoc | 2m 16s | | the patch passed with JDK Private Build-1.8.0_352-8u352-ga-1~20.04-b08 | | +1 :green_heart: | spotbugs | 6m 3s | | the patch passed | | +1 :green_heart: | shadedclient | 23m 57s | | patch has no errors when building and testing our client artifacts. | _ Other Tests _ | | +1 :green_heart: | unit | 0m 39s | | hadoop-minikdc in the patch passed. | | +1 :green_heart: | unit | 3m 21s | | hadoop-auth in the patch passed. | | +1 :green_heart: | unit | 18m 53s | | hadoop-common in the patch passed. | | +1 :green_heart: | unit | 1m 16s | | hadoop-registry in the patch passed. | | +1 :green_heart: | asflicense | 0m 52s | | The patch does not generate ASF License warnings. | | | | 267m 18s | | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4537/12/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hadoop/pull/4537 | | Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs che
[jira] [Commented] (HADOOP-18329) Add support for IBM Semeru OE JRE 11.0.15.0 and greater
[ https://issues.apache.org/jira/browse/HADOOP-18329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17645622#comment-17645622 ] ASF GitHub Bot commented on HADOOP-18329: - steveloughran commented on code in PR #4537: URL: https://github.com/apache/hadoop/pull/4537#discussion_r1045091286 ## hadoop-common-project/hadoop-minikdc/src/test/java/org/apache/hadoop/minikdc/TestMiniKdc.java: ## @@ -38,8 +38,35 @@ import java.util.Arrays; public class TestMiniKdc extends KerberosSecurityTestcase { - private static final boolean IBM_JAVA = System.getProperty("java.vendor") - .contains("IBM"); + private static final boolean IBM_JAVA = shouldUseIbmPackages(); + // duplicated to avoid cycles in the build + private static boolean shouldUseIbmPackages() { +final List ibmTechnologyEditionSecurityModules = Arrays.asList( +"com.ibm.security.auth.module.JAASLoginModule", +"com.ibm.security.auth.module.Win64LoginModule", +"com.ibm.security.auth.module.NTLoginModule", +"com.ibm.security.auth.module.AIX64LoginModule", +"com.ibm.security.auth.module.LinuxLoginModule", +"com.ibm.security.auth.module.Krb5LoginModule" +); + +if (System.getProperty("java.vendor").contains("IBM")) { + return ibmTechnologyEditionSecurityModules + .stream().anyMatch((module) -> isSystemClassAvailable(module)); +} + +return false; + } + + private static boolean isSystemClassAvailable(String className) { Review Comment: ahh, lets keep it simple for now > Add support for IBM Semeru OE JRE 11.0.15.0 and greater > --- > > Key: HADOOP-18329 > URL: https://issues.apache.org/jira/browse/HADOOP-18329 > Project: Hadoop Common > Issue Type: Bug > Components: auth, common >Affects Versions: 3.0.0, 3.1.0, 3.0.1, 3.2.0, 3.0.2, 3.1.1, 3.0.3, 3.3.0, > 3.1.2, 3.2.1, 3.1.3, 3.1.4, 3.2.2, 3.3.1, 3.2.3, 3.3.2, 3.3.3 > Environment: Running Hadoop (or Apache Spark 3.2.1 or above) on IBM > Semeru runtimes open edition 11.0.15.0 or greater. >Reporter: Jack >Priority: Major > Labels: pull-request-available > Original Estimate: 1h > Time Spent: 2.5h > Remaining Estimate: 0h > > There are checks within the PlatformName class that use the Vendor property > of the provided runtime JVM specifically looking for `IBM` within the name. > Whilst this check worked for IBM's [java technology > edition|https://www.ibm.com/docs/en/sdk-java-technology] it fails to work on > [Semeru|https://developer.ibm.com/languages/java/semeru-runtimes/] since > 11.0.15.0 due to the following change: > h4. java.vendor system property > In this release, the {{java.vendor}} system property has been changed from > "International Business Machines Corporation" to "IBM Corporation". > Modules such as the below are not provided in these runtimes. > com.ibm.security.auth.module.JAASLoginModule -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18329) Add support for IBM Semeru OE JRE 11.0.15.0 and greater
[ https://issues.apache.org/jira/browse/HADOOP-18329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17645626#comment-17645626 ] ASF GitHub Bot commented on HADOOP-18329: - steveloughran merged PR #4537: URL: https://github.com/apache/hadoop/pull/4537 > Add support for IBM Semeru OE JRE 11.0.15.0 and greater > --- > > Key: HADOOP-18329 > URL: https://issues.apache.org/jira/browse/HADOOP-18329 > Project: Hadoop Common > Issue Type: Bug > Components: auth, common >Affects Versions: 3.0.0, 3.1.0, 3.0.1, 3.2.0, 3.0.2, 3.1.1, 3.0.3, 3.3.0, > 3.1.2, 3.2.1, 3.1.3, 3.1.4, 3.2.2, 3.3.1, 3.2.3, 3.3.2, 3.3.3 > Environment: Running Hadoop (or Apache Spark 3.2.1 or above) on IBM > Semeru runtimes open edition 11.0.15.0 or greater. >Reporter: Jack >Priority: Major > Labels: pull-request-available > Original Estimate: 1h > Time Spent: 2.5h > Remaining Estimate: 0h > > There are checks within the PlatformName class that use the Vendor property > of the provided runtime JVM specifically looking for `IBM` within the name. > Whilst this check worked for IBM's [java technology > edition|https://www.ibm.com/docs/en/sdk-java-technology] it fails to work on > [Semeru|https://developer.ibm.com/languages/java/semeru-runtimes/] since > 11.0.15.0 due to the following change: > h4. java.vendor system property > In this release, the {{java.vendor}} system property has been changed from > "International Business Machines Corporation" to "IBM Corporation". > Modules such as the below are not provided in these runtimes. > com.ibm.security.auth.module.JAASLoginModule -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18329) Add support for IBM Semeru OE JRE 11.0.15.0 and greater
[ https://issues.apache.org/jira/browse/HADOOP-18329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17645629#comment-17645629 ] ASF GitHub Bot commented on HADOOP-18329: - steveloughran commented on PR #4537: URL: https://github.com/apache/hadoop/pull/4537#issuecomment-1345277471 @JackBuggins merged to trunk, but it has merge conflicts into 3.3 with SSLFactory.java. can you do a quick review and PR there? I'm busy testing the abfs blockers hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/ssl/SSLFactory.java ``` "ssl.server.exclude.cipher.list"; <<< HEAD public static final String SSLCERTIFICATE = IBM_JAVA?"ibmX509":"SunX509"; === public static final String KEY_MANAGER_SSLCERTIFICATE = IBM_JAVA ? "ibmX509" : KeyManagerFactory.getDefaultAlgorithm(); public static final String TRUST_MANAGER_SSLCERTIFICATE = IBM_JAVA ? "ibmX509" : TrustManagerFactory.getDefaultAlgorithm(); >>> a46b20d25f1 (HADOOP-18329. Support for IBM Semeru JVM > 11.0.15.0 Vendor Name Changes (#4537)) ``` > Add support for IBM Semeru OE JRE 11.0.15.0 and greater > --- > > Key: HADOOP-18329 > URL: https://issues.apache.org/jira/browse/HADOOP-18329 > Project: Hadoop Common > Issue Type: Bug > Components: auth, common >Affects Versions: 3.0.0, 3.1.0, 3.0.1, 3.2.0, 3.0.2, 3.1.1, 3.0.3, 3.3.0, > 3.1.2, 3.2.1, 3.1.3, 3.1.4, 3.2.2, 3.3.1, 3.2.3, 3.3.2, 3.3.3 > Environment: Running Hadoop (or Apache Spark 3.2.1 or above) on IBM > Semeru runtimes open edition 11.0.15.0 or greater. >Reporter: Jack >Priority: Major > Labels: pull-request-available > Original Estimate: 1h > Time Spent: 2.5h > Remaining Estimate: 0h > > There are checks within the PlatformName class that use the Vendor property > of the provided runtime JVM specifically looking for `IBM` within the name. > Whilst this check worked for IBM's [java technology > edition|https://www.ibm.com/docs/en/sdk-java-technology] it fails to work on > [Semeru|https://developer.ibm.com/languages/java/semeru-runtimes/] since > 11.0.15.0 due to the following change: > h4. java.vendor system property > In this release, the {{java.vendor}} system property has been changed from > "International Business Machines Corporation" to "IBM Corporation". > Modules such as the below are not provided in these runtimes. > com.ibm.security.auth.module.JAASLoginModule -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18329) Add support for IBM Semeru OE JRE 11.0.15.0 and greater
[ https://issues.apache.org/jira/browse/HADOOP-18329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17646103#comment-17646103 ] ASF GitHub Bot commented on HADOOP-18329: - JackBuggins opened a new pull request, #5208: URL: https://github.com/apache/hadoop/pull/5208 ### Description of PR Applies patches provided in to branch-3.3 via cherry-picking a merge commit of the proposed changes, eg. ``` git cherry-pick a46b20d25f12dfb6af1d89c6bd8e39220cc8c928 -m 1 ``` The original change request can be found at https://github.com/apache/hadoop/pull/4537 --- There are checks within the PlatformName class that use the Vendor property of the provided runtime JVM specifically looking for `IBM` within the name. Whilst this check worked for IBM's [java technology edition](https://www.ibm.com/docs/en/sdk-java-technology) it fails to work on [Semeru](https://developer.ibm.com/languages/java/semeru-runtimes/) since 11.0.15.0 due to the following change: **java.vendor system property** In this release, the java.vendor system property has been changed from "International Business Machines Corporation" to "IBM Corporation". Modules such as the below are not provided in these runtimes. com.ibm.security.auth.module.JAASLoginModule This change attempts to use reflection to ensure that a class common to IBM JT runtimes exists, extending upon the vendor check, since IBM vendored JVM's may not actually require special logic to use custom security modules. The same 3.3.3 versions were working correctly until the vendor name change was observed during routine upgrades by internal CI. ### How was this patch tested? CI + Unit test ### How was this patch tested? - C.I ### For code changes: - [x] Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')? - [ ] Object storage: have the integration tests been executed and the endpoint declared according to the connector-specific documentation? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the `LICENSE`, `LICENSE-binary`, `NOTICE-binary` files? > Add support for IBM Semeru OE JRE 11.0.15.0 and greater > --- > > Key: HADOOP-18329 > URL: https://issues.apache.org/jira/browse/HADOOP-18329 > Project: Hadoop Common > Issue Type: Bug > Components: auth, common >Affects Versions: 3.0.0, 3.1.0, 3.0.1, 3.2.0, 3.0.2, 3.1.1, 3.0.3, 3.3.0, > 3.1.2, 3.2.1, 3.1.3, 3.1.4, 3.2.2, 3.3.1, 3.2.3, 3.3.2, 3.3.3 > Environment: Running Hadoop (or Apache Spark 3.2.1 or above) on IBM > Semeru runtimes open edition 11.0.15.0 or greater. >Reporter: Jack >Priority: Major > Labels: pull-request-available > Original Estimate: 1h > Time Spent: 2.5h > Remaining Estimate: 0h > > There are checks within the PlatformName class that use the Vendor property > of the provided runtime JVM specifically looking for `IBM` within the name. > Whilst this check worked for IBM's [java technology > edition|https://www.ibm.com/docs/en/sdk-java-technology] it fails to work on > [Semeru|https://developer.ibm.com/languages/java/semeru-runtimes/] since > 11.0.15.0 due to the following change: > h4. java.vendor system property > In this release, the {{java.vendor}} system property has been changed from > "International Business Machines Corporation" to "IBM Corporation". > Modules such as the below are not provided in these runtimes. > com.ibm.security.auth.module.JAASLoginModule -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18329) Add support for IBM Semeru OE JRE 11.0.15.0 and greater
[ https://issues.apache.org/jira/browse/HADOOP-18329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17646156#comment-17646156 ] ASF GitHub Bot commented on HADOOP-18329: - JackBuggins commented on PR #4537: URL: https://github.com/apache/hadoop/pull/4537#issuecomment-1346679972 @steveloughran I've popped up a PR against branch 3.3; should I do the same for 3.3.5? > Add support for IBM Semeru OE JRE 11.0.15.0 and greater > --- > > Key: HADOOP-18329 > URL: https://issues.apache.org/jira/browse/HADOOP-18329 > Project: Hadoop Common > Issue Type: Bug > Components: auth, common >Affects Versions: 3.0.0, 3.1.0, 3.0.1, 3.2.0, 3.0.2, 3.1.1, 3.0.3, 3.3.0, > 3.1.2, 3.2.1, 3.1.3, 3.1.4, 3.2.2, 3.3.1, 3.2.3, 3.3.2, 3.3.3 > Environment: Running Hadoop (or Apache Spark 3.2.1 or above) on IBM > Semeru runtimes open edition 11.0.15.0 or greater. >Reporter: Jack >Priority: Major > Labels: pull-request-available > Original Estimate: 1h > Time Spent: 2.5h > Remaining Estimate: 0h > > There are checks within the PlatformName class that use the Vendor property > of the provided runtime JVM specifically looking for `IBM` within the name. > Whilst this check worked for IBM's [java technology > edition|https://www.ibm.com/docs/en/sdk-java-technology] it fails to work on > [Semeru|https://developer.ibm.com/languages/java/semeru-runtimes/] since > 11.0.15.0 due to the following change: > h4. java.vendor system property > In this release, the {{java.vendor}} system property has been changed from > "International Business Machines Corporation" to "IBM Corporation". > Modules such as the below are not provided in these runtimes. > com.ibm.security.auth.module.JAASLoginModule -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18329) Add support for IBM Semeru OE JRE 11.0.15.0 and greater
[ https://issues.apache.org/jira/browse/HADOOP-18329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17646195#comment-17646195 ] ASF GitHub Bot commented on HADOOP-18329: - hadoop-yetus commented on PR #5208: URL: https://github.com/apache/hadoop/pull/5208#issuecomment-1346830962 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reexec | 10m 21s | | Docker mode activated. | _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | | No case conflicting files found. | | +0 :ok: | codespell | 0m 0s | | codespell was not available. | | +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available. | | +1 :green_heart: | @author | 0m 0s | | The patch does not contain any @author tags. | | +1 :green_heart: | test4tests | 0m 0s | | The patch appears to include 1 new or modified test files. | _ branch-3.3 Compile Tests _ | | +0 :ok: | mvndep | 15m 31s | | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 26m 48s | | branch-3.3 passed | | +1 :green_heart: | compile | 18m 46s | | branch-3.3 passed | | +1 :green_heart: | checkstyle | 1m 0s | | branch-3.3 passed | | +1 :green_heart: | mvnsite | 1m 7s | | branch-3.3 passed | | +1 :green_heart: | javadoc | 1m 4s | | branch-3.3 passed | | +1 :green_heart: | spotbugs | 1m 39s | | branch-3.3 passed | | +1 :green_heart: | shadedclient | 25m 22s | | branch has no errors when building and testing our client artifacts. | _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 25s | | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 0m 37s | | the patch passed | | +1 :green_heart: | compile | 17m 54s | | the patch passed | | +1 :green_heart: | javac | 17m 54s | | the patch passed | | +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks issues. | | +1 :green_heart: | checkstyle | 0m 54s | | hadoop-common-project: The patch generated 0 new + 3 unchanged - 1 fixed = 3 total (was 4) | | +1 :green_heart: | mvnsite | 1m 6s | | the patch passed | | +1 :green_heart: | javadoc | 0m 57s | | the patch passed | | +1 :green_heart: | spotbugs | 1m 48s | | the patch passed | | +1 :green_heart: | shadedclient | 25m 37s | | patch has no errors when building and testing our client artifacts. | _ Other Tests _ | | +1 :green_heart: | unit | 0m 37s | | hadoop-minikdc in the patch passed. | | +1 :green_heart: | unit | 3m 34s | | hadoop-auth in the patch passed. | | +1 :green_heart: | asflicense | 0m 50s | | The patch does not generate ASF License warnings. | | | | 158m 35s | | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5208/1/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hadoop/pull/5208 | | Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets | | uname | Linux b3f0d71badd5 4.15.0-200-generic #211-Ubuntu SMP Thu Nov 24 18:16:04 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/bin/hadoop.sh | | git revision | branch-3.3 / 3774de951867c1c2250851d6eeb21b0ef239e051 | | Default Java | Private Build-1.8.0_352-8u352-ga-1~18.04-b08 | | Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5208/1/testReport/ | | Max. process+thread count | 527 (vs. ulimit of 5500) | | modules | C: hadoop-common-project/hadoop-minikdc hadoop-common-project/hadoop-auth U: hadoop-common-project | | Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5208/1/console | | versions | git=2.17.1 maven=3.6.0 spotbugs=4.2.2 | | Powered by | Apache Yetus 0.14.0 https://yetus.apache.org | This message was automatically generated. > Add support for IBM Semeru OE JRE 11.0.15.0 and greater > --- > > Key: HADOOP-18329 > URL: https://issues.apache.org/jira/browse/HADOOP-18329 > Project: Hadoop Common > Issue Type: Bug > Components: auth, common >Affects Versions: 3.0.0, 3.1.0, 3.0.1, 3.2.0, 3.0.2, 3.1.1, 3.0.3, 3.3.0, > 3.1.2, 3.2.1, 3.1.3, 3.1.4, 3.2.2, 3.3.1, 3.2.3, 3.3.2, 3.3.3 > Environment: Running Hadoop (or Apache Spark 3.2.1 or above) on IBM > Semeru runtimes open edition 11.0.15.0 or greater. >Reporter: Jack >
[jira] [Commented] (HADOOP-18329) Add support for IBM Semeru OE JRE 11.0.15.0 and greater
[ https://issues.apache.org/jira/browse/HADOOP-18329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17646230#comment-17646230 ] ASF GitHub Bot commented on HADOOP-18329: - steveloughran commented on PR #4537: URL: https://github.com/apache/hadoop/pull/4537#issuecomment-1346920543 get it into 3.3 and i will pull to 3.3.5, they are almost identical. if there *are* merge problems again, then we can worry about it > Add support for IBM Semeru OE JRE 11.0.15.0 and greater > --- > > Key: HADOOP-18329 > URL: https://issues.apache.org/jira/browse/HADOOP-18329 > Project: Hadoop Common > Issue Type: Bug > Components: auth, common >Affects Versions: 3.0.0, 3.1.0, 3.0.1, 3.2.0, 3.0.2, 3.1.1, 3.0.3, 3.3.0, > 3.1.2, 3.2.1, 3.1.3, 3.1.4, 3.2.2, 3.3.1, 3.2.3, 3.3.2, 3.3.3 > Environment: Running Hadoop (or Apache Spark 3.2.1 or above) on IBM > Semeru runtimes open edition 11.0.15.0 or greater. >Reporter: Jack >Priority: Major > Labels: pull-request-available > Original Estimate: 1h > Time Spent: 2.5h > Remaining Estimate: 0h > > There are checks within the PlatformName class that use the Vendor property > of the provided runtime JVM specifically looking for `IBM` within the name. > Whilst this check worked for IBM's [java technology > edition|https://www.ibm.com/docs/en/sdk-java-technology] it fails to work on > [Semeru|https://developer.ibm.com/languages/java/semeru-runtimes/] since > 11.0.15.0 due to the following change: > h4. java.vendor system property > In this release, the {{java.vendor}} system property has been changed from > "International Business Machines Corporation" to "IBM Corporation". > Modules such as the below are not provided in these runtimes. > com.ibm.security.auth.module.JAASLoginModule -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18329) Add support for IBM Semeru OE JRE 11.0.15.0 and greater
[ https://issues.apache.org/jira/browse/HADOOP-18329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17646235#comment-17646235 ] ASF GitHub Bot commented on HADOOP-18329: - steveloughran merged PR #5208: URL: https://github.com/apache/hadoop/pull/5208 > Add support for IBM Semeru OE JRE 11.0.15.0 and greater > --- > > Key: HADOOP-18329 > URL: https://issues.apache.org/jira/browse/HADOOP-18329 > Project: Hadoop Common > Issue Type: Bug > Components: auth, common >Affects Versions: 3.0.0, 3.1.0, 3.0.1, 3.2.0, 3.0.2, 3.1.1, 3.0.3, 3.3.0, > 3.1.2, 3.2.1, 3.1.3, 3.1.4, 3.2.2, 3.3.1, 3.2.3, 3.3.2, 3.3.3 > Environment: Running Hadoop (or Apache Spark 3.2.1 or above) on IBM > Semeru runtimes open edition 11.0.15.0 or greater. >Reporter: Jack >Priority: Major > Labels: pull-request-available > Original Estimate: 1h > Time Spent: 2.5h > Remaining Estimate: 0h > > There are checks within the PlatformName class that use the Vendor property > of the provided runtime JVM specifically looking for `IBM` within the name. > Whilst this check worked for IBM's [java technology > edition|https://www.ibm.com/docs/en/sdk-java-technology] it fails to work on > [Semeru|https://developer.ibm.com/languages/java/semeru-runtimes/] since > 11.0.15.0 due to the following change: > h4. java.vendor system property > In this release, the {{java.vendor}} system property has been changed from > "International Business Machines Corporation" to "IBM Corporation". > Modules such as the below are not provided in these runtimes. > com.ibm.security.auth.module.JAASLoginModule -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18329) Add support for IBM Semeru OE JRE 11.0.15.0 and greater
[ https://issues.apache.org/jira/browse/HADOOP-18329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17583470#comment-17583470 ] ASF GitHub Bot commented on HADOOP-18329: - steveloughran commented on code in PR #4537: URL: https://github.com/apache/hadoop/pull/4537#discussion_r952388846 ## hadoop-common-project/hadoop-auth/src/main/java/org/apache/hadoop/util/PlatformName.java: ## @@ -45,9 +45,23 @@ public class PlatformName { /** * A public static variable to indicate the current java vendor is - * IBM java or not. + * IBM and the type is Java Technology Edition which provides its + * own implementations of many security packages and Cipher suites. + * Note that these are not provided in Semeru runtimes: + * See https://developer.ibm.com/languages/java/semeru-runtimes/ + * The class used is present in any supported IBM JTE Runtimes. */ - public static final boolean IBM_JAVA = JAVA_VENDOR_NAME.contains("IBM"); + public static final boolean IBM_JAVA = JAVA_VENDOR_NAME.contains("IBM") && +hasClass("com.ibm.security.auth.module.JAASLoginModule"); + + private static boolean hasClass(String className) { +try { + Thread.currentThread().getContextClassLoader().loadClass(className); + return true; +} catch(ClassNotFoundException ignored) { Review Comment: can any other exception get raised here? if so, best to log and downgrade to false too > Add support for IBM Semeru OE JRE 11.0.15.0 and greater > --- > > Key: HADOOP-18329 > URL: https://issues.apache.org/jira/browse/HADOOP-18329 > Project: Hadoop Common > Issue Type: Bug > Components: auth, common >Affects Versions: 3.0.0, 3.1.0, 3.0.1, 3.2.0, 3.0.2, 3.1.1, 3.0.3, 3.3.0, > 3.1.2, 3.2.1, 3.1.3, 3.1.4, 3.2.2, 3.3.1, 3.2.3, 3.3.2, 3.3.3 > Environment: Running Hadoop (or Apache Spark 3.2.1 or above) on IBM > Semeru runtimes open edition 11.0.15.0 or greater. >Reporter: Jack >Priority: Major > Labels: pull-request-available > Original Estimate: 1h > Time Spent: 2.5h > Remaining Estimate: 0h > > There are checks within the PlatformName class that use the Vendor property > of the provided runtime JVM specifically looking for `IBM` within the name. > Whilst this check worked for IBM's [java technology > edition|https://www.ibm.com/docs/en/sdk-java-technology] it fails to work on > [Semeru|https://developer.ibm.com/languages/java/semeru-runtimes/] since > 11.0.15.0 due to the following change: > h4. java.vendor system property > In this release, the {{java.vendor}} system property has been changed from > "International Business Machines Corporation" to "IBM Corporation". > Modules such as the below are not provided in these runtimes. > com.ibm.security.auth.module.JAASLoginModule -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18329) Add support for IBM Semeru OE JRE 11.0.15.0 and greater
[ https://issues.apache.org/jira/browse/HADOOP-18329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17584138#comment-17584138 ] ASF GitHub Bot commented on HADOOP-18329: - JackBuggins commented on code in PR #4537: URL: https://github.com/apache/hadoop/pull/4537#discussion_r953550055 ## hadoop-common-project/hadoop-auth/src/main/java/org/apache/hadoop/util/PlatformName.java: ## @@ -45,9 +45,23 @@ public class PlatformName { /** * A public static variable to indicate the current java vendor is - * IBM java or not. + * IBM and the type is Java Technology Edition which provides its + * own implementations of many security packages and Cipher suites. + * Note that these are not provided in Semeru runtimes: + * See https://developer.ibm.com/languages/java/semeru-runtimes/ + * The class used is present in any supported IBM JTE Runtimes. */ - public static final boolean IBM_JAVA = JAVA_VENDOR_NAME.contains("IBM"); + public static final boolean IBM_JAVA = JAVA_VENDOR_NAME.contains("IBM") && +hasClass("com.ibm.security.auth.module.JAASLoginModule"); + + private static boolean hasClass(String className) { +try { + Thread.currentThread().getContextClassLoader().loadClass(className); + return true; +} catch(ClassNotFoundException ignored) { Review Comment: Thanks @steveloughran - Would you suggest we are catching the generic Exception instead? > Add support for IBM Semeru OE JRE 11.0.15.0 and greater > --- > > Key: HADOOP-18329 > URL: https://issues.apache.org/jira/browse/HADOOP-18329 > Project: Hadoop Common > Issue Type: Bug > Components: auth, common >Affects Versions: 3.0.0, 3.1.0, 3.0.1, 3.2.0, 3.0.2, 3.1.1, 3.0.3, 3.3.0, > 3.1.2, 3.2.1, 3.1.3, 3.1.4, 3.2.2, 3.3.1, 3.2.3, 3.3.2, 3.3.3 > Environment: Running Hadoop (or Apache Spark 3.2.1 or above) on IBM > Semeru runtimes open edition 11.0.15.0 or greater. >Reporter: Jack >Priority: Major > Labels: pull-request-available > Original Estimate: 1h > Time Spent: 2.5h > Remaining Estimate: 0h > > There are checks within the PlatformName class that use the Vendor property > of the provided runtime JVM specifically looking for `IBM` within the name. > Whilst this check worked for IBM's [java technology > edition|https://www.ibm.com/docs/en/sdk-java-technology] it fails to work on > [Semeru|https://developer.ibm.com/languages/java/semeru-runtimes/] since > 11.0.15.0 due to the following change: > h4. java.vendor system property > In this release, the {{java.vendor}} system property has been changed from > "International Business Machines Corporation" to "IBM Corporation". > Modules such as the below are not provided in these runtimes. > com.ibm.security.auth.module.JAASLoginModule -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18329) Add support for IBM Semeru OE JRE 11.0.15.0 and greater
[ https://issues.apache.org/jira/browse/HADOOP-18329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17584152#comment-17584152 ] ASF GitHub Bot commented on HADOOP-18329: - steveloughran commented on code in PR #4537: URL: https://github.com/apache/hadoop/pull/4537#discussion_r953589723 ## hadoop-common-project/hadoop-auth/src/main/java/org/apache/hadoop/util/PlatformName.java: ## @@ -45,9 +45,23 @@ public class PlatformName { /** * A public static variable to indicate the current java vendor is - * IBM java or not. + * IBM and the type is Java Technology Edition which provides its + * own implementations of many security packages and Cipher suites. + * Note that these are not provided in Semeru runtimes: + * See https://developer.ibm.com/languages/java/semeru-runtimes/ + * The class used is present in any supported IBM JTE Runtimes. */ - public static final boolean IBM_JAVA = JAVA_VENDOR_NAME.contains("IBM"); + public static final boolean IBM_JAVA = JAVA_VENDOR_NAME.contains("IBM") && +hasClass("com.ibm.security.auth.module.JAASLoginModule"); + + private static boolean hasClass(String className) { +try { + Thread.currentThread().getContextClassLoader().loadClass(className); + return true; +} catch(ClassNotFoundException ignored) { Review Comment: yes > Add support for IBM Semeru OE JRE 11.0.15.0 and greater > --- > > Key: HADOOP-18329 > URL: https://issues.apache.org/jira/browse/HADOOP-18329 > Project: Hadoop Common > Issue Type: Bug > Components: auth, common >Affects Versions: 3.0.0, 3.1.0, 3.0.1, 3.2.0, 3.0.2, 3.1.1, 3.0.3, 3.3.0, > 3.1.2, 3.2.1, 3.1.3, 3.1.4, 3.2.2, 3.3.1, 3.2.3, 3.3.2, 3.3.3 > Environment: Running Hadoop (or Apache Spark 3.2.1 or above) on IBM > Semeru runtimes open edition 11.0.15.0 or greater. >Reporter: Jack >Priority: Major > Labels: pull-request-available > Original Estimate: 1h > Time Spent: 2.5h > Remaining Estimate: 0h > > There are checks within the PlatformName class that use the Vendor property > of the provided runtime JVM specifically looking for `IBM` within the name. > Whilst this check worked for IBM's [java technology > edition|https://www.ibm.com/docs/en/sdk-java-technology] it fails to work on > [Semeru|https://developer.ibm.com/languages/java/semeru-runtimes/] since > 11.0.15.0 due to the following change: > h4. java.vendor system property > In this release, the {{java.vendor}} system property has been changed from > "International Business Machines Corporation" to "IBM Corporation". > Modules such as the below are not provided in these runtimes. > com.ibm.security.auth.module.JAASLoginModule -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18329) Add support for IBM Semeru OE JRE 11.0.15.0 and greater
[ https://issues.apache.org/jira/browse/HADOOP-18329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17656655#comment-17656655 ] ASF GitHub Bot commented on HADOOP-18329: - ivrisivris commented on PR #4537: URL: https://github.com/apache/hadoop/pull/4537#issuecomment-1377379603 Hi, thank you for solving this problem. Is there any plan when 3.3.5 will be released? Thanks fo the reply Best regards Kamil > Add support for IBM Semeru OE JRE 11.0.15.0 and greater > --- > > Key: HADOOP-18329 > URL: https://issues.apache.org/jira/browse/HADOOP-18329 > Project: Hadoop Common > Issue Type: Bug > Components: auth, common >Affects Versions: 3.0.0, 3.1.0, 3.0.1, 3.2.0, 3.0.2, 3.1.1, 3.0.3, 3.3.0, > 3.1.2, 3.2.1, 3.1.3, 3.1.4, 3.2.2, 3.3.1, 3.2.3, 3.3.2, 3.3.3 > Environment: Running Hadoop (or Apache Spark 3.2.1 or above) on IBM > Semeru runtimes open edition 11.0.15.0 or greater. >Reporter: Jack >Assignee: Jack >Priority: Major > Labels: pull-request-available > Fix For: 3.4.0, 3.3.5 > > Original Estimate: 1h > Time Spent: 2.5h > Remaining Estimate: 0h > > There are checks within the PlatformName class that use the Vendor property > of the provided runtime JVM specifically looking for `IBM` within the name. > Whilst this check worked for IBM's [java technology > edition|https://www.ibm.com/docs/en/sdk-java-technology] it fails to work on > [Semeru|https://developer.ibm.com/languages/java/semeru-runtimes/] since > 11.0.15.0 due to the following change: > h4. java.vendor system property > In this release, the {{java.vendor}} system property has been changed from > "International Business Machines Corporation" to "IBM Corporation". > Modules such as the below are not provided in these runtimes. > com.ibm.security.auth.module.JAASLoginModule -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18329) Add support for IBM Semeru OE JRE 11.0.15.0 and greater
[ https://issues.apache.org/jira/browse/HADOOP-18329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17656772#comment-17656772 ] ASF GitHub Bot commented on HADOOP-18329: - JackBuggins commented on PR #4537: URL: https://github.com/apache/hadoop/pull/4537#issuecomment-1377805782 @ivrisivris Sorry, I don't know the answer to that one, but there is a workaround you might consider in the meantime which makes use of an agent to manipulate the vendor name - you can see an example [here](https://github.com/eclipse-openj9/openj9/issues/14950#issuecomment-1105797810) > Add support for IBM Semeru OE JRE 11.0.15.0 and greater > --- > > Key: HADOOP-18329 > URL: https://issues.apache.org/jira/browse/HADOOP-18329 > Project: Hadoop Common > Issue Type: Bug > Components: auth, common >Affects Versions: 3.0.0, 3.1.0, 3.0.1, 3.2.0, 3.0.2, 3.1.1, 3.0.3, 3.3.0, > 3.1.2, 3.2.1, 3.1.3, 3.1.4, 3.2.2, 3.3.1, 3.2.3, 3.3.2, 3.3.3 > Environment: Running Hadoop (or Apache Spark 3.2.1 or above) on IBM > Semeru runtimes open edition 11.0.15.0 or greater. >Reporter: Jack >Assignee: Jack >Priority: Major > Labels: pull-request-available > Fix For: 3.4.0, 3.3.5 > > Original Estimate: 1h > Time Spent: 2.5h > Remaining Estimate: 0h > > There are checks within the PlatformName class that use the Vendor property > of the provided runtime JVM specifically looking for `IBM` within the name. > Whilst this check worked for IBM's [java technology > edition|https://www.ibm.com/docs/en/sdk-java-technology] it fails to work on > [Semeru|https://developer.ibm.com/languages/java/semeru-runtimes/] since > 11.0.15.0 due to the following change: > h4. java.vendor system property > In this release, the {{java.vendor}} system property has been changed from > "International Business Machines Corporation" to "IBM Corporation". > Modules such as the below are not provided in these runtimes. > com.ibm.security.auth.module.JAASLoginModule -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18329) Add support for IBM Semeru OE JRE 11.0.15.0 and greater
[ https://issues.apache.org/jira/browse/HADOOP-18329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17666250#comment-17666250 ] ASF GitHub Bot commented on HADOOP-18329: - ivrisivris commented on PR #4537: URL: https://github.com/apache/hadoop/pull/4537#issuecomment-1378370099 @JackBuggins Thank you for responding so quickly. > Add support for IBM Semeru OE JRE 11.0.15.0 and greater > --- > > Key: HADOOP-18329 > URL: https://issues.apache.org/jira/browse/HADOOP-18329 > Project: Hadoop Common > Issue Type: Bug > Components: auth, common >Affects Versions: 3.0.0, 3.1.0, 3.0.1, 3.2.0, 3.0.2, 3.1.1, 3.0.3, 3.3.0, > 3.1.2, 3.2.1, 3.1.3, 3.1.4, 3.2.2, 3.3.1, 3.2.3, 3.3.2, 3.3.3 > Environment: Running Hadoop (or Apache Spark 3.2.1 or above) on IBM > Semeru runtimes open edition 11.0.15.0 or greater. >Reporter: Jack >Assignee: Jack >Priority: Major > Labels: pull-request-available > Fix For: 3.4.0, 3.3.5 > > Original Estimate: 1h > Time Spent: 2.5h > Remaining Estimate: 0h > > There are checks within the PlatformName class that use the Vendor property > of the provided runtime JVM specifically looking for `IBM` within the name. > Whilst this check worked for IBM's [java technology > edition|https://www.ibm.com/docs/en/sdk-java-technology] it fails to work on > [Semeru|https://developer.ibm.com/languages/java/semeru-runtimes/] since > 11.0.15.0 due to the following change: > h4. java.vendor system property > In this release, the {{java.vendor}} system property has been changed from > "International Business Machines Corporation" to "IBM Corporation". > Modules such as the below are not provided in these runtimes. > com.ibm.security.auth.module.JAASLoginModule -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18329) Add support for IBM Semeru OE JRE 11.0.15.0 and greater
[ https://issues.apache.org/jira/browse/HADOOP-18329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17676733#comment-17676733 ] ASF GitHub Bot commented on HADOOP-18329: - steveloughran commented on PR #4537: URL: https://github.com/apache/hadoop/pull/4537#issuecomment-1382265647 The 3.3.5 RC0 is up for testing. Grab it from the Hadoop web site and make sure it works for you. Do not wait until the final release-as if that is broken or will still take a while for a patched release to ship. > Add support for IBM Semeru OE JRE 11.0.15.0 and greater > --- > > Key: HADOOP-18329 > URL: https://issues.apache.org/jira/browse/HADOOP-18329 > Project: Hadoop Common > Issue Type: Bug > Components: auth, common >Affects Versions: 3.0.0, 3.1.0, 3.0.1, 3.2.0, 3.0.2, 3.1.1, 3.0.3, 3.3.0, > 3.1.2, 3.2.1, 3.1.3, 3.1.4, 3.2.2, 3.3.1, 3.2.3, 3.3.2, 3.3.3 > Environment: Running Hadoop (or Apache Spark 3.2.1 or above) on IBM > Semeru runtimes open edition 11.0.15.0 or greater. >Reporter: Jack >Assignee: Jack >Priority: Major > Labels: pull-request-available > Fix For: 3.4.0, 3.3.5 > > Original Estimate: 1h > Time Spent: 2.5h > Remaining Estimate: 0h > > There are checks within the PlatformName class that use the Vendor property > of the provided runtime JVM specifically looking for `IBM` within the name. > Whilst this check worked for IBM's [java technology > edition|https://www.ibm.com/docs/en/sdk-java-technology] it fails to work on > [Semeru|https://developer.ibm.com/languages/java/semeru-runtimes/] since > 11.0.15.0 due to the following change: > h4. java.vendor system property > In this release, the {{java.vendor}} system property has been changed from > "International Business Machines Corporation" to "IBM Corporation". > Modules such as the below are not provided in these runtimes. > com.ibm.security.auth.module.JAASLoginModule -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18329) Add support for IBM Semeru OE JRE 11.0.15.0 and greater
[ https://issues.apache.org/jira/browse/HADOOP-18329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17677757#comment-17677757 ] ASF GitHub Bot commented on HADOOP-18329: - ivrisivris commented on PR #4537: URL: https://github.com/apache/hadoop/pull/4537#issuecomment-1385411337 Hi, I tried hadoop-3.3.5 RC0 but still got errors. I thought it was a https://github.com/apache/hadoop/pull/4537 issue and it was fixed in RC0. **Errors** `Caused by: javax.security.auth.login.LoginException: No LoginModule found for com.ibm.security.auth.module.LinuxLoginModule at java.base/javax.security.auth.login.LoginContext.invoke(LoginContext.java:731) at java.base/javax.security.auth.login.LoginContext$4.run(LoginContext.java:672) at java.base/javax.security.auth.login.LoginContext$4.run(LoginContext.java:670) at java.base/java.security.AccessController.doPrivileged(AccessController.java:783) at java.base/javax.security.auth.login.LoginContext.invokePriv(LoginContext.java:670) at java.base/javax.security.auth.login.LoginContext.login(LoginContext.java:581) at org.apache.hadoop.security.UserGroupInformation.loginUserFromSubject(UserGroupInformation.java:812)` **Enviroment** openjdk version "11.0.17" 2022-10-18 IBM Semeru Runtime Open Edition 11.0.17.0 (build 11.0.17+8) Eclipse OpenJ9 VM 11.0.17.0 (build openj9-0.35.0, JRE 11 Linux amd64-64-Bit Compressed References 20221031_559 (JIT enabled, AOT enabled) OpenJ9 - e04a7f6c1 OMR - 85a21674f JCL - a94c231303 based on jdk-11.0.17+8) Thank you for your help Kamil > Add support for IBM Semeru OE JRE 11.0.15.0 and greater > --- > > Key: HADOOP-18329 > URL: https://issues.apache.org/jira/browse/HADOOP-18329 > Project: Hadoop Common > Issue Type: Bug > Components: auth, common >Affects Versions: 3.0.0, 3.1.0, 3.0.1, 3.2.0, 3.0.2, 3.1.1, 3.0.3, 3.3.0, > 3.1.2, 3.2.1, 3.1.3, 3.1.4, 3.2.2, 3.3.1, 3.2.3, 3.3.2, 3.3.3 > Environment: Running Hadoop (or Apache Spark 3.2.1 or above) on IBM > Semeru runtimes open edition 11.0.15.0 or greater. >Reporter: Jack >Assignee: Jack >Priority: Major > Labels: pull-request-available > Fix For: 3.4.0, 3.3.5 > > Original Estimate: 1h > Time Spent: 2.5h > Remaining Estimate: 0h > > There are checks within the PlatformName class that use the Vendor property > of the provided runtime JVM specifically looking for `IBM` within the name. > Whilst this check worked for IBM's [java technology > edition|https://www.ibm.com/docs/en/sdk-java-technology] it fails to work on > [Semeru|https://developer.ibm.com/languages/java/semeru-runtimes/] since > 11.0.15.0 due to the following change: > h4. java.vendor system property > In this release, the {{java.vendor}} system property has been changed from > "International Business Machines Corporation" to "IBM Corporation". > Modules such as the below are not provided in these runtimes. > com.ibm.security.auth.module.JAASLoginModule -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18329) Add support for IBM Semeru OE JRE 11.0.15.0 and greater
[ https://issues.apache.org/jira/browse/HADOOP-18329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17677801#comment-17677801 ] ASF GitHub Bot commented on HADOOP-18329: - JackBuggins commented on PR #4537: URL: https://github.com/apache/hadoop/pull/4537#issuecomment-1385582733 I'll check this out against 11.0.17 today > Add support for IBM Semeru OE JRE 11.0.15.0 and greater > --- > > Key: HADOOP-18329 > URL: https://issues.apache.org/jira/browse/HADOOP-18329 > Project: Hadoop Common > Issue Type: Bug > Components: auth, common >Affects Versions: 3.0.0, 3.1.0, 3.0.1, 3.2.0, 3.0.2, 3.1.1, 3.0.3, 3.3.0, > 3.1.2, 3.2.1, 3.1.3, 3.1.4, 3.2.2, 3.3.1, 3.2.3, 3.3.2, 3.3.3 > Environment: Running Hadoop (or Apache Spark 3.2.1 or above) on IBM > Semeru runtimes open edition 11.0.15.0 or greater. >Reporter: Jack >Assignee: Jack >Priority: Major > Labels: pull-request-available > Fix For: 3.4.0, 3.3.5 > > Original Estimate: 1h > Time Spent: 2.5h > Remaining Estimate: 0h > > There are checks within the PlatformName class that use the Vendor property > of the provided runtime JVM specifically looking for `IBM` within the name. > Whilst this check worked for IBM's [java technology > edition|https://www.ibm.com/docs/en/sdk-java-technology] it fails to work on > [Semeru|https://developer.ibm.com/languages/java/semeru-runtimes/] since > 11.0.15.0 due to the following change: > h4. java.vendor system property > In this release, the {{java.vendor}} system property has been changed from > "International Business Machines Corporation" to "IBM Corporation". > Modules such as the below are not provided in these runtimes. > com.ibm.security.auth.module.JAASLoginModule -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18329) Add support for IBM Semeru OE JRE 11.0.15.0 and greater
[ https://issues.apache.org/jira/browse/HADOOP-18329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17677908#comment-17677908 ] ASF GitHub Bot commented on HADOOP-18329: - steveloughran commented on PR #4537: URL: https://github.com/apache/hadoop/pull/4537#issuecomment-1385924218 jack, be good to know. we are going to do a new RC next week and this is the kind of stabilisation issue we can address > Add support for IBM Semeru OE JRE 11.0.15.0 and greater > --- > > Key: HADOOP-18329 > URL: https://issues.apache.org/jira/browse/HADOOP-18329 > Project: Hadoop Common > Issue Type: Bug > Components: auth, common >Affects Versions: 3.0.0, 3.1.0, 3.0.1, 3.2.0, 3.0.2, 3.1.1, 3.0.3, 3.3.0, > 3.1.2, 3.2.1, 3.1.3, 3.1.4, 3.2.2, 3.3.1, 3.2.3, 3.3.2, 3.3.3 > Environment: Running Hadoop (or Apache Spark 3.2.1 or above) on IBM > Semeru runtimes open edition 11.0.15.0 or greater. >Reporter: Jack >Assignee: Jack >Priority: Major > Labels: pull-request-available > Fix For: 3.4.0, 3.3.5 > > Original Estimate: 1h > Time Spent: 2.5h > Remaining Estimate: 0h > > There are checks within the PlatformName class that use the Vendor property > of the provided runtime JVM specifically looking for `IBM` within the name. > Whilst this check worked for IBM's [java technology > edition|https://www.ibm.com/docs/en/sdk-java-technology] it fails to work on > [Semeru|https://developer.ibm.com/languages/java/semeru-runtimes/] since > 11.0.15.0 due to the following change: > h4. java.vendor system property > In this release, the {{java.vendor}} system property has been changed from > "International Business Machines Corporation" to "IBM Corporation". > Modules such as the below are not provided in these runtimes. > com.ibm.security.auth.module.JAASLoginModule -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18329) Add support for IBM Semeru OE JRE 11.0.15.0 and greater
[ https://issues.apache.org/jira/browse/HADOOP-18329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17677931#comment-17677931 ] ASF GitHub Bot commented on HADOOP-18329: - JackBuggins commented on PR #4537: URL: https://github.com/apache/hadoop/pull/4537#issuecomment-1385975581 I've just span up a version of rc-0 3.3.5 against semeru 11.0.17 from and executed a few of the sample jobs and checked out the dev/null out. eg. ``` yarn jar $HADOOP_HOME/share/hadoop/mapreduce/hadoop-mapreduce-examples-3.3.5.jar teragen 100 /test yarn jar $HADOOP_HOME/share/hadoop/mapreduce/hadoop-mapreduce-examples-3.3.5.jar terasort /test /testout ``` I'm not seeing anything suspicious in the stderr - @ivrisivris can you give me some more detail on the specific cluster config / actions / workloads that triggered this so I can reproduce and diagnose a bit better? Are you using this with Spark or anything else? Are these being added to the classpath in some other way? I currently don't believe anything packaged IBM semeru 11.0.17 or hadoop rc-0 3.3.5 contains something to trigger the check added, (I've even ran a jar with just that class to check that, and this looks good). Whilst I wait on a response on the above I'll check that there isn't anywhere else that is not using the `IBM_JAVA` check to determine the use of those modules and report back shortly. > Add support for IBM Semeru OE JRE 11.0.15.0 and greater > --- > > Key: HADOOP-18329 > URL: https://issues.apache.org/jira/browse/HADOOP-18329 > Project: Hadoop Common > Issue Type: Bug > Components: auth, common >Affects Versions: 3.0.0, 3.1.0, 3.0.1, 3.2.0, 3.0.2, 3.1.1, 3.0.3, 3.3.0, > 3.1.2, 3.2.1, 3.1.3, 3.1.4, 3.2.2, 3.3.1, 3.2.3, 3.3.2, 3.3.3 > Environment: Running Hadoop (or Apache Spark 3.2.1 or above) on IBM > Semeru runtimes open edition 11.0.15.0 or greater. >Reporter: Jack >Assignee: Jack >Priority: Major > Labels: pull-request-available > Fix For: 3.4.0, 3.3.5 > > Original Estimate: 1h > Time Spent: 2.5h > Remaining Estimate: 0h > > There are checks within the PlatformName class that use the Vendor property > of the provided runtime JVM specifically looking for `IBM` within the name. > Whilst this check worked for IBM's [java technology > edition|https://www.ibm.com/docs/en/sdk-java-technology] it fails to work on > [Semeru|https://developer.ibm.com/languages/java/semeru-runtimes/] since > 11.0.15.0 due to the following change: > h4. java.vendor system property > In this release, the {{java.vendor}} system property has been changed from > "International Business Machines Corporation" to "IBM Corporation". > Modules such as the below are not provided in these runtimes. > com.ibm.security.auth.module.JAASLoginModule -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18329) Add support for IBM Semeru OE JRE 11.0.15.0 and greater
[ https://issues.apache.org/jira/browse/HADOOP-18329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17677959#comment-17677959 ] ASF GitHub Bot commented on HADOOP-18329: - JackBuggins commented on PR #4537: URL: https://github.com/apache/hadoop/pull/4537#issuecomment-1385992169 From what I can see, the below could still exhibit the same behaviour @steveloughran - I'll see if I can figure out how to get into these paths to validate it so I understand the scenario @ivrisivris is likely hitting. - https://github.com/apache/hadoop/blob/cf33164857530a38aa3edc99a9051a47aae9dede/hadoop-common-project/hadoop-registry/src/main/java/org/apache/hadoop/registry/client/impl/zk/RegistrySecurity.java#L606-L612 - https://github.com/apache/hadoop/blob/cf33164857530a38aa3edc99a9051a47aae9dede/hadoop-common-project/hadoop-auth/src/main/java/org/apache/hadoop/security/authentication/util/JaasConfiguration.java#L70-L75 > Add support for IBM Semeru OE JRE 11.0.15.0 and greater > --- > > Key: HADOOP-18329 > URL: https://issues.apache.org/jira/browse/HADOOP-18329 > Project: Hadoop Common > Issue Type: Bug > Components: auth, common >Affects Versions: 3.0.0, 3.1.0, 3.0.1, 3.2.0, 3.0.2, 3.1.1, 3.0.3, 3.3.0, > 3.1.2, 3.2.1, 3.1.3, 3.1.4, 3.2.2, 3.3.1, 3.2.3, 3.3.2, 3.3.3 > Environment: Running Hadoop (or Apache Spark 3.2.1 or above) on IBM > Semeru runtimes open edition 11.0.15.0 or greater. >Reporter: Jack >Assignee: Jack >Priority: Major > Labels: pull-request-available > Fix For: 3.4.0, 3.3.5 > > Original Estimate: 1h > Time Spent: 2.5h > Remaining Estimate: 0h > > There are checks within the PlatformName class that use the Vendor property > of the provided runtime JVM specifically looking for `IBM` within the name. > Whilst this check worked for IBM's [java technology > edition|https://www.ibm.com/docs/en/sdk-java-technology] it fails to work on > [Semeru|https://developer.ibm.com/languages/java/semeru-runtimes/] since > 11.0.15.0 due to the following change: > h4. java.vendor system property > In this release, the {{java.vendor}} system property has been changed from > "International Business Machines Corporation" to "IBM Corporation". > Modules such as the below are not provided in these runtimes. > com.ibm.security.auth.module.JAASLoginModule -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18329) Add support for IBM Semeru OE JRE 11.0.15.0 and greater
[ https://issues.apache.org/jira/browse/HADOOP-18329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17678037#comment-17678037 ] ASF GitHub Bot commented on HADOOP-18329: - JackBuggins commented on PR #4537: URL: https://github.com/apache/hadoop/pull/4537#issuecomment-1386308333 I hit a similar error trivially in spark 3.4.0/dev > Add support for IBM Semeru OE JRE 11.0.15.0 and greater > --- > > Key: HADOOP-18329 > URL: https://issues.apache.org/jira/browse/HADOOP-18329 > Project: Hadoop Common > Issue Type: Bug > Components: auth, common >Affects Versions: 3.0.0, 3.1.0, 3.0.1, 3.2.0, 3.0.2, 3.1.1, 3.0.3, 3.3.0, > 3.1.2, 3.2.1, 3.1.3, 3.1.4, 3.2.2, 3.3.1, 3.2.3, 3.3.2, 3.3.3 > Environment: Running Hadoop (or Apache Spark 3.2.1 or above) on IBM > Semeru runtimes open edition 11.0.15.0 or greater. >Reporter: Jack >Assignee: Jack >Priority: Major > Labels: pull-request-available > Fix For: 3.4.0, 3.3.5 > > Original Estimate: 1h > Time Spent: 2.5h > Remaining Estimate: 0h > > There are checks within the PlatformName class that use the Vendor property > of the provided runtime JVM specifically looking for `IBM` within the name. > Whilst this check worked for IBM's [java technology > edition|https://www.ibm.com/docs/en/sdk-java-technology] it fails to work on > [Semeru|https://developer.ibm.com/languages/java/semeru-runtimes/] since > 11.0.15.0 due to the following change: > h4. java.vendor system property > In this release, the {{java.vendor}} system property has been changed from > "International Business Machines Corporation" to "IBM Corporation". > Modules such as the below are not provided in these runtimes. > com.ibm.security.auth.module.JAASLoginModule -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18329) Add support for IBM Semeru OE JRE 11.0.15.0 and greater
[ https://issues.apache.org/jira/browse/HADOOP-18329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17678198#comment-17678198 ] ASF GitHub Bot commented on HADOOP-18329: - steveloughran commented on PR #4537: URL: https://github.com/apache/hadoop/pull/4537#issuecomment-1386898828 maybe start with logging that java.vendor sysprop at debug. reopened the JIRA > Add support for IBM Semeru OE JRE 11.0.15.0 and greater > --- > > Key: HADOOP-18329 > URL: https://issues.apache.org/jira/browse/HADOOP-18329 > Project: Hadoop Common > Issue Type: Bug > Components: auth, common >Affects Versions: 3.0.0, 3.1.0, 3.0.1, 3.2.0, 3.0.2, 3.1.1, 3.0.3, 3.3.0, > 3.1.2, 3.2.1, 3.1.3, 3.1.4, 3.2.2, 3.3.1, 3.2.3, 3.3.2, 3.3.3 > Environment: Running Hadoop (or Apache Spark 3.2.1 or above) on IBM > Semeru runtimes open edition 11.0.15.0 or greater. >Reporter: Jack >Assignee: Jack >Priority: Major > Labels: pull-request-available > Fix For: 3.4.0, 3.3.5 > > Original Estimate: 1h > Time Spent: 2.5h > Remaining Estimate: 0h > > There are checks within the PlatformName class that use the Vendor property > of the provided runtime JVM specifically looking for `IBM` within the name. > Whilst this check worked for IBM's [java technology > edition|https://www.ibm.com/docs/en/sdk-java-technology] it fails to work on > [Semeru|https://developer.ibm.com/languages/java/semeru-runtimes/] since > 11.0.15.0 due to the following change: > h4. java.vendor system property > In this release, the {{java.vendor}} system property has been changed from > "International Business Machines Corporation" to "IBM Corporation". > Modules such as the below are not provided in these runtimes. > com.ibm.security.auth.module.JAASLoginModule -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18329) Add support for IBM Semeru OE JRE 11.0.15.0 and greater
[ https://issues.apache.org/jira/browse/HADOOP-18329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17685720#comment-17685720 ] ASF GitHub Bot commented on HADOOP-18329: - Tre2878 commented on PR #4537: URL: https://github.com/apache/hadoop/pull/4537#issuecomment-1422125911 hi,[JackBuggins](https://github.com/JackBuggins),I am using in the merged hadoop3.3.4 your https://github.com/apache/hadoop/pull/5208 code, start the datanode error, an error is as follows: `STARTUP_MSG: java = 11.0.15 / INFO org.apache.hadoop.hdfs.server.datanode.DataNode: registered UNIX signal handlers for [TERM, HUP, INT] ERROR org.apache.hadoop.security.UserGroupInformation: Unable to find JAAS classes:com.ibm.security.auth.UsernamePrincipal ERROR org.apache.hadoop.hdfs.server.datanode.DataNode: Exception in secureMain org.apache.hadoop.security.KerberosAuthException: failure to login: for principal: work/***@HADOOP.COM from keytab /etc/security/keytab/.hdfs.keytab javax.security.auth.login.LoginException: No LoginModule found for com.ibm.security.auth.module.Krb5LoginModule at org.apache.hadoop.security.UserGroupInformation.doSubjectLogin(UserGroupInformation.java:1986) at org.apache.hadoop.security.UserGroupInformation.loginUserFromKeytabAndReturnUGI(UserGroupInformation.java:1361) at org.apache.hadoop.security.UserGroupInformation.loginUserFromKeytab(UserGroupInformation.java:1122) at org.apache.hadoop.security.SecurityUtil.login(SecurityUtil.java:315) at org.apache.hadoop.hdfs.server.datanode.DataNode.instantiateDataNode(DataNode.java:2732) at org.apache.hadoop.hdfs.server.datanode.DataNode.createDataNode(DataNode.java:2778) at org.apache.hadoop.hdfs.server.datanode.DataNode.secureMain(DataNode.java:2922) at org.apache.hadoop.hdfs.server.datanode.DataNode.main(DataNode.java:2946) Caused by: javax.security.auth.login.LoginException: No LoginModule found for com.ibm.security.auth.module.Krb5LoginModule at java.base/javax.security.auth.login.LoginContext.invoke(LoginContext.java:731) at java.base/javax.security.auth.login.LoginContext$4.run(LoginContext.java:672) at java.base/javax.security.auth.login.LoginContext$4.run(LoginContext.java:670) at java.base/java.security.AccessController.doPrivileged(AccessController.java:783) at java.base/javax.security.auth.login.LoginContext.invokePriv(LoginContext.java:670) at java.base/javax.security.auth.login.LoginContext.login(LoginContext.java:581) at org.apache.hadoop.security.UserGroupInformation$HadoopLoginContext.login(UserGroupInformation.java:2065) at org.apache.hadoop.security.UserGroupInformation.doSubjectLogin(UserGroupInformation.java:1975) ... 7 more` java -version: `openjdk version "11.0.15" 2022-04-19 IBM Semeru Runtime Open Edition 11.0.15.0 (build 11.0.15+10) Eclipse OpenJ9 VM 11.0.15.0 (build openj9-0.32.0, JRE 11 Linux amd64-64-Bit Compressed References 20220422_425 (JIT enabled, AOT enabled) OpenJ9 - 9a84ec34e OMR - ab24b JCL - b7b5b42ea6 based on jdk-11.0.15+10)` Do I need to use a later version of the jdk or is this an unsolved bug > Add support for IBM Semeru OE JRE 11.0.15.0 and greater > --- > > Key: HADOOP-18329 > URL: https://issues.apache.org/jira/browse/HADOOP-18329 > Project: Hadoop Common > Issue Type: Bug > Components: auth, common >Affects Versions: 3.0.0, 3.1.0, 3.0.1, 3.2.0, 3.0.2, 3.1.1, 3.0.3, 3.3.0, > 3.1.2, 3.2.1, 3.1.3, 3.1.4, 3.2.2, 3.3.1, 3.2.3, 3.3.2, 3.3.3 > Environment: Running Hadoop (or Apache Spark 3.2.1 or above) on IBM > Semeru runtimes open edition 11.0.15.0 or greater. >Reporter: Jack >Assignee: Jack >Priority: Major > Labels: pull-request-available > Fix For: 3.4.0, 3.3.5 > > Original Estimate: 1h > Time Spent: 2.5h > Remaining Estimate: 0h > > There are checks within the PlatformName class that use the Vendor property > of the provided runtime JVM specifically looking for `IBM` within the name. > Whilst this check worked for IBM's [java technology > edition|https://www.ibm.com/docs/en/sdk-java-technology] it fails to work on > [Semeru|https://developer.ibm.com/languages/java/semeru-runtimes/] since > 11.0.15.0 due to the following change: > h4. java.vendor system property > In this release, the {{java.vendor}} system property has been changed from > "International Business Machines Corporation" to "IBM Corporation". > Modules such as the below are not provided in these runtimes. > com.ibm.security.auth.module.
[jira] [Commented] (HADOOP-18329) Add support for IBM Semeru OE JRE 11.0.15.0 and greater
[ https://issues.apache.org/jira/browse/HADOOP-18329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17685771#comment-17685771 ] ASF GitHub Bot commented on HADOOP-18329: - JackBuggins commented on PR #4537: URL: https://github.com/apache/hadoop/pull/4537#issuecomment-1422263273 @Tre2878 did you build the branch it was previously merged to from source? It's merged back retrospectively but once published initially there aren't more updates to a stable stream in terms of distributions. Ie. If you aren't switching versions and aren't building you aren't getting any back ported fixes. Please us 3.3.5 RC or build the branch yourself to pick up the changes 👍🏼 > Add support for IBM Semeru OE JRE 11.0.15.0 and greater > --- > > Key: HADOOP-18329 > URL: https://issues.apache.org/jira/browse/HADOOP-18329 > Project: Hadoop Common > Issue Type: Bug > Components: auth, common >Affects Versions: 3.0.0, 3.1.0, 3.0.1, 3.2.0, 3.0.2, 3.1.1, 3.0.3, 3.3.0, > 3.1.2, 3.2.1, 3.1.3, 3.1.4, 3.2.2, 3.3.1, 3.2.3, 3.3.2, 3.3.3 > Environment: Running Hadoop (or Apache Spark 3.2.1 or above) on IBM > Semeru runtimes open edition 11.0.15.0 or greater. >Reporter: Jack >Assignee: Jack >Priority: Major > Labels: pull-request-available > Fix For: 3.4.0, 3.3.5 > > Original Estimate: 1h > Time Spent: 2.5h > Remaining Estimate: 0h > > There are checks within the PlatformName class that use the Vendor property > of the provided runtime JVM specifically looking for `IBM` within the name. > Whilst this check worked for IBM's [java technology > edition|https://www.ibm.com/docs/en/sdk-java-technology] it fails to work on > [Semeru|https://developer.ibm.com/languages/java/semeru-runtimes/] since > 11.0.15.0 due to the following change: > h4. java.vendor system property > In this release, the {{java.vendor}} system property has been changed from > "International Business Machines Corporation" to "IBM Corporation". > Modules such as the below are not provided in these runtimes. > com.ibm.security.auth.module.JAASLoginModule -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18329) Add support for IBM Semeru OE JRE 11.0.15.0 and greater
[ https://issues.apache.org/jira/browse/HADOOP-18329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17685778#comment-17685778 ] ASF GitHub Bot commented on HADOOP-18329: - JackBuggins commented on PR #4537: URL: https://github.com/apache/hadoop/pull/4537#issuecomment-1422280251 If you are finding some more areas where IBM classes are being called relating to auth, if you can demonstrate a configuration with specific details as well as how you reproduce it I will take a look. I need the JRE details, OS details and full hadoop config and command executed to hit this. If this is specifically relating to Kerberos, please add details for that too but obviously I won't need any secrets. I've spent some good hours trying to replicate it so far so having this detail will be awesome for me to understand where it's failing for some. May be best to pop this on the jira only. Thanks! > Add support for IBM Semeru OE JRE 11.0.15.0 and greater > --- > > Key: HADOOP-18329 > URL: https://issues.apache.org/jira/browse/HADOOP-18329 > Project: Hadoop Common > Issue Type: Bug > Components: auth, common >Affects Versions: 3.0.0, 3.1.0, 3.0.1, 3.2.0, 3.0.2, 3.1.1, 3.0.3, 3.3.0, > 3.1.2, 3.2.1, 3.1.3, 3.1.4, 3.2.2, 3.3.1, 3.2.3, 3.3.2, 3.3.3 > Environment: Running Hadoop (or Apache Spark 3.2.1 or above) on IBM > Semeru runtimes open edition 11.0.15.0 or greater. >Reporter: Jack >Assignee: Jack >Priority: Major > Labels: pull-request-available > Fix For: 3.4.0, 3.3.5 > > Original Estimate: 1h > Time Spent: 2.5h > Remaining Estimate: 0h > > There are checks within the PlatformName class that use the Vendor property > of the provided runtime JVM specifically looking for `IBM` within the name. > Whilst this check worked for IBM's [java technology > edition|https://www.ibm.com/docs/en/sdk-java-technology] it fails to work on > [Semeru|https://developer.ibm.com/languages/java/semeru-runtimes/] since > 11.0.15.0 due to the following change: > h4. java.vendor system property > In this release, the {{java.vendor}} system property has been changed from > "International Business Machines Corporation" to "IBM Corporation". > Modules such as the below are not provided in these runtimes. > com.ibm.security.auth.module.JAASLoginModule -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18329) Add support for IBM Semeru OE JRE 11.0.15.0 and greater
[ https://issues.apache.org/jira/browse/HADOOP-18329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17685780#comment-17685780 ] ASF GitHub Bot commented on HADOOP-18329: - JackBuggins commented on PR #4537: URL: https://github.com/apache/hadoop/pull/4537#issuecomment-1422283214 @Tre2878 - any of the files I listed here look sus to you https://github.com/apache/hadoop/pull/4537#issuecomment-1385992169? > Add support for IBM Semeru OE JRE 11.0.15.0 and greater > --- > > Key: HADOOP-18329 > URL: https://issues.apache.org/jira/browse/HADOOP-18329 > Project: Hadoop Common > Issue Type: Bug > Components: auth, common >Affects Versions: 3.0.0, 3.1.0, 3.0.1, 3.2.0, 3.0.2, 3.1.1, 3.0.3, 3.3.0, > 3.1.2, 3.2.1, 3.1.3, 3.1.4, 3.2.2, 3.3.1, 3.2.3, 3.3.2, 3.3.3 > Environment: Running Hadoop (or Apache Spark 3.2.1 or above) on IBM > Semeru runtimes open edition 11.0.15.0 or greater. >Reporter: Jack >Assignee: Jack >Priority: Major > Labels: pull-request-available > Fix For: 3.4.0, 3.3.5 > > Original Estimate: 1h > Time Spent: 2.5h > Remaining Estimate: 0h > > There are checks within the PlatformName class that use the Vendor property > of the provided runtime JVM specifically looking for `IBM` within the name. > Whilst this check worked for IBM's [java technology > edition|https://www.ibm.com/docs/en/sdk-java-technology] it fails to work on > [Semeru|https://developer.ibm.com/languages/java/semeru-runtimes/] since > 11.0.15.0 due to the following change: > h4. java.vendor system property > In this release, the {{java.vendor}} system property has been changed from > "International Business Machines Corporation" to "IBM Corporation". > Modules such as the below are not provided in these runtimes. > com.ibm.security.auth.module.JAASLoginModule -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18329) Add support for IBM Semeru OE JRE 11.0.15.0 and greater
[ https://issues.apache.org/jira/browse/HADOOP-18329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17685831#comment-17685831 ] ASF GitHub Bot commented on HADOOP-18329: - steveloughran commented on PR #4537: URL: https://github.com/apache/hadoop/pull/4537#issuecomment-1422368836 ok, i think we can hopefully say this is a cannot reproduce state, especially if @Tre2878 is using their own build. 1. can you do the kdiag command to get the diags, after sanitising anything you don't want to share, attach to the jira 2. then grab my cloudstore.jar and run its storediag command with an hdfs url to see how interaction with hdfs goes: https://github.com/steveloughran/cloudstore > Add support for IBM Semeru OE JRE 11.0.15.0 and greater > --- > > Key: HADOOP-18329 > URL: https://issues.apache.org/jira/browse/HADOOP-18329 > Project: Hadoop Common > Issue Type: Bug > Components: auth, common >Affects Versions: 3.0.0, 3.1.0, 3.0.1, 3.2.0, 3.0.2, 3.1.1, 3.0.3, 3.3.0, > 3.1.2, 3.2.1, 3.1.3, 3.1.4, 3.2.2, 3.3.1, 3.2.3, 3.3.2, 3.3.3 > Environment: Running Hadoop (or Apache Spark 3.2.1 or above) on IBM > Semeru runtimes open edition 11.0.15.0 or greater. >Reporter: Jack >Assignee: Jack >Priority: Major > Labels: pull-request-available > Fix For: 3.4.0, 3.3.5 > > Original Estimate: 1h > Time Spent: 2.5h > Remaining Estimate: 0h > > There are checks within the PlatformName class that use the Vendor property > of the provided runtime JVM specifically looking for `IBM` within the name. > Whilst this check worked for IBM's [java technology > edition|https://www.ibm.com/docs/en/sdk-java-technology] it fails to work on > [Semeru|https://developer.ibm.com/languages/java/semeru-runtimes/] since > 11.0.15.0 due to the following change: > h4. java.vendor system property > In this release, the {{java.vendor}} system property has been changed from > "International Business Machines Corporation" to "IBM Corporation". > Modules such as the below are not provided in these runtimes. > com.ibm.security.auth.module.JAASLoginModule -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18329) Add support for IBM Semeru OE JRE 11.0.15.0 and greater
[ https://issues.apache.org/jira/browse/HADOOP-18329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17685938#comment-17685938 ] ASF GitHub Bot commented on HADOOP-18329: - JackBuggins commented on PR #4537: URL: https://github.com/apache/hadoop/pull/4537#issuecomment-1422734164 Yeah, I just need a way I can reliably hit the errors to understand a bit better what isn't covered so far to proceed further. This indeed looks similar to the previous issue reported, so it would seem to be a common use case. > Add support for IBM Semeru OE JRE 11.0.15.0 and greater > --- > > Key: HADOOP-18329 > URL: https://issues.apache.org/jira/browse/HADOOP-18329 > Project: Hadoop Common > Issue Type: Bug > Components: auth, common >Affects Versions: 3.0.0, 3.1.0, 3.0.1, 3.2.0, 3.0.2, 3.1.1, 3.0.3, 3.3.0, > 3.1.2, 3.2.1, 3.1.3, 3.1.4, 3.2.2, 3.3.1, 3.2.3, 3.3.2, 3.3.3 > Environment: Running Hadoop (or Apache Spark 3.2.1 or above) on IBM > Semeru runtimes open edition 11.0.15.0 or greater. >Reporter: Jack >Assignee: Jack >Priority: Major > Labels: pull-request-available > Fix For: 3.4.0, 3.3.5 > > Original Estimate: 1h > Time Spent: 2.5h > Remaining Estimate: 0h > > There are checks within the PlatformName class that use the Vendor property > of the provided runtime JVM specifically looking for `IBM` within the name. > Whilst this check worked for IBM's [java technology > edition|https://www.ibm.com/docs/en/sdk-java-technology] it fails to work on > [Semeru|https://developer.ibm.com/languages/java/semeru-runtimes/] since > 11.0.15.0 due to the following change: > h4. java.vendor system property > In this release, the {{java.vendor}} system property has been changed from > "International Business Machines Corporation" to "IBM Corporation". > Modules such as the below are not provided in these runtimes. > com.ibm.security.auth.module.JAASLoginModule -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18329) Add support for IBM Semeru OE JRE 11.0.15.0 and greater
[ https://issues.apache.org/jira/browse/HADOOP-18329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17686153#comment-17686153 ] ASF GitHub Bot commented on HADOOP-18329: - Tre2878 commented on PR #4537: URL: https://github.com/apache/hadoop/pull/4537#issuecomment-1423544578 > @Tre2878 did you build the branch it was previously merged to from source? > > It's merged back retrospectively but once published initially there aren't more updates to a stable stream in terms of distributions. Ie. If you aren't switching versions and aren't building you aren't getting any back ported fixes. > > Please us 3.3.5 RC or build the branch yourself to pick up the changes 👍🏼 I am in the branch: https://github.com/apache/hadoop/tree/branch-3.3.4 to merge your https://github.com/apache/hadoop/pull/5208 code, compile, deploy start an error, My understanding is that there is no patch for 3.3.4 yet, right? > Add support for IBM Semeru OE JRE 11.0.15.0 and greater > --- > > Key: HADOOP-18329 > URL: https://issues.apache.org/jira/browse/HADOOP-18329 > Project: Hadoop Common > Issue Type: Bug > Components: auth, common >Affects Versions: 3.0.0, 3.1.0, 3.0.1, 3.2.0, 3.0.2, 3.1.1, 3.0.3, 3.3.0, > 3.1.2, 3.2.1, 3.1.3, 3.1.4, 3.2.2, 3.3.1, 3.2.3, 3.3.2, 3.3.3 > Environment: Running Hadoop (or Apache Spark 3.2.1 or above) on IBM > Semeru runtimes open edition 11.0.15.0 or greater. >Reporter: Jack >Assignee: Jack >Priority: Major > Labels: pull-request-available > Fix For: 3.4.0, 3.3.5 > > Original Estimate: 1h > Time Spent: 2.5h > Remaining Estimate: 0h > > There are checks within the PlatformName class that use the Vendor property > of the provided runtime JVM specifically looking for `IBM` within the name. > Whilst this check worked for IBM's [java technology > edition|https://www.ibm.com/docs/en/sdk-java-technology] it fails to work on > [Semeru|https://developer.ibm.com/languages/java/semeru-runtimes/] since > 11.0.15.0 due to the following change: > h4. java.vendor system property > In this release, the {{java.vendor}} system property has been changed from > "International Business Machines Corporation" to "IBM Corporation". > Modules such as the below are not provided in these runtimes. > com.ibm.security.auth.module.JAASLoginModule -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18329) Add support for IBM Semeru OE JRE 11.0.15.0 and greater
[ https://issues.apache.org/jira/browse/HADOOP-18329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17686373#comment-17686373 ] ASF GitHub Bot commented on HADOOP-18329: - JackBuggins commented on PR #4537: URL: https://github.com/apache/hadoop/pull/4537#issuecomment-1424011963 @Tre2878 - That's correct, it's just branch-3.3 and trunk, which is in turn in the 3.3.5 stream, so right now upgrading/testing out the 3.3.5 stream is the way to get these changes. Looks like the 3.3.4 distro was cut around August 2022 (based on git releases), and the changes go to branch-3.3 during December 2022. I would need to defer to project owners if porting to branch 3.3.4 for others to pickup when building from source would be an option here. You could try cherry-picking the commit against 3.3 to branch 3.3.4 then build your own copy to test, or otherwise consider applying some type of agent workaround to modify the vendor name until you're in a position to do this. > Add support for IBM Semeru OE JRE 11.0.15.0 and greater > --- > > Key: HADOOP-18329 > URL: https://issues.apache.org/jira/browse/HADOOP-18329 > Project: Hadoop Common > Issue Type: Bug > Components: auth, common >Affects Versions: 3.0.0, 3.1.0, 3.0.1, 3.2.0, 3.0.2, 3.1.1, 3.0.3, 3.3.0, > 3.1.2, 3.2.1, 3.1.3, 3.1.4, 3.2.2, 3.3.1, 3.2.3, 3.3.2, 3.3.3 > Environment: Running Hadoop (or Apache Spark 3.2.1 or above) on IBM > Semeru runtimes open edition 11.0.15.0 or greater. >Reporter: Jack >Assignee: Jack >Priority: Major > Labels: pull-request-available > Fix For: 3.4.0, 3.3.5 > > Original Estimate: 1h > Time Spent: 2.5h > Remaining Estimate: 0h > > There are checks within the PlatformName class that use the Vendor property > of the provided runtime JVM specifically looking for `IBM` within the name. > Whilst this check worked for IBM's [java technology > edition|https://www.ibm.com/docs/en/sdk-java-technology] it fails to work on > [Semeru|https://developer.ibm.com/languages/java/semeru-runtimes/] since > 11.0.15.0 due to the following change: > h4. java.vendor system property > In this release, the {{java.vendor}} system property has been changed from > "International Business Machines Corporation" to "IBM Corporation". > Modules such as the below are not provided in these runtimes. > com.ibm.security.auth.module.JAASLoginModule -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18329) Add support for IBM Semeru OE JRE 11.0.15.0 and greater
[ https://issues.apache.org/jira/browse/HADOOP-18329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17686486#comment-17686486 ] ASF GitHub Bot commented on HADOOP-18329: - steveloughran commented on PR #4537: URL: https://github.com/apache/hadoop/pull/4537#issuecomment-1424181715 > My understanding is that there is no patch for 3.3.4 yet, right? 3.3.5 is the successor to 3.3.4 from the ASF, therefore there is a patch for 3.3.4 and it is called "upgrade to 3.3.5". you can take our 3.3.4 release, fork it and make whatever changes you want, but you will have a private release at that point. > Add support for IBM Semeru OE JRE 11.0.15.0 and greater > --- > > Key: HADOOP-18329 > URL: https://issues.apache.org/jira/browse/HADOOP-18329 > Project: Hadoop Common > Issue Type: Bug > Components: auth, common >Affects Versions: 3.0.0, 3.1.0, 3.0.1, 3.2.0, 3.0.2, 3.1.1, 3.0.3, 3.3.0, > 3.1.2, 3.2.1, 3.1.3, 3.1.4, 3.2.2, 3.3.1, 3.2.3, 3.3.2, 3.3.3 > Environment: Running Hadoop (or Apache Spark 3.2.1 or above) on IBM > Semeru runtimes open edition 11.0.15.0 or greater. >Reporter: Jack >Assignee: Jack >Priority: Major > Labels: pull-request-available > Fix For: 3.4.0, 3.3.5 > > Original Estimate: 1h > Time Spent: 2.5h > Remaining Estimate: 0h > > There are checks within the PlatformName class that use the Vendor property > of the provided runtime JVM specifically looking for `IBM` within the name. > Whilst this check worked for IBM's [java technology > edition|https://www.ibm.com/docs/en/sdk-java-technology] it fails to work on > [Semeru|https://developer.ibm.com/languages/java/semeru-runtimes/] since > 11.0.15.0 due to the following change: > h4. java.vendor system property > In this release, the {{java.vendor}} system property has been changed from > "International Business Machines Corporation" to "IBM Corporation". > Modules such as the below are not provided in these runtimes. > com.ibm.security.auth.module.JAASLoginModule -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org