[jira] [Updated] (SPARK-45508) org.apache.spark.unsafe.Platform uses wrong cleaner class name in JDK 9.b110+
[ https://issues.apache.org/jira/browse/SPARK-45508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-45508: --- Description: In JDK >= 9.b110, the code at [https://github.com/apache/spark/blob/v3.5.0/common/unsafe/src/main/java/org/apache/spark/unsafe/Platform.java#L213] hits a fallback path because we are using the wrong cleaner class name: `jdk.internal.ref.Cleaner` was removed in [https://bugs.openjdk.org/browse/JDK-8149925] This can be verified via ``` val f = classOf[org.apache.spark.unsafe.Platform].getDeclaredField("CLEANER_CREATE_METHOD") f.setAccessible(true) f.get(null) ``` returning `null` instead of a method. was: In JDK >= 9.b110, the code at [https://github.com/apache/spark/blob/v3.5.0/common/unsafe/src/main/java/org/apache/spark/unsafe/Platform.java#L213] hits a fallback path because we are using the wrong cleaner class name: `jdk.internal.ref.Cleaner` was removed in JDK-8149925 [https://bugs.java.com/bugdatabase/view_bug.do?bug_id=8149925] This can be verified via ``` val f = classOf[org.apache.spark.unsafe.Platform].getDeclaredField("CLEANER_CREATE_METHOD") f.setAccessible(true) f.get(null) ``` returning `null` instead of a method. > org.apache.spark.unsafe.Platform uses wrong cleaner class name in JDK 9.b110+ > - > > Key: SPARK-45508 > URL: https://issues.apache.org/jira/browse/SPARK-45508 > Project: Spark > Issue Type: Bug > Components: Spark Core >Affects Versions: 3.5.0 >Reporter: Josh Rosen >Priority: Major > > In JDK >= 9.b110, the code at > [https://github.com/apache/spark/blob/v3.5.0/common/unsafe/src/main/java/org/apache/spark/unsafe/Platform.java#L213] > hits a fallback path because we are using the wrong cleaner class name: > `jdk.internal.ref.Cleaner` was removed in > [https://bugs.openjdk.org/browse/JDK-8149925] > This can be verified via > > ``` > val f = > classOf[org.apache.spark.unsafe.Platform].getDeclaredField("CLEANER_CREATE_METHOD") > f.setAccessible(true) > f.get(null) > ``` > returning `null` instead of a method. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-45508) org.apache.spark.unsafe.Platform uses wrong cleaner class name in JDK 9.b110+
[ https://issues.apache.org/jira/browse/SPARK-45508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-45508: --- Description: In JDK >= 9.b110, the code at [https://github.com/apache/spark/blob/v3.5.0/common/unsafe/src/main/java/org/apache/spark/unsafe/Platform.java#L213] hits a fallback path because we are using the wrong cleaner class name: `jdk.internal.ref.Cleaner` was removed in JDK-8149925 [https://bugs.java.com/bugdatabase/view_bug.do?bug_id=8149925] This can be verified via ``` val f = classOf[org.apache.spark.unsafe.Platform].getDeclaredField("CLEANER_CREATE_METHOD") f.setAccessible(true) f.get(null) ``` returning `null` instead of a method. was: In JDK 11+, the code at [https://github.com/apache/spark/blob/v3.5.0/common/unsafe/src/main/java/org/apache/spark/unsafe/Platform.java#L213] hits a fallback path because we are using the wrong cleaner class name. This can be verified via ``` val f = classOf[org.apache.spark.unsafe.Platform].getDeclaredField("CLEANER_CREATE_METHOD") f.setAccessible(true) f.get(null) ``` returning `null` instead of a method. Summary: org.apache.spark.unsafe.Platform uses wrong cleaner class name in JDK 9.b110+ (was: org.apache.spark.unsafe.Platform uses wrong cleaner class name in JDK 11+) > org.apache.spark.unsafe.Platform uses wrong cleaner class name in JDK 9.b110+ > - > > Key: SPARK-45508 > URL: https://issues.apache.org/jira/browse/SPARK-45508 > Project: Spark > Issue Type: Bug > Components: Spark Core >Affects Versions: 3.5.0 >Reporter: Josh Rosen >Priority: Major > > In JDK >= 9.b110, the code at > [https://github.com/apache/spark/blob/v3.5.0/common/unsafe/src/main/java/org/apache/spark/unsafe/Platform.java#L213] > hits a fallback path because we are using the wrong cleaner class name: > `jdk.internal.ref.Cleaner` was removed in JDK-8149925 > [https://bugs.java.com/bugdatabase/view_bug.do?bug_id=8149925] > This can be verified via > > ``` > val f = > classOf[org.apache.spark.unsafe.Platform].getDeclaredField("CLEANER_CREATE_METHOD") > f.setAccessible(true) > f.get(null) > ``` > returning `null` instead of a method. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org