pan3793 commented on PR #56006: URL: https://github.com/apache/spark/pull/56006#issuecomment-4496136568
The vanilla netty lib Java 25 support has been addressed by adding `-Dio.netty.noUnsafe=false` in SPARK-55670 (https://github.com/apache/spark/pull/54470), but your stacktrace indicates the error is thrown from a shaded arrow lib `org.sparkproject.org.apache.arrow` according to https://github.com/netty/netty/issues/14942#issuecomment-2737878083, we can use `--sun-misc-unsafe-memory-access=true` and remove `-Dio.netty.noUnsafe=false`, this works for both vanilla and shaded arrow libs. so please update all places as the opts spread in many places. > You can either ignore the warning, since Netty is auto-detecting availability, or you can disable it explicitly with -Dio.netty.noUnsafe=true (note, this property have have a different name in the shaded version). > > You can also control this behavior with the --sun-misc-unsafe-memory-access argument: > > ``` > --sun-misc-unsafe-memory-access=<value> > allow or deny usage of unsupported API sun.misc.Unsafe > <value> is one of "allow", "warn", "debug", or "deny". > The default value is "warn". > ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
