attilapiros commented on PR #55720:
URL: https://github.com/apache/spark/pull/55720#issuecomment-4493448506
@dongjoon-hyun, @huaxingao, @yadavay-amzn
Sorry for the confusion!
I was wrong before: this is NOT a simple test issue but a bug in the
`spark-shell` as well on JDK25:
```
./bin/spark-shell --remote sc://localhost:15002
WARNING: Using incubator modules: jdk.incubator.vector
WARNING: package sun.security.action not in java.base
26/05/19 17:38:31 INFO BaseAllocator: Debug mode disabled. Enable with the
VM option -Darrow.memory.debug.allocator=true.
26/05/19 17:38:31 INFO DefaultAllocationManagerOption: allocation manager
type not specified, using netty as the default type
26/05/19 17:38:31 INFO CheckAllocator: Using DefaultAllocationManager at
memory/netty/DefaultAllocationManagerFactory.class
Exception in thread "main" java.lang.ExceptionInInitializerError
at
org.sparkproject.org.apache.arrow.memory.netty.DefaultAllocationManagerFactory.<clinit>(DefaultAllocationManagerFactory.java:26)
at java.base/java.lang.Class.forName0(Native Method)
at java.base/java.lang.Class.forName(Class.java:467)
at java.base/java.lang.Class.forName(Class.java:458)
at
org.sparkproject.org.apache.arrow.memory.DefaultAllocationManagerOption.getFactory(DefaultAllocationManagerOption.java:105)
at
org.sparkproject.org.apache.arrow.memory.DefaultAllocationManagerOption.getDefaultAllocationManagerFactory(DefaultAllocationManagerOption.java:92)
at
org.sparkproject.org.apache.arrow.memory.BaseAllocator$Config.getAllocationManagerFactory(BaseAllocator.java:826)
at
org.sparkproject.org.apache.arrow.memory.ImmutableConfig.access$001(ImmutableConfig.java:20)
at
org.sparkproject.org.apache.arrow.memory.ImmutableConfig$InitShim.getAllocationManagerFactory(ImmutableConfig.java:80)
at
org.sparkproject.org.apache.arrow.memory.ImmutableConfig.<init>(ImmutableConfig.java:43)
at
org.sparkproject.org.apache.arrow.memory.ImmutableConfig$Builder.build(ImmutableConfig.java:492)
at
org.sparkproject.org.apache.arrow.memory.BaseAllocator.<clinit>(BaseAllocator.java:72)
at
org.apache.spark.sql.connect.SparkSession.<init>(SparkSession.scala:89)
at
org.apache.spark.sql.connect.SparkSession$Builder.tryCreateSessionFromClient(SparkSession.scala:1059)
at
org.apache.spark.sql.connect.SparkSession$Builder.$anonfun$getOrCreate$1(SparkSession.scala:1119)
at
org.apache.spark.sql.connect.SparkSession$.withLocalConnectServer(SparkSession.scala:949)
at
org.apache.spark.sql.connect.SparkSession$Builder.getOrCreate(SparkSession.scala:1118)
at
org.apache.spark.sql.application.ConnectRepl$.$anonfun$doMain$1(ConnectRepl.scala:91)
at
org.apache.spark.sql.connect.SparkSession$.withLocalConnectServer(SparkSession.scala:949)
at
org.apache.spark.sql.application.ConnectRepl$.doMain(ConnectRepl.scala:68)
at
org.apache.spark.sql.application.ConnectRepl$.main(ConnectRepl.scala:58)
at org.apache.spark.sql.application.ConnectRepl.main(ConnectRepl.scala)
at
java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:104)
at java.base/java.lang.reflect.Method.invoke(Method.java:565)
at
org.apache.spark.deploy.JavaMainApplication.start(SparkApplication.scala:52)
at
org.apache.spark.deploy.SparkSubmit.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:1033)
at
org.apache.spark.deploy.SparkSubmit.doRunMain$1(SparkSubmit.scala:203)
at org.apache.spark.deploy.SparkSubmit.submit(SparkSubmit.scala:226)
at org.apache.spark.deploy.SparkSubmit.doSubmit(SparkSubmit.scala:95)
at
org.apache.spark.deploy.SparkSubmit$$anon$2.doSubmit(SparkSubmit.scala:1171)
at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:1180)
at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
Caused by: java.lang.UnsupportedOperationException
at
org.sparkproject.io.netty.buffer.EmptyByteBuf.memoryAddress(EmptyByteBuf.java:961)
at
org.sparkproject.io.netty.buffer.DuplicatedByteBuf.memoryAddress(DuplicatedByteBuf.java:115)
at
org.sparkproject.io.netty.buffer.UnsafeDirectLittleEndian.<init>(UnsafeDirectLittleEndian.java:45)
at
org.sparkproject.io.netty.buffer.PooledByteBufAllocatorL.<init>(PooledByteBufAllocatorL.java:47)
at
org.sparkproject.org.apache.arrow.memory.netty.NettyAllocationManager.<clinit>(NettyAllocationManager.java:54)
... 32 more
```
Still I think it is unrelated to the ammonite fix!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
