pan3793 opened a new pull request, #4880:
URL: https://github.com/apache/zeppelin/pull/4880
### What is this PR for?
Migrate the Livy test from Spark 2.4 with Java 8 to Spark 3.2 with Java 11.
There is a failed test caused by a classloader issue, likely a Livy-side
issue, temporarily disable it.
```
Error: Failures:
Error: LivyInterpreterIT.testLivyTutorialNote:689 %text
org.apache.spark.SparkException: Job aborted due to stage failure: Task 0 in
stage 0.0 failed 1 times, most recent failure: Lost task 0.0 in stage 0.0 (TID
0) (localhost executor driver): java.lang.ClassCastException: class Bank cannot
be cast to class Bank (Bank is in unnamed module of loader
org.apache.spark.repl.ExecutorClassLoader @6e7a7cff; Bank is in unnamed module
of loader scala.tools.nsc.interpreter.IMain$TranslatingClassLoader @67f10f45)
at
org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIteratorForCodegenStage1.agg_doAggregateWithKeys_0$(Unknown
Source)
at
org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIteratorForCodegenStage1.processNext(Unknown
Source)
at
org.apache.spark.sql.execution.BufferedRowIterator.hasNext(BufferedRowIterator.java:43)
at
org.apache.spark.sql.execution.WholeStageCodegenExec$$anon$1.hasNext(WholeStageCodegenExec.scala:759)
at scala.collection.Iterator$$anon$10.hasNext(Iterator.scala:460)
at
org.apache.spark.shuffle.sort.BypassMergeSortShuffleWriter.write(BypassMergeSortShuffleWriter.java:140)
at
org.apache.spark.shuffle.ShuffleWriteProcessor.write(ShuffleWriteProcessor.scala:59)
at
org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:99)
at
org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:52)
at org.apache.spark.scheduler.Task.run(Task.scala:131)
at
org.apache.spark.executor.Executor$TaskRunner.$anonfun$run$3(Executor.scala:506)
at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1491)
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:509)
at
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)
```
### What type of PR is it?
Improvement
### Todos
* [ ] - Investigate the classloader issue - likely a Livy-side issue.
### What is the Jira issue?
ZEPPELIN-6134
### How should this be tested?
Pass GHA.
### Screenshots (if appropriate)
### Questions:
* Does the license files need to update?
* Is there breaking changes for older versions?
* Does this needs documentation?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
