[
https://issues.apache.org/jira/browse/SPARK-44784?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17753974#comment-17753974
]
Juliusz Sompolski commented on SPARK-44784:
-------------------------------------------
This is a classloader issue with the serialization of an UDF pulling classes
from the client's class context that it doesn't necessarily need, and then
resulting in ClassNotFoundException on the server.
org.apache.spark.SparkException: org/apache/spark/sql/connect/client/SparkResult
It's not specifically related to the tests in that suite, but can happen
anyplace with UDFs.
> Failure in testing `SparkSessionE2ESuite` using Maven
> -----------------------------------------------------
>
> Key: SPARK-44784
> URL: https://issues.apache.org/jira/browse/SPARK-44784
> Project: Spark
> Issue Type: Bug
> Components: Connect, Tests
> Affects Versions: 3.5.0
> Reporter: Yang Jie
> Priority: Blocker
>
> [https://github.com/apache/spark/actions/runs/5832898984/job/15819181762]
>
> The following failures exist in the daily Maven tests, we should fix them
> before Apache Spark 3.5.0 release:
> {code:java}
> SparkSessionE2ESuite:
> 4638- interrupt all - background queries, foreground interrupt *** FAILED ***
> 4639 The code passed to eventually never returned normally. Attempted 30
> times over 20.092924822 seconds. Last failure message: Some("unexpected
> failure in q1: org.apache.spark.SparkException:
> org/apache/spark/sql/connect/client/SparkResult") was not empty Error not
> empty: Some(unexpected failure in q1: org.apache.spark.SparkException:
> org/apache/spark/sql/connect/client/SparkResult).
> (SparkSessionE2ESuite.scala:71)
> 4640- interrupt all - foreground queries, background interrupt *** FAILED ***
> 4641 "org/apache/spark/sql/connect/client/SparkResult" did not contain
> "OPERATION_CANCELED" Unexpected exception: org.apache.spark.SparkException:
> org/apache/spark/sql/connect/client/SparkResult
> (SparkSessionE2ESuite.scala:105)
> 4642- interrupt tag *** FAILED ***
> 4643 The code passed to eventually never returned normally. Attempted 30
> times over 20.069445587 seconds. Last failure message: ListBuffer() had
> length 0 instead of expected length 2 Interrupted operations: ListBuffer()..
> (SparkSessionE2ESuite.scala:199)
> 4644- interrupt operation *** FAILED ***
> 4645 org.apache.spark.SparkException:
> org/apache/spark/sql/connect/client/SparkResult
> 4646 at
> org.apache.spark.sql.connect.client.GrpcExceptionConverter$.toThrowable(GrpcExceptionConverter.scala:89)
> 4647 at
> org.apache.spark.sql.connect.client.GrpcExceptionConverter$.convert(GrpcExceptionConverter.scala:38)
> 4648 at
> org.apache.spark.sql.connect.client.GrpcExceptionConverter$$anon$1.hasNext(GrpcExceptionConverter.scala:46)
> 4649 at
> org.apache.spark.sql.connect.client.SparkResult.org$apache$spark$sql$connect$client$SparkResult$$processResponses(SparkResult.scala:83)
> 4650 at
> org.apache.spark.sql.connect.client.SparkResult.operationId(SparkResult.scala:174)
> 4651 at
> org.apache.spark.sql.SparkSessionE2ESuite.$anonfun$new$31(SparkSessionE2ESuite.scala:243)
> 4652 at
> org.apache.spark.sql.connect.client.util.RemoteSparkSession.$anonfun$test$1(RemoteSparkSession.scala:243)
> 4653 at org.scalatest.OutcomeOf.outcomeOf(OutcomeOf.scala:85)
> 4654 at org.scalatest.OutcomeOf.outcomeOf$(OutcomeOf.scala:83)
> 4655 at org.scalatest.OutcomeOf$.outcomeOf(OutcomeOf.scala:104)
> 4656 ... {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
