This is an automated email from the ASF dual-hosted git repository. hvanhovell pushed a commit to branch branch-3.5 in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/branch-3.5 by this push: new 2d7e05ddd98 [SPARK-44816][CONNECT] Improve error message when UDF class is not found 2d7e05ddd98 is described below commit 2d7e05ddd980ad7a934e5c8fde77335ef6591c06 Author: Niranjan Jayakar <n...@databricks.com> AuthorDate: Wed Aug 23 18:42:04 2023 +0200 [SPARK-44816][CONNECT] Improve error message when UDF class is not found ### What changes were proposed in this pull request? Improve the error messaging on the connect client when using a UDF whose corresponding class has not been sync'ed with the spark connect service. Prior to this change, the client receives a cryptic error: ``` Exception in thread "main" org.apache.spark.SparkException: Main$ ``` With this change, the message is improved to be: ``` Exception in thread "main" org.apache.spark.SparkException: Failed to load class: Main$. Make sure the artifact where the class is defined is installed by calling session.addArtifact. ``` ### Why are the changes needed? This change makes it clear to the user on what the error is. ### Does this PR introduce _any_ user-facing change? Yes. The error message is improved. See details above. ### How was this patch tested? Manually by running a connect server and client. Closes #42500 from nija-at/improve-error. Authored-by: Niranjan Jayakar <n...@databricks.com> Signed-off-by: Herman van Hovell <her...@databricks.com> (cherry picked from commit 2d0a0a00cb5dde6bcb8e561278357b6bb8b76dcc) Signed-off-by: Herman van Hovell <her...@databricks.com> --- .../org/apache/spark/sql/connect/planner/SparkConnectPlanner.scala | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/connector/connect/server/src/main/scala/org/apache/spark/sql/connect/planner/SparkConnectPlanner.scala b/connector/connect/server/src/main/scala/org/apache/spark/sql/connect/planner/SparkConnectPlanner.scala index e81e9bb51cb..46c465e4deb 100644 --- a/connector/connect/server/src/main/scala/org/apache/spark/sql/connect/planner/SparkConnectPlanner.scala +++ b/connector/connect/server/src/main/scala/org/apache/spark/sql/connect/planner/SparkConnectPlanner.scala @@ -1527,6 +1527,11 @@ class SparkConnectPlanner(val sessionHolder: SessionHolder) extends Logging { s"Failed to load class correctly due to $nsm. " + "Make sure the artifact where the class is defined is installed by calling" + " session.addArtifact.") + case cnf: ClassNotFoundException => + throw new ClassNotFoundException( + s"Failed to load class: ${cnf.getMessage}. " + + "Make sure the artifact where the class is defined is installed by calling" + + " session.addArtifact.") case _ => throw t } } --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org