Hyukjin Kwon created SPARK-60911:
------------------------------------
Summary: Raise server errors in Spark Connect Rust with the
exception class and details PySpark gives
Key: SPARK-60911
URL: https://issues.apache.org/jira/browse/SPARK-60911
Project: Spark
Issue Type: Bug
Components: Connect
Affects Versions: connect-rust-4.3.0
Reporter: Hyukjin Kwon
PySpark Connect builds the exception for a server error from the error's
{{ErrorInfo}}, and from {{FetchErrorDetails}} when the server has more. It
picks the class by walking the server's exception classes from the most
specific one, and passes on the error condition, message parameters, SQL state,
server stack trace and query contexts. The Rust-backed Python client raises the
exception with a message only, so these PySpark 4.3.0 results differ:
* A syntax error, as in {{spark.sql("SELEC 1")}}, raises {{ParseException}}.
The client raises {{AnalysisException}}: it looks for {{AnalysisException}} in
the class list before {{ParseException}}, which is a subclass of it.
* {{getCondition()}}, {{getMessageParameters()}} and {{getSqlState()}} return
the server's values, for example {{TABLE_OR_VIEW_NOT_FOUND}}, the
{{relationName}} and {{searchPath}} parameters, and {{42P01}}. The client
returns {{None}} for all three.
* {{str(e)}} is the server's message. The client repeats the condition, as in
{{[TABLE_OR_VIEW_NOT_FOUND] [TABLE_OR_VIEW_NOT_FOUND] The table or view ...}},
and {{getMessage()}} starts with {{[None]}}.
* {{getQueryContext()}} returns the query contexts, such as a
{{SQLQueryContext}} for {{1/0}} in {{SELECT 1/0}}, or a
{{DataFrameQueryContext}} for a column that does not resolve. The client
returns none.
* {{getStackTrace()}} returns the server's stack trace, which the message shows
after "JVM stacktrace:" when {{spark.sql.connect.serverStacktrace.enabled}} is
true, its default. The client has no {{getStackTrace()}} and shows no stack
trace.
* The exceptions are those of {{pyspark.errors.exceptions.connect}}: subclasses
of {{SparkConnectGrpcException}} and of the base classes in {{pyspark.errors}}.
The client raises the base classes.
Build the exceptions as PySpark Connect does.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
