Hyukjin Kwon created SPARK-61397:
------------------------------------
Summary: Raise SparkConnectException for a malformed chunked Arrow
batch, as PySpark does
Key: SPARK-61397
URL: https://issues.apache.org/jira/browse/SPARK-61397
Project: Spark
Issue Type: Bug
Components: Connect
Affects Versions: connect-rust-4.3.0
Reporter: Hyukjin Kwon
A follow-up of SPARK-61262, found in a post-merge review. The chunk assembly
errors of ExecutePlan (an unexpected chunk index, a missing first chunk of the
next batch, an unexpected row count) are raised as PySparkRuntimeError in the
Python client, where PySpark raises SparkConnectException, so except
SparkConnectException does not catch them. The core client's raw stream also
yields the partial chunks without saying so in its documentation.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]