Hyukjin Kwon created SPARK-61397:
------------------------------------

             Summary: Raise SparkConnectException for a malformed chunked Arrow 
batch, as PySpark does
                 Key: SPARK-61397
                 URL: https://issues.apache.org/jira/browse/SPARK-61397
             Project: Spark
          Issue Type: Bug
          Components: Connect
    Affects Versions: connect-rust-4.3.0
            Reporter: Hyukjin Kwon


A follow-up of SPARK-61262, found in a post-merge review. The chunk assembly 
errors of ExecutePlan (an unexpected chunk index, a missing first chunk of the 
next batch, an unexpected row count) are raised as PySparkRuntimeError in the 
Python client, where PySpark raises SparkConnectException, so except 
SparkConnectException does not catch them. The core client's raw stream also 
yields the partial chunks without saying so in its documentation.



--
This message was sent by Atlassian Jira
(v10.3.26#10030026)

---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to