DB Tsai created SPARK-61099:
-------------------------------
Summary: Honor binaryAsBytes when collecting rows
Key: SPARK-61099
URL: https://issues.apache.org/jira/browse/SPARK-61099
Project: Spark
Issue Type: Bug
Components: Connect
Affects Versions: connect-rust-4.3.0
Reporter: DB Tsai
Set spark.sql.execution.pyspark.binaryAsBytes to false, then collect SELECT
X'0102' AS b or iterate it with toLocalIterator(). Official PySpark returns
bytearray; the Rust-backed Python client always constructs bytes. Honor the
setting for top-level and nested binary result values and preserve user-defined
type conversion. Cover collection, lazy iteration and both settings with
regression tests.
Observed by source inspection of public apache/spark-connect-rust master
83d74462ec41012993176cf3af99017576688eb8. Runtime regression tests will
accompany the fix.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
