[
https://issues.apache.org/jira/browse/SPARK-61099?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
DB Tsai updated SPARK-61099:
----------------------------
Fix Version/s: connect-rust-4.3.0
> Honor binaryAsBytes when collecting rows
> ----------------------------------------
>
> Key: SPARK-61099
> URL: https://issues.apache.org/jira/browse/SPARK-61099
> Project: Spark
> Issue Type: Bug
> Components: Connect
> Affects Versions: connect-rust-4.3.0
> Reporter: DB Tsai
> Priority: Major
> Fix For: connect-rust-4.3.0
>
>
> Set spark.sql.execution.pyspark.binaryAsBytes to false, then collect SELECT
> X'0102' AS b or iterate it with toLocalIterator(). Official PySpark returns
> bytearray; the Rust-backed Python client always constructs bytes. Honor the
> setting for top-level and nested binary result values and preserve
> user-defined type conversion. Cover collection, lazy iteration and both
> settings with regression tests.
> Observed by source inspection of public apache/spark-connect-rust master
> 83d74462ec41012993176cf3af99017576688eb8. Runtime regression tests will
> accompany the fix.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
