[
https://issues.apache.org/jira/browse/SPARK-60678?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated SPARK-60678:
-----------------------------------
Labels: pull-request-available (was: )
> Accept DDL return schemas for Python UDTFs in Spark Connect Rust
> ----------------------------------------------------------------
>
> Key: SPARK-60678
> URL: https://issues.apache.org/jira/browse/SPARK-60678
> Project: Spark
> Issue Type: Bug
> Components: Connect
> Affects Versions: connect-rust-4.3.0
> Reporter: DB Tsai
> Assignee: DB Tsai
> Priority: Major
> Labels: pull-request-available
> Fix For: connect-rust-4.3.0
>
>
> The Rust Python client rejects the public Python UDTF returnType string form
> (for example a struct DDL schema) with TypeError: argument return_type: str
> object is not an instance of DataType. Official PySpark Connect accepts this
> API. The benchmark udf-udtf-construct, udf-udtf-first-query and
> udf-udtf-repeat-query all fail on Rust and pass on official Python. Support
> the string schema form with matching parsing/validation and retain support
> for StructType. Add focused positive/negative compatibility and plan/protocol
> tests; rerun the three benchmark cases and UDF regression guards. Baseline
> Rust: d884e347452c. This is the client schema serialization path, not
> worker-side UDTF execution.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
