[
https://issues.apache.org/jira/browse/SPARK-61057?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hyukjin Kwon updated SPARK-61057:
---------------------------------
Fix Version/s: connect-rust-4.3.0
> Take the arguments of DataStreamReader in the Python client as PySpark does
> ---------------------------------------------------------------------------
>
> Key: SPARK-61057
> URL: https://issues.apache.org/jira/browse/SPARK-61057
> Project: Spark
> Issue Type: Bug
> Components: Connect
> Affects Versions: connect-rust-4.3.0
> Reporter: Hyukjin Kwon
> Priority: Major
> Fix For: connect-rust-4.3.0
>
>
> In PySpark Connect, {{spark.readStream.name(source_name)}} raises
> {{NOT_EXPECTED_TYPE}} of {{source_name}} for one that is not a str, and the
> {{PySparkValueError}} {{INVALID_STREAMING_SOURCE_NAME}} for one of other
> characters than ASCII letters, digits and underscores. {{load(path)}} raises
> {{VALUE_NOT_NON_EMPTY_STR}} of {{path}} for a path that is not a non-empty
> str, and {{csv}}, {{json}}, {{orc}}, {{parquet}}, {{text}} and {{xml}} raise
> {{NOT_EXPECTED_TYPE}} of {{path}} for one that is not a str.
> {{changes(tableName)}} of a reader of a schema raises the
> {{AnalysisException}} "User specified schema not supported with `changes`.".
> The Rust-backed Python client raises {{TypeError}}s and a plain
> {{ValueError}} for these, sends an empty path to the server, and sends
> {{changes}} of a schema to the server, which raises another error. Raise them
> as PySpark does.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
