Hyukjin Kwon created SPARK-61057:
------------------------------------
Summary: Take the arguments of DataStreamReader in the Python
client as PySpark does
Key: SPARK-61057
URL: https://issues.apache.org/jira/browse/SPARK-61057
Project: Spark
Issue Type: Bug
Components: Connect
Affects Versions: connect-rust-4.3.0
Reporter: Hyukjin Kwon
In PySpark Connect, {{spark.readStream.name(source_name)}} raises
{{NOT_EXPECTED_TYPE}} of {{source_name}} for one that is not a str, and the
{{PySparkValueError}} {{INVALID_STREAMING_SOURCE_NAME}} for one of other
characters than ASCII letters, digits and underscores. {{load(path)}} raises
{{VALUE_NOT_NON_EMPTY_STR}} of {{path}} for a path that is not a non-empty str,
and {{csv}}, {{json}}, {{orc}}, {{parquet}}, {{text}} and {{xml}} raise
{{NOT_EXPECTED_TYPE}} of {{path}} for one that is not a str.
{{changes(tableName)}} of a reader of a schema raises the {{AnalysisException}}
"User specified schema not supported with `changes`.".
The Rust-backed Python client raises {{TypeError}}s and a plain {{ValueError}}
for these, sends an empty path to the server, and sends {{changes}} of a schema
to the server, which raises another error. Raise them as PySpark does.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
