[
https://issues.apache.org/jira/browse/SPARK-61083?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hyukjin Kwon updated SPARK-61083:
---------------------------------
Fix Version/s: connect-rust-4.3.0
> Raise the errors of PySpark for the tags, the table name, the paths of
> copyFromLocalToFs and the JVM attributes of a session in the Python client
> -------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: SPARK-61083
> URL: https://issues.apache.org/jira/browse/SPARK-61083
> Project: Spark
> Issue Type: Bug
> Components: Connect
> Affects Versions: connect-rust-4.3.0
> Reporter: Hyukjin Kwon
> Priority: Major
> Fix For: connect-rust-4.3.0
>
>
> In PySpark Connect, the methods of a session raise these errors for their
> arguments:
> * {{addTag}} and {{removeTag}} raise {{CANNOT_BE_NONE}} for None,
> {{VALUE_ALLOWED}} for a tag of a comma, and {{VALUE_NOT_NON_EMPTY_STR}} for
> an empty tag, of the {{arg_name}} "Spark Connect tag".
> * {{table}} raises {{NOT_EXPECTED_TYPE}} for a name that is not a str.
> * {{copyFromLocalToFs}} raises {{NO_SCHEMA_AND_DRIVER_DEFAULT_SCHEME}} for a
> destination of a scheme, and an {{AssertionError}} ("local path must be a
> file path.") for a local path that is not a file.
> * {{_jsc}}, {{_jconf}}, {{_jvm}}, {{_jsparkSession}} and {{sparkContext}}
> raise the {{PySparkAttributeError}} {{JVM_ATTRIBUTE_NOT_SUPPORTED}}.
> The Rust-backed Python client raises {{INVALID_TAG}} for the tags of
> {{addTag}}, takes any tag in {{removeTag}}, raises a plain {{TypeError}} of
> {{table(1)}}, a {{PySparkRuntimeError}} of a missing local file, and a plain
> {{AttributeError}} of {{sparkContext}}. Raise them as PySpark does.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
