[
https://issues.apache.org/jira/browse/SPARK-61389?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hyukjin Kwon updated SPARK-61389:
---------------------------------
Description: A follow-up of SPARK-61319 and SPARK-61362, found in a
post-merge review. The port of a connection string is parsed with Rust's u16
parser: sc://localhost:99999 reports that the port could not be cast to an
integer where PySpark reports 'Port out of range 0-65535', and a port with a
leading + is accepted where urllib rejects it. These errors, and the keepalive
int() error of SPARK-61362, are raised as PySparkValueError where PySpark
raises the builtin ValueError. (was: A follow-up of SPARK-61319, found in a
post-merge review. A non-integer grpc_keepalive_time_ms or
grpc_keepalive_timeout_ms in the connection string is silently ignored
(int_param falls back to the default), where PySpark raises ValueError (invalid
literal for int()). An out-of-range port reports that the port could not be
cast to an integer, where PySpark reports 'Port out of range 0-65535'. The user
agent sent as client_type reports os/macos on macOS, where PySpark reports
platform.uname().system.lower() (os/darwin).)
Summary: Parse the port of the connection string as urllib does, and
raise the builtin ValueError for a bad port or keepalive value, as PySpark
does, in Spark Connect Rust (was: Raise for a bad keepalive value or port in
the connection string, and report the OS in the user agent as PySpark does, in
Spark Connect Rust)
> Parse the port of the connection string as urllib does, and raise the builtin
> ValueError for a bad port or keepalive value, as PySpark does, in Spark
> Connect Rust
> ------------------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: SPARK-61389
> URL: https://issues.apache.org/jira/browse/SPARK-61389
> Project: Spark
> Issue Type: Bug
> Components: Connect
> Affects Versions: connect-rust-4.3.0
> Reporter: Hyukjin Kwon
> Priority: Major
> Fix For: connect-rust-4.3.0
>
>
> A follow-up of SPARK-61319 and SPARK-61362, found in a post-merge review. The
> port of a connection string is parsed with Rust's u16 parser:
> sc://localhost:99999 reports that the port could not be cast to an integer
> where PySpark reports 'Port out of range 0-65535', and a port with a leading
> + is accepted where urllib rejects it. These errors, and the keepalive int()
> error of SPARK-61362, are raised as PySparkValueError where PySpark raises
> the builtin ValueError.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]