Hyukjin Kwon created SPARK-61362:
------------------------------------
Summary: Report and retry transport failures as grpc-python does,
and send the request fields, keepalive errors and user agent of PySpark
Key: SPARK-61362
URL: https://issues.apache.org/jira/browse/SPARK-61362
Project: Spark
Issue Type: Bug
Components: Connect
Affects Versions: connect-rust-4.3.0
Reporter: Hyukjin Kwon
Low-priority transport findings of the s16 review of the Rust core
(spark-connect-core), each checked against PySpark 4.3.0 / grpc-python 1.76:
- Retries: is_transport_error retried any HTTP/2 error, also a stream that the
server resets with a reason that tonic leaves UNKNOWN (STREAM_CLOSED,
FRAME_SIZE_ERROR, HTTP_1_1_REQUIRED): 15 retries over about 10 minutes, where
grpc-python reports INTERNAL and PySpark does not retry. A GOAWAY that
processed no stream was retried only when tonic happened to report it as
UNKNOWN; grpc-python always reports UNAVAILABLE.
- Codes: a dropped connection is UNKNOWN in tonic but UNAVAILABLE in
grpc-python, and the Rust transport that the pandas-parity gate runs official
PySpark over (RustConnectStub) passed UNKNOWN on, so official PySpark did not
reattach a reset. SparkError::grpc_code and the drop-in's getGrpcStatusCode
reported the tonic codes too.
- Request fields: a user_id gRPC header that PySpark never sends;
client_observed_server_side_session_id on ReleaseExecute, CloneSession,
AddArtifacts and ArtifactStatus, where PySpark sends none; no client_type on
ExecutePlan, AnalyzePlan, Config, Interrupt, FetchErrorDetails,
ReattachExecute, ReleaseExecute and ReleaseSession; CloneSession with the user
agent instead of "python"; and a first ExecutePlan without the observed
server-side session ID.
- A keepalive parameter that is not an integer (grpc_keepalive_time_ms=abc) was
ignored, where PySpark raises ValueError "invalid literal for int() with base
10: 'abc'" when it opens its channel; and the user agent named macOS "macos"
where PySpark's platform.uname().system.lower() gives "darwin".
- Plan compression encoded every relation and command only to compare its size
with the threshold; the plan of a DataFrame that an on_execute_plan hook of the
drop-in put in went uncompressed, where PySpark's hook would put in the plan of
to_proto, which compresses it; and no live test checked that a server runs
compressed plans.
The fix maps transport failures to grpc-python's code (new
retries::grpc_python_code) for retries, errors and the transport seam, sends
the request fields as PySpark per request type, raises PySpark's ValueError for
a bad keepalive parameter, names the OS as Python does, sizes plans with
encoded_len, compresses the plan again after a hook ran, and adds a live test
of compressed plans.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]