L. C. Hsieh created SPARK-61440:
-----------------------------------
Summary: Make the retries of the calls of the Spark Connect Rust
client in one place, and do not panic on a stream deadline too far to be a time
Key: SPARK-61440
URL: https://issues.apache.org/jira/browse/SPARK-61440
Project: Spark
Issue Type: Improvement
Components: Connect
Affects Versions: connect-rust-4.3.0
Reporter: L. C. Hsieh
The Rust client (crates/spark-connect-core/src/client.rs) retries its calls in
four hand-written loops: the unary and stream-opening calls (with_retry), the
artifact upload, the background ReleaseExecute, and the reattach of a result
stream. Each has its own copy of the decision "can the policy retry this
failure, is a retry left, wait its backoff", so a change to the retry policy
has to be made four times, and the loops can drift apart.
This makes that decision in one place in retries.rs, as PySpark's Retrying: a
retry of the policy with its backoff, and the retry made at once that PySpark
raises as a RetryException, bounded by its elapsed time. The differences
between the loops that are on purpose stay, as parameters: the calls of a
client check that the client is still open before a retry and ReleaseExecute
does not, an attempt of an upload can fail with an error that is not retried,
and the reader of a reattachable stream shares the attempts of the policy among
the failures of a response (SPARK-61415).
Two related fixes in the same code:
* The deadline of a stream is computed as Instant::now() + deadline at three
places, which panics for a configured deadline too large to add to the time. A
deadline that cannot be a time is never reached, so it is none.
* INVALID_CURSOR.DISCONNECTED and INVALID_HANDLE.OPERATION_NOT_FOUND /
SESSION_NOT_FOUND are recognized by their name in the message of the error, as
PySpark does. The match stays, in one named function with tests.
No other change of behavior.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]