HyukjinKwon opened a new pull request, #58: URL: https://github.com/apache/spark-connect-rust/pull/58
### What changes were proposed in this pull request? Start the CI Spark Connect server with `spark.connect.execute.reattachable.senderMaxStreamDuration=1200s` in the Python-Connect-parity workflow. ### Why are the changes needed? After #54 merged, `test_parity_arrow_python_udf.py` failed the master parity gate with a whole-file **TIMEOUT (p=0)** — zero tests completed in the 1200s per-file cap — even though the exact same file passes **deterministically locally in ~113s (320 passed, 64 skipped)**, matching the reference client's ~120s. So it is not a client correctness bug and not mere slowness (it ran the full 1200s doing nothing). Root cause: on the loaded single-node CI server, a slow first arrow-Python-UDF worker lets the default `senderMaxStreamDuration` elapse, so the server ends the reattachable `ExecutePlan` response stream early and the client must `ReattachExecute`; that reattach path stalls under these conditions and hangs the whole file. Raising the sender duration keeps the stream open until results are produced, so no reattach is triggered and the file completes normally. (The underlying reattach-path robustness is worth hardening in the client separately; this unblocks the gate without masking a client bug.) ### Does this PR introduce _any_ user-facing change? No — CI configuration only. ### How was this patch tested? CI: the Python Connect parity job on this PR. Locally the file already passes in ~113s. This pull request and its description were written by Isaac. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
