Haotian Sun created SPARK-60767:
-----------------------------------
Summary: Flush Python worker init responses for initialization
timing
Key: SPARK-60767
URL: https://issues.apache.org/jira/browse/SPARK-60767
Project: Spark
Issue Type: Sub-task
Components: PySpark
Affects Versions: 4.4.0
Reporter: Haotian Sun
The successful init response added in SPARK-60613 can remain buffered until
task output is written, so its receipt cannot mark initialization completion
promptly on the JVM.
Flush the response immediately and timestamp successful receipt in
BasePythonRunner for pythonInitTime. Preserve the existing Python timestamps,
boot and total-time accounting, and finish log calculation. Add buffered-socket
and reader-timing regressions.
Generated-by: Codex (GPT-6)
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
