DB Tsai created SPARK-61098:
-------------------------------
Summary: Deliver streaming query start callbacks before returning
the query
Key: SPARK-61098
URL: https://issues.apache.org/jira/browse/SPARK-61098
Project: Spark
Issue Type: Bug
Components: Connect
Affects Versions: connect-rust-4.3.0
Reporter: DB Tsai
DataStreamWriter.start() reads the query ID and name from
WriteStreamOperationStartResult but ignores query_started_event_json. The
listener dispatcher accepts only progress, idle and termination events.
Consequently a registered onQueryStarted callback is never called. Official
PySpark dispatches this callback synchronously before start() returns. Deliver
the start event exactly once to the session listeners and preserve callback
error behavior, with regression coverage.
Observed by source inspection of public apache/spark-connect-rust master
83d74462ec41012993176cf3af99017576688eb8. Runtime regression tests will
accompany the fix.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
