[
https://issues.apache.org/jira/browse/SPARK-61532?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hyukjin Kwon updated SPARK-61532:
---------------------------------
Description:
>From a review of SPARK-61470 (Ctrl-C ends the waits of the Python client) and
>its follow-ups. The listener bus of the Python client still differed from
>PySpark's in two cases of a signal:
1. A signal during addListener before the server's response headers came: an
unreachable or slow server, or the plan compression lookup. That signal dropped
the listener from the bus, while PySpark's append, which puts the listener in
the bus first, keeps it. The listener now stays, as for a signal during the
wait for the acknowledgement, and the server's listener, which the ADD may have
created, is removed by the next removal or subscription.
2. A failed Interrupt (the server fails the interrupt of the wait) during
addListener, removeListener or close was raised as the interrupt's error.
PySpark's append and remove catch it with `except Exception`, warn, and return;
append also removes the listener from its bus. These calls now warn and return,
and addListener drops the registration that the interrupted add kept.
Not covered: PySpark's remove blocks on the lock that post_to_all holds while a
callback runs, so a Ctrl-C then sends no REMOVE. The drop-in sends the REMOVE
while a callback runs, so that a callback can make lifecycle calls during an
external close, which
test_streaming_listener_callback_lifecycle_calls_during_external_close pins. In
PySpark those calls deadlock on its non-reentrant lock. Waiting for the
delivery first needs a separate design decision.
was:
>From a review of SPARK-61470 (Ctrl-C ends the waits of the Python client) and
>its follow-ups. The listener bus of the Python client still differed from
>PySpark's in three cases of a signal:
1. removeListener of the last listener (and streams.close()) while a listener
callback runs on the worker. PySpark's remove acquires the lock of the bus,
which post_to_all holds while it calls the listeners, and a signal interrupts
that acquire. So PySpark raises KeyboardInterrupt at once, sends no REMOVE, and
keeps the listener. Ours sent the REMOVE at once, and then waited for the
callback in the join of the worker. The last-listener removal now waits for the
delivery of the worker, as the removal of another listener already does,
through the bus's wait, which a signal ends.
2. A signal during addListener before the server's response headers came: an
unreachable or slow server, or the plan compression lookup. That signal dropped
the listener from the bus, while PySpark's append, which puts the listener in
the bus first, keeps it. The listener now stays, as for a signal during the
wait for the acknowledgement, and the server's listener, which the ADD may have
created, is removed by the next removal or subscription.
3. A failed Interrupt (the server fails the interrupt of the wait) during
addListener, removeListener or close was raised as the interrupt's error.
PySpark's append and remove catch it with `except Exception`, warn, and return;
append also removes the listener from its bus. These calls now warn and return,
and addListener drops the registration that the interrupted add kept.
Summary: Keep the listener of an add that a signal ends before the
server answers, and warn of a failed interrupt of the listener bus, as PySpark
does (was: Let a signal during a callback end the removal of the last listener
before its REMOVE, keep the listener of an add that a signal ends before the
server answers, and warn of a failed interrupt of the listener bus, as PySpark
does)
> Keep the listener of an add that a signal ends before the server answers, and
> warn of a failed interrupt of the listener bus, as PySpark does
> ---------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: SPARK-61532
> URL: https://issues.apache.org/jira/browse/SPARK-61532
> Project: Spark
> Issue Type: Bug
> Components: Connect
> Affects Versions: connect-rust-4.3.0
> Reporter: Hyukjin Kwon
> Priority: Major
> Labels: pull-request-available
> Fix For: connect-rust-4.3.0
>
>
> From a review of SPARK-61470 (Ctrl-C ends the waits of the Python client) and
> its follow-ups. The listener bus of the Python client still differed from
> PySpark's in two cases of a signal:
> 1. A signal during addListener before the server's response headers came: an
> unreachable or slow server, or the plan compression lookup. That signal
> dropped the listener from the bus, while PySpark's append, which puts the
> listener in the bus first, keeps it. The listener now stays, as for a signal
> during the wait for the acknowledgement, and the server's listener, which the
> ADD may have created, is removed by the next removal or subscription.
> 2. A failed Interrupt (the server fails the interrupt of the wait) during
> addListener, removeListener or close was raised as the interrupt's error.
> PySpark's append and remove catch it with `except Exception`, warn, and
> return; append also removes the listener from its bus. These calls now warn
> and return, and addListener drops the registration that the interrupted add
> kept.
> Not covered: PySpark's remove blocks on the lock that post_to_all holds while
> a callback runs, so a Ctrl-C then sends no REMOVE. The drop-in sends the
> REMOVE while a callback runs, so that a callback can make lifecycle calls
> during an external close, which
> test_streaming_listener_callback_lifecycle_calls_during_external_close pins.
> In PySpark those calls deadlock on its non-reentrant lock. Waiting for the
> delivery first needs a separate design decision.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]