Hyukjin Kwon created SPARK-61388:
------------------------------------

             Summary: Interrupt only the execution that an interrupted call 
waits for, in the Python client as PySpark does
                 Key: SPARK-61388
                 URL: https://issues.apache.org/jira/browse/SPARK-61388
             Project: Spark
          Issue Type: Bug
          Components: Connect
    Affects Versions: connect-rust-4.3.0
            Reporter: Hyukjin Kwon


A follow-up of SPARK-61259 and SPARK-61353, found in a post-merge review. The 
in-flight registry that a SIGINT interrupts is keyed by thread only, and its 
innermost entry wins, so SIGINT during next() of one toLocalIterator interrupts 
another half-consumed iterator of the thread instead. With such an iterator or 
a streaming listener alive on the thread, SIGINT during SparkSession.stop() 
also interrupts that execution and then waits for the release. The watch also 
latches the first signal it sees when no execution is in flight (for example 
during the plan compression Config RPC), so an execution that the same call 
starts afterwards runs to its end before KeyboardInterrupt is raised.



--
This message was sent by Atlassian Jira
(v10.3.26#10030026)

---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to