Jungtaek Lim created SPARK-36764:
------------------------------------
Summary: Fix race-condition on "ensure continuous stream is being
used" in KafkaContinuousTest
Key: SPARK-36764
URL: https://issues.apache.org/jira/browse/SPARK-36764
Project: Spark
Issue Type: Bug
Components: Structured Streaming
Affects Versions: 3.1.2, 3.0.3, 2.4.8, 3.2.0
Reporter: Jungtaek Lim
The test “ensure continuous stream is being used“ in KafkaContinuousTestquickly
checks the actual type of the execution, and stop the query. Stopping the
streaming query in continuous mode is done by interrupting query execution
thread and join indefinitely.
In parallel, started streaming query is going to generate execution plan,
including running optimizer. Some parts of SessionState can be built at that
time, as they are defined as lazy. The problem is, some of them seem to be able
to “swallow” the InterruptedException and let the thread run continuously.
That said, the query can’t indicate whether there is a request on stopping
query, so the query won’t stop.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
