[
https://issues.apache.org/jira/browse/SPARK-39218?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hyukjin Kwon updated SPARK-39218:
---------------------------------
Affects Version/s: 3.1.2
3.0.3
3.3.0
> Python foreachBatch streaming query cannot be stopped gracefully after pin
> thread mode is enabled
> -------------------------------------------------------------------------------------------------
>
> Key: SPARK-39218
> URL: https://issues.apache.org/jira/browse/SPARK-39218
> Project: Spark
> Issue Type: Bug
> Components: PySpark, Structured Streaming
> Affects Versions: 3.0.3, 3.1.2, 3.2.1, 3.3.0
> Reporter: Hyukjin Kwon
> Priority: Major
>
> For example,
> {code}
> import time
> def func(batch_df, batch_id):
> time.sleep(10)
> print(batch_df.count())
> q =
> spark.readStream.format("rate").load().writeStream.foreachBatch(func).start()
> time.sleep(5)
> q.stop()
> {code}
> works find with pinned thread mode is disabled. Whe pinned thread mode is
> enabled:
> {code}
> {code}
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
