[ https://issues.apache.org/jira/browse/SPARK-44462?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Hyukjin Kwon resolved SPARK-44462. ---------------------------------- Resolution: Duplicate > Fix the session passed to foreachBatch. > ---------------------------------------- > > Key: SPARK-44462 > URL: https://issues.apache.org/jira/browse/SPARK-44462 > Project: Spark > Issue Type: Task > Components: Connect, Structured Streaming > Affects Versions: 3.4.1 > Reporter: Raghu Angadi > Priority: Major > > foreachBatch() in Connect uses initial session used while starting the > streaming query. But streaming query uses a cloned session, not the the > original session. We should set up the mapping for the cloned session and > pass that in. Look for this ticket ID in the code for more context inline. > > Another issue with not creating new session id: foreachBatch worker keeps the > session alive. The session mapping at Connect server does not expire and > query keeps running even if the original client disappears. This keeps the > query running. -- This message was sent by Atlassian Jira (v8.20.10#820010) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org