[jira] [Commented] (SPARK-36667) Close resources properly in StateStoreSuite/RocksDBStateStoreSuite
[ https://issues.apache.org/jira/browse/SPARK-36667?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17410293#comment-17410293 ] Apache Spark commented on SPARK-36667: -- User 'HeartSaVioR' has created a pull request for this issue: https://github.com/apache/spark/pull/33916 > Close resources properly in StateStoreSuite/RocksDBStateStoreSuite > -- > > Key: SPARK-36667 > URL: https://issues.apache.org/jira/browse/SPARK-36667 > Project: Spark > Issue Type: Bug > Components: Structured Streaming >Affects Versions: 3.2.0 >Reporter: Jungtaek Lim >Priority: Major > > The StateStoreProvider instances created from "newStoreProvider" are NOT > automatically closed. > While this is trivial for HDFSBackedStateStoreProvider, for > RocksDBStateStoreProvider we leak RocksDB instance as well which should have > closed. Most tests in the RocksDBStateStoreSuite initialize > RocksDBStateStoreProvider, meaning that 60+ RocksDB instances are not closed > in the suite. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-36667) Close resources properly in StateStoreSuite/RocksDBStateStoreSuite
[ https://issues.apache.org/jira/browse/SPARK-36667?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17409779#comment-17409779 ] Jungtaek Lim commented on SPARK-36667: -- Will submit a PR soon. > Close resources properly in StateStoreSuite/RocksDBStateStoreSuite > -- > > Key: SPARK-36667 > URL: https://issues.apache.org/jira/browse/SPARK-36667 > Project: Spark > Issue Type: Bug > Components: Structured Streaming >Affects Versions: 3.2.0 >Reporter: Jungtaek Lim >Priority: Major > > The StateStoreProvider instances created from "newStoreProvider" are NOT > automatically closed. > While this is trivial for HDFSBackedStateStoreProvider, for > RocksDBStateStoreProvider we leak RocksDB instance as well which should have > closed. Most tests in the RocksDBStateStoreSuite initialize > RocksDBStateStoreProvider, meaning that 60+ RocksDB instances are not closed > in the suite. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
