[ https://issues.apache.org/jira/browse/SPARK-29911?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Lantao Jin updated SPARK-29911: ------------------------------- Description: How to reproduce: 1. create a local temporary view v1 2. cache it in memory 3. close session without drop v1. The application will hold the memory forever. In a long running thrift server scenario. It's worse. {code} 0: jdbc:hive2://localhost:10000> CACHE TABLE testCacheTable AS SELECT 1; CACHE TABLE testCacheTable AS SELECT 1; +---------+--+ | Result | +---------+--+ +---------+--+ No rows selected (1.498 seconds) 0: jdbc:hive2://localhost:10000> !close !close Closing: 0: jdbc:hive2://localhost:10000 0: jdbc:hive2://localhost:10000 (closed)> !connect 'jdbc:hive2://localhost:10000' !connect 'jdbc:hive2://localhost:10000' Connecting to jdbc:hive2://localhost:10000 Enter username for jdbc:hive2://localhost:10000: lajin Enter password for jdbc:hive2://localhost:10000: *** Connected to: Spark SQL (version 3.0.0-SNAPSHOT) Driver: Hive JDBC (version 1.2.1.spark2) Transaction isolation: TRANSACTION_REPEATABLE_READ 1: jdbc:hive2://localhost:10000> select * from testCacheTable; select * from testCacheTable; Error: Error running query: org.apache.spark.sql.AnalysisException: Table or view not found: testCacheTable; line 1 pos 14; 'Project [*] +- 'UnresolvedRelation [testCacheTable] (state=,code=0) {code} was: How to reproduce: 1. create a local temporary view v1 2. cache it in memory 3. close session without drop v1. The application will hold the memory forever. In a long running thrift server scenario. It's worse. {code} 0: jdbc:hive2://localhost:10000> CACHE TABLE testCacheTable AS SELECT 1; CACHE TABLE testCacheTable AS SELECT 1; +---------+--+ | Result | +---------+--+ +---------+--+ No rows selected (1.498 seconds) 0: jdbc:hive2://localhost:10000> !close !close Closing: 0: jdbc:hive2://localhost:10000 0: jdbc:hive2://localhost:10000 (closed)> !connect 'jdbc:hive2://localhost:10000' !connect 'jdbc:hive2://localhost:10000' Connecting to jdbc:hive2://localhost:10000 Enter username for jdbc:hive2://localhost:10000: lajin lajin Enter password for jdbc:hive2://localhost:10000: 123 *** Connected to: Spark SQL (version 3.0.0-SNAPSHOT) Driver: Hive JDBC (version 1.2.1.spark2) Transaction isolation: TRANSACTION_REPEATABLE_READ 1: jdbc:hive2://localhost:10000> select * from testCacheTable; select * from testCacheTable; Error: Error running query: org.apache.spark.sql.AnalysisException: Table or view not found: testCacheTable; line 1 pos 14; 'Project [*] +- 'UnresolvedRelation [testCacheTable] (state=,code=0) {code} > Cache table may memory leak when session stopped > ------------------------------------------------ > > Key: SPARK-29911 > URL: https://issues.apache.org/jira/browse/SPARK-29911 > Project: Spark > Issue Type: Bug > Components: SQL > Affects Versions: 3.0.0 > Reporter: Lantao Jin > Priority: Major > Attachments: Screen Shot 2019-11-15 at 2.03.49 PM.png > > > How to reproduce: > 1. create a local temporary view v1 > 2. cache it in memory > 3. close session without drop v1. > The application will hold the memory forever. In a long running thrift server > scenario. It's worse. > {code} > 0: jdbc:hive2://localhost:10000> CACHE TABLE testCacheTable AS SELECT 1; > CACHE TABLE testCacheTable AS SELECT 1; > +---------+--+ > | Result | > +---------+--+ > +---------+--+ > No rows selected (1.498 seconds) > 0: jdbc:hive2://localhost:10000> !close > !close > Closing: 0: jdbc:hive2://localhost:10000 > 0: jdbc:hive2://localhost:10000 (closed)> !connect > 'jdbc:hive2://localhost:10000' > !connect 'jdbc:hive2://localhost:10000' > Connecting to jdbc:hive2://localhost:10000 > Enter username for jdbc:hive2://localhost:10000: > lajin > Enter password for jdbc:hive2://localhost:10000: > *** > Connected to: Spark SQL (version 3.0.0-SNAPSHOT) > Driver: Hive JDBC (version 1.2.1.spark2) > Transaction isolation: TRANSACTION_REPEATABLE_READ > 1: jdbc:hive2://localhost:10000> select * from testCacheTable; > select * from testCacheTable; > Error: Error running query: org.apache.spark.sql.AnalysisException: Table or > view not found: testCacheTable; line 1 pos 14; > 'Project [*] > +- 'UnresolvedRelation [testCacheTable] (state=,code=0) > {code} -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org