[ https://issues.apache.org/jira/browse/SPARK-30494?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Dongjoon Hyun updated SPARK-30494: ---------------------------------- Fix Version/s: 2.4.6 > Duplicates cached RDD when create or replace an existing view > ------------------------------------------------------------- > > Key: SPARK-30494 > URL: https://issues.apache.org/jira/browse/SPARK-30494 > Project: Spark > Issue Type: Bug > Components: SQL > Affects Versions: 2.0.2, 2.1.3, 2.2.3, 2.3.4, 2.4.5, 3.0.0 > Reporter: Lantao Jin > Assignee: Lantao Jin > Priority: Major > Fix For: 3.0.0, 2.4.6 > > > We can reproduce by below commands: > {code} > beeline> create or replace temporary view temp1 as select 1 > beeline> cache table temp1 > beeline> create or replace temporary view temp1 as select 1, 2 > beeline> cache table temp1 > {code} > The cached RDD for plan "select 1" stays in memory forever until the session > close. This cached data cannot be used since the view temp1 has been replaced > by another plan. It's a memory leak. > assert(spark.sharedState.cacheManager.lookupCachedData(sql("select 1, > 2")).isDefined) > assert(spark.sharedState.cacheManager.lookupCachedData(sql("select > 1")).isDefined) -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org