[ https://issues.apache.org/jira/browse/SPARK-30494?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17064093#comment-17064093 ]
Dongjoon Hyun commented on SPARK-30494: --------------------------------------- Hi, [~cltlfcjin]. How about older versions, 2.3.4 and 2.4.5? > Duplicates cached RDD when create or replace an existing view > ------------------------------------------------------------- > > Key: SPARK-30494 > URL: https://issues.apache.org/jira/browse/SPARK-30494 > Project: Spark > Issue Type: Bug > Components: SQL > Affects Versions: 3.0.0 > Reporter: Lantao Jin > Priority: Major > > We can reproduce by below commands: > {code} > beeline> create or replace temporary view temp1 as select 1 > beeline> cache table tempView > beeline> create or replace temporary view temp1 as select 1, 2 > beeline> cache table tempView > {code} > The cached RDD for plan "select 1" stays in memory forever until the session > close. This cached data cannot be used since the view temp1 has been replaced > by another plan. It's a memory leak. > assert(spark.sharedState.cacheManager.lookupCachedData(sql("select 1, > 2")).isDefined) > assert(spark.sharedState.cacheManager.lookupCachedData(sql("select > 1")).isDefined) -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org