[ https://issues.apache.org/jira/browse/SPARK-21579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16107371#comment-16107371 ]
ant_nebula commented on SPARK-21579: ------------------------------------ It will keep ods_table1 in memory, although it will not been used anymore. It waste memory, especially when my service diagram much more complex !screenshot-1.png! > dropTempView has a critical BUG > ------------------------------- > > Key: SPARK-21579 > URL: https://issues.apache.org/jira/browse/SPARK-21579 > Project: Spark > Issue Type: Bug > Components: SQL > Affects Versions: 2.1.1, 2.2.0 > Reporter: ant_nebula > Priority: Critical > Attachments: screenshot-1.png > > > when I dropTempView dwd_table1 only, sub table dwd_table2 also disappear from > http://127.0.0.1:4040/storage/. > It affect version 2.1.1 and 2.2.0, 2.1.0 is ok for this problem. > {code:java} > val spark = > SparkSession.builder.master("local").appName("sparkTest").getOrCreate() > val rows = Seq(Row("p1", 30), Row("p2", 20), Row("p3", 25), Row("p4", 10), > Row("p5", 40), Row("p6", 15)) > val schema = new StructType().add(StructField("name", > StringType)).add(StructField("age", IntegerType)) > val rowRDD = spark.sparkContext.parallelize(rows, 3) > val df = spark.createDataFrame(rowRDD, schema) > df.createOrReplaceTempView("ods_table") > spark.sql("cache table ods_table") > spark.sql("cache table dwd_table1 as select * from ods_table where age>=25") > spark.sql("cache table dwd_table2 as select * from dwd_table1 where > name='p1'") > spark.catalog.dropTempView("dwd_table1") > //spark.catalog.dropTempView("ods_table") > spark.sql("select * from dwd_table2").show() > {code} -- This message was sent by Atlassian JIRA (v6.4.14#64029) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org