[ https://issues.apache.org/jira/browse/SPARK-33987?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17258179#comment-17258179 ]
Maxim Gekk commented on SPARK-33987: ------------------------------------ I am working on this > v2 ALTER TABLE .. DROP PARTITION does not refresh cached table > -------------------------------------------------------------- > > Key: SPARK-33987 > URL: https://issues.apache.org/jira/browse/SPARK-33987 > Project: Spark > Issue Type: Sub-task > Components: SQL > Affects Versions: 3.2.0 > Reporter: Maxim Gekk > Priority: Major > > The test below portraits the issue: > {code:scala} > test("SPARK-33950: refresh cache after partition dropping") { > withNamespaceAndTable("ns", "tbl") { t => > sql(s"CREATE TABLE $t (id int, part int) $defaultUsing PARTITIONED BY > (part)") > sql(s"INSERT INTO $t PARTITION (part=0) SELECT 0") > sql(s"INSERT INTO $t PARTITION (part=1) SELECT 1") > assert(!spark.catalog.isCached(t)) > sql(s"CACHE TABLE $t") > assert(spark.catalog.isCached(t)) > QueryTest.checkAnswer(sql(s"SELECT * FROM $t"), Seq(Row(0, 0), Row(1, > 1))) > sql(s"ALTER TABLE $t DROP PARTITION (part=0)") > assert(spark.catalog.isCached(t)) > QueryTest.checkAnswer(sql(s"SELECT * FROM $t"), Seq(Row(1, 1))) > } > } > {code} > The last check fails: > {code} > == Results == > !== Correct Answer - 1 == == Spark Answer - 2 == > !struct<> struct<id:int,part:int> > ![1,1] [0,0] > ! [1,1] > {code} > -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org