[ https://issues.apache.org/jira/browse/SPARK-34149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Maxim Gekk updated SPARK-34149: ------------------------------- Description: For example, the test below: {code:scala} test("SPARK-XXXXX: refresh cache in partition adding") { withNamespaceAndTable("ns", "tbl") { t => sql(s"CREATE TABLE $t (part int) $defaultUsing PARTITIONED BY (part)") sql(s"ALTER TABLE $t ADD PARTITION (part=0)") assert(!spark.catalog.isCached(t)) sql(s"CACHE TABLE $t") assert(spark.catalog.isCached(t)) checkAnswer(sql(s"SELECT * FROM $t"), Row(0)) sql(s"ALTER TABLE $t ADD PARTITION (part=1)") assert(spark.catalog.isCached(t)) checkAnswer(sql(s"SELECT * FROM $t"), Seq(Row(0), Row(1))) } } {code} fails with; {code} !== Correct Answer - 2 == == Spark Answer - 1 == !struct<> struct<part:int> [0] [0] ![1] ScalaTestFailureLocation: org.apache.spark.sql.QueryTest$ at (QueryTest.scala:243) {code} because the command doesn't refresh the cache. > DSv2: `ALTER TABLE .. ADD PARTITION` does not refresh table cache > ----------------------------------------------------------------- > > Key: SPARK-34149 > URL: https://issues.apache.org/jira/browse/SPARK-34149 > Project: Spark > Issue Type: Sub-task > Components: SQL > Affects Versions: 3.2.0 > Reporter: Maxim Gekk > Priority: Major > > For example, the test below: > {code:scala} > test("SPARK-XXXXX: refresh cache in partition adding") { > withNamespaceAndTable("ns", "tbl") { t => > sql(s"CREATE TABLE $t (part int) $defaultUsing PARTITIONED BY (part)") > sql(s"ALTER TABLE $t ADD PARTITION (part=0)") > assert(!spark.catalog.isCached(t)) > sql(s"CACHE TABLE $t") > assert(spark.catalog.isCached(t)) > checkAnswer(sql(s"SELECT * FROM $t"), Row(0)) > sql(s"ALTER TABLE $t ADD PARTITION (part=1)") > assert(spark.catalog.isCached(t)) > checkAnswer(sql(s"SELECT * FROM $t"), Seq(Row(0), Row(1))) > } > } > {code} > fails with; > {code} > !== Correct Answer - 2 == == Spark Answer - 1 == > !struct<> struct<part:int> > [0] [0] > ![1] > > > ScalaTestFailureLocation: org.apache.spark.sql.QueryTest$ at > (QueryTest.scala:243) > {code} > because the command doesn't refresh the cache. -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org