[ https://issues.apache.org/jira/browse/SPARK-24867?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16554986#comment-16554986 ]
Saisai Shao commented on SPARK-24867: ------------------------------------- [~smilegator] what's the ETA of this issue? > Add AnalysisBarrier to DataFrameWriter > --------------------------------------- > > Key: SPARK-24867 > URL: https://issues.apache.org/jira/browse/SPARK-24867 > Project: Spark > Issue Type: Bug > Components: SQL > Affects Versions: 2.3.0, 2.3.1 > Reporter: Xiao Li > Assignee: Xiao Li > Priority: Blocker > > {code} > val udf1 = udf({(x: Int, y: Int) => x + y}) > val df = spark.range(0, 3).toDF("a") > .withColumn("b", udf1($"a", udf1($"a", lit(10)))) > df.cache() > df.write.saveAsTable("t") > df.write.saveAsTable("t1") > {code} > Cache is not being used because the plans do not match with the cached plan. > This is a regression caused by the changes we made in AnalysisBarrier, since > not all the Analyzer rules are idempotent. We need to fix it to Spark 2.3.2 -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org