viirya commented on code in PR #38558:
URL: https://github.com/apache/spark/pull/38558#discussion_r1018726898
##########
sql/core/src/main/scala/org/apache/spark/sql/execution/columnar/InMemoryTableScanExec.scala:
##########
@@ -111,10 +112,15 @@ case class InMemoryTableScanExec(
override def output: Seq[Attribute] = attributes
+ private def cachedPlan = relation.cachedPlan match {
+ case adaptive: AdaptiveSparkPlanExec if adaptive.isFinalized =>
adaptive.executedPlan
+ case other => other
Review Comment:
I think for good side, this can improve AQE coverage for some limited cases.
Although I'm also worrying about some inconsistent behaviors that could be seen
by end-users regarding shuffling/sorting. They might ask for questions why
sometimes shuffle/sort is added but sometimes it isn't.
I guess that might require users more tricks on optimizing cached AQE
relation in practice.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
