cloud-fan commented on code in PR #38558: URL: https://github.com/apache/spark/pull/38558#discussion_r1018732458
########## sql/core/src/main/scala/org/apache/spark/sql/execution/columnar/InMemoryTableScanExec.scala: ########## @@ -111,10 +112,15 @@ case class InMemoryTableScanExec( override def output: Seq[Attribute] = attributes + private def cachedPlan = relation.cachedPlan match { + case adaptive: AdaptiveSparkPlanExec if adaptive.isFinalized => adaptive.executedPlan + case other => other Review Comment: Another idea is to materialize the AQE plan eagerly so that even the first cache access can be optimized. However, this requires triggering query execution during query planning, which is a bit risky. A good practice is to ask users to do query caching eagerly, e.g. do a `df.count` right after the `df` is cached. Then they won't observe inconsistencies. Anyway, I think this PR is a net win as it optimizes all the following cache accesses after the first access. This is important for query caching as cache is meant to be accessed repeatedly. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org