ChenMichael commented on pull request #34684: URL: https://github.com/apache/spark/pull/34684#issuecomment-976858837
Oh I realized if a plan containing InMemoryRelation is converted to a RDD, but the RDD is never submitted to DAG scheduler, then the stats would also be inaccurate. I think in this case we would also see the problem with AQE off. Let me try to come up with a test for this scenario. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
