shahidki31 commented on issue #24715: [SPARK-25474][SQL] Data source tables support fallback to HDFS for size estimation URL: https://github.com/apache/spark/pull/24715#issuecomment-522293215 If we do 3 times computes statistics like, ``` sql(s"EXPLAIN COST SELECT * FROM spark_25474 '") sql(s"EXPLAIN COST SELECT * FROM spark_25474 '") sql(s"EXPLAIN COST SELECT * FROM spark_25474 '") ``` Then all the 3 times it falls back to HDFS or only the first time, as the table statistics hasn't changed? Then shouldn't we update the table statistics after it computes first time, and next time onwards we can directly gets from the catalog table?.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
