Eric Liang created SPARK-16596:
----------------------------------
Summary: Refactor DataSourceScanExec to do partition discovery at
execution instead of planning time
Key: SPARK-16596
URL: https://issues.apache.org/jira/browse/SPARK-16596
Project: Spark
Issue Type: Improvement
Components: SQL
Reporter: Eric Liang
Priority: Minor
Partition discovery is rather expensive, so we should do it at execution time
instead of during physical planning. Right now there is not much benefit since
ListingFileCatalog will read scan for all partitions at planning time anyways,
but this can be optimized in the future.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
