[ https://issues.apache.org/jira/browse/SPARK-12890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15115204#comment-15115204 ]
Hyukjin Kwon commented on SPARK-12890: -------------------------------------- Actually I don't still understand what is an issue here. This might not be merging schemas as it is disabled by default and any filter is not being pushed down here. I mean, the referenced column would be {{date}} and given filters would be empty. So it tries to read all the files regardless of file format as long as it supports to partitioned files. > Spark SQL query related to only partition fields should not scan the whole > data. > -------------------------------------------------------------------------------- > > Key: SPARK-12890 > URL: https://issues.apache.org/jira/browse/SPARK-12890 > Project: Spark > Issue Type: Improvement > Components: SQL > Reporter: Prakash Chockalingam > > I have a SQL query which has only partition fields. The query ends up > scanning all the data which is unnecessary. > Example: select max(date) from table, where the table is partitioned by date. -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org