[ https://issues.apache.org/jira/browse/SPARK-12890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15115558#comment-15115558 ]
Simeon Simeonov commented on SPARK-12890: ----------------------------------------- [~viirya] If schema merging is the cause of the problem then this is clearly a bug. The resulting schema for a query using only partition columns is completely independent of the schema in the data files. There is no merging to do at all. > Spark SQL query related to only partition fields should not scan the whole > data. > -------------------------------------------------------------------------------- > > Key: SPARK-12890 > URL: https://issues.apache.org/jira/browse/SPARK-12890 > Project: Spark > Issue Type: Improvement > Components: SQL > Reporter: Prakash Chockalingam > > I have a SQL query which has only partition fields. The query ends up > scanning all the data which is unnecessary. > Example: select max(date) from table, where the table is partitioned by date. -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org