[
https://issues.apache.org/jira/browse/SPARK-12890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15115144#comment-15115144
]
Liang-Chi Hsieh commented on SPARK-12890:
-----------------------------------------
For the original issue, I think it might because you enable schema merging. In
order to get the correct schema, it will scan all footer and parquet files to
merge their schema. Try to disable the schema merging if you don't need it, and
see if it solves your problem.
> Spark SQL query related to only partition fields should not scan the whole
> data.
> --------------------------------------------------------------------------------
>
> Key: SPARK-12890
> URL: https://issues.apache.org/jira/browse/SPARK-12890
> Project: Spark
> Issue Type: Improvement
> Components: SQL
> Reporter: Prakash Chockalingam
>
> I have a SQL query which has only partition fields. The query ends up
> scanning all the data which is unnecessary.
> Example: select max(date) from table, where the table is partitioned by date.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
