[ https://issues.apache.org/jira/browse/SPARK-10301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14721312#comment-14721312 ]
Yin Huai commented on SPARK-10301: ---------------------------------- https://github.com/apache/spark/pull/8515 has been merged. It is not the fix for this issue but will give users a nice error message when the global schema as less struct fields than local parquet file schema (it will ask users to enable schema merging). I am re-targeting this issue to 1.6 for the proper fix (https://github.com/apache/spark/pull/8509). > For struct type, if parquet's global schema has less fields than a file's > schema, data reading will fail > -------------------------------------------------------------------------------------------------------- > > Key: SPARK-10301 > URL: https://issues.apache.org/jira/browse/SPARK-10301 > Project: Spark > Issue Type: Bug > Components: SQL > Affects Versions: 1.5.0 > Reporter: Yin Huai > Assignee: Yin Huai > Priority: Critical > > When parquet's global schema has less number of fields than the local schema > of a file, the data reading path will fail. -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org