GitHub user Sephiroth-Lin opened a pull request: https://github.com/apache/spark/pull/7209
[SPARK-8811][SQL] Read array struct data from parquet error JIRA:https://issues.apache.org/jira/browse/SPARK-8811 For example: we have a table: ``` t1(c1 string, c2 string, arr_c1 array<struct<in_c1 string, in_c2 string>>, arr_c2 array<struct<in_c1 string, in_c2 string>>) we save data in parquet. for select * from t1, we know in parquet the fileSchema may be: message hive_schema { optional binary c1; optional binary c2; optional group arr_c1 (LIST) { repeated group bag { optional group array_element { optional binary IN_C1; optional binary IN_C2; } } } optional group arr_c2 (LIST) { repeated group bag { optional group array_element { optional binary IN_C1; optional binary IN_C2; } } } } but the requestSchema is: message root { optional binary c1; optional binary c2; optional group arr_c1 (LIST) { repeated group bag { optional group element { optional binary IN_C1; optional binary IN_C2; } } } optional group arr_c2 (LIST) { repeated group bag { optional group element { optional binary IN_C1; optional binary IN_C2; } } } } ``` You can merge this pull request into a Git repository by running: $ git pull https://github.com/Sephiroth-Lin/spark SPARK-8811 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/spark/pull/7209.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #7209 ---- commit ecd25477abd6735514ab48549a4a937bf6d00f42 Author: linweizhong <linweizh...@huawei.com> Date: 2015-07-03T07:55:00Z Change schema for array type from element to array_element ---- --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org