[
https://issues.apache.org/jira/browse/ARROW-14196?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17423328#comment-17423328
]
Joris Van den Bossche commented on ARROW-14196:
-----------------------------------------------
> I'm also surprised that a non-existing column name wouldn't return an error
> instead of selecting nothing?
With the new datasets API, it actually raises an error if a column name is not
found. With the legacy implementation (or the plain {{ParquetFile}} interface),
it ignores those.
(and I was using use_legacy_dataset=True because with Datasets we don't yet
support selecting nested fields ..)
> [C++][Parquet] Default to compliant nested types in Parquet writer
> ------------------------------------------------------------------
>
> Key: ARROW-14196
> URL: https://issues.apache.org/jira/browse/ARROW-14196
> Project: Apache Arrow
> Issue Type: Improvement
> Components: C++, Parquet
> Reporter: Joris Van den Bossche
> Priority: Major
>
> In C++ there is already an option to get the "compliant_nested_types" (to
> have the list columns follow the Parquet specification), and ARROW-11497
> exposed this option in Python.
> This is still set to False by default, but in the source it says "TODO: At
> some point we should flip this.", and in ARROW-11497 there was also some
> discussion about what it would take to change the default.
> cc [~emkornfield] [~apitrou]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
