neilz commented on issue #18055:
URL: https://github.com/apache/arrow/issues/18055#issuecomment-3132984441
> In my opinion the biggest advantage of using the `_common_metadata` is in
case of schema evolution.
I am running into this right now where my dataset is date-partitioned and
the schema expands over time. It would be very convenient if the ParquetDataset
automatically were to detect and use `<dataset_path>/_common_metadata` as the
schema. The alternative is to tell users to always call
```
import pyarrow.parquet as pq
URI = "<path_to_dataset>"
s = pq.read_metadata(f"{URI}/_common_metadata").schema
pd.read_table(
URI,
schema=s.to_arrow_schema()
)
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
