[
https://issues.apache.org/jira/browse/ARROW-8284?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joris Van den Bossche updated ARROW-8284:
-----------------------------------------
Description: In a dataset, one can have timestamp columns with different
resolutions. There should be an optional to cast all timestamps to the type
mentioned in the schema. A typical example could be that we store a pandas
DataFrame with {{ns}} precision to Parquet files that only support {{us}}
resolution in their most widespread from. Then the dataset schema and the
actual file content don't match anymore. (was: In a dataset, one can timestamp
columns with different resolutions. There should be an optional to cast all
timestamps to the type mentioned in the schema. A typical example could be that
we store a pandas DataFrame with {{ns}} precision to Parquet files that only
support {{us}} resolution in their most widespread from. Then the dataset
schema and the actual file content don't match anymore.)
> [C++][Dataset] Schema evolution for timestamp columns
> -----------------------------------------------------
>
> Key: ARROW-8284
> URL: https://issues.apache.org/jira/browse/ARROW-8284
> Project: Apache Arrow
> Issue Type: Improvement
> Components: C++ - Dataset
> Reporter: Uwe Korn
> Priority: Major
>
> In a dataset, one can have timestamp columns with different resolutions.
> There should be an optional to cast all timestamps to the type mentioned in
> the schema. A typical example could be that we store a pandas DataFrame with
> {{ns}} precision to Parquet files that only support {{us}} resolution in
> their most widespread from. Then the dataset schema and the actual file
> content don't match anymore.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
