[ https://issues.apache.org/jira/browse/ARROW-8284?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Joris Van den Bossche updated ARROW-8284: ----------------------------------------- Description: In a dataset, one can have timestamp columns with different resolutions. There should be an optional to cast all timestamps to the type mentioned in the schema. A typical example could be that we store a pandas DataFrame with {{ns}} precision to Parquet files that only support {{us}} resolution in their most widespread from. Then the dataset schema and the actual file content don't match anymore. (was: In a dataset, one can timestamp columns with different resolutions. There should be an optional to cast all timestamps to the type mentioned in the schema. A typical example could be that we store a pandas DataFrame with {{ns}} precision to Parquet files that only support {{us}} resolution in their most widespread from. Then the dataset schema and the actual file content don't match anymore.) > [C++][Dataset] Schema evolution for timestamp columns > ----------------------------------------------------- > > Key: ARROW-8284 > URL: https://issues.apache.org/jira/browse/ARROW-8284 > Project: Apache Arrow > Issue Type: Improvement > Components: C++ - Dataset > Reporter: Uwe Korn > Priority: Major > > In a dataset, one can have timestamp columns with different resolutions. > There should be an optional to cast all timestamps to the type mentioned in > the schema. A typical example could be that we store a pandas DataFrame with > {{ns}} precision to Parquet files that only support {{us}} resolution in > their most widespread from. Then the dataset schema and the actual file > content don't match anymore. -- This message was sent by Atlassian Jira (v8.3.4#803005)