tustvold opened a new issue, #6381:
URL: https://github.com/apache/arrow-datafusion/issues/6381
### Is your feature request related to a problem or challenge?
Currently SchemaAdapter::map_projections returns an error if the provided
`file_schema` contains columns of a different type to the expected
`table_schema`.
In order to accomodate schema evolution, it would be advantageous to be able
to instead cast the array data from the file schema to the table schema where
possible.
### Describe the solution you'd like
```
impl SchemaAdapter {
fn map_schema(&self, file_schema: &Schema) -> Result<SchemaMapping> {
...
}
}
struct SchemaMapping {
...
}
impl SchemaMapping {
fn map_batch(batch: RecordBatch) -> Result<RecordBatch>;
}
```
Where `map_schema` would use
[can_cast_types](https://docs.rs/arrow-cast/latest/arrow_cast/cast/fn.can_cast_types.html)
to determine if conversion is possible, and `map_batch` would use
[cast](https://docs.rs/arrow-cast/latest/arrow_cast/cast/fn.cast.html) to
perform this conversion
### Describe alternatives you've considered
We could not do this
### Additional context
_No response_
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
