niebayes opened a new issue, #8780:
URL: https://github.com/apache/arrow-rs/issues/8780
The `skip_arrow_schema` option in the `ArrowReaderOptions` actually refers
to skipping decoding parquet file key-value metadata.
See the following code:
``` Rust
/// Create a new [`ArrowReaderMetadata`]
///
/// # Notes
///
/// This function does not attempt to load the PageIndex if not present
in the metadata.
/// See [`Self::load`] for more details.
pub fn try_new(metadata: Arc<ParquetMetaData>, options:
ArrowReaderOptions) -> Result<Self> {
match options.supplied_schema {
Some(supplied_schema) => Self::with_supplied_schema(metadata,
supplied_schema.clone()),
None => {
let kv_metadata = match options.skip_arrow_metadata {
true => None,
false => metadata.file_metadata().key_value_metadata(),
};
let (schema, fields) = parquet_to_arrow_schema_and_fields(
metadata.file_metadata().schema_descr(),
ProjectionMask::all(),
kv_metadata,
)?;
Ok(Self {
metadata,
schema: Arc::new(schema),
fields: fields.map(Arc::new),
})
}
}
}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
