kylebarron opened a new issue, #7364:
URL: https://github.com/apache/arrow-rs/issues/7364
**Describe the bug**
The `StatisticsConverter` produces all-null columns for struct-type fields.
**To Reproduce**
```rs
#[cfg(test)]
mod test_geoparquet {
use std::sync::Arc;
use arrow::array::AsArray;
use arrow::datatypes::Float32Type;
use object_store::aws::AmazonS3Builder;
use parquet::arrow::arrow_reader::ArrowReaderMetadata;
use parquet::arrow::async_reader::ParquetObjectReader;
use super::*;
#[tokio::test]
async fn test_struct_geoparquet() {
let store = Arc::new(
AmazonS3Builder::new()
.with_bucket_name("overturemaps-us-west-2")
.with_skip_signature(true)
.with_region("us-west-2")
.build()
.unwrap(),
);
let path =
"release/2025-02-19.0/theme=addresses/type=address/part-00010-e084a2d7-fea9-41e5-a56f-e638a3307547-c000.zstd.parquet";
let mut object_reader = ParquetObjectReader::new(store, path.into());
let meta = ArrowReaderMetadata::load_async(&mut object_reader,
Default::default())
.await
.unwrap();
let parquet_schema = meta.parquet_schema();
let column_desc = parquet_schema.column(2);
let min_bytes = meta
.metadata()
.row_group(0)
.column(2)
.statistics()
.unwrap()
.min_bytes_opt()
.unwrap();
let statistics_value_direct =
f32::from_le_bytes(min_bytes.try_into().unwrap());
let converter =
StatisticsConverter::try_new("bbox", meta.schema(),
meta.parquet_schema()).unwrap();
let mins = converter
.row_group_mins(meta.metadata().row_groups())
.unwrap();
let mins_struct = mins.as_struct();
let minx_bbox_minx =
mins_struct.column(0).as_primitive::<Float32Type>();
// This line fails
assert!(minx_bbox_minx.is_valid(0));
let statistics_value_via_converter = minx_bbox_minx.value(0);
assert_eq!(statistics_value_direct, statistics_value_via_converter)
}
}
```
**Expected behavior**
Some way to handle struct columns in `StatisticsConverter`.
**Additional context**
It looks like this issue has been documented in datafusion, before the
`StatisticsConverter` was moved into `parquet` core.
https://github.com/apache/datafusion/issues/10609,
https://github.com/apache/datafusion/issues/8334
It doesn't look like there was previously an issue in this repo for this.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
