alamb opened a new issue, #10824:
URL: https://github.com/apache/arrow-rs/issues/10824
**Which part is this question about**
The new `PageIndex` struct introduced in #10719.
**Describe the problem / reproducer**
`PageIndex::new` is `pub(crate)`, and `PageIndex` has no other constructor
or mutators. `ParquetMetaDataBuilder::set_page_index` takes an
`Option<PageIndex>`, but since external code cannot create (or modify) a
`PageIndex`, the only thing a user outside the `parquet` crate can do with that
method is remove or move an existing index.
Before #10719, external code could build `ParquetMetaData` values containing
page indexes, e.g.:
```rust
ParquetMetaData::new(file_metadata, row_groups)
.into_builder()
.set_column_index(Some(vec![vec![column_index_builder.build().unwrap()]]))
.set_offset_index(Some(vec![vec![offset_index_builder.build()]]))
.build()
```
`ColumnIndexBuilder` and `OffsetIndexBuilder` are still public, so the
individual `ColumnIndexMetaData` / `OffsetIndexMetaData` values can still be
built — they just cannot be assembled into a `PageIndex` / attached to a
`ParquetMetaData` anymore.
**Expected behavior**
External code can construct a `ParquetMetaData` with synthetic page indexes
(DataFusion does this in tests that exercise page-level statistics handling,
e.g. `datafusion/datasource-parquet/src/statistics_order_tests.rs`).
**Actual behavior**
There is no public way to construct or modify a `PageIndex`, so such code no
longer compiles against `main`.
**Additional context**
Found while testing DataFusion against arrow-rs `main` ahead of the arrow 60
release (#9880, apache/datafusion#24366).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
