Ted-Jiang opened a new pull request, #1791:
URL: https://github.com/apache/arrow-rs/pull/1791
# Which issue does this PR close?
Closes #1790.
# Rationale for this change
For now row group filter in datafusion pass a closure to arrow-rs
```
fn build_row_group_predicate(
pruning_predicate: &PruningPredicate,
metrics: ParquetFileMetrics,
) -> Box<dyn FnMut(&RowGroupMetaData, usize) -> bool> {
```
https://github.com/apache/arrow-datafusion/blob/585bc3a629b92ea7a86ebfe8bf762dbef4155710/datafusion/core/src/physical_plan/file_format/parquet.rs#L559-L562
So for page filter i try to pass
```
Box<dyn FnMut(&[pageIndex], &[pageLocation], usize) -> &[ranges]>
```
then use ranges to construct `RowRanges ` : row ranges in a row-group (one
col) if col is sorted vec size will be 1.
For multi filter combine:
if there are two filters use `and` connect，use `RowRanges::intersection` to
get the final rowRange; two filters use `or` connect，use `RowRanges::union` to
get the final rowRange.
After this PR: i will working on `column_page_reader`, enable read specific
row ranges record.
# What changes are included in this PR?
<!---
There is no need to duplicate the description in the issue here but it is
sometimes worth providing a summary of the individual changes in this PR.
-->
# Are there any user-facing changes?
<!---
If there are user-facing changes then we may require documentation to be
updated before approving the PR.
-->
<!---
If there are any breaking changes to public APIs, please add the `breaking
change` label.
-->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
