Dandandan opened a new issue, #9416: URL: https://github.com/apache/arrow-rs/issues/9416
**Is your feature request related to a problem or challenge? Please describe what you are trying to do.** We can create mask based row selections for parquet filters. However, before converting them, `RowSelection::from_filters` is called to convert them in `ReadPlanBuilder::with_predicate`, storing them as indices and using `and_then` to combine indices. In very effective filters, this is not a problem, but it can generate some overhead in non-effective filters. **Describe the solution you'd like** Optimize the `ReadPlanBuilder` to: * store the current selection as `RowSelectionCursor` (or as mask?) * leave the boolean masks as is for non-effective filters * use bitwise `and` to combine masks when possible (or efficiently combine `RowSelection` and mask) **Describe alternatives you've considered** Keep it as is, optimize `RowSelection` code (from_filters / and_then...) **Additional context** Might be necessary to remove all regressions: https://github.com/apache/datafusion/issues/20324 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
