alamb opened a new issue, #11428: URL: https://github.com/apache/arrow-rs/issues/11428
### Is your feature request related to a problem or challenge? @adriangb added the `ScanPlan` API in the Parquet decoder in - https://github.com/apache/arrow-rs/pull/10555 This is a very neat API that exposes what steps will be taken by the internal decoder next so external crates can plan / schedule their I/O needs However, it effectively duplicates the logic to decide what to do next when reading a row group across * `Planner::plan_next_row_group` * `RowGroupReaderBuilder::try_transition` This means that it those two copies may drift over time and some of the most complicated logic in the decoder is replicated twice ### Describe the solution you'd like Remove the duplication. ### Describe alternatives you've considered In my mind, the ideal outcome is that RowGroupReaderBuilder holds ScanPlan field or something similar internally, and uses the same exact logic to calculate what to do next as is exposed in the public API ### Additional context This is NOT a good ticket for someone new to the codebase to work on (or their agent). The amount of time required to review will be substantial -- please only create a PR that is already carefully reviewed and reduces complexity -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
