ump45nose opened a new pull request, #26935: URL: https://github.com/apache/datafusion/pull/26935
## Which issue does this PR close? - Closes #26690. ## Rationale for this change On a pre-sorted multi-file table, `SELECT * FROM (SELECT * FROM t OFFSET 5) ORDER BY a LIMIT 10` can return fewer than 10 rows (sometimes none). `replace_with_order_preserving_variants` turns the `CoalescePartitionsExec` under that `OFFSET` into a `SortPreservingMergeExec` and gives it the TopK `fetch` (10). The merge therefore stops after 10 rows, and the outer `GlobalLimitExec` then skips 5 of them, so only 5 come back. Sort pushdown already accounts for this with `input_fetch`: when a fetch moves below a `GlobalLimitExec`, it adds `skip` first. This path did not. ## What changes are included in this PR? - `plan_with_order_preserving_variants` now passes `input_fetch(plan, fetch)` to children instead of the bare parent fetch, so a merge under `OFFSET 5` with TopK 10 gets `fetch=15`. - Made `sort_pushdown::input_fetch` `pub(super)` so both sites share the same rule. - Added a unit test that checks the merge receives `fetch=15` for that case. ## What is the testing strategy for this PR? - New unit test: `test_plan_with_order_preserving_variants_accounts_for_limit_skip`. - Existing `replace_with_order_preserving_variants` and `input_fetch` tests still pass. ## Are there any user-facing changes? No API changes. Queries that previously under-returned after `OFFSET` + `ORDER BY ... LIMIT` on sorted multi-partition data should now return the expected number of rows. ## AI assistance Used an AI coding assistant to help locate the bug site and draft the change. I reviewed the logic against #26690 and the existing `input_fetch` helper, ran the tests above, and wrote this description myself. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
