555aaditya opened a new pull request, #55297: URL: https://github.com/apache/arrow/pull/55297
### Rationale for this change `dataset.scanner(fragment_readahead=0).to_table()` never returns. GH-15264 (#29185) made the scan node read fragments with a concatenated generator when `fragment_readahead` is 0 or 1. That change only covered `require_sequenced_output`. The unsequenced path, which `ToTable` and `ScanBatchesUnordered` use, still calls `MakeMergedGenerator(..., fragment_readahead)`. With 0, the merged generator never subscribes to a fragment, so the scan waits forever. `ScannerBuilder::FragmentReadahead` accepts 0, so 0 is a valid value. ### What changes are included in this PR? `MakeScanNode` now uses the concatenated generator for `fragment_readahead <= 1` on both paths. For `fragment_readahead > 1` nothing changes. ### Are these changes tested? Yes. - C++: added `TestScanner.ScanWithoutFragmentReadahead`. It runs ordered and unordered scans with `fragment_readahead` 0 and 1. Without the fix it hangs on the first case. With the fix all 4 parameterizations pass. - Python: added `test_scanner_no_fragment_readahead` in `test_dataset.py`, which reproduces the snippet from the issue. - `ctest -R "arrow-dataset|arrow-acero"` passes locally (26 tests, macOS arm64, debug build). ### Are there any user-facing changes? Scans with `fragment_readahead=0` no longer hang. ### Was AI used for this PR? In accordance to the [AI generation guidelines](https://arrow.apache.org/docs/dev/developers/overview.html#ai-generated-code), please disclose below whether and how AI was used in this PR. **PR code and description written by:** - [x] Human - [x] AI **Reviewed before submission by:** - [x] Human - [ ] AI - [ ] Not reviewed -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
