geoffreyclaude opened a new pull request, #26933: URL: https://github.com/apache/datafusion/pull/26933
## Which issue does this PR close? Follow-up to the [merge-initialization discussion in #26242](https://github.com/apache/datafusion/pull/26242#issuecomment-6340207237). ## Rationale for this change Before producing output, a sorted merge opens its inputs and encodes their comparison keys. With many ready inputs, initialization can monopolize a Tokio worker and delay unrelated queries. ## What changes are included in this PR? Yield after approximately the configured batch size's worth of rows during row-key initialization. Preserve scan progress across yields and revisit pending inputs when they wake. Optimized single-column cursors keep their existing initialization path. Each individual batch conversion remains synchronous. The sort-task limit is unchanged. ## What is the testing strategy for this PR? Three deterministic tests cover the yield boundary, pending-input wakeups, and keeping optimized field cursors inline. Formatting, full-workspace Clippy and extended workspace tests passed on the rebased branch. ## Are there any user-facing changes? More responsive merge startup, with no new configuration or output-ordering changes. Prepared with AI assistance. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
