calvinchengx commented on PR #24193: URL: https://github.com/apache/datafusion/pull/24193#issuecomment-5229094711
Tested this against the repro from #24044, with a control on the parent commit so the result shows the fix flipping the behaviour rather than just passing on its own. | | no repartition | RoundRobin(4), batch_size 8192 | RoundRobin(4), batch_size 1 | |---|---|---|---| | `eec8b947c` (parent) | 59 | **0** | 59 | | `f26704525` (this PR) | 59 | **59** | 59 | Rows delivered in a six second window from a source emitting one row per 100ms, so 59 is the expected full count. The middle column is the bug. The other two are unchanged across both commits: the first never had a repartition to coalesce, and the third already worked because `batch_size=1` completes every batch immediately. So the change is scoped to the case it targets. Method: a single clone of this repo with the repro added as `datafusion-examples/examples/repartition_repro.rs`, one `git checkout` between runs, rebuilt in place. I confirmed `is_unbounded` was absent from `repartition/mod.rs` on the parent rather than assuming the checkout took. This resolves the issue as I reported it. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
