rluvaton commented on PR #22945: URL: https://github.com/apache/datafusion/pull/22945#issuecomment-4846938409
It is ready for merge, but looking at the code I see that if we have 5 streams and batch size is 1000; the first stream is large, set batch size / 2 = 500, and spilt it, the next stream is large as well, set batch size / 2 = 250 and split it while we should not set batch size / 2 for streams that were not splitt yet. as we might get away with the old split, but this can be in a follow up PR. merging, thanks @EmilyMatt -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
