sunchao commented on PR #7258: URL: https://github.com/apache/datafusion-comet/pull/7258#issuecomment-6455360136
@andygrove, addressed both questions in `b97daacb45`: 1. The mode remains default-off for both reasons: wide-row results do not establish a consistent additional gain, and the reader retains native buffers outside the execution pool. I added a concrete enablement bar to the description: broader whole-query benefit over the improved JVM serializer, neutral large/wide controls, and concurrent-task memory measurements. The new complete local-shuffle query does show a residual benefit (930–1,114 ms OFF versus 644–823 ms ON across both orders), but it uses a controlled explicit broadcast, AQE off, and the benchmark's unbounded on-heap execution mode. It is useful current-path evidence, not enough by itself to flip the default. 2. Added the JVM boundary case with distinct rows in frames `[3,3,8,2]` and target 16. It asserts output sizes `[6,8,2]` and every row in order, exercising the pending-group flush and the simultaneously ready passthrough before the final group. These local tests now live in `NativeBatchDecoderIteratorSuite`, registered in both CI matrices. The driver-side size cap remains independent. Local validation passed 17 focused Rust tests and 169 JVM tests, with six existing version-gated cancellations; strict compilation, formatting and Clippy also passed. Hosted CI is running on the updated head. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
