sunchao opened a new pull request, #7173: URL: https://github.com/apache/datafusion-comet/pull/7173
## Which issue does this PR close? Closes #7172. ## Rationale for this change Grouped Bloom aggregates can receive several partial states for one group during spill replay. `merge_batch` currently asserts that there is exactly one row, so replay can panic. It also reads the binary array's complete backing buffer, which can include rows excluded by an Arrow slice. For example, merging partial filters containing 11 and 22 should retain both values. Passing a one-row slice containing the filter for 11 should merge only that row, even if the original buffer also contains a filter for 99. ## What changes are included in this PR? Merge each non-null binary row using its own slice. Empty batches and null states are no-ops. The existing version, hash-count, seed, and bit-array compatibility checks still apply to every merged state. This change contains only the merge fix and regression tests. The accounting changes from #6370 are already on main. ## How are these changes tested? Four unit regressions cover multi-row merges, sliced offsets, empty/null batches, and incompatible states. Multi-row and slice coverage exercise both Bloom serialization versions. A native integration test runs grouped final aggregation with an 8 MiB pool and two passes over 1,024 groups. It explicitly selects DataFusion's legacy sorted-spill path, requires actual spilling, verifies every group's exact merged bytes, rejects duplicate/missing groups, and checks that reservations are released. The repository's existing Cargo nextest job discovers this test automatically. Local `rustfmt --check` for both changed files and `git diff --check` pass. The attempted focused command was: ```sh cargo test --manifest-path native/Cargo.toml --locked -p datafusion-comet-spark-expr --lib --test bloom_filter_spill bloom_filter ``` Local test execution is blocked before compilation: the configured dependency mirror does not provide `aws-config 1.13.0`, pinned by the current upstream workspace lockfile. The new tests have not been executed locally on this upstream revision; CI results will provide runtime validation. No JVM tests or performance benchmarks were run. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
