sunchao opened a new issue, #7172: URL: https://github.com/apache/datafusion-comet/issues/7172
### Describe the bug `SparkBloomFilter::merge_batch` assumes its binary state array contains exactly one row and passes `BinaryArray::value_data()` to `merge_filter`. There are two problems: grouped aggregate spill replay can provide several partial states for one group in a batch, causing the one-row assertion to panic; and a sliced one-row array can retain a backing buffer containing other rows, so reading the entire buffer merges the wrong state. ### Steps to reproduce At the native accumulator level: 1. Serialize two compatible Bloom filters containing different values and pass both rows to `merge_batch`. The accumulator should merge both, but currently asserts that the array length is one. 2. Build a binary array containing states for values 99, 11, and 99, then pass `array.slice(1, 1)`. Only the state for 11 should be merged, but `value_data()` still exposes the original backing buffer. A grouped final aggregate with a bounded memory pool can also exercise the batched case during sorted-spill replay. The accompanying fix includes a regression that requires actual spilling and verifies the exact merged state for every group. ### Expected behavior Merge every non-null state using its individual row slice, accept empty batches, and retain the existing compatibility checks. ### Additional context The code remains present at main commit `785fdbcb4cbf9057dfcfae3bd9dd45a77bb505bb`. This is separate from the memory-accounting changes in #6370. Local runtime validation of the proposed fix is currently blocked during dependency resolution because the configured registry does not provide the locked `aws-config 1.13.0`. ### Willingness to contribute I can contribute a fix for this bug independently. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
