haohuaijin opened a new pull request, #11390: URL: https://github.com/apache/arrow-rs/pull/11390
# Which issue does this PR close? Closes #10423. # Rationale for this change Mixed mask/selector `intersection` and `union` currently expand the bitmap into RLE before merging. An alternating 3M-row bitmap can produce 3M selectors, turning roughly 375 KB into 48 MB of temporary storage. # What changes are included in this PR? Convert the selector operand to a bitmap and reuse the existing bitwise operations. Nonempty mixed inputs return a mask-backed result without materializing the mask's RLE representation. Empty operands preserve the other operand's representation, and unequal lengths retain the existing tail behavior. Remove the unused `borrowed_selectors` helper. Benchmark coverage is submitted separately in #11389 and should land first. # Are these changes tested? All 86 selection tests pass, including mixed-input equivalence, offsets, unequal lengths, empty operands, cached inputs, and result representation. Formatting and Clippy checks pass. Representative local Criterion results on Apple M2 Max, in µs (main → this change), using the equal-length, uncached inputs from #11389: | Case | Intersection | Union | | --- | ---: | ---: | | `alternating_1m` | 5870.78 → 6.63 | 5892.17 → 6.65 | | `alternating_3m` | 19495.49 → 16.83 | 19666.48 → 16.84 | | `compact_1m` | 13.70 → 7.11 | 13.62 → 7.21 | | `fragmented_selectors` | 362.98 → 110.84 | 338.59 → 111.61 | | `chain_mixed_3` | 5756.80 → 18.08 | 5038.74 → 18.11 | Chain timings include all three operations. These measurements exclude input construction and include result allocation and destruction. # Are there any significant user-facing changes? No API signature changes. Mixed nonempty results use bitmap storage even when the input mask already has cached selectors. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
