comphead commented on issue #7660: URL: https://github.com/apache/datafusion-comet/issues/7660#issuecomment-6963884082
I tried item 1 for grouped AVG, and it fails the bench gate, so I did not open it. The numbers are from `benches/decimal_groups.rs`, 8192-row batches. | Shape | Bound pass + unchecked loop | Unchecked loop alone (no valid bound) | |---|---|---| | (25,2), 4 groups (Q1) | +44% | +13% | | (25,2), 1024 groups | +3% | −32% | | (25,2), 64K groups | +12% | −6% | | (25,2), 1024 groups, 10% nulls | −5% | −36% | | (38,6), near limit | +11% | n/a | - **The bound pass costs about as much as it saves.** Computing `R` from the values takes about 0.33 ns per value. That is what the per-row check costs in bench builds, where arrow's call is inlined. - **The unchecked loop is slower with few groups.** When consecutive rows hit the same few slots, as in Q1, it loses about 13% even without the pass. The "No check" column in the description shows the same for AVG with 4 groups. So #7674 does item 2 first. Grouped SUM and AVG compare against precision bounds loaded once per accumulator, which removes arrow's out-of-line call from release builds. A fast path would need two things, and both need a prototype before a PR: - A bound that costs nothing per batch. The declared input precision would do it, but it trusts that inputs respect that precision, and Parquet values are not validated against it. - Separate accumulator lanes when there are few groups, so consecutive rows of the same group do not update the same slot. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
