comphead commented on issue #7660:
URL: 
https://github.com/apache/datafusion-comet/issues/7660#issuecomment-6963884082

   I tried item 1 for grouped AVG, and it fails the bench gate, so I did not 
open it. The numbers are from `benches/decimal_groups.rs`, 8192-row batches.
   
   | Shape | Bound pass + unchecked loop | Unchecked loop alone (no valid 
bound) |
   |---|---|---|
   | (25,2), 4 groups (Q1) | +44% | +13% |
   | (25,2), 1024 groups | +3% | −32% |
   | (25,2), 64K groups | +12% | −6% |
   | (25,2), 1024 groups, 10% nulls | −5% | −36% |
   | (38,6), near limit | +11% | n/a |
   
   - **The bound pass costs about as much as it saves.** Computing `R` from the 
values takes about 0.33 ns per value. That is what the per-row check costs in 
bench builds, where arrow's call is inlined.
   - **The unchecked loop is slower with few groups.** When consecutive rows 
hit the same few slots, as in Q1, it loses about 13% even without the pass. The 
"No check" column in the description shows the same for AVG with 4 groups.
   
   So #7674 does item 2 first. Grouped SUM and AVG compare against precision 
bounds loaded once per accumulator, which removes arrow's out-of-line call from 
release builds.
   
   A fast path would need two things, and both need a prototype before a PR:
   - A bound that costs nothing per batch. The declared input precision would 
do it, but it trusts that inputs respect that precision, and Parquet values are 
not validated against it.
   - Separate accumulator lanes when there are few groups, so consecutive rows 
of the same group do not update the same slot.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to