mbutrovich opened a new issue, #7694:
URL: https://github.com/apache/datafusion-comet/issues/7694

   ### What is the problem the feature request solves?
   
   Decimal `/` and `div` run 
[`spark_decimal_div_internal`](https://github.com/apache/datafusion-comet/blob/d187e6823f9052e7331e8a810b08a1202debb91c/native/spark-expr/src/math_funcs/div.rs#L72-L178).
 Integer `div` runs it too, because the serde casts integer operands to 
`Decimal(19, 0)` 
([`CometIntegralDivide`](https://github.com/apache/datafusion-comet/blob/d187e6823f9052e7331e8a810b08a1202debb91c/spark/src/main/scala/org/apache/comet/serde/arithmetic.scala#L324-L329)).
 When no rescaled operand passes 38 digits, the kernel runs its `i128` branch 
([L147-L173](https://github.com/apache/datafusion-comet/blob/d187e6823f9052e7331e8a810b08a1202debb91c/native/spark-expr/src/math_funcs/div.rs#L147-L173)),
 and integer `div` always does. Per row, that branch scales the dividend by one 
more power of ten than the result needs, divides by the divisor, and divides 
the quotient by 10. The last division drops the extra digit, rounding half away 
from zero for `/` and truncating for `div`.
   
   Neither aarch64 nor x86_64 has a 128-bit divide instruction. In an aarch64 
release build, the closure of the `i128` branch calls `__divti3` twice per row, 
once for the literal 10. The `BigInt` branch 
([L114-L145](https://github.com/apache/datafusion-comet/blob/d187e6823f9052e7331e8a810b08a1202debb91c/native/spark-expr/src/math_funcs/div.rs#L114-L145))
 has the same structure, with two `BigInt` divisions per row.
   
   ### Describe the potential solution
   
   Compute the quotient at the result's scale, without the extra digit, so that 
each row divides once.
   
   - For `div`, truncating twice is the same as truncating once: 
`trunc(trunc(10a / b) / 10) = trunc(a / b)`. The division by 10 can go without 
changing any result.
   - For `/`, compute the remainder `rem = a - q * b` from the truncated 
quotient `q`. Because `|q * b| <= |a|`, the product can't overflow. Round `q` 
away from zero when `|rem| >= |b| - |rem|`. That is the test `2 * |rem| >= |b|` 
without the overflow of `2 * |rem|`. It rounds the same rows as today's `(q10 
+/- 5) / 10`, because both round the exact quotient half away from zero. 
apache/datafusion-comet#7690 makes the same change in `div_round_half_up`.
   - The exponents lose the extra digit: `l_exp = (s2 + s3) - s1` and `r_exp = 
s1 - (s2 + s3)`, each saturating at zero. The precision check that picks the 
`BigInt` branch then uses the new exponents. When the dividend is scaled, its 
precision drops by one, so some inputs move from the `BigInt` branch to the 
`i128` branch. When the divisor is scaled, its precision rises by one, so some 
inputs move the other way. Both branches compute the same quotient, so this 
changes which branch runs, not the result.
   - For integer `div`, all three scales are 0, so neither operand is scaled. A 
further step could divide in `i64`, which is a hardware divide, after a 
per-batch check that both operands fit. `i64::MIN div -1` stays on the `i128` 
path. The check is needed because a `Decimal(19, 0)` that comes from a decimal 
column can hold values up to 10^19 - 1.
   
   Tests should compare the new kernel with today's on both branches. They 
should cover negative operands, quotients exactly halfway between two values at 
the result scale, scale combinations where either exponent is zero, and 
operands at the 38-digit bound on each side of the branch choice.
   
   
[`benches/decimal_div.rs`](https://github.com/apache/datafusion-comet/blob/d187e6823f9052e7331e8a810b08a1202debb91c/native/spark-expr/benches/decimal_div.rs#L38-L66)
 measures `/` and `div` on `Decimal(10, 4)` and `Decimal(10, 3)`. Its 
`decimal_integral_div` case passes `Decimal128(10, 4)` as the return type, 
while the serde sends scale 0 for `div`. Could it get a case with that return 
type, and one for integer `div` on `Decimal(19, 0)`?
   
   ### Additional context
   
   apache/arrow-rs#11421 adds precomputed divisors to arrow-buffer. They help 
this kernel only when the divisor is a scalar, such as `x div 7` or `price / 
100`, and the kernel expands a scalar to an array today 
([L87-L96](https://github.com/apache/datafusion-comet/blob/d187e6823f9052e7331e8a810b08a1202debb91c/native/spark-expr/src/math_funcs/div.rs#L87-L96)).
 `DivisorI64` from apache/arrow-rs#11467 is on arrow-rs `main`, and 
apache/arrow-rs#12232 adds `DivisorI128`. The changes above don't depend on 
them.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to