aweltsch opened a new pull request, #7021:
URL: https://github.com/apache/arrow-rs/pull/7021
# Which issue does this PR close?
<!--
We generally require a GitHub issue to be filed for all bug fixes and
enhancements and this helps us generate change logs for our releases. You can
link an issue to this PR using the GitHub syntax. For example `Closes #123`
indicates that this PR will close issue #123.
-->
Closes #6877.
# Rationale for this change
As mentioned in the issue there are certain conversions for which we don't
need to check precision / overflow and can thus improve performance by using
the infallible unary kernel.
## Explanation of the optimisation:
### Case 1, increasing scale:
Increasing the scale will result in multiplication with powers of 10, thus
being equivalent to a "shift left" of the digits.
Every number will thus "gain" additional digits. The operation is safe when
the output type gives enough precision (i.e. digits) to contain the original
digits, as well as the digits gained.
This can be boiled down to the condition `input_prec + (output_scale -
input_scale) <= output_prec`
#### Example:
consider a conversion from `Decimal(5, 0)` to `Decimal(8, 3)` with the
number `12345 * 10^0 = 12345000 * 10^(-3)`
If we are starting with any number `xxxxx`, then an increase of scale by 3
will have the following effect on the representation: `[xxxxx] -> [xxxxx000]`.
So for this to work, every output type needs to have at least 8 digits of
precision.
### Case 2, decreasing scale:
Decreasing the scale will result in division with powers of 10, thus
"shifting right" of the digits __and adding a rounding term__. We usually need
less precision to represent the result. By shifting right we lose `input_scale
- output_scale` digits, but adding a rounding term can result in one additional
digit being required, therefore we can boil this down to
`input_prec - (input_scale - output_scale) < output_prec`
#### Example:
consider a conversion from `Decimal(5, 0)` to `Decimal(3, -3)` with the
number `99900 * 10^0 = 99.9 * 10^(3)` which is then rounded to `100 * 10^3 =
100000`
If we are starting with any number `xxxxx`, then and decrease the scale by 3
will have the following effect on the representation: `[xxxxx] -> [xx] (+ 1
possibly)`. In the example plus one adds an additional digit, so the conversion
to work, every output type needs to have at least 3 digits of precision.
A conversion to `Decimal(2, -3)` would not be possible.
### Performance impact
The only cases affected are between decimal128 types, for increasing scale
there is a considerable improvements that I measured of around 80%.
For decreasing the scale there was an improvement of around 25%.
```
cast decimal128 to decimal128 512 6.98
2.8±0.00µs ? ?/sec 1.00 402.4±0.35ns ? ?/sec
cast decimal128 to decimal128 512 lower precision 1.01
3.0±0.00µs ? ?/sec 1.00 2.9±0.01µs ? ?/sec
cast decimal128 to decimal128 512 with lower scale (infallible) 1.31
3.3±0.00µs ? ?/sec 1.00 2.5±0.00µs ? ?/sec
cast decimal128 to decimal128 512 with same scale 1.07
54.2±1.47ns ? ?/sec 1.00 50.5±1.41ns ? ?/sec
cast decimal128 to decimal256 512 1.00
6.1±0.01µs ? ?/sec 1.00 6.1±0.01µs ? ?/sec
cast decimal256 to decimal128 512 1.02
13.2±0.67µs ? ?/sec 1.00 12.9±0.12µs ? ?/sec
cast decimal256 to decimal256 512 1.01
6.1±0.02µs ? ?/sec 1.00 6.1±0.03µs ? ?/sec
cast decimal256 to decimal256 512 with same scale 1.05
54.1±1.98ns ? ?/sec 1.00 51.6±1.15ns ? ?/sec
```
# What changes are included in this PR?
I've added a new specialization for dealing with "safe" casts between the
same decimal type both when reducing and increasing scale
A new benchmark for reducing scale
# Are there any user-facing changes?
No, the behavior of the casts should stay the same.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
