neilconway opened a new pull request, #10869: URL: https://github.com/apache/arrow-rs/pull/10869
# Which issue does this PR close? - Closes #10866. - Closes #10865. # Rationale for this change By default, decimal values are not validated against their type's declared precision. Such out-of-precision values must therefore be handled correctly through arrow-rs. `format_decimal_str` attempted to do so by truncating the digit string to the declared precision, but that is not correct: for example, the value `12345` in a `Decimal128(3, 1)` context was formatted as `12.3`, i.e., it silently produced incorrect values. There are two principled ways to handle such values: format them in their full precision or return an error. Returning an error is impractical: (1) the formatting code doesn't have an easy way to surface such errors (2) this would be inconsistent with how out-of-precision decimals are handled throughout the rest of the Arrow ecosystem. Hence, this PR arranges to format out-of-precision values without truncation; the declared precision no longer affects formatting. Along the way, fix a second bug: a `0` decimal value with a negative scale was zero-padded; for example, formatting with scale `-2` resulted in the string "000". That is not a legal number in JSON, and is also just a surprising and likely unexpected behavior. # What changes are included in this PR? * Fix decimal formatting bugs as described above * Add/update tests # Are these changes tested? Yes; new tests added. # Are there any user-facing changes? Yes, but only in corner-cases. Both scenarios are rare to begin with, and the implemented behavior is unlikely to be what a user would desire. `format_decimal_str` still takes a `precision` parameter that is now unused. I've kept that in the function signature to avoid breaking a public API. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
