[
https://issues.apache.org/jira/browse/SPARK-59686?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated SPARK-59686:
-----------------------------------
Labels: pull-request-available (was: )
> round and bround fail for a large rounding scale instead of returning the
> rounded value
> ---------------------------------------------------------------------------------------
>
> Key: SPARK-59686
> URL: https://issues.apache.org/jira/browse/SPARK-59686
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Affects Versions: 3.5.8, 4.2.0, 4.3.0, 4.1.3, 4.0.4
> Reporter: sepuri sai krishna
> Priority: Major
> Labels: pull-request-available
>
> h3. Problem
> {{round}} and {{bround}} pass the user supplied scale straight to
> {{BigDecimal.setScale}}, which
> throws once the magnitude passes roughly 1e9. Neighbouring scales return the
> result fine, so a
> well defined answer becomes an error:
> {code:sql}
> SELECT round(1.5, -10000000); -- 0.0
> SELECT round(1.5, -1000000000); -- java.lang.ArithmeticException
> SELECT round(1.5, 100000); -- 1.5
> SELECT round(1.5, 2147483647); -- java.lang.ArithmeticException
> {code}
> The answers are not in doubt. A scale far to the left of the decimal point
> rounds any finite value
> to zero, and a scale past the digits a value carries leaves it unchanged,
> which is exactly what the
> smaller scales above return.
> h3. Two symptoms, one cause
> * On the ANSI integral path the exception is wrapped by
> {{MathUtils.withOverflow}} and surfaces as
> {{ARITHMETIC_OVERFLOW}}. That is spurious: nothing overflowed, and the
> result {{0}} is
> representable.
> * Everywhere else, so {{FLOAT}}, {{DOUBLE}}, {{DECIMAL}} with a negative
> scale, and the integral
> types in non-ANSI mode, the raw {{java.lang.ArithmeticException}} reaches
> the user. It is not a
> {{SparkThrowable}}, so it carries no error condition and no SQLSTATE.
> h3. Also affected
> {{RoundBase.dataType}} computes {{-_scale + 1}} for a negative scale. For
> {{Int.MinValue}} the
> negation overflows, so a {{DECIMAL}} input gets a narrower result precision
> than intended.
> h3. Suggested fix
> Clamp the scale once, where it is read. {{_scale}} is a single {{protected
> lazy val}} that every
> type branch uses and that codegen interpolates, so clamping there covers
> interpreted and generated
> code for all numeric types.
> Clamping cannot change a result. No finite value has more than 309 integral
> digits, so any scale at
> or below the negative bound rounds to zero, and the exact decimal expansion
> of a finite double needs
> at most 1074 fractional digits, so any scale at or above the positive bound
> leaves the value
> unchanged.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
