[
https://issues.apache.org/jira/browse/SPARK-39316?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
XiDuo You updated SPARK-39316:
------------------------------
Description:
Fix the bug of `TypeCoercion`, for example:
{code:java}
SELECT CAST(1 AS DECIMAL(28, 2))
UNION ALL
SELECT CAST(1 AS DECIMAL(18, 2)) / CAST(1 AS DECIMAL(18, 2));
{code}
The union result data type is not correct according to the formula:
|| Operation || Result Precision || Result Scale ||
|e1 union e2 | max(s1, s2) + max(p1-s1, p2-s2) | max(s1, s2) |
{code:java}
-- before
-- query schema
decimal(28,2)
-- query output
1.00
1.00
-- after
-- query schema
decimal(38,20)
-- query output
1.00000000000000000000
1.00000000000000000000
{code}
was:
Merge {{PromotePrecision}} into {{{}dataType{}}}, for example, {{{}Add{}}}:
{code:java}
override def dataType: DataType = (left, right) match {
case (DecimalType.Expression(p1, s1), DecimalType.Expression(p2, s2)) =>
val resultScale = max(s1, s2)
if (allowPrecisionLoss) {
DecimalType.adjustPrecisionScale(max(p1 - s1, p2 - s2) + resultScale + 1,
resultScale)
} else {
DecimalType.bounded(max(p1 - s1, p2 - s2) + resultScale + 1, resultScale)
}
case _ => super.dataType
} {code}
Merge {{{}CheckOverflow{}}}, for example, {{Add}} eval:
{code:java}
dataType match {
case decimalType: DecimalType =>
val value = numeric.plus(input1, input2)
checkOverflow(value.asInstanceOf[Decimal], decimalType)
...
} {code}
> Merge PromotePrecision and CheckOverflow into decimal binary arithmetic
> -----------------------------------------------------------------------
>
> Key: SPARK-39316
> URL: https://issues.apache.org/jira/browse/SPARK-39316
> Project: Spark
> Issue Type: Sub-task
> Components: SQL
> Affects Versions: 3.4.0
> Reporter: XiDuo You
> Assignee: XiDuo You
> Priority: Major
> Fix For: 3.4.0
>
>
> Fix the bug of `TypeCoercion`, for example:
> {code:java}
> SELECT CAST(1 AS DECIMAL(28, 2))
> UNION ALL
> SELECT CAST(1 AS DECIMAL(18, 2)) / CAST(1 AS DECIMAL(18, 2));
> {code}
> The union result data type is not correct according to the formula:
> || Operation || Result Precision || Result Scale ||
> |e1 union e2 | max(s1, s2) + max(p1-s1, p2-s2) | max(s1, s2) |
> {code:java}
> -- before
> -- query schema
> decimal(28,2)
> -- query output
> 1.00
> 1.00
> -- after
> -- query schema
> decimal(38,20)
> -- query output
> 1.00000000000000000000
> 1.00000000000000000000
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
