maropu commented on a change in pull request #25136: [SPARK-28322][SQL] Add
support to Decimal type for integral divide
URL: https://github.com/apache/spark/pull/25136#discussion_r312702648
##########
File path:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/DecimalPrecision.scala
##########
@@ -174,6 +174,20 @@ object DecimalPrecision extends TypeCoercionRule {
CheckOverflow(Pmod(promotePrecision(e1, widerType), promotePrecision(e2,
widerType)),
resultType, nullOnOverflow)
+ case expr @ IntegralDivide(
+ e1 @ DecimalType.Expression(p1, s1), e2 @ DecimalType.Expression(p2,
s2)) =>
+ val widerType = widerDecimalType(p1, s1, p2, s2)
+ val promotedExpr =
+ IntegralDivide(promotePrecision(e1, widerType), promotePrecision(e2,
widerType))
+ if (expr.dataType.isInstanceOf[DecimalType]) {
+ // This follows division rule
+ val intDig = p1 - s1 + s2
+ // No precision loss can happen as the result scale is 0, so only
overflow can happen
+ CheckOverflow(promotedExpr, DecimalType.bounded(intDig, 0),
nullOnOverflow)
Review comment:
For example `select div(-9223372036854775808, -1)` should cause overflow
with `spark.sql.legacy.integralDivide.returnBigint=true`? In case of
`spark.sql.legacy.integralDivide.returnBigint=false` and the numeric input,
IIUC we have no chance to hit overflow. Actually, I think we should adjust
scale/precision without overflow.
But, I understand your concern since `Divide` already uses `CheckOverflow`
for nuemric input. cc: @gatorsmile @cloud-fan @dongjoon-hyun
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
