Github user mgaido91 commented on a diff in the pull request:
https://github.com/apache/spark/pull/20023#discussion_r158225279
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/DecimalPrecision.scala
---
@@ -243,17 +248,43 @@ object DecimalPrecision extends TypeCoercionRule {
// Promote integers inside a binary expression with fixed-precision
decimals to decimals,
// and fixed-precision decimals in an expression with floats / doubles
to doubles
case b @ BinaryOperator(left, right) if left.dataType !=
right.dataType =>
- (left.dataType, right.dataType) match {
- case (t: IntegralType, DecimalType.Fixed(p, s)) =>
- b.makeCopy(Array(Cast(left, DecimalType.forType(t)), right))
- case (DecimalType.Fixed(p, s), t: IntegralType) =>
- b.makeCopy(Array(left, Cast(right, DecimalType.forType(t))))
- case (t, DecimalType.Fixed(p, s)) if isFloat(t) =>
- b.makeCopy(Array(left, Cast(right, DoubleType)))
- case (DecimalType.Fixed(p, s), t) if isFloat(t) =>
- b.makeCopy(Array(Cast(left, DoubleType), right))
- case _ =>
- b
- }
+ nondecimalLiteralAndDecimal(b).lift((left, right)).getOrElse(
+ nondecimalNonliteralAndDecimal(b).applyOrElse((left.dataType,
right.dataType),
+ (_: (DataType, DataType)) => b))
}
+
+ /**
+ * Type coercion for BinaryOperator in which one side is a non-decimal
literal numeric, and the
+ * other side is a decimal.
+ */
+ private def nondecimalLiteralAndDecimal(
--- End diff --
Yes, it is. If we don't introduce this, we have a failure in Hive
compatibility tests, because Hive use the exact precision and scale needed by
the literals, while we, before this change, were using conservative values for
each type. For instance, if we have a `select 123.12345*3`, before this change
`3` would have been interpreted as `Decimal(10, 0)`, which is the type for
integers. After the change, `3` would become `Decimal(1, 0)`, as Hive does.
This prevents from needing more precision that what is actually needed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
