Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/23042#discussion_r233816966
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/TypeCoercion.scala
---
@@ -138,6 +138,11 @@ object TypeCoercion {
case (DateType, TimestampType)
=> if (conf.compareDateTimestampInTimestamp) Some(TimestampType)
else Some(StringType)
+ // to support a popular use case of tables using Decimal(X, 0) for
long IDs instead of strings
+ // see SPARK-26070 for more details
+ case (n: DecimalType, s: StringType) if n.scale == 0 =>
Some(DecimalType(n.precision, n.scale))
--- End diff --
what if the decimal is (1, 0) and the string is something like `1111.1111`?
The string can be anything: a very big integer, a fraction with many digits
after the dot, etc. I don't think there is a perfect solution, casting to
double is the best we can do here.
I'd suggest end users to manually do the cast which fits their data best.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
