Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/21074#discussion_r182301592
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/TypeCoercion.scala
---
@@ -176,10 +176,18 @@ object TypeCoercion {
}
private def findWiderCommonType(types: Seq[DataType]): Option[DataType]
= {
- types.foldLeft[Option[DataType]](Some(NullType))((r, c) => r match {
- case Some(d) => findWiderTypeForTwo(d, c)
- case None => None
- })
+ // findWiderTypeForTwo doesn't satisfy the associative law, i.e. (a op
b) op c may not equal
+ // to a op (b op c). This is only a problem for StringType. Excluding
StringType,
+ // findWiderTypeForTwo satisfies the associative law. For instance,
(TimestampType,
+ // IntegerType, StringType) should have StringType as the wider common
type.
+ val (stringTypes, nonStringTypes) = types.partition { t =>
+ t == StringType || t == ArrayType(StringType)
--- End diff --
we need something like
```
def hasStringType(dt: DataType): Boolean = dt match {
case StringType => true
case ArrayType(et, _) => hasStringType(et)
case _ => false // Add StructType if we support string promotion for
struct fields in the future.
}
```
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
