[ https://issues.apache.org/jira/browse/SPARK-22826?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16295673#comment-16295673 ]
Aleksander Eskilson commented on SPARK-22826: --------------------------------------------- I created a PR for this issue, [#20010|https://github.com/apache/spark/pull/20010]. > [SQL] findWiderTypeForTwo Fails over StructField of Array > --------------------------------------------------------- > > Key: SPARK-22826 > URL: https://issues.apache.org/jira/browse/SPARK-22826 > Project: Spark > Issue Type: Bug > Components: SQL > Affects Versions: 2.3.0 > Reporter: Aleksander Eskilson > > The {{findWiderTypeForTwo}} codepath in Catalyst {{TypeCoercion}} fails when > applied to to {{StructType}} having the following fields: > {noformat} > StructType(StructField("a", ArrayType(StringType, containsNull=true)) > :: Nil), > StructType(StructField("a", ArrayType(StringType, containsNull=false)) > :: Nil) > {noformat} > When in {{findTightestCommonType}}, the function attempts to recursively find > the tightest common type of two arrays. These two arrays are not equal types > (since one would admit null elements and the other would not), but > {{findTightestCommonType}} has no match case for {{ArrayType}} (or > {{MapType}}), so the > [get|https://github.com/apache/spark/blob/master/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/TypeCoercion.scala#L108] > operation on the dataType of the {{StructField}} throws a > {{NoSuchElementException}}. -- This message was sent by Atlassian JIRA (v6.4.14#64029) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org