[ https://issues.apache.org/jira/browse/SPARK-32641?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17182731#comment-17182731 ]
Apache Spark commented on SPARK-32641: -------------------------------------- User 'fqaiser94' has created a pull request for this issue: https://github.com/apache/spark/pull/29522 > withField + getField on null structs returns incorrect results > -------------------------------------------------------------- > > Key: SPARK-32641 > URL: https://issues.apache.org/jira/browse/SPARK-32641 > Project: Spark > Issue Type: Bug > Components: SQL > Affects Versions: 3.1.0 > Reporter: fqaiser94 > Priority: Major > > > There is a bug in the way the optimizer rule in SimplifyExtractValueOps > (sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/ComplexTypes.scala) > is currently coded which yields incorrect results in scenarios like the > following: > {code:java} > sql("SELECT CAST(NULL AS struct<a:int,b:int>) struct_col") > .select($"struct_col".withField("d", lit(4)).getField("d").as("d")) > // currently returns this: > +---+ > |d | > +---+ > |4 | > +---+ > // when in fact it should return this: > +----+ > |d | > +----+ > |null| > +----+ > {code} > > -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org