[ https://issues.apache.org/jira/browse/SPARK-32167?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Dongjoon Hyun updated SPARK-32167: ---------------------------------- Fix Version/s: 2.4.7 > nullability of GetArrayStructFields is incorrect > ------------------------------------------------ > > Key: SPARK-32167 > URL: https://issues.apache.org/jira/browse/SPARK-32167 > Project: Spark > Issue Type: Bug > Components: SQL > Affects Versions: 1.6.3, 2.0.2, 2.1.3, 2.2.3, 2.3.4, 2.4.6, 3.0.0 > Reporter: Wenchen Fan > Assignee: Wenchen Fan > Priority: Blocker > Labels: correctness > Fix For: 2.4.7, 3.0.1, 3.1.0 > > > The following should be `Array([WrappedArray(1, null)])` instead of > `Array([WrappedArray(1, 0)])` > {code:java} > import scala.collection.JavaConverters._ > import org.apache.spark.sql.Row > import org.apache.spark.sql.types.{ArrayType, StructType} > val innerStruct = new StructType().add("i", "int", nullable = true) > val schema = new StructType().add("arr", ArrayType(innerStruct, containsNull > = false)) > val df = spark.createDataFrame(List(Row(Seq(Row(1), Row(null)))).asJava, > schema) > df.select($"arr".getField("i")).collect > {code} > -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org