[ https://issues.apache.org/jira/browse/SPARK-9763?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15136480#comment-15136480 ]
Junyang Shen commented on SPARK-9763: ------------------------------------- May I ask why the StructType is always defined as "asNullable" in the following three places? Line 109: StructType(schema.filterNot(f => partitionColumns.contains(f.name))).asNullable Line 149: StructType(partitionColumns.map { col => schema.find(_.name == col).getOrElse { throw new RuntimeException(s"Partition column $col not found in schema $schema") } }).asNullable Line 186: Some(dataSchema.asNullable) The "asNullable()" function would override the "StructField.nullable". Is it because the inside SQL data has to be nullable? Thanks! > Minimize exposure of internal SQL classes > ----------------------------------------- > > Key: SPARK-9763 > URL: https://issues.apache.org/jira/browse/SPARK-9763 > Project: Spark > Issue Type: Sub-task > Components: SQL > Reporter: Reynold Xin > Assignee: Reynold Xin > Fix For: 1.5.0 > > -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org