[ https://issues.apache.org/jira/browse/SPARK-19950?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Apache Spark reassigned SPARK-19950: ------------------------------------ Assignee: Apache Spark > nullable ignored when df.load() is executed for file-based data source > ---------------------------------------------------------------------- > > Key: SPARK-19950 > URL: https://issues.apache.org/jira/browse/SPARK-19950 > Project: Spark > Issue Type: Bug > Components: SQL > Affects Versions: 2.2.0 > Reporter: Kazuaki Ishizaki > Assignee: Apache Spark > > This problem is reported in [Databricks > forum|https://forums.databricks.com/questions/7123/nullable-seemingly-ignored-when-reading-parquet.html]. > When we execute the following code, a schema for "id" in {{dfRead}} has > {{nullable = true}}. It should be {{nullable = false}}. > {code:java} > val field = "id" > val df = spark.range(0, 5, 1, 1).toDF(field) > val fmt = "parquet" > val path = "/tmp/parquet" > val schema = StructType(Seq(StructField(field, LongType, false))) > df.write.format(fmt).mode("overwrite").save(path) > val dfRead = spark.read.format(fmt).schema(schema).load(path) > dfRead.printSchema > {code} -- This message was sent by Atlassian JIRA (v6.3.15#6346) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org