[ https://issues.apache.org/jira/browse/SPARK-27559?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Hyukjin Kwon resolved SPARK-27559. ---------------------------------- Resolution: Duplicate > Nullable in a given schema is not respected when reading from parquet > --------------------------------------------------------------------- > > Key: SPARK-27559 > URL: https://issues.apache.org/jira/browse/SPARK-27559 > Project: Spark > Issue Type: Improvement > Components: SQL > Affects Versions: 2.4.2 > Reporter: colin fang > Priority: Minor > > Even if I specify a schema when reading from parquet, nullable is not reset. > {code:java} > spark.range(10, numPartitions=1).write.mode('overwrite').parquet('tmp') > df1 = spark.read.parquet('tmp') > df1.printSchema() > # root > # |-- id: long (nullable = true) > df2 = spark.read.schema(StructType([StructField('id', LongType(), > False)])).parquet('tmp') > df2.printSchema() > # root > # |-- x: long (nullable = true) > {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org