colin fang created SPARK-27559: ---------------------------------- Summary: Nullable in a given schema is not respected when reading from parquet Key: SPARK-27559 URL: https://issues.apache.org/jira/browse/SPARK-27559 Project: Spark Issue Type: Improvement Components: Spark Core Affects Versions: 2.4.2 Reporter: colin fang
Even if I specify a schema when reading from parquet, nullable is not reset. {code:java} spark.range(10, numPartitions=1).write.mode('overwrite').parquet('tmp') df1 = spark.read.parquet('tmp') df1.printSchema() # root # |-- id: long (nullable = true) df2 = spark.read.schema(StructType([StructField('id', LongType(), False)])).parquet('tmp') df2.printSchema() # root # |-- x: long (nullable = true) {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org