Repository: spark Updated Branches: refs/heads/master 9565c246e -> 2f3837885
[SPARK-11360][DOC] Loss of nullability when writing parquet files This fix is to add one line to explain the current behavior of Spark SQL when writing Parquet files. All columns are forced to be nullable for compatibility reasons. Author: gatorsmile <gatorsm...@gmail.com> Closes #9314 from gatorsmile/lossNull. Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/2f383788 Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/2f383788 Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/2f383788 Branch: refs/heads/master Commit: 2f38378856fb56bdd9be7ccedf56427e81701f4e Parents: 9565c24 Author: gatorsmile <gatorsm...@gmail.com> Authored: Mon Nov 9 16:06:48 2015 -0800 Committer: Michael Armbrust <mich...@databricks.com> Committed: Mon Nov 9 16:06:48 2015 -0800 ---------------------------------------------------------------------- docs/sql-programming-guide.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/spark/blob/2f383788/docs/sql-programming-guide.md ---------------------------------------------------------------------- diff --git a/docs/sql-programming-guide.md b/docs/sql-programming-guide.md index ccd2690..6e02d65 100644 --- a/docs/sql-programming-guide.md +++ b/docs/sql-programming-guide.md @@ -982,7 +982,8 @@ when a table is dropped. [Parquet](http://parquet.io) is a columnar format that is supported by many other data processing systems. Spark SQL provides support for both reading and writing Parquet files that automatically preserves the schema -of the original data. +of the original data. When writing Parquet files, all columns are automatically converted to be nullable for +compatibility reasons. ### Loading Data Programmatically --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
