This is an automated email from the ASF dual-hosted git repository. srowen pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push: new 078f50b16f0 [SPARK-38826][SQL][DOCS] dropFieldIfAllNull option does not work for empty JSON struct 078f50b16f0 is described below commit 078f50b16f07db6df169254fac36f8c41201ca61 Author: morvenhuang <morven.hu...@gmail.com> AuthorDate: Sun Apr 17 08:51:20 2022 -0500 [SPARK-38826][SQL][DOCS] dropFieldIfAllNull option does not work for empty JSON struct ### What changes were proposed in this pull request? Make a minor correction for the documentation of json option `dropFieldIfAllNull` ### Why are the changes needed? The `dropFieldIfAllNull` option actually does not work for empty json struct due to [SPARK-8093](https://issues.apache.org/jira/browse/SPARK-8093), the empty struct will be dropped anyway. We should update the doc, otherwise it would be confusing. ### Does this PR introduce _any_ user-facing change? Yes ### How was this patch tested? I've built the documentation locally and tested my change. Closes #36111 from morvenhuang/SPARK-38826. Authored-by: morvenhuang <morven.hu...@gmail.com> Signed-off-by: Sean Owen <sro...@gmail.com> --- docs/sql-data-sources-json.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/sql-data-sources-json.md b/docs/sql-data-sources-json.md index b5f27aacf41..f75efd1108a 100644 --- a/docs/sql-data-sources-json.md +++ b/docs/sql-data-sources-json.md @@ -235,7 +235,7 @@ Data source options of JSON can be set via: <tr> <td><code>dropFieldIfAllNull</code></td> <td><code>false</code></td> - <td>Whether to ignore column of all null values or empty array/struct during schema inference.</td> + <td>Whether to ignore column of all null values or empty array during schema inference.</td> <td>read</td> </tr> <tr> --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org