Repository: spark Updated Branches: refs/heads/master 33e337c11 -> f1891ff1e
[SPARK-25760][DOCS][FOLLOWUP] Add note about AddJar return value change in migration guide ## What changes were proposed in this pull request? Add note about AddJar return value change in migration guide ## How was this patch tested? n/a Closes #22826 from srowen/SPARK-25760.2. Authored-by: Sean Owen <sean.o...@databricks.com> Signed-off-by: Sean Owen <sean.o...@databricks.com> Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/f1891ff1 Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/f1891ff1 Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/f1891ff1 Branch: refs/heads/master Commit: f1891ff1e3f03668ac21b352b009bfea5e3c2b7f Parents: 33e337c Author: Sean Owen <sean.o...@databricks.com> Authored: Fri Oct 26 09:48:17 2018 -0500 Committer: Sean Owen <sean.o...@databricks.com> Committed: Fri Oct 26 09:48:17 2018 -0500 ---------------------------------------------------------------------- docs/sql-migration-guide-upgrade.md | 2 ++ 1 file changed, 2 insertions(+) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/spark/blob/f1891ff1/docs/sql-migration-guide-upgrade.md ---------------------------------------------------------------------- diff --git a/docs/sql-migration-guide-upgrade.md b/docs/sql-migration-guide-upgrade.md index 38c03d3..c9685b8 100644 --- a/docs/sql-migration-guide-upgrade.md +++ b/docs/sql-migration-guide-upgrade.md @@ -15,6 +15,8 @@ displayTitle: Spark SQL Upgrading Guide - Since Spark 3.0, the `from_json` functions supports two modes - `PERMISSIVE` and `FAILFAST`. The modes can be set via the `mode` option. The default mode became `PERMISSIVE`. In previous versions, behavior of `from_json` did not conform to either `PERMISSIVE` nor `FAILFAST`, especially in processing of malformed JSON records. For example, the JSON string `{"a" 1}` with the schema `a INT` is converted to `null` by previous versions but Spark 3.0 converts it to `Row(null)`. + - The `ADD JAR` command previously returned a result set with the single value 0. It now returns an empty result set. + ## Upgrading From Spark SQL 2.3 to 2.4 - In Spark version 2.3 and earlier, the second parameter to array_contains function is implicitly promoted to the element type of first array type parameter. This type promotion can be lossy and may cause `array_contains` function to return wrong result. This problem has been addressed in 2.4 by employing a safer type promotion mechanism. This can cause some change in behavior and are illustrated in the table below. --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
