This is an automated email from the ASF dual-hosted git repository.
dongjoon pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push:
new e1d021214c61 [SPARK-45425][DOCS][FOLLOWUP] Add a migration guide for
TINYINT type mapping change
e1d021214c61 is described below
commit e1d021214c6130588e69dfa05e0391d89b463f9d
Author: Kent Yao <y...@apache.org>
AuthorDate: Thu Apr 25 08:19:40 2024 -0700
[SPARK-45425][DOCS][FOLLOWUP] Add a migration guide for TINYINT type
mapping change
### What changes were proposed in this pull request?
Followup of SPARK-45425, adding migration guide.
### Why are the changes needed?
migration guide
### Does this PR introduce _any_ user-facing change?
no
### How was this patch tested?
existing build
### Was this patch authored or co-authored using generative AI tooling?
no
Closes #46224 from yaooqinn/SPARK-45425.
Authored-by: Kent Yao <y...@apache.org>
Signed-off-by: Dongjoon Hyun <dh...@apple.com>
---
docs/sql-migration-guide.md | 1 +
1 file changed, 1 insertion(+)
diff --git a/docs/sql-migration-guide.md b/docs/sql-migration-guide.md
index 9b189eee6ad1..024423fb145a 100644
--- a/docs/sql-migration-guide.md
+++ b/docs/sql-migration-guide.md
@@ -47,6 +47,7 @@ license: |
- Since Spark 4.0, MySQL JDBC datasource will read BIT(n > 1) as BinaryType,
while in Spark 3.5 and previous, read as LongType. To restore the previous
behavior, set `spark.sql.legacy.mysql.bitArrayMapping.enabled` to `true`.
- Since Spark 4.0, MySQL JDBC datasource will write ShortType as SMALLINT,
while in Spark 3.5 and previous, write as INTEGER. To restore the previous
behavior, you can replace the column with IntegerType whenever before writing.
- Since Spark 4.0, Oracle JDBC datasource will write TimestampType as
TIMESTAMP WITH LOCAL TIME ZONE, while in Spark 3.5 and previous, write as
TIMESTAMP. To restore the previous behavior, set
`spark.sql.legacy.oracle.timestampMapping.enabled` to `true`.
+- Since Spark 4.0, MsSQL Server JDBC datasource will read TINYINT as
ShortType, while in Spark 3.5 and previous, read as IntegerType. To restore the
previous behavior, set `spark.sql.legacy.mssqlserver.numericMapping.enabled` to
`true`.
- Since Spark 4.0, The default value for
`spark.sql.legacy.ctePrecedencePolicy` has been changed from `EXCEPTION` to
`CORRECTED`. Instead of raising an error, inner CTE definitions take precedence
over outer definitions.
- Since Spark 4.0, The default value for `spark.sql.legacy.timeParserPolicy`
has been changed from `EXCEPTION` to `CORRECTED`. Instead of raising an
`INCONSISTENT_BEHAVIOR_CROSS_VERSION` error, `CANNOT_PARSE_TIMESTAMP` will be
raised if ANSI mode is enable. `NULL` will be returned if ANSI mode is
disabled. See [Datetime Patterns for Formatting and
Parsing](sql-ref-datetime-pattern.html).
- Since Spark 4.0, A bug falsely allowing `!` instead of `NOT` when `!` is not
a prefix operator has been fixed. Clauses such as `expr ! IN (...)`, `expr !
BETWEEN ...`, or `col ! NULL` now raise syntax errors. To restore the previous
behavior, set `spark.sql.legacy.bangEqualsNot` to `true`.
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org
For additional commands, e-mail: commits-h...@spark.apache.org
