GitHub user gatorsmile opened a pull request:
https://github.com/apache/spark/pull/19939
[SPARK-20557] [SQL] Only support TIMESTAMP WITH TIME ZONE for Oracle Dialect
## What changes were proposed in this pull request?
In the previous PRs, https://github.com/apache/spark/pull/17832 and
https://github.com/apache/spark/pull/17835 , we convert `TIMESTAMP WITH TIME
ZONE` and `TIME WITH TIME ZONE` to `TIMESTAMP` for all the JDBC sources.
However, this conversion could be risky since it does not respect our SQL
configuration `spark.sql.session.timeZone`.
In addition, each vendor might have different semantics for these two
types. For example, Postgres simply returns `TIMESTAMP` types for `TIMESTAMP
WITH TIME ZONE`. For such supports, we should do it case by case. This PR
reverts the general support of `TIMESTAMP WITH TIME ZONE` and `TIME WITH TIME
ZONE` for JDBC sources, except ORACLE Dialect.
When supporting the ORACLE's `TIMESTAMP WITH TIME ZONE`, we only support it
when the JVM default timezone is the same as the user-specified configuration
`spark.sql.session.timeZone`. Now, we still treat `TIMESTAMP WITH TIME ZONE` as
`TIMESTAMP` when fetching the values via the Oracle JDBC connector, whose
client converts the timestamp values with time zone to the timestamp values
using the local JVM default timezone (a test case is added to
`OracleIntegrationSuite.scala` in this PR for showing the behavior). Thus, to
avoid any future behavior change, we will not support it if JVM default
timezone is different from `spark.sql.session.timeZone`
No regression because the previous two PRs were just merged to be
unreleased master branch.
## How was this patch tested?
Added the test cases
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/gatorsmile/spark timezoneUpdate
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/19939.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #19939
----
commit 69c6e3ef97b6450da5250171d47cf6c7f0faa9ff
Author: gatorsmile <gatorsm...@gmail.com>
Date: 2017-12-10T07:06:37Z
fix
----
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
