[
https://issues.apache.org/jira/browse/SPARK-14057?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15275987#comment-15275987
]
Vijay Parmar commented on SPARK-14057:
--------------------------------------
I have few suggestions to make here after looking into the issue along with
referring google and other sources:-
1. We can make use of the built-in java.time.package which is available in Java
8 and higher versions
(http://hg.openjdk.java.net/jdk8/jdk8/jdk/file/tip/src/share/classes/java/util/Date.java).
In short, here a new instance would be created to adjust the Time-Zone.
In java.util.date package the class in most of the cases ignores the
Time-Zone.
We could try implementing this package.
2. This code snippet can be handy :-
ZoneId zoneLondon = ZoneId.of("London");
ZonedDateTime nowLondon = ZonedDateTime.now ( zoneLondon );
ZoneId zoneSingapore = ZoneId.of("Singapore");
ZonedDateTime nowSingapore = nowLondon.withZoneSameInstant( zoneSingapore );
ZonedDateTime nowUTC = nowLondon.withZoneSameInstant( ZoneOffset.UTC );
3. We need to look into the SQL side code also To have an understanding how the
Time is getting captured and stored once it is received from this end.
I will still keep on looking iinto the issue and will update you. Meanwhile, I
also wait for your comment(s) on my suggestions.
> sql time stamps do not respect time zones
> -----------------------------------------
>
> Key: SPARK-14057
> URL: https://issues.apache.org/jira/browse/SPARK-14057
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Affects Versions: 1.6.0
> Reporter: Andrew Davidson
> Priority: Minor
>
> we have time stamp data. The time stamp data is UTC how ever when we load the
> data into spark data frames, the system assume the time stamps are in the
> local time zone. This causes problems for our data scientists. Often they
> pull data from our data center into their local macs. The data centers run
> UTC. There computers are typically in PST or EST.
> It is possible to hack around this problem
> This cause a lot of errors in their analysis
> A complete description of this issue can be found in the following mail msg
> https://www.mail-archive.com/user@spark.apache.org/msg48121.html
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
