[ https://issues.apache.org/jira/browse/SPARK-22632?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Hyukjin Kwon resolved SPARK-22632. ---------------------------------- Resolution: Incomplete > Fix the behavior of timestamp values for R's DataFrame to respect session > timezone > ---------------------------------------------------------------------------------- > > Key: SPARK-22632 > URL: https://issues.apache.org/jira/browse/SPARK-22632 > Project: Spark > Issue Type: Bug > Components: SparkR, SQL > Affects Versions: 2.3.0 > Reporter: Hyukjin Kwon > Priority: Major > Labels: bulk-closed > > Note: wording is borrowed from SPARK-22395. Symptom is similar and I think > that JIRA is well descriptive. > When converting R's DataFrame from/to Spark DataFrame using > {{createDataFrame}} or {{collect}}, timestamp values behave to respect R > system timezone instead of session timezone. > For example, let's say we use "America/Los_Angeles" as session timezone and > have a timestamp value "1970-01-01 00:00:01" in the timezone. Btw, I'm in > South Korea so R timezone would be "KST". > The timestamp value from current collect() will be the following: > {code} > > sparkR.session(master = "local[*]", sparkConfig = > > list(spark.sql.session.timeZone = "America/Los_Angeles")) > > collect(sql("SELECT cast(cast(28801 as timestamp) as string) as ts")) > ts > 1 1970-01-01 00:00:01 > > collect(sql("SELECT cast(28801 as timestamp) as ts")) > ts > 1 1970-01-01 17:00:01 > {code} > As you can see, the value becomes "1970-01-01 17:00:01" because it respects R > system timezone. -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org