[
https://issues.apache.org/jira/browse/SPARK-36045?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gengliang Wang updated SPARK-36045:
-----------------------------------
Description:
Currently, the SQL function to_utc_timestamp is confusing: it just takes the
timestamp value in the local timezone and then pretends it’s in the provided
timezone and then returns the UTC value, but the result is still treated as
local timezone!
The same issue happens in from_utc_timestamp as well.
We even tried to deprecated in the OSS community:
https://github.com/apache/spark/commit/c5e83ab92c0cb514963209dc3e70ba0e24570082
We should make TO_UTC_TIMESTAMP and FROM_UTC_TIMESTAMP return TimestampNTZ,
which makes a lot of sense. converting the current local time to/from UTC local
time.
The functions should accept both Timestamp types:
1. given TimestampLTZ, convert it to TimestampNTZ and continue step #2
2. given TimestampNTZ, convert it as to/from UTC local time.
was:
Currently, the SQL function to_utc_timestamp is confusing: it just takes the
timestamp value in the local timezone and then pretends it’s in the provided
timezone and then returns the UTC value, but the result is still treated as
local timezone!
The same issue happens in from_utc_timestamp as well.
We even tried to deprecated in the OSS community:
https://github.com/apache/spark/commit/c5e83ab92c0cb514963209dc3e70ba0e24570082
We should make
> TO_UTC_TIMESTAMP and FROM_UTC_TIMESTAMP should return TimestampNTZ
> ------------------------------------------------------------------
>
> Key: SPARK-36045
> URL: https://issues.apache.org/jira/browse/SPARK-36045
> Project: Spark
> Issue Type: Sub-task
> Components: SQL
> Affects Versions: 3.2.0
> Reporter: Gengliang Wang
> Priority: Major
>
> Currently, the SQL function to_utc_timestamp is confusing: it just takes the
> timestamp value in the local timezone and then pretends it’s in the provided
> timezone and then returns the UTC value, but the result is still treated as
> local timezone!
> The same issue happens in from_utc_timestamp as well.
> We even tried to deprecated in the OSS community:
> https://github.com/apache/spark/commit/c5e83ab92c0cb514963209dc3e70ba0e24570082
> We should make TO_UTC_TIMESTAMP and FROM_UTC_TIMESTAMP return TimestampNTZ,
> which makes a lot of sense. converting the current local time to/from UTC
> local time.
> The functions should accept both Timestamp types:
> 1. given TimestampLTZ, convert it to TimestampNTZ and continue step #2
> 2. given TimestampNTZ, convert it as to/from UTC local time.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
