[
https://issues.apache.org/jira/browse/SPARK-36065?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sumeet updated SPARK-36065:
---------------------------
Affects Version/s: 3.2.0
> date_trunc returns incorrect output
> -----------------------------------
>
> Key: SPARK-36065
> URL: https://issues.apache.org/jira/browse/SPARK-36065
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Affects Versions: 3.2.0, 3.3.0
> Reporter: Sumeet
> Priority: Major
> Labels: date_trunc, sql, timestamp
>
> Hi,
> Running date_trunc on any hour of "1891-10-01" returns incorrect output for
> "Europe/Bratislava" timezone.
> Use the following steps in order to reproduce the issue:
> * Run spark-shell using:
> {code:java}
> TZ="Europe/Bratislava" ./bin/spark-shell --conf
> spark.driver.extraJavaOptions='-Duser.timezone=Europe/Bratislava' --conf
> spark.executor.extraJavaOptions='-Duser.timezone=Europe/Bratislava' --conf
> spark.sql.session.timeZone="Europe/Bratislava"{code}
> * Generate test data:
> {code:java}
> ((0 until 9).map(hour => s"1891-10-01 00:0$hour:00") ++ (10 until
> 24).map(hour => s"1891-10-01
> 00:$hour:00")).toDF("ts_string").createOrReplaceTempView("temp_ts")
> {code}
> * Run query:
> {code:java}
> sql("select ts_string, cast(ts_string as TIMESTAMP) as ts, date_trunc('day',
> ts_string) from temp_ts").show(false)
> {code}
> * Output:
> {code:java}
> +-------------------+-------------------+--------------------------+
> |ts_string |ts |date_trunc(day, ts_string)|
> +-------------------+-------------------+--------------------------+
> |1891-10-01 00:00:00|1891-10-01 00:02:16|1891-10-01 00:02:16 |
> |1891-10-01 00:01:00|1891-10-01 00:03:16|1891-10-01 00:02:16 |
> |1891-10-01 00:02:00|1891-10-01 00:04:16|1891-10-01 00:02:16 |
> |1891-10-01 00:03:00|1891-10-01 00:03:00|1891-10-01 00:02:16 |
> |1891-10-01 00:04:00|1891-10-01 00:04:00|1891-10-01 00:02:16 |
> |1891-10-01 00:05:00|1891-10-01 00:05:00|1891-10-01 00:02:16 |
> |1891-10-01 00:06:00|1891-10-01 00:06:00|1891-10-01 00:02:16 |
> |1891-10-01 00:07:00|1891-10-01 00:07:00|1891-10-01 00:02:16 |
> |1891-10-01 00:08:00|1891-10-01 00:08:00|1891-10-01 00:02:16 |
> |1891-10-01 00:10:00|1891-10-01 00:10:00|1891-10-01 00:02:16 |
> |1891-10-01 00:11:00|1891-10-01 00:11:00|1891-10-01 00:02:16 |
> |1891-10-01 00:12:00|1891-10-01 00:12:00|1891-10-01 00:02:16 |
> |1891-10-01 00:13:00|1891-10-01 00:13:00|1891-10-01 00:02:16 |
> |1891-10-01 00:14:00|1891-10-01 00:14:00|1891-10-01 00:02:16 |
> |1891-10-01 00:15:00|1891-10-01 00:15:00|1891-10-01 00:02:16 |
> |1891-10-01 00:16:00|1891-10-01 00:16:00|1891-10-01 00:02:16 |
> |1891-10-01 00:17:00|1891-10-01 00:17:00|1891-10-01 00:02:16 |
> |1891-10-01 00:18:00|1891-10-01 00:18:00|1891-10-01 00:02:16 |
> |1891-10-01 00:19:00|1891-10-01 00:19:00|1891-10-01 00:02:16 |
> |1891-10-01 00:20:00|1891-10-01 00:20:00|1891-10-01 00:02:16 |
> +-------------------+-------------------+--------------------------+
> only showing top 20 rows
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
