[
https://issues.apache.org/jira/browse/SPARK-33404?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Takeshi Yamamuro updated SPARK-33404:
-------------------------------------
Affects Version/s: 3.1.0
> "date_trunc" expression returns incorrect results
> -------------------------------------------------
>
> Key: SPARK-33404
> URL: https://issues.apache.org/jira/browse/SPARK-33404
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Affects Versions: 3.0.0, 3.0.1, 3.1.0
> Reporter: Utkarsh Agarwal
> Priority: Major
> Labels: correctness
>
> `date_trunc` SQL expression returns incorrect results for {{minute}}
> formatting string.
> Context: The {{minute}} formatting string should truncate the timestamps such
> that the seconds is set to ZERO.
> Repro (run the following commands in spark-shell):
> {quote}
> spark.conf.set("spark.sql.session.timeZone", "America/Los_Angeles")
> spark.sql("SELECT date_trunc('minute', '1769-10-17 17:10:02')").show()
> {quote}
> Spark currently incorrectly returns
> {quote}
> 1769-10-17 17:10:02
> {quote}
> against the expected return value of
> {quote}
> 1769-10-17 17:10:00
> {quote}
> This happens as {{truncTimestamp}} in package
> {{org.apache.spark.sql.catalyst.util.DateTimeUtils}} incorrectly assumes that
> time zone offsets can never have the granularity of a second and thus does
> not account for time zone adjustment when truncating the timestamp to
> {{minute}}.
> This assumption is currently used when truncating the timestamps to
> {{microsecond, millisecond, second, or minute}}.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
