[ https://issues.apache.org/jira/browse/SPARK-30766?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Wenchen Fan resolved SPARK-30766. --------------------------------- Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 27512 [https://github.com/apache/spark/pull/27512] > Wrong truncation of old timestamps to hours and days > ---------------------------------------------------- > > Key: SPARK-30766 > URL: https://issues.apache.org/jira/browse/SPARK-30766 > Project: Spark > Issue Type: Bug > Components: SQL > Affects Versions: 3.0.0 > Reporter: Maxim Gekk > Assignee: Maxim Gekk > Priority: Major > Fix For: 3.0.0 > > > The date_trunc() function incorrectly truncates old timestamps to HOUR and > DAY: > {code:scala} > scala> import org.apache.spark.sql.functions._ > import org.apache.spark.sql.functions._ > scala> :paste > // Entering paste mode (ctrl-D to finish) > Seq("0010-01-01 01:02:03.123456").toDF() > .select($"value".cast("timestamp").as("ts")) > .select(date_trunc("HOUR", $"ts").cast("string")) > .show(false) > // Exiting paste mode, now interpreting. > +------------------------------------+ > |CAST(date_trunc(HOUR, ts) AS STRING)| > +------------------------------------+ > |0010-01-01 01:30:17 | > +------------------------------------+ > {code} > the result must be *0010-01-01 01:00:00* > {code:scala} > scala> :paste > // Entering paste mode (ctrl-D to finish) > Seq("0010-01-01 01:02:03.123456").toDF() > .select($"value".cast("timestamp").as("ts")) > .select(date_trunc("DAY", $"ts").cast("string")) > .show(false) > // Exiting paste mode, now interpreting. > +-----------------------------------+ > |CAST(date_trunc(DAY, ts) AS STRING)| > +-----------------------------------+ > |0010-01-01 23:30:17 | > +-----------------------------------+ > {code} > the result must be *0010-01-01 00:00:00* -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org