[ https://issues.apache.org/jira/browse/SPARK-38990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Apache Spark reassigned SPARK-38990: ------------------------------------ Assignee: (was: Apache Spark) > date_trunc and trunc both fail with format from column in inline table > ---------------------------------------------------------------------- > > Key: SPARK-38990 > URL: https://issues.apache.org/jira/browse/SPARK-38990 > Project: Spark > Issue Type: Bug > Components: SQL > Affects Versions: 3.1.3, 3.2.1, 3.3.0, 3.4.0 > Reporter: Bruce Robbins > Priority: Minor > > This fails: > {noformat} > create or replace temp view v1 as > select * from values ('week', timestamp'2012-01-01') as data(col1, col2); > select date_trunc(col1, col2) from v1; > {noformat} > It fails with a {{NullPointerException}}: > {noformat} > java.lang.NullPointerException: null > at > org.apache.spark.sql.catalyst.InternalRow$.$anonfun$getAccessor$8(InternalRow.scala:141) > ~[spark-catalyst_2.12-3.4.0-SNAPSHOT.jar:3.4.0-SNAPSHOT] > at > org.apache.spark.sql.catalyst.InternalRow$.$anonfun$getAccessor$8$adapted(InternalRow.scala:141) > ~[spark-catalyst_2.12-3.4.0-SNAPSHOT.jar:3.4.0-SNAPSHOT] > at > org.apache.spark.sql.catalyst.expressions.BoundReference.eval(BoundAttribute.scala:40) > ~[spark-catalyst_2.12-3.4.0-SNAPSHOT.jar:3.4.0-SNAPSHOT] > at > org.apache.spark.sql.catalyst.expressions.TruncInstant.evalHelper(datetimeExpressions.scala:2117) > ~[spark-catalyst_2.12-3.4.0-SNAPSHOT.jar:3.4.0-SNAPSHOT] > at > org.apache.spark.sql.catalyst.expressions.TruncInstant.evalHelper$(datetimeExpressions.scala:2112) > ~[spark-catalyst_2.12-3.4.0-SNAPSHOT.jar:3.4.0-SNAPSHOT] > at > org.apache.spark.sql.catalyst.expressions.TruncTimestamp.evalHelper(datetimeExpressions.scala:2277) > ~[spark-catalyst_2.12-3.4.0-SNAPSHOT.jar:3.4.0-SNAPSHOT] > at > org.apache.spark.sql.catalyst.expressions.TruncTimestamp.eval(datetimeExpressions.scala:2295) > ~[spark-catalyst_2.12-3.4.0-SNAPSHOT.jar:3.4.0-SNAPSHOT] > at > org.apache.spark.sql.catalyst.expressions.Alias.eval(namedExpressions.scala:157) > ~[spark-catalyst_2.12-3.4.0-SNAPSHOT.jar:3.4.0-SNAPSHOT] > at > org.apache.spark.sql.catalyst.expressions.InterpretedMutableProjection.apply(InterpretedMutableProjection.scala:97) > ~[spark-catalyst_2.12-3.4.0-SNAPSHOT.jar:3.4.0-SNAPSHOT] > at > org.apache.spark.sql.catalyst.optimizer.ConvertToLocalRelation$$anonfun$apply$47.$anonfun$applyOrElse$77(Optimizer.scala:1987) > ~[spark-catalyst_2.12-3 > {noformat} > However, if you cache {{v1}}, then it works: > {noformat} > spark-sql> cache table v1; > Time taken: 2.086 seconds > spark-sql> select date_trunc(col1, col2) from v1; > 2011-12-26 00:00:00 > Time taken: 0.172 seconds, Fetched 1 row(s) > spark-sql> > {noformat} > Both {{date_trunc}} and {{trunc}} exhibit this behavior. -- This message was sent by Atlassian Jira (v8.20.7#820007) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org