[
https://issues.apache.org/jira/browse/SPARK-38990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hyukjin Kwon resolved SPARK-38990.
----------------------------------
Fix Version/s: 3.3.0
3.0.4
3.2.2
3.1.3
Resolution: Fixed
Issue resolved by pull request 36312
[https://github.com/apache/spark/pull/36312]
> date_trunc and trunc both fail with format from column in inline table
> ----------------------------------------------------------------------
>
> Key: SPARK-38990
> URL: https://issues.apache.org/jira/browse/SPARK-38990
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Affects Versions: 3.1.3, 3.2.1, 3.3.0, 3.4.0
> Reporter: Bruce Robbins
> Assignee: Bruce Robbins
> Priority: Minor
> Fix For: 3.3.0, 3.0.4, 3.2.2, 3.1.3
>
>
> This fails:
> {noformat}
> create or replace temp view v1 as
> select * from values ('week', timestamp'2012-01-01') as data(col1, col2);
> select date_trunc(col1, col2) from v1;
> {noformat}
> It fails with a {{NullPointerException}}:
> {noformat}
> java.lang.NullPointerException: null
> at
> org.apache.spark.sql.catalyst.InternalRow$.$anonfun$getAccessor$8(InternalRow.scala:141)
> ~[spark-catalyst_2.12-3.4.0-SNAPSHOT.jar:3.4.0-SNAPSHOT]
> at
> org.apache.spark.sql.catalyst.InternalRow$.$anonfun$getAccessor$8$adapted(InternalRow.scala:141)
> ~[spark-catalyst_2.12-3.4.0-SNAPSHOT.jar:3.4.0-SNAPSHOT]
> at
> org.apache.spark.sql.catalyst.expressions.BoundReference.eval(BoundAttribute.scala:40)
> ~[spark-catalyst_2.12-3.4.0-SNAPSHOT.jar:3.4.0-SNAPSHOT]
> at
> org.apache.spark.sql.catalyst.expressions.TruncInstant.evalHelper(datetimeExpressions.scala:2117)
> ~[spark-catalyst_2.12-3.4.0-SNAPSHOT.jar:3.4.0-SNAPSHOT]
> at
> org.apache.spark.sql.catalyst.expressions.TruncInstant.evalHelper$(datetimeExpressions.scala:2112)
> ~[spark-catalyst_2.12-3.4.0-SNAPSHOT.jar:3.4.0-SNAPSHOT]
> at
> org.apache.spark.sql.catalyst.expressions.TruncTimestamp.evalHelper(datetimeExpressions.scala:2277)
> ~[spark-catalyst_2.12-3.4.0-SNAPSHOT.jar:3.4.0-SNAPSHOT]
> at
> org.apache.spark.sql.catalyst.expressions.TruncTimestamp.eval(datetimeExpressions.scala:2295)
> ~[spark-catalyst_2.12-3.4.0-SNAPSHOT.jar:3.4.0-SNAPSHOT]
> at
> org.apache.spark.sql.catalyst.expressions.Alias.eval(namedExpressions.scala:157)
> ~[spark-catalyst_2.12-3.4.0-SNAPSHOT.jar:3.4.0-SNAPSHOT]
> at
> org.apache.spark.sql.catalyst.expressions.InterpretedMutableProjection.apply(InterpretedMutableProjection.scala:97)
> ~[spark-catalyst_2.12-3.4.0-SNAPSHOT.jar:3.4.0-SNAPSHOT]
> at
> org.apache.spark.sql.catalyst.optimizer.ConvertToLocalRelation$$anonfun$apply$47.$anonfun$applyOrElse$77(Optimizer.scala:1987)
> ~[spark-catalyst_2.12-3
> {noformat}
> However, if you cache {{v1}}, then it works:
> {noformat}
> spark-sql> cache table v1;
> Time taken: 2.086 seconds
> spark-sql> select date_trunc(col1, col2) from v1;
> 2011-12-26 00:00:00
> Time taken: 0.172 seconds, Fetched 1 row(s)
> spark-sql>
> {noformat}
> Both {{date_trunc}} and {{trunc}} exhibit this behavior.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
