harikishore created SPARK-49689:
-----------------------------------
Summary: EventTimeWatermark to support TimestampNTZType
Key: SPARK-49689
URL: https://issues.apache.org/jira/browse/SPARK-49689
Project: Spark
Issue Type: Bug
Components: Structured Streaming
Affects Versions: 3.4.3, 3.5.2, 3.5.1, 3.5.0, 3.4.1, 3.4.0
Reporter: harikishore
Fix For: 3.5.2
Spark added TimestampNTZType support since 3.4.0.
[https://spark.apache.org/docs/3.4.0/api/java/org/apache/spark/sql/types/TimestampNTZType.html]
However, EventTimeWatermark with TimestampNTZType fails analysis check. Both
FilterPushdownSuite.scala and AnalysisSuite.scala don't explicitly avoid
timestampNTZ. Moreover, from StatefulAggregationStrategy -->
EventTimeWatermarkExec, even if the evenTime is of type TimestampNTZType, it
should be handled as per datetimeExpressions.scala. It is also supported in
Window Expressions `WindowSpecDefinition -> isValidFrameType`.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
