Jungtaek Lim created SPARK-40892:
------------------------------------
Summary: Loosen the requirement of window_time rule - allow
multiple window_time calls
Key: SPARK-40892
URL: https://issues.apache.org/jira/browse/SPARK-40892
Project: Spark
Issue Type: Improvement
Components: Structured Streaming
Affects Versions: 3.4.0
Reporter: Jungtaek Lim
SPARK-40821 introduces a new SQL function "window_time" to extract the
representative time from window (which also carries over the event time
metadata as well if feasible).
SPARK-40821 followed the existing rule of time window / session window which
only allows a single function call in a same projection (strictly saying, it
considers the call of function as once if the function is called with same
parameters).
For existing rules, the restriction makes sense since allowing this would
produce cartesian product of rows (although Spark can handle it). But given
that window_time only produces one value, the restriction no longer makes sense.
It would be better to unlock the functionality. Note that this means the
resulting column of "window_time()" is no longer be "window_time".
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
