Arun Mahadevan created SPARK-25555:
--------------------------------------
Summary: Generic constructs for windowing and support for custom
windows
Key: SPARK-25555
URL: https://issues.apache.org/jira/browse/SPARK-25555
Project: Spark
Issue Type: Improvement
Components: Structured Streaming
Affects Versions: 2.4.0
Reporter: Arun Mahadevan
Refactor windows logic with generic "assignWindows" and "mergeWindows"
constructs. The existing windows (Tumbling and sliding) and generic session
windows can be built on top of this. It could be extended to support different
types of custom windowing.
K,Values -> AssignWindows (produces [k, v, timestamp, window]) -> GroupByKey
(shuffle) -> MergeWindows (optional step) -> GroupWindows -> aggregate values.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
