[
https://issues.apache.org/jira/browse/SPARK-10816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17287585#comment-17287585
]
Yuanjian Li commented on SPARK-10816:
-------------------------------------
Great thanks for your heads up! [~viirya] [~kabhwan]
{quote}Now that there're two committers from different teams finding the
feature as useful, looks like we could try pushing this out again.
{quote}
Big +1. Really excited to revive this feature with you. I'll also take some
time to reload the old context soon.
{quote}Probably the code size is different because the design is actually quite
different
{quote}
That's right. From my roughly investigation, The main difference list below:
* State store format design: As Shixiong described in [this
comment|https://issues.apache.org/jira/browse/SPARK-10816?focusedCommentId=16645370&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16645370],
my approach is easy to implement but not scale well in the case of non-numeric
aggregate.
* The structure of the physical plan node: Jungtaek's approach leverages the
aggregation iterator. My approach reused the way of `WindowExec`.
About authorship, really appreciate your trust [~kabhwan]! I can help with
confirming with the co-authors. Comparing with other issues, I think this
should be the easiest one and can be discussed at the end. :)
> EventTime based sessionization (session window)
> -----------------------------------------------
>
> Key: SPARK-10816
> URL: https://issues.apache.org/jira/browse/SPARK-10816
> Project: Spark
> Issue Type: New Feature
> Components: Structured Streaming
> Reporter: Reynold Xin
> Priority: Major
> Attachments: SPARK-10816 Support session window natively.pdf, Session
> Window Support For Structure Streaming.pdf
>
>
> Currently structured streaming supports two kinds of windows: tumbling window
> and sliding window. Another useful window function is session window. Which
> is not supported by SS.
> Unlike time window (tumbling window and sliding window), session window
> doesn't have static window begin and end time. Session window creation
> depends on defined session gap which can be static or dynamic.
> For static session gap, the events which are falling in a certain period of
> time (gap) are considered as a session window. A session window closes when
> it does not receive events for the gap. For dynamic gap, the gap could be
> changed from event to event.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
