[
https://issues.apache.org/jira/browse/SPARK-10816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16576503#comment-16576503
]
Mike Sukmanowsky commented on SPARK-10816:
------------------------------------------
Voted, but want to express that we're definitely interested in native support
for session windows/merging windows in structured streaming. We're heavy
PySpark users and mapGroupsWithState is Java/Scala only so being able to
support this via SQL or PySpark DSL would be a great win.
Apache Beam supports a syntax like so (see
https://www.oreilly.com/ideas/the-world-beyond-batch-streaming-102):
{code:java}
PCollection<KV<String, Integer>> scores = input
.apply(Window.into(Sessions.withGapDuration(Duration.standardMinutes(1)))
.triggering(
AtWatermark()
.withEarlyFirings(AtPeriod(Duration.standardMinutes(1)))
.withLateFirings(AtCount(1)))
.accumulatingAndRetractingFiredPanes())
.apply(Sum.integersPerKey());{code}
> EventTime based sessionization
> ------------------------------
>
> Key: SPARK-10816
> URL: https://issues.apache.org/jira/browse/SPARK-10816
> Project: Spark
> Issue Type: New Feature
> Components: Structured Streaming
> Reporter: Reynold Xin
> Priority: Major
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
