[
https://issues.apache.org/jira/browse/FLINK-8384?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16325566#comment-16325566
]
ASF GitHub Bot commented on FLINK-8384:
---------------------------------------
GitHub user dyanarose opened a pull request:
https://github.com/apache/flink/pull/5295
[FLINK-8384] [streaming] Session Window Assigner with Dynamic Gaps
## What is the purpose of the change
This PR adds the ability for the Session Window assigners to to have
dynamic inactivity gaps in addition to the existing static inactivity gaps.
**Behaviour of dynamic gaps within existing sessions:**
- scenario 1 - the new timeout is prior to the old timeout. The old timeout
(the furthest in the future) is respected.
- scenario 2 - the new timeout is after the old timeout. The new timeout is
respected.
- scenario 3 - a session is in flight, a new timeout is calculated, however
no new events arrive for that session after the new timeout is calculated. This
session will not have its timeout changed
## Brief change log
**What's New**
- SessionWindowTimeGapExtractor\<T\> - Generic Interface with one extract
method that returns the time gap
- DynamicEventTimeSessionWindows\<T\> - Generic event time session window
- DynamicProcessingTimeSessionWindows\<T\> - Generic processing time
session window
- TypedEventTimeTrigger\<T\> - Generic event time trigger
- TypedProcessingTimeTrigger\<T\> - Generic processing time trigger
- Tests for all the above
## Verifying this change
This change added tests and can be verified as follows:
- added tests for the typed triggers that duplicate the existing trigger
tests to prove parity
- added unit tests for the dynamic session window assigners that mimic the
existing static session window assigner tests to prove parity in the static case
- added tests to the WindowOperatorTest class to prove the behaviour of
changing inactivity gaps
## Does this pull request potentially affect one of the following parts:
- Dependencies (does it add or upgrade a dependency): (no)
- The public API, i.e., is any changed class annotated with
`@Public(Evolving)`: (no, though the two typed trigger classes are marked
`@Public(Evolving)`)
- The serializers: (no)
- The runtime per-record code paths (performance sensitive): (no)
- Anything that affects deployment or recovery: JobManager (and its
components), Checkpointing, Yarn/Mesos, ZooKeeper: (no)
- The S3 file system connector: (no)
## Documentation
- Does this pull request introduce a new feature? (yes)
- If yes, how is the feature documented? (docs && JavaDocs)
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/SaleCycle/flink dynamic-session-window-gaps
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/flink/pull/5295.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #5295
----
commit 399522a3e23a51ce1e860e5e09499ef98a7e340d
Author: Dyana Rose <dyana.rose@...>
Date: 2018-01-10T15:50:00Z
[FLINK-8384] [streaming] Dynamic Gap Session Window Assigner
----
> Session Window Assigner with Dynamic Gaps
> -----------------------------------------
>
> Key: FLINK-8384
> URL: https://issues.apache.org/jira/browse/FLINK-8384
> Project: Flink
> Issue Type: Improvement
> Components: Streaming
> Reporter: Dyana Rose
> Priority: Minor
>
> *Reason for Improvement*
> Currently both Session Window assigners only allow a static inactivity gap.
> Given the following scenario, this is too restrictive:
> * Given a stream of IoT events from many device types
> * Assume each device type could have a different inactivity gap
> * Assume each device type gap could change while sessions are in flight
> By allowing dynamic inactivity gaps, the correct gap can be determined in the
> [assignWindows
> function|https://github.com/apache/flink/blob/master/flink-streaming-java/src/main/java/org/apache/flink/streaming/api/windowing/assigners/EventTimeSessionWindows.java#L59]
> by passing the element currently under consideration, the timestamp, and the
> context to a user defined function. This eliminates the need to create
> unwieldy work arounds if you only have static gaps.
> Dynamic Session Window gaps should be available for both Event Time and
> Processing Time streams.
> (short preliminary discussion:
> https://lists.apache.org/thread.html/08a011c0039e826343e9be0b1bb4e0000cfc2e12976bc65f8a43ee88@%3Cdev.flink.apache.org%3E)
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
