Gyula Fora created FLINK-1618:
---------------------------------
Summary: Add parallel time discretisation for time-window
transformations
Key: FLINK-1618
URL: https://issues.apache.org/jira/browse/FLINK-1618
Project: Flink
Issue Type: Improvement
Components: Streaming
Reporter: Gyula Fora
Currently discretizers for all windowing policies including time are executed
with parallelism 1 when they define global windows. (for instance: sum of the
last 10 minutes)
While this is necessary for arbitrary policies like delta based or user-defined
policies. Some discretizers such as Time can be implemented in a distributed
fashion.
Distributed time discretisers (and other types) can be implemented in the
following way:
-The discretisers should create StreamWindow s with incrementally increasing
ID-s starting from the same value so that it is possible to merge them after
the transformation
- The partitioner for each discretizer should send the number of partitions
created to the merger (the merger should be aware of the number of partitioners
present to wait for all the information)
- Based on all the partitioning info the merger can merge the windows properly
afterwards
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
