[
https://issues.apache.org/jira/browse/FLINK-5572?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15853453#comment-15853453
]
Syinchwun Leo commented on FLINK-5572:
--------------------------------------
I try to find a common way to abstract the States in WindowOperator, and it is
very hard to find a balance between memory and performance. The current
framework is a good method to express most States except ListState. ListState
is commonly used in UDF Windows. ListState keeps raw data, and other States may
keep intermediate data. It's hard to unite all States using a common way to
keep, fire and clean up window. If we can keep the current framework, and only
optimize each specific State?
> ListState in SlidingEventTimeWindow and SlidingProcessingTimeWindow
> optimization
> --------------------------------------------------------------------------------
>
> Key: FLINK-5572
> URL: https://issues.apache.org/jira/browse/FLINK-5572
> Project: Flink
> Issue Type: Improvement
> Components: DataStream API
> Affects Versions: 1.2.1
> Environment: CentOS 7.2
> Reporter: Syinchwun Leo
>
> When using ListState in SlidingEventTimeWindow and
> SlidingProcessingTimeWindow, an element may be assigned to multiple
> overlapped windows. It may lead to storage consuming. for example,
> window(SlidingEventTimeWindows.of(Time.seconds(10),
> Time.seconds(2))).apply(UDF window function), each element is assigned to 5
> windows, When the window size is very large, it is unacceptable(size/slide is
> very large).
> We plan to make a little optimization, and the doc is in
> https://docs.google.com/document/d/1HCt1Si3YNGFwsl2H5SO0f7WD69DdBBPVJA6abd3oFWo/edit?usp=sharing
>
> Comments?
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
