[
https://issues.apache.org/jira/browse/FLINK-7606?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16168099#comment-16168099
]
Kostas Kloudas commented on FLINK-7606:
---------------------------------------
Also, on another note, I read that you have fewer results than expected.
Could you check if some of these patterns are emitted as timed out patterns? In
essence, could you check if timed-out patterns + successful ones == 30000.
This could happen if, for example, a "Start" arrives before the "Idle" (due to
parallelism), and there is a watermark in between them whose timestamp is
greater than that of the "Idle" element. In this case, the "Idle" will be
dropped as late and the pattern will time out. This is due to the fact that as
stated in the documentation, the CEP library assumes correctness of the
watermark.
> CEP operator leaks state
> ------------------------
>
> Key: FLINK-7606
> URL: https://issues.apache.org/jira/browse/FLINK-7606
> Project: Flink
> Issue Type: Bug
> Components: CEP
> Affects Versions: 1.3.1
> Reporter: Matteo Ferrario
> Attachments: heap-dump1.png, heap-dump2.png, heap-dump3.png
>
>
> The NestedMapsStateTable grows up continuously without free the heap memory.
> We created a simple job that processes a stream of messages and uses CEP to
> generate an outcome message when a specific pattern is identified.
> The messages coming from the stream are grouped by a key defined in a
> specific field of the message.
> We've also added the "within" clause (set as 5 minutes), indicating that two
> incoming messages match the pattern only if they come in a certain time
> window.
> What we've seen is that for every key present in the message, an NFA object
> is instantiated in the NestedMapsStateTable and it is never deallocated.
> Also the "within" clause didn't help: we've seen that if we send messages
> that don't match the pattern, the memory grows up (I suppose that the state
> of NFA is updated) but it is not cleaned also after the 5 minutes of time
> window defined in "within" clause.
> If you need, I can provide more details about the job we've implemented and
> also the screenshots about the memory leak.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
