Yes. In general, and for every keyed state, the amount of state you store is proportional to the number of active keys that you expect to have.
On Sep 15, 2017 5:33 PM, "Paolo Rendano (JIRA)" <j...@apache.org> wrote: > > [ https://issues.apache.org/jira/browse/FLINK-7606?page= > com.atlassian.jira.plugin.system.issuetabpanels:comment- > tabpanel&focusedCommentId=16168025#comment-16168025 ] > > Paolo Rendano commented on FLINK-7606: > -------------------------------------- > > Hi [~kkl0u], > from your comment I understand that the basic memory (so with minimal load > of messages) I have to setup for a single instance is directly related with > the number of keys I want to be able to manage. I correctly understood, do > I? > > > CEP operator leaks state > > ------------------------ > > > > Key: FLINK-7606 > > URL: https://issues.apache.org/jira/browse/FLINK-7606 > > Project: Flink > > Issue Type: Bug > > Components: CEP > > Affects Versions: 1.3.1 > > Reporter: Matteo Ferrario > > Attachments: heap-dump1.png, heap-dump2.png, heap-dump3.png > > > > > > The NestedMapsStateTable grows up continuously without free the heap > memory. > > We created a simple job that processes a stream of messages and uses CEP > to generate an outcome message when a specific pattern is identified. > > The messages coming from the stream are grouped by a key defined in a > specific field of the message. > > We've also added the "within" clause (set as 5 minutes), indicating that > two incoming messages match the pattern only if they come in a certain time > window. > > What we've seen is that for every key present in the message, an NFA > object is instantiated in the NestedMapsStateTable and it is never > deallocated. > > Also the "within" clause didn't help: we've seen that if we send > messages that don't match the pattern, the memory grows up (I suppose that > the state of NFA is updated) but it is not cleaned also after the 5 minutes > of time window defined in "within" clause. > > If you need, I can provide more details about the job we've implemented > and also the screenshots about the memory leak. > > > > -- > This message was sent by Atlassian JIRA > (v6.4.14#64029) >
