Hi, Memory used by session windows should be released once window is triggered (allowedLateness can prolong window’s life). Unless your code introduces some memory leak (by not releasing references) everything should be garbage collected.
Keep in mind that session windows with time gap of 10 minute, doesn’t mean that you will have a constant/fixed memory usage. As the times goes by and you process more and more elements your memory usage can increase indefinitely as events with new keys arrive and if older windows are still being kept alive (by arrival of events with existing keys). Also check what you are doing in your reduce function. If you collect incoming events on some collection then your memory footprint also can grow over time. Piotrek > On 20 Oct 2017, at 12:55, Rahul Raj <rahulrajms...@gmail.com> wrote: > > Hi All, > > I have been facing "GC Overhead limit exceeded" exception for quite a time > with my flink program. In my flink program, I have created a session window > with time gap of 10 minutes and I am performing some custom reduce operation > using Window function. The current statebackend is File System. The program > works fine for 15-20 mins and then fails with "GC overhead limited exception" > . My understanding of flink is that, the task manager's memory will be > released as soon as operation under window function will be completed. But > then why I am getting this error? What should I do to avoid this? > > Rahul Raj
