Re: [External] Re: Memory Issue When Running Beam On Flink

2020-08-31 Thread David Gogokhiya
> >>>>>>>     >>>          expected to > >>>>>>>     >>>          have as many > >>>>>>>     &g

Beam Memory Issue Followup

2020-08-31 Thread David Gogokhiya
Sorry for starting a new thread. I couldn't add a response to the existing thread at [1] for some reason. I have a very naive question related to finding a temporary fix to the memory issue raised in [1]. I know Jan suggested to use 2 successive fixed overlapping windows with offset as a temporary

Re: Memory Issue When Running Beam On Flink

2020-08-10 Thread David Gogokhiya
it takes for memory usage to reach the limit. Could you also say how > long it currently takes for memory to reach the limit? > > On Thu, Aug 6, 2020 at 4:23 PM David Gogokhiya wrote: > > > Hi, > > > > We recently started using Apache Beam version 2.20.0 running on

Re: Memory Issue When Running Beam On Flink

2020-08-10 Thread David Gogokhiya
r's memory usage is an expected behavior or not? Sincerely, David On Thu, Aug 6, 2020 at 3:19 PM David Gogokhiya wrote: > Hi, > > We recently started using Apache Beam version 2.20.0 running on Flink > version 1.9 deployed on kubernetes to process unbounded streams of data. > Howeve

Memory Issue When Running Beam On Flink

2020-08-06 Thread David Gogokhiya
Hi, We recently started using Apache Beam version 2.20.0 running on Flink version 1.9 deployed on kubernetes to process unbounded streams of data. However, we noticed that the memory consumed by stateful Beam is steadily increasing over time with no drops no matter what the current bandwidth is. W