Re: memory tuning

2021-01-27 Thread Matthias Pohl
Thanks for sharing the logs. The configuration looks fine. Have you analyzed the memory usage? On Tue, Jan 26, 2021 at 5:02 PM Marco Villalobos wrote: > Yes, I will do that. > > PRODUCTION > > 2021-01-26 04:03:50,804 INFO org.apache.flink.yarn.YarnTaskExecutorRunner > [] - > ---

Re: memory tuning

2021-01-26 Thread Marco Villalobos
Yes, I will do that. PRODUCTION 2021-01-26 04:03:50,804 INFO org.apache.flink.yarn.YarnTaskExecutorRunner [] - 2021-01-26 04:03:50,807 INFO org.apache.flink.yarn.YarnTaskExecutorRunner

Re: memory tuning

2021-01-26 Thread Matthias Pohl
Hi Marco, Could you share the preconfiguration logs? They are printed in the beginning of the taskmanagers' logs and contain a summary of the used memory configuration? Best, Matthias On Tue, Jan 26, 2021 at 6:35 AM Marco Villalobos wrote: > > I have a flink job that collects and aggregates tim

memory tuning

2021-01-25 Thread Marco Villalobos
I have a flink job that collects and aggregates time-series data from many devices into one object (let's call that X) that was collected by a window. X contains time-series data, so it contains many String, Instant, a HashMap, and another type (Let's call Y) objects. When I collect 4 X instances