Hi Barisa,
by looking at the 1.8 documentation [1] it was possible to configure the
off heap memory as well. Also other memory options were already present.
So I don't think that you need an upgrade to 1.11 immediately. Please
let us know if you could fix your problem, otherwise we can try to loop
in other people that should know better.
Regards,
Timo
[1]
https://ci.apache.org/projects/flink/flink-docs-release-1.8/ops/config.html#taskmanager-memory-off-heap
On 25.02.21 15:50, Bariša wrote:
Small update:
we believe that the off heap memory is used by the parquet writer (
used in sink to write to S3 )
On Wed, 24 Feb 2021 at 23:25, Bariša <barisa.obrado...@gmail.com
<mailto:barisa.obrado...@gmail.com>> wrote:
I'm running flink 1.8.2 in a container, and under heavy load,
container gets OOM from the kernel.
I'm guessing that that reason for the kernel OOM is large size of
the off-heap memory. Is there a way I can limit it in flink 1.8.2?
I can see that newer version of flink has a config param, checking
here is it possible to do something similar in flink 1.8.2, without
a flink upgrade?
Cheers,
Barisa
