Hi Dimitris,
Regarding your questions.
a) For standalone clusters, the recommended way is to use `.flink.size`
rather than `.process.size`. `.process.size` includes JVM metaspace and
overhead in addition to `.flink.size`, which usually do not really matter
for standalone clusters.
b) In case of direct OOMs, you should increase
`taskmanager.memory.task.off-heap.size`. There's no fraction for that.
c) Your understanding is correct. And you can also specify the absolute
network memory size by setting the min and max to the same value.
Here are my suggestions according to what you described.
1. Since both off-heap and network memory seems insufficient, I would
suggest to increase `taskmanager.memory.flink.size` to give your task
managers more memory in total.
2. If 1) does not work, I would suggest not to set the total memory
(means configure neither `.flink.size` nor `process.size`), but go for the
fine grained configuration where explicitly specify the individual memory
components. Flink will automatically add them up to derive the total memory.
1. In addition to `.task.off-heap.size` and `.network.[min|max]`, you
will also need to set `.task.heap.size` and `managed.size`.
2. If you don't know how many heap/managed memory to configure, you
can look for the configuration options in the beginning of the TM logs
(`-Dkey=value`). Those are the values derived from your current
configuration.
Thank you~
Xintong Song
On Sat, Jun 27, 2020 at 10:56 PM Dimitris Vogiatzidakis <
dimitrisvogiatzida...@gmail.com> wrote:
> Hello,
>
> I'm having a bit of trouble understanding the memory configuration on
> flink.
> I'm using flink10.0.0 to read some datasets of edges and extract features.
> I run this on a cluster consisting of 4 nodes , with 32cores and 252GB Ram
> each, and hopefully I could expand this as long as I can add extra nodes to
> the cluster.
>
> So regarding the configuration file (flink-conf.yaml).
> a) I can't understand when should I use process.size and when .flink.size.
>
> b) From the detailed memory model I understand that Direct memory is
> included in both of flink and process size, however if I don't specify
> off-heap.task.size I get
> " OutOfMemoryError: Direct buffer memory " . Also should I change
> off-heap.fraction as well?
>
> c)When I fix this, I get network buffers error, which if I understand
> correctly, flink.size * network fraction , should be between min and max.
>
> I can't find the 'perfect' configuration regarding my setup. What is the
> optimal way to use the system I have currently?
>
> Thank you for your time.
>
>
>
