>
> How is the memory measured?
I meant which flink or k8s metric is collected? I'm asking because
depending on which metric is used, the *container memory usage* can be
defined differently. E.g., whether mmap memory is included.
Also, could you share the effective memory configurations for the
taskmanagers? You should find something like the following at the
beginning of taskmanger logs.
INFO [] - Final TaskExecutor Memory configuration:
> INFO [] - Total Process Memory: 1.688gb (1811939328 bytes)
> INFO [] - Total Flink Memory: 1.250gb (1342177280 bytes)
> INFO [] - Total JVM Heap Memory: 512.000mb (536870902 bytes)
> INFO [] - Framework: 128.000mb (134217728 bytes)
> INFO [] - Task: 384.000mb (402653174 bytes)
> INFO [] - Total Off-heap Memory: 768.000mb (805306378 bytes)
> INFO [] - Managed: 512.000mb (536870920 bytes)
> INFO [] - Total JVM Direct Memory: 256.000mb (268435458 bytes)
> INFO [] - Framework: 128.000mb (134217728 bytes)
> INFO [] - Task: 0 bytes
> INFO [] - Network: 128.000mb (134217730 bytes)
> INFO [] - JVM Metaspace: 256.000mb (268435456 bytes)
> INFO [] - JVM Overhead: 192.000mb (201326592 bytes)
Thank you~
Xintong Song
On Tue, Feb 2, 2021 at 8:59 PM Randal Pitt <randal.p...@foresite.com> wrote:
> Hi Xintong Song,
>
> Correct, we are using standalone k8s. Task managers are deployed as a
> statefulset so have consistent pod names. We tried using native k8s (in
> fact
> I'd prefer to) but got persistent
> "io.fabric8.kubernetes.client.KubernetesClientException: too old resource
> version: 242214695 (242413759)" errors which resulted in jobs being
> restarted every 30-60 minutes.
>
> We are using Prometheus Node Exporter to capture memory usage. The graph
> shows the metric:
>
>
> sum(container_memory_usage_bytes{container_name="taskmanager",pod_name=~"$flink_task_manager"})
> by (pod_name)
>
> I've attached the original
> <
> http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/file/t2869/Screenshot_2021-02-02_at_11.png>
>
> so Nabble doesn't shrink it.
>
> Best regards,
>
> Randal.
>
>
>
>
>
> --
> Sent from:
> http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/
>
