MarkSfik commented on a change in pull request #373:
URL: https://github.com/apache/flink-web/pull/373#discussion_r479940171
##########
File path: _posts/2020-09-01-flink-1.11-memory-management-improvements.md
##########
@@ -0,0 +1,66 @@
+---
+layout: post
+title: "Memory Management improvements for Flink’s JobManager in Apache Flink
1.11"
+date: 2020-09-01T15:30:00.000Z
+authors:
+- Andrey:
+ name: "Andrey Zagrebin"
+categories: news
+excerpt: In a previous blog post focused on the memory model of the
TaskManagers and how it was improved with the Apache Flink 1.10 release. This
blog post addresses the same topic but for the JobManager instead.
+---
+
+Apache Flink 1.11 comes with significant changes to the memory model of
Flink’s JobManager and configuration options for your Flink clusters. These
recently-introduced changes make Flink adaptable to all kinds of deployment
environments (e.g. Kubernetes, Yarn, Mesos), providing better control over its
memory consumption.
+
+The [previous blog post]({{ site.baseurl
}}/news/2020/04/21/memory-management-improvements-flink-1.10.html), focused on
the memory model of the TaskManagers and how it was improved with the Apache
Flink 1.10 release. This blog post addresses the same topic but for the
JobManager instead. Flink 1.11 unifies the memory model of Flink’s processes.
The newly introduced memory model of the JobManager follows a similar approach
to that of the TaskManagers; it is simpler and has fewer components and tuning
knobs. This post might then seem very similar to our previous story on Flink’s
memory, but aims at providing a complete overview of Flink’s JobManager memory
model as of Flink 1.11. Read on for a full list of updates and changes below!
+
+## Introduction to Flink’s process memory model
+
+Having a clear understanding of Apache Flink’s process memory model allows you
to manage resources for the various workloads more efficiently. The following
diagram illustrates the main memory components of a Flink process:
+
+<center>
+<img src="{{ site.baseurl
}}/img/blog/2020-09-01-flink-1.11-memory-management-improvements/total-process-memory-flink-1.11.png"
width="400px" alt="Backpressure sampling:high"/>
+<br/>
+<i><small>Flink: Total Process Memory</small></i>
+</center>
+<br/>
+
+The JobManager process is a JVM process. On a high level, its memory consists
of the JVM Heap and Off-Heap memory. These types of memory are consumed by
Flink directly or by JVM for its specific purposes (i.e. metaspace etc). There
are two major memory consumers within the JobManager process: the framework
itself consuming memory for internal data structures, network communication,
etc. and the user code which runs within the JobManager process, e.g. in
certain batch sources or in checkpoint completion callbacks.
Review comment:
```suggestion
The JobManager process is a JVM process. On a high level, its memory
consists of the JVM Heap and Off-Heap memory. These types of memory are
consumed by Flink directly or by JVM for its specific purposes (i.e.
metaspace). There are two major memory consumers within the JobManager process:
the framework itself consuming memory for internal data structures, network
communication, etc. and the user code which runs within the JobManager process,
e.g. in certain batch sources or during checkpoint completion callbacks.
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
