MarkSfik commented on a change in pull request #328:
URL: https://github.com/apache/flink-web/pull/328#discussion_r411337034
##########
File path: _posts/2020-04-17-memory-management-improvements-flink-1.10.md
##########
@@ -0,0 +1,87 @@
+---
+layout: post
+title: "Memory Management improvements with Apache Flink 1.10"
+date: 2020-04-17T12:00:00.000Z
+authors:
+- andrey:
+ name: "Andrey Zagrebin"
+categories: news
+excerpt: This post discusses the recent changes to the memory model of the
task managers and configuration options for your Flink applications in Flink
1.10.
+---
+
+Apache Flink 1.10 comes with significant changes to the memory model of the
task managers and configuration options for your Flink applications. These
recently-introduced changes make Flink more adaptable to all kinds of
deployment environments (e.g. Kubernetes, Yarn, Mesos), providing strict
control over its memory consumption. In this post, we describe Flink’s memory
model, as it stands in Flink 1.10, how to set up and manage memory consumption
of your Flink applications and the recent changes the community implemented in
the latest Apache Flink release.
+
+## Introduction to Flink’s memory model
+
+Having a clear understanding of Apache Flink’s memory model allows you to
manage resources for the various workloads more efficiently. The following
diagram illustrates the main memory components in Flink:
+
+<center>
+<img src="{{ site.baseurl
}}/img/blog/2020-04-17-memory-management-improvements-flink-1.10/total-process-memory.svg"
width="400px" alt="Figure 1: Rule definition"/>
+<br/>
+<i><small>Flink: Total Process Memory</small></i>
+</center>
+<br/>
+
+The task manager process is a JVM process. On a high level, its memory
consists of the *JVM Heap* and *Off-Heap* memory. These types of memory are
consumed by Flink directly or by JVM for its specific purposes (i.e. metaspace
etc). There are two major memory consumers within Flink: the user code of job
operator tasks and the framework itself consuming memory for internal data
structures, network buffers etc.
+
+**Please note that** the user code has direct access to all memory types: *JVM
Heap, Direct* and *Native memory*. Therefore, Flink cannot really control its
allocation and usage. There are however two types of Off-Heap memory which are
consumed by tasks and controlled explicitly by Flink:
+
+- *Managed Off-Heap Memory*
+- *Network Buffers*
+
+The latter is part of the *JVM Direct Memory*, allocated for user record data
exchange between operator tasks.
+
+## How to set up Flink memory
+
+With the latest release of Flink 1.10 and in order to provide better user
experience, the framework comes with both high-level and fine-grained tuning of
memory components. There are essentially three alternatives to setting up
memory in task managers.
+
+The first two — and simplest — alternatives are configuring one of the two
following options for total memory available for the task manager:
Review comment:
```suggestion
The first two — and simplest — alternatives are configuring one of the two
following options for total memory available for the Task Manager:
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
