Re: Flink memory usage

[AndreaKinn]

Anyway, If I understood how system metrics works (the results seems to be
showed in browser) I can't use it because my cluster is accessible only with
terminal via ssh 



--
Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/

Re: Flink memory usage

[AndreaKinn]

I have used sysstat linux tool.

On the node the only one application running is Flink. The outcomes measured
with metric system could be different?




--
Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/

Re: Flink memory usage

[Kien Truong]

Hi,

How did you measure the memory usage ?

JVM processes tend to occupy the maximum memory allocated to them, 
regardless of whether those memory are actively in used or not. To 
correctly measure the memory usage, you should use Flink's metric system[1]


Regards,

Kien

[1] 
https://ci.apache.org/projects/flink/flink-docs-release-1.3/monitoring/metrics.html#system-metrics


On 11/4/2017 3:15 AM, AndreaKinn wrote:

Hi,
I would like to share some considerations about Flink memory consumption.
I have a cluster composed of three nodes: 1 used both as JM and TM and other
2 TM.

I ran two identical applications (in different moments) on it. The only
difference is that on the second one I doubled every operators, essentially
to check what changes in resource's usage.

Analysing the outcomes on cpu side effectively the efforts are doubled.
Doing the same with memory I had these results:



which to me seems completely counterintuitive since the results are
essentially equal.
I can imagine in the second case the memory was effectively almost full but
why Flink gets such a lot of memory even in the first case?
How it is explained this behaviour?



--
Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/