Nisanth Simon created AMBARI-21124:
--------------------------------------
Summary: Used Average Memory in Ambari dashborad is different from
Used Average got from free command in RHEL 7
Key: AMBARI-21124
URL: https://issues.apache.org/jira/browse/AMBARI-21124
Project: Ambari
Issue Type: Bug
Components: ambari-metrics
Affects Versions: 2.2.0
Environment: RHEL 7
Reporter: Nisanth Simon
Priority: Minor
There is a change in formula for calculating memory_used in "free command" for
RHEL 6 & RHEL 7.
In RHEL 6, memory_used = ( total - free). You can use "man free" on RHEL 6, to
get this formula.
[root@host1 ~]# free
total used free shared
buffers cached
Mem: 24607596 1799392 22808204 6368
95624 608592
-/+ buffers/cache: 1095176 23512420
Swap: 4194300 0 4194300
[root@host1 ~]#
In the above output, [used = total -free = 24607596 - 22808204 = 1799392]
>From RHEL 7 onwards, there is an extra attribute (MemAvailable) in
>/proc/meminfo. In RHEL 7, the memory_used = (total - free - buffers - cache).
>Refer the link for more details - https://access.redhat.com/solutions/406773
>for formula.
[root@host2 ~]# free
total used free shared
buff/cache available
Mem: 24524432 1002640 22724020 28232 797772 23247360
Swap: 4194300 0 4194300
[root@host2 ~]#
In the above output, [used = total - free - buffers - cache = 24524432 -
22724020 - 797772 = 22724020]
The Ambari Code calculate the memory used using the formula (mem_used =
psutil.virtual_memory().used) . Refer the code,
/usr/lib/python2.6/site-packages/resource_monitoring/core/host_info.py
def get_mem_info(self):
"""
Return memory statistics at current time
"""
mem_stats = psutil.virtual_memory()
swap_stats = psutil.swap_memory()
disk_usage = self.get_combined_disk_usage()
mem_total = self.__host_static_info.get('mem_total')
bytes2kilobytes = lambda x: x / 1024
return {
'mem_total': bytes2kilobytes(mem_total) if mem_total else 0,
'mem_used': bytes2kilobytes(mem_stats.used) if hasattr(mem_stats, 'used')
else 0,
'mem_free': bytes2kilobytes(mem_stats.free) if hasattr(mem_stats, 'free')
else 0,
'mem_shared': bytes2kilobytes(mem_stats.shared) if hasattr(mem_stats,
'shared') else 0,
'mem_buffered': bytes2kilobytes(mem_stats.buffers) if hasattr(mem_stats,
'buffers') else 0,
'mem_cached': bytes2kilobytes(mem_stats.cached) if hasattr(mem_stats,
'cached') else 0,
'swap_free': bytes2kilobytes(swap_stats.free) if hasattr(swap_stats,
'free') else 0,
'disk_free' : disk_usage.get("disk_free"),
# todo: cannot send string
#'part_max_used' : disk_usage.get("max_part_used")[0],
'disk_total' : disk_usage.get("disk_total")
}
pass
The Ambari Code psutil.virtual_memory() calculates the memory_used using the
formula (total -free). In the code,
/usr/lib/python2.6/site-packages/resource_monitoring/psutil/psutil/_pslinux.py
def virtual_memory():
total, free, buffers, shared, _, _ = cext.linux_sysinfo()
cached = active = inactive = None
f = open('/proc/meminfo', 'rb')
CACHED, ACTIVE, INACTIVE = b("Cached:"), b("Active:"), b("Inactive:")
try:
for line in f:
if line.startswith(CACHED):
cached = int(line.split()[1]) * 1024
elif line.startswith(ACTIVE):
active = int(line.split()[1]) * 1024
elif line.startswith(INACTIVE):
inactive = int(line.split()[1]) * 1024
if (cached is not None
and active is not None
and inactive is not None):
break
else:
# we might get here when dealing with exotic Linux flavors, see:
# http://code.google.com/p/psutil/issues/detail?id=313
msg = "'cached', 'active' and 'inactive' memory stats couldn't " \
"be determined and were set to 0"
warnings.warn(msg, RuntimeWarning)
cached = active = inactive = 0
finally:
f.close()
avail = free + buffers + cached
used = total - free
percent = usage_percent((total - avail), total, _round=1)
return svmem(total, avail, percent, used, free,
active, inactive, buffers, cached)
>From the above, we can conclude that the memory_used_average displayed in
>Ambari dashboard will be same with free command average for the cluster for
>RHEL 6.
In RHEL 7, the memory_used_average displayed in Ambari dashboard will be
different from free command average for the cluster.
Ambari adheres to the formula used by free command of RHEL 6 not of RHEL 7.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
