Srimanth Gunturi created AMBARI-10979:
-----------------------------------------
Summary: MRv2 Config Screen allows me to allocate sort buffers
larger than heap.
Key: AMBARI-10979
URL: https://issues.apache.org/jira/browse/AMBARI-10979
Project: Ambari
Issue Type: Bug
Components: contrib
Affects Versions: 2.1.0
Reporter: Srimanth Gunturi
Assignee: Srimanth Gunturi
Fix For: 2.1.0
{{mapreduce.task.io.sort.mb}} needs to be smaller than the min Xmx for maps and
reduces (also applies to Tez).
Ambari allows me to specify an invalid state without any warnings.
The formulas should be as follows
{code}
'mapreduce.map.memory.mb' =
int(configurations["yarn-site"]["properties"]["yarn.scheduler.minimum-allocation-mb"])
'mapreduce.reduce.memory.mb' =
min(2*int(configurations["yarn-site"]["properties"]["yarn.scheduler.minimum-allocation-mb"]),
int(nodemanagerMinRam))
mapredMapXmx =
int(0.8*int(configurations["mapred-site"]["properties"]["mapreduce.map.memory.mb"]));
'mapreduce.map.java.opts' = "-Xmx" + str(mapredMapXmx) + "m"
'mapreduce.reduce.java.opts' = "-Xmx" +
str(int(0.8*int(configurations["mapred-site"]["properties"]["mapreduce.reduce.memory.mb"])))
+ "m"
'mapreduce.task.io.sort.mb' = str(min(int(0.7*mapredMapXmx), 2047))
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
