[
https://issues.apache.org/jira/browse/AMBARI-10979?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Srimanth Gunturi resolved AMBARI-10979.
---------------------------------------
Resolution: Fixed
Tested on live cluster and stack-advisor tests pass.
> MRv2 Config Screen allows me to allocate sort buffers larger than heap.
> -----------------------------------------------------------------------
>
> Key: AMBARI-10979
> URL: https://issues.apache.org/jira/browse/AMBARI-10979
> Project: Ambari
> Issue Type: Bug
> Components: contrib
> Affects Versions: 2.1.0
> Reporter: Srimanth Gunturi
> Assignee: Srimanth Gunturi
> Fix For: 2.1.0
>
> Attachments: AMBARI-10979.patch
>
>
> {{mapreduce.task.io.sort.mb}} needs to be smaller than the min Xmx for maps
> and reduces (also applies to Tez).
> Ambari allows me to specify an invalid state without any warnings.
> The formulas should be as follows
> {code}
> 'mapreduce.map.memory.mb' =
> int(configurations["yarn-site"]["properties"]["yarn.scheduler.minimum-allocation-mb"])
> 'mapreduce.reduce.memory.mb' =
> min(2*int(configurations["yarn-site"]["properties"]["yarn.scheduler.minimum-allocation-mb"]),
> int(nodemanagerMinRam))
> mapredMapXmx =
> int(0.8*int(configurations["mapred-site"]["properties"]["mapreduce.map.memory.mb"]));
> 'mapreduce.map.java.opts' = "-Xmx" + str(mapredMapXmx) + "m"
> 'mapreduce.reduce.java.opts' = "-Xmx" +
> str(int(0.8*int(configurations["mapred-site"]["properties"]["mapreduce.reduce.memory.mb"])))
> + "m"
> 'mapreduce.task.io.sort.mb' = str(min(int(0.7*mapredMapXmx), 2047))
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
