[ https://issues.apache.org/jira/browse/YARN-168?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13632868#comment-13632868 ]
Krishna Kishore Bonagiri commented on YARN-168: ----------------------------------------------- How can I get this fix if I want it now? When would the next release be? I mean, the one having this fix! > No way to turn off virtual memory limits without turning off physical memory > limits > ----------------------------------------------------------------------------------- > > Key: YARN-168 > URL: https://issues.apache.org/jira/browse/YARN-168 > Project: Hadoop YARN > Issue Type: Bug > Components: nodemanager > Affects Versions: 3.0.0 > Reporter: Harsh J > > Asked and reported by a user (Krishna) on ML: > {quote} > This is possible to do, but you've hit a bug with the current YARN > implementation. Ideally you should be able to configure the vmem-pmem > ratio (or an equivalent config) to be -1, to indicate disabling of > virtual memory checks completely (and there's indeed checks for this), > but it seems like we are enforcing the ratio to be at least 1.0 (and > hence negatives are disallowed). > You can't workaround by setting the NM's offered resource.mb to -1 > either, as you'll lose out on controlling maximum allocations. > Please file a YARN bug on JIRA. The code at fault lies under > ContainersMonitorImpl#init(…). > On Thu, Oct 18, 2012 at 4:00 PM, Krishna Kishore Bonagiri > <write2kish...@gmail.com> wrote: > > Hi, > > > > Is there a way we can ask the YARN RM for not killing a container when it > > uses excess virtual memory than the maximum it can use as per the > > specification in the configuration file yarn-site.xml? We can't always > > estimate the amount of virtual memory needed for our application running on > > a container, but we don't want to get it killed in a case it exceeds the > > maximum limit. > > > > Please suggest as to how can we come across this issue. > > > > Thanks, > > Kishore > {quote} > Basically, we're doing: > {code} > // ///////// Virtual memory configuration ////// > float vmemRatio = conf.getFloat( > YarnConfiguration.NM_VMEM_PMEM_RATIO, > YarnConfiguration.DEFAULT_NM_VMEM_PMEM_RATIO); > Preconditions.checkArgument(vmemRatio > 0.99f, > YarnConfiguration.NM_VMEM_PMEM_RATIO + > " should be at least 1.0"); > this.maxVmemAllottedForContainers = > (long)(vmemRatio * maxPmemAllottedForContainers); > {code} > For virtual memory monitoring to be disabled, maxVmemAllottedForContainers > has to be -1. For that to be -1, given the above buggy computation, vmemRatio > must be -1 or maxPmemAllottedForContainers must be -1. > If vmemRatio were -1, we fail the precondition check and exit. > If maxPmemAllottedForContainers, we also end up disabling physical memory > monitoring. > Or perhaps that makes sense - to disable both physical and virtual memory > monitoring, but that way your NM becomes infinite in resource grants, I think. > We need a way to selectively disable kills done via virtual memory > monitoring, which is the base request here. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira