> On 30 Apr 2018, at 22:37, Nate Coraor <n...@bx.psu.edu> wrote: > > Hi Shawn, > > I'm wondering if you're still seeing this. I've recently enabled task/cgroup > on 17.11.5 running on CentOS 7 and just discovered that jobs are escaping > their cgroups. For me this is resulting in a lot of jobs ending in > OUT_OF_MEMORY that shouldn't, because it appears slurmd thinks the oom-killer > has triggered when it hasn't. I'm not using GRES or devices, only:
I am not sure that you are making the correct conclusion here. There is a known cgroups issue, due to https://www.kernel.org/doc/Documentation/cgroup-v1/memory.txt Relevant part: The memory controller has a long history. A request for comments for the memory controller was posted by Balbir Singh [1]. At the time the RFC was posted there were several implementations for memory control. The goal of the RFC was to build consensus and agreement for the minimal features required for memory control. The first RSS controller was posted by Balbir Singh[2] in Feb 2007. Pavel Emelianov [3][4][5] has since posted three versions of the RSS controller. At OLS, at the resource management BoF, everyone suggested that we handle both page cache and RSS together. Another request was raised to allow user space handling of OOM. The current memory controller is at version 6; it combines both mapped (RSS) and unmapped Page Cache Control [11]. Are the jobs killed prematurely? If not, then you ran into the above. Kind regards. — Andy
signature.asc
Description: Message signed with OpenPGP
