[issue42411] respect cgroups limits when trying to allocate memory

Caleb Collins-Parks added the comment: @christian.heimes following up on this - we have been having frequent memory issues with Python 3.7 in Kubernetes. It could just be the code, but if it does turn out this is a bug then fixing it could be very beneficial. -- nosy: +caleb2

[issue42411] respect cgroups limits when trying to allocate memory

Carlos Alexandro Becker added the comment: Any updates? -- ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue42411] respect cgroups limits when trying to allocate memory

Christian Heimes added the comment: Even if we would decide to add a memory limit based on cgroups, there is no way to implement a limit in Python correctly. We rely on the platforms malloc() implementation to handle memory allocation for us. Python has an abstraction layer for memory

[issue42411] respect cgroups limits when trying to allocate memory

Carlos Alexandro Becker added the comment: Just did more tests here: **on my machine**: $ docker run --name test -m 1GB fedora:33 python3 -c 'import resource; m = int(open("/sys/fs/cgroup/memory/memory.limit_in_bytes").read()); resource.setrlimit(resource.RLIMIT_AS, (m, m));

[issue42411] respect cgroups limits when trying to allocate memory

Christian Heimes added the comment: I doubt it. My test hosts have between 16G and 64G of RAM + plenty of swap. What's your platform, distribution, Kernel version, Docker version, and libseccomp version? -- ___ Python tracker

[issue42411] respect cgroups limits when trying to allocate memory

Carlos Alexandro Becker added the comment: FWIW, here, both cases: ``` ❯ docker ps -a CONTAINER IDIMAGE COMMAND CREATED STATUSPORTS NAMES 30fc350a8dbdpython:rc-alpine"python -c 'x =

[issue42411] respect cgroups limits when trying to allocate memory

Carlos Alexandro Becker added the comment: Maybe you're trying to allocate more memory than the host has available? I found out that it gives MemoryError in those cases too (kind of easy to reproduce on docker for mac)... -- ___ Python tracker

[issue42411] respect cgroups limits when trying to allocate memory

Christian Heimes added the comment: I can neither reproduce the issue with podman and cgroupv2 nor with docker and cgroupsv1. In both cases I'm getting a MemoryError as expected: # podman run -m 1G --cpus 1 python:rc-alpine python -c 'x = bytearray(80 * 1024 * 1024 * 1000)' Traceback (most

[issue42411] respect cgroups limits when trying to allocate memory

Carlos Alexandro Becker added the comment: The problem is that, instead of getting a MemoryError, Python tries to "go out of bounds" and allocate more memory than the cgroup allows, causing Linux to kill the process. A workaround is to set RLIMIT_AS to the contents of

[issue42411] respect cgroups limits when trying to allocate memory

Andrew Svetlov added the comment: Could you explain the proposal? How "+X:UseContainerSupport" behaves for Java? Sorry, I did not use Java for ages and don't follow the modern Java best practices. >From my understanding, without the Docker the allocation of `bytearray(80 * >1024 * 1024 *

[issue42411] respect cgroups limits when trying to allocate memory

New submission from Carlos Alexandro Becker : A common use case is running python inside containers, for instance, for training models and things like that. The python process sees the host memory/cpu, and ignores its limits, which often leads to OOMKills, for instance: docker run -m 1G