Hi there,
If I may can resuscitate a very old thread, I think we might still be
interested in per-worker cgroup settings, something like
cgroup = /sys/fs/cgroups/memory/domains/vassal-domain-name/worker-$WORKER_ID
We can't run the master as root, but can run the emperor as root. We're
using various other cgroups already, so it'd just be a matter of setting
a unique name per worker to the cgroup, from within the vassal config
file -- is there any way to do that?
evil-reload-on-as *might* work, but the advantage of the cgroup is the
user will actually see a MemoryError exception, rather than just a
random "killed". I think...
rgds,
Harry & the PythonAnywhere team
On 25/05/16 15:58, Roberto De Ioris wrote:
Hi there,
We're trying to apply some memory limitations to apps that may contain a
variable number of workers.
We know we can set
cgroup = /mnt/cgroups/memory/domains/vassal-domain-name
cgroup-opt = memory.limit_in_bytes=1800M
cgroup-opt = memory.memsw.limit_in_bytes=1800M
And that's fine for limiting a user with one worker to 1.8GB of RAM, but
the problem is that if a user has 10 workers, the cgroup limits the
*total* memory usage to 1.8GB, which is now down to less than 200Mb per
worker, and that's too little. We also don't want to just bump that
user up to an 18GB limit, since that's potentially higher than the
system ram, and besides, this is mainly about catching accidental memory
leaks in individual processes rather than limiting total consumption.
So the question is:
* Is there any way to put each worker in its own cgroup? *
I could imagine it working if uwsgi had some magical template variable
syntax, a bit like nginx? eg:
cgroup =
/mnt/cgroups/memory/domains/vassal-domain-name/worker-$WORKER_ID
Is there any such thing?
Or perhaps there would be a way of running a script, pre-jail put
post-fork, that ran as root/privileged user, that would be able to
identify the spawned worker processes and manually put them into their
own cgroups? Could exec-post-jail or exec-pre-app work for this?
Would there be any way for such a job to identify the worker process ids?
Hi albeit, technically possible, the problem is that it would require the
master to run as root (cgroup filesystem cannot be written by unprivileged
user) and i am not sure it would be a great advantage. Even if you charge
the users by their amount of workers having 10 cgroup by 1.8 gigs, or 1 of
18 does not change the fact that a user could bring down your machine.
Maybe you charge users by workers ? (so a user paying for 2 workers will
get potentially twice the memory ?). I would go by setting a "safe" cgroup
value by instance and then i would enable the memory checks features of
the master (reload-on-rss/evil-reload...and so on) that will try to
protect the user from memory leaks in his app
--
Harry Percival
Developer
ha...@pythonanywhere.com
PythonAnywhere - a fully browser-based Python development and hosting
environment
<http://www.pythonanywhere.com/>
PythonAnywhere LLP
17a Clerkenwell Road, London EC1M 5RD, UK
VAT No.: GB 893 5643 79
Registered in England and Wales as company number OC378414.
Registered address: 28 Ely Place, 3rd Floor, London EC1N 6TD, UK
_______________________________________________
uWSGI mailing list
uWSGI@lists.unbit.it
http://lists.unbit.it/cgi-bin/mailman/listinfo/uwsgi
