Re: [uWSGI] Linux KSM
Le 01/02/2019 à 10:32 a.m., Krzysztof Warzecha a écrit : Hello, docs: https://uwsgi-docs.readthedocs.io/en/latest/KSM.html https://www.kernel.org/doc/Documentation/vm/ksm.txt https://www.kernel.org/doc/html/latest/admin-guide/mm/ksm.html The way I understand it, --ksm=10 tells uwsgi to call madvise(MADV_MERGABLE) on worker's memory, on every 10th request. Linux kernel will then consider that memory for same-page de-duplication. It works nice if you have uwsgi emperor with many vassals. It should work with many separate uwsgi instances too. would you recommend to enable it In theory you can save a bit of memory, but I'm pretty sure you'll have to benchmark it. Enable it and use /sys/kernel/mm/ksm/ to trigger merge and measure how effective it is (with stats from /sys/kernel/mm/ksm/pages_sharing). Thanks a lot for sharing this! I'm looking forward to try enabling ksmd with uwsgi and pypy for testing purposes... :) kind regards, tk -- tkad...@yandex.com | Twitter: @wise_project https://www.isotoperesearch.ca/ Not everyone who wander are lost. ___ uWSGI mailing list uWSGI@lists.unbit.it http://lists.unbit.it/cgi-bin/mailman/listinfo/uwsgi
Re: [uWSGI] Linux KSM
Hello, docs: https://uwsgi-docs.readthedocs.io/en/latest/KSM.html https://www.kernel.org/doc/Documentation/vm/ksm.txt https://www.kernel.org/doc/html/latest/admin-guide/mm/ksm.html The way I understand it, --ksm=10 tells uwsgi to call madvise(MADV_MERGABLE) on worker's memory, on every 10th request. Linux kernel will then consider that memory for same-page de-duplication. It works nice if you have uwsgi emperor with many vassals. It should work with many separate uwsgi instances too. > would you recommend to enable it In theory you can save a bit of memory, but I'm pretty sure you'll have to benchmark it. Enable it and use /sys/kernel/mm/ksm/ to trigger merge and measure how effective it is (with stats from /sys/kernel/mm/ksm/pages_sharing). -- Krzysztof Warzecha ___ uWSGI mailing list uWSGI@lists.unbit.it http://lists.unbit.it/cgi-bin/mailman/listinfo/uwsgi
[uWSGI] Linux KSM
Good morning, im looking for documentation about the "--ksm" option in uWSGI. What does this is supposed doing and would you recommend to enable it when using PyPy (JIT) under the hood? Thanks in advance, tk -- tkad...@yandex.com | Twitter: @wise_project https://www.isotopesoftware.ca ___ uWSGI mailing list uWSGI@lists.unbit.it http://lists.unbit.it/cgi-bin/mailman/listinfo/uwsgi
Re: [uWSGI] Linux KSM support
Dnia sobota, 26 listopada 2011 12:55:49 Roberto De Ioris pisze: > > I've started uWSGI and django app on 2 ubuntu 10.04 servers (64 bit), > > one > > with ksm=50 and other with ksm disabled. After 10h of siege stress > > testing both servers, the one running with ksm enabled in uWSGI have > > one worker eating all memory. uWSGI is running in emperor mode so I > > guess that master process for this vassal is leaking memory when ksm is > > enabled in uWSGI. > > > > uWSGI 1.0-dev-1706 > > Should be fixed now. > > Many thanks for your reports, really I want to use uWSGI in production so we both benefit from fixing its bugs. Łukasz Mierzwa ___ uWSGI mailing list uWSGI@lists.unbit.it http://lists.unbit.it/cgi-bin/mailman/listinfo/uwsgi
Re: [uWSGI] Linux KSM support
> I've started uWSGI and django app on 2 ubuntu 10.04 servers (64 bit), one > with ksm=50 and other with ksm disabled. After 10h of siege stress testing > both servers, the one running with ksm enabled in uWSGI have one worker > eating all memory. uWSGI is running in emperor mode so I guess that master > process for this vassal is leaking memory when ksm is enabled in uWSGI. > > uWSGI 1.0-dev-1706 Should be fixed now. Many thanks for your reports, really -- Roberto De Ioris http://unbit.it ___ uWSGI mailing list uWSGI@lists.unbit.it http://lists.unbit.it/cgi-bin/mailman/listinfo/uwsgi
Re: [uWSGI] Linux KSM support
I've started uWSGI and django app on 2 ubuntu 10.04 servers (64 bit), one with ksm=50 and other with ksm disabled. After 10h of siege stress testing both servers, the one running with ksm enabled in uWSGI have one worker eating all memory. uWSGI is running in emperor mode so I guess that master process for this vassal is leaking memory when ksm is enabled in uWSGI. uWSGI 1.0-dev-1706 2011/11/17 Roberto De Ioris > Hi everyone, the support for Linux KSM has been added in current mercurial > tip > > http://projects.unbit.it/uwsgi/wiki/KSM > > If you host a good amount of uWSGi instances/processes on a recent > (>=2.6.32) linux kernel, try it. > > -- > Roberto De Ioris > http://unbit.it > JID: robe...@jabber.unbit.it > > ___ > uWSGI mailing list > uWSGI@lists.unbit.it > http://lists.unbit.it/cgi-bin/mailman/listinfo/uwsgi > -- Łukasz Mierzwa ___ uWSGI mailing list uWSGI@lists.unbit.it http://lists.unbit.it/cgi-bin/mailman/listinfo/uwsgi
Re: [uWSGI] Linux KSM support
On Thu, Nov 17, 2011 at 5:04 AM, Łukasz Mierzwa wrote:
> Dnia czwartek, 17 listopada 2011 11:40:55 Roberto De Ioris pisze:
>> Hi everyone, the support for Linux KSM has been added in current mercurial
>> tip
>>
>> http://projects.unbit.it/uwsgi/wiki/KSM
>>
>> If you host a good amount of uWSGi instances/processes on a recent
>> (>=2.6.32) linux kernel, try it.
>
> This is great feature that all who need to run multiple ruby apps will be
> thankful for, but I guess that one should add few zeros at the end of max-
> requests value and maybe disable idle option.
> The great thing about KSM is that it requires cpu power only to scan/merge
> pages, so when under high load one can disable KSM (echo 1 > ksm/run) and all
> the saving are still going to be there.
i dont think it's too expensive for KSM to merge pages later on if as
they die, so long as your aren't killing stuff off very quickly.
slightly OT, but this irritated me for hours -- apparently sysctl only
works with /proc/sys? and not /sys?? @#$@#??? no way to apply
settings at boot (debian has custom /etc/sysfs.conf, though). systemd
(what i use for init) only applies sysctl variables, not sysfs ...
so i wrote this:
# awk -vs=/sys/ -ve=
"!/^\s*(#|$)/{match($0,/^([^=]+)=(.*)$/,a);t(a);print a[2] > s
a[1];}function t(a, i){for(i=1;i<3;i++){gsub(/^\s*|\s*$/,e,a[i]);}}"
/etc/sysfs.conf
... and as a .service file (modeled from sysctl.service)
-
[Unit]
Description=Apply Sysfs Variables
DefaultDependencies=no
Conflicts=shutdown.target
After=systemd-readahead-collect.service systemd-readahead-replay.service
Before=sysinit.target shutdown.target
ConditionPathExists=|/etc/sysfs.conf
[Service]
Type=oneshot
RemainAfterExit=yes
ExecStart=/usr/bin/awk -vs=/sys/ -ve=
"!/^\s*(#|$)/{match($0,/^([^=]+)=(.*)$/,a);t(a);print a[2] > s
a[1];}function t(a, i){for(i=1;i<3;i++){gsub(/^\s*|\s*$/,e,a[i]);}}"
/etc/sysfs.conf
-
... might be useful to some:
-
# cat /etc/sysfs.conf
# Set KSM config at boot
kernel/mm/ksm/sleep_millisecs = 100
kernel/mm/ksm/run = 1
-
... btw, as an example of KSMs power:
-
# for x in /sys/kernel/mm/ksm/*; do printf "%s\t%s\n" "$(basename
"$x")" "$(<"$x")"; done | column -t; echo; free -m
full_scans 173
pages_shared 10039
pages_sharing34206
pages_to_scan100
pages_unshared 107191
pages_volatile 2578
run 1
sleep_millisecs 100
total used free sharedbuffers cached
Mem: 3958 1072 2886 0 30293
-/+ buffers/cache:748 3210
Swap: 2047 0 2047
-
... this machine runs 4 KVM VMs at 256M each (ie. MORE than 1GB
dedicated), and guest + host RAM usage is less than 800MB ... savings
will be even more once i convert to 9p2000.L rootfs in all VMs, but
that's way OT for this list.
also see: http://code.google.com/p/uksm/
--
C Anthony
___
uWSGI mailing list
uWSGI@lists.unbit.it
http://lists.unbit.it/cgi-bin/mailman/listinfo/uwsgi
Re: [uWSGI] Linux KSM support
Il giorno 17/nov/2011, alle ore 17:52, Andy ha scritto: > This sounds like a great feature! > > What about "non-virgin" worker that has been around for a while, on 64-bit > system. What type of memory savings can be expected there? > > Andy Theoretically (or statistically ?), bigger apps will get more shared pages. I have noted not differences between 64 and 32 bit. > -- Roberto De Ioris http://unbit.it JID: robe...@jabber.unbit.it ___ uWSGI mailing list uWSGI@lists.unbit.it http://lists.unbit.it/cgi-bin/mailman/listinfo/uwsgi
Re: [uWSGI] Linux KSM support
This sounds like a great feature! What about "non-virgin" worker that has been around for a while, on 64-bit system. What type of memory savings can be expected there? Andy From: Roberto De Ioris To: uWSGI developers and users list Sent: Thursday, November 17, 2011 6:57 AM Subject: Re: [uWSGI] Linux KSM support > On Thu, 2011-11-17 at 11:40 +0100, Roberto De Ioris wrote: >> Hi everyone, the support for Linux KSM has been added in current >> mercurial tip >> >> http://projects.unbit.it/uwsgi/wiki/KSM >> >> If you host a good amount of uWSGi instances/processes on a recent >> (>=2.6.32) linux kernel, try it. > > This is GREAT. I tried out some time ago with a friend (who wrote the > code) a different approach to the problem (using LD_PRELOAD dlsym to > replace malloc/realloc to madvise pages), and it was working but savings > were very low (it was probably the wrong approach). But it was just a > Saturday afternoon hack (well, code for ~1h, then tests :P) > I'll try KSM with uwsgi very soon, thanks again for your work :) > I tried something similar, some month ago, but there are so many ways to allocate memory (in uWSGI for example we use 5 different ways) that managing the thing asynchronously is the only way to go. Luckily enough, linux exports process mappings via proc file, so it has been only a matter of parsing that file when needed and calling madvise() Each "virgin"+python uWSGI worker, can "share" about 200 pages on a 32 bit system (1 meg of about 4, so it is 25% less memory per worker) -- Roberto De Ioris http://unbit.it ___ uWSGI mailing list uWSGI@lists.unbit.it http://lists.unbit.it/cgi-bin/mailman/listinfo/uwsgi___ uWSGI mailing list uWSGI@lists.unbit.it http://lists.unbit.it/cgi-bin/mailman/listinfo/uwsgi
Re: [uWSGI] Linux KSM support
> On Thu, 2011-11-17 at 11:40 +0100, Roberto De Ioris wrote: >> Hi everyone, the support for Linux KSM has been added in current >> mercurial tip >> >> http://projects.unbit.it/uwsgi/wiki/KSM >> >> If you host a good amount of uWSGi instances/processes on a recent >> (>=2.6.32) linux kernel, try it. > > This is GREAT. I tried out some time ago with a friend (who wrote the > code) a different approach to the problem (using LD_PRELOAD dlsym to > replace malloc/realloc to madvise pages), and it was working but savings > were very low (it was probably the wrong approach). But it was just a > Saturday afternoon hack (well, code for ~1h, then tests :P) > I'll try KSM with uwsgi very soon, thanks again for your work :) > I tried something similar, some month ago, but there are so many ways to allocate memory (in uWSGI for example we use 5 different ways) that managing the thing asynchronously is the only way to go. Luckily enough, linux exports process mappings via proc file, so it has been only a matter of parsing that file when needed and calling madvise() Each "virgin"+python uWSGI worker, can "share" about 200 pages on a 32 bit system (1 meg of about 4, so it is 25% less memory per worker) -- Roberto De Ioris http://unbit.it ___ uWSGI mailing list uWSGI@lists.unbit.it http://lists.unbit.it/cgi-bin/mailman/listinfo/uwsgi
Re: [uWSGI] Linux KSM support
On Thu, 2011-11-17 at 11:40 +0100, Roberto De Ioris wrote: > Hi everyone, the support for Linux KSM has been added in current mercurial tip > > http://projects.unbit.it/uwsgi/wiki/KSM > > If you host a good amount of uWSGi instances/processes on a recent (>=2.6.32) > linux kernel, try it. This is GREAT. I tried out some time ago with a friend (who wrote the code) a different approach to the problem (using LD_PRELOAD dlsym to replace malloc/realloc to madvise pages), and it was working but savings were very low (it was probably the wrong approach). But it was just a Saturday afternoon hack (well, code for ~1h, then tests :P) I'll try KSM with uwsgi very soon, thanks again for your work :) Thanks for the credits on the doc page, btw :) Giacomo ___ uWSGI mailing list uWSGI@lists.unbit.it http://lists.unbit.it/cgi-bin/mailman/listinfo/uwsgi
Re: [uWSGI] Linux KSM support
Dnia czwartek, 17 listopada 2011 11:40:55 Roberto De Ioris pisze: > Hi everyone, the support for Linux KSM has been added in current mercurial > tip > > http://projects.unbit.it/uwsgi/wiki/KSM > > If you host a good amount of uWSGi instances/processes on a recent > (>=2.6.32) linux kernel, try it. This is great feature that all who need to run multiple ruby apps will be thankful for, but I guess that one should add few zeros at the end of max- requests value and maybe disable idle option. The great thing about KSM is that it requires cpu power only to scan/merge pages, so when under high load one can disable KSM (echo 1 > ksm/run) and all the saving are still going to be there. Łukasz Mierzwa ___ uWSGI mailing list uWSGI@lists.unbit.it http://lists.unbit.it/cgi-bin/mailman/listinfo/uwsgi
[uWSGI] Linux KSM support
Hi everyone, the support for Linux KSM has been added in current mercurial tip http://projects.unbit.it/uwsgi/wiki/KSM If you host a good amount of uWSGi instances/processes on a recent (>=2.6.32) linux kernel, try it. -- Roberto De Ioris http://unbit.it JID: robe...@jabber.unbit.it ___ uWSGI mailing list uWSGI@lists.unbit.it http://lists.unbit.it/cgi-bin/mailman/listinfo/uwsgi
