The 'swappiness' proc knob was on its default of 60, but I just don't
get it... the VIRT columns of all running process don't add up to
anywhere near (physical RAM + swap in use) (they add to under 10GB)
It can't even be disk cache because there's only 660MB in use on the
rootfs...
Filesystem 1K-blocks Used Available Use% Mounted on
/dev/sda1 230528596 669032 218149388 1% /
I don't want to dive deep into a discussion about how Linux swap works
and such. I will leave that for a Linux mailing list and people who have
the full deep understanding. I will only say that, in my experience, if
you give Linux swap it is going to use it for something sometime.
The big issue I see here is that you have allocated 7.5GB to memcached
on a system with only 8GB or RAM. That leaves only 500MB for the system
AND memcached overhead. For example, on one of my servers, I have
memcached set to use 512MB of RAM. Its RSS in ps is 590712. It is using
80MB of memory for other uses. This includes the application itself,
connection overhead, etc. I like to keep 1GB of free memory on all my
systems for the system.
> I fully expect if I restart the 3 memcache processes (not something I
> want to do since it will hurt live performance) the swap will zero,
> but that doesn't solve the root cause.
Well, that is a whole other discussion. Restarting one memcached daemon
should never be a problem assuming the rest of the system is up and
running. If it is, you may want to reconsider your cache architecture.
--
Brian.
--------
http://brian.moonspot.net/
