*First - Possible workarounds* If you feel secure to use the remaining RAM, you need to reduce swappiness value (and clean swap) or to disable swap. There are two drawbacks in this procedure, however. One is that you will need to better monitor the memory usage to avoid containers and vms stop working. And second, you will still get some performance degradation by the lack of RAM space to provide filesystem caching. This performance degradation, though, theoretically could be less worse than the generated by the swapping movement.
*Why you are seeing high load average and performance degradation* The increase in load average is not only incurred by the CPU usage of kswapd process, but by the I/O and "I/O wait" it generates on your storage. Even if it is a SSD (NVMe or not), its speed is not unlimited and will be saturated until the swap movement finishes, as the RAM speed is noticeably faster. This saturation on your storage I/O capacity impacts everything on your server, including its containers and vms. *Final considerations - My 2 cents* After the observations above, I do note that 30GB of free RAM on a 128GB server indicates that it has only 25% of free memory. Server admin decisions depend on the type of use and server scenario. *But*, it could be beneficial to your server (and to the workloads it is responsible to provide, like OpenVZ containers) if you maintain at least 25% of free RAM. Operating with less than this threshold could represent a risk to your operations if, for whatever reason, your server and/or its containers suffer a spike on processing/access/usage/etc, since it will not have a secure reserve of resources to respond properly. Again, as the server administrator, you can make a better decision on this threshold since you know what your server scenario and use is. Cordially, Paulo Coghi On Sun, Jun 6, 2021 at 11:37 PM George <g...@sturley.me> wrote: > Hello, > > For a very long time now (we're talking the past few years) I've faced an > issue with a number of my nodes where by two kswapd processes fire up > frequently (using ~200% CPU between them). > It's at those times that the overall load average shoots up (from a normal > range of 30-50 to 100+), resulting in temporary performance degradation. > > I'm currently using the latest stable kernel (3.10.0-1160.21.1.vz7.174.13) > with all available updates installed, but have seen it on every other > kernel in the past as well. > > There has been some suggestion from various people that upgrading the > amount of physical RAM would be beneficial, which doesn't make a great deal > of sense to me as there's always at least 30 GB RAM available on the nodes, > and 40 GB+ in most cases. > > The node specifications are as follows: > > 2x Intel Xeon E5-2670v2 > 128 GB DDR3 ECC RAM (upgradable to 256 GB if needed) > 4x 2 TB Samsung 870 SSD > Hardware RAID 10 w/ BBU (WriteBack, ReadAheadNone, Direct, No Write Cache > if Bad BBU) > > Output from top -c: https://i.imgur.com/C4Yo0V1.png > Output from free -m: https://i.imgur.com/utkZNFf.png > > My question is: has anyone seen anything like this? Is there anything that > I should consider doing to help remedy the problem? Upgrading the RAM is an > option, but I am not entirely convinced it will be beneficial. > > Many thanks. > > _______________________________________________ > Users mailing list > Users@openvz.org > https://lists.openvz.org/mailman/listinfo/users >
_______________________________________________ Users mailing list Users@openvz.org https://lists.openvz.org/mailman/listinfo/users