Brad Schoening created CASSANDRA-19387:
------------------------------------------
Summary: Transition Micro config to 48GB RAM
Key: CASSANDRA-19387
URL: https://issues.apache.org/jira/browse/CASSANDRA-19387
Project: Cassandra
Issue Type: Improvement
Reporter: Brad Schoening
I think we have been starving the OS of RAM on the micro 32GB hosts. Cassandra
4.0 switched to Netty (nio), which allocates Direct Memory buffers (off heap)
and may have accentuated the issue. Reports are that Netty can be memory
intensive [Netty 4.x High Memory
Usage|https://access.redhat.com/solutions/5286701].
Some symptoms:
* Summer 2023 RHEL begins swapping to favor page cache
* Fall 2023 OutOfMemory DirectMemory seen on several clusters with 4.0.6
* Cluster 733 speed up 2X simply by switching from 32GB -> 64GB RAM
Our typical configuration for Micro:
16GB JVM
16GB Direct Memory buffers for netty I/O (default is equal to
heap size)
---------
0 memory left over for page cache or anything else
DataStax DSE
[recommends|https://docs.datastax.com/eol/en/dse/6.7/dse-admin/datastax_enterprise/config/configRecommendedSettings.html#configFileCacheSettings__chunkCacheHistory]:
* Max direct memory = (system memory – JVM heap size) / 2
* Chunk cache = (Max direct memory ) / 2
For CaaS micro nodes, this suggests
16GB (heap) + 8GB (nio) + 2GB (chunk) + 0.5GB (networking) =
26.5 GB + CaaS agent (3GB?) = 29.5GB
I’d like to see us use 48GB VMs for micro nodes.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
