A caveat to the 31GB recommendation for G1GC. If you have tight latency SLAs instead of throughput SLAs then this doesn’t necessary pan out to be beneficial.
Yes the GCs are less frequent, but they can hurt more when they do happen. The win is if your usage pattern is such that the added time helps you squeak past deciding copying into old gen when a smaller heap/more frequent GC cycle would have decided it had to do promotions. C* tends to have a lot of medium-lifetime objects on the heap so it can really come down to the specifics of what your clients are typically doing. Also, reallocation of RAM from O/S buffer cache to Java heap will also change the dynamics of dirty page flushes from your writes, which again directly surfaces in C* read latency numbers during I/O stalls from the write spikes in the background. So really bumping up heap is an alteration that can be a double-whammy for the latency sensitive. Those only caring about throughput won’t care and it’s probably unconditionally a win to go to 31GB. R From: Erick Ramirez <erick.rami...@datastax.com> Reply-To: "user@cassandra.apache.org" <user@cassandra.apache.org> Date: Monday, February 10, 2020 at 3:55 PM To: "user@cassandra.apache.org" <user@cassandra.apache.org> Subject: Re: Overload because of hint pressure + MVs Message from External Sender Currently the value of phi_convict_threshold is not set which makes it to 8 (default) . Can this also cause hints buildup even when we can see that all nodes are UP ? You can bump it up to 12 to reduce the sensitivity but it's likely GC pauses causing it. Phi convict is the side-effect, not the cause. Just to add , we are using 24GB heap size. Are you using CMS? If using G1, I'd recommend bumping it up to 31GB if the servers have 40+ GB of RAM. Cheers!
