[
https://issues.apache.org/jira/browse/CASSANDRA-13241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15886689#comment-15886689
]
Jeff Jirsa commented on CASSANDRA-13241:
----------------------------------------
I suspect 1-2TB/node / 128GB RAM would really nice suggestions that many people
dont follow in practice (at least in 2017). Past employer we ran "lots" of
nodes (multiple petabytes worth) with 3-4T of data on 32G of RAM, and I've
heard of people trying to go as dense as 8-10TB/node (DTCS/TWCS, in particular,
are designed to let you run right up to the edge of your disk capacity, and
don't have the tons-of-open-files problem that a very dense LCS node might
have).
It IS true that a few extra GB of RAM to get faster IO is a tradeoff that some
people would glady take, and with dense nodes that may be really important
(since you're less likely to fit everything in page cache). It's also true that
if you have 32G of RAM (say you're using an AWS c4.4xl or m4.2xl or r4.xl ),
that extra 2GB of RAM may be a big deal).
I'm not saying I object to changing the default, I'm just saying I don't think
we should just jump to 4k because it's faster.
> Lower default chunk_length_in_kb from 64kb to 4kb
> -------------------------------------------------
>
> Key: CASSANDRA-13241
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13241
> Project: Cassandra
> Issue Type: Wish
> Components: Core
> Reporter: Benjamin Roth
>
> Having a too low chunk size may result in some wasted disk space. A too high
> chunk size may lead to massive overreads and may have a critical impact on
> overall system performance.
> In my case, the default chunk size lead to peak read IOs of up to 1GB/s and
> avg reads of 200MB/s. After lowering chunksize (of course aligned with read
> ahead), the avg read IO went below 20 MB/s, rather 10-15MB/s.
> The risk of (physical) overreads is increasing with lower (page cache size) /
> (total data size) ratio.
> High chunk sizes are mostly appropriate for bigger payloads pre request but
> if the model consists rather of small rows or small resultsets, the read
> overhead with 64kb chunk size is insanely high. This applies for example for
> (small) skinny rows.
> Please also see here:
> https://groups.google.com/forum/#!topic/scylladb-dev/j_qXSP-6-gY
> To give you some insights what a difference it can make (460GB data, 128GB
> RAM):
> - Latency of a quite large CF: https://cl.ly/1r3e0W0S393L
> - Disk throughput: https://cl.ly/2a0Z250S1M3c
> - This shows, that the request distribution remained the same, so no "dynamic
> snitch magic": https://cl.ly/3E0t1T1z2c0J
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
