[
https://issues.apache.org/jira/browse/CASSANDRA-15229?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17076712#comment-17076712
]
Benedict Elliott Smith commented on CASSANDRA-15229:
----------------------------------------------------
bq. memory wasted due to fragmentation is perhaps not an issue with a cache as
little as 512 MB
My view is that having a significant proportion of memory wasted to
fragmentation is a serious bug, irregardless of the total amount of memory that
is wasted.
bq. The point is that it is not suitable for long lived buffers, similarly to
our bump the pointer strategy.
It's not poorly suited to long lived buffers its it? Only to buffers with
widely divergent lifetimes. If the lifetimes are loosely correlated then the
length of the lifetime is mostly irrelevant I think.
bq. The changes to the buffer pool can be dropped in 4.0 if you think that
If you mean introducing a new pool specifically for {{ChunkCache}}. I'm fine
with it as an alternative to permitting {{BufferPool}} to mitigate worst case
behaviour for the {{ChunkCache}}. But verifying a replacement for
{{BufferPool}} is a lot more work, and we use the {{BufferPool}} extensively in
networking now, which requires non-uniform buffer sizes.
Honestly, given chunks are normally the same size, simply re-using the evicted
buffer if possible, and if not allocating new system memory, seems probably
sufficient to me.
> BufferPool Regression
> ---------------------
>
> Key: CASSANDRA-15229
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15229
> Project: Cassandra
> Issue Type: Bug
> Components: Local/Caching
> Reporter: Benedict Elliott Smith
> Assignee: ZhaoYang
> Priority: Normal
> Fix For: 4.0, 4.0-beta
>
>
> The BufferPool was never intended to be used for a {{ChunkCache}}, and we
> need to either change our behaviour to handle uncorrelated lifetimes or use
> something else. This is particularly important with the default chunk size
> for compressed sstables being reduced. If we address the problem, we should
> also utilise the BufferPool for native transport connections like we do for
> internode messaging, and reduce the number of pooling solutions we employ.
> Probably the best thing to do is to improve BufferPool’s behaviour when used
> for things with uncorrelated lifetimes, which essentially boils down to
> tracking those chunks that have not been freed and re-circulating them when
> we run out of completely free blocks. We should probably also permit
> instantiating separate {{BufferPool}}, so that we can insulate internode
> messaging from the {{ChunkCache}}, or at least have separate memory bounds
> for each, and only share fully-freed chunks.
> With these improvements we can also safely increase the {{BufferPool}} chunk
> size to 128KiB or 256KiB, to guarantee we can fit compressed pages and reduce
> the amount of global coordination and per-allocation overhead. We don’t need
> 1KiB granularity for allocations, nor 16 byte granularity for tiny
> allocations.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
