[
https://issues.apache.org/jira/browse/CASSANDRA-15229?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17077609#comment-17077609
]
Benedict Elliott Smith commented on CASSANDRA-15229:
----------------------------------------------------
{quote}That's absolutely true. However, it's also true that none of our users
reported any problems when the cache was 512 MB... So my point was simply
meant to say that we should be mindful of changing critical code late in a
release cycle if the existing code is performing adequately.
{quote}
There has been at least one report, I'm fairly sure, but it is besides the
point IMO. I personally don't endorse this adequacy criterion: we've done a
lot of inadequate things users haven't noticed.
As also mentioned in other fora (perhaps not this ticket), the situation will
be much worse in 4.0, since the default page size has shrunk and the
{{BufferPool}} page size has grown. These changes are very likely to lead to a
minority of assigned memory being usable, and a majority being served from
system memory. In which case we might as well just do that for everything.
This is a simple solution with similar performance characteristics and no
wasted memory. Given the cache fronts decompression the advantage of an
internal memory pool here was anyway surely always fairly minimal? Was it ever
demonstrated to be beneficial?
An alternative is perhaps to remove the {{ChunkCache}} entirely, or at least
deprecate it, or mark it "experimental"
{quote}Also, indexes would use different chunk sizes surely?
{quote}
Perhaps, but should we be caching indexes at all? I thought the purpose of the
chunk cache was for compressed data that has been uncompressed, and I'm fairly
sure we don't compress indexes? Since it is probably of limited value for data
that is already uncompressed, versus the file system cache. But I have not
looked closely at any of the {{ChunkCache;}} only close enough to notice the
problematic behaviour mentioned in this ticket.
{quote}I've dropped some files on this
[branch|https://github.com/stef1927/cassandra/tree/15229-4.0].
{quote}
Thanks, that's greatly appreciated, though I won't be able to take a look for a
couple of weeks at least unfortunately.
> BufferPool Regression
> ---------------------
>
> Key: CASSANDRA-15229
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15229
> Project: Cassandra
> Issue Type: Bug
> Components: Local/Caching
> Reporter: Benedict Elliott Smith
> Assignee: ZhaoYang
> Priority: Normal
> Fix For: 4.0, 4.0-beta
>
>
> The BufferPool was never intended to be used for a {{ChunkCache}}, and we
> need to either change our behaviour to handle uncorrelated lifetimes or use
> something else. This is particularly important with the default chunk size
> for compressed sstables being reduced. If we address the problem, we should
> also utilise the BufferPool for native transport connections like we do for
> internode messaging, and reduce the number of pooling solutions we employ.
> Probably the best thing to do is to improve BufferPool’s behaviour when used
> for things with uncorrelated lifetimes, which essentially boils down to
> tracking those chunks that have not been freed and re-circulating them when
> we run out of completely free blocks. We should probably also permit
> instantiating separate {{BufferPool}}, so that we can insulate internode
> messaging from the {{ChunkCache}}, or at least have separate memory bounds
> for each, and only share fully-freed chunks.
> With these improvements we can also safely increase the {{BufferPool}} chunk
> size to 128KiB or 256KiB, to guarantee we can fit compressed pages and reduce
> the amount of global coordination and per-allocation overhead. We don’t need
> 1KiB granularity for allocations, nor 16 byte granularity for tiny
> allocations.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
