[ https://issues.apache.org/jira/browse/CASSANDRA-15229?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17074656#comment-17074656 ]
Benedict Elliott Smith commented on CASSANDRA-15229: ---------------------------------------------------- bq. a bump-the-pointer slab approach for the transient pool, not to dissimilar from the current implementation. We then exploit our thread per core architecture: core threads get a dedicated slab each, other threads share a global slab. The current implementation isn't really a bump the pointer allocator? It's bitmap based, though with a very tiny bitmap. Could you elaborate on how these work, as my intuition is that anything designed for a thread-per-core architecture probably won't translate so well to the present state of the world. Though, either way, I suppose this is probably orthogonal to this ticket as we only need to address the {{ChunkCache}} part. bq. We also optimized the chunk cache to store memory addresses rather than byte buffers, which significantly reduced heap usage. The byte buffers are materialized on the fly. This would be a huge improvement, and a welcome backport if it is easy - though it might (I would guess) depend on {{Unsafe}}, which may be going away soon. It's orthogonal to this ticket, though, I think. bq. We changed the chunk cache to always store buffers of the same size. bq. We have global lists of these slabs, sorted by buffer size where each size is a power-of-two. How do these two statements reconcile? Is it your opinion that your entire {{ChunkCache}} implementation can be dropped wholesale into 4.0? I would assume it is still primarily multi-threaded. If so, it might be preferable to trying to fix the existing {{ChunkCache}} > BufferPool Regression > --------------------- > > Key: CASSANDRA-15229 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15229 > Project: Cassandra > Issue Type: Bug > Components: Local/Caching > Reporter: Benedict Elliott Smith > Assignee: ZhaoYang > Priority: Normal > Fix For: 4.0, 4.0-beta > > > The BufferPool was never intended to be used for a {{ChunkCache}}, and we > need to either change our behaviour to handle uncorrelated lifetimes or use > something else. This is particularly important with the default chunk size > for compressed sstables being reduced. If we address the problem, we should > also utilise the BufferPool for native transport connections like we do for > internode messaging, and reduce the number of pooling solutions we employ. > Probably the best thing to do is to improve BufferPool’s behaviour when used > for things with uncorrelated lifetimes, which essentially boils down to > tracking those chunks that have not been freed and re-circulating them when > we run out of completely free blocks. We should probably also permit > instantiating separate {{BufferPool}}, so that we can insulate internode > messaging from the {{ChunkCache}}, or at least have separate memory bounds > for each, and only share fully-freed chunks. > With these improvements we can also safely increase the {{BufferPool}} chunk > size to 128KiB or 256KiB, to guarantee we can fit compressed pages and reduce > the amount of global coordination and per-allocation overhead. We don’t need > 1KiB granularity for allocations, nor 16 byte granularity for tiny > allocations. -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org