[
https://issues.apache.org/jira/browse/CASSANDRA-9738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14721715#comment-14721715
]
Robert Stupp edited comment on CASSANDRA-9738 at 8/30/15 9:23 PM:
------------------------------------------------------------------
Code's been refactored and unit tests have no failures, but some dtests failed
recently - so it's not yet fully done.
There are several commits on the branch.
It starts with cleanup refactorings and preparation by encapsulating all
RowIndexEntry and IndexInfo accesses.
Next is introduction of ByteBuffer (744dde329a084a7fdab52fbbed11a15b296bab35),
which is the "main" commit.
Following commits are optimizations on top of that commit.
The main change is that all IndexInfo objects are just present as a single
(on-heap) ByteBuffer. IndexInfo objects are created when needed. This helps to
reduce the amount of small, temporary and maybe unused objects - e.g. during
binary search.
I've got no benchmark in place yet and latest cstar tests neither show anything
new nor a regression. I just didn't succeed to get really big partitions - so
I'll try to build some artificial microbenchmarks to get some comparable
numbers and prove that the approach also works for really big partitions (at
least in theory it should, but who knows).
The plan to do binary search in off-heap key cache or deserialize IndexInfo
directly from off-heap or even just retrieve requested information without
deserialization at all is not yet done. It requires more refactoring than I
expected as RowIndexEntry and IndexInfo objects are used at many distinct
places and "exposing" the off-heap ByteBuffer containing the serialized
RowIndexEntry+IndexInfos requires proper reference counting (or risk off-heap
memory leaks).
EDIT: [dtest build
#53|http://cassci.datastax.com/view/Dev/view/snazy/job/snazy-9738-key-cache-ohc-dtest/53/testReport/]
show a regression in compaction throughput, which definitely needs to be fixed
- but no functional failures as far as I can see.
was (Author: snazy):
Code's been refactored and unit tests have no failures, but some dtests failed
recently - so it's not yet fully done.
There are several commits on the branch.
It starts with cleanup refactorings and preparation by encapsulating all
RowIndexEntry and IndexInfo accesses.
Next is introduction of ByteBuffer (744dde329a084a7fdab52fbbed11a15b296bab35),
which is the "main" commit.
Following commits are optimizations on top of that commit.
The main change is that all IndexInfo objects are just present as a single
(on-heap) ByteBuffer. IndexInfo objects are created when needed. This helps to
reduce the amount of small, temporary and maybe unused objects - e.g. during
binary search.
I've got no benchmark in place yet and latest cstar tests neither show anything
new nor a regression. I just didn't succeed to get really big partitions - so
I'll try to build some artificial microbenchmarks to get some comparable
numbers and prove that the approach also works for really big partitions (at
least in theory it should, but who knows).
The plan to do binary search in off-heap key cache or deserialize IndexInfo
directly from off-heap or even just retrieve requested information without
deserialization at all is not yet done. It requires more refactoring than I
expected as RowIndexEntry and IndexInfo objects are used at many distinct
places and "exposing" the off-heap ByteBuffer containing the serialized
RowIndexEntry+IndexInfos requires proper reference counting (or risk off-heap
memory leaks).
> Migrate key-cache to be fully off-heap
> --------------------------------------
>
> Key: CASSANDRA-9738
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9738
> Project: Cassandra
> Issue Type: Sub-task
> Reporter: Robert Stupp
> Assignee: Robert Stupp
> Fix For: 3.0 beta 2
>
>
> Key cache still uses a concurrent map on-heap. This could go to off-heap and
> feels doable now after CASSANDRA-8099.
> Evaluation should be done in advance based on a POC to prove that pure
> off-heap counter cache buys a performance and/or gc-pressure improvement.
> In theory, elimination of on-heap management of the map should buy us some
> benefit.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
