[jira] [Updated] (CASSANDRA-5863) In process (uncompressed) page cache
[ https://issues.apache.org/jira/browse/CASSANDRA-5863?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksey Yeschenko updated CASSANDRA-5863: - Fix Version/s: (was: 3.x) 3.6 > In process (uncompressed) page cache > > > Key: CASSANDRA-5863 > URL: https://issues.apache.org/jira/browse/CASSANDRA-5863 > Project: Cassandra > Issue Type: Sub-task >Reporter: T Jake Luciani >Assignee: Branimir Lambov > Labels: performance > Fix For: 3.6 > > > Currently, for every read, the CRAR reads each compressed chunk into a > byte[], sends it to ICompressor, gets back another byte[] and verifies a > checksum. > This process is where the majority of time is spent in a read request. > Before compression, we would have zero-copy of data and could respond > directly from the page-cache. > It would be useful to have some kind of Chunk cache that could speed up this > process for hot data, possibly off heap. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-5863) In process (uncompressed) page cache
[ https://issues.apache.org/jira/browse/CASSANDRA-5863?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pavel Yaskevich updated CASSANDRA-5863: --- Resolution: Fixed Status: Resolved (was: Patch Available) Ok, makes sense, since this way only caching rebufferer can invalidate and as everything is wrapping we'll have to double check anyway, maybe cleaner way was to include invalidate into factory or rebuffer interface but I will leave it for the future to decide :) +1 and committed. Thanks! > In process (uncompressed) page cache > > > Key: CASSANDRA-5863 > URL: https://issues.apache.org/jira/browse/CASSANDRA-5863 > Project: Cassandra > Issue Type: Sub-task >Reporter: T Jake Luciani >Assignee: Branimir Lambov > Labels: performance > Fix For: 3.x > > > Currently, for every read, the CRAR reads each compressed chunk into a > byte[], sends it to ICompressor, gets back another byte[] and verifies a > checksum. > This process is where the majority of time is spent in a read request. > Before compression, we would have zero-copy of data and could respond > directly from the page-cache. > It would be useful to have some kind of Chunk cache that could speed up this > process for hot data, possibly off heap. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-5863) In process (uncompressed) page cache
[
https://issues.apache.org/jira/browse/CASSANDRA-5863?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Branimir Lambov updated CASSANDRA-5863:
---
Status: Patch Available (was: In Progress)
Marking patch ready for review, as I believe it is worth adding to the codebase
in its current form while I continue working on replacing and improving the
actual cache implementation.
The main changes the patch does are to extract rebuffering from
{{\[Compressed\]RandomAccessReader}} into a separate interface {{Rebufferer}},
and provide different rebuffering implementations that build upon each other to
provide functionality.
The simplest one is {{SimpleReadRebufferer}} which just reads from a channel,
and does not own a buffer. When cache is not enabled, a
{{BufferManagingRebufferer}} is used to provide it with a buffer only used by
the specific {{RandomAccessReader}}. If cache is enabled, it is used in
combination with a shared {{ReaderCache.CachingRebufferer}} which operates
cached buffers and gives them as required to (multiple) readers, without any
data copying. If rate limitation is required, a {{LimitingRebufferer}} wrapper
(specific to the reader) is applied.
In the case of compressed data, the {{SimpleReadRebufferer}} is replaced by
{{CompressedRandomAccessReader.StandardRebufferer}} or
{{CompressedRandomAccessReader.MmapRebufferer}}; the stack on top of it stays
the same.
Separately, uncompressed memory-mapped access is implemented in one step by
{{MmapRebufferer}} which can directly provide the mapped buffers (+ a
{{LimitingRebufferer}} if required).
Most of the rebufferer stack is thread-safe and is placed in {{SegmentedFile}}
to be shared among readers.
The patch also changes {{ChecksummedDataInput}} and the other hinting-related
readers to be based on {{RebufferingInputStream}} instead of
{{RandomAccessReader}} as they don't need random access (or caching).
> In process (uncompressed) page cache
>
>
> Key: CASSANDRA-5863
> URL: https://issues.apache.org/jira/browse/CASSANDRA-5863
> Project: Cassandra
> Issue Type: Sub-task
>Reporter: T Jake Luciani
>Assignee: Branimir Lambov
> Labels: performance
> Fix For: 3.x
>
>
> Currently, for every read, the CRAR reads each compressed chunk into a
> byte[], sends it to ICompressor, gets back another byte[] and verifies a
> checksum.
> This process is where the majority of time is spent in a read request.
> Before compression, we would have zero-copy of data and could respond
> directly from the page-cache.
> It would be useful to have some kind of Chunk cache that could speed up this
> process for hot data, possibly off heap.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-5863) In process (uncompressed) page cache
[ https://issues.apache.org/jira/browse/CASSANDRA-5863?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pavel Yaskevich updated CASSANDRA-5863: --- Reviewer: Pavel Yaskevich > In process (uncompressed) page cache > > > Key: CASSANDRA-5863 > URL: https://issues.apache.org/jira/browse/CASSANDRA-5863 > Project: Cassandra > Issue Type: Sub-task >Reporter: T Jake Luciani >Assignee: Branimir Lambov > Labels: performance > Fix For: 3.x > > > Currently, for every read, the CRAR reads each compressed chunk into a > byte[], sends it to ICompressor, gets back another byte[] and verifies a > checksum. > This process is where the majority of time is spent in a read request. > Before compression, we would have zero-copy of data and could respond > directly from the page-cache. > It would be useful to have some kind of Chunk cache that could speed up this > process for hot data, possibly off heap. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-5863) In process (uncompressed) page cache
[ https://issues.apache.org/jira/browse/CASSANDRA-5863?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksey Yeschenko updated CASSANDRA-5863: - Issue Type: Sub-task (was: New Feature) Parent: CASSANDRA-10994 > In process (uncompressed) page cache > > > Key: CASSANDRA-5863 > URL: https://issues.apache.org/jira/browse/CASSANDRA-5863 > Project: Cassandra > Issue Type: Sub-task >Reporter: T Jake Luciani > Labels: performance > Fix For: 3.x > > > Currently, for every read, the CRAR reads each compressed chunk into a > byte[], sends it to ICompressor, gets back another byte[] and verifies a > checksum. > This process is where the majority of time is spent in a read request. > Before compression, we would have zero-copy of data and could respond > directly from the page-cache. > It would be useful to have some kind of Chunk cache that could speed up this > process for hot data, possibly off heap. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-5863) In process (uncompressed) page cache
[ https://issues.apache.org/jira/browse/CASSANDRA-5863?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksey Yeschenko updated CASSANDRA-5863: - Assignee: Branimir Lambov > In process (uncompressed) page cache > > > Key: CASSANDRA-5863 > URL: https://issues.apache.org/jira/browse/CASSANDRA-5863 > Project: Cassandra > Issue Type: Sub-task >Reporter: T Jake Luciani >Assignee: Branimir Lambov > Labels: performance > Fix For: 3.x > > > Currently, for every read, the CRAR reads each compressed chunk into a > byte[], sends it to ICompressor, gets back another byte[] and verifies a > checksum. > This process is where the majority of time is spent in a read request. > Before compression, we would have zero-copy of data and could respond > directly from the page-cache. > It would be useful to have some kind of Chunk cache that could speed up this > process for hot data, possibly off heap. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-5863) In process (uncompressed) page cache
[ https://issues.apache.org/jira/browse/CASSANDRA-5863?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis updated CASSANDRA-5863: -- Fix Version/s: (was: 3.0) 3.1 In process (uncompressed) page cache Key: CASSANDRA-5863 URL: https://issues.apache.org/jira/browse/CASSANDRA-5863 Project: Cassandra Issue Type: New Feature Components: Core Reporter: T Jake Luciani Labels: performance Fix For: 3.1 Currently, for every read, the CRAR reads each compressed chunk into a byte[], sends it to ICompressor, gets back another byte[] and verifies a checksum. This process is where the majority of time is spent in a read request. Before compression, we would have zero-copy of data and could respond directly from the page-cache. It would be useful to have some kind of Chunk cache that could speed up this process for hot data, possibly off heap. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-5863) In process (uncompressed) page cache
[ https://issues.apache.org/jira/browse/CASSANDRA-5863?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis updated CASSANDRA-5863: -- Description: Currently, for every read, the CRAR reads each compressed chunk into a byte[], sends it to ICompressor, gets back another byte[] and verifies a checksum. This process is where the majority of time is spent in a read request. Before compression, we would have zero-copy of data and could respond directly from the page-cache. It would be useful to have some kind of Chunk cache that could speed up this process for hot data, possibly off heap. was: Currently, for every read, the CRAR reads each compressed chunk into a byte[], sends it to ICompressor, gets back another byte[] and verifies a checksum. This process is where the majority of time is spent in a read request. Before compression, we would have zero-copy of data and could respond directly from the page-cache. It would be useful to have some kind of Chunk cache that could speed up this process for hot data. Initially this could be a off heap cache but it would be great to put these decompressed chunks onto a SSD so the hot data lives on a fast disk similar to https://github.com/facebook/flashcache. In process (uncompressed) page cache Key: CASSANDRA-5863 URL: https://issues.apache.org/jira/browse/CASSANDRA-5863 Project: Cassandra Issue Type: New Feature Components: Core Reporter: T Jake Luciani Labels: performance Fix For: 3.0 Currently, for every read, the CRAR reads each compressed chunk into a byte[], sends it to ICompressor, gets back another byte[] and verifies a checksum. This process is where the majority of time is spent in a read request. Before compression, we would have zero-copy of data and could respond directly from the page-cache. It would be useful to have some kind of Chunk cache that could speed up this process for hot data, possibly off heap. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-5863) In process (uncompressed) page cache
[ https://issues.apache.org/jira/browse/CASSANDRA-5863?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis updated CASSANDRA-5863: -- Fix Version/s: (was: 2.1 rc1) 3.0 In process (uncompressed) page cache Key: CASSANDRA-5863 URL: https://issues.apache.org/jira/browse/CASSANDRA-5863 Project: Cassandra Issue Type: New Feature Components: Core Reporter: T Jake Luciani Labels: performance Fix For: 3.0 Currently, for every read, the CRAR reads each compressed chunk into a byte[], sends it to ICompressor, gets back another byte[] and verifies a checksum. This process is where the majority of time is spent in a read request. Before compression, we would have zero-copy of data and could respond directly from the page-cache. It would be useful to have some kind of Chunk cache that could speed up this process for hot data. Initially this could be a off heap cache but it would be great to put these decompressed chunks onto a SSD so the hot data lives on a fast disk similar to https://github.com/facebook/flashcache. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (CASSANDRA-5863) In process (uncompressed) page cache
[ https://issues.apache.org/jira/browse/CASSANDRA-5863?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis updated CASSANDRA-5863: -- Assignee: (was: Pavel Yaskevich) Thanks. Will unassign in case someone has another idea to try. In process (uncompressed) page cache Key: CASSANDRA-5863 URL: https://issues.apache.org/jira/browse/CASSANDRA-5863 Project: Cassandra Issue Type: New Feature Components: Core Reporter: T Jake Luciani Labels: performance Fix For: 2.1 beta2 Currently, for every read, the CRAR reads each compressed chunk into a byte[], sends it to ICompressor, gets back another byte[] and verifies a checksum. This process is where the majority of time is spent in a read request. Before compression, we would have zero-copy of data and could respond directly from the page-cache. It would be useful to have some kind of Chunk cache that could speed up this process for hot data. Initially this could be a off heap cache but it would be great to put these decompressed chunks onto a SSD so the hot data lives on a fast disk similar to https://github.com/facebook/flashcache. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (CASSANDRA-5863) In process (uncompressed) page cache
[ https://issues.apache.org/jira/browse/CASSANDRA-5863?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Benedict updated CASSANDRA-5863: Summary: In process (uncompressed) page cache (was: Create a Decompressed Chunk [block] Cache) In process (uncompressed) page cache Key: CASSANDRA-5863 URL: https://issues.apache.org/jira/browse/CASSANDRA-5863 Project: Cassandra Issue Type: New Feature Components: Core Reporter: T Jake Luciani Assignee: Pavel Yaskevich Labels: performance Fix For: 2.1 beta2 Currently, for every read, the CRAR reads each compressed chunk into a byte[], sends it to ICompressor, gets back another byte[] and verifies a checksum. This process is where the majority of time is spent in a read request. Before compression, we would have zero-copy of data and could respond directly from the page-cache. It would be useful to have some kind of Chunk cache that could speed up this process for hot data. Initially this could be a off heap cache but it would be great to put these decompressed chunks onto a SSD so the hot data lives on a fast disk similar to https://github.com/facebook/flashcache. -- This message was sent by Atlassian JIRA (v6.2#6252)
