[ https://issues.apache.org/jira/browse/CASSANDRA-10520?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15875966#comment-15875966 ]
Robert Stupp commented on CASSANDRA-10520: ------------------------------------------ re: microbench: yea - page cache, that makes sense. don't have the possibility to run it on an appropriate linux box until the weekend. re: exceptions: exceptions vs. JIT - that was one of the question marks behind my question. The 2 new dtest failures look to be fixed by the dtest patch, which LGTM. Anyway, +1 on the patch. Nice work! > Compressed writer and reader should support non-compressed data. > ---------------------------------------------------------------- > > Key: CASSANDRA-10520 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10520 > Project: Cassandra > Issue Type: Improvement > Components: Local Write-Read Paths > Reporter: Branimir Lambov > Assignee: Branimir Lambov > Labels: messaging-service-bump-required > Fix For: 4.x > > Attachments: ReadWriteTestCompression.java > > > Compressing uncompressible data, as done, for instance, to write SSTables > during stress-tests, results in chunks larger than 64k which are a problem > for the buffer pooling mechanisms employed by the > {{CompressedRandomAccessReader}}. This results in non-negligible performance > issues due to excessive memory allocation. > To solve this problem and avoid decompression delays in the cases where it > does not provide benefits, I think we should allow compressed files to store > uncompressed chunks as alternative to compressed data. Such a chunk could be > written after compression returns a buffer larger than, for example, 90% of > the input, and would not result in additional delays in writing. On reads it > could be recognized by size (using a single global threshold constant in the > compression metadata) and data could be directly transferred into the > decompressed buffer, skipping the decompression step and ensuring a 64k > buffer for compressed data always suffices. -- This message was sent by Atlassian JIRA (v6.3.15#6346)