[
https://issues.apache.org/jira/browse/CASSANDRA-21677?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18115085#comment-18115085
]
Stefan Miklosovic commented on CASSANDRA-21677:
-----------------------------------------------
For Zstd on dicts, by moving to pooled contexts we have flatten the price for
going from larger chunks to smaller ones. It basically does not matter what
chunk size one uses when on dicts, when it comes to CPU spend (the figures
there is in percents). On the other hand, pooling it in plain Zstd is not
flattening it. BUT. The second image shows the run where Cassandra was patched
with already pooled contexts for ZSTD dict and pooled contexts for plain ZSTD
for the first time.
There we can see that Zstd dicts did not gain anything new (it was the second
run with pooled contexts, nothing new), but for the first run with pooled plain
Zstd contexts we see that even it did not flatten 64->4 chunks CPU usage, it
indeed helped, it helps most for smaller chunks.
> Pool Native Zstd Contexts in Plain ZstdCompressor
> -------------------------------------------------
>
> Key: CASSANDRA-21677
> URL: https://issues.apache.org/jira/browse/CASSANDRA-21677
> Project: Apache Cassandra
> Issue Type: Improvement
> Components: Feature/Compression
> Reporter: Yifan Cai
> Priority: Normal
> Attachments: dicts.png, image2.png
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> ZstdCompressorBase.compress()/uncompress() call Zstd's static one-shot
> helpers (Zstd.compress, Zstd.decompress, Zstd.decompressByteArray), each of
> which allocates a native ZSTD_CCtx/ZSTD_DCtx and frees it before returning,
> once per chunk.
> According to CASSANDRA-21672, profiling of the dictionary-compression path
> found roughly 6.5% of node CPU in ZSTD_createCCtx alone at small chunk sizes,
> and the same per-chunk allocation cost applies to plain, non-dictionary Zstd
> compression, which is unaffected by that work. ZstdCompressor should pool a
> reusable ZstdCompressCtx/ZstdDecompressCtx per compressor instance, instead
> of allocating and freeing a native context on every chunk.
> That said, the plain ZstdCompressor is likely less impacted by the per chunk
> allocation, as it typically requires larger chunk size to achieve a good
> compression ratio. The effect of per chunk allocation is more significant for
> small chunks such as 4KiB.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
