[
https://issues.apache.org/jira/browse/CASSANDRA-21672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18116799#comment-18116799
]
Yifan Cai commented on CASSANDRA-21672:
---------------------------------------
+1 on the new PR.
> ZstdDictionaryCompressor allocates a native zstd context per chunk
> ------------------------------------------------------------------
>
> Key: CASSANDRA-21672
> URL: https://issues.apache.org/jira/browse/CASSANDRA-21672
> Project: Apache Cassandra
> Issue Type: Bug
> Components: Feature/Compression
> Reporter: Stefan Miklosovic
> Assignee: Stefan Miklosovic
> Priority: Normal
> Fix For: 6.x
>
> Attachments: 2026-09-10-13-33-41.html
>
>
> h3. Summary
>
> {{ZstdDictionaryCompressor}} creates and destroys a native zstd context
> ({{{}ZSTD_CCtx{}}}/{{{}ZSTD_DCtx{}}}) for every chunk, adding avoidable
> per-chunk overhead that is disproportionately expensive at small chunk sizes.
> h3. Root cause
>
> {{compress(ByteBuffer)}} and {{uncompress(ByteBuffer)}} use the static
> one-shot helpers {{Zstd.compressDirectByteBufferFastDict}} /
> {{{}Zstd.decompressDirectByteBufferFastDict{}}}, each of which internally
> constructs and closes a {{ZstdCompressCtx}} / {{ZstdDecompressCtx}} per
> invocation. The digested dictionary (CDict) is already cached on the
> compressor, so this is purely context-lifecycle overhead. The cost is fixed
> per chunk and therefore scales with chunk count.
> h3. Impact
>
> Async-profiler sampling of a dictionary-compressed flush at
> {{{}chunk_length_in_kb = 4{}}}:
> * {{Zstd.compressDirectByteBufferFastDict}} — ~25% of on-CPU samples
> * {{ZSTD_createCCtx}} (context initialisation) — ~6.5% of on-CPU samples
> * dictionary load — negligible (CDict cached)
>
> The effect is elevated flush/compaction CPU and increased write-path pressure
> at small chunk sizes.
> h3. Fix
>
> Reuse contexts instead of creating one per chunk: maintain a per-compressor
> pool of {{{}ZstdCompressCtx{}}}/{{{}ZstdDecompressCtx{}}}, each with the
> dictionary loaded once, borrowed for a single
> {{{}compress{}}}/{{{}uncompress{}}} call and returned afterwards
> ({{{}ZstdCompressCtx{}}}/{{{}ZstdDecompressCtx{}}} are not thread-safe and
> these paths run concurrently). Pooled contexts are closed when the compressor
> is evicted from its cache. No changes to zstd-jni are required.
> h3. Verification
>
> * Compression round-trip correctness tests.
> * Re-profiling confirms {{ZSTD_createCCtx}} no longer appears on the flush
> path.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
