Stefan Miklosovic created CASSANDRA-21672:
---------------------------------------------
Summary: ZstdDictionaryCompressor allocates a native zstd context
per chunk
Key: CASSANDRA-21672
URL: https://issues.apache.org/jira/browse/CASSANDRA-21672
Project: Apache Cassandra
Issue Type: Bug
Components: Feature/Compression
Reporter: Stefan Miklosovic
h3. Summary
{\{ZstdDictionaryCompressor}} creates and destroys a native zstd context
(\{{ZSTD_CCtx}}/\{{ZSTD_DCtx}}) for every chunk, adding avoidable per-chunk
overhead that is disproportionately expensive at small chunk sizes.
h3. Root cause
{\{compress(ByteBuffer)}} and \{{uncompress(ByteBuffer)}} use the static
one-shot helpers \{{Zstd.compressDirectByteBufferFastDict}} /
\{{Zstd.decompressDirectByteBufferFastDict}}, each of which internally
constructs and closes a \{{ZstdCompressCtx}} / \{{ZstdDecompressCtx}} per
invocation. The digested dictionary (CDict) is already cached on the
compressor, so this is purely context-lifecycle overhead. The cost is fixed per
chunk and therefore scales with chunk count.
h3. Impact
Async-profiler sampling of a dictionary-compressed flush at
\{{chunk_length_in_kb = 4}}:
* \{{Zstd.compressDirectByteBufferFastDict}} — ~25% of on-CPU samples
* \{{ZSTD_createCCtx}} (context initialisation) — ~6.5% of on-CPU samples
* dictionary load — negligible (CDict cached)
The effect is elevated flush/compaction CPU and increased write-path pressure
at small chunk sizes.
h3. Fix
Reuse contexts instead of creating one per chunk: maintain a per-compressor
pool of \{{ZstdCompressCtx}}/\{{ZstdDecompressCtx}}, each with the dictionary
loaded once, borrowed for a single \{{compress}}/\{{uncompress}} call and
returned afterwards (\{{ZstdCompressCtx}}/\{{ZstdDecompressCtx}} are not
thread-safe and these paths run concurrently). Pooled contexts are closed when
the compressor is evicted from its cache. No changes to zstd-jni are required.
h3. Verification
* Compression round-trip correctness tests.
* Re-profiling confirms \{{ZSTD_createCCtx}} no longer appears on the flush
path.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
