anton-vinogradov opened a new pull request, #13325: URL: https://github.com/apache/ignite/pull/13325
https://issues.apache.org/jira/browse/IGNITE-28853 `CompressedMessage` moves the same bytes through memory 3-5 times and allocates direct `ByteBuffer`s per message on both sides of the wire. Direct allocation is expensive (memory zeroing, Cleaner-based release, potential `System.gc()` inside `Bits.reserveMemory`), while no point of the path actually needs a direct buffer: data arrives in and leaves through heap arrays. **Send path:** `compress()` copies the whole source buffer into a `byte[]`, deflates via `DeflaterOutputStream` (512-byte internal buffer → many small JNI calls) into a `ByteArrayOutputStream` pre-sized to the *uncompressed* length, then copies again via `toByteArray()`; `ChunkedByteReader` then copies every 10K chunk into a fresh array one more time. **Receive path:** `CompressedMessageSerializer.readFrom()` accumulates incoming chunks into a 100KB direct `ByteBuffer` allocated per message (grown by doubling through another copy), although each chunk is already a fresh heap array returned by `readByteArray()`; `uncompress()` copies it all back into a heap array and inflates via `InflaterInputStream.readAllBytes()` (internal 8K buffers + final consolidation copy) despite the exact result size being known upfront; `DirectMessageReader.readCompressedMessageAndDeserialize()` then copies the whole uncompressed payload into yet another per-message direct buffer, although `DirectByteBufferStream` fully supports heap buffers. **Fix (wire format unchanged):** * Internal representation switched to `List<byte[]> chunks` for both directions, `ChunkedByteReader` removed. * `compress()`: raw `Deflater` with `setInput(ByteBuffer)` (no input copy), deflating straight into wire-ready chunks — compressed bytes are written exactly once. * `readFrom()`: a received chunk is simply added to the list — zero copies, zero direct allocations. * `uncompress()`: raw `Inflater` fed chunk by chunk into an exact-size `byte[dataSize]`. * `readCompressedMessageAndDeserialize()`: `ByteBuffer.wrap(uncompressed)` instead of `allocateDirect`+`put`+`flip`. **JMH** (`GridDhtPartitionsFullMessage` receive round-trip with two `@Compress` map fields, JDK 17, M-series; benchmark source attached in a comment below): | entries | master | patched | heap alloc | |---|---|---|---| | 30 | 15.2K ± 34.8K ops/s | **100.9K ± 6.2K** (~6.6x) | 66.7K → 25.2K B/op (−62%) | | 500 | 4.38K ± 0.3K ops/s | **5.76K ± 0.5K** (+31%) | 522K → 431K B/op (−18%) | Master's huge variance at 30 entries is caused by the per-message direct allocations triggering GC storms. On top of the heap savings, all per-message direct buffer allocations (~365KB/op at 500 entries, invisible to `gc.alloc.rate`) are eliminated. 🤖 Generated with [Claude Code](https://claude.com/claude-code) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
