This is an automated email from the ASF dual-hosted git repository.
zaynt4606 pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/celeborn.git
The following commit(s) were added to refs/heads/main by this push:
new 924b054f96 [CELEBORN-2500] Use ArrayDeque instead of LinkedList in
TransportFrameDecoder
924b054f96 is described below
commit 924b054f9691a9c80c82f80a7637718c9fd394af
Author: Nicholas Jiang <[email protected]>
AuthorDate: Fri Oct 23 10:21:37 2026 +0800
[CELEBORN-2500] Use ArrayDeque instead of LinkedList in
TransportFrameDecoder
### What changes were proposed in this pull request?
Replace `LinkedList<ByteBuf>` with `ArrayDeque<ByteBuf>` for the pending
input buffers in `TransportFrameDecoder`.
The decoder assembles incoming buffers into complete frames before decoding
messages. Buffers that have not yet been consumed are kept in a FIFO queue:
`channelRead` appends at the tail, decoding reads and removes from the head,
and `handlerRemoved` releases any remaining buffers before clearing the queue.
`ArrayDeque` supports all operations used here (`add`, `getFirst`,
`removeFirst`, iteration, and `clear`) with the same ordering. Frame assembly,
buffer ownership, and reference-count release behavior remain unchanged.
Backport: https://github.com/apache/spark/pull/58857.
### Why are the changes needed?
Each incoming `ByteBuf` currently causes `LinkedList.add` to allocate a
separate list node. This adds allocation and garbage-collection work to the
Netty receive path even when the buffer can be consumed immediately.
`ArrayDeque` stores buffer references in a reusable backing array, removing
the per-buffer list-node allocation while retaining efficient FIFO access. This
is a small allocation optimization; its effect on end-to-end throughput has not
been benchmarked.
### Does this PR resolve a correctness bug?
- [ ] Yes
### Does this PR introduce _any_ user-facing change?
- [ ] Yes
No protocol, configuration, or API changes.
### How was this patch tested?
Existing RPC integration tests passed on JDK 17: 7 tests in
`RpcIntegrationSuiteJ` and 8 in `SSLRpcIntegrationSuiteJ`, with no failures or
skipped tests. These exercise single and multiple RPCs, error responses, and
one-way messages over plain and SSL connections.
Closes #3893 from SteNicholas/CELEBORN-2500.
Authored-by: Nicholas Jiang <[email protected]>
Signed-off-by: zhengtao <[email protected]>
AI-Contributed/Feature: 0/6
AI-Contributed/UT: 0/0
---
.../apache/celeborn/common/network/util/TransportFrameDecoder.java | 6 ++++--
1 file changed, 4 insertions(+), 2 deletions(-)
diff --git
a/common/src/main/java/org/apache/celeborn/common/network/util/TransportFrameDecoder.java
b/common/src/main/java/org/apache/celeborn/common/network/util/TransportFrameDecoder.java
index 83c89afbd5..972b312c7e 100644
---
a/common/src/main/java/org/apache/celeborn/common/network/util/TransportFrameDecoder.java
+++
b/common/src/main/java/org/apache/celeborn/common/network/util/TransportFrameDecoder.java
@@ -17,7 +17,7 @@
package org.apache.celeborn.common.network.util;
-import java.util.LinkedList;
+import java.util.ArrayDeque;
import com.google.common.base.Preconditions;
import io.netty.buffer.ByteBuf;
@@ -51,7 +51,9 @@ public class TransportFrameDecoder extends
ChannelInboundHandlerAdapter implemen
private static final int MAX_FRAME_SIZE = Integer.MAX_VALUE;
private static final int UNKNOWN_FRAME_SIZE = -1;
- private final LinkedList<ByteBuf> buffers = new LinkedList<>();
+ // Pending input buffers not yet consumed into a frame. ArrayDeque avoids a
+ // per-channelRead list-node allocation; used strictly FIFO.
+ private final ArrayDeque<ByteBuf> buffers = new ArrayDeque<>();
private long totalSize = 0;
private long nextFrameSize = UNKNOWN_FRAME_SIZE;
