david-mollitor-db opened a new pull request, #58857: URL: https://github.com/apache/spark/pull/58857
### What changes were proposed in this pull request? `TransportFrameDecoder` sits at the head of the Netty receive pipeline and reassembles length-prefixed frames from incoming socket reads. It keeps the input `ByteBuf`s that have not yet been consumed into a frame in a `buffers` field, which is accessed purely as a FIFO queue: appended at the tail on each `channelRead`, read/removed from the head as frames are decoded, plus iteration and `clear` on cleanup. This changes `buffers` from `LinkedList<ByteBuf>` to `ArrayDeque<ByteBuf>`. Every operation used (`add`/`addLast`, `getFirst`, `removeFirst`, enhanced-for iteration, `clear`) has identical FIFO semantics on `ArrayDeque`, so the swap is behavior-preserving. ### Why are the changes needed? `LinkedList` allocates a node object on every `add`. `TransportFrameDecoder.channelRead` runs on every inbound read of every connection (RPC and block transfer), so the old code produced a steady stream of short-lived list-node allocations on a hot path. `ArrayDeque` supports the same add-at-tail / poll-at-head access pattern with a single reused backing array -- the queue is typically just one buffer -- eliminating the per-read node allocation and improving cache locality. This mirrors SPARK-59431, which replaced `LinkedList` with `ArrayDeque` for `BytesToBytesMap.dataPages` for the same reason. It is an allocation micro-optimization; it is not expected to move throughput on its own, since the shuffle data path is dominated by zero-copy `FileRegion` transfers rather than framing. ### Does this PR introduce _any_ user-facing change? No. ### How was this patch tested? Existing `TransportFrameDecoderSuite` passes (7 tests), covering frame decoding, a length field split across buffers, retained frames, interception, empty/negative frame sizes, and consolidation -- the cases that exercise `buffers` across multiple reads. This is an internal data-structure change with no behavior difference, so no new tests were added. ### Was this patch authored or co-authored using generative AI tooling? Generated-by: Isaac This pull request and its description were written by Isaac. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
