This is an automated email from the ASF dual-hosted git repository.
dongjoon-hyun pushed a commit to branch branch-4.x
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/branch-4.x by this push:
new 3a8786b10845 [SPARK-57713][CORE] Deduplicate AUTO IOMode resolution in
NettyUtils
3a8786b10845 is described below
commit 3a8786b10845d990def29431fcd44ef3fa27759c
Author: YangJie <[email protected]>
AuthorDate: Fri Jun 26 08:06:41 2026 -0700
[SPARK-57713][CORE] Deduplicate AUTO IOMode resolution in NettyUtils
### What changes were proposed in this pull request?
`NettyUtils.createEventLoop`, `getClientChannelClass`, and
`getServerChannelClass` each contained an identical copy of the `IOMode.AUTO`
resolution (EPOLL on Linux, KQUEUE on macOS, NIO otherwise). This PR extracts
that logic into a single private `resolveMode` helper, and each method now
switches on the resolved concrete mode. The `AUTO` switch arm becomes
unreachable, so it is kept as an explicit arm that throws, which preserves the
exhaustiveness check over `IOMode`.
### Why are the changes needed?
The resolution logic was duplicated three times, so the methods could
easily drift apart when one is updated and the others are not. Keeping it in
one place removes that risk. Because every switch still lists all `IOMode`
constants explicitly, adding a future transport (for example io_uring) will
fail to compile at all three call sites and force each to be handled
deliberately.
### Does this PR introduce _any_ user-facing change?
No. This is a behavior-preserving refactor: non-AUTO modes pass through
unchanged, and AUTO resolves to the same transport as before.
### How was this patch tested?
No new tests. The change has no functional difference, and the affected
code paths are exercised by the existing `common/network-common` transport
suites that run in CI. Built locally with `build/sbt network-common/compile`.
### Was this patch authored or co-authored using generative AI tooling?
Generated-by: Claude Code (Claude Opus 4.8)
Closes #56806 from LuciferYang/nettyutils-auto-iomode.
Authored-by: YangJie <[email protected]>
Signed-off-by: Dongjoon Hyun <[email protected]>
(cherry picked from commit 688064e691a63a44075dee769eb07ceed19068f6)
Signed-off-by: Dongjoon Hyun <[email protected]>
---
.../org/apache/spark/network/util/NettyUtils.java | 58 +++++++++++-----------
1 file changed, 28 insertions(+), 30 deletions(-)
diff --git
a/common/network-common/src/main/java/org/apache/spark/network/util/NettyUtils.java
b/common/network-common/src/main/java/org/apache/spark/network/util/NettyUtils.java
index c113b72f557c..2a1931290dba 100644
---
a/common/network-common/src/main/java/org/apache/spark/network/util/NettyUtils.java
+++
b/common/network-common/src/main/java/org/apache/spark/network/util/NettyUtils.java
@@ -65,60 +65,58 @@ public class NettyUtils {
return new DefaultThreadFactory(threadPoolPrefix, true);
}
+ /** Message for the unreachable AUTO arms below; resolveMode never returns
AUTO. */
+ private static final String UNRESOLVED_AUTO_MODE = "AUTO should be resolved
by resolveMode";
+
+ /**
+ * Resolves {@link IOMode#AUTO} to a concrete transport for the current
platform: EPOLL on
+ * Linux, KQUEUE on macOS, and NIO otherwise (including when the native
transport is not
+ * available). Any other mode is returned unchanged. Keeping this in one
place stops the
+ * event-loop and channel factories below from drifting apart.
+ */
+ private static IOMode resolveMode(IOMode mode) {
+ if (mode != IOMode.AUTO) {
+ return mode;
+ }
+ if (JavaUtils.isLinux && Epoll.isAvailable()) {
+ return IOMode.EPOLL;
+ } else if (JavaUtils.isMac && KQueue.isAvailable()) {
+ return IOMode.KQUEUE;
+ } else {
+ return IOMode.NIO;
+ }
+ }
+
/** Creates a Netty EventLoopGroup based on the IOMode. */
public static EventLoopGroup createEventLoop(IOMode mode, int numThreads,
String threadPrefix) {
ThreadFactory threadFactory = createThreadFactory(threadPrefix);
- IoHandlerFactory handlerFactory = switch (mode) {
+ IoHandlerFactory handlerFactory = switch (resolveMode(mode)) {
case NIO -> NioIoHandler.newFactory();
case EPOLL -> EpollIoHandler.newFactory();
case KQUEUE -> KQueueIoHandler.newFactory();
- case AUTO -> {
- if (JavaUtils.isLinux && Epoll.isAvailable()) {
- yield EpollIoHandler.newFactory();
- } else if (JavaUtils.isMac && KQueue.isAvailable()) {
- yield KQueueIoHandler.newFactory();
- } else {
- yield NioIoHandler.newFactory();
- }
- }
+ case AUTO -> throw new IllegalStateException(UNRESOLVED_AUTO_MODE);
};
return new MultiThreadIoEventLoopGroup(numThreads, threadFactory,
handlerFactory);
}
/** Returns the correct (client) SocketChannel class based on IOMode. */
public static Class<? extends Channel> getClientChannelClass(IOMode mode) {
- return switch (mode) {
+ return switch (resolveMode(mode)) {
case NIO -> NioSocketChannel.class;
case EPOLL -> EpollSocketChannel.class;
case KQUEUE -> KQueueSocketChannel.class;
- case AUTO -> {
- if (JavaUtils.isLinux && Epoll.isAvailable()) {
- yield EpollSocketChannel.class;
- } else if (JavaUtils.isMac && KQueue.isAvailable()) {
- yield KQueueSocketChannel.class;
- } else {
- yield NioSocketChannel.class;
- }
- }
+ case AUTO -> throw new IllegalStateException(UNRESOLVED_AUTO_MODE);
};
}
/** Returns the correct ServerSocketChannel class based on IOMode. */
public static Class<? extends ServerChannel> getServerChannelClass(IOMode
mode) {
- return switch (mode) {
+ return switch (resolveMode(mode)) {
case NIO -> NioServerSocketChannel.class;
case EPOLL -> EpollServerSocketChannel.class;
case KQUEUE -> KQueueServerSocketChannel.class;
- case AUTO -> {
- if (JavaUtils.isLinux && Epoll.isAvailable()) {
- yield EpollServerSocketChannel.class;
- } else if (JavaUtils.isMac && KQueue.isAvailable()) {
- yield KQueueServerSocketChannel.class;
- } else {
- yield NioServerSocketChannel.class;
- }
- }
+ case AUTO -> throw new IllegalStateException(UNRESOLVED_AUTO_MODE);
};
}
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
