zhengchenyu commented on issue #2404:
URL: https://github.com/apache/uniffle/issues/2404#issuecomment-2723430740
It is not falut of
org.apache.uniffle.test.RemoteMergeShuffleWithRssClientTestWhenShuffleFlushed.
I've tested it many times with no problems.
I found shuffle server error like this:
```
[2025-03-14 03:39:30.870] [nioEventLoopGroup-71-2] [WARN]
ReferenceCountUtil.safeRelease - Failed to release a message:
org.apache.uniffle.common.netty.protocol.MessageWithHeader@5640c830
java.lang.NullPointerException
at
org.apache.uniffle.common.netty.buffer.FileSegmentManagedBuffer.release(FileSegmentManagedBuffer.java:111)
at
org.apache.uniffle.common.netty.protocol.MessageWithHeader.deallocate(MessageWithHeader.java:131)
at
io.netty.util.AbstractReferenceCounted.handleRelease(AbstractReferenceCounted.java:86)
at
io.netty.util.AbstractReferenceCounted.release(AbstractReferenceCounted.java:76)
at io.netty.util.ReferenceCountUtil.release(ReferenceCountUtil.java:90)
at
io.netty.util.ReferenceCountUtil.safeRelease(ReferenceCountUtil.java:116)
at
io.netty.channel.ChannelOutboundBuffer.remove(ChannelOutboundBuffer.java:301)
at
io.netty.channel.nio.AbstractNioByteChannel.doWriteInternal(AbstractNioByteChannel.java:242)
at
io.netty.channel.nio.AbstractNioByteChannel.doWrite0(AbstractNioByteChannel.java:212)
at
io.netty.channel.socket.nio.NioSocketChannel.doWrite(NioSocketChannel.java:407)
at
io.netty.channel.AbstractChannel$AbstractUnsafe.flush0(AbstractChannel.java:931)
at
io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.flush0(AbstractNioChannel.java:359)
at
io.netty.channel.AbstractChannel$AbstractUnsafe.flush(AbstractChannel.java:895)
at
io.netty.channel.DefaultChannelPipeline$HeadContext.flush(DefaultChannelPipeline.java:1372)
at
io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:935)
at
io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:921)
at
io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:907)
at
io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:127)
at
io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:937)
at
io.netty.channel.AbstractChannelHandlerContext.invokeWriteAndFlush(AbstractChannelHandlerContext.java:957)
at
io.netty.channel.AbstractChannelHandlerContext.write(AbstractChannelHandlerContext.java:982)
at
io.netty.channel.AbstractChannelHandlerContext.writeAndFlush(AbstractChannelHandlerContext.java:950)
at
io.netty.channel.AbstractChannelHandlerContext.writeAndFlush(AbstractChannelHandlerContext.java:1000)
at
io.netty.channel.DefaultChannelPipeline.writeAndFlush(DefaultChannelPipeline.java:1025)
at
io.netty.channel.AbstractChannel.writeAndFlush(AbstractChannel.java:306)
at
org.apache.uniffle.server.netty.ShuffleServerNettyHandler.handleGetSortedShuffleDataRequest(ShuffleServerNettyHandler.java:847)
at
org.apache.uniffle.server.netty.ShuffleServerNettyHandler.receive(ShuffleServerNettyHandler.java:133)
at
org.apache.uniffle.common.netty.handle.TransportRequestHandler.handle(TransportRequestHandler.java:62)
at
org.apache.uniffle.common.netty.handle.TransportChannelHandler.channelRead(TransportChannelHandler.java:100)
at
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444)
at
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
at
io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412)
at
io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:289)
at
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442)
at
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
at
io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412)
at
org.apache.uniffle.common.netty.TransportFrameDecoder.channelRead(TransportFrameDecoder.java:81)
at
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444)
at
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
at
io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412)
at
io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410)
at
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:440)
at
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
at
io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919)
at
io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:166)
at
io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:788)
at
io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:724)
at
io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:650)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:562)
at
io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997)
at
io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
at
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.lang.Thread.run(Thread.java:750)
[2025-03-14 03:39:30.870] [DefaultMergeEventHandler-1] [INFO]
ShuffleBufferManager.flushIfNecessary - Start to flush with usedMemory[90],
preAllocatedSize[0], inFlushSize[0]
```
The problem is introduce by #1658. The below code is not reasonable. The
problem is that Multiple releases will result in NPE.
<img width="981" alt="Image"
src="https://github.com/user-attachments/assets/4171e3bb-4542-4dff-ac33-ff463c11d8f0";
/>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
