[
https://issues.apache.org/jira/browse/DRILL-8489?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
shihuafeng updated DRILL-8489:
------------------------------
Description:
When encode throw Exception, if encode msg instanceof ReferenceCounted, netty
can release msg, but drill convert msg to OutboundRpcMessage, so netty can not
release msg. this causes sender memory leaks
exception info
{code:java}
2024-04-16 16:25:57,998 [DataClient-7] ERROR o.a.d.exec.rpc.RpcExceptionHandler
- Exception in RPC communication. Connection: /10.32.112.138:47924 <-->
/10.32.112.138:31012 (data client). Closing connection.
io.netty.handler.codec.EncoderException:
org.apache.drill.exec.exception.OutOfMemoryException: Unable to allocate buffer
of size 4096 due to memory limit (9223372036854775807). Current allocation: 0
at
io.netty.handler.codec.MessageToMessageEncoder.write(MessageToMessageEncoder.java:107)
at
io.netty.channel.AbstractChannelHandlerContext.invokeWrite0(AbstractChannelHandlerContext.java:881)
at
io.netty.channel.AbstractChannelHandlerContext.invokeWriteAndFlush(AbstractChannelHandlerContext.java:940)
at
io.netty.channel.AbstractChannelHandlerContext$WriteTask.run(AbstractChannelHandlerContext.java:1247)
at
io.netty.util.concurrent.AbstractEventExecutor.runTask(AbstractEventExecutor.java:173)
at
io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:166)
at
io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:470)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:569)
at
io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997)
at
io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.drill.exec.exception.OutOfMemoryException: Unable to
allocate buffer of size 4096 due to memory limit (9223372036854775807). Current
allocation: 0
at
org.apache.drill.exec.memory.BaseAllocator.buffer(BaseAllocator.java:245)
at
org.apache.drill.exec.memory.BaseAllocator.buffer(BaseAllocator.java:220)
at
org.apache.drill.exec.memory.DrillByteBufAllocator.buffer(DrillByteBufAllocator.java:55)
at
org.apache.drill.exec.memory.DrillByteBufAllocator.buffer(DrillByteBufAllocator.java:50)
at org.apache.drill.exec.rpc.RpcEncoder.encode(safeRelease.java:87)
at org.apache.drill.exec.rpc.RpcEncoder.encode(RpcEncoder.java:38)
at
io.netty.handler.codec.MessageToMessageEncoder.write(MessageToMessageEncoder.java:90){code}
> Sender memory leak when rpc encode exception
> --------------------------------------------
>
> Key: DRILL-8489
> URL: https://issues.apache.org/jira/browse/DRILL-8489
> Project: Apache Drill
> Issue Type: Bug
> Components: Server
> Affects Versions: 1.21.1
> Reporter: shihuafeng
> Priority: Major
> Fix For: 1.22.0
>
>
> When encode throw Exception, if encode msg instanceof ReferenceCounted, netty
> can release msg, but drill convert msg to OutboundRpcMessage, so netty can
> not release msg. this causes sender memory leaks
> exception info
> {code:java}
> 2024-04-16 16:25:57,998 [DataClient-7] ERROR
> o.a.d.exec.rpc.RpcExceptionHandler - Exception in RPC communication.
> Connection: /10.32.112.138:47924 <--> /10.32.112.138:31012 (data client).
> Closing connection.
> io.netty.handler.codec.EncoderException:
> org.apache.drill.exec.exception.OutOfMemoryException: Unable to allocate
> buffer of size 4096 due to memory limit (9223372036854775807). Current
> allocation: 0
> at
> io.netty.handler.codec.MessageToMessageEncoder.write(MessageToMessageEncoder.java:107)
> at
> io.netty.channel.AbstractChannelHandlerContext.invokeWrite0(AbstractChannelHandlerContext.java:881)
> at
> io.netty.channel.AbstractChannelHandlerContext.invokeWriteAndFlush(AbstractChannelHandlerContext.java:940)
> at
> io.netty.channel.AbstractChannelHandlerContext$WriteTask.run(AbstractChannelHandlerContext.java:1247)
> at
> io.netty.util.concurrent.AbstractEventExecutor.runTask(AbstractEventExecutor.java:173)
> at
> io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:166)
> at
> io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:470)
> at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:569)
> at
> io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997)
> at
> io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: org.apache.drill.exec.exception.OutOfMemoryException: Unable to
> allocate buffer of size 4096 due to memory limit (9223372036854775807).
> Current allocation: 0
> at
> org.apache.drill.exec.memory.BaseAllocator.buffer(BaseAllocator.java:245)
> at
> org.apache.drill.exec.memory.BaseAllocator.buffer(BaseAllocator.java:220)
> at
> org.apache.drill.exec.memory.DrillByteBufAllocator.buffer(DrillByteBufAllocator.java:55)
> at
> org.apache.drill.exec.memory.DrillByteBufAllocator.buffer(DrillByteBufAllocator.java:50)
> at org.apache.drill.exec.rpc.RpcEncoder.encode(safeRelease.java:87)
> at org.apache.drill.exec.rpc.RpcEncoder.encode(RpcEncoder.java:38)
> at
> io.netty.handler.codec.MessageToMessageEncoder.write(MessageToMessageEncoder.java:90){code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
