deepthi912 opened a new pull request, #15335:
URL: https://github.com/apache/pinot/pull/15335
**Context:** Some expensive queries from broker caused servers to go into
OOM state causing the consuming segments to stop consumption and we had to
increase the direct max memory on the servers and do a manual restart inorder
to mitigate the issue. The channels on the Request handler from broker seems to
be taking up most of the space for large queries.
```
ERROR [InstanceRequestHandler] [nioEventLoopGroup-3-20] Unhandled Exception
in org.apache.pinot.core.transport.InstanceRequestHandler
java.lang.OutOfMemoryError: Cannot reserve 4194304 bytes of direct buffer
memory (allocated: 5364982965, limit: 5368709120)
at java.base/java.nio.Bits.reserveMemory(Bits.java:178) ~[?:?]
at
java.base/java.nio.DirectByteBuffer.<init>(DirectByteBuffer.java:111) ~[?:?]
at java.base/java.nio.ByteBuffer.allocateDirect(ByteBuffer.java:360)
~[?:?]
at
```
Closing these active channels when OOM is caused because of all the buffer
allocations from broker, releasing the channel and freeing up the space.
Reference for broker OOM fix: https://github.com/apache/pinot/pull/11496
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
