[GitHub] [incubator-uniffle] zuston opened a new issue, #230: Possible memory leak in shuffle-server

[GitBox]

zuston opened a new issue, #230:
URL: https://github.com/apache/incubator-uniffle/issues/230

   ### Background
   
   I found the metric size of `res`  is too large when using the command of 
`top` to inspect the shuffle-server process, which has been excluded in the 
cluster and no app runs on it.
   
   When I dump the jvm heap, I found the heap size is normal. But it has too 
many netty `PooledUnsafeDirectByteBuf` objects, it this normal?
   
   
![image](https://user-images.githubusercontent.com/8609142/191160103-1ee4719d-db81-410d-838f-97b40075ffcf.png)
   
   
   ### Shuffle-server startup Environment
   XMX=30g
   top RES = 54g
   
   
![image](https://user-images.githubusercontent.com/8609142/191160294-9ab64262-b63c-4648-a737-de57002fa3c6.png)
   
   
   We have made the shuffle-servers colocate with the Yarn nodemangers. So we 
hope the used memory of shuffle-server could be limited by using the JVM 
options `-XX:MaxDirectMemorySize` or other netty config.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscr...@uniffle.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org