xianjingfeng opened a new issue, #76: URL: https://github.com/apache/incubator-uniffle/issues/76
We found shuffle server which under high load is easy encounter `java.lang.OutOfMemoryError: Java heap space` even we allocate more jvm heap memory and less `rss.server.buffer.capacity ` The steps for the exception above: 1. When shuffle server under high load, `requireBufferId` is easy to expire, and Suffle server release `usedMemory` 2. Client `sendShuffleData` using a expired `requireBufferId`, 3. Suffle server recive shuffle data and store _in rpc queue_(this part of memory usage was not be added to `usedMemory`) 4. Other clients `requireBuffer` because `usedMemory` is enough -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
