zuston opened a new issue, #674: URL: https://github.com/apache/incubator-uniffle/issues/674
### Code of Conduct - [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) ### Search before asking - [X] I have searched in the [issues](https://github.com/apache/incubator-uniffle/issues?q=is%3Aissue) and found no similar issues. ### What would you like to be improved? In our production environment, the shuffle server is always stopped due to the full GC. After increasing more memory, it still occurs, that makes me confused. So I suspect some abusing memory exists in uniffle. After digging into the Grpc service api of `sendShuffleData`, I found something is abnormal. Please see this part code. https://github.com/apache/incubator-uniffle/blob/1fbdfe576f0a164f173cbff9ad83dccccb909ee2/server/src/main/java/org/apache/uniffle/server/ShuffleServerGrpcService.java#L751-L769 The byte[] will always be created in the invoking of `block.getData().toByteArray()`, which will cause GC if receiving data speed is high. And this has been proved in our dashboard Besides, the byte[] will be created when reading shuffle data in memory, which may be not the main factor to trigger this. Anyway, it also should be fixed. ### How should we improve? We could use the `block.getData().asReadOnlyByteBuffer()` to replace `toByteArray` to avoid useless memory allocation. ### Are you willing to submit PR? - [X] Yes I am willing to submit a PR! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
