spacemonkd opened a new pull request, #1540: URL: https://github.com/apache/ratis/pull/1540
## What changes were proposed in this pull request? RATIS-2632. Apply backpressure on install-snapshot chunk loop When the leader streams a snapshot to a lagging follower, the chunk send loop in [GrpcLogAppender.installSnapshot()](https://github.com/apache/ratis/blob/a1bbf47f98a83d3bbddb4357c8a9b96bcfd7c727/ratis-grpc/src/main/java/org/apache/ratis/grpc/server/GrpcLogAppender.java#L775-L783) calls `snapshotRequestObserver.onNext(request)` for every chunk without ever checking `CallStreamObserver.isReady()`. The append-entries path already [checks isReady()](https://github.com/apache/ratis/blob/a1bbf47f98a83d3bbddb4357c8a9b96bcfd7c727/ratis-grpc/src/main/java/org/apache/ratis/grpc/server/GrpcLogAppender.java#L366), but the snapshot path does not, so it is possible that chunks are enqueued faster than the network can drain them. While on the default path `INSTALL_SNAPSHOT_REQUEST_ELEMENT_LIMIT_DEFAULT` is set to 8 so `StreamObserverWithTimeout` applies a request-response window, but since we allow setting the limit to 0, the semaphore can be `null` and cause: - Unbounded outbound buffering of snapshot chunks in the gRPC/Netty write queue when the follower is slow. - Leader OOM / direct-memory exhaustion during large-snapshot catch-up, which can crash the leader and impact the whole group. This is handled by having an explicit isReady() check, which causes minimal overhead on the default path (`element-limit=8`) but handles the case when it might be set to 0. ## What is the link to the Apache JIRA https://issues.apache.org/jira/browse/RATIS-2632 ## How was this patch tested? Patch was tested via unit tests. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
