On Thu, Jan 19, 2017 at 8:06 AM, Jeff Darcy <jda...@redhat.com> wrote:
> > The more relevant question would be with TCP_KEEPALIVE and > TCP_USER_TIMEOUT > > on sockets, do we really need ping-pong framework in Clients? We might > need > > that in transport/rdma setups, but my question is concentrating on > > transport/rdma. In other words would like to hear why do we need > heart-beat > > mechanism in the first place. One scenario might be a healthy socket > level > > connection but an unhealthy brick/client (like a deadlocked one). > > This is an important case to consider. On the one hand, I think it answers > your question about TCP_KEEPALIVE. What we really care about is whether a > brick's request queue is moving. In other words, what's the time since the > last reply from that brick, and does that time exceed some threshold? On a > busy system, we don't even need ping packets to know that. We can just use > responses on other requests to set/reset that timer. We only need to send > ping packets when our *outbound* queue has remained empty for some fraction > of our timeout. > > However, it's important that our measurements be *end to end* and not just > at the transport level. This is particularly true with multiplexing, > where multiple bricks will share and contend on various resources. We > should ping *through* client and server, with separate translators above > and below each. This would give us a true end-to-end ping *for that > brick*, and also keep the code nicely modular. > +1 to this. Having ping, pong xlators immediately above and below protocol translators would also address the problem of epoll threads getting blocked in gluster's xlator stacks in busy systems. Having said that, I do see value in Rafi's patch that prompted this thread. Would it not help to prioritize ping - pong traffic in all parts of the gluster stack including the send queue on the client? Regards, Vijay
_______________________________________________ Gluster-devel mailing list Gluster-devel@gluster.org http://lists.gluster.org/mailman/listinfo/gluster-devel