On 15 Feb 2017 7:59 pm, "Juho Mäkinen" <j...@unity3d.com> wrote:
We have a setup which requires us to have two haproxy tiers so that first forwards connections to the second. What I want to know is the theory how (and why) I should tune my maxconn, backlog and timeout settings to handle queues overloads and back pressure in situations where my backends are overloaded. Think an array of virtual called A, which size is between 10-20. Then another array B which size is 100-200. There is a client in the A servers which want to connect to a random worker in array B. Each machine in B contains a haproxy which will have a single listen clause with backends which are pointed to workers in the same host. This means that all connections to a worker in a host in the array B will go through the haproxy in that host. Servers A have haproxies which have a listen clause so that each server in the B array will have one backend set. Clients in servers A will connect to localhost so they will reach the haproxy in machine A which will route the request to a suitable server B haproxy and where that haproxy will route it to a worker in that node. This works, but I'm not sure how I should tune my configurations that if any server in the B array gets overloaded then the haproxies in server array A would avoid this server? I'm thinking that I should use the "retries" setting in haproxy A so that if it can't connect to the firstly selected server B it would try another. If you have a single B array server in the A listener the retries will not help. But I'm not sure how I should configure haproxy B so that this is done? Sounds like you need more than one B array server per A listener so A can retry or load balance if you like different one in case the first one chosen is not responding. If I set both maxconn and backlog settings low enough in B will this cause this to happen and what is actually going in terms of SYN, SYN+ACK, kernel backlog queues and in haproxy frontend queues? I'm pretty sure I need to lab this out so that I can use wireshark to really look what is going on, but the lab setup is non-trivial and I could use some good theory how this should work. - Garo
