On Thu, Oct 8, 2015 at 12:18 AM, Dmitry Sivachenko <[email protected]> wrote:
> Hello, > > I am using haproxy-1.5.14 and sometimes I see the following errors in the > log: > > Oct 7 08:33:03 srv1 haproxy[77565]: unix:1 [07/Oct/2015:08:33:02.428] > MT-front MT_RU_EN-back/<NOSRV> 0/1000/-1/-1/1000 503 212 - - sQ-- > 125/124/108/0/0 0/28 "POST /some/url HTTP/1.1" > (many similar at one moment) > > Common part in these errors is "1000" in Tw and Tt, and "sQ--" termination > state. > > Here is the relevant part on my config (I can post more if needed): > > defaults > balance roundrobin > maxconn 10000 > timeout queue 1s > fullconn 3000 > default-server inter 5s downinter 1s fastinter 500ms fall 3 rise 1 > slowstart 60s maxqueue 1 minconn 5 maxconn 150 > > backend MT_RU_EN-back > mode http > timeout server 30s > server mt1-34 mt1-34:19016 track MT-back/mt1-34 weight 38 > server mt1-35 mt1-35:19016 track MT-back/mt1-35 weight 38 > <total 18 of similar servers> > > So this error log indicates that request was sitting in the queue for > timeout queue==1s and his turn did not come. > > In the stats web interface for MT_RU_EN-back backend I see the following > numbers: > > Sessions: limit=3000, max=126 (for the whole backend) > Limit=150, max=5 or 6 (for each server) > > If I understand minconn/maxconn meaning right, each server should accept > up to min(150, 3000/18) connections > > So according to stats the load were far from limits. > > What can be the cause of such errors? > > Thanks! > The only thing I can think of is you have left net.core.somaxconn = 128, try increasing it to 4096 lets say to match your planned capacity of 3000
