Hi Terje, On Tue, Dec 04, 2012 at 12:58:17PM +0100, Borgen, Terje wrote: > Hi, > We are using haproxy in an main/failover scenario in front of Jetty-servers. > With approximately every fifth config-reload all the requests gets 503 error > in the next minute. > It seems very much like the issue described here: > http://marc.info/?t=121146026200009&r=1&w=2 > > Some info: > RHEL 5.3 > Haproxy 1.4.22 > The traffic is approximately 30 request/second. > There is a Apache HTTP Server in front of Haproxy > > Attached is the configuration and haproxy-log for normal reload and reload > with hang. > Can You help me indentify what???s wrong?
I'm having an idea right now. If you look below, the pause happens between the old and the new process : Nov 30 11:00:13 alp-stb-004 haproxy[29722]: 127.0.0.1:48843 [30/Nov/2012:10:59:44.396] client skade_p/skade_p_main 0/0/0/29383/29387 302 408 - - --VN 0/0/0/0/0 0/0 "POST /skade/public/storage/submitRetrieveShoppingCart.action HTTP/1.1" Nov 30 11:01:01 alp-stb-004 haproxy[32609]: 127.0.0.1:51371 [30/Nov/2012:11:01:01.365] client skade_p/skade_p_main 0/0/0/47/47 200 12828 - - --VN 0/0/0/0/0 0/0 "GET /skade/partner HTTP/1.1" One thing that the new process does is to try to bind the listeners, and if any fails, it asks the old process to release the port so that it can try again. Since you're running on a non-privileged port (8088), what could be possible is that during the reload, when the port is remporarily released, it's used as a source port by the old process, preventing any of them from binding to it. Could you please check /proc/sys/net/ipv4/ip_local_port_range ? If it spans further than the default 32768-61000, please ensure the range does not cover 8088 and try again. If it solves the issue, maybe we should document that in the doc. Regards, Willy
