Hi Matthias, I do have some good news. I could reproduce your issue on a FreeBSD machine (thanks Olivier!). This issue isn't FreeBSD specific, it's a bug in haproxy. It just happens that FreeBSD *sometimes* manages to get connect() to immediately succeed over the loopback, that you're indeed connecting to a backend server on the loopack, that you're using send-proxy and that your front sslh sometimes doesn't deliver a request immediately (probably due to client's pre-connect feature).
In my tests I saw about 1 connection out of 19000 being immediately accepted but didn't see any problem. So I cheated by disabling non- blocking during the connect(), resulting in 100% of the connections being non-blocking :-) There's a race between the removal of the CO_FL_WAIT_L4_CONN flag on the connection, the sending of the proxy protocol line and the enabling of polling for sending pending data. If I send some data early or force "data=1" in tcp_connect_server(), it works, just like if I don't remove CO_FL_WAIT_L4_CONN or if I don't use send-proxy. All this to say that the issue is now under investigation even if not yet fully understood. I already suspect I'll discover something scary down the pipe, I hope it will be reasonably fixable :-) Thanks for all the elements you provided, that was a key in being able to go that close to the bug. I'll keep you updated. Cheers, Willy
