Connection tracking in tcp mode

2011-06-09 Thread Andreas

Hi,

we are using haproxy only in tcp mode.

Actually we have the problem that we see broken requests on our backend 
server, but we cannot see which client group is causing the problem.


That's because we only see the IP and the source port of haproxy in our 
server logs and the client IP and source port in the haproxy log.


It would be cool to have the local source port of haproxy logged so that 
we can find the corresponding logfile entry from the haproxy logs.


Or does somebody have a better idea to find out the right connections?

Thanks for your help
Andreas



Re: Connection tracking in tcp mode

2011-06-09 Thread Willy Tarreau
Hi Andreas,

On Thu, Jun 09, 2011 at 03:55:46PM +0200, Andreas wrote:
 Hi,
 
 we are using haproxy only in tcp mode.
 
 Actually we have the problem that we see broken requests on our backend 
 server, but we cannot see which client group is causing the problem.
 
 That's because we only see the IP and the source port of haproxy in our 
 server logs and the client IP and source port in the haproxy log.
 
 It would be cool to have the local source port of haproxy logged so that 
 we can find the corresponding logfile entry from the haproxy logs.

I know what you're talking about, it happened to me to have the same issue
too. While I don't want to touch the HTTP log format until we can make it
totally flexible, I think that extending the TCP one is not a big deal.
We should have a backend option log-outgoing-address which when enabled
would perform the getsockname() on the server-side socket and add that in
the logs. It would tremendously help making the relation between both sides.
If you want to work on such a thing, I'm willing to merge it into 1.5.

 Or does somebody have a better idea to find out the right connections?

If you don't have too many new connections per second and if your servers
are NTP-synchronized, you can use the connection accept date which should
be the same on the server modulo 1 ms.

Alternatives involve working in transparent mode or implementing the PROXY
protocol in your servers and enabling it on haproxy. It will then pass all
the info there.

If your issue happens often enough and the load is not too high (less than
about 10% CPU), then you can run strace on haproxy (strace -tto log -p PID).
I'm often doing that to guess what FD is associated to what other FD, because
you can see contents and precise timing which let you match a connect() with
an accept(). It's not always usable but sometimes quite handy. Sometimes you
have to combine it with tcpdump running on the same system if you absolutely
need the source port of the outgoing connection.

Hoping this helps,
Willy