Hi Aleks, Patrick's solution is correct. What I was expecting was that HAProxy would take
X-Forwarded-For: 1.1.1.1 and produce X-Forwarded-For: 1.1.1.1,2.2.2.2 but what it actually does is produce X-Forwarded-For: 1.1.1.1 X-Forwarded-For: 2.2.2.2 The backend concatenates the lines and treats them like what I was expecting, but when I do a header capture in the second HAProxy it only reads the second line. And yes, my snippet is missing the header capture on the http frontend, thanks for spotting that. On Jul 23, 2017 03:50, "Aleksandar Lazic" <al-hapr...@none.at> wrote: Hi Patrick Hemmer, Patrick Hemmer wrote on 22.07.2017: > On 2017/7/22 11:11, Claus Strommer wrote: > > Hi all, I'm seeing some odd behaviour with our > haproxy balancer and am looking for some insights. > > The setup: > > I have a webserver that is behind two haproxy > balancers (version 1.5.18 on EL7), which are > behind CloudFlare. In effect the request goes > > > client->CF->haproxy1->haproxy2->server. > > On both haproxy balancers I have "option > forwardfor" and "capture request header > X-Forwarded-For len 128" set. > On the server I also capture X-Forwarded-For > > Now here is where the odd behaviour > (highlighted) happens: > > * haproxy1 logs the full X-Forwarded-For header. > * haproxy2 only logs the IP of the CF proxy (the last address in X-Forwarded-For) > * server logs the full X-Forwarded-For header. > * If I turn off "option forwardfor" on haproxy1, then > haproxy2 logs the full header as received by CF. > * Changing the length of the capture request does not > seem to make a difference. > > * I noticed that haproxy uses spaces after the comma > between the header entries, but CF does not. I tried > replicating this issue with a direct curl request to haproxy2 > replicating the x-forwarded-for header that haproxy1 would > have sent, and I cannot reproduce the issue. > > The only thing that I notice is that CF > > > Am I missing something obvious here? Below are the full > options I'm using on haproxy1 and haproxy2. Everything after that is ACLs > > defaults > mode http > log global > option httplog > option dontlognull > option http-server-close > option forwardfor except 127.0.0.0/8 > option redispatch > retries 3 > > frontend http *:80 > mode http > reqadd X-Forwarded-Proto:\ https I miss here the capture request header X-Forwarded-For len 15 > redirect scheme https code 301 > > frontend https > bind *:443 ssl crt /etc/pki/tls/certs/hacert.pem > mode http > capture request header Host len 50 I miss here the capture request header X-Forwarded-For len 15 Can you try to run haproxy for a short time in debug mode? http://cbonte.github.io/haproxy-dconv/1.7/management.html#3 -d > The "option forwardfor" setting appends a complete new header, > not appends the value to an existing header. From the docs on "option forwardfor": > > Enable insertion of the X-Forwarded-For header to requests sent to servers > ... > > this header is always appended at the end of the existing header list > > Your header capture is grabbing the last X-Forwarded-For header. > On issues like this, you should perform a packet capture. It > would make the issue immediately apparent. > > Personally I use 2 rules similar to the following to append to X-Forwarded-For: > > http-request set-header X-Forwarded-For > %[req.fhdr(X-Forwarded-For)],\ %[src] if { req.fhdr(X-Forwarded-For) -m found } > http-request set-header X-Forwarded-For %[src] if !{ > req.fhdr(X-Forwarded-For) -m found } > > -Patrick But doesn't haproxy do this already? http://git.haproxy.org/?p=haproxy-1.7.git;a=blob;f=src/proto_http.c;h= 94c8d639f6f777241109f605e1e1742f9a39bf33;hb=HEAD#l4639 -- Best Regards Aleks