Re: bug? reqidel and pcre
Hi Willy and Hank, Le jeudi 25 novembre 2010 07:10:20, Willy Tarreau a écrit : > Cyril, Hank is right, there's a bug. Here's how I can reproduce it with > both 1.4.9 and 1.5-dev3: > > Config : > listen echo > bind:8000 > modehttp > option http-server-close > timeout client 50s > timeout server 50s > timeout connect 50s > server srv 127.0.0.1:8001 > reqidel ^X > reqidel ^Via: > > Request sent: > $ printf "GET / HTTP/1.0\r\nX-truc1: blah\r\nX-truc2: blah\r\nVia: > here\r\nX-truc3: blah\r\nVia: there\r\nNon-X: blah\r\n\r\n" | nc 0 8000 > > Request received: > $ nc -lp8001 > GET / HTTP/1.0 > Via: here > Via: there > Non-X: blah Oh I see, I didn't understand the issue like this. > If I switch the two reqidel lines, I don't have the problem anymore. But if you add a second Via header before X-truc3, this X-truc3 won't be removed. > If I remove only one of the first X-truc headers, it's OK too. I suspect > that the removal of more than one headers from the beginning prevents > the remaining first header from being matched by next rule :-/ Not only the first one. With this order : reqidel ^Via: reqidel ^X and this request : printf "GET / HTTP/1.0\r\nX-truc1: blah\r\nX-truc2: blah\r\nVia: here\r\nVia: here\r\nX-truc3: blah\r\nX-truc4: blah\r\nX-truc5: blah\r\nX-truc6: blah\r\nVia: there\r\nNon-X: blah\r\n\r\n" X-truc3, X-truc4, X-truc5, X-truc6 are not removed. Interesting, I'll have to check some servers configurations today. > I'll have to debug that ! > > Cheers, > Willy -- Cyril Bonté
host specific error messages
Hello guys, Is it possible to set host specific error files in the config? That is to server /etc/error/503.html for www.this.com but /etc/error/503_alter.html for www.that.com ? If so how? :) Both hosts are served by the same listen directive atm. Was looking at acls and hdr_beg(host) -i, but how could I make use of that inside listen? Thanks, Joe
Re: IIS7 with IIRF, 403 - Forbidden: Access is denied after HAproxy
On Tue, Nov 23, 2010 at 07:18:11PM +0800, Hanky Cheng(SingTao) wrote: > Hi All > > I have a question about HAproxy with IIS7 & IIRF. > > As before i running website on IIS7 & IIRF for URLrewrite. Before 1 mth i > using HAproxy for loading balance (backend is same server but difference > port). > > After that website show "403 - Forbidden: Access is denied" randomly. Did > anyone to face this problem? Could you please send a little bit more info ? Config, where you observe this error (haproxy or IIS), the haproxy log line showing the error, etc ? Right now it's too vague to even be investigated :-( Regards, Willy
Re: option httpchk and SSL
On Wed, Nov 24, 2010 at 07:24:10PM -0500, John Marrett wrote: > Arthur, > > I believe that you will need to use stunnel in client mode to initiate > the ssl connections to the back end. You'll set up multiple instances, > presumably on the machine running haproxy, each confired to point to a > specific backend. > > httpchk will not do https. Confirmed. Also, maybe it would be fine for you to send your check to nginx on port 80 in clear text ? If so, just add "port 80" after "check" on your server lines. It will not validate the SSL part but will at least validate that the whole chain is running. And the SSL part is not much likely to fail without the rest anyway. Regards, Willy
Re: bug? reqidel and pcre
Hi guys, On Wed, Nov 24, 2010 at 08:30:38PM +0100, Cyril Bonté wrote: > Hi Hank, > > Le mercredi 24 novembre 2010 19:55:32, Hank A. Paulson a écrit : > > I have a possible bug, I have a backend I want strip all the X-* headers > > off the requests. But I found that if I did: > > > > reqidel ^X > > reqidel ^Via:\ > > > > or > > > > reqdel ^x-.*:\ > > reqdel ^Via > > > > or similar > > > > haproxy [1.4.8 (Fedora package version) and hand compiled 1.4.9 version > > both using pcre] both would not remove the "Via:" header (the Via header > > was the first header after Host, so initially I thought it was header > > order. > > > > I also had some other reqidel's but I removed all of them except ^X and > > ^Via and the problem remained. > > > > I tried many combinations of reqidel/reqdel and removed all the other > > reqdel/rspadd lines during testing and I still see the same results: > > X- or x- headers are removed but the Via header is not. > > > > The only other thing is that I am capturing an x- header, but why not an > > explicit error message if that is the cause of the problem. > > You should provide your configuration file and an example of the buggy > request > (without sensitive data like IP addresses) because it's not clear for me, and > it will be easier to reproduce the issue if there's one ;-) Cyril, Hank is right, there's a bug. Here's how I can reproduce it with both 1.4.9 and 1.5-dev3: Config : listen echo bind:8000 modehttp option http-server-close timeout client 50s timeout server 50s timeout connect 50s server srv 127.0.0.1:8001 reqidel ^X reqidel ^Via: Request sent: $ printf "GET / HTTP/1.0\r\nX-truc1: blah\r\nX-truc2: blah\r\nVia: here\r\nX-truc3: blah\r\nVia: there\r\nNon-X: blah\r\n\r\n" | nc 0 8000 Request received: $ nc -lp8001 GET / HTTP/1.0 Via: here Via: there Non-X: blah If I switch the two reqidel lines, I don't have the problem anymore. If I remove only one of the first X-truc headers, it's OK too. I suspect that the removal of more than one headers from the beginning prevents the remaining first header from being matched by next rule :-/ I'll have to debug that ! Cheers, Willy
Re: Session Definition
Hi Richard, On Wed, Nov 24, 2010 at 04:17:31PM +, Richard Shaw wrote: > > Hi, > > Could someone please give me the correct definition of a session under > HAProxy, i've confused myself! A session is the entity instanciated when an incoming connection is accepted. Until recently, it could only be a TCP/TCPv6 session, but now it can also be instanciated from a UNIX socket. It lives until that connection is closed. The session owns the request and response buffers, and knows what server it is talking to, if any. In HTTP, it is possible to process multiple requests in a single session when keep-alive is enabled. Depending who I'm talking to, I sometimes prefer to use the term "connection" because application people tend to think about application sessions which are completely different things. In haproxy, a session must be understood as a layer 4 session. Regards, Willy
Re: [SOLVED] Re: appsession does not work in Haproxy 1.4.9
Hi Willy, you are awesome! I am trying it now and give you feedback. Thanks! Hogan On Thu, Nov 25, 2010 at 1:46 AM, Willy Tarreau wrote: > Hi Hogan, > > On Wed, Nov 24, 2010 at 09:33:37PM +0800, Hogan Yu wrote: > > Hi Willy, > > I send you an dump core file and I sure that we have some free memory on > > our platform. > > Thank you very much for your core. I could find the issue and indeed it's > not related to a memory shortage, which was the issue I first encountered > when trying your config. > > The issue comes from the mixing of cookies in indirect mode and appsession. > The bug is that the indirect cookie is removed but 3 pointers that point to > the attribute of the next cookies are not updated after the removal, and > they are used by the appsession code. One length computation can get wrong > and cause a memcpy() to be called with a negative value, thus crashing the > process. > > I have issued a critical fix for this, I'd like you to give it a try : > > http://git.1wt.eu/web?p=haproxy-1.4.git;a=commitdiff_plain;h=c5f374 > > In my tests the problem is correctly solved. Once you confirm the issue > is gone, I'll release 1.4.10 with it. > > In the mean time, if there are any user mixing cookies + appsession, I > strongly encourage them to disable the "indirect" keyword on the cookie > line, or to disable either of the two mechanisms until they upgrade to > 1.4.10. > > I know that 1.5-dev3 is affected too, though it's a development version > so it's less critical. I don't know for older 1.4 versions nor for 1.3. > > Cheers, > Willy > > -- Hogan Yu Technical Operations Director Ice BreakerSoftware (Beijing) Lmt. Mobile: 18611746815 Tel：86-10-82800259 82800942 Fax：86-10-82800941
Re: option httpchk and SSL
Arthur, I believe that you will need to use stunnel in client mode to initiate the ssl connections to the back end. You'll set up multiple instances, presumably on the machine running haproxy, each confired to point to a specific backend. httpchk will not do https. -JohnF
option httpchk and SSL
Hi, I am running haproxy 1.3.2 as a load balancer for some servers running nginx as an ssl terminator which forwards to squid running as a reverse proxy which falls back to apache. I want to use 'option httpchk' to check whether or not a specific resource is available to our users - the idea being that the health-check will actually detect if any particular piece of the webserver is down (nginx, squid or apache). However, I cannot seem to get it working properly. I've tried a variety of things along the lines of: option httpchk GET /index.php HTTP/1.1\r\nHost: foo.bar.org server foo1.private.net 10.0.0.1:443 check inter 2000 rise 5 fall 2 but nothing appears to work - all the requests show up as 400 errors in the nginx logs. My hunch is that this is because the httpchk is not even attempting to negotiate the SSL handshake, but perhaps I am just doing something wrong. Is there any way to get this to work? If not, is there any other httpchk-like thing for https? I know of ssl-hello-chk, but I want to know if I can actually pull a resource from a webserver, not just if my HTTPS service is alive. Thanks very much, Arthur Richards
Re: [PATCH] bind non local ip on FreeBSD
2010/11/24 Willy Tarreau : > On Tue, Nov 23, 2010 at 05:28:26PM +0300, Alexandre Snarskii wrote: >> On Mon, Nov 22, 2010 at 02:03:42PM +0100, joris dedieu wrote: >> > Hi list, >> > FreeBSD (and maybe other BSD) use IP_BINDANY flag to permite bind() to >> > bind a non local ip >> >> Please note that this flag is available only since FreeBSD 8, >> so your patch will break haproxy builds on older versions, like >> FreeBSD 7.* or 6.*. > > In my opinion, we should not have a build option for this, we should > just rely on the definition of the IP_BINDANY macro. And the config > option must always be parsed, then we should report "unsupported on > this build" if IP_BINDANY is not defined, so that users don't waste > hours trying to figure why their keyword is rejected. I tried to do something this way. I included openbsd's SO_BINDANY and the old IP_NONLOCALOK (see http://lists.freebsd.org/pipermail/svn-src-all/2009-June/009805.html ). > > I too am for a "non-local" bind option if this is required. I was wondering, > isn't there a system-global option to allow non-local binding as on Linux ? No there is not. There were a kernel build option which was removed. But in any case you need a userland explicit decision. > I still fail to see why doing so requires special privileges, the user is > not adding nor removing any IP from the system ! Well I investigate this point. In fact there is no root privilege needed but the PRIV_NETINET_BINDANY privilege. So as far as has I know It could be configured with the mandatory access control (mac). Regards Joris > > Regards, > Willy > > haproxy-non-local.patch Description: Binary data
Re: bug? reqidel and pcre
Hi Hank, Le mercredi 24 novembre 2010 19:55:32, Hank A. Paulson a écrit : > I have a possible bug, I have a backend I want strip all the X-* headers > off the requests. But I found that if I did: > > reqidel ^X > reqidel ^Via:\ > > or > > reqdel ^x-.*:\ > reqdel ^Via > > or similar > > haproxy [1.4.8 (Fedora package version) and hand compiled 1.4.9 version > both using pcre] both would not remove the "Via:" header (the Via header > was the first header after Host, so initially I thought it was header > order. > > I also had some other reqidel's but I removed all of them except ^X and > ^Via and the problem remained. > > I tried many combinations of reqidel/reqdel and removed all the other > reqdel/rspadd lines during testing and I still see the same results: > X- or x- headers are removed but the Via header is not. > > The only other thing is that I am capturing an x- header, but why not an > explicit error message if that is the cause of the problem. You should provide your configuration file and an example of the buggy request (without sensitive data like IP addresses) because it's not clear for me, and it will be easier to reproduce the issue if there's one ;-) -- Cyril Bonté
bug? reqidel and pcre
I have a possible bug, I have a backend I want strip all the X-* headers off the requests. But I found that if I did: reqidel ^X reqidel ^Via:\ or reqdel ^x-.*:\ reqdel ^Via or similar haproxy [1.4.8 (Fedora package version) and hand compiled 1.4.9 version both using pcre] both would not remove the "Via:" header (the Via header was the first header after Host, so initially I thought it was header order. I also had some other reqidel's but I removed all of them except ^X and ^Via and the problem remained. I tried many combinations of reqidel/reqdel and removed all the other reqdel/rspadd lines during testing and I still see the same results: X- or x- headers are removed but the Via header is not. The only other thing is that I am capturing an x- header, but why not an explicit error message if that is the cause of the problem.
[SOLVED] Re: appsession does not work in Haproxy 1.4.9
Hi Hogan, On Wed, Nov 24, 2010 at 09:33:37PM +0800, Hogan Yu wrote: > Hi Willy, > I send you an dump core file and I sure that we have some free memory on > our platform. Thank you very much for your core. I could find the issue and indeed it's not related to a memory shortage, which was the issue I first encountered when trying your config. The issue comes from the mixing of cookies in indirect mode and appsession. The bug is that the indirect cookie is removed but 3 pointers that point to the attribute of the next cookies are not updated after the removal, and they are used by the appsession code. One length computation can get wrong and cause a memcpy() to be called with a negative value, thus crashing the process. I have issued a critical fix for this, I'd like you to give it a try : http://git.1wt.eu/web?p=haproxy-1.4.git;a=commitdiff_plain;h=c5f374 In my tests the problem is correctly solved. Once you confirm the issue is gone, I'll release 1.4.10 with it. In the mean time, if there are any user mixing cookies + appsession, I strongly encourage them to disable the "indirect" keyword on the cookie line, or to disable either of the two mechanisms until they upgrade to 1.4.10. I know that 1.5-dev3 is affected too, though it's a development version so it's less critical. I don't know for older 1.4 versions nor for 1.3. Cheers, Willy
Session Definition
Hi, Could someone please give me the correct definition of a session under HAProxy, i've confused myself! Thanks Richard
Re: Meta backend ?
> Currently there is no "meta backend" functionality, so you need to duplicate > your config manually or to write a script that generates it in a desired > way. Regarding your concerns related to health checks performance, you can > and even should use "track backend/server" option: > > http://haproxy.1wt.eu/git?p=haproxy.git;a=commitdiff;h=c8b16fc94885adbf7b0146b847ce50530eab9e40 > > It allows you to enable health checks only once for each server and > configure other servers to follow state of a checked one. > Hi Krzysztof, Thanks a lot for your answer... "track" is exactly what I need :-) Cheers, Bartek
Re: Meta backend ?
On 2010-11-18 16:57, Bartłomiej Jarocki wrote: Hi, Hello Bartłomiej, I'm using haproxy for few years on quite big environments so I'd like to say 'BIG THANKS' before asking a question :-) simple scenario: backend www1 [ few hundreds of servers ] backend www2 [ few hundreds of servers ] backend www3 [ few hundreds of servers ] backend www4 [ few hundreds of servers ] frontend http bind :80 acl host-www1 hdr(host) -i www1 acl host-www2 hdr(host) -i www2 acl host-www3 hdr(host) -i www3 acl host-www4 hdr(host) -i www4 use_backend www1 if host-www1 use_backend www2 if host-www2 use_backend www3 if host-www3 use_backend www4 if host-www4 Is there any way to create _meta_ backend over www1+www2+www3+www4? I need to push traffic to all servers (from all backends) in case of one specific acl case. Is it necessary to create 'backend all' with list of all servers? What about amount of health checks if server is present in _many_ backends ? I saw no one responded to your question, so here I come. ;) Currently there is no "meta backend" functionality, so you need to duplicate your config manually or to write a script that generates it in a desired way. Regarding your concerns related to health checks performance, you can and even should use "track backend/server" option: http://haproxy.1wt.eu/git?p=haproxy.git;a=commitdiff;h=c8b16fc94885adbf7b0146b847ce50530eab9e40 It allows you to enable health checks only once for each server and configure other servers to follow state of a checked one. HTH. Best regards, Krzysztof Olędzki
Re: appsession does not work in Haproxy 1.4.9
Hi Willy, I send you an dump core file and I sure that we have some free memory on our platform. # free -m total used free sharedbuffers cached Mem: 16046 15919127 0365 1729 -/+ buffers/cache: 13824 Swap: 2047 40 2007 If you need more dump core files ,I can send you another one. pls help to check this problem out. thanks! Best Regards, Hogan On Wed, Nov 24, 2010 at 5:31 PM, Willy Tarreau wrote: > Hi Hogan, > > On Wed, Nov 24, 2010 at 03:32:14PM +0800, Hogan Yu wrote: > > Hi Willy, > > > > Sorry for give your feedback late. I test it several days and it does > not > > work with the new two patchs. It hang two times one day. > > It still get the errors as follows, > > haproxy[8728]: segfault at 1e90f000 rip 003ef9c7c51a rsp > > 7fff80e29208 error 6 > > haproxy[18325]: segfault at 1e397000 rip 003ef9c7c373 rsp > > 7fff071afbd8 error 4 > > haproxy[19660]: segfault at 1067a000 rip 003ef9c7c506 rsp > > 70703288 error 6 > > haproxy[19955]: segfault at 06a2e000 rip 003ef9c7c3df rsp > > 7fff15517608 error 4 > > haproxy[20180]: segfault at 0958b000 rip 003ef9c7c4c8 rsp > > 7fff64b3ecb8 error 6 > > > > > > do you think it maybe caused by enable the cookie and appsession > together? > > I don't see any reason why this could happen. Do you have a core file this > time ? It would help a lot ! > > >cookie HTTPSERVERID insert nocache indirect > >appsession JSESSIONID len 34 timeout 1h > > should I disable the cookie and just enable the appsession to test it? > > You could very well try to disable one or the other in turn. > > > If it is cause by memory shortage, do you know how to disable the > memory > > limitation? > > Your ulimits don't report any memory limitation. You then just have > to check if you still have some free memory on the system (just run > the "free" command). > > Otherwise it's possible that we're on a very different bug. > > Thanks! > Willy > > -- Hogan Yu Technical Operations Director Ice BreakerSoftware (Beijing) Lmt. Mobile: 18611746815 Tel：86-10-82800259 82800942 Fax：86-10-82800941
Re: appsession does not work in Haproxy 1.4.9
Hi Hogan, On Wed, Nov 24, 2010 at 03:32:14PM +0800, Hogan Yu wrote: > Hi Willy, > > Sorry for give your feedback late. I test it several days and it does not > work with the new two patchs. It hang two times one day. > It still get the errors as follows, > haproxy[8728]: segfault at 1e90f000 rip 003ef9c7c51a rsp > 7fff80e29208 error 6 > haproxy[18325]: segfault at 1e397000 rip 003ef9c7c373 rsp > 7fff071afbd8 error 4 > haproxy[19660]: segfault at 1067a000 rip 003ef9c7c506 rsp > 70703288 error 6 > haproxy[19955]: segfault at 06a2e000 rip 003ef9c7c3df rsp > 7fff15517608 error 4 > haproxy[20180]: segfault at 0958b000 rip 003ef9c7c4c8 rsp > 7fff64b3ecb8 error 6 > > > do you think it maybe caused by enable the cookie and appsession together? I don't see any reason why this could happen. Do you have a core file this time ? It would help a lot ! >cookie HTTPSERVERID insert nocache indirect >appsession JSESSIONID len 34 timeout 1h > should I disable the cookie and just enable the appsession to test it? You could very well try to disable one or the other in turn. > If it is cause by memory shortage, do you know how to disable the memory > limitation? Your ulimits don't report any memory limitation. You then just have to check if you still have some free memory on the system (just run the "free" command). Otherwise it's possible that we're on a very different bug. Thanks! Willy
