Re: Possible bug - request forwarding to wrong backend

[Vivek Malik]

Hi Cyril,

Thanks for the quick response. I tried adding option foreclose and option
http-pretend-keepalive and it seems to fix the situation.

I learned a few things about http keep-alive today ...

Thanks :)

Vivek

On Mon, Apr 4, 2011 at 2:33 AM, Cyril Bonté  wrote:

>  Hi Vivek,
>
>
> Le lundi 04 avril 2011 11:21:55, Vivek Malik a écrit :
>
> > Hi,
>
> >
>
> > I have been struggling with this bug for a little while and wanted to
> report
>
> > it and see if you can either help me or fix this bug. Here is a sample
>
> > config file which shows the bug
>
> < (...)
>
> > Basically, the config is just distributing http connections between 2
>
> > servers based on the path. However, I am constantly seeing haproxy
> sending
>
> > traffic to the wrong backend. The backend back_b in the example above is
>
> > responsible for all pages except a few like /login, /logout, /admin,
> /secure
>
> > etc.
>
> >
>
> > The bug can be reproduced by
>
> > a) User goes to home page. Haproxy forwards the request to back_b.
>
> > b) User clicks the login link (/login which is supposed to be sent to
>
> > back_a).
>
> > c) User receives response from back_b instead of back_a
>
> >
>
> > If the user clicks on a link quickly the request ends up going to back_b
>
> > instead of back_a as expected based on ACL. Moreover, haproxy doesn't log
>
> > the request and its routing to back_b. Whenever haproxy logs the request,
> it
>
> > is sending to the right backend. The wrong backend logs do show them
>
> > receiving requests from ip running haproxy. So, it has to be haproxy
>
> > forwarding those requests.
>
>
> Your sample config file uses tunnel mode, which means it can only analyze
> the first request of a connection. Everything after the headers of this
> first request is considered as data.
>
> The issue you encounter is probably due to HTTP KeepAlive.
>
>
> You should try to add "option httpclose", or better "option
> http-server-close" (which enables client HTTP keepalive and allows to
> analyze each request of a connection).
>
> With "option http-server-close", you can also have a look at "timeout
> http-keep-alive" and "option http-pretend-keepalive".
>
>
> --
>
> Cyril Bonté
>

Re: Possible bug - request forwarding to wrong backend

[Cyril Bonté]

Hi Vivek,

Le lundi 04 avril 2011 11:21:55, Vivek Malik a écrit :
> Hi,
> 
> I have been struggling with this bug for a little while and wanted to report
> it and see if you can either help me or fix this bug. Here is a sample
> config file which shows the bug
< (...)
> Basically, the config is just distributing http connections between 2
> servers based on the path. However, I am constantly seeing haproxy sending
> traffic to the wrong backend. The backend back_b in the example above is
> responsible for all pages except a few like /login, /logout, /admin, /secure
> etc.
> 
> The bug can be reproduced by
> a) User goes to home page. Haproxy forwards the request to back_b.
> b) User clicks the login link (/login which is supposed to be sent to
> back_a).
> c) User receives response from back_b instead of back_a
> 
> If the user clicks on a link quickly the request ends up going to back_b
> instead of back_a as expected based on ACL. Moreover, haproxy doesn't log
> the request and its routing to back_b. Whenever haproxy logs the request, it
> is sending to the right backend. The wrong backend logs do show them
> receiving requests from ip running haproxy. So, it has to be haproxy
> forwarding those requests.

Your sample config file uses tunnel mode, which means it can only analyze the 
first request of a connection. Everything after the headers of this first 
request is considered as data.
The issue you encounter is probably due to HTTP KeepAlive.

You should try to add "option httpclose", or better "option http-server-close" 
(which enables client HTTP keepalive and allows to analyze each request of a 
connection).
With "option http-server-close", you can also have a look at "timeout 
http-keep-alive" and "option http-pretend-keepalive".

-- 
Cyril Bonté

Possible bug - request forwarding to wrong backend

[Vivek Malik]

Hi,

I have been struggling with this bug for a little while and wanted to report
it and see if you can either help me or fix this bug. Here is a sample
config file which shows the bug

*global*
*log 127.0.0.1 local0*
*user nobody*
*group nogroup*
*daemon*
*pidfile /var/run/haproxy.pid*
*
*
*defaults*
*log global*
*mode http*
*option httplog*
*
*
*timeout client 10s*
*timeout connect 5s*
*timeout http-request 10s*
*timeout server 60s*
*
*
*frontend http*
*bind 0.0.0.0:80 *
* acl path_group_a path_beg /login /logout /admin /secure /js/ /css/*
* *
* user_backend back_a if path_group_a*
* default_backend back_b*
* *
*backend back_a*
*server a1 10.20.203.165:80 *
*backend back_b*
*server b1 10.20.122.68:80 *
*
*
*
*
Basically, the config is just distributing http connections between 2
servers based on the path. However, I am constantly seeing haproxy sending
traffic to the wrong backend. The backend back_b in the example above is
responsible for all pages except a few like /login, /logout, /admin, /secure
etc.

The bug can be reproduced by
a) User goes to home page. Haproxy forwards the request to back_b.
b) User clicks the login link (/login which is supposed to be sent to
back_a).
c) User receives response from back_b instead of back_a

If the user clicks on a link quickly the request ends up going to back_b
instead of back_a as expected based on ACL. Moreover, haproxy doesn't log
the request and its routing to back_b. Whenever haproxy logs the request, it
is sending to the right backend. The wrong backend logs do show them
receiving requests from ip running haproxy. So, it has to be haproxy
forwarding those requests.

I have tried running haproxy in debug mode too but am unable to see the
requests there too. Somehow the request is getting routed to wrong backend
and there is no log generated for that bug in info mode or debug mode.

Please help ! I am trying to set up a production infrastructure and solving
this problem would be a great help.

Thanks,
Vivek
310-754-5861