Re: Multiple Load Balancers, stick table and url-embedded session support
Hi Jeffrey, On 12/09/2010 10:13 PM, Jeffrey 'jf' Lim wrote: On Thu, Dec 9, 2010 at 7:27 PM, Hank A. Paulson mailto:h...@spamproof.nospammail.net>> wrote: Please see the thread: "need help figuring out a sticking method" I asked about this, Willie says there are issues figuring out a workable config syntax for 'regex to pull the URL/URI substring' but (I think) that coding the functionality is not technically super-difficult just not enough hands maybe and the config syntax? Actually if the "key" is to taken from a query param, that is relatively easy enough (I coded something myself for a client some time back based on 1.3.15.4). Would that be something you could and would be willing to share ? regards, David
Re: Multiple Load Balancers, stick table and url-embedded session support
On 12/09/2010 08:27 PM, Hank A. Paulson wrote: Please see the thread: "need help figuring out a sticking method" I asked about this, Willie says there are issues figuring out a workable config syntax for 'regex to pull the URL/URI substring' but (I think) that coding the functionality is not technically super-difficult just not enough hands maybe and the config syntax? I have a feeling this would be a fairly commonly used feature, so it is good to see others asking the same question :) My use-case is pretty simple, since it was fullfilled without stick table using the appsession configuration parameter, e.g. appsession MYSESSIONID len 64 timeout 1h request-learn mode query-string IOW, I only need simple string matching, and that maybe be an acceptable first step ? How long do you think it would take ? I am relatively familiar with C, but have no knowledge of network programming nor I have ever looked into haproxy source code. How are you planning to distribute the traffic to the different haproxy instances? LVS? Some hardware? I don't know exactly - I think using somekind of DNS round robin scheme between our set of load balancers (we have something similar a the moment with pound as load balancer, but without any kind of sticky session support). We are required not to use hardware. cheers, David
Re: haproxy 1.4.10 with option http-pretent-keepalive is become very slow
Hi, Le jeudi 9 d??cembre 2010 07:40:20, ?? a ??crit : > My english is very poor. If nginx proxy server is front of haproxy > server with option http-pretent-keeplive , the requests become very > slow. But is normal of direct request to haproxy server . > please help me. > Changrong.zhou Can you provide your nginx and haproxy configuration (or part of them). Recently I had some issues like this with nginx as a proxy (with or whithout haproxy). I could reproduce it with : - ssl on - several proxy_set_header - proxy_buffering on as if the last buffer data were not sent until the http keep alive timeout ! As soon as I removed one of this condition, it was OK. I didn't have time to make more tests. -- Cyril Bont??
Re: Adding/Removing servers for a backend without restarting haproxy
On Thu, 2010-12-09 at 14:28 +0100, Willy Tarreau wrote: > Hi Andreas, > > On Thu, Dec 09, 2010 at 02:00:30PM +0100, Andreas Kohn wrote: > > Ok, > > > > attached a set of patches that allows an admin user to add and remove a > > server from an existing backend using the stats interface. > > > > The use-case we have for this is that we have a lot of servers that come > > and go (amazon cloud environment with automatic scaling), and to add > > them to a haproxy we want to avoid restarts of the haproxy. > > [...] > > > > I currently have open "issues" with this: > > 1) is the adding doing the right things in terms of putting it in > >in the struct proxy and setting it up? > > 2) I still need to check what other structures need to be free()'d > >when removing a server > > 3) is there any kind of locking requirements if requests come in while > >a server is added/removed? > > > > > > What do you think? > > I still have a problem with doing that because I don't see how you can > easily fix some parameters that are determined at config time : > > - the size of the LB map for hash/static-rr is determined as the sum of > the servers' weights divided by their GCD. Adding a server or changing > even one weight will result in a change of the map size. This must be > reallocated and recomputed. Indeed, need to look into that I guess. The tests I did so far were only with a single server, and there was a disabled server in the config for another test. > - the number of file descriptors is set to the total amount of required > sockets (listeners, client, server, pipes, checks). If you add servers, > you will need more sockets to run the checks, but you don't have the > required privileges anymore to change the ulimit. That means that > connections or health checks may randomly fail when working close to > connection limit if some servers are added. Hmm. One quick way around that could be to statically configure a number of extra file descriptors as "slack" in the backend options. Usually we don't add an excessive new amount of servers without removing similar amounts of other servers, so we should be able to estimate maximum/average numbers. But I definitely should check this, and refuse the addition if the limit is reached. In that case the caller could fall-back to restarting the haproxy instance with a higher maximum. > - what happens when you remove a server that is being tracked or referenced > by an ACL ? We don't use those features, so I fear my answer would be a clear "I don't know". Will have a look. I hope that the patches in the current state could be helpful already for looking at them, but most certainly they should be flagged "highly experimental". I'm still learning how haproxy works internally :) Regards, -- Andreas > Thanks, > Willy -- Never attribute to malice that which can be adequately explained by stupidity.-- Hanlon's Razor signature.asc Description: This is a digitally signed message part
Re: Adding/Removing servers for a backend without restarting haproxy
Hi Andreas, On Thu, Dec 09, 2010 at 02:00:30PM +0100, Andreas Kohn wrote: > Ok, > > attached a set of patches that allows an admin user to add and remove a > server from an existing backend using the stats interface. > > The use-case we have for this is that we have a lot of servers that come > and go (amazon cloud environment with automatic scaling), and to add > them to a haproxy we want to avoid restarts of the haproxy. > > A simple example: > > Adding a serverA ipA:portA to the 'query_backend': > echo "add server query_backend/serverA ipA:portA check inter 3000 rise 2 fall > 3 maxconn 50" | socat stdio unix-connect:/var/haproxy.socket > > Removing serverA again: > echo "remove server query_backend/serverA" | socat stdio > unix-connect:/var/haproxy.socket > > > The commands do not modify the configuration file, so a restart of haproxy > will > "forget" about the added servers. In our use case we'll modify the > configuration > using some scripts, so this was not a problem, and changing the configuration > files > would be quite a bit of work. > > > I've tested the actual commands by repeatedly adding/removing a web server > and verifying that haproxy returns the proper answer (200 if the server is > there, 503 > if it was removed) > > > Patch index: > > 0001-Move-the-Wall-from-COPTS-into-a-separate-variable-WA.patch > 0002-Do-not-return-a-value-from-a-void-function.patch > Two patches I initially did because I tried compiling with Sun cc > first. Unfortunately there is a lot more issues with this > compiler, so I stopped there and used gcc again. > > 0003-Set-the-session-previous-server-to-the-current-serve.patch > Patch for the issue in assign_server() I reported in the > previous mail. > > 0004-Fix-typos-in-comments.patch > Some comment typos fixed. > > The actual dynamic adding/removing: > 0005-Pull-out-the-checks-for-a-single-server-configuratio.patch > Factor out the server configuration validation from > cfgparse:check_config_validity() > > 0006-Factor-out-allocation-and-configuration-of-a-new-ser.patch > Factor out the server creation and application of configuration options > from cfgparse:cfg_parse_listen() > > 0007-Add-support-for-adding-removing-servers-from-an-exis.patch > The new 'add server'/'remove server' commands for > dumpstats:stats_sock_parse_request() > This also adds two SRV_ flags: SRV_REMOVED is used to remove the > server when bringing it down, SRV_ADDED is currently unused. > > > I currently have open "issues" with this: > 1) is the adding doing the right things in terms of putting it in >in the struct proxy and setting it up? > 2) I still need to check what other structures need to be free()'d >when removing a server > 3) is there any kind of locking requirements if requests come in while >a server is added/removed? > > > What do you think? I still have a problem with doing that because I don't see how you can easily fix some parameters that are determined at config time : - the size of the LB map for hash/static-rr is determined as the sum of the servers' weights divided by their GCD. Adding a server or changing even one weight will result in a change of the map size. This must be reallocated and recomputed. - the number of file descriptors is set to the total amount of required sockets (listeners, client, server, pipes, checks). If you add servers, you will need more sockets to run the checks, but you don't have the required privileges anymore to change the ulimit. That means that connections or health checks may randomly fail when working close to connection limit if some servers are added. - what happens when you remove a server that is being tracked or referenced by an ACL ? Thanks, Willy
Re: [PATCH] Potential issue in backend.c:assign_server()
Hi, On Thu, Dec 09, 2010 at 01:37:15PM +0100, Andreas Kohn wrote: > Hi, > > while trying to implement a feature to dynamically add/remove servers > from a backend (patches for that in another mail soon) I found an odd > line in assign_server(): > > s->prev_srv = s->prev_srv; > > Shouldn't this be > > s->prev_srv = s->srv? > > Or, am I missing something here? You're obviously right. In fact it has never caused any issue because it's already done right in srv_redispatch_connect(). That said, it's better to fix it. I think the best fix would consist in removing the line to avoid any confusion. Regards, Willy
Re: Multiple Load Balancers, stick table and url-embedded session support
On Thu, Dec 9, 2010 at 7:27 PM, Hank A. Paulson < h...@spamproof.nospammail.net> wrote: > Please see the thread: > "need help figuring out a sticking method" > > I asked about this, Willie says there are issues figuring out a workable > config syntax for 'regex to pull the URL/URI substring' but (I think) that > coding the functionality is not technically super-difficult just not enough > hands maybe and the config syntax? > > Actually if the "key" is to taken from a query param, that is relatively easy enough (I coded something myself for a client some time back based on 1.3.15.4). If, however, more flexibility is required (like in your case), then the point that Willie has mentioned will definitely come into play. -jf I have a feeling this would be a fairly commonly used feature, so it is good > to see others asking the same question :) > > How are you planning to distribute the traffic to the different haproxy > instances? LVS? Some hardware? > > > On 12/8/10 8:58 PM, David wrote: > >> Hi there, >> >> I have been asked to design an architecture for our load-balancing needs, >> and >> it looks like haproxy can do almost everything needed in a fairly >> straightfoward way. Two of the requirements are stickiness support (always >> send a request for a given session to the same backend) as well as >> multiple >> load balancers running at the same time to avoid single point of failure >> (hotbackup with only one haproxy running at a time is not considered >> acceptable). >> >> Using multiple HAproxy instances in parallel with stickiness support looks >> relatively easy if cookies are allowed (through e.g. cookie prefixing) >> since >> no information needs to be shared. Unfortunately, we also need to support >> session id embedded in URL (e.g. http://example.com/foo?sess=someid), and >> I >> was hoping that the new sticky table replication in 1.5 could help for >> that, >> but I am not sure it is the case. >> >> As far as I understand, I need to first define a table with string type, >> and >> then use the store-request to store the necessary information. I cannot >> see a >> way to get some information embedded in the URL using the existing query >> extraction methods. Am I missing something, or is it difficult to do this >> with >> haproxy ? >> >> regards, >> >> David >> >> >
[PATCH] Potential issue in backend.c:assign_server()
Hi, while trying to implement a feature to dynamically add/remove servers from a backend (patches for that in another mail soon) I found an odd line in assign_server(): s->prev_srv = s->prev_srv; Shouldn't this be s->prev_srv = s->srv? Or, am I missing something here? Regards, -- Andreas -- Never attribute to malice that which can be adequately explained by stupidity.-- Hanlon's Razor signature.asc Description: This is a digitally signed message part
Re: Multiple Load Balancers, stick table and url-embedded session support
Please see the thread: "need help figuring out a sticking method" I asked about this, Willie says there are issues figuring out a workable config syntax for 'regex to pull the URL/URI substring' but (I think) that coding the functionality is not technically super-difficult just not enough hands maybe and the config syntax? I have a feeling this would be a fairly commonly used feature, so it is good to see others asking the same question :) How are you planning to distribute the traffic to the different haproxy instances? LVS? Some hardware? On 12/8/10 8:58 PM, David wrote: Hi there, I have been asked to design an architecture for our load-balancing needs, and it looks like haproxy can do almost everything needed in a fairly straightfoward way. Two of the requirements are stickiness support (always send a request for a given session to the same backend) as well as multiple load balancers running at the same time to avoid single point of failure (hotbackup with only one haproxy running at a time is not considered acceptable). Using multiple HAproxy instances in parallel with stickiness support looks relatively easy if cookies are allowed (through e.g. cookie prefixing) since no information needs to be shared. Unfortunately, we also need to support session id embedded in URL (e.g. http://example.com/foo?sess=someid), and I was hoping that the new sticky table replication in 1.5 could help for that, but I am not sure it is the case. As far as I understand, I need to first define a table with string type, and then use the store-request to store the necessary information. I cannot see a way to get some information embedded in the URL using the existing query extraction methods. Am I missing something, or is it difficult to do this with haproxy ? regards, David
