Re:
John, As you are using cookies it is safe to use the standard soft reload on HAProxy. So just change the configuration file as required and restart. We have a (very) simple API script on our EC2 (HAProxy based) appliance which allows auto-scaling servers in the cluster to register their IP address with the HAProxy load balancer and automatically join the cluster when they boot: Assuming your dynamic servers have the same SSH key, they can just locate the load balancer by DNS and run the API command remotely, passing their own IP details. http://www.loadbalancer.org/ec2.php You would put something like the following in the init script on your dynamic (auto scaling) servers: #!/bin/sh PATH=/sbin:/bin:/usr/sbin:/usr/bin; AMI_KEY_PAIR=path-to-ssh-key; EC2_LOADBALANCER_IP=ip-address-of-ec2-loadbalancer; CURL=`which curl`; SSH=`which ssh`; AMI_ID=`$CURL -s http://169.254.169.254/latest/meta-data/ami-id`;; AMI_IP=`$CURL -s http://169.254.169.254/latest/meta-data/local-ipv4`;; case $1 in start) $SSH -i $AMI_KEY_PAIR root@$EC2_LOADBALANCER_IP \lb_modify -l $AMI_ID -d $AMI_IP \; exit 0; ;; stop) $SSH -i $AMI_KEY_PAIR root@$EC2_LOADBALANCER_IP \lb_modify -l $AMI_ID -d $AMI_IP -r \; ;; *) exit 1; ;; esac; exit 0; On 8 February 2012 01:09, John Langley dige...@gmail.com wrote: We are looking for a solution for sticky bit routing based on cookies that will run on Amazon's EC2 cloud. I've looked at the architecture guide for HAProxy (although not the source yet) and it ~may~ be capable of doing what we need, but I thought I'd ask the mailing list to see if anyone else has already tried this solution. (Without knowing the implementation, it's impossible to say if our requirements can be met by the implementation) The challenge that we have is that unlike a traditional system where the sticky bit routing would be to one of a set of predefined servers, in our case, the servers will be created dynamically in the cloud. We can't configure them when we start the HAProxy routing layer. Although we may have some back up servers, that can be used if no cookie is in the request OR if the cookie specifies a server that has died, in general the servers that the cookie will be specifying will be dynamically created and we will assign them to the requests ourselves (not needing the nginx layer to round-robin assign them to one of a pool of fixed address servers). So my question may come down to: Can HAProxy route to servers not predefined in the initial configuration? I can easily imagine an implementation that could handle this, but wanted to ask if HAProxy already does this. Thanks in advance -- Langley -- Regards, Malcolm Turnbull. Loadbalancer.org Ltd. Phone: +44 (0)870 443 8779 http://www.loadbalancer.org/
Possible error in changing port between frontend and backend in 1.4.18
Hello.
I have setup haproxy as frontend for php/nginx farm for our company application.
Our app contains two main modules - http frontend and webservice
apllication server.
http uses normal port - 80 (ssl through stunel) and webservice apps
are available on port 8000
Ive decided to use one backend server with leastconn algorithm and
setting + prefix for each server to redirect connection to port 81
or 8001 on which runs nginx.
Everything worked perfect until i checked log on client application
wich uses our webservice application.
For some request there were 503 responses service unavailble
Digging into logs and tcpdump i found out that haproxy sometimes
manages to send client_port instead of destination port to rewriting
algoritm which ends up in requesting backend server on nonexisting
port.
Previous config
backend app
balance leastconn
server diabel2_php diabel2_php:+1 check port 81 weight 2 maxconn 1
server diabel3_php diabel3_php:+1 check port 81 weight 2 maxconn 2
server diabel1_php diabel1_php:+1 check port 81 weight 2 maxconn 4
server bies1_php bies1_php:+1 check port 81 weight 3 maxconn 250
server bies2_php bies2_php:+1 check port 81 weight 3 maxconn 250
server bies3_php bies3_php:+1 check port 81 weight 3 maxconn 250
server bies4_php bies4_php:+1 check port 81 weight 3 maxconn 250
haproxy.strace
grep connect haproxy.strace |grep 10.6.10|grep -v 8001|grep -v 81
10:10:52 connect(11, {sa_family=AF_INET, sin_port=htons(60988),
sin_addr=inet_addr(10.6.10.102)}, 16) = -1 EINPROGRESS (Operation
now in progress)
10:10:53 connect(17, {sa_family=AF_INET, sin_port=htons(60988),
sin_addr=inet_addr(10.6.10.102)}, 16) = -1 EINPROGRESS (Operation
now in progress)
10:10:54 connect(1, {sa_family=AF_INET, sin_port=htons(60988),
sin_addr=inet_addr(10.6.10.102)}, 16) = -1 EINPROGRESS (Operation
now in progress)
10:10:55 connect(12, {sa_family=AF_INET, sin_port=htons(60988),
sin_addr=inet_addr(10.6.10.1)}, 16) = -1 EINPROGRESS (Operation now
in progress)
10:10:56 connect(13, {sa_family=AF_INET, sin_port=htons(1973),
sin_addr=inet_addr(10.6.10.1)}, 16) = -1 EINPROGRESS (Operation now
in progress)
10:10:56 connect(13, {sa_family=AF_INET, sin_port=htons(4756),
sin_addr=inet_addr(10.6.10.104)}, 16) = -1 EINPROGRESS (Operation
now in progress)
10:10:56 connect(11, {sa_family=AF_INET, sin_port=htons(4037),
sin_addr=inet_addr(10.6.10.102)}, 16) = -1 EINPROGRESS (Operation
now in progress)
coresponding haproxy.log:
Feb 8 10:10:55 localhost haproxy[25750]: 79.189.159.77:60987
[08/Feb/2012:10:10:52.036] http app/diabel1_php 6/3005/-1/-1/3012 503
212 - - SC-- 4/4/2/0/+3 0/0 GET
/f82b0a1554d0426a476c391ce49442ba/?request=PopReceipt HTTP/1.1
Feb 8 10:10:59 localhost haproxy[25750]: 79.187.248.101:1972
[08/Feb/2012:10:10:56.213] http app/bies1_php 1/3004/-1/-1/3006 503
212 - - SC-- 9/9/9/1/+3 0/0 GET
/56b919f0b3592d6624ef32e3e96f1de9/?request=PopReceipt HTTP/1.1
Feb 8 10:10:59 localhost haproxy[25750]: 95.49.216.250:4755
[08/Feb/2012:10:10:56.347] http app/bies2_php 8/3006/-1/-1/3016 503
212 - - SC-- 6/6/6/0/+3 0/0 GET
/ab48266e45a77df9ba5f693c05484691/?request=PopReceipt HTTP/1.1
10.6.10.* are backend servers (diabel|bies)[1234]_php
For now i created separated backends for webservice and normal web
frontend and i havent spoted errors anymore.
If i can provide anymore info just let me know.
Greetins
--
Pozdrawiam
Mateusz Ratajski // http://exp.pl
Possible error in changing port between frontend and backend in 1.4.18
Hello.
I have setup haproxy as frontend for php/nginx farm for our company application.
Our app contains two main modules - http frontend and webservice
apllication server.
http uses normal port - 80 (ssl through stunel) and webservice apps
are available on port 8000
Ive decided to use one backend server with leastconn algorithm and
setting + prefix for each server to redirect connection to port 81
or 8001 on which runs nginx.
Everything worked perfect until i checked log on client application
wich uses our webservice application.
For some request there were 503 responses service unavailble
Digging into logs and tcpdump i found out that haproxy sometimes
manages to send client_port instead of destination port to rewriting
algoritm which ends up in requesting backend server on nonexisting
port.
Previous config
backend app
balance leastconn
server diabel2_php diabel2_php:+1 check port 81 weight 2 maxconn 1
server diabel3_php diabel3_php:+1 check port 81 weight 2 maxconn 2
server diabel1_php diabel1_php:+1 check port 81 weight 2 maxconn 4
server bies1_php bies1_php:+1 check port 81 weight 3 maxconn 250
server bies2_php bies2_php:+1 check port 81 weight 3 maxconn 250
server bies3_php bies3_php:+1 check port 81 weight 3 maxconn 250
server bies4_php bies4_php:+1 check port 81 weight 3 maxconn 250
haproxy.strace
grep connect haproxy.strace |grep 10.6.10|grep -v 8001|grep -v 81
10:10:52 connect(11, {sa_family=AF_INET, sin_port=htons(60988),
sin_addr=inet_addr(10.6.10.102)}, 16) = -1 EINPROGRESS (Operation
now in progress)
10:10:53 connect(17, {sa_family=AF_INET, sin_port=htons(60988),
sin_addr=inet_addr(10.6.10.102)}, 16) = -1 EINPROGRESS (Operation
now in progress)
10:10:54 connect(1, {sa_family=AF_INET, sin_port=htons(60988),
sin_addr=inet_addr(10.6.10.102)}, 16) = -1 EINPROGRESS (Operation
now in progress)
10:10:55 connect(12, {sa_family=AF_INET, sin_port=htons(60988),
sin_addr=inet_addr(10.6.10.1)}, 16) = -1 EINPROGRESS (Operation now
in progress)
10:10:56 connect(13, {sa_family=AF_INET, sin_port=htons(1973),
sin_addr=inet_addr(10.6.10.1)}, 16) = -1 EINPROGRESS (Operation now
in progress)
10:10:56 connect(13, {sa_family=AF_INET, sin_port=htons(4756),
sin_addr=inet_addr(10.6.10.104)}, 16) = -1 EINPROGRESS (Operation
now in progress)
10:10:56 connect(11, {sa_family=AF_INET, sin_port=htons(4037),
sin_addr=inet_addr(10.6.10.102)}, 16) = -1 EINPROGRESS (Operation
now in progress)
coresponding haproxy.log:
Feb 8 10:10:55 localhost haproxy[25750]: 79.189.159.77:60987
[08/Feb/2012:10:10:52.036] http app/diabel1_php 6/3005/-1/-1/3012 503
212 - - SC-- 4/4/2/0/+3 0/0 GET
/f82b0a1554d0426a476c391ce49442ba/?request=PopReceipt HTTP/1.1
Feb 8 10:10:59 localhost haproxy[25750]: 79.187.248.101:1972
[08/Feb/2012:10:10:56.213] http app/bies1_php 1/3004/-1/-1/3006 503
212 - - SC-- 9/9/9/1/+3 0/0 GET
/56b919f0b3592d6624ef32e3e96f1de9/?request=PopReceipt HTTP/1.1
Feb 8 10:10:59 localhost haproxy[25750]: 95.49.216.250:4755
[08/Feb/2012:10:10:56.347] http app/bies2_php 8/3006/-1/-1/3016 503
212 - - SC-- 6/6/6/0/+3 0/0 GET
/ab48266e45a77df9ba5f693c05484691/?request=PopReceipt HTTP/1.1
10.6.10.* are backend servers (diabel|bies)[1234]_php
For now i created separated backends for webservice and normal web
frontend and i havent spoted errors anymore.
If i can provide anymore info just let me know.
Greetins
--
Pozdrawiam
Mateusz Ratajski // http://exp.pl
--
Pozdrawiam
Mateusz Ratajski // http://exp.pl
509305409
Re: Resp error on stats page
Hi, Can you enable logging and check out what kind of errors are generated? If you can share it with us (or at least with me), we (I) could give you some clues. Cheers On Wed, Feb 8, 2012 at 2:47 AM, Dwyer, Simon sdw...@federalit.net wrote: Hi All, We are doing some testing of HAProxy as a frontend for sharepoint servers and all is going well. I am however seeing errors arising in the status page for Resp. Is this something to worry about. We havnt seen issues from a users point of view. I have googled it and havnt found too much explaining what it is. Cheers, Simon
Re: Possible error in changing port between frontend and backend in 1.4.18
Hi,
It seems the servers have explicitly refused the connection (TCP RST).
Have you enabled the redispatch option?
It should be used in pair with the retry 3 you may have already set up.
cheers
On Wed, Feb 8, 2012 at 11:30 AM, Mateusz Ratajski
mateusz.rataj...@exp.net.pl wrote:
Hello.
I have setup haproxy as frontend for php/nginx farm for our company
application.
Our app contains two main modules - http frontend and webservice
apllication server.
http uses normal port - 80 (ssl through stunel) and webservice apps
are available on port 8000
Ive decided to use one backend server with leastconn algorithm and
setting + prefix for each server to redirect connection to port 81
or 8001 on which runs nginx.
Everything worked perfect until i checked log on client application
wich uses our webservice application.
For some request there were 503 responses service unavailble
Digging into logs and tcpdump i found out that haproxy sometimes
manages to send client_port instead of destination port to rewriting
algoritm which ends up in requesting backend server on nonexisting
port.
Previous config
backend app
balance leastconn
server diabel2_php diabel2_php:+1 check port 81 weight 2 maxconn 1
server diabel3_php diabel3_php:+1 check port 81 weight 2 maxconn 2
server diabel1_php diabel1_php:+1 check port 81 weight 2 maxconn 4
server bies1_php bies1_php:+1 check port 81 weight 3 maxconn 250
server bies2_php bies2_php:+1 check port 81 weight 3 maxconn 250
server bies3_php bies3_php:+1 check port 81 weight 3 maxconn 250
server bies4_php bies4_php:+1 check port 81 weight 3 maxconn 250
haproxy.strace
grep connect haproxy.strace |grep 10.6.10|grep -v 8001|grep -v 81
10:10:52 connect(11, {sa_family=AF_INET, sin_port=htons(60988),
sin_addr=inet_addr(10.6.10.102)}, 16) = -1 EINPROGRESS (Operation
now in progress)
10:10:53 connect(17, {sa_family=AF_INET, sin_port=htons(60988),
sin_addr=inet_addr(10.6.10.102)}, 16) = -1 EINPROGRESS (Operation
now in progress)
10:10:54 connect(1, {sa_family=AF_INET, sin_port=htons(60988),
sin_addr=inet_addr(10.6.10.102)}, 16) = -1 EINPROGRESS (Operation
now in progress)
10:10:55 connect(12, {sa_family=AF_INET, sin_port=htons(60988),
sin_addr=inet_addr(10.6.10.1)}, 16) = -1 EINPROGRESS (Operation now
in progress)
10:10:56 connect(13, {sa_family=AF_INET, sin_port=htons(1973),
sin_addr=inet_addr(10.6.10.1)}, 16) = -1 EINPROGRESS (Operation now
in progress)
10:10:56 connect(13, {sa_family=AF_INET, sin_port=htons(4756),
sin_addr=inet_addr(10.6.10.104)}, 16) = -1 EINPROGRESS (Operation
now in progress)
10:10:56 connect(11, {sa_family=AF_INET, sin_port=htons(4037),
sin_addr=inet_addr(10.6.10.102)}, 16) = -1 EINPROGRESS (Operation
now in progress)
coresponding haproxy.log:
Feb 8 10:10:55 localhost haproxy[25750]: 79.189.159.77:60987
[08/Feb/2012:10:10:52.036] http app/diabel1_php 6/3005/-1/-1/3012 503
212 - - SC-- 4/4/2/0/+3 0/0 GET
/f82b0a1554d0426a476c391ce49442ba/?request=PopReceipt HTTP/1.1
Feb 8 10:10:59 localhost haproxy[25750]: 79.187.248.101:1972
[08/Feb/2012:10:10:56.213] http app/bies1_php 1/3004/-1/-1/3006 503
212 - - SC-- 9/9/9/1/+3 0/0 GET
/56b919f0b3592d6624ef32e3e96f1de9/?request=PopReceipt HTTP/1.1
Feb 8 10:10:59 localhost haproxy[25750]: 95.49.216.250:4755
[08/Feb/2012:10:10:56.347] http app/bies2_php 8/3006/-1/-1/3016 503
212 - - SC-- 6/6/6/0/+3 0/0 GET
/ab48266e45a77df9ba5f693c05484691/?request=PopReceipt HTTP/1.1
10.6.10.* are backend servers (diabel|bies)[1234]_php
For now i created separated backends for webservice and normal web
frontend and i havent spoted errors anymore.
If i can provide anymore info just let me know.
Greetins
--
Pozdrawiam
Mateusz Ratajski // http://exp.pl
--
Pozdrawiam
Mateusz Ratajski // http://exp.pl
509305409
Re: HAProxy in Amazon EC2 in peer mode
Hi, This kind of scenario is really dependent from your configuration. Basically, you want to configure HAProxy to load-balance SSL services hosted by your application servers. Persistence based on SSLID into HAProxy, sharing the stick-table between both haproxy. This may work, but be carefull with the peers section, since your LB will change their IP addresses each time they'll reboot... cheers On Tue, Feb 7, 2012 at 1:56 AM, Mir Islam mis...@mirislam.com wrote: Hi, I am thinking of the following scenario to implement in AWS. Two HAProxies with SSL stickiness, TCP pass through and in peer mode. Using Amazon ELB to do round robin load balance between the two server. Basically the idea is to create automatic failover for HA proxy servers. Theoretically it should work since both of the servers have the same session information. Am I wrong on this assumption? Thanks Mir
Re: HAProxy in Amazon EC2 in peer mode
You can also write a script that gather instances' data (using the EC2 APIs) in order to get the IPs. That way you can update the HAProxy configuration files when a change in your setup occurs. Cheers, Michele On Feb 8, 2012, at 5:33 PM, Mir Islam wrote: Right, the internal IP changes since it is on DHCP. But I was thinking of assigning static IP (EIP in Amazon term) to the HAProxies. And then refer to each other via those EIP. Here is a quick and dirty diagram of howI am thinking of implementing. Ofcourse the main cost is the static IP addresses that I have to get from Amazon.
Re: HAProxy in Amazon EC2 in peer mode
Just in case anyone was following in my effort to get HAProxy running in EC2. The peers mode will not work in EC2 because the IP address identified in the peer section must be an ip which is bound to the interface. I could not find a way to bypass it. Since the static IP assigned in EC2 are not really bound to the interfaces on the vm itself, it appears for the time being it is not possible to run in peer mode with static IP. It is possible to use the internal 10.x.x.x IP assigned by Amazon but that may change after a reboot. Oh, well I guess it is what it is. On Feb 8, 2012, at 7:33 AM, Mir Islam wrote: On Feb 8, 2012, at 2:49 AM, Baptiste wrote: Hi, This kind of scenario is really dependent from your configuration. Basically, you want to configure HAProxy to load-balance SSL services hosted by your application servers. That is correct. Persistence based on SSLID into HAProxy, sharing the stick-table between both haproxy. Right, that way any one of the pair of HAProxy server would know where to send the request from client. This may work, but be carefull with the peers section, since your LB will change their IP addresses each time they'll reboot... Right, the internal IP changes since it is on DHCP. But I was thinking of assigning static IP (EIP in Amazon term) to the HAProxies. And then refer to each other via those EIP. Here is a quick and dirty diagram of howI am thinking of implementing. Ofcourse the main cost is the static IP addresses that I have to get from Amazon. Thanks Baptiste. PastedGraphic-1.pdf cheers On Tue, Feb 7, 2012 at 1:56 AM, Mir Islam mis...@mirislam.com wrote: Hi, I am thinking of the following scenario to implement in AWS. Two HAProxies with SSL stickiness, TCP pass through and in peer mode. Using Amazon ELB to do round robin load balance between the two server. Basically the idea is to create automatic failover for HA proxy servers. Theoretically it should work since both of the servers have the same session information. Am I wrong on this assumption? Thanks Mir
RE: Resp error on stats page
Ok so I worked out how to do logging in haproxy and how to read it (great to learn) The two logs that seem to be the resp are... Feb 9 14:26:18 localhost haproxy[25784]: 10.35.31.12:54422 [09/Feb/2012:14:24:10.920] 10-35-31-135 CTHUB/SPWFW1 0/0/0/4/127355 401 112211 - - SDNI 1/1/1/0/0 0/0 GET /SitePages/Home.aspx HTTP/1.0 Feb 9 14:26:19 localhost haproxy[25784]: 10.35.31.12:54419 [09/Feb/2012:14:24:10.395] 10-35-31-135 CTHUB/SPWFW2 0/0/1/2/129120 401 112211 - - SDNI 0/0/0/0/0 0/0 GET /SitePages/Home.aspx HTTP/1.0 From what i read SD means that ther is a rst coming from the server? is that something i should be worried about or could that just be normal process? Cheers all, Simon From: Baptiste [bed...@gmail.com] Sent: Wednesday, February 08, 2012 9:34 PM To: Dwyer, Simon Cc: haproxy@formilux.org Subject: Re: Resp error on stats page Hi, Can you enable logging and check out what kind of errors are generated? If you can share it with us (or at least with me), we (I) could give you some clues. Cheers On Wed, Feb 8, 2012 at 2:47 AM, Dwyer, Simon sdw...@federalit.net wrote: Hi All, We are doing some testing of HAProxy as a frontend for sharepoint servers and all is going well. I am however seeing errors arising in the status page for Resp. Is this something to worry about. We havnt seen issues from a users point of view. I have googled it and havnt found too much explaining what it is. Cheers, Simon
Re: HAProxy in Amazon EC2 in peer mode
welcome to the cloud :) On Thu, Feb 9, 2012 at 2:26 AM, Mir Islam mis...@mirislam.com wrote: Just in case anyone was following in my effort to get HAProxy running in EC2. The peers mode will not work in EC2 because the IP address identified in the peer section must be an ip which is bound to the interface. I could not find a way to bypass it. Since the static IP assigned in EC2 are not really bound to the interfaces on the vm itself, it appears for the time being it is not possible to run in peer mode with static IP. It is possible to use the internal 10.x.x.x IP assigned by Amazon but that may change after a reboot. Oh, well I guess it is what it is. On Feb 8, 2012, at 7:33 AM, Mir Islam wrote: On Feb 8, 2012, at 2:49 AM, Baptiste wrote: Hi, This kind of scenario is really dependent from your configuration. Basically, you want to configure HAProxy to load-balance SSL services hosted by your application servers. That is correct. Persistence based on SSLID into HAProxy, sharing the stick-table between both haproxy. Right, that way any one of the pair of HAProxy server would know where to send the request from client. This may work, but be carefull with the peers section, since your LB will change their IP addresses each time they'll reboot... Right, the internal IP changes since it is on DHCP. But I was thinking of assigning static IP (EIP in Amazon term) to the HAProxies. And then refer to each other via those EIP. Here is a quick and dirty diagram of howI am thinking of implementing. Ofcourse the main cost is the static IP addresses that I have to get from Amazon. Thanks Baptiste. PastedGraphic-1.pdf cheers On Tue, Feb 7, 2012 at 1:56 AM, Mir Islam mis...@mirislam.com wrote: Hi, I am thinking of the following scenario to implement in AWS. Two HAProxies with SSL stickiness, TCP pass through and in peer mode. Using Amazon ELB to do round robin load balance between the two server. Basically the idea is to create automatic failover for HA proxy servers. Theoretically it should work since both of the servers have the same session information. Am I wrong on this assumption? Thanks Mir
