Re: Who replies to client requests?

2012-12-11 Thread Hermes Flying
But what happens with TLS? Am I supposed to configure my server certificates to 
HAProxy?




 From: David Coulson 
To: Hermes Flying  
Cc: "haproxy@formilux.org"  
Sent: Tuesday, December 11, 2012 1:16 PM
Subject: Re: Who replies to client requests?
 

HAProxy is a proxy, so there is one TCP connection from client to HAProxy, then 
a new TCP connection is built between HAProxy and backend. Backend server 
responds to HAProxy, then HAProxy sends that to client.



On 12/11/12 3:52 AM, Hermes Flying wrote:

Hi,
>By using a HAProxy as load balancer, the reply to the client is send directly 
>from the server node that processed the request or the reply is send by 
>HAProxy?
>If the reply is send directly by the server node, do you have any issues by 
>the fact that the MAC address in the response is different than the MAC 
>address that was the request for? E.g. like spoofing issue?
>
>
>
>Thanks
>

Re: HAProxy basic setup question

2012-12-08 Thread Hermes Flying
But if one server is lost then the client requests will all be served by 1 of 
the 2 servers instead of distributing the requests among the 2 servers.
So due to overload we could have from degradation to e.g. SW crash due to 
OutOfMemory exceptions. I mean doesn't this "avalanche" into a SPOF?




 From: Willy Tarreau 
To: Hermes Flying  
Cc: David Coulson ; "haproxy@formilux.org" 
 
Sent: Saturday, December 8, 2012 7:25 PM
Subject: Re: HAProxy basic setup question
 
On Sat, Dec 08, 2012 at 09:14:48AM -0800, Hermes Flying wrote:
> Hi Willy, thanks for this.
> 1)I wanted to ask does the oblique lines indicates 2-port NIC on each server?

Exactly. Note that almost all servers nowadays come with 2 onboard ports.

> 2) If I remove the oblique line as you note and have the 2 switches
> interconnected this is still considered a SPOF right? As loss of either
> switch1 or switch 2 brings the design down, right?

No it's not a spof because if you lose a switch, you only lose the
attached server and the attached router, so your architecture works
in degraded mode but you still have one component of each kind to
provide the service.

Willy

Re: HAProxy basic setup question

2012-12-08 Thread Hermes Flying
Hi Willy, thanks for this.
1)I wanted to ask does the oblique lines indicates 2-port NIC on each server?
2) If I remove the oblique line as you note and have the 2 switches 
interconnected this is still considered a SPOF right? As loss of either switch1 
or switch 2 brings the design down, right?





 From: Willy Tarreau 
To: Hermes Flying  
Cc: David Coulson ; "haproxy@formilux.org" 
 
Sent: Saturday, December 8, 2012 4:49 PM
Subject: Re: HAProxy basic setup question
 
Hi,

On Sat, Dec 08, 2012 at 05:45:38AM -0800, Hermes Flying wrote:
> I see. With the 2 switches you mention I guess it would not be possible to
> set the 2 lixux which have a single port NIC.Right?

In fact yes you can but in the event of a switch loss, you also lose the
machine attached to that switch. Still that's a risk many hosters take
when they have numerous machines, because switch ports are expensive and
they don't always want to double the number of switches just for the rare
case of a lost switch.

In general your setup would look like this :

         router 1           router 2
            |                  |
     +-+    +-+
     |  switch 1   ||  switch 2   |
     +-++--+    +--++-+
       |         \        /         |
       |           \    /           |
       |             \/             |
       |            /  \            |
       |          /      \          |
       |        /          \        |
       |      /              \      |
       |    /                  \    | 
     +-+---+-+                +-+---+-+
     | srv1  |                | srv2  |
     +---+                +---+

Each front router is connected to one switch. Both are connected to the
internet and make use of a dynamic routing protocol to reach the net
(typically BGP).

Both switches are connected using a redundant link (etherchannel/trunk/lacp,
name depends on brands).

Each server is configured with bonded interfaces with the active interface
connected to one switch and the backup interface to the other switch. Some
people prefer to have the active interface on a different switch so that
all switches are constantly used. Other people prefer to have all active
interfaces on the same switch and the backup ones on the other switch so
that they know which switch gets the traffic (useful for network operations).

Note that the architecture works well with servers wihch have a single network
interface. Just cut the oblique links above and it's still OK. However the
loss of a switch causes the loss of the attached server and router.

And David was right, most incompletely redundant architectures are commonly
worse than simple SPOFs. For example, if you can't afford two switches or
two routers, you might prefer to have a spare cheap $100 8-port switch to
quickly move your servers in case the main switch dies. That's better than
something complex that nobody understands and knows how to fix.

In general, you need to have redundant servers because servers commonly
run bogus software that regularly dies. We could say that each of your
servers will probably unexpectedly disappear once every 1-2 years, and
you need to add planned reboots to that. A correct switch will rarely
die before the first 3 years unless it's incorrectly cooled (eg: closed
rack with glass doors). Good quality routers tend to last forever. Link
to the operator my randomly dysfunction more often than the servers
themselves.

With this in mind, it should be clear that you at least need two servers,
ideally double-attached. Two switches are highly recommended to avoid
the complexity of moving all cables in case of sudden death. A single
router might be acceptable if the internal cable is correctly identified
and easy to move by hand to the second switch very quickly (eg: always
on port 1).

Hoping this helps,
Willy

Re: HAProxy basic setup question

2012-12-08 Thread Hermes Flying
I see. With the 2 switches you mention I guess it would not be possible to set 
the 2 lixux which have a single port NIC.Right?




 From: David Coulson 
To: Hermes Flying  
Cc: "haproxy@formilux.org"  
Sent: Saturday, December 8, 2012 3:41 PM
Subject: Re: HAProxy basic setup question
 

Yep - The systems we use in clustered environments typically have 4 or 6 NICs 
for redundant front-end and back-end networks.

That's why I told you to pay someone to build it.


On 12/8/12 8:38 AM, Hermes Flying wrote:

Are you assuming multi-port NICs? Sorry if this is a trivial question but I am 
an application programmer and lack your background.
>
>
>
>
>____
> From: David Coulson 
>To: Hermes Flying  
>Cc: "haproxy@formilux.org"  
>Sent: Saturday, December 8, 2012 3:36 PM
>Subject: Re: HAProxy basic setup question
> 
>
>Well, since it's 2012 you use a switch instead of a hub. And as I described 
>earlier you can take two switches and connect systems to both, reducing the 
>risk of a hardware fault taking everything down. You use the bonding 
>capability in Linux to make the two NIC ports appear as one logical interface 
>in the OS.
>
>If you are so worried about building a massively
resilient system, you need to pay someone to build it
for you. In my experience, a poorly built 'redundant'
environment ends up with more downtime than a one with
multiple single points of failure.
>
>
>On 12/8/12 8:33 AM, Hermes Flying wrote:
>
>2 Linuxes connecting LBs over the same hub. Not sure what you mean by 2 
>switches
>>Isn't it SPOF? If the hub breaks then no load balancing
>>
>>
>>
>>
>>
>> From: David Coulson 
>>To: Hermes Flying  
>>Cc: "haproxy@formilux.org"  
>>Sent: Saturday, December 8, 2012 3:31 PM
>>Subject: Re: HAProxy basic setup question
>> 
>>
>>
>>
>>On 12/8/12 8:30 AM, Hermes Flying wrote:
>>
>>So this would be e.g. Pacemaker? 
Yes
>>
>>
>>Also such a setup is considered a SPOF right?
>>>
No - Two switches, right?
>>
>>
>>
>
>
>

Re: HAProxy basic setup question

2012-12-08 Thread Hermes Flying
I see. With the 2 switches you mention I guess it would not be possible to set 
the 2 lixux which have a single port NIC.Right?




 From: David Coulson 
To: Hermes Flying  
Cc: "haproxy@formilux.org"  
Sent: Saturday, December 8, 2012 3:41 PM
Subject: Re: HAProxy basic setup question
 

Yep - The systems we use in clustered environments typically have 4 or 6 NICs 
for redundant front-end and back-end networks.

That's why I told you to pay someone to build it.


On 12/8/12 8:38 AM, Hermes Flying wrote:

Are you assuming multi-port NICs? Sorry if this is a trivial question but I am 
an application programmer and lack your background.
>
>
>
>
>____
> From: David Coulson 
>To: Hermes Flying  
>Cc: "haproxy@formilux.org"  
>Sent: Saturday, December 8, 2012 3:36 PM
>Subject: Re: HAProxy basic setup question
> 
>
>Well, since it's 2012 you use a switch instead of a hub. And as I described 
>earlier you can take two switches and connect systems to both, reducing the 
>risk of a hardware fault taking everything down. You use the bonding 
>capability in Linux to make the two NIC ports appear as one logical interface 
>in the OS.
>
>If you are so worried about building a massively
resilient system, you need to pay someone to build it
for you. In my experience, a poorly built 'redundant'
environment ends up with more downtime than a one with
multiple single points of failure.
>
>
>On 12/8/12 8:33 AM, Hermes Flying wrote:
>
>2 Linuxes connecting LBs over the same hub. Not sure what you mean by 2 
>switches
>>Isn't it SPOF? If the hub breaks then no load balancing
>>
>>
>>
>>
>>
>> From: David Coulson 
>>To: Hermes Flying  
>>Cc: "haproxy@formilux.org"  
>>Sent: Saturday, December 8, 2012 3:31 PM
>>Subject: Re: HAProxy basic setup question
>> 
>>
>>
>>
>>On 12/8/12 8:30 AM, Hermes Flying wrote:
>>
>>So this would be e.g. Pacemaker? 
Yes
>>
>>
>>Also such a setup is considered a SPOF right?
>>>
No - Two switches, right?
>>
>>
>>
>
>
>

Re: HAProxy basic setup question

2012-12-08 Thread Hermes Flying
Are you assuming multi-port NICs? Sorry if this is a trivial question but I am 
an application programmer and lack your background.




 From: David Coulson 
To: Hermes Flying  
Cc: "haproxy@formilux.org"  
Sent: Saturday, December 8, 2012 3:36 PM
Subject: Re: HAProxy basic setup question
 

Well, since it's 2012 you use a switch instead of a hub. And as I described 
earlier you can take two switches and connect systems to both, reducing the 
risk of a hardware fault taking everything down. You use the bonding capability 
in Linux to make the two NIC ports appear as one logical interface in the OS.

If you are so worried about building a massively resilient system,
you need to pay someone to build it for you. In my experience, a
poorly built 'redundant' environment ends up with more downtime than
a one with multiple single points of failure.


On 12/8/12 8:33 AM, Hermes Flying wrote:

2 Linuxes connecting LBs over the same hub. Not sure what you mean by 2 switches
>Isn't it SPOF? If the hub breaks then no load balancing
>
>
>
>
>________
> From: David Coulson 
>To: Hermes Flying  
>Cc: "haproxy@formilux.org"  
>Sent: Saturday, December 8, 2012 3:31 PM
>Subject: Re: HAProxy basic setup question
> 
>
>
>
>On 12/8/12 8:30 AM, Hermes Flying wrote:
>
>So this would be e.g. Pacemaker? 
Yes
>
>
>Also such a setup is considered a SPOF right?
>>
No - Two switches, right?
>
>
>

Re: HAProxy basic setup question

2012-12-08 Thread Hermes Flying
2 Linuxes connecting LBs over the same hub. Not sure what you mean by 2 switches
Isn't it SPOF? If the hub breaks then no load balancing




 From: David Coulson 
To: Hermes Flying  
Cc: "haproxy@formilux.org"  
Sent: Saturday, December 8, 2012 3:31 PM
Subject: Re: HAProxy basic setup question
 



On 12/8/12 8:30 AM, Hermes Flying wrote:

So this would be e.g. Pacemaker? 
Yes


Also such a setup is considered a SPOF right?
>
No - Two switches, right?

Re: HAProxy basic setup question

2012-12-08 Thread Hermes Flying
So this would be e.g. Pacemaker? Also such a setup is considered a SPOF right?




 From: David Coulson 
To: Hermes Flying  
Cc: "haproxy@formilux.org"  
Sent: Saturday, December 8, 2012 3:22 PM
Subject: Re: HAProxy basic setup question
 

No. HAProxy does not care if the systems are on the same subnet. Whatever you 
are using for VIP failover probably will though.

Most people use bonded interfaces and multiple switches. Nothing to
do with HAProxy.

David


On 12/8/12 8:20 AM, Hermes Flying wrote:

Hi,
>I wanted to ask:
>If I have linux-1 and linux-2 running HAProxy instances  each
and let's say that HAProxy in linux-1 is active and does load
balancing between linux-1 and linux-2 (both run Tomcat instance)
is there a requirement that the 2 linux machines be connected to
the same hub for load balancing/forwarding to succeed? If yes,
isn't this a single point of failure? If not, what is the
standard setup for what I describe?
>
>Thank you!
>
>
>

Re: HAproxy and detect split-brain (network failures)

2012-11-29 Thread Hermes Flying
Great help! Thank you for your time! Much appreciated!

 


 From: David Coulson 
To: Hermes Flying  
Cc: Baptiste ; "haproxy@formilux.org"  
Sent: Thursday, November 29, 2012 11:13 PM
Subject: Re: HAproxy and detect split-brain (network failures)
  

In general, yes, Pacemaker is reliable. If your config is wrong, you may still 
have an outage in the event of a failure.

That said, if you are a business and need support, you probably want
to use whatever clustering software ships with the distribution you
use. I belive SuSE uses pacemaker, but RedHat still uses rgmanager.
Pacemaker is tech preview in RHEL6 but will be mainline in 7. I
believe RedHat employ some core developers of pacemaker.

David


On 11/29/12 4:10 PM, Hermes Flying wrote:
 
Thank you for your help. 
>I take it that you are find Pacemaker reliable in your experience? Should I 
>look into it?  
>
> 
>From: David Coulson mailto:da...@davidcoulson.net
>To: Hermes Flying mailto:flyingher...@yahoo.com 
>Cc: Baptiste mailto:bed...@gmail.com; mailto:haproxy@formilux.org 
>mailto:haproxy@formilux.org 
>Sent: Thursday, November 29, 2012 11:04 PM
>Subject: Re: HAproxy and detect split-brain (network failures)
>  
>
>Again, you need to talk to the pacemaker people for actual clustering 
>information.
>
>The ping was so a node could detect it lost upstream
connectivity, and move the VIP, otherwise the VIP may
continue to run on a system which does not have access
to your network. This has nothing at all to do with
split brain.
>
>If you want to deal with split brain, add a third node.
Period. You also want to have redundant heartbeat
communication paths. You also want STONITH/fencing so if
one node detects the other is down it'll power it off or
crash it. I've not had issues with a two-node cluster
with two diverse backend communication links and fencing
enabled.
>
>David
>
>
>On 11/29/12 3:58 PM, Hermes Flying wrote:
> 
>"You can have pacemaker ping an IP (gateway for example) and migrate the VIP 
>based on that" 
>>How does this help for splitbrain? 
>>If I understand what you say, pacemaker will ping an IP and if successfull 
>>will assume that the other node has crashed. But what if the other node 
>>hasn't and it is just their communication link that failed? Won't both become 
>>primary? 
>>How does the ping help? 
>>  
>>
>> 
>>From: David Coulson mailto:da...@davidcoulson.net
>>To: Hermes Flying mailto:flyingher...@yahoo.com 
>>Cc: Baptiste mailto:bed...@gmail.com; mailto:haproxy@formilux.org 
>>mailto:haproxy@formilux.org 
>>Sent: Thursday, November 29, 2012 10:26 PM
>>Subject: Re: HAproxy and detect split-brain (network failures)
>>  
>>
>>
>>
>>On 11/29/12 3:11 PM, Hermes Flying wrote:
>> 
>>I see now!
>>>One last question since you are
using Pacemaker. Do you recommend it
for splitbrain so that I look into
that direction?
>>>   
>>Any two node cluster has risk of split
brain. if you implement fencing/STONITH, you
are in a better place. If you have a third
node, that's even better, even if it does
not actually run any services beyond the
cluster software
>>
>>I mean when you say that pacemaker restart HAProxy, does it detect network 
>>failures as well? Or only SW crashes?  
>>>I assume pacemaker will be aware of
both HAProxy1 and HAProxy2 in my
described deployment
>>>  
You can have pacemaker ping an IP (gateway for example) and migrate the VIP 
based on that. In my config I have haproxy configured as a cloned resource in 
pacemaker, so all nodes have the same pacemaker config for haproxy and it keeps 
haproxy running on all nodes all of the time.
>>  
>>
>>
>  
>
>

Re: HAproxy and detect split-brain (network failures)

2012-11-29 Thread Hermes Flying
Thank you for your help.
I take it that you are find Pacemaker reliable in your experience? Should I 
look into it? 
 


 From: David Coulson 
To: Hermes Flying  
Cc: Baptiste ; "haproxy@formilux.org"  
Sent: Thursday, November 29, 2012 11:04 PM
Subject: Re: HAproxy and detect split-brain (network failures)
  

Again, you need to talk to the pacemaker people for actual clustering 
information.

The ping was so a node could detect it lost upstream connectivity,
and move the VIP, otherwise the VIP may continue to run on a system
which does not have access to your network. This has nothing at all
to do with split brain.

If you want to deal with split brain, add a third node. Period. You
also want to have redundant heartbeat communication paths. You also
want STONITH/fencing so if one node detects the other is down it'll
power it off or crash it. I've not had issues with a two-node
cluster with two diverse backend communication links and fencing
enabled.

David


On 11/29/12 3:58 PM, Hermes Flying wrote:
 
"You can have pacemaker ping an IP (gateway for example) and migrate the VIP 
based on that" 
>How does this help for splitbrain? 
>If I understand what you say, pacemaker will ping an IP and if successfull 
>will assume that the other node has crashed. But what if the other node hasn't 
>and it is just their communication link that failed? Won't both become 
>primary? 
>How does the ping help? 
>  
>
> 
>From: David Coulson mailto:da...@davidcoulson.net
>To: Hermes Flying mailto:flyingher...@yahoo.com 
>Cc: Baptiste mailto:bed...@gmail.com; mailto:haproxy@formilux.org 
>mailto:haproxy@formilux.org 
>Sent: Thursday, November 29, 2012 10:26 PM
>Subject: Re: HAproxy and detect split-brain (network failures)
>  
>
>
>
>On 11/29/12 3:11 PM, Hermes Flying wrote:
> 
>I see now!
>>One last question since you are using Pacemaker.
Do you recommend it for splitbrain so that I
look into that direction?
>>   
>Any two node cluster has risk of split brain. if you
implement fencing/STONITH, you are in a better place. If
you have a third node, that's even better, even if it
does not actually run any services beyond the cluster
software
>
>I mean when you say that pacemaker restart HAProxy, does it detect network 
>failures as well? Or only SW crashes?  
>>I assume pacemaker will be aware of both
HAProxy1 and HAProxy2 in my described deployment
>>  
You can have pacemaker ping an IP (gateway for example) and migrate the VIP 
based on that. In my config I have haproxy configured as a cloned resource in 
pacemaker, so all nodes have the same pacemaker config for haproxy and it keeps 
haproxy running on all nodes all of the time.
>  
>
>

Re: HAproxy and detect split-brain (network failures)

2012-11-29 Thread Hermes Flying
Hi Owen,
 How does the heartbeat this help for splitbrain?
With heartbeat the nodes know that it can't talk to each other. They don't know 
if the other is down. If there is a different communication path between the 
nodes and the incoming requests, both can become primary assuming the other is 
down due to network failure of the communcation link
So how does this work for your system?
 


 From: Owen MArinas 
To: haproxy@formilux.org 
Sent: Thursday, November 29, 2012 10:40 PM
Subject: Re: HAproxy and detect split-brain (network failures)
  

we have exactly that setup with heartbeat, and 2 floating IPs. 
Working in production for 3 years now

Owen



On 29/11/2012 3:26 PM, David Coulson wrote:
 

>
>On 11/29/12 3:11 PM, Hermes Flying wrote:
> 
>I see now!
>>One last question since you are using Pacemaker. Do you
  recommend it for splitbrain so that I look into that
  direction?
>>   
>Any two node cluster has risk of split brain. if you implement
  fencing/STONITH, you are in a better place. If you have a third
  node, that's even better, even if it does not actually run any
  services beyond the cluster software
>
>I mean when you say that pacemaker restart HAProxy, does it detect network 
>failures as well? Or only SW crashes?  
>>I assume pacemaker will be aware of both HAProxy1 and
  HAProxy2 in my described deployment
>>  
You can have pacemaker ping an IP (gateway for example) and migrate the VIP 
based on that. In my config I have haproxy configured as a cloned resource in 
pacemaker, so all nodes have the same pacemaker config for haproxy and it keeps 
haproxy running on all nodes all of the time.
> 

Re: HAproxy and detect split-brain (network failures)

2012-11-29 Thread Hermes Flying
"You can have pacemaker ping an IP (gateway for example) and migratethe VIP 
based on that"
How does this help for splitbrain?
If I understand what you say, pacemaker will ping an IP and if successfull will 
assume that the other node has crashed. But what if the other node hasn't and 
it is just their communication link that failed? Won't both become primary?
How does the ping help?
 
 


 From: David Coulson 
To: Hermes Flying  
Cc: Baptiste ; "haproxy@formilux.org"  
Sent: Thursday, November 29, 2012 10:26 PM
Subject: Re: HAproxy and detect split-brain (network failures)
  



On 11/29/12 3:11 PM, Hermes Flying wrote:
 
I see now!
>One last question since you are using Pacemaker. Do you
recommend it for splitbrain so that I look into that
direction?
>   
Any two node cluster has risk of split brain. if you implement
fencing/STONITH, you are in a better place. If you have a third
node, that's even better, even if it does not actually run any
services beyond the cluster software

I mean when you say that pacemaker restart HAProxy, does it detect network 
failures as well? Or only SW crashes?  
>I assume pacemaker will be aware of both HAProxy1 and
HAProxy2 in my described deployment
>  
You can have pacemaker ping an IP (gateway for example) and migrate the VIP 
based on that. In my config I have haproxy configured as a cloned resource in 
pacemaker, so all nodes have the same pacemaker config for haproxy and it keeps 
haproxy running on all nodes all of the time.

Re: HAproxy and detect split-brain (network failures)

2012-11-29 Thread Hermes Flying
I see now!
One last question since you are using Pacemaker. Do you recommend it for 
splitbrain so that I look into that direction?
I mean when you say that pacemaker restart HAProxy, does it detect network 
failures as well? Or only SW crashes?  
I assume pacemaker will be aware of both HAProxy1 and HAProxy2 in my described 
deployment

 


 From: David Coulson 
To: Hermes Flying  
Cc: Baptiste ; "haproxy@formilux.org"  
Sent: Thursday, November 29, 2012 9:29 PM
Subject: Re: HAproxy and detect split-brain (network failures)
  

Both haproxy instances have the same config, with the tomcat instances with the 
same weight, etc. Run something like keepalived or pacemaker to manage a VIP 
between the two boxes. That's it. Not sure about keepalived, but pacemaker can 
make sure haproxy is running, then either restart it or move the VIP if it is 
not running.

David


On 11/29/12 2:27 PM, Hermes Flying wrote:
 
Something like the following: 
>  
> HAProxy1 > Tomcat1  
> |    +/\
> |   +
> |+--->Tomcat2
>+    /+\
>    + + 
>HAProxy2+++ 
>  
>HAProxy1 is in the same machine as Tomcat1 
>HAproxy2 is in the same machine as Tomcat2 
>HAProxy1 distributes the load among Tomcat1 and Tomcat2. 
>I erroneously thought that HAProxy2 would take over when HAProxy1 crashed to 
>distribute the load among Tomcat1/Tomcat2.   
>So if both are independent what can I do? 
>  
>
> 
>From: David Coulson mailto:da...@davidcoulson.net
>To: Hermes Flying mailto:flyingher...@yahoo.com 
>Cc: Baptiste mailto:bed...@gmail.com; mailto:haproxy@formilux.org 
>mailto:haproxy@formilux.org 
>Sent: Thursday, November 29, 2012 9:12 PM
>Subject: Re: HAproxy and detect split-brain (network failures)
>  
>
>Again, you are mixing everything up.
>
>HAProxy has it's own configuration - It defines what
nodes your port 80 traffic (or whatever) is routed to.
Haproxy does periodic health checks of these backend
services to make sure they are available for requests.
If you have multiple haproxy instances they will all
independently do health checks and not share any of that
information with each other. HAProxy will route traffic
to all systems defined as a backend for a particular
service based upon whatever criteria is in the haproxy
config. 
>
>You can run a two-node environment that is active/backup
from a VIP perspective, but active/active from a haproxy
service perspective - Each node would run Apache (or
whatever your service is) and haproxy would distribute
requests across both based on your haproxy config. But,
at any point in time only one node would actually be
routing requests through it's local instance of haproxy.
>
>I can't make it any simpler than that. Draw a diagram of
what you are trying to do if it doesn't make sense.
>
>
>
>On 11/29/12 2:06 PM, Hermes Flying wrote:
> 
>You are saying that one instance of HAProxy runs in each system and one 
>instance is assigned the VIP that clients hit-on (out of scope for HAProxy). 
>>But this HAProxy distributes the requests according to the load, either on 
>>system-A or system-B for which you seem to refer to as backup system. In what 
>>way are you now refering to it as backup system? Because I am interested in 
>>distributing the load to all the nodes.
>> 
>>
>> 
>>From: David Coulson mailto:da...@davidcoulson.net
>>To: Hermes Flying mailto:flyingher...@yahoo.com 
>>Cc: Baptiste mailto:bed...@gmail.com; mailto:haproxy@formilux.org 
>>mailto:haproxy@formilux.org 
>>Sent: Thursday, November 29, 2012 8:57 PM
>>Subject: Re: HAproxy and detect split-brain (network failures)
>>  
>>
>>You can do that, but haproxy doesn't have anything to do with the failover 
>>process, other than you run an instance of haproxy on one server, and another 
>>instance on your backup system. As I said, neither of the haproxy instances 
>>communicate anything, so all you need to do is move the IP clients are using 
>>from one server to the other in order to handle a failure. Moving the IP 
>>around is something keepalived, pacemaker, etc handles - Look at their 
>>documentation for specifics and challenges in a two-node config.
>>
>>HAProxy doesn't have a concent of primary
      and backup in terms of it's own instances.
  Each of them is stand alone. It's up to
   

Re: HAproxy and detect split-brain (network failures)

2012-11-29 Thread Hermes Flying
Something like the following:
 
 HAProxy1 > Tomcat1  
 |    +/\
 |   +
 |+--->Tomcat2
+    /+\
    + +
HAProxy2+++
 
HAProxy1 is in the same machine as Tomcat1
HAproxy2 is in the same machine as Tomcat2
HAProxy1 distributes the load among Tomcat1 and Tomcat2.
I erroneously thought that HAProxy2 would take over when HAProxy1 crashed to 
distribute the load among Tomcat1/Tomcat2.  
So if both are independent what can I do?
 
 


 From: David Coulson 
To: Hermes Flying  
Cc: Baptiste ; "haproxy@formilux.org"  
Sent: Thursday, November 29, 2012 9:12 PM
Subject: Re: HAproxy and detect split-brain (network failures)
  

Again, you are mixing everything up.

HAProxy has it's own configuration - It defines what nodes your port
80 traffic (or whatever) is routed to. Haproxy does periodic health
checks of these backend services to make sure they are available for
requests. If you have multiple haproxy instances they will all
independently do health checks and not share any of that information
with each other. HAProxy will route traffic to all systems defined
as a backend for a particular service based upon whatever criteria
is in the haproxy config. 

You can run a two-node environment that is active/backup from a VIP
perspective, but active/active from a haproxy service perspective -
Each node would run Apache (or whatever your service is) and haproxy
would distribute requests across both based on your haproxy config.
But, at any point in time only one node would actually be routing
requests through it's local instance of haproxy.

I can't make it any simpler than that. Draw a diagram of what you
are trying to do if it doesn't make sense.



On 11/29/12 2:06 PM, Hermes Flying wrote:
 
You are saying that one instance of HAProxy runs in each system and one 
instance is assigned the VIP that clients hit-on (out of scope for HAProxy). 
>But this HAProxy distributes the requests according to the load, either on 
>system-A or system-B for which you seem to refer to as backup system. In what 
>way are you now refering to it as backup system? Because I am interested in 
>distributing the load to all the nodes.
> 
>
> 
>From: David Coulson mailto:da...@davidcoulson.net
>To: Hermes Flying mailto:flyingher...@yahoo.com 
>Cc: Baptiste mailto:bed...@gmail.com; mailto:haproxy@formilux.org 
>mailto:haproxy@formilux.org 
>Sent: Thursday, November 29, 2012 8:57 PM
>Subject: Re: HAproxy and detect split-brain (network failures)
>  
>
>You can do that, but haproxy doesn't have anything to do with the failover 
>process, other than you run an instance of haproxy on one server, and another 
>instance on your backup system. As I said, neither of the haproxy instances 
>communicate anything, so all you need to do is move the IP clients are using 
>from one server to the other in order to handle a failure. Moving the IP 
>around is something keepalived, pacemaker, etc handles - Look at their 
>documentation for specifics and challenges in a two-node config.
>
>HAProxy doesn't have a concent of primary and backup
  in terms of it's own instances. Each of them is stand
  alone. It's up to you, based on your network/IP config
  which one has traffic routed to it. 
>
>David
>
>
>
>On 11/29/12 1:53 PM, Hermes Flying wrote:
> 
>But if I install 2 HAProxy as load balancers, doesn't one act as the primary 
>loadbalancer directing the load to the known servers while the secondary takes 
>over load distribution as soon as the heartbeat fails? I remember reading 
>this. Is this wrong? 
>>
>> 
>>From: David Coulson mailto:da...@davidcoulson.net
>>To: Hermes Flying mailto:flyingher...@yahoo.com 
>>Cc: Baptiste mailto:bed...@gmail.com; mailto:haproxy@formilux.org 
>>mailto:haproxy@formilux.org 
>>Sent: Thursday, November 29, 2012 8:39 PM
>>Subject: Re: HAproxy and detect split-brain (network failures)
>>  
>>
>>You are mixing two totally different things together.
>>
>>1) HAProxy will do periodic health checks
  of backend systems you are routing to.
  Depending if you configure something as
  'backup' or 'not backup' will determine
  if/how traffic is routed to it. The
  backend systems do not 'take over'.
  Haproxy just routes traffic to systems
  based on your configuration. The backend
  systems don't know/care about the

Re: HAproxy and detect split-brain (network failures)

2012-11-29 Thread Hermes Flying
You are saying that one instance of HAProxy runs in each system and one 
instance is assigned the VIP that clients hit-on (out of scope for HAProxy).
But this HAProxy distributes the requests according to the load, either on 
system-A or system-B for which you seem to refer to as backup system. In what 
way are you now refering to it as backup system? Because I am interested in 
distributing the load to all the nodes.


 


 From: David Coulson 
To: Hermes Flying  
Cc: Baptiste ; "haproxy@formilux.org"  
Sent: Thursday, November 29, 2012 8:57 PM
Subject: Re: HAproxy and detect split-brain (network failures)
  

You can do that, but haproxy doesn't have anything to do with the failover 
process, other than you run an instance of haproxy on one server, and another 
instance on your backup system. As I said, neither of the haproxy instances 
communicate anything, so all you need to do is move the IP clients are using 
from one server to the other in order to handle a failure. Moving the IP around 
is something keepalived, pacemaker, etc handles - Look at their documentation 
for specifics and challenges in a two-node config.

HAProxy doesn't have a concent of primary and backup in terms of
it's own instances. Each of them is stand alone. It's up to you,
based on your network/IP config which one has traffic routed to it. 

David



On 11/29/12 1:53 PM, Hermes Flying wrote:
 
But if I install 2 HAProxy as load balancers, doesn't one act as the primary 
loadbalancer directing the load to the known servers while the secondary takes 
over load distribution as soon as the heartbeat fails? I remember reading this. 
Is this wrong? 
>
> 
>From: David Coulson mailto:da...@davidcoulson.net
>To: Hermes Flying mailto:flyingher...@yahoo.com 
>Cc: Baptiste mailto:bed...@gmail.com; mailto:haproxy@formilux.org 
>mailto:haproxy@formilux.org 
>Sent: Thursday, November 29, 2012 8:39 PM
>Subject: Re: HAproxy and detect split-brain (network failures)
>  
>
>You are mixing two totally different things together.
>
>1) HAProxy will do periodic health checks of backend
systems you are routing to. Depending if you configure
something as 'backup' or 'not backup' will determine
if/how traffic is routed to it. The backend systems do
not 'take over'. Haproxy just routes traffic to systems
based on your configuration. The backend systems don't
know/care about the other backend nodes, unless your
application requires it which is a different story and
nothing to do with haproxy. HAproxy only cares about a
single instance of itself - If you have more than one
haproxy instance, they do NOT communicate anything
between each other.
>
>2) In terms of keepalived, pacemaker, etc, it makes no
difference which you use with haproxy - all they do is
manage the IP address(es) which haproxy is listening on,
and perhaps restart haproxy if it dies. Their
configuration and how you maintain quorum in a two-node
configuration is a question for one of their mailing
lists, or just read their documentation. I personally
use pacemaker.
>
>
>On 11/29/12 1:35 PM, Hermes Flying wrote:
> 
>Well I don't follow:
>>"You can have a pool of primary that it routes across, then backup systems 
>>that are only used when all primary systems are unavailable."
>>When you are saying that "the backup systems that are used when primary 
>>systems are unavailable", how do they decide to take over? How do they know 
>>that the other systems are unavailable? 
>>Are you saying that they depend on third party components like the ones you 
>>mentioned (Keepalived etc)? In this case, what is the most suitable tool to 
>>be used along with HAProxy? Is there a reference manual for this somewhere?
>>  
>>
>> 
>>From: David Coulson mailto:da...@davidcoulson.net
>>To: Hermes Flying mailto:flyingher...@yahoo.com 
>>Cc: Baptiste mailto:bed...@gmail.com; mailto:haproxy@formilux.org 
>>mailto:haproxy@formilux.org 
>>Sent: Thursday, November 29, 2012 8:21 PM
>>Subject: Re: HAproxy and detect split-brain (network failures)
>>  
>>
>>HAProxy only does primary and backup in terms of active backend systems - You 
>>can have a pool of primary that it routes across, then backup systems that 
>>are only used when all primary systems are unavailable.
>>
>>There is no concept of a cluster in terms
  of haproxy instances, although you can run
 

Re: HAproxy and detect split-brain (network failures)

2012-11-29 Thread Hermes Flying
But if I install 2 HAProxy as load balancers, doesn't one act as the primary 
loadbalancer directing the load to the known servers while the secondary takes 
over load distribution as soon as the heartbeat fails? I remember reading this. 
Is this wrong?

 


 From: David Coulson 
To: Hermes Flying  
Cc: Baptiste ; "haproxy@formilux.org"  
Sent: Thursday, November 29, 2012 8:39 PM
Subject: Re: HAproxy and detect split-brain (network failures)
  

You are mixing two totally different things together.

1) HAProxy will do periodic health checks of backend systems you are
routing to. Depending if you configure something as 'backup' or 'not
backup' will determine if/how traffic is routed to it. The backend
systems do not 'take over'. Haproxy just routes traffic to systems
based on your configuration. The backend systems don't know/care
about the other backend nodes, unless your application requires it
which is a different story and nothing to do with haproxy. HAproxy
only cares about a single instance of itself - If you have more than
one haproxy instance, they do NOT communicate anything between each
other.

2) In terms of keepalived, pacemaker, etc, it makes no difference
which you use with haproxy - all they do is manage the IP
address(es) which haproxy is listening on, and perhaps restart
haproxy if it dies. Their configuration and how you maintain quorum
in a two-node configuration is a question for one of their mailing
lists, or just read their documentation. I personally use pacemaker.


On 11/29/12 1:35 PM, Hermes Flying wrote:
 
Well I don't follow:
>"You can have a pool of primary that it routes across, then backup systems 
>that are only used when all primary systems are unavailable."
>When you are saying that "the backup systems that are used when primary 
>systems are unavailable", how do they decide to take over? How do they know 
>that the other systems are unavailable? 
>Are you saying that they depend on third party components like the ones you 
>mentioned (Keepalived etc)? In this case, what is the most suitable tool to be 
>used along with HAProxy? Is there a reference manual for this somewhere?
>  
>
> 
>From: David Coulson mailto:da...@davidcoulson.net
>To: Hermes Flying mailto:flyingher...@yahoo.com 
>Cc: Baptiste mailto:bed...@gmail.com; mailto:haproxy@formilux.org 
>mailto:haproxy@formilux.org 
>Sent: Thursday, November 29, 2012 8:21 PM
>Subject: Re: HAproxy and detect split-brain (network failures)
>  
>
>HAProxy only does primary and backup in terms of active backend systems - You 
>can have a pool of primary that it routes across, then backup systems that are 
>only used when all primary systems are unavailable.
>
>There is no concept of a cluster in terms of haproxy
  instances, although you can run more than one and
  manage them via something like pacemaker, keepalived
  or rgmanager.
>
>
>On 11/29/12 1:19 PM, Hermes Flying wrote:
> 
>Hi, 
>>From a quick look into HAProxy, I see that it is a Primary/backup 
>>architecture. So isn't ensuring that both "nodes" don't become primary part 
>>of HAProxy's primary/backup "protocol" ? 
>>
>> 
>>From: Baptiste mailto:bed...@gmail.com
>>To: Hermes Flying mailto:flyingher...@yahoo.com 
>>Cc: mailto:haproxy@formilux.org mailto:haproxy@formilux.org 
>>Sent: Thursday, November 29, 2012 3:02 PM
>>Subject: Re: HAproxy and detect split-brain (network failures)
>>  
>>Hi,
>>
>>This is not HAProxy's role, this is the tool
  you use to ensure high
>>availability to do that.
>>
>>I could see a way where HAProxy can report one
  interface failing,
>>maybe this could help you to detect if you're
  in a split brain
>>situation.
>>
>>cheers
>>
>>
>>
>>On Thu, Nov 29, 2012 at 11:51 AM, Hermes
  Flying  wrote:
>>> Hi,
>>> I am looking into using HAProxy as our
  load balancer.
>>> I see that you are using a primary/backup
  approach. I was wondering how does
>>> HAProxy (if it does) address split-brain
  situation? Do you have a mechanism
>>> to detect and avoid it? Do you have some
  standard recommendation to all
>>> those using your solution?
>>>
>>> Thanks
>>
>>
>>
>  
>
>

Re: HAproxy and detect split-brain (network failures)

2012-11-29 Thread Hermes Flying
Well I don't follow:  
"You can have a pool of primary that it routes across, thenbackup systems 
that are only used when all primary systems areunavailable."  
When you are saying that "the backup systems that are used when primary systems 
are unavailable", how do they decide to take over? How do they know that the 
other systems are unavailable?
Are you saying that they depend on third party components like the ones you 
mentioned (Keepalived etc)? In this case, what is the most suitable tool to be 
used along with HAProxy? Is there a reference manual for this somewhere?



 


 From: David Coulson 
To: Hermes Flying  
Cc: Baptiste ; "haproxy@formilux.org"  
Sent: Thursday, November 29, 2012 8:21 PM
Subject: Re: HAproxy and detect split-brain (network failures)
  

HAProxy only does primary and backup in terms of active backend systems - You 
can have a pool of primary that it routes across, then backup systems that are 
only used when all primary systems are unavailable.

There is no concept of a cluster in terms of haproxy instances,
although you can run more than one and manage them via something
like pacemaker, keepalived or rgmanager.


On 11/29/12 1:19 PM, Hermes Flying wrote:
 
Hi, 
>From a quick look into HAProxy, I see that it is a Primary/backup 
>architecture. So isn't ensuring that both "nodes" don't become primary part of 
>HAProxy's primary/backup "protocol" ? 
>
> 
>From: Baptiste mailto:bed...@gmail.com
>To: Hermes Flying mailto:flyingher...@yahoo.com 
>Cc: mailto:haproxy@formilux.org mailto:haproxy@formilux.org 
>Sent: Thursday, November 29, 2012 3:02 PM
>Subject: Re: HAproxy and detect split-brain (network failures)
>  
>Hi,
>
>This is not HAProxy's role, this is the tool you use to
ensure high
>availability to do that.
>
>I could see a way where HAProxy can report one interface
failing,
>maybe this could help you to detect if you're in a split
brain
>situation.
>
>cheers
>
>
>
>On Thu, Nov 29, 2012 at 11:51 AM, Hermes Flying  wrote:
>> Hi,
>> I am looking into using HAProxy as our load balancer.
>> I see that you are using a primary/backup approach. I
was wondering how does
>> HAProxy (if it does) address split-brain situation? Do
you have a mechanism
>> to detect and avoid it? Do you have some standard
recommendation to all
>> those using your solution?
>>
>> Thanks
>
>
>

Re: HAproxy and detect split-brain (network failures)

2012-11-29 Thread Hermes Flying
Hi Robert,
But with keep alive you can only detect that the 2 nodes can not contact each 
other (network failure). How do you know if the other node/process actually 
crashed so that the secondary can become the primary? 
 


 From: Robert Snyder 
To: Baptiste  
Cc: Hermes Flying ; "haproxy@formilux.org" 
 
Sent: Thursday, November 29, 2012 3:13 PM
Subject: Re: HAproxy and detect split-brain (network failures)
  
Hi,

We use Keepalived http://www.keepalived.org/index.html to manage the Virtual IP 
address management between our two physical HAproxy servers. It maintains 
heartbeat between the servers, and in the event of failure passes ensures that 
the VIPs are migrated and the service is brought up. Also handles migration 
back after a restart of our primary, so that if available, that is the server 
that owns the IPs. 

We use Mercurial to manage the configuration files between the two servers to 
maintain consistency so that we are prepared for consistent fail overs. 

Robert

On Nov 29, 2012, at 8 :02 AM, Baptiste  wrote:

> Hi,
> 
> This is not HAProxy's role, this is the tool you use to ensure high
> availability to do that.
> 
> I could see a way where HAProxy can report one interface failing,
> maybe this could help you to detect if you're in a split brain
> situation.
> 
> cheers
> 
> 
> 
> On Thu, Nov 29, 2012 at 11:51 AM, Hermes Flying  
> wrote:
>> Hi,
>> I am looking into using HAProxy as our load balancer.
>> I see that you are using a primary/backup approach. I was wondering how does
>> HAProxy (if it does) address split-brain situation? Do you have a mechanism
>> to detect and avoid it? Do you have some standard recommendation to all
>> those using your solution?
>> 
>> Thanks
> 





Robert Snyder
Outreach Technology Services
The Pennsylvania State University
The 329 Building, Suite 306E
University Park  PA  16802
Phone: 814-865-0912
E-mail: rsny...@psu.edu

Re: HAproxy and detect split-brain (network failures)

2012-11-29 Thread Hermes Flying
Hi,
>From a quick look into HAProxy, I see that it is a Primary/backup 
>architecture. So isn't ensuring that both "nodes" don't become primary part of 
>HAProxy's primary/backup "protocol" ?
 

________
 From: Baptiste 
To: Hermes Flying  
Cc: "haproxy@formilux.org"  
Sent: Thursday, November 29, 2012 3:02 PM
Subject: Re: HAproxy and detect split-brain (network failures)
  
Hi,

This is not HAProxy's role, this is the tool you use to ensure high
availability to do that.

I could see a way where HAProxy can report one interface failing,
maybe this could help you to detect if you're in a split brain
situation.

cheers



On Thu, Nov 29, 2012 at 11:51 AM, Hermes Flying  wrote:
> Hi,
> I am looking into using HAProxy as our load balancer.
> I see that you are using a primary/backup approach. I was wondering how does
> HAProxy (if it does) address split-brain situation? Do you have a mechanism
> to detect and avoid it? Do you have some standard recommendation to all
> those using your solution?
>
> Thanks