On 13/08/14 08:37 AM, Andrey Borzenkov wrote:
Hi,
Sorry for may be basic question, but it is my first Linux HA project.
I (will) have two node cluster in active/passive configuration -
single application on one node and second as standby; application si
implemented as master/slave clone. Is it possible to prioritize node,
that have active application? So that in case of split brain passive
node gets killed?
Usually this is done using staggered delay for fencing requests. I
think that it may be possible to implement in pacemaker using rules,
but I'm a bit uneasy about how to express it. Rule should select a
node where master is currently active, not fixed node.
Thank you in advance!
-andrei
Hi Andrei,
"Basic questions" is how you avoid mistakes, so please never
apologize for asking them. I sure ask my own basic questions... :P
First up, a little semantics; "split-brain" is what happens when
fencing fails. Your asking what happens when the connection between the
nodes break when both nodes are otherwise happy (sometimes called a
"partitioning of the cluster", though I don't think there is an official
term).
You are right in guessing that it is "delay" to set this. You add the
attribute 'delay="15"' to the fence method. You put the delay attribute
on the node you want to win. Here is an example;
http://clusterlabs.org/wiki/STONITH_Levels#Configuring_The_Fence_Methods
In that example, the node called "pcmk-1" has the 'delay' set, so it
will get a 15 second head start in fencing "pcmk-2". It works that way
because what it does is tell the cluster "If you want to fence 'pcmk-1',
pause for 15 seconds before doing so". So in a 2-node cluster
partitioning, both would initiate a fence against the other immediately,
but pcmk-2 would pause before fencing pcmk-1, where pcmk-1 would *not*
pause before fencing pcmk-2.
There is another important note when using IPMI-based fencing in
2-node clusters.
If you have acpid running on the nodes, then when the node is fenced
over IPMI, it take ~4 seconds to be forced off. This is because the
power button is effectively pressed and held. This four seconds is time
in which the node could get a fence call started against the peer,
causing a dual-fence. On the surface, the 'delay="15"' should deal with
this, because 15 > 4, but there are corner cases where the delay alone
isn't enough.
Consider a broadcast storm on your network that takes, say, 30
seconds to fix. By the time that has passed, the delay has expired and
both nodes will be sitting there trying to fence the other. When the
storm ends, both will potentially immediately begin a fence against the
other.
So to reduce the chance of a dual-fence in this corner case, you want
to disable acpid. Most servers will react to a power button even by
nearly instantly powering off, thus further reducing the chance of a
dual fence because now, even if the delay has failed, there is only a
fraction of a second between the slower node being fenced and being
disabled.
hth
digimer
--
Digimer
Papers and Projects: https://alteeve.ca/w/
What if the cure for cancer is trapped in the mind of a person without
access to education?
_______________________________________________
Pacemaker mailing list: Pacemaker@oss.clusterlabs.org
http://oss.clusterlabs.org/mailman/listinfo/pacemaker
Project Home: http://www.clusterlabs.org
Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
Bugs: http://bugs.clusterlabs.org
