Re: [ClusterLabs] Colocation constraint for grouping all master-mode stateful resources with important stateless resources

2018-03-26 Thread Ken Gaillot 
On Mon, 2018-03-26 at 15:42 +, Sam Gardner wrote:
> Thanks, Andrei and Alberto.
> 
> Alberto, I will look into the node-constraint parameters, though I
> suspect Andrei is correct - my "base" resource is DRBDFS in this
> case, and the issue I'm seeing is that a failure in my secondary
> resources does not cause the other secondary resources nor the "base"
> resource to move to the other node.
> 
> Andrei, I have no restrictions on the particulars of the rules that
> I'm putting in place - I can completely discard the rules that I have
> implemented already.
> 
> Here's a simple diagram:
> https://imgur.com/a/5LTmJ
> 
> These are my restrictions:
> 1) If any of DRBD-Master, DRBDFS, INIF-Master, or OUTIF-Master moves
> to D2, all other resources should move to D2.
> 2) If DRBDFS or DRBD-Master cannot run on either D1 or D2, all other
> resources should be stopped.
> 3) If INIF-Master or OUTIF-Master cannot run on either D1 or D2, no
> other resources should be stopped.

4) Keep INIF-Master with a working IN interface and OUTIF-Master with a
working OUT interface.

One problem is that 4) conflicts with 1). It's possible for INIF to be
working on only one node and OUTIF to be working only on the other
node.

I'm thinking you want something like this:

* Don't constrain DRBD-Master

* Colocate DRBDFS with DRBD-Master, +INFINITY

* Colocate INIF-Master and OUTIF-Master with DRBDFS, +INFINITY (using
separate individual constraints)

* Keep your INIF/OUTIF rules, but use a finite negative score. That
way, they *must* stay with DRBFDS and DRBD-Master (due to the previous
constraints), but will *prefer* a node where their interface is up. If
one of them is more important than the other, give it a stronger score,
to break the tie when one interface is up on each node.

* Order DRBDFS start after DRBD-Master promote

No order is necessary on INIF/OUTIF, since an IP can work regardless of
the file system.

That should meet your intentions.

> This sounds like a particular constraint that may not be possible to
> do per our discussions in this thread.
> 
> I can get pretty close with a workaround - I'm using ethmonitor on
> the Master/Slave resources as you can see in the config, so if I
> create new "heartbeat:Dummy" active resources with the same
> ethmonitor location constraint, unplugging the interface will move
> everything over.
> 
> However, a failure of a different type on the master/slave VIPs that
> would not also be apparent on the dummy base resource would not cause
> a failover of the entire group, which isn't ideal (though admittedly
> unlikely in this particular use case).
> 
> Thanks much for all of the help,
> -- 
> Sam Gardner
> Trustwave | SMART SECURITY ON DEMAND
> 
> 
> 
> 
> 
> 
> 
> 
> On 3/25/18, 6:06 AM, "Users on behalf of Andrei Borzenkov"  n...@clusterlabs.org on behalf of arvidj...@gmail.com> wrote:
> 
> > 25.03.2018 10:21, Alberto Mijares пишет:
> > > On Sat, Mar 24, 2018 at 2:16 PM, Andrei Borzenkov  > > l.com> wrote:
> > > > 23.03.2018 20:42, Sam Gardner пишет:
> > > > > Thanks, Ken.
> > > > > 
> > > > > I just want all master-mode resources to be running wherever
> > > > > DRBDFS is running (essentially). If the cluster detects that
> > > > > any of the master-mode resources can't run on the current
> > > > > node (but can run on the other per ethmon), all other master-
> > > > > mode resources as well as DRBDFS should move over to the
> > > > > other node.
> > > > > 
> > > > > The current set of constraints I have will let DRBDFS move to
> > > > > the standby node and "take" the Master mode resources with
> > > > > it, but the Master mode resources failing over to the other
> > > > > node won't take the other Master resources or DRBDFS.
> > > > > 
> > > > 
> > > > I do not think it is possible. There is no way to express
> > > > symmetrical
> > > > colocation rule like "always run A and B together". You start
> > > > with A and
> > > > place B relative to A; but then A is not affected by B's state.
> > > > Attempting now to place A relative to B will result in a loop
> > > > and is
> > > > ignored. See also old discussion:
> > > > 
> > > 
> > > 
> > > It is possible. Check this thread
> > > https://scanmail.trustwave.com/?c=4062=qYK32i8YnPIdkrPQRoURDTOy
> > > qGVIytWo2-
> > > H2bJ__2w=5=https%3a%2f%2flists%2eclusterlabs%2eorg%2fpipermai
> > > l%2fusers%2f2017-November%2f006788%2ehtml
> > > 
> > 
> > I do not see how it answers the question. It explains how to use
> > other
> > criteria than node name for colocating resources, but it does not
> > change
> > basic fact that colocating is asymmetrical. Actually this thread
> > explicitly suggests "Pick one resource as your base resource that
> > everything else should go along with".
> > 
> > If you you actually have configuration that somehow implements
> > symmetrical colocation between resources, I would appreciate if you
> > could post your configuration.
> > 
> > Regarding the original problem, the root cause is

Re: [ClusterLabs] Colocation constraint for grouping all master-mode stateful resources with important stateless resources

2018-03-26 Thread Alberto Mijares 
>>
>> It is possible. Check this thread
>> https://lists.clusterlabs.org/pipermail/users/2017-November/006788.html
>>
>
> I do not see how it answers the question. It explains how to use other
> criteria than node name for colocating resources, but it does not change
> basic fact that colocating is asymmetrical. Actually this thread
> explicitly suggests "Pick one resource as your base resource that
> everything else should go along with".
>


To be honest, I didn't fully read the OP. I just read a fragment and
it looked to me like the grouping constraint could work.

Sorry, guys. Please ignore my post.

Best regards,


Alberto Mijares
___
Users mailing list: Users@clusterlabs.org
https://lists.clusterlabs.org/mailman/listinfo/users

Project Home: http://www.clusterlabs.org
Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
Bugs: http://bugs.clusterlabs.org

Re: [ClusterLabs] Colocation constraint for grouping all master-mode stateful resources with important stateless resources

2018-03-26 Thread Sam Gardner 
Thanks, Andrei and Alberto.

Alberto, I will look into the node-constraint parameters, though I suspect 
Andrei is correct - my "base" resource is DRBDFS in this case, and the issue 
I'm seeing is that a failure in my secondary resources does not cause the other 
secondary resources nor the "base" resource to move to the other node.

Andrei, I have no restrictions on the particulars of the rules that I'm putting 
in place - I can completely discard the rules that I have implemented already.

Here's a simple diagram:
https://imgur.com/a/5LTmJ

These are my restrictions:
1) If any of DRBD-Master, DRBDFS, INIF-Master, or OUTIF-Master moves to D2, all 
other resources should move to D2.
2) If DRBDFS or DRBD-Master cannot run on either D1 or D2, all other resources 
should be stopped.
3) If INIF-Master or OUTIF-Master cannot run on either D1 or D2, no other 
resources should be stopped.


This sounds like a particular constraint that may not be possible to do per our 
discussions in this thread.

I can get pretty close with a workaround - I'm using ethmonitor on the 
Master/Slave resources as you can see in the config, so if I create new 
"heartbeat:Dummy" active resources with the same ethmonitor location 
constraint, unplugging the interface will move everything over.

However, a failure of a different type on the master/slave VIPs that would not 
also be apparent on the dummy base resource would not cause a failover of the 
entire group, which isn't ideal (though admittedly unlikely in this particular 
use case).

Thanks much for all of the help,
-- 
Sam Gardner
Trustwave | SMART SECURITY ON DEMAND








On 3/25/18, 6:06 AM, "Users on behalf of Andrei Borzenkov" 
 wrote:

>25.03.2018 10:21, Alberto Mijares пишет:
>> On Sat, Mar 24, 2018 at 2:16 PM, Andrei Borzenkov  
>> wrote:
>>> 23.03.2018 20:42, Sam Gardner пишет:
 Thanks, Ken.

 I just want all master-mode resources to be running wherever DRBDFS is 
 running (essentially). If the cluster detects that any of the master-mode 
 resources can't run on the current node (but can run on the other per 
 ethmon), all other master-mode resources as well as DRBDFS should move 
 over to the other node.

 The current set of constraints I have will let DRBDFS move to the standby 
 node and "take" the Master mode resources with it, but the Master mode 
 resources failing over to the other node won't take the other Master 
 resources or DRBDFS.

>>>
>>> I do not think it is possible. There is no way to express symmetrical
>>> colocation rule like "always run A and B together". You start with A and
>>> place B relative to A; but then A is not affected by B's state.
>>> Attempting now to place A relative to B will result in a loop and is
>>> ignored. See also old discussion:
>>>
>> 
>> 
>> It is possible. Check this thread
>> https://scanmail.trustwave.com/?c=4062=qYK32i8YnPIdkrPQRoURDTOyqGVIytWo2-H2bJ__2w=5=https%3a%2f%2flists%2eclusterlabs%2eorg%2fpipermail%2fusers%2f2017-November%2f006788%2ehtml
>> 
>
>I do not see how it answers the question. It explains how to use other
>criteria than node name for colocating resources, but it does not change
>basic fact that colocating is asymmetrical. Actually this thread
>explicitly suggests "Pick one resource as your base resource that
>everything else should go along with".
>
>If you you actually have configuration that somehow implements
>symmetrical colocation between resources, I would appreciate if you
>could post your configuration.
>
>Regarding the original problem, the root cause is slightly different though.
>
>@Sam, the behavior you describe is correct for your constraints that you
>show. When colocating with resource set, all resources in the set must
>be active on the same node. It means that in your case of
>
>  id="pcs_rsc_colocation_set_drbdfs_set_drbd.master_inside-interface-sameip.master_outside-interface-sameip.master"
>score="INFINITY">
>   
> 
>   
>   id="pcs_rsc_set_drbd.master_inside-interface-sameip.master_outside-interface-sameip.master"
>role="Master" sequential="false">
> 
> 
> 
>   
>  
>
>if one IP resource (master) is moved to another node, dependent resource
>(drbdfs) simply cannot run anywhere.
>
>Before discussing low level pacemaker implementation you really need to
>have high level model of resources relationship. On one hand you
>apparently intend to always run everything on the same node - on the
>other hand you have two rules that independently decide where to place
>two resources. That does not fit together.
>___
>Users mailing list: Users@clusterlabs.org
>https://scanmail.trustwave.com/?c=4062=qYK32i8YnPIdkrPQRoURDTOyqGVIytWo2-H0aZn72g=5=https%3a%2f%2flists%2eclusterlabs%2eorg%2fmailman%2flistinfo%2fusers
>
>Project Home:

Re: [ClusterLabs] Colocation constraint for grouping all master-mode stateful resources with important stateless resources

2018-03-25 Thread Andrei Borzenkov 
25.03.2018 10:21, Alberto Mijares пишет:
> On Sat, Mar 24, 2018 at 2:16 PM, Andrei Borzenkov  wrote:
>> 23.03.2018 20:42, Sam Gardner пишет:
>>> Thanks, Ken.
>>>
>>> I just want all master-mode resources to be running wherever DRBDFS is 
>>> running (essentially). If the cluster detects that any of the master-mode 
>>> resources can't run on the current node (but can run on the other per 
>>> ethmon), all other master-mode resources as well as DRBDFS should move over 
>>> to the other node.
>>>
>>> The current set of constraints I have will let DRBDFS move to the standby 
>>> node and "take" the Master mode resources with it, but the Master mode 
>>> resources failing over to the other node won't take the other Master 
>>> resources or DRBDFS.
>>>
>>
>> I do not think it is possible. There is no way to express symmetrical
>> colocation rule like "always run A and B together". You start with A and
>> place B relative to A; but then A is not affected by B's state.
>> Attempting now to place A relative to B will result in a loop and is
>> ignored. See also old discussion:
>>
> 
> 
> It is possible. Check this thread
> https://lists.clusterlabs.org/pipermail/users/2017-November/006788.html
> 

I do not see how it answers the question. It explains how to use other
criteria than node name for colocating resources, but it does not change
basic fact that colocating is asymmetrical. Actually this thread
explicitly suggests "Pick one resource as your base resource that
everything else should go along with".

If you you actually have configuration that somehow implements
symmetrical colocation between resources, I would appreciate if you
could post your configuration.

Regarding the original problem, the root cause is slightly different though.

@Sam, the behavior you describe is correct for your constraints that you
show. When colocating with resource set, all resources in the set must
be active on the same node. It means that in your case of

  

  


  
  
  

  

if one IP resource (master) is moved to another node, dependent resource
(drbdfs) simply cannot run anywhere.

Before discussing low level pacemaker implementation you really need to
have high level model of resources relationship. On one hand you
apparently intend to always run everything on the same node - on the
other hand you have two rules that independently decide where to place
two resources. That does not fit together.
___
Users mailing list: Users@clusterlabs.org
https://lists.clusterlabs.org/mailman/listinfo/users

Project Home: http://www.clusterlabs.org
Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
Bugs: http://bugs.clusterlabs.org

Re: [ClusterLabs] Colocation constraint for grouping all master-mode stateful resources with important stateless resources

2018-03-25 Thread Alberto Mijares 
On Sat, Mar 24, 2018 at 2:16 PM, Andrei Borzenkov  wrote:
> 23.03.2018 20:42, Sam Gardner пишет:
>> Thanks, Ken.
>>
>> I just want all master-mode resources to be running wherever DRBDFS is 
>> running (essentially). If the cluster detects that any of the master-mode 
>> resources can't run on the current node (but can run on the other per 
>> ethmon), all other master-mode resources as well as DRBDFS should move over 
>> to the other node.
>>
>> The current set of constraints I have will let DRBDFS move to the standby 
>> node and "take" the Master mode resources with it, but the Master mode 
>> resources failing over to the other node won't take the other Master 
>> resources or DRBDFS.
>>
>
> I do not think it is possible. There is no way to express symmetrical
> colocation rule like "always run A and B together". You start with A and
> place B relative to A; but then A is not affected by B's state.
> Attempting now to place A relative to B will result in a loop and is
> ignored. See also old discussion:
>


It is possible. Check this thread
https://lists.clusterlabs.org/pipermail/users/2017-November/006788.html

Best regards,


Alberto Mijares
___
Users mailing list: Users@clusterlabs.org
https://lists.clusterlabs.org/mailman/listinfo/users

Project Home: http://www.clusterlabs.org
Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
Bugs: http://bugs.clusterlabs.org

Re: [ClusterLabs] Colocation constraint for grouping all master-mode stateful resources with important stateless resources

2018-03-24 Thread Andrei Borzenkov 
23.03.2018 20:42, Sam Gardner пишет:
> Thanks, Ken.
> 
> I just want all master-mode resources to be running wherever DRBDFS is 
> running (essentially). If the cluster detects that any of the master-mode 
> resources can't run on the current node (but can run on the other per 
> ethmon), all other master-mode resources as well as DRBDFS should move over 
> to the other node.
> 
> The current set of constraints I have will let DRBDFS move to the standby 
> node and "take" the Master mode resources with it, but the Master mode 
> resources failing over to the other node won't take the other Master 
> resources or DRBDFS.
> 

I do not think it is possible. There is no way to express symmetrical
colocation rule like "always run A and B together". You start with A and
place B relative to A; but then A is not affected by B's state.
Attempting now to place A relative to B will result in a loop and is
ignored. See also old discussion:

https://lists.gt.net/linuxha/pacemaker/68099


> As a side note, there are other resources I have in play (some active/passive 
> like DRBDFS, some Master/Slave like the ship resources) that are related, but 
> not shown here - I'm just having a hard time reasoning about the generalized 
> form that my constraints should take to make this sort of thing work.
> 

___
Users mailing list: Users@clusterlabs.org
https://lists.clusterlabs.org/mailman/listinfo/users

Project Home: http://www.clusterlabs.org
Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
Bugs: http://bugs.clusterlabs.org

Re: [ClusterLabs] Colocation constraint for grouping all master-mode stateful resources with important stateless resources

2018-03-23 Thread Sam Gardner 
The shared-ip resource agents are our own custom RA - they do what we want and 
the failover mechanism for the single resource works fine as far as I can tell 
- using conntrackd or some other non-IP master/slave resource would be the same 
sort of failure that we're running into because the constraints aren't set up 
correctly.
-- 
Sam Gardner
Trustwave | SMART SECURITY ON DEMAND









On 3/23/18, 12:42 PM, "Users on behalf of Sam Gardner" 
 wrote:

>Thanks, Ken.
>
>I just want all master-mode resources to be running wherever DRBDFS is running 
>(essentially). If the cluster detects that any of the master-mode resources 
>can't run on the current node (but can run on the other per ethmon), all other 
>master-mode resources as well as DRBDFS should move over to the other node.
>
>The current set of constraints I have will let DRBDFS move to the standby node 
>and "take" the Master mode resources with it, but the Master mode resources 
>failing over to the other node won't take the other Master resources or DRBDFS.
>
>As a side note, there are other resources I have in play (some active/passive 
>like DRBDFS, some Master/Slave like the ship resources) that are related, but 
>not shown here - I'm just having a hard time reasoning about the generalized 
>form that my constraints should take to make this sort of thing work.
>-- 
>Sam Gardner
>Trustwave | SMART SECURITY ON DEMAND
>
>
>
>
>
>
>
>
>On 3/23/18, 12:34 PM, "Users on behalf of Ken Gaillot" 
> wrote:
>
>>On Tue, 2018-03-20 at 16:34 +, Sam Gardner wrote:
>>> Hi All -
>>> 
>>> I've implemented a simple two-node cluster with DRBD and a couple of
>>> network-based Master/Slave resources.
>>> 
>>> Using the ethmonitor RA, I set up failover whenever the
>>> Master/Primary node loses link on the specified ethernet physical
>>> device by constraining the Master role only on nodes where the ethmon
>>> variable is "1".
>>> 
>>> Something is going wrong with my colocation constraint, however - if
>>> I set up the DRBDFS resource to monitor link on eth1, unplugging eth1
>>> on the Primary node causes a failover as expected - all Master
>>> resources are demoted to "slave" and promoted on the opposite node,
>>> and the "normal" DRBDFS moves to the other node as expected.
>>> 
>>> However, if I put the same ethmonitor constraint on the network-based 
>>> Master/Slave resource, only that specific resource fails over -
>>> DRBDFS stays in the same location (though it stops) as do the other
>>> Master/Slave resources.
>>> 
>>> This *smells* like a constraints issue to me - does anyone know what
>>> I might be doing wrong?
>>>
>>> PCS before:
>>> Cluster name: 
>>> http://scanmail.trustwave.com/?c=4062=mby12vU1sHtzTmhG_RQD80FeoJfoSXp5PKS2oHmF9g=5=http%3a%2f%2fnode1%2ehostname%2ecom%5fnode2%2ehostname%2ecom
>>> Stack: corosync
>>> Current DC: node2.hostname.com_0 (version 1.1.16-12.el7_4.4-94ff4df)
>>> - partition with quorum
>>> Last updated: Tue Mar 20 16:25:47 2018
>>> Last change: Tue Mar 20 16:00:33 2018 by hacluster via crmd on
>>> node2.hostname.com_0
>>> 
>>> 2 nodes configured
>>> 11 resources configured
>>> 
>>> Online: [ node1.hostname.com_0 node2.hostname.com_0 ]
>>> 
>>> Full list of resources:
>>> 
>>>  Master/Slave Set: drbd.master [drbd.slave]
>>>  Masters: [ node1.hostname.com_0 ]
>>>  Slaves: [ node2.hostname.com_0 ]
>>>  drbdfs (ocf::heartbeat:Filesystem):Started node1.hostname.com_0
>>>  Master/Slave Set: inside-interface-sameip.master [inside-interface-
>>> sameip.slave]
>>>  Masters: [ node1.hostname.com_0 ]
>>>  Slaves: [ node2.hostname.com_0 ]
>>>  Master/Slave Set: outside-interface-sameip.master [outside-
>>> interface-sameip.slave]
>>>  Masters: [ node1.hostname.com_0 ]
>>>  Slaves: [ node2.hostname.com_0 ]
>>>  Clone Set: monitor-eth1-clone [monitor-eth1]
>>>  Started: [ node1.hostname.com_0 node2.hostname.com_0 ]
>>>  Clone Set: monitor-eth2-clone [monitor-eth2]
>>>  Started: [ node1.hostname.com_0 node2.hostname.com_0 ]
>>
>>What agent are the two IP resources using? I'm not familiar with any IP
>>resource agents that are master/slave clones.
>>
>>> Daemon Status:
>>>   corosync: active/enabled
>>>   pacemaker: active/enabled
>>>   pcsd: inactive/disabled
>>> 
>>> PCS after:
>>> Cluster name: 
>>> http://scanmail.trustwave.com/?c=4062=mby12vU1sHtzTmhG_RQD80FeoJfoSXp5PKS2oHmF9g=5=http%3a%2f%2fnode1%2ehostname%2ecom%5fnode2%2ehostname%2ecom
>>> Stack: corosync
>>> Current DC: node2.hostname.com_0 (version 1.1.16-12.el7_4.4-94ff4df)
>>> - partition with quorum
>>> Last updated: Tue Mar 20 16:29:40 2018
>>> Last change: Tue Mar 20 16:00:33 2018 by hacluster via crmd on
>>> node2.hostname.com_0
>>> 
>>> 2 nodes configured
>>> 11 resources configured
>>> 
>>> Online: [ node1.hostname.com_0 node2.hostname.com_0 ]
>>> 
>>> Full list of resources:
>>> 
>>>  Master/Slave Set:

Re: [ClusterLabs] Colocation constraint for grouping all master-mode stateful resources with important stateless resources

2018-03-23 Thread Sam Gardner 
Thanks, Ken.

I just want all master-mode resources to be running wherever DRBDFS is running 
(essentially). If the cluster detects that any of the master-mode resources 
can't run on the current node (but can run on the other per ethmon), all other 
master-mode resources as well as DRBDFS should move over to the other node.

The current set of constraints I have will let DRBDFS move to the standby node 
and "take" the Master mode resources with it, but the Master mode resources 
failing over to the other node won't take the other Master resources or DRBDFS.

As a side note, there are other resources I have in play (some active/passive 
like DRBDFS, some Master/Slave like the ship resources) that are related, but 
not shown here - I'm just having a hard time reasoning about the generalized 
form that my constraints should take to make this sort of thing work.
-- 
Sam Gardner
Trustwave | SMART SECURITY ON DEMAND








On 3/23/18, 12:34 PM, "Users on behalf of Ken Gaillot" 
 wrote:

>On Tue, 2018-03-20 at 16:34 +, Sam Gardner wrote:
>> Hi All -
>> 
>> I've implemented a simple two-node cluster with DRBD and a couple of
>> network-based Master/Slave resources.
>> 
>> Using the ethmonitor RA, I set up failover whenever the
>> Master/Primary node loses link on the specified ethernet physical
>> device by constraining the Master role only on nodes where the ethmon
>> variable is "1".
>> 
>> Something is going wrong with my colocation constraint, however - if
>> I set up the DRBDFS resource to monitor link on eth1, unplugging eth1
>> on the Primary node causes a failover as expected - all Master
>> resources are demoted to "slave" and promoted on the opposite node,
>> and the "normal" DRBDFS moves to the other node as expected.
>> 
>> However, if I put the same ethmonitor constraint on the network-based 
>> Master/Slave resource, only that specific resource fails over -
>> DRBDFS stays in the same location (though it stops) as do the other
>> Master/Slave resources.
>> 
>> This *smells* like a constraints issue to me - does anyone know what
>> I might be doing wrong?
>>
>> PCS before:
>> Cluster name: 
>> http://scanmail.trustwave.com/?c=4062=o7q12l4ebrGiubIsdGUsPkgY15QLdpI4OU2Ogkq_xg=5=http%3a%2f%2fnode1%2ehostname%2ecom%5fnode2%2ehostname%2ecom
>> Stack: corosync
>> Current DC: node2.hostname.com_0 (version 1.1.16-12.el7_4.4-94ff4df)
>> - partition with quorum
>> Last updated: Tue Mar 20 16:25:47 2018
>> Last change: Tue Mar 20 16:00:33 2018 by hacluster via crmd on
>> node2.hostname.com_0
>> 
>> 2 nodes configured
>> 11 resources configured
>> 
>> Online: [ node1.hostname.com_0 node2.hostname.com_0 ]
>> 
>> Full list of resources:
>> 
>>  Master/Slave Set: drbd.master [drbd.slave]
>>  Masters: [ node1.hostname.com_0 ]
>>  Slaves: [ node2.hostname.com_0 ]
>>  drbdfs (ocf::heartbeat:Filesystem):Started node1.hostname.com_0
>>  Master/Slave Set: inside-interface-sameip.master [inside-interface-
>> sameip.slave]
>>  Masters: [ node1.hostname.com_0 ]
>>  Slaves: [ node2.hostname.com_0 ]
>>  Master/Slave Set: outside-interface-sameip.master [outside-
>> interface-sameip.slave]
>>  Masters: [ node1.hostname.com_0 ]
>>  Slaves: [ node2.hostname.com_0 ]
>>  Clone Set: monitor-eth1-clone [monitor-eth1]
>>  Started: [ node1.hostname.com_0 node2.hostname.com_0 ]
>>  Clone Set: monitor-eth2-clone [monitor-eth2]
>>  Started: [ node1.hostname.com_0 node2.hostname.com_0 ]
>
>What agent are the two IP resources using? I'm not familiar with any IP
>resource agents that are master/slave clones.
>
>> Daemon Status:
>>   corosync: active/enabled
>>   pacemaker: active/enabled
>>   pcsd: inactive/disabled
>> 
>> PCS after:
>> Cluster name: 
>> http://scanmail.trustwave.com/?c=4062=o7q12l4ebrGiubIsdGUsPkgY15QLdpI4OU2Ogkq_xg=5=http%3a%2f%2fnode1%2ehostname%2ecom%5fnode2%2ehostname%2ecom
>> Stack: corosync
>> Current DC: node2.hostname.com_0 (version 1.1.16-12.el7_4.4-94ff4df)
>> - partition with quorum
>> Last updated: Tue Mar 20 16:29:40 2018
>> Last change: Tue Mar 20 16:00:33 2018 by hacluster via crmd on
>> node2.hostname.com_0
>> 
>> 2 nodes configured
>> 11 resources configured
>> 
>> Online: [ node1.hostname.com_0 node2.hostname.com_0 ]
>> 
>> Full list of resources:
>> 
>>  Master/Slave Set: drbd.master [drbd.slave]
>>  Masters: [ node1.hostname.com_0 ]
>>  Slaves: [ node2.hostname.com_0 ]
>>  drbdfs (ocf::heartbeat:Filesystem):Stopped
>>  Master/Slave Set: inside-interface-sameip.master [inside-interface-
>> sameip.slave]
>>  Masters: [ node2.hostname.com_0 ]
>>  Stopped: [ node1.hostname.com_0 ]
>>  Master/Slave Set: outside-interface-sameip.master [outside-
>> interface-sameip.slave]
>>  Masters: [ node1.hostname.com_0 ]
>>  Slaves: [ node2.hostname.com_0 ]
>>  Clone Set: monitor-eth1-clone [monitor-eth1]
>>  Started: [ node1.hostname.com_0 node2.hostname.com_0 ]
>>  Clone Set:

Re: [ClusterLabs] Colocation constraint for grouping all master-mode stateful resources with important stateless resources

2018-03-23 Thread Ken Gaillot 
On Tue, 2018-03-20 at 16:34 +, Sam Gardner wrote:
> Hi All -
> 
> I've implemented a simple two-node cluster with DRBD and a couple of
> network-based Master/Slave resources.
> 
> Using the ethmonitor RA, I set up failover whenever the
> Master/Primary node loses link on the specified ethernet physical
> device by constraining the Master role only on nodes where the ethmon
> variable is "1".
> 
> Something is going wrong with my colocation constraint, however - if
> I set up the DRBDFS resource to monitor link on eth1, unplugging eth1
> on the Primary node causes a failover as expected - all Master
> resources are demoted to "slave" and promoted on the opposite node,
> and the "normal" DRBDFS moves to the other node as expected.
> 
> However, if I put the same ethmonitor constraint on the network-based 
> Master/Slave resource, only that specific resource fails over -
> DRBDFS stays in the same location (though it stops) as do the other
> Master/Slave resources.
> 
> This *smells* like a constraints issue to me - does anyone know what
> I might be doing wrong?
>
> PCS before:
> Cluster name: node1.hostname.com_node2.hostname.com
> Stack: corosync
> Current DC: node2.hostname.com_0 (version 1.1.16-12.el7_4.4-94ff4df)
> - partition with quorum
> Last updated: Tue Mar 20 16:25:47 2018
> Last change: Tue Mar 20 16:00:33 2018 by hacluster via crmd on
> node2.hostname.com_0
> 
> 2 nodes configured
> 11 resources configured
> 
> Online: [ node1.hostname.com_0 node2.hostname.com_0 ]
> 
> Full list of resources:
> 
>  Master/Slave Set: drbd.master [drbd.slave]
>      Masters: [ node1.hostname.com_0 ]
>      Slaves: [ node2.hostname.com_0 ]
>  drbdfs (ocf::heartbeat:Filesystem):    Started node1.hostname.com_0
>  Master/Slave Set: inside-interface-sameip.master [inside-interface-
> sameip.slave]
>      Masters: [ node1.hostname.com_0 ]
>      Slaves: [ node2.hostname.com_0 ]
>  Master/Slave Set: outside-interface-sameip.master [outside-
> interface-sameip.slave]
>      Masters: [ node1.hostname.com_0 ]
>      Slaves: [ node2.hostname.com_0 ]
>  Clone Set: monitor-eth1-clone [monitor-eth1]
>      Started: [ node1.hostname.com_0 node2.hostname.com_0 ]
>  Clone Set: monitor-eth2-clone [monitor-eth2]
>      Started: [ node1.hostname.com_0 node2.hostname.com_0 ]

What agent are the two IP resources using? I'm not familiar with any IP
resource agents that are master/slave clones.

> Daemon Status:
>   corosync: active/enabled
>   pacemaker: active/enabled
>   pcsd: inactive/disabled
> 
> PCS after:
> Cluster name: node1.hostname.com_node2.hostname.com
> Stack: corosync
> Current DC: node2.hostname.com_0 (version 1.1.16-12.el7_4.4-94ff4df)
> - partition with quorum
> Last updated: Tue Mar 20 16:29:40 2018
> Last change: Tue Mar 20 16:00:33 2018 by hacluster via crmd on
> node2.hostname.com_0
> 
> 2 nodes configured
> 11 resources configured
> 
> Online: [ node1.hostname.com_0 node2.hostname.com_0 ]
> 
> Full list of resources:
> 
>  Master/Slave Set: drbd.master [drbd.slave]
>      Masters: [ node1.hostname.com_0 ]
>      Slaves: [ node2.hostname.com_0 ]
>  drbdfs (ocf::heartbeat:Filesystem):    Stopped
>  Master/Slave Set: inside-interface-sameip.master [inside-interface-
> sameip.slave]
>      Masters: [ node2.hostname.com_0 ]
>      Stopped: [ node1.hostname.com_0 ]
>  Master/Slave Set: outside-interface-sameip.master [outside-
> interface-sameip.slave]
>      Masters: [ node1.hostname.com_0 ]
>      Slaves: [ node2.hostname.com_0 ]
>  Clone Set: monitor-eth1-clone [monitor-eth1]
>      Started: [ node1.hostname.com_0 node2.hostname.com_0 ]
>  Clone Set: monitor-eth2-clone [monitor-eth2]
>      Started: [ node1.hostname.com_0 node2.hostname.com_0 ]
> 
> Daemon Status:
>   corosync: active/enabled
>   pacemaker: active/enabled
>   pcsd: inactive/disabled
> 
> This is the "constraints" section of my CIB (full CIB is attached):
>        id="pcs_rsc_colocation_set_drbdfs_set_drbd.master_inside-interface-
> sameip.master_outside-interface-sameip.master" score="INFINITY">
>         
>           
>         
>          sequential="false">
>           
>           
>           
>         
>       

Resource sets can be confusing in the best of cases.

The above constraint says: Place drbdfs only on a node where the master
instances of drbd.master and the two IPs are running (without any
dependencies between those resources).

This explains why the master instances can run on different nodes, and
why drbdfs was stopped when they did.

>        kind="Serialize" symmetrical="false">
>          id="pcs_rsc_set_drbd.master_inside-interface-sameip.master_outside-
> interface-sameip.master-1" role="Master">
>           
>           
>           
>         
>         
>           
>         
>       

The above constraint says: if promoting any of drbd.master and the two
interfaces and/or starting drbdfs, do each action one at a time (in any
order). Other actions (including demoting and stopping) can happen in
any order.

>      

[ClusterLabs] Colocation constraint for grouping all master-mode stateful resources with important stateless resources

2018-03-20 Thread Sam Gardner 
Hi All -

I've implemented a simple two-node cluster with DRBD and a couple of 
network-based Master/Slave resources.

Using the ethmonitor RA, I set up failover whenever the Master/Primary node 
loses link on the specified ethernet physical device by constraining the Master 
role only on nodes where the ethmon variable is "1".

Something is going wrong with my colocation constraint, however - if I set up 
the DRBDFS resource to monitor link on eth1, unplugging eth1 on the Primary 
node causes a failover as expected - all Master resources are demoted to 
"slave" and promoted on the opposite node, and the "normal" DRBDFS moves to the 
other node as expected.

However, if I put the same ethmonitor constraint on the network-based 
Master/Slave resource, only that specific resource fails over - DRBDFS stays in 
the same location (though it stops) as do the other Master/Slave resources.

This *smells* like a constraints issue to me - does anyone know what I might be 
doing wrong?

PCS before:
Cluster name: node1.hostname.com_node2.hostname.com
Stack: corosync
Current DC: node2.hostname.com_0 (version 1.1.16-12.el7_4.4-94ff4df) - 
partition with quorum
Last updated: Tue Mar 20 16:25:47 2018
Last change: Tue Mar 20 16:00:33 2018 by hacluster via crmd on 
node2.hostname.com_0

2 nodes configured
11 resources configured

Online: [ node1.hostname.com_0 node2.hostname.com_0 ]

Full list of resources:

 Master/Slave Set: drbd.master [drbd.slave]
 Masters: [ node1.hostname.com_0 ]
 Slaves: [ node2.hostname.com_0 ]
 drbdfs (ocf::heartbeat:Filesystem):Started node1.hostname.com_0
 Master/Slave Set: inside-interface-sameip.master 
[inside-interface-sameip.slave]
 Masters: [ node1.hostname.com_0 ]
 Slaves: [ node2.hostname.com_0 ]
 Master/Slave Set: outside-interface-sameip.master 
[outside-interface-sameip.slave]
 Masters: [ node1.hostname.com_0 ]
 Slaves: [ node2.hostname.com_0 ]
 Clone Set: monitor-eth1-clone [monitor-eth1]
 Started: [ node1.hostname.com_0 node2.hostname.com_0 ]
 Clone Set: monitor-eth2-clone [monitor-eth2]
 Started: [ node1.hostname.com_0 node2.hostname.com_0 ]

Daemon Status:
  corosync: active/enabled
  pacemaker: active/enabled
  pcsd: inactive/disabled

PCS after:
Cluster name: node1.hostname.com_node2.hostname.com
Stack: corosync
Current DC: node2.hostname.com_0 (version 1.1.16-12.el7_4.4-94ff4df) - 
partition with quorum
Last updated: Tue Mar 20 16:29:40 2018
Last change: Tue Mar 20 16:00:33 2018 by hacluster via crmd on 
node2.hostname.com_0

2 nodes configured
11 resources configured

Online: [ node1.hostname.com_0 node2.hostname.com_0 ]

Full list of resources:

 Master/Slave Set: drbd.master [drbd.slave]
 Masters: [ node1.hostname.com_0 ]
 Slaves: [ node2.hostname.com_0 ]
 drbdfs (ocf::heartbeat:Filesystem):Stopped
 Master/Slave Set: inside-interface-sameip.master 
[inside-interface-sameip.slave]
 Masters: [ node2.hostname.com_0 ]
 Stopped: [ node1.hostname.com_0 ]
 Master/Slave Set: outside-interface-sameip.master 
[outside-interface-sameip.slave]
 Masters: [ node1.hostname.com_0 ]
 Slaves: [ node2.hostname.com_0 ]
 Clone Set: monitor-eth1-clone [monitor-eth1]
 Started: [ node1.hostname.com_0 node2.hostname.com_0 ]
 Clone Set: monitor-eth2-clone [monitor-eth2]
 Started: [ node1.hostname.com_0 node2.hostname.com_0 ]

Daemon Status:
  corosync: active/enabled
  pacemaker: active/enabled
  pcsd: inactive/disabled

This is the "constraints" section of my CIB (full CIB is attached):
  

  


  
  
  

  
  

  
  
  


  

  
  

  

  
  

  

  

--
Sam Gardner
Trustwave | SMART SECURITY ON DEMAND


cib-details.xml
Description: cib-details.xml
___
Users mailing list: Users@clusterlabs.org
https://lists.clusterlabs.org/mailman/listinfo/users

Project Home: http://www.clusterlabs.org
Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
Bugs: http://bugs.clusterlabs.org