[ClusterLabs] Booth fail-over conditions

2018-04-13 Thread Zach Anderson 
 Hey all,

new user to pacemaker/booth and I'm fumbling my way through my first proof
of concept. I have a 2 site configuration setup with local pacemaker
clusters at each site (running rabbitmq) and a booth arbitrator. I've
successfully validated the base failover when the "granted" site has
failed. My question is if there are any other ways to configure failover,
i.e. using resource health checks or the like?

Thanks!
___
Users mailing list: Users@clusterlabs.org
https://lists.clusterlabs.org/mailman/listinfo/users

Project Home: http://www.clusterlabs.org
Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
Bugs: http://bugs.clusterlabs.org

Re: [ClusterLabs] General Capabilities Question

2018-04-13 Thread Ken Gaillot 
On Thu, 2018-04-12 at 21:09 -0700, Cliff Burdick wrote:
> Hi, I had a general question about Pacemaker to see if it would work
> for a somewhat unique situation. I have a cluster of 10 active
> machines + 2 standby that each have 3 interfaces (2 control, 1
> management). I want each of the control interfaces to use virtual
> IPs, such that if any of those 10 nodes fail, one of the two standby
> nodes can assume the virtual IPs of the failed node. For the virtual
> IPs, I wanted the failover to happen entirely in userspace and
> essentially just receive a notification that a node failed, and allow
> my application on the standby node to do all of the virtual IP
> reconfiguration. By userspace I mean that these interfaces are
> typically not configured by Linux, so the failure of those interfaces
> and the configuration would be done by a user script.
> 
> The reason I'm looking at pacemaker is it appears to be robust on
> node failure detections, and the STONITH is something I'd like to do.
> Is this something pacemaker could be configured to do, and if so,
> which part of the documentation should I focus on?

Yes, that's what Pacemaker is designed for. I recommend starting with
the "Clusters from Scratch" document to get familiar with the basics.
Then you can look at "Pacemaker Explained" to get detailed info about
particular configuration options (especially constraints).

I'm not sure what your interface/VIP stuff does, but you probably want
to write your own OCF resource agent (see IPaddr2 as an example) to
manage the IP, and let Pacemaker call it as needed.
-- 
Ken Gaillot 
___
Users mailing list: Users@clusterlabs.org
https://lists.clusterlabs.org/mailman/listinfo/users

Project Home: http://www.clusterlabs.org
Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
Bugs: http://bugs.clusterlabs.org

Re: [ClusterLabs] Corosync 2.4.4 is available at corosync.org!

2018-04-13 Thread Jan Friesse 

Ferenc Wágner napsal(a):

Jan Pokorný  writes:


On 12/04/18 14:33 +0200, Jan Friesse wrote:


This release contains a lot of fixes, including fix for
CVE-2018-1084.


Security related updates would preferably provide more context


Absolutely, thanks for providing that!  Looking at the git log, I wonder
if c139255 (totemsrp: Implement sanity checks of received msgs) has
direct security relevance as well.  Should I include that too in the


Not entirely direct, but quite similar.


Debian security update?  Debian stable has 2.4.2, so I'm cherry picking


Yes, please include all
fc1d5418533c1faf21616b282c2559bed7d361c4..b25b029fe186bacf089ab8136da58390945eb35c

Regards,
  Honza


into that version.



___
Users mailing list: Users@clusterlabs.org
https://lists.clusterlabs.org/mailman/listinfo/users

Project Home: http://www.clusterlabs.org
Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
Bugs: http://bugs.clusterlabs.org

Re: [ClusterLabs] No slave is promoted to be master

2018-04-13 Thread Jehan-Guillaume de Rorthais 
OK, I know what happen.

It seems like your standbies were not replicating when the master "crashed",
you can find tons of messages like this in the log files:

  WARNING: No secondary connected to the master
  WARNING: "db2" is not connected to the primary
  WARNING: "db3" is not connected to the primary

When a standby is not replicating, the master set negative master score to them
to forbid the promotion on them, as they are probably lagging for some
undefined time.

The following command shows the scores just before the simulated master crash:

  $ crm_simulate -x pe-input-2039.bz2 -s|grep -E 'date|promotion'
  Using the original execution date of: 2018-04-11 16:23:07Z
  pgsqld:0 promotion score on db1: 1001
  pgsqld:1 promotion score on db2: -1000
  pgsqld:2 promotion score on db3: -1000

"1001" score design the master. Streaming standbies always have a
positive master score between 1000 and 1000-N*10 where N is the number of
connected standbies.



On Fri, 13 Apr 2018 01:37:54 +
范国腾  wrote:

> The log is in the attachment.
> 
> We make a bug in the PG code in master node to make it not be restarted any
> more in order to test the following scenario: One slave could be promoted
> when the master crashed,  
> 
> -邮件原件-
> 发件人: Jehan-Guillaume de Rorthais [mailto:j...@dalibo.com] 
> 发送时间: 2018年4月12日 17:39
> 收件人: 范国腾 
> 抄送: Cluster Labs - All topics related to open-source clustering welcomed
>  主题: Re: [ClusterLabs] No slave is promoted to be
> master
> 
> Hi,
> On Thu, 12 Apr 2018 08:31:39 +
> 范国腾  wrote:
> 
> > Thank you very much for help check this issue. The information is in 
> > the attachment.
> > 
> > I have restarted the cluster after I send my first email. Not sure if 
> > it affects the checking of "the result of "crm_simulate -sL"
> 
> It does...
> 
> Could you please provide files
> from /var/lib/pacemaker/pengine/pe-input-2039.bz2 to  pe-input-2065.bz2 ?
> 
> [...]
> > Then the master is restarted and it could not start（that is ok and we 
> > know the reason）。
> 
> Why couldn't it start ?



-- 
Jehan-Guillaume de Rorthais
Dalibo
___
Users mailing list: Users@clusterlabs.org
https://lists.clusterlabs.org/mailman/listinfo/users

Project Home: http://www.clusterlabs.org
Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
Bugs: http://bugs.clusterlabs.org

Re: [ClusterLabs] heartbeat/anything Resource Agent : "wait for proper service before ending the start operation"

2018-04-13 Thread Oyvind Albrigtsen 

On 13/04/18 11:53 +0200, Nicolas Huillard wrote:

Le vendredi 13 avril 2018 à 11:15 +0200, Oyvind Albrigtsen a écrit :

On 13/04/18 11:07 +0200, Nicolas Huillard wrote:
> One of my resources is a pppd process, which is started with the
> heartbeat/anything RA. That RA just spawn the pppd process with the
> correct parameters and return OCF_SUCCESS if the process started.
> The problem is that the service provided by pppd is only available
> after some time (a few seconds to 30s), ie. when it have
> successfully
> negotiated a connection. At this time, the interface it creates is
> UP.
>
> The issue here is that other resources that depend on this
> connection
> are started by Pacemaker just after it starts pppd, thus before the
> interface is UP. This creates various problems.
>
> I figured that fixing this would require to add a monitor call
> inside
> the start operation, and wait for a successful monitor before
> returning
> OCF_SUCCESS, within the start timeout.
>
> Is it a correct approach?
> Are there some other standard way to fix this, like a "wait for
> condition" Resource Agent?

You could try using the monitor_hook parameter to check the status,


The issue here is the monitor will at first return a "fail", which is
considered fatal by Pacemaker unless property start-failure-is-fatal is
set to false, which may come with side-effects.
That's what I do now with a ping RA inserted before the service which
may fail if the interface is not UP. It works, but triggers some "fail"
events which are not really "fails" but "not started yet".

You might try setting it to e.g. "sleep 30;
" and see if that works.



or
use the Delay agent between the anything resource and the other
resources.


I'll try this. Hoping a sensible delay can be derived from the logs.

Thanks,

--
Nicolas Huillard
___
Users mailing list: Users@clusterlabs.org
https://lists.clusterlabs.org/mailman/listinfo/users

Project Home: http://www.clusterlabs.org
Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
Bugs: http://bugs.clusterlabs.org

___
Users mailing list: Users@clusterlabs.org
https://lists.clusterlabs.org/mailman/listinfo/users

Project Home: http://www.clusterlabs.org
Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
Bugs: http://bugs.clusterlabs.org

Re: [ClusterLabs] heartbeat/anything Resource Agent : "wait for proper service before ending the start operation"

2018-04-13 Thread Nicolas Huillard 
Le vendredi 13 avril 2018 à 11:15 +0200, Oyvind Albrigtsen a écrit :
> On 13/04/18 11:07 +0200, Nicolas Huillard wrote:
> > One of my resources is a pppd process, which is started with the
> > heartbeat/anything RA. That RA just spawn the pppd process with the
> > correct parameters and return OCF_SUCCESS if the process started.
> > The problem is that the service provided by pppd is only available
> > after some time (a few seconds to 30s), ie. when it have
> > successfully
> > negotiated a connection. At this time, the interface it creates is
> > UP.
> > 
> > The issue here is that other resources that depend on this
> > connection
> > are started by Pacemaker just after it starts pppd, thus before the
> > interface is UP. This creates various problems.
> > 
> > I figured that fixing this would require to add a monitor call
> > inside
> > the start operation, and wait for a successful monitor before
> > returning
> > OCF_SUCCESS, within the start timeout.
> > 
> > Is it a correct approach?
> > Are there some other standard way to fix this, like a "wait for
> > condition" Resource Agent?
> 
> You could try using the monitor_hook parameter to check the status, 

The issue here is the monitor will at first return a "fail", which is
considered fatal by Pacemaker unless property start-failure-is-fatal is
set to false, which may come with side-effects.
That's what I do now with a ping RA inserted before the service which
may fail if the interface is not UP. It works, but triggers some "fail"
events which are not really "fails" but "not started yet".

> or
> use the Delay agent between the anything resource and the other
> resources.

I'll try this. Hoping a sensible delay can be derived from the logs.

Thanks,

-- 
Nicolas Huillard
___
Users mailing list: Users@clusterlabs.org
https://lists.clusterlabs.org/mailman/listinfo/users

Project Home: http://www.clusterlabs.org
Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
Bugs: http://bugs.clusterlabs.org

[ClusterLabs] heartbeat/anything Resource Agent : "wait for proper service before ending the start operation"

2018-04-13 Thread Nicolas Huillard 
Hello all,

One of my resources is a pppd process, which is started with the
heartbeat/anything RA. That RA just spawn the pppd process with the
correct parameters and return OCF_SUCCESS if the process started.
The problem is that the service provided by pppd is only available
after some time (a few seconds to 30s), ie. when it have successfully
negotiated a connection. At this time, the interface it creates is UP.

The issue here is that other resources that depend on this connection
are started by Pacemaker just after it starts pppd, thus before the
interface is UP. This creates various problems.

I figured that fixing this would require to add a monitor call inside
the start operation, and wait for a successful monitor before returning
 OCF_SUCCESS, within the start timeout.

Is it a correct approach?
Are there some other standard way to fix this, like a "wait for
condition" Resource Agent?

Using Pacemaker 1.1.16 on Debian stretch.

-- 
Nicolas Huillard
___
Users mailing list: Users@clusterlabs.org
https://lists.clusterlabs.org/mailman/listinfo/users

Project Home: http://www.clusterlabs.org
Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
Bugs: http://bugs.clusterlabs.org

Re: [ClusterLabs] heartbeat/anything Resource Agent : "wait for proper service before ending the start operation"

2018-04-13 Thread Nicolas Huillard 
Le vendredi 13 avril 2018 à 11:59 +0200, Oyvind Albrigtsen a écrit :
> On 13/04/18 11:53 +0200, Nicolas Huillard wrote:
> > Le vendredi 13 avril 2018 à 11:15 +0200, Oyvind Albrigtsen a
> > écrit :
> > The issue here is the monitor will at first return a "fail", which
> > is considered fatal by Pacemaker unless property start-failure-is-
> > fatal is set to false, which may come with side-effects.
> > That's what I do now with a ping RA inserted before the service
> > which may fail if the interface is not UP. It works, but triggers
> > some "fail" events which are not really "fails" but "not started
> > yet".
> 
> You might try setting it to e.g. "sleep 30;
> " and see if that works.

I'm using resource-agent package 4.0.0, and just noticed that what I
was thinking about was implemented more recently in :
https://github.com/ClusterLabs/resource-agents/commit/ee099d62c23e0afd0
442a4febde80412b8ac22f1#diff-07b3e128cbd8576888076cc71c00233b

I'll use that one, thanks !

-- 
Nicolas Huillard
___
Users mailing list: Users@clusterlabs.org
https://lists.clusterlabs.org/mailman/listinfo/users

Project Home: http://www.clusterlabs.org
Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
Bugs: http://bugs.clusterlabs.org

Re: [ClusterLabs] Corosync 2.4.4 is available at corosync.org!

2018-04-13 Thread Jan Friesse 

Ferenc Wágner napsal(a):

Jan Friesse  writes:


Ferenc Wágner napsal(a):


I wonder if c139255 (totemsrp: Implement sanity checks of received
msgs) has direct security relevance as well.


Not entirely direct, but quite similar.


Should I include that too in the Debian security update?  Debian
stable has 2.4.2, so I'm cherry picking into that version.


Yes, please include all
fc1d5418533c1faf21616b282c2559bed7d361c4..b25b029fe186bacf089ab8136da58390945eb35c


Hi Honza,


Ferenc,



I'm confused, the commit I mentioned above is not in the range you
provided.  Besides, I can only include targeted security fixes for


Actually it is. c139255 = master/camelback branch, 
50e17ffc736f0052e921c861b6953ba8938e4103 = needle branch.



exploitable vulnerabilities in a stable security update.  A pre-
authentication buffer overflow (CVE-2018-1084) most certainly qualifies,
while the msgio cleanup does not.  Missing checks for messages being


Patch "msgio: Fix reading of msg longer than i32" is not only cleanup. 
It also fixes real problem when message length > 2^31 .



sent (08cb237) are hard to judge for me... wouldn't expoiting this
require root privileges to start with?  Also, how much of these issues


None of these require root privileges


can be mitigated by enabling encryption or strict firewalling?


All (including the CVE one) can be mitigated by strict firewall. The CVE 
one and the msgio cannot be mitigated by encryption, other issues can be.



Basically, I'll need more ammo to push all these changes through the
Security Team.


We can probably do CVE for others.

Honza



(I'll package 2.4.4 for testing/unstable and eventually provide a stable
backport of it, but that goes through different channels.)



___
Users mailing list: Users@clusterlabs.org
https://lists.clusterlabs.org/mailman/listinfo/users

Project Home: http://www.clusterlabs.org
Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
Bugs: http://bugs.clusterlabs.org

Re: [ClusterLabs] [solved] heartbeat/anything Resource Agent : "wait for proper service before ending the start operation"

2018-04-13 Thread Nicolas Huillard 
Le vendredi 13 avril 2018 à 11:15 +0200, Oyvind Albrigtsen a écrit :
> On 13/04/18 11:07 +0200, Nicolas Huillard wrote:
> > I figured that fixing this would require to add a monitor call
> > inside the start operation, and wait for a successful monitor
> > before returning OCF_SUCCESS, within the start timeout.
> > 
> > Is it a correct approach?
> > Are there some other standard way to fix this, like a "wait for
> > condition" Resource Agent?
> 
> You could try using the monitor_hook parameter to check the status, 

What I came up with is:

monitor_hook="(i=50; while [ $i -gt 0 ]; do ip link show ppp0 && exit
0; sleep 5; i=$((i+5)); done; exit 1)"

The "active" part of it being "ip link show ppp0". The rest is a loop
to wait the least possible (at 5s interval).

This works with resource-agents 4.1.1, not 4.0.0 that I initially used.

-- 
Nicolas Huillard
___
Users mailing list: Users@clusterlabs.org
https://lists.clusterlabs.org/mailman/listinfo/users

Project Home: http://www.clusterlabs.org
Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
Bugs: http://bugs.clusterlabs.org

Re: [ClusterLabs] [solved] heartbeat/anything Resource Agent : "wait for proper service before ending the start operation"

2018-04-13 Thread Oyvind Albrigtsen 

On 13/04/18 14:11 +0200, Nicolas Huillard wrote:

Le vendredi 13 avril 2018 à 11:15 +0200, Oyvind Albrigtsen a écrit :

On 13/04/18 11:07 +0200, Nicolas Huillard wrote:
> I figured that fixing this would require to add a monitor call
> inside the start operation, and wait for a successful monitor
> before returning OCF_SUCCESS, within the start timeout.
>
> Is it a correct approach?
> Are there some other standard way to fix this, like a "wait for
> condition" Resource Agent?

You could try using the monitor_hook parameter to check the status,


What I came up with is:

monitor_hook="(i=50; while [ $i -gt 0 ]; do ip link show ppp0 && exit
0; sleep 5; i=$((i+5)); done; exit 1)"

You could simplify it by using for:
for x in $(seq 1 10); do ip link show ppp0 && exit; sleep 5; done; exit 1


The "active" part of it being "ip link show ppp0". The rest is a loop
to wait the least possible (at 5s interval).

This works with resource-agents 4.1.1, not 4.0.0 that I initially used.

--
Nicolas Huillard
___
Users mailing list: Users@clusterlabs.org
https://lists.clusterlabs.org/mailman/listinfo/users

Project Home: http://www.clusterlabs.org
Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
Bugs: http://bugs.clusterlabs.org

___
Users mailing list: Users@clusterlabs.org
https://lists.clusterlabs.org/mailman/listinfo/users

Project Home: http://www.clusterlabs.org
Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
Bugs: http://bugs.clusterlabs.org

Re: [ClusterLabs] Failing operations immediately when node is known to be down

2018-04-13 Thread Ken Gaillot 
On Tue, 2018-04-10 at 12:56 -0500, Ryan Thomas wrote:
> I’m trying to implement a HA solution which recovers very quickly
> when a node fails.  It my configuration, when I reboot a node, I see
> in the logs that pacemaker realizes the node is down, and decides to
> move all resources to the surviving node.  To do this, it initiates a
> ‘stop’ operation on each of the resources to perform the move.  The
> ‘stop’ fails as expected after 20s (the default action timeout). 
> However, in this case, with the node known to be down,  I’d like to
> avoid this 20 second delay.  The node is known to be down, so any
> operations sent to the node will fail.  It would be nice if
> operations sent to a down node would immediately fail, thus reducing
> the time it takes the resource to be started on the surviving node.
>  I do not want to reduce the timeout for the operation, because the
> timeout is sensible for when a resource moves due to a non-node-
> failure.  Is there a way to accomplish this? 
> 
> Thanks for your help.

How are you rebooting -- cleanly (normal shutdown) or simulating a
failure (e.g. power button)?

In a normal shutdown, pacemaker will move all resources off the node
before it shuts down. These operations shouldn't fail, because the node
isn't down yet.

When a node fails, corosync should detect this and notify pacemaker.
Pacemaker will not try to execute any operations on a failed node.
Instead, it will fence it.

What log messages do you see from corosync and pacemaker indicating
that the node is down? Do you have fencing configured and tested?
-- 
Ken Gaillot 
___
Users mailing list: Users@clusterlabs.org
https://lists.clusterlabs.org/mailman/listinfo/users

Project Home: http://www.clusterlabs.org
Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
Bugs: http://bugs.clusterlabs.org

[ClusterLabs] General Capabilities Question

2018-04-13 Thread Cliff Burdick 
Hi, I had a general question about Pacemaker to see if it would work for a
somewhat unique situation. I have a cluster of 10 active machines + 2
standby that each have 3 interfaces (2 control, 1 management). I want each
of the control interfaces to use virtual IPs, such that if any of those 10
nodes fail, one of the two standby nodes can assume the virtual IPs of the
failed node. For the virtual IPs, I wanted the failover to happen entirely
in userspace and essentially just receive a notification that a node
failed, and allow my application on the standby node to do all of the
virtual IP reconfiguration. By userspace I mean that these interfaces are
typically not configured by Linux, so the failure of those interfaces and
the configuration would be done by a user script.

The reason I'm looking at pacemaker is it appears to be robust on node
failure detections, and the STONITH is something I'd like to do. Is this
something pacemaker could be configured to do, and if so, which part of the
documentation should I focus on?
___
Users mailing list: Users@clusterlabs.org
https://lists.clusterlabs.org/mailman/listinfo/users

Project Home: http://www.clusterlabs.org
Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
Bugs: http://bugs.clusterlabs.org

Re: [ClusterLabs] HALVM monitor action fail on slave node. Possible bug?

2018-04-13 Thread emmanuel segura 
the first thing that you need to configure is the stonith, because you have
this constraint "constraint order promote DrbdResClone then start HALVM"

To recover and promote drbd to master when you crash a node, configurare
the drbd fencing handler.

pacemaker execute monitor in both nodes, so this is normal, to test why
monitor fail, use ocf-tester

2018-04-13 15:29 GMT+02:00 Marco Marino :

> Hello, I'm trying to configure a simple 2 node cluster with drbd and HALVM
> (ocf:heartbeat:LVM) but I have a problem that I'm not able to solve, to I
> decided to write this long post. I need to really understand what I'm doing
> and where I'm doing wrong.
> More precisely, I'm configuring a pacemaker cluster with 2 nodes and only
> one drbd resource. Here all operations:
>
> - System configuration
> hostnamectl set-hostname pcmk[12]
> yum update -y
> yum install vim wget git -y
> vim /etc/sysconfig/selinux  -> permissive mode
> systemctl disable firewalld
> reboot
>
> - Network configuration
> [pcmk1]
> nmcli connection modify corosync ipv4.method manual ipv4.addresses
> 192.168.198.201/24 ipv6.method ignore connection.autoconnect yes
> nmcli connection modify replication ipv4.method manual ipv4.addresses
> 192.168.199.201/24 ipv6.method ignore connection.autoconnect yes
> [pcmk2]
> nmcli connection modify corosync ipv4.method manual ipv4.addresses
> 192.168.198.202/24 ipv6.method ignore connection.autoconnect yes
> nmcli connection modify replication ipv4.method manual ipv4.addresses
> 192.168.199.202/24 ipv6.method ignore connection.autoconnect yes
>
> ssh-keyget -t rsa
> ssh-copy-id root@pcmk[12]
> scp /etc/hosts root@pcmk2:/etc/hosts
>
> - Drbd Repo configuration and drbd installation
> rpm --import https://www.elrepo.org/RPM-GPG-KEY-elrepo.org
> rpm -Uvh http://www.elrepo.org/elrepo-release-7.0-3.el7.elrepo.
> noarch.rpm
> yum update -y
> yum install drbd84-utils kmod-drbd84 -y
>
> - Drbd Configuration:
> Creating a new partition on top of /dev/vdb -> /dev/vdb1 of type
> "Linux" (83)
> [/etc/drbd.d/global_common.conf]
> usage-count no;
> [/etc/drbd.d/myres.res]
> resource myres {
> on pcmk1 {
> device /dev/drbd0;
> disk /dev/vdb1;
> address 192.168.199.201:7789;
> meta-disk internal;
> }
> on pcmk2 {
> device /dev/drbd0;
> disk /dev/vdb1;
> address 192.168.199.202:7789;
> meta-disk internal;
> }
> }
>
> scp /etc/drbd.d/myres.res root@pcmk2:/etc/drbd.d/myres.res
> systemctl start drbd <-- only for test. The service is disabled at
> boot!
> drbdadm create-md myres
> drbdadm up myres
> drbdadm primary --force myres
>
> - LVM Configuration
> [root@pcmk1 ~]# lsblk
> NAMEMAJ:MIN RM  SIZE RO TYPE MOUNTPOINT
> sr0  11:01 1024M  0 rom
> vda 252:00   20G  0 disk
> ├─vda1  252:101G  0 part /boot
> └─vda2  252:20   19G  0 part
>   ├─cl-root 253:00   17G  0 lvm  /
>   └─cl-swap 253:102G  0 lvm  [SWAP]
> vdb 252:16   08G  0 disk
> └─vdb1  252:17   08G  0 part  <--- /dev/vdb1 is the partition
> I'd like to use as backing device for drbd
>   └─drbd0   147:008G  0 disk
>
> [/etc/lvm/lvm.conf]
> write_cache_state = 0
> use_lvmetad = 0
> filter = [ "a|drbd.*|", "a|vda.*|", "r|.*|" ]
>
> Disabling lvmetad service
> systemctl disable lvm2-lvmetad.service
> systemctl disable lvm2-lvmetad.socket
> reboot
>
> - Creating volume group and logical volume
> systemctl start drbd (both nodes)
> drbdadm primary myres
> pvcreate /dev/drbd0
> vgcreate havolumegroup /dev/drbd0
> lvcreate -n c-vol1 -L1G havolumegroup
> [root@pcmk1 ~]# lvs
> LV VGAttr   LSize   Pool Origin Data%  Meta%
> Move Log Cpy%Sync Convert
> root   cl-wi-ao <17.00g
>
> swap   cl-wi-ao   2.00g
>
> c-vol1 havolumegroup -wi-a-   1.00g
>
>
> - Cluster Configuration
> yum install pcs fence-agents-all -y
> systemctl enable pcsd
> systemctl start pcsd
> echo redhat | passwd --stdin hacluster
> pcs cluster auth pcmk1 pcmk2
> pcs cluster setup --name ha_cluster pcmk1 pcmk2
> pcs cluster start --all
> pcs cluster enable --all
> pcs property set stonith-enabled=false<--- Just for test!!!
> pcs property set no-quorum-policy=ignore
>
> - Drbd resource configuration
> pcs cluster cib drbd_cfg
> pcs -f drbd_cfg resource create DrbdRes ocf:linbit:drbd
> drbd_resource=myres op monitor interval=60s
> pcs -f drbd_cfg resource master DrbdResClone DrbdRes master-max=1
> master-node-max=1 clone-max=2 clone-node-max=1 notify=true
> [root@pcmk1 ~]# pcs -f drbd_cfg

[ClusterLabs] HALVM monitor action fail on slave node. Possible bug?

2018-04-13 Thread Marco Marino 
Hello, I'm trying to configure a simple 2 node cluster with drbd and HALVM
(ocf:heartbeat:LVM) but I have a problem that I'm not able to solve, to I
decided to write this long post. I need to really understand what I'm doing
and where I'm doing wrong.
More precisely, I'm configuring a pacemaker cluster with 2 nodes and only
one drbd resource. Here all operations:

- System configuration
hostnamectl set-hostname pcmk[12]
yum update -y
yum install vim wget git -y
vim /etc/sysconfig/selinux  -> permissive mode
systemctl disable firewalld
reboot

- Network configuration
[pcmk1]
nmcli connection modify corosync ipv4.method manual ipv4.addresses
192.168.198.201/24 ipv6.method ignore connection.autoconnect yes
nmcli connection modify replication ipv4.method manual ipv4.addresses
192.168.199.201/24 ipv6.method ignore connection.autoconnect yes
[pcmk2]
nmcli connection modify corosync ipv4.method manual ipv4.addresses
192.168.198.202/24 ipv6.method ignore connection.autoconnect yes
nmcli connection modify replication ipv4.method manual ipv4.addresses
192.168.199.202/24 ipv6.method ignore connection.autoconnect yes

ssh-keyget -t rsa
ssh-copy-id root@pcmk[12]
scp /etc/hosts root@pcmk2:/etc/hosts

- Drbd Repo configuration and drbd installation
rpm --import https://www.elrepo.org/RPM-GPG-KEY-elrepo.org
rpm -Uvh
http://www.elrepo.org/elrepo-release-7.0-3.el7.elrepo.noarch.rpm
yum update -y
yum install drbd84-utils kmod-drbd84 -y

- Drbd Configuration:
Creating a new partition on top of /dev/vdb -> /dev/vdb1 of type
"Linux" (83)
[/etc/drbd.d/global_common.conf]
usage-count no;
[/etc/drbd.d/myres.res]
resource myres {
on pcmk1 {
device /dev/drbd0;
disk /dev/vdb1;
address 192.168.199.201:7789;
meta-disk internal;
}
on pcmk2 {
device /dev/drbd0;
disk /dev/vdb1;
address 192.168.199.202:7789;
meta-disk internal;
}
}

scp /etc/drbd.d/myres.res root@pcmk2:/etc/drbd.d/myres.res
systemctl start drbd <-- only for test. The service is disabled at boot!
drbdadm create-md myres
drbdadm up myres
drbdadm primary --force myres

- LVM Configuration
[root@pcmk1 ~]# lsblk
NAMEMAJ:MIN RM  SIZE RO TYPE MOUNTPOINT
sr0  11:01 1024M  0 rom
vda 252:00   20G  0 disk
├─vda1  252:101G  0 part /boot
└─vda2  252:20   19G  0 part
  ├─cl-root 253:00   17G  0 lvm  /
  └─cl-swap 253:102G  0 lvm  [SWAP]
vdb 252:16   08G  0 disk
└─vdb1  252:17   08G  0 part  <--- /dev/vdb1 is the partition
I'd like to use as backing device for drbd
  └─drbd0   147:008G  0 disk

[/etc/lvm/lvm.conf]
write_cache_state = 0
use_lvmetad = 0
filter = [ "a|drbd.*|", "a|vda.*|", "r|.*|" ]

Disabling lvmetad service
systemctl disable lvm2-lvmetad.service
systemctl disable lvm2-lvmetad.socket
reboot

- Creating volume group and logical volume
systemctl start drbd (both nodes)
drbdadm primary myres
pvcreate /dev/drbd0
vgcreate havolumegroup /dev/drbd0
lvcreate -n c-vol1 -L1G havolumegroup
[root@pcmk1 ~]# lvs
LV VGAttr   LSize   Pool Origin Data%  Meta%
Move Log Cpy%Sync Convert
root   cl-wi-ao
<17.00g
swap   cl-wi-ao
2.00g
c-vol1 havolumegroup -wi-a-   1.00g


- Cluster Configuration
yum install pcs fence-agents-all -y
systemctl enable pcsd
systemctl start pcsd
echo redhat | passwd --stdin hacluster
pcs cluster auth pcmk1 pcmk2
pcs cluster setup --name ha_cluster pcmk1 pcmk2
pcs cluster start --all
pcs cluster enable --all
pcs property set stonith-enabled=false<--- Just for test!!!
pcs property set no-quorum-policy=ignore

- Drbd resource configuration
pcs cluster cib drbd_cfg
pcs -f drbd_cfg resource create DrbdRes ocf:linbit:drbd
drbd_resource=myres op monitor interval=60s
pcs -f drbd_cfg resource master DrbdResClone DrbdRes master-max=1
master-node-max=1 clone-max=2 clone-node-max=1 notify=true
[root@pcmk1 ~]# pcs -f drbd_cfg resource show
 Master/Slave Set: DrbdResClone [DrbdRes]
 Stopped: [ pcmk1 pcmk2 ]
[root@pcmk1 ~]#

Testing the failover with a forced shutoff of pcmk1. When pcmk1 returns
up, drbd is slave but logical volume is not active on pcmk2. So I need HALVM
[root@pcmk2 ~]# lvs
  LV VGAttr   LSize   Pool Origin Data%  Meta%
Move Log Cpy%Sync Convert
  root   cl-wi-ao
<17.00g
  swap   cl-wi-ao
2.00g
  c-vol1 havolumegroup -wi---
1.00g
[root@pcmk2 ~]#



- Lvm resource and constraints
pcs cluster cib lvm_cfg
pcs -f lvm_cfg resource create HALVM

[ClusterLabs] HAProxy resource agent

2018-04-13 Thread Tomer Azran 
Hello,

I'm planning to install an active\active HAProxy cluster on CentOS 7
I didn't found that there is any RA for HAproxy.
I found some on the net but I'm not sure if I need it. For example: 
https://raw.githubusercontent.com/thisismitch/cluster-agents/master/haproxy
I can always use the systemd service RA

What is your recommendation?

Thanks,
Tomer.
___
Users mailing list: Users@clusterlabs.org
https://lists.clusterlabs.org/mailman/listinfo/users

Project Home: http://www.clusterlabs.org
Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
Bugs: http://bugs.clusterlabs.org

Re: [ClusterLabs] HAProxy resource agent

2018-04-13 Thread RaSca 
On 13/04/2018 17:56, Tomer Azran wrote:
> Hello,
> I'm planning to install an active\active HAProxy cluster on CentOS 7
> I didn't found that there is any RA for HAproxy.
> I found some on the net but I'm not sure if I need it. For example:
>
https://raw.githubusercontent.com/thisismitch/cluster-agents/master/haproxy
> I can always use the systemd service RA
> What is your recommendation?

Systemd is fine for a service like haproxy, go for it!

-- 
RaSca
Mia Mamma Usa Linux: Niente è impossibile da capire, se lo spieghi bene!
ra...@miamammausalinux.org
http://www.miamammausalinux.org
___
Users mailing list: Users@clusterlabs.org
https://lists.clusterlabs.org/mailman/listinfo/users

Project Home: http://www.clusterlabs.org
Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
Bugs: http://bugs.clusterlabs.org

[ClusterLabs] Pacemaker resources are not scheduled

2018-04-13 Thread lkxjtu 
My cluster version:
Corosync 2.4.0
Pacemaker 1.1.16

There are many resource anomalies. Some resources are only monitored and not 
recovered. Some resources are not monitored or recovered. Only one resource of 
vnm is scheduled normally, but this resource cannot be started because other 
resources in the cluster are abnormal. Just like a deadlock. I have been 
plagued by this problem for a long time. I just want a stable and highly 
available resource with infinite recovery for everyone. Is my resource 
configure correct?



$ cat /etc/corosync/corosync.conf
mpatibility: whitetank

quorum {
  provider: corosync_votequorum
two_node: 0
   }

nodelist {
  node {
ring0_addr: 122.0.1.8
name: 122.0.1.8
nodeid: 01008
  }
  node {
ring0_addr: 122.0.1.9
name: 122.0.1.9
nodeid: 01009
  }
  node {
ring0_addr: 122.0.1.10
name: 122.0.1.10
nodeid: 01010
  }
}

totem {
  version: 2
  token:   3000
  token_retransmits_before_loss_const: 10
  join:60
  consensus:   3600
  vsftype: none
  max_messages:20
  clear_node_high_bit: yes
  rrp_mode:none
  secauth: off
  threads: 2
  transport:   udpu
  interface {
ringnumber:  0
bindnetaddr: 122.0.1.8
mcastport:   5405
  }
}

logging {
  fileline:off
  to_stderr:   no
  to_logfile:  yes
  logfile: /root/info/logs/pacemaker_cluster/corosync.log
  to_syslog:   yes
  syslog_facility: daemon
  syslog_priority: info
  debug:   off
  function_name:   on
  timestamp:   on
  logger_subsys {
subsys: AMF
debug:  off
tags:   enter|leave|trace1|trace2|trace3|trace4|trace6
  }
}

amf {
  mode: disabled
}

aisexec {
  user:  root
  group: root
}





$ crm configure show
node 1008: 122.0.1.8
node 1009: 122.0.1.9
node 1010: 122.0.1.10
primitive apigateway apigateway \
op monitor interval=20s timeout=220 \
op stop interval=0 timeout=120s on-fail=restart \
op start interval=0 timeout=120s on-fail=restart \
meta failure-timeout=60s target-role=Started
primitive apigateway_vip IPaddr2 \
params ip=122.0.1.203 cidr_netmask=24 \
op start interval=0 timeout=20 \
op stop interval=0 timeout=20 \
op monitor timeout=20s interval=10s depth=0 \
meta migration-threshold=3 failure-timeout=60s
primitive inetmanager inetmanager \
op monitor interval=10s timeout=160 \
op stop interval=0 timeout=60s on-fail=restart \
op start interval=0 timeout=60s on-fail=restart \
meta migration-threshold=2 failure-timeout=60s resource-stickiness=100
primitive inetmanager_vip IPaddr2 \
params ip=122.0.1.201 cidr_netmask=24 \
op start interval=0 timeout=20 \
op stop interval=0 timeout=20 \
op monitor timeout=20s interval=10s depth=0 \
meta migration-threshold=3 failure-timeout=60s
primitive logserver logserver \
op monitor interval=20s timeout=220 \
op stop interval=0 timeout=120s on-fail=restart \
op start interval=0 timeout=120s on-fail=restart \
meta failure-timeout=60s target-role=Started
primitive mariadb_vip IPaddr2 \
params ip=122.0.1.204 cidr_netmask=24 \
op start interval=0 timeout=20 \
op stop interval=0 timeout=20 \
op monitor timeout=20s interval=10s depth=0 \
meta migration-threshold=3 failure-timeout=60s
primitive mysql mysql \
op monitor interval=20s timeout=220 \
op stop interval=0 timeout=120s on-fail=restart \
op start interval=0 timeout=120s on-fail=restart \
meta failure-timeout=60s target-role=Started
primitive p_rdbserver hardb_docker \
op start timeout=3600 interval=0 \
op stop timeout=1260 interval=0 \
op promote timeout=3600 interval=0 \
op monitor role=Master interval=1 timeout=30 \
op monitor interval=15 timeout=7200 \
meta migration-threshold=3 failure-timeout=3600s
primitive p_rdbvip IPaddr2 \
params ip=100.0.1.203 cidr_netmask=24 \
op start interval=0 timeout=20 \
op stop interval=0 timeout=20 \
op monitor timeout=20s interval=10s depth=0 \
meta migration-threshold=3 failure-timeout=60s resource-stickiness=100
primitive rabbitmq rabbitmq \
op monitor interval=20s timeout=220 \
op stop interval=0 timeout=120s on-fail=restart \
op start interval=0 timeout=120s on-fail=restart \
meta failure-timeout=60s target-role=Started
primitive rabbitmq_vip IPaddr2 \
params ip=122.0.1.200 \
op start interval=0 timeout=20 \
op stop interval=0 timeout=20 \
op monitor timeout=20s interval=10s depth=0 \
meta