Re: [ClusterLabs] Pacemaker 1.1.15 - Release Candidate 3

[Jan Pokorný]

On 27/05/16 16:21 -0500, Ken Gaillot wrote:
> The third release candidate for Pacemaker version 1.1.15 is now
> available at:
> 
> https://github.com/ClusterLabs/pacemaker/releases/tag/Pacemaker-1.1.15-rc3

Once gain, to check this release candidate out using Fedora/EPEL builds,
there's a COPR link for your convenience (you can also stick with
repo file downloaded from Overview page and install the packages
in a common way):

https://copr.fedorainfracloud.org/coprs/jpokorny/pacemaker/

Fedora rawhide will be updated shortly and the -rc3 build may also
take a few minutes (from now) to finish.

> Perhaps the most visible change since 1.1.15-rc2 is that many log
> messages have been made more user-friendly. Partly this is due to taking
> advantage of the "extended information logging" feature of libqb 0.17.2
> and greater (if installed on the build machine); the same message may be
> logged to the system log without developer-oriented details, and to the
> pacemaker detail log with the extra detail.
> 
> This release candidate includes multiple bugfixes since 1.1.15-rc2, most
> importantly:
> 
> * In 1.1.14, the controld resource agent was modified to return a
> monitor error when DLM is in the "wait fencing" state. This turned out
> to be too aggressive, resulting in fencing the monitored node
> unnecessarily if a slow fencing operation against another node was in
> progress. The agent now does additional checking to determine whether to
> return an error or not.
> 
> * A bug introduced in 1.1.14, resulting in the have-watchdog property
> always being set to true, has been fixed. The cluster now properly
> checks for a running sbd process.
> 
> * A regression introduced in 1.1.15-rc1 has been fixed. When a node ID
> is reused, attrd would have problems setting attributes for the new node.
> 
> Everyone is encouraged to download, compile and test the new release.
> Your feedback is important and appreciated. I am aiming for one more
> release candidate, with the final release in mid- to late June.

-- 
Jan (Poki)


pgpLgrkWGr10X.pgp
Description: PGP signature
___
Users mailing list: Users@clusterlabs.org
http://clusterlabs.org/mailman/listinfo/users

Project Home: http://www.clusterlabs.org
Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
Bugs: http://bugs.clusterlabs.org

[ClusterLabs] Pacemaker 1.1.15 - Release Candidate 3

[Ken Gaillot]

The third release candidate for Pacemaker version 1.1.15 is now
available at:

https://github.com/ClusterLabs/pacemaker/releases/tag/Pacemaker-1.1.15-rc3

Perhaps the most visible change since 1.1.15-rc2 is that many log
messages have been made more user-friendly. Partly this is due to taking
advantage of the "extended information logging" feature of libqb 0.17.2
and greater (if installed on the build machine); the same message may be
logged to the system log without developer-oriented details, and to the
pacemaker detail log with the extra detail.

This release candidate includes multiple bugfixes since 1.1.15-rc2, most
importantly:

* In 1.1.14, the controld resource agent was modified to return a
monitor error when DLM is in the "wait fencing" state. This turned out
to be too aggressive, resulting in fencing the monitored node
unnecessarily if a slow fencing operation against another node was in
progress. The agent now does additional checking to determine whether to
return an error or not.

* A bug introduced in 1.1.14, resulting in the have-watchdog property
always being set to true, has been fixed. The cluster now properly
checks for a running sbd process.

* A regression introduced in 1.1.15-rc1 has been fixed. When a node ID
is reused, attrd would have problems setting attributes for the new node.

Everyone is encouraged to download, compile and test the new release.
Your feedback is important and appreciated. I am aiming for one more
release candidate, with the final release in mid- to late June.
-- 
Ken Gaillot 

___
Users mailing list: Users@clusterlabs.org
http://clusterlabs.org/mailman/listinfo/users

Project Home: http://www.clusterlabs.org
Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
Bugs: http://bugs.clusterlabs.org

Re: [ClusterLabs] Antw: RES: Performance of a mirrored LV (cLVM) with OCFS: Attempt to monitor it

[emmanuel segura]

Hi,

But the latest lvm version doesn't worries about the aligned?

2016-05-27 18:37 GMT+02:00 Ken Gaillot :
> On 05/27/2016 12:58 AM, Ulrich Windl wrote:
>> Hi!
>>
>> Thanks for this info. We actually run the "noop" scheduler for  the SAN
>> storage (as per menufacturer's recommendation), because on "disk" is actually
>> spread over up to 40 disks.
>> Other settings we changes was:
>> queue/rotational:0
>> queue/add_random:0
>> queue/max_sectors_kb:128 (manufacturer's recommendation, before up to 1MB
>> transfers were seen)
>> queue/read_ahead_kb:0
>>
>> And we apply those setting (where available) the the whole stack (disk
>> devices, multipath device, LV).
>>
>> Regards,
>> Ulrich
>
> I don't have anything to add about clvm specifically, but some general
> RAID tips that are often overlooked:
>
> If you're using striped RAID (i.e. >1), it's important to choose a
> stripe size wisely and make sure everything is aligned with it. Somewhat
> counterintuitively, smaller stripe sizes are better for large reads and
> writes, while larger stripe sizes are better for small reads and writes.
> There's a big performance penalty by setting a stripe size too small,
> but not much penalty from setting it too large.
>
> Things that should be aligned:
>
> * Partition sizes. A disk's first usable partition will generally start
> at (your stripe size in kilobytes * 2) sectors.
>
> * LVM physical volume metadata (via the --metadatasize option to
> pvcreate). It will set the metadata size to the next 64K boundary above
> the value, so set it to be just under the size you want, ex.
> --metadatasize 1.99M will get a metadata size of 2MB.
>
> * The filesystem creation options (varies by fs type). For example, with
> ext3/ext4, where N1 is stripe size in kilobytes / 4, and N2 is $N1 times
> the number of nonparity disks in the array, use -E
> stride=$N1,stripe-width=$N2. For xfs, where STRIPE is the stripe size in
> kilobytes and NONPARITY is the number of nonparity disks in the array,
> use -d su=${STRIPE}k,sw=${NONPARITY} -l su=${STRIPE}k.
>
> If your RAID controller has power backup (BBU or supercapacitor), mount
> filesystems with the nobarrier option.
>
> "Carlos Xavier"  schrieb am 25.05.2016 um 22:25
>> in
>> Nachricht <01da01d1b6c3$8f5c3dc0$ae14b940$@com.br>:
>>> Hi.
>>>
>>> I have been running OCFS2 on clusters for quite long time.
>>> We started running it over DRBD and now we have it running on a Dell
>>> storage.
>>> Over DRBD it showed a very poor performance, most because the way DRBD
>>> works.
>>> To improve the performance we had to change the I/O Scheduler of the disk to
>>
>>> "Deadline"
>>>
>>> When we migrate the system to the storage, the issue show up again.
>>> Sometimes the system was hanging due to disk access, to solve the issue I
>>> changed the I/O Schedule To Deadline and the trouble vanished.
>>>
>>> Regards,
>>> Carlos.
>>>
>>>
 -Mensagem original-
 De: Kristoffer Grönlund [mailto:kgronl...@suse.com]
 Enviada em: quarta-feira, 25 de maio de 2016 06:55
 Para: Ulrich Windl; users@clusterlabs.org
 Assunto: Re: [ClusterLabs] Performance of a mirrored LV (cLVM) with OCFS:
>>> Attempt to monitor it

 Ulrich Windl  writes:

> cLVM has never made a good impression regarding performance, so I wonder
>> if
>>> there's anything we
 could do to improve the4 performance. I suspect that one VM paging heavily
>>
>>> on OCFS2 kills the
 performance of the whole cluster (that hosts Xen PV guests only). Anyone
>>> with deeper insights?

 My understanding is that this is a problem inherent in the design of CLVM
>>> and there is work ongoing to
 mitigate this by handling clustering in md instead. See this LWN article
>> for
>>> more details:

 http://lwn.net/Articles/674085/

 Cheers,
 Kristoffer

 --
 // Kristoffer Grönlund
 // kgronl...@suse.com
>
> ___
> Users mailing list: Users@clusterlabs.org
> http://clusterlabs.org/mailman/listinfo/users
>
> Project Home: http://www.clusterlabs.org
> Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
> Bugs: http://bugs.clusterlabs.org



-- 
  .~.
  /V\
 //  \\
/(   )\
^`~'^

___
Users mailing list: Users@clusterlabs.org
http://clusterlabs.org/mailman/listinfo/users

Project Home: http://www.clusterlabs.org
Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
Bugs: http://bugs.clusterlabs.org

Re: [ClusterLabs] Antw: RES: Performance of a mirrored LV (cLVM) with OCFS: Attempt to monitor it

[Ken Gaillot]

On 05/27/2016 12:58 AM, Ulrich Windl wrote:
> Hi!
> 
> Thanks for this info. We actually run the "noop" scheduler for  the SAN
> storage (as per menufacturer's recommendation), because on "disk" is actually
> spread over up to 40 disks.
> Other settings we changes was:
> queue/rotational:0
> queue/add_random:0
> queue/max_sectors_kb:128 (manufacturer's recommendation, before up to 1MB
> transfers were seen)
> queue/read_ahead_kb:0
> 
> And we apply those setting (where available) the the whole stack (disk
> devices, multipath device, LV).
> 
> Regards,
> Ulrich

I don't have anything to add about clvm specifically, but some general
RAID tips that are often overlooked:

If you're using striped RAID (i.e. >1), it's important to choose a
stripe size wisely and make sure everything is aligned with it. Somewhat
counterintuitively, smaller stripe sizes are better for large reads and
writes, while larger stripe sizes are better for small reads and writes.
There's a big performance penalty by setting a stripe size too small,
but not much penalty from setting it too large.

Things that should be aligned:

* Partition sizes. A disk's first usable partition will generally start
at (your stripe size in kilobytes * 2) sectors.

* LVM physical volume metadata (via the --metadatasize option to
pvcreate). It will set the metadata size to the next 64K boundary above
the value, so set it to be just under the size you want, ex.
--metadatasize 1.99M will get a metadata size of 2MB.

* The filesystem creation options (varies by fs type). For example, with
ext3/ext4, where N1 is stripe size in kilobytes / 4, and N2 is $N1 times
the number of nonparity disks in the array, use -E
stride=$N1,stripe-width=$N2. For xfs, where STRIPE is the stripe size in
kilobytes and NONPARITY is the number of nonparity disks in the array,
use -d su=${STRIPE}k,sw=${NONPARITY} -l su=${STRIPE}k.

If your RAID controller has power backup (BBU or supercapacitor), mount
filesystems with the nobarrier option.

 "Carlos Xavier"  schrieb am 25.05.2016 um 22:25
> in
> Nachricht <01da01d1b6c3$8f5c3dc0$ae14b940$@com.br>:
>> Hi.
>>
>> I have been running OCFS2 on clusters for quite long time.
>> We started running it over DRBD and now we have it running on a Dell 
>> storage.
>> Over DRBD it showed a very poor performance, most because the way DRBD 
>> works.
>> To improve the performance we had to change the I/O Scheduler of the disk to
> 
>> "Deadline"
>>
>> When we migrate the system to the storage, the issue show up again. 
>> Sometimes the system was hanging due to disk access, to solve the issue I 
>> changed the I/O Schedule To Deadline and the trouble vanished.
>>
>> Regards,
>> Carlos.
>>
>>
>>> -Mensagem original-
>>> De: Kristoffer Grönlund [mailto:kgronl...@suse.com]
>>> Enviada em: quarta-feira, 25 de maio de 2016 06:55
>>> Para: Ulrich Windl; users@clusterlabs.org 
>>> Assunto: Re: [ClusterLabs] Performance of a mirrored LV (cLVM) with OCFS: 
>> Attempt to monitor it
>>>
>>> Ulrich Windl  writes:
>>>
 cLVM has never made a good impression regarding performance, so I wonder
> if 
>> there's anything we
>>> could do to improve the4 performance. I suspect that one VM paging heavily
> 
>> on OCFS2 kills the
>>> performance of the whole cluster (that hosts Xen PV guests only). Anyone 
>> with deeper insights?
>>>
>>> My understanding is that this is a problem inherent in the design of CLVM 
>> and there is work ongoing to
>>> mitigate this by handling clustering in md instead. See this LWN article
> for 
>> more details:
>>>
>>> http://lwn.net/Articles/674085/ 
>>>
>>> Cheers,
>>> Kristoffer
>>>
>>> --
>>> // Kristoffer Grönlund
>>> // kgronl...@suse.com 

___
Users mailing list: Users@clusterlabs.org
http://clusterlabs.org/mailman/listinfo/users

Project Home: http://www.clusterlabs.org
Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
Bugs: http://bugs.clusterlabs.org

Re: [ClusterLabs] crm_attribute bug in 1.1.15-rc1

[Jehan-Guillaume de Rorthais]

Le Mon, 23 May 2016 19:21:23 +0300,
Andrey Rogovsky  a écrit :

> Hi
> Any idea why it not work on my cluster?

Ok, I think I understood the problem. By default, crm_master use "forever" as
lifetime attribute. So my commands were incomplete to get live master score
set from the RA itself. Try the following command:

  crm_master -l reboot -r pgsqld -Q

or 

  crm_master -l reboot -r pgsqld -N $NODENAME -Q

> 2016-05-23 19:00 GMT+03:00 Jehan-Guillaume de Rorthais :
> 
> > Le Mon, 23 May 2016 15:42:55 +0300,
> > Andrey Rogovsky  a écrit :
> >
> > > Hi
> > > Your commands is not works
> > > root@c:~#   crm_master -r pgsqld -N $HOSTNAME -Q
> > > Error performing operation: No such device or address
> > > root@c:~#   crm_master -r pgsqld -Q
> > > Error performing operation: No such device or address
> >
> >
> > It works here:
> >
> > root@srv1:~$ crm_master -r pgsqld -Q
> > 1
> > root@srv1:~$ crm_master -r pgsqld -N srv2 -Q
> > 1001
> >
> > >
> > >
> > > 2016-05-23 13:38 GMT+03:00 Jehan-Guillaume de Rorthais  > >:
> > >
> > > > ok, you were trying with the attribute name. I wrote you had a use the
> > > > **resource** name.
> > > >
> > > > Your command should be (again):
> > > >
> > > >   crm_master -r pgsqld -N $HOSTNAME -Q
> > > >
> > > > Or simply this if you want to check the score on the local node:
> > > >
> > > >   crm_master -r pgsqld -Q
> > > >
> > > > Moreover, you should really consider doing some cleanup in your
> > attributes,
> > > > "pgsql-data-status" and "maintenance" definitely does not comes from
> > the
> > > > PAF
> > > > project.
> > > >
> > > >
> > > > Le Mon, 23 May 2016 12:44:29 +0300,
> > > > Andrey Rogovsky  a écrit :
> > > >
> > > > > Stack: corosync
> > > > > Current DC: b (version 1.1.12-cdf310a) - partition with quorum
> > > > > Last updated: Mon May 23 12:43:57 2016 Last change: Wed May  4
> > 12:15:06
> > > > > 2016 via crm_attribute on c
> > > > >
> > > > > 3 nodes and 7 resources configured
> > > > >
> > > > > Online: [ a b c ]
> > > > >
> > > > >  Resource Group: master
> > > > >  pgsql-master-ip (ocf::heartbeat:IPaddr2): Started a
> > > > >  Master/Slave Set: msPostgresql [pgsqld]
> > > > >  Masters: [ a ]
> > > > >  Slaves: [ b c ]
> > > > >  Clone Set: WebFarm [apache]
> > > > >  Started: [ a b c ]
> > > > >
> > > > > Node Attributes:
> > > > > * Node a:
> > > > > + maintenance : off
> > > > > + master-pgsqld   : 1001
> > > > > + pgsql-data-status   : STREAMING|ASYNC
> > > > > * Node b:
> > > > > + maintenance : off
> > > > > + master-pgsqld   : 1000
> > > > > + pgsql-data-status   : LATEST
> > > > > * Node c:
> > > > > + maintenance : off
> > > > > + master-pgsqld   : 1
> > > > > + pgsql-data-status   : STREAMING|ASYNC
> > > > >
> > > > >
> > > > > 2016-05-23 12:35 GMT+03:00 Jehan-Guillaume de Rorthais <
> > j...@dalibo.com
> > > > >:
> > > > >
> > > > > > Le Mon, 23 May 2016 12:31:37 +0300,
> > > > > > Andrey Rogovsky  a écrit :
> > > > > >
> > > > > > > This is not work
> > > > > > > #   crm_master -r master-pgsqld -N $HOSTNAME -Q
> > > > > > > Error performing operation: No such device or address
> > > > > >
> > > > > > as I wrote: you must use the resource name that is cloned by your
> > > > master
> > > > > > resource.
> > > > > >
> > > > > > Could you show us your configuration please?
> > > > > >
> > > > > > > 2016-05-23 11:46 GMT+03:00 Jehan-Guillaume de Rorthais <
> > > > j...@dalibo.com
> > > > > > >:
> > > > > > >
> > > > > > > > Le Mon, 23 May 2016 11:36:37 +0300,
> > > > > > > > Andrey Rogovsky  a écrit :
> > > > > > > >
> > > > > > > > > Hi
> > > > > > > > > This is not work for me:
> > > > > > > > > #   crm_master -r pgsqld -N $HOSTNAME $HOSTNAME -Q
> > > > > > > > > Error performing operation: No such device or address
> > > > > > > >
> > > > > > > > This should be :
> > > > > > > >
> > > > > > > >   crm_master -r pgsqld -N $HOSTNAME -Q
> > > > > > > >
> > > > > > > > (supposing as your resource name is "pgsqld")
> > > > > > > >
> > > > > > > > > 2016-05-23 11:19 GMT+03:00 Jehan-Guillaume de Rorthais <
> > > > > > j...@dalibo.com
> > > > > > > > >:
> > > > > > > > >
> > > > > > > > > > Le Mon, 23 May 2016 09:28:41 +0300,
> > > > > > > > > > Andrey Rogovsky  a écrit :
> > > > > > > > > >
> > > > > > > > > > > I try crm_master, but it not works:
> > > > > > > > > > > # LC_ALL=C /usr/sbin/crm_master -q -t nodes --node-uname
> > > > > > $HOSTNAME
> > > > > > > > > > > --attr-name master-pgsqld --get-value
> > > > > > > > > > > crm_master: invalid option -- 't'
> > > > > > > > > > > crm_master: unrecognized option '--node-uname'
> > > > > > > > > > > crm_master: unrecognized option '--attr-name'
> > > > 

Re: [ClusterLabs] Using pacemaker for manual failover only?

[Klaus Wenninger]

On 05/26/2016 08:55 PM, Stephano-Shachter, Dylan wrote:
> I tried the location -INFINITY trick and it seems to work quite well.
> Thanks for the advice.
>
> It seems to me that if I am not failing over automatically, then there
> is no good reason to run a stonith resource. Is this correct or is it
> still needed for some reason?

Well, even if you configured your cluster in a way that the rule-set
allows the critical resource/role
to be running on just one node the cluster will still help you enforce
that it is running nowhere else.
So e.g. in the situation where you switched around your -INFINITY and
the resource/role refuses
to die on the former node fencing would help you that you wouldn't have
to kill the node manually.

But I'd say that is just convenience. If you are there and doing the
config-change manually then
you can as well do the observation if everything is reacting as desired ...

>
> On Tue, May 24, 2016 at 11:02 AM, Ken Gaillot  > wrote:
>
> On 05/24/2016 04:13 AM, Klaus Wenninger wrote:
> > On 05/24/2016 09:50 AM, Jehan-Guillaume de Rorthais wrote:
> >> Le Tue, 24 May 2016 01:53:22 -0400,
> >> Digimer > a écrit :
> >>
> >>> On 23/05/16 03:03 PM, Stephano-Shachter, Dylan wrote:
>  Hello,
> 
>  I am using pacemaker 1.1.14 with pcs 0.9.149. I have successfully
>  configured pacemaker for highly available nfs with drbd.
> Pacemaker
>  allows me to easily failover without interrupting nfs
> connections. I,
>  however, am only interested in failing over manually
> (currently I use
>  "pcs resource move   --master"). I
> would like for
>  the cluster to do nothing when a node fails unexpectedly.
> 
>  Right now the solution I am going with is to run
>  "pcs property set is-managed-default=no"
>  until I need to failover, at which point I set
> is-managed-default=yes,
>  then failover, then set it back to no.
> 
>  While this method works for me, it can be unpredictable if
> people run
>  move commands at the wrong time.
> 
>  Is there a way to disable automatic failover permanently
> while still
>  allowing manual failover (with "pcs resource move" or with
> something else)?
> >> Try to set up your cluster without the "interval" parameter on
> the monitor
> >> action? The resource will be probed during the target-action
> (start/promote I
> >> suppose), but then it should not get monitored anymore.
> >
> > Ignoring the general cluster yes/no question a simple solution would
> > be to bind the master-role to a node-attribute that you move around
> > manually.
>
> This is the right track. There are a number of ways you could do
> it, but
> the basic idea is to use constraints to only allow the resources
> to run
> on one node. When you want to fail over, flip the constraints.
>
> I'd colocate everything with one (most basic) resource, so then
> all you
> need is one constraint for that resource to flip. It could be as
> simple
> as a -INFINITY location constraint on the node you don't want to
> run on.
>
> ___
> Users mailing list: Users@clusterlabs.org
> 
> http://clusterlabs.org/mailman/listinfo/users
>
> Project Home: http://www.clusterlabs.org
> Getting started:
> http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
> Bugs: http://bugs.clusterlabs.org
>
>
>
>
> ___
> Users mailing list: Users@clusterlabs.org
> http://clusterlabs.org/mailman/listinfo/users
>
> Project Home: http://www.clusterlabs.org
> Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
> Bugs: http://bugs.clusterlabs.org


___
Users mailing list: Users@clusterlabs.org
http://clusterlabs.org/mailman/listinfo/users

Project Home: http://www.clusterlabs.org
Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
Bugs: http://bugs.clusterlabs.org