[ClusterLabs] Pacemaker: pgsql
Hello, We have setup active-passive cluster using streaming replication on Rhel7.5. We are testing pacemaker for automated failover. We are seeing below issues with the setup : 1. When a failover is triggered when data is being added to the primary by killing primary (killall -9 postgres), the standby doesnt come up in sync. On pacemaker, the crm_mon -Afr shows standby in disconnected and HS:alone state. On postgres, we see below error: < 2019-09-20 17:07:46.266 IST > LOG: entering standby mode < 2019-09-20 17:07:46.267 IST > LOG: database system was not properly shut down; automatic recovery in progress < 2019-09-20 17:07:46.270 IST > LOG: redo starts at 1/680A2188 < 2019-09-20 17:07:46.370 IST > LOG: consistent recovery state reached at 1/6879D9F8 < 2019-09-20 17:07:46.370 IST > LOG: database system is ready to accept read only connections cp: cannot stat '/var/lib/pgsql/9.6/data/archivedir/000100010068': No such file or directory < 2019-09-20 17:07:46.751 IST > LOG: statement: select pg_is_in_recovery() < 2019-09-20 17:07:46.782 IST > LOG: statement: show synchronous_standby_names < 2019-09-20 17:07:50.993 IST > LOG: statement: select pg_is_in_recovery() < 2019-09-20 17:07:53.395 IST > LOG: started streaming WAL from primary at 1/6800 on timeline 1 < 2019-09-20 17:07:53.436 IST > LOG: invalid contrecord length 2662 at 1/6879D9F8 < 2019-09-20 17:07:53.438 IST > FATAL: terminating walreceiver process due to administrator command cp: cannot stat '/var/lib/pgsql/9.6/data/archivedir/0002.history': No such file or directory cp: cannot stat '/var/lib/pgsql/9.6/data/archivedir/000100010068': No such file or directory When we try to restart postgres on the standby, using pg_ctl restart, the standby start syncing. 2. After standby syncs using pg_ctl restart as mentioned above, we found out that 1-2 records are missing on the standby. Need help to check: 1. why the standby starts in disconnect, HS:alone state? f you have faced this issue/have knowledge, please let us know. Thanks. ___ Manage your subscription: https://lists.clusterlabs.org/mailman/listinfo/users ClusterLabs home: https://www.clusterlabs.org/
[ClusterLabs] Coming in Pacemaker 2.0.3: You can (mostly) forget cluster-recheck-interval exists
Hi all, Currently, if you configure a failure-timeout, a reconnect_interval for an ocf:pacemaker:remote resource, or a rule with a date_expression, Pacemaker doesn't guarantee checking them more often than the cluster- recheck-interval (which defaults to 15 minutes). Often, users either forget to change cluster-recheck-interval, in which case the action may occur well after intended, or set cluster-recheck- interval very low, which is inefficient. With the upcoming Pacemaker 2.0.3, Pacemaker will dynamically compute the recheck interval based on the configuration. If there's a failure that will expire 9 minutes from now, Pacemaker will recheck in 9 minutes. The lone exception is for date expressions using the cron-like "date_spec" format, due to the difficulty of determining the next recheck time in that case -- cluster-recheck-interval is still your friend if you use date_spec. Pacemaker will continue to check at least as often as cluster-recheck- interval, both to evaluate date_spec entries and as a fail-safe in case of certain types of scheduler bugs. -- Ken Gaillot ___ Manage your subscription: https://lists.clusterlabs.org/mailman/listinfo/users ClusterLabs home: https://www.clusterlabs.org/
[ClusterLabs] fence-agents v4.5.1
ClusterLabs is happy to announce fence-agents v4.5.1, which is a bugfix release for v4.5.0. The source code is available at: https://github.com/ClusterLabs/fence-agents/releases/tag/v4.5.1 The most significant enhancements in this release are: - bugfixes and enhancements: - fence_mpath/fence_openstack: fix fail_usage() issue and a couple of other minor issues w/the newly added plug/port parameter Everyone is encouraged to download and test the new release. We do many regression tests and simulations, but we can't cover all possible use cases, so your feedback is important and appreciated. Many thanks to all the contributors to this release. Best, The fence-agents maintainers ___ Manage your subscription: https://lists.clusterlabs.org/mailman/listinfo/users ClusterLabs home: https://www.clusterlabs.org/
Re: [ClusterLabs] Fence_sbd script in Fedora30?
Thank you Everybody for quick reply! I was a little confused that fence_sbd script was removed from sbd package. I guess now it is living only in fence_agents. Also, I was looking for some guidance on new (for me) parameters in fence_sbd, but I think I have figured that out. Another problem I have is that we modify scripts to work with our hardware and I am in the process of going through these changes. Thanks again! _Vitaly > On September 24, 2019 at 12:29 AM Andrei Borzenkov > wrote: > > > 23.09.2019 23:23, Vitaly Zolotusky пишет: > > Hello, > > I am trying to upgrade to Fedora 30. The platform is two node cluster with > > pacemaker. > > It Fedora 28 we were using old fence_sbd script from 2013: > > > > # This STONITH script drives the shared-storage stonith plugin. > > # Copyright (C) 2013 Lars Marowsky-Bree > > > > We were overwriting the distribution script in custom built RPM with the > > one from 2013. > > It looks like there is no fence_sbd script any more in the agents source > > and some apis changed so that the old script would not work. > > Do you have any documentation / suggestions on how to move from old > > fence_sbd script to the latest? > > What's wrong with external/sbd stonith resource? > ___ > Manage your subscription: > https://lists.clusterlabs.org/mailman/listinfo/users > > ClusterLabs home: https://www.clusterlabs.org/ ___ Manage your subscription: https://lists.clusterlabs.org/mailman/listinfo/users ClusterLabs home: https://www.clusterlabs.org/
Re: [ClusterLabs] Fence_sbd script in Fedora30?
On 24/09/19 08:08 +0200, Klaus Wenninger wrote: > On 9/23/19 10:23 PM, Vitaly Zolotusky wrote: >> I am trying to upgrade to Fedora 30. The platform is two node >> cluster with pacemaker. >> It Fedora 28 we were using old fence_sbd script from 2013: >> >> # This STONITH script drives the shared-storage stonith plugin. >> # Copyright (C) 2013 Lars Marowsky-Bree >> >> We were overwriting the distribution script in custom built RPM >> with the one from 2013. >> It looks like there is no fence_sbd script any more in the agents >> source and some apis changed so that the old script would not work. > > What about fence_sbd from fence-agents-sbd-package from fc30? > That should work with pacemaker & sbd from fc30. From current Rawhide ~ Fedora 32: # dnf repoquery -f '*/fence_sbd' 2>/dev/null fence-agents-sbd-0:4.4.0-2.fc31.noarch >> Do you have any documentation / suggestions on how to move from old >> fence_sbd script to the latest? -- Jan (Poki) pgpaDNXbUjBnR.pgp Description: PGP signature ___ Manage your subscription: https://lists.clusterlabs.org/mailman/listinfo/users ClusterLabs home: https://www.clusterlabs.org/
Re: [ClusterLabs] Fence_sbd script in Fedora30?
On 9/23/19 10:23 PM, Vitaly Zolotusky wrote: > Hello, > I am trying to upgrade to Fedora 30. The platform is two node cluster with > pacemaker. > It Fedora 28 we were using old fence_sbd script from 2013: > > # This STONITH script drives the shared-storage stonith plugin. > # Copyright (C) 2013 Lars Marowsky-Bree > > We were overwriting the distribution script in custom built RPM with the one > from 2013. > It looks like there is no fence_sbd script any more in the agents source and > some apis changed so that the old script would not work. What about fence_sbd from fence-agents-sbd-package from fc30? That should work with pacemaker & sbd from fc30. Klaus > Do you have any documentation / suggestions on how to move from old fence_sbd > script to the latest? > Thank you very much! > _Vitaly > ___ > Manage your subscription: > https://lists.clusterlabs.org/mailman/listinfo/users > > ClusterLabs home: https://www.clusterlabs.org/ ___ Manage your subscription: https://lists.clusterlabs.org/mailman/listinfo/users ClusterLabs home: https://www.clusterlabs.org/
