Hi, I have a problem with fencing on a two node cluster. It seems that randomly the cluster cannot complete monitor operation for fence devices. In log I see: crmd[8206]: error: Result of monitor operation for fence-node2 on ld2.mydomain.it: Timed Out As attachment there is - /var/log/messages for node1 (only the important part) - /var/log/messages for node2 (only the important part) <-- Problem starts here - pcs status - pcs stonith show (for both fence devices)
I think it could be a timeout problem, so how can I see timeout value for monitor operation in stonith devices? Please, someone can help me with this problem? Furthermore, how can I fix the state of fence devices without downtime? Thank you
############PCS STATUS######################## root@ld1 ~]# pcs status Cluster name: ldcluster Stack: corosync Current DC: ld1.mydomain.it (version 1.1.19-8.el7_6.4-c3c624ea3d) - partition with quorum Last updated: Tue Sep 3 09:37:27 2019 Last change: Thu Jul 4 21:36:07 2019 by root via cibadmin on ld1.mydomain.it 2 nodes configured 10 resources configured Online: [ ld1.mydomain.it ld2.mydomain.it ] Full list of resources: fence-node1 (stonith:fence_ipmilan): Stopped fence-node2 (stonith:fence_ipmilan): Stopped Master/Slave Set: DrbdResClone [DrbdRes] Masters: [ ld1.mydomain.it ] Slaves: [ ld2.mydomain.it ] HALVM (ocf::heartbeat:LVM): Started ld1.mydomain.it PgsqlFs (ocf::heartbeat:Filesystem): Started ld1.mydomain.it PostgresqlD (systemd:postgresql-9.6.service): Started ld1.mydomain.it LegaldocapiD (systemd:legaldocapi.service): Started ld1.mydomain.it PublicVIP (ocf::heartbeat:IPaddr2): Started ld1.mydomain.it DefaultRoute (ocf::heartbeat:Route): Started ld1.mydomain.it Failed Actions: * fence-node1_start_0 on ld1.mydomain.it 'unknown error' (1): call=221, status=Timed Out, exitreason='', last-rc-change='Wed Aug 21 12:49:00 2019', queued=0ms, exec=20006ms * fence-node2_start_0 on ld1.mydomain.it 'unknown error' (1): call=222, status=Timed Out, exitreason='', last-rc-change='Wed Aug 21 12:49:00 2019', queued=1ms, exec=20013ms * fence-node1_start_0 on ld2.mydomain.it 'unknown error' (1): call=182, status=Timed Out, exitreason='', last-rc-change='Wed Aug 21 14:26:09 2019', queued=0ms, exec=20006ms * fence-node2_start_0 on ld2.mydomain.it 'unknown error' (1): call=176, status=Timed Out, exitreason='', last-rc-change='Wed Aug 21 12:48:40 2019', queued=1ms, exec=20008ms Daemon Status: corosync: active/disabled pacemaker: active/disabled pcsd: active/enabled [root@ld1 ~]# ########################STONITH SHOW########################################### [root@ld1 ~]# pcs stonith show fence-node1 Resource: fence-node1 (class=stonith type=fence_ipmilan) Attributes: ipaddr=192.168.254.250 lanplus=1 login=root passwd=XXXXXXX pcmk_host_check=static-list pcmk_host_list=ld1.mydomain.it Operations: monitor interval=60s (fence-node1-monitor-interval-60s) [root@ld1 ~]# pcs stonith show fence-node2 Resource: fence-node2 (class=stonith type=fence_ipmilan) Attributes: ipaddr=192.168.254.251 lanplus=1 login=root passwd=XXXXXXXX pcmk_host_check=static-list pcmk_host_list=ld2.mydomain.it delay=12 Operations: monitor interval=60s (fence-node2-monitor-interval-60s) [root@ld1 ~]# ###########################NODE 2 /var/log/messages############################## Aug 21 12:48:40 ld2 stonith-ng[8202]: notice: Child process 46006 performing action 'monitor' timed out with signal 15 Aug 21 12:48:40 ld2 stonith-ng[8202]: notice: Operation 'monitor' [46006] for device 'fence-node2' returned: -62 (Timer expired) Aug 21 12:48:40 ld2 crmd[8206]: error: Result of monitor operation for fence-node2 on ld2.mydomain.it: Timed Out Aug 21 12:48:40 ld2 stonith-ng[8202]: notice: On loss of CCM Quorum: Ignore Aug 21 12:48:40 ld2 crmd[8206]: notice: Result of stop operation for fence-node2 on ld2.mydomain.it: 0 (ok) Aug 21 12:48:40 ld2 stonith-ng[8202]: notice: On loss of CCM Quorum: Ignore Aug 21 12:48:40 ld2 stonith-ng[8202]: notice: On loss of CCM Quorum: Ignore Aug 21 12:48:40 ld2 stonith-ng[8202]: notice: On loss of CCM Quorum: Ignore Aug 21 12:48:59 ld2 stonith-ng[8202]: notice: On loss of CCM Quorum: Ignore Aug 21 12:49:00 ld2 stonith-ng[8202]: notice: Child process 46053 performing action 'monitor' timed out with signal 15 Aug 21 12:49:00 ld2 stonith-ng[8202]: notice: Operation 'monitor' [46053] for device 'fence-node2' returned: -62 (Timer expired) Aug 21 12:49:00 ld2 crmd[8206]: error: Result of start operation for fence-node2 on ld2.mydomain.it: Timed Out Aug 21 12:49:00 ld2 stonith-ng[8202]: notice: On loss of CCM Quorum: Ignore Aug 21 12:49:00 ld2 stonith-ng[8202]: notice: On loss of CCM Quorum: Ignore Aug 21 12:49:00 ld2 stonith-ng[8202]: notice: On loss of CCM Quorum: Ignore Aug 21 12:49:00 ld2 stonith-ng[8202]: notice: On loss of CCM Quorum: Ignore Aug 21 12:49:00 ld2 crmd[8206]: notice: Result of stop operation for fence-node2 on ld2.mydomain.it: 0 (ok) Aug 21 12:49:00 ld2 stonith-ng[8202]: notice: On loss of CCM Quorum: Ignore Aug 21 12:49:00 ld2 stonith-ng[8202]: notice: On loss of CCM Quorum: Ignore Aug 21 12:49:00 ld2 stonith-ng[8202]: notice: On loss of CCM Quorum: Ignore Aug 21 12:49:20 ld2 stonith-ng[8202]: notice: On loss of CCM Quorum: Ignore Aug 21 12:49:21 ld2 stonith-ng[8202]: notice: On loss of CCM Quorum: Ignore Aug 21 12:49:21 ld2 stonith-ng[8202]: notice: On loss of CCM Quorum: Ignore Aug 21 12:49:21 ld2 stonith-ng[8202]: notice: On loss of CCM Quorum: Ignore Aug 21 12:49:21 ld2 stonith-ng[8202]: notice: On loss of CCM Quorum: Ignore Aug 21 12:49:21 ld2 stonith-ng[8202]: notice: On loss of CCM Quorum: Ignore Aug 21 12:49:21 ld2 stonith-ng[8202]: notice: On loss of CCM Quorum: Ignore Aug 21 12:49:31 ld2 fence_ipmilan: Failed: Unable to obtain correct plug status or plug is not available Aug 21 12:49:31 ld2 stonith-ng[8202]: warning: fence_ipmilan[46088] stderr: [ 2019-08-21 12:49:31,216 ERROR: Failed: Unable to obtain correct plug status or plug is not available ] Aug 21 12:49:31 ld2 stonith-ng[8202]: warning: fence_ipmilan[46088] stderr: [ ] Aug 21 12:49:31 ld2 stonith-ng[8202]: warning: fence_ipmilan[46088] stderr: [ ] Aug 21 12:49:32 ld2 crmd[8206]: notice: Result of start operation for fence-node1 on ld2.mydomain.it: 0 (ok) Aug 21 12:49:32 ld2 stonith-ng[8202]: notice: On loss of CCM Quorum: Ignore Aug 21 12:49:32 ld2 stonith-ng[8202]: notice: On loss of CCM Quorum: Ignore Aug 21 12:49:32 ld2 stonith-ng[8202]: notice: On loss of CCM Quorum: Ignore Aug 21 12:50:01 ld2 systemd: Started Session 28060 of user root. Aug 21 13:00:01 ld2 systemd: Started Session 28061 of user root. Aug 21 13:01:01 ld2 systemd: Started Session 28062 of user root. Aug 21 13:10:01 ld2 systemd: Started Session 28063 of user root. Aug 21 13:20:01 ld2 systemd: Started Session 28064 of user root. Aug 21 13:30:01 ld2 systemd: Started Session 28065 of user root. Aug 21 13:40:01 ld2 systemd: Started Session 28066 of user root. Aug 21 13:50:01 ld2 systemd: Started Session 28067 of user root. Aug 21 14:00:01 ld2 systemd: Started Session 28068 of user root. Aug 21 14:01:01 ld2 systemd: Started Session 28069 of user root. Aug 21 14:10:01 ld2 systemd: Started Session 28070 of user root. Aug 21 14:20:01 ld2 systemd: Started Session 28071 of user root. Aug 21 14:26:08 ld2 stonith-ng[8202]: notice: Child process 4835 performing action 'monitor' timed out with signal 15 Aug 21 14:26:08 ld2 stonith-ng[8202]: notice: Operation 'monitor' [4835] for device 'fence-node1' returned: -62 (Timer expired) Aug 21 14:26:08 ld2 crmd[8206]: error: Result of monitor operation for fence-node1 on ld2.mydomain.it: Timed Out Aug 21 14:26:08 ld2 stonith-ng[8202]: notice: On loss of CCM Quorum: Ignore Aug 21 14:26:09 ld2 crmd[8206]: notice: Result of stop operation for fence-node1 on ld2.mydomain.it: 0 (ok) Aug 21 14:26:09 ld2 stonith-ng[8202]: notice: On loss of CCM Quorum: Ignore Aug 21 14:26:09 ld2 stonith-ng[8202]: notice: On loss of CCM Quorum: Ignore Aug 21 14:26:09 ld2 stonith-ng[8202]: notice: On loss of CCM Quorum: Ignore Aug 21 14:26:29 ld2 stonith-ng[8202]: notice: Child process 4892 performing action 'monitor' timed out with signal 15 Aug 21 14:26:29 ld2 stonith-ng[8202]: notice: Operation 'monitor' [4892] for device 'fence-node1' returned: -62 (Timer expired) Aug 21 14:26:29 ld2 crmd[8206]: error: Result of start operation for fence-node1 on ld2.mydomain.it: Timed Out Aug 21 14:26:29 ld2 stonith-ng[8202]: notice: On loss of CCM Quorum: Ignore Aug 21 14:26:29 ld2 crmd[8206]: notice: Result of stop operation for fence-node1 on ld2.mydomain.it: 0 (ok) Aug 21 14:26:29 ld2 stonith-ng[8202]: notice: On loss of CCM Quorum: Ignore Aug 21 14:26:29 ld2 stonith-ng[8202]: notice: On loss of CCM Quorum: Ignore ###########################NODE 1 /var/log/messages############################## Aug 21 12:48:40 ld1 crmd[8457]: notice: State transition S_IDLE -> S_POLICY_ENGINE Aug 21 12:48:40 ld1 stonith-ng[8453]: notice: On loss of CCM Quorum: Ignore Aug 21 12:48:40 ld1 pengine[8456]: notice: On loss of CCM Quorum: Ignore Aug 21 12:48:40 ld1 pengine[8456]: warning: Processing failed monitor of fence-node2 on ld2.mydomain.it: unknown error Aug 21 12:48:40 ld1 pengine[8456]: notice: * Recover fence-node2 ( ld2.mydomain.it ) Aug 21 12:48:40 ld1 pengine[8456]: notice: Calculated transition 15937, saving inputs in /var/lib/pacemaker/pengine/pe-input-95.bz2 Aug 21 12:48:40 ld1 pengine[8456]: notice: On loss of CCM Quorum: Ignore Aug 21 12:48:40 ld1 pengine[8456]: warning: Processing failed monitor of fence-node2 on ld2.mydomain.it: unknown error Aug 21 12:48:40 ld1 pengine[8456]: notice: * Recover fence-node2 ( ld2.mydomain.it ) Aug 21 12:48:40 ld1 pengine[8456]: notice: Calculated transition 15938, saving inputs in /var/lib/pacemaker/pengine/pe-input-96.bz2 Aug 21 12:48:40 ld1 crmd[8457]: notice: Initiating stop operation fence-node2_stop_0 on ld2.mydomain.it Aug 21 12:48:40 ld1 stonith-ng[8453]: notice: On loss of CCM Quorum: Ignore Aug 21 12:48:40 ld1 crmd[8457]: notice: Initiating start operation fence-node2_start_0 on ld2.mydomain.it Aug 21 12:48:40 ld1 stonith-ng[8453]: notice: On loss of CCM Quorum: Ignore Aug 21 12:48:40 ld1 stonith-ng[8453]: notice: On loss of CCM Quorum: Ignore Aug 21 12:48:43 ld1 snmpd[20884]: refused smux peer: oid SNMPv2-SMI::enterprises.674.10892.1, descr Systems Management SNMP MIB Plug-in Manager Aug 21 12:48:46 ld1 snmpd[20884]: refused smux peer: oid SNMPv2-SMI::enterprises.674.10892.1, descr Systems Management SNMP MIB Plug-in Manager Aug 21 12:48:49 ld1 snmpd[20884]: refused smux peer: oid SNMPv2-SMI::enterprises.674.10892.1, descr Systems Management SNMP MIB Plug-in Manager Aug 21 12:48:52 ld1 snmpd[20884]: refused smux peer: oid SNMPv2-SMI::enterprises.674.10892.1, descr Systems Management SNMP MIB Plug-in Manager Aug 21 12:48:55 ld1 snmpd[20884]: refused smux peer: oid SNMPv2-SMI::enterprises.674.10892.1, descr Systems Management SNMP MIB Plug-in Manager Aug 21 12:48:58 ld1 snmpd[20884]: refused smux peer: oid SNMPv2-SMI::enterprises.674.10892.1, descr Systems Management SNMP MIB Plug-in Manager Aug 21 12:48:59 ld1 stonith-ng[8453]: notice: Child process 13446 performing action 'monitor' timed out with signal 15 Aug 21 12:48:59 ld1 stonith-ng[8453]: notice: Operation 'monitor' [13446] for device 'fence-node1' returned: -62 (Timer expired) Aug 21 12:48:59 ld1 crmd[8457]: error: Result of monitor operation for fence-node1 on ld1.mydomain.it: Timed Out Aug 21 12:48:59 ld1 crmd[8457]: notice: Transition aborted by operation fence-node1_monitor_60000 'create' on ld1.mydomain.it: Old event Aug 21 12:48:59 ld1 stonith-ng[8453]: notice: On loss of CCM Quorum: Ignore Aug 21 12:49:00 ld1 crmd[8457]: warning: Action 14 (fence-node2_start_0) on ld2.mydomain.it failed (target: 0 vs. rc: 1): Error Aug 21 12:49:00 ld1 crmd[8457]: notice: Transition 15938 (Complete=2, Pending=0, Fired=0, Skipped=0, Incomplete=1, Source=/var/lib/pacemaker/pengine/pe-input-96.bz2): Complete Aug 21 12:49:00 ld1 stonith-ng[8453]: notice: On loss of CCM Quorum: Ignore Aug 21 12:49:00 ld1 pengine[8456]: notice: On loss of CCM Quorum: Ignore Aug 21 12:49:00 ld1 pengine[8456]: warning: Processing failed monitor of fence-node1 on ld1.mydomain.it: unknown error Aug 21 12:49:00 ld1 pengine[8456]: warning: Processing failed start of fence-node2 on ld2.mydomain.it: unknown error Aug 21 12:49:00 ld1 pengine[8456]: warning: Processing failed start of fence-node2 on ld2.mydomain.it: unknown error Aug 21 12:49:00 ld1 pengine[8456]: notice: * Recover fence-node1 ( ld1.mydomain.it ) Aug 21 12:49:00 ld1 pengine[8456]: notice: * Recover fence-node2 ( ld2.mydomain.it ) Aug 21 12:49:00 ld1 pengine[8456]: notice: Calculated transition 15939, saving inputs in /var/lib/pacemaker/pengine/pe-input-97.bz2 Aug 21 12:49:00 ld1 pengine[8456]: notice: On loss of CCM Quorum: Ignore Aug 21 12:49:00 ld1 pengine[8456]: warning: Processing failed monitor of fence-node1 on ld1.mydomain.it: unknown error Aug 21 12:49:00 ld1 pengine[8456]: warning: Processing failed start of fence-node2 on ld2.mydomain.it: unknown error Aug 21 12:49:00 ld1 pengine[8456]: warning: Processing failed start of fence-node2 on ld2.mydomain.it: unknown error Aug 21 12:49:00 ld1 pengine[8456]: warning: Forcing fence-node2 away from ld2.mydomain.it after 1000000 failures (max=1000000) Aug 21 12:49:00 ld1 pengine[8456]: notice: * Recover fence-node1 ( ld1.mydomain.it ) Aug 21 12:49:00 ld1 pengine[8456]: notice: * Recover fence-node2 ( ld2.mydomain.it -> ld1.mydomain.it ) Aug 21 12:49:00 ld1 pengine[8456]: notice: Calculated transition 15940, saving inputs in /var/lib/pacemaker/pengine/pe-input-98.bz2 Aug 21 12:49:00 ld1 crmd[8457]: notice: Initiating stop operation fence-node1_stop_0 locally on ld1.mydomain.it Aug 21 12:49:00 ld1 stonith-ng[8453]: notice: On loss of CCM Quorum: Ignore Aug 21 12:49:00 ld1 crmd[8457]: notice: Initiating stop operation fence-node2_stop_0 on ld2.mydomain.it Aug 21 12:49:00 ld1 crmd[8457]: notice: Result of stop operation for fence-node1 on ld1.mydomain.it: 0 (ok) Aug 21 12:49:00 ld1 crmd[8457]: notice: Initiating start operation fence-node1_start_0 locally on ld1.mydomain.it Aug 21 12:49:00 ld1 stonith-ng[8453]: notice: On loss of CCM Quorum: Ignore Aug 21 12:49:00 ld1 crmd[8457]: notice: Initiating start operation fence-node2_start_0 locally on ld1.mydomain.it Aug 21 12:49:00 ld1 stonith-ng[8453]: notice: On loss of CCM Quorum: Ignore Aug 21 12:49:00 ld1 stonith-ng[8453]: notice: On loss of CCM Quorum: Ignore Aug 21 12:49:00 ld1 stonith-ng[8453]: notice: On loss of CCM Quorum: Ignore Aug 21 12:49:00 ld1 stonith-ng[8453]: notice: On loss of CCM Quorum: Ignore Aug 21 12:49:01 ld1 snmpd[20884]: refused smux peer: oid SNMPv2-SMI::enterprises.674.10892.1, descr Systems Management SNMP MIB Plug-in Manager Aug 21 12:49:04 ld1 snmpd[20884]: refused smux peer: oid SNMPv2-SMI::enterprises.674.10892.1, descr Systems Management SNMP MIB Plug-in Manager Aug 21 12:49:07 ld1 snmpd[20884]: refused smux peer: oid SNMPv2-SMI::enterprises.674.10892.1, descr Systems Management SNMP MIB Plug-in Manager Aug 21 12:49:10 ld1 snmpd[20884]: refused smux peer: oid SNMPv2-SMI::enterprises.674.10892.1, descr Systems Management SNMP MIB Plug-in Manager Aug 21 12:49:13 ld1 snmpd[20884]: refused smux peer: oid SNMPv2-SMI::enterprises.674.10892.1, descr Systems Management SNMP MIB Plug-in Manager Aug 21 12:49:16 ld1 snmpd[20884]: refused smux peer: oid SNMPv2-SMI::enterprises.674.10892.1, descr Systems Management SNMP MIB Plug-in Manager Aug 21 12:49:19 ld1 snmpd[20884]: refused smux peer: oid SNMPv2-SMI::enterprises.674.10892.1, descr Systems Management SNMP MIB Plug-in Manager Aug 21 12:49:20 ld1 stonith-ng[8453]: notice: Child process 13654 performing action 'monitor' timed out with signal 15 Aug 21 12:49:20 ld1 stonith-ng[8453]: notice: Operation 'monitor' [13654] for device 'fence-node1' returned: -62 (Timer expired) Aug 21 12:49:20 ld1 crmd[8457]: error: Result of start operation for fence-node1 on ld1.mydomain.it: Timed Out Aug 21 12:49:20 ld1 crmd[8457]: warning: Action 12 (fence-node1_start_0) on ld1.mydomain.it failed (target: 0 vs. rc: 1): Error Aug 21 12:49:20 ld1 crmd[8457]: notice: Transition aborted by operation fence-node1_start_0 'modify' on ld1.mydomain.it: Event failed Aug 21 12:49:20 ld1 stonith-ng[8453]: notice: On loss of CCM Quorum: Ignore Aug 21 12:49:20 ld1 crmd[8457]: notice: Transition aborted by status-1-fail-count-fence-node1.start_0 doing create fail-count-fence-node1#start_0=INFINITY: Transient attribute change Aug 21 12:49:20 ld1 stonith-ng[8453]: notice: Child process 13656 performing action 'monitor' timed out with signal 15 Aug 21 12:49:20 ld1 stonith-ng[8453]: notice: Operation 'monitor' [13656] for device 'fence-node2' returned: -62 (Timer expired) Aug 21 12:49:21 ld1 crmd[8457]: error: Result of start operation for fence-node2 on ld1.mydomain.it: Timed Out Aug 21 12:49:21 ld1 crmd[8457]: warning: Action 13 (fence-node2_start_0) on ld1.mydomain.it failed (target: 0 vs. rc: 1): Error Aug 21 12:49:21 ld1 crmd[8457]: notice: Transition 15940 (Complete=4, Pending=0, Fired=0, Skipped=0, Incomplete=2, Source=/var/lib/pacemaker/pengine/pe-input-98.bz2): Complete Aug 21 12:49:21 ld1 stonith-ng[8453]: notice: On loss of CCM Quorum: Ignore Aug 21 12:49:21 ld1 pengine[8456]: notice: On loss of CCM Quorum: Ignore Aug 21 12:49:21 ld1 pengine[8456]: warning: Processing failed start of fence-node1 on ld1.mydomain.it: unknown error Aug 21 12:49:21 ld1 pengine[8456]: warning: Processing failed start of fence-node1 on ld1.mydomain.it: unknown error Aug 21 12:49:21 ld1 pengine[8456]: warning: Processing failed start of fence-node2 on ld1.mydomain.it: unknown error Aug 21 12:49:21 ld1 pengine[8456]: warning: Processing failed start of fence-node2 on ld1.mydomain.it: unknown error Aug 21 12:49:21 ld1 pengine[8456]: warning: Processing failed start of fence-node2 on ld2.mydomain.it: unknown error Aug 21 12:49:21 ld1 pengine[8456]: warning: Forcing fence-node1 away from ld1.mydomain.it after 1000000 failures (max=1000000) Aug 21 12:49:21 ld1 pengine[8456]: warning: Forcing fence-node2 away from ld2.mydomain.it after 1000000 failures (max=1000000) Aug 21 12:49:21 ld1 pengine[8456]: notice: * Recover fence-node1 ( ld1.mydomain.it -> ld2.mydomain.it ) Aug 21 12:49:21 ld1 pengine[8456]: notice: * Recover fence-node2 ( ld1.mydomain.it ) Aug 21 12:49:21 ld1 pengine[8456]: notice: Calculated transition 15941, saving inputs in /var/lib/pacemaker/pengine/pe-input-99.bz2 Aug 21 12:49:21 ld1 pengine[8456]: notice: On loss of CCM Quorum: Ignore Aug 21 12:49:21 ld1 pengine[8456]: warning: Processing failed start of fence-node1 on ld1.mydomain.it: unknown error Aug 21 12:49:21 ld1 pengine[8456]: warning: Processing failed start of fence-node1 on ld1.mydomain.it: unknown error Aug 21 12:49:21 ld1 pengine[8456]: warning: Processing failed start of fence-node2 on ld1.mydomain.it: unknown error Aug 21 12:49:21 ld1 pengine[8456]: warning: Processing failed start of fence-node2 on ld1.mydomain.it: unknown error Aug 21 12:49:21 ld1 pengine[8456]: warning: Processing failed start of fence-node2 on ld2.mydomain.it: unknown error Aug 21 12:49:21 ld1 pengine[8456]: warning: Forcing fence-node1 away from ld1.mydomain.it after 1000000 failures (max=1000000) Aug 21 12:49:21 ld1 pengine[8456]: warning: Forcing fence-node2 away from ld1.mydomain.it after 1000000 failures (max=1000000) Aug 21 12:49:21 ld1 pengine[8456]: warning: Forcing fence-node2 away from ld2.mydomain.it after 1000000 failures (max=1000000) Aug 21 12:49:21 ld1 pengine[8456]: notice: * Recover fence-node1 ( ld1.mydomain.it -> ld2.mydomain.it ) Aug 21 12:49:21 ld1 pengine[8456]: notice: * Stop fence-node2 ( ld1.mydomain.it ) due to node availability Aug 21 12:49:21 ld1 pengine[8456]: notice: Calculated transition 15942, saving inputs in /var/lib/pacemaker/pengine/pe-input-100.bz2 Aug 21 12:49:21 ld1 crmd[8457]: notice: Initiating stop operation fence-node1_stop_0 locally on ld1.mydomain.it Aug 21 12:49:21 ld1 crmd[8457]: notice: Initiating stop operation fence-node2_stop_0 locally on ld1.mydomain.it Aug 21 12:49:21 ld1 crmd[8457]: notice: Result of stop operation for fence-node1 on ld1.mydomain.it: 0 (ok) Aug 21 12:49:21 ld1 crmd[8457]: notice: Result of stop operation for fence-node2 on ld1.mydomain.it: 0 (ok) Aug 21 12:49:21 ld1 stonith-ng[8453]: notice: On loss of CCM Quorum: Ignore Aug 21 12:49:21 ld1 stonith-ng[8453]: notice: On loss of CCM Quorum: Ignore Aug 21 12:49:21 ld1 crmd[8457]: notice: Initiating start operation fence-node1_start_0 on ld2.mydomain.it Aug 21 12:49:21 ld1 stonith-ng[8453]: notice: On loss of CCM Quorum: Ignore Aug 21 12:49:21 ld1 stonith-ng[8453]: notice: On loss of CCM Quorum: Ignore Aug 21 12:49:21 ld1 stonith-ng[8453]: notice: On loss of CCM Quorum: Ignore Aug 21 12:49:22 ld1 snmpd[20884]: refused smux peer: oid SNMPv2-SMI::enterprises.674.10892.1, descr Systems Management SNMP MIB Plug-in Manager Aug 21 12:49:25 ld1 snmpd[20884]: refused smux peer: oid SNMPv2-SMI::enterprises.674.10892.1, descr Systems Management SNMP MIB Plug-in Manager Aug 21 12:49:28 ld1 snmpd[20884]: refused smux peer: oid SNMPv2-SMI::enterprises.674.10892.1, descr Systems Management SNMP MIB Plug-in Manager Aug 21 12:49:31 ld1 snmpd[20884]: refused smux peer: oid SNMPv2-SMI::enterprises.674.10892.1, descr Systems Management SNMP MIB Plug-in Manager Aug 21 12:49:32 ld1 crmd[8457]: notice: Initiating monitor operation fence-node1_monitor_60000 on ld2.mydomain.it Aug 21 12:49:32 ld1 stonith-ng[8453]: notice: On loss of CCM Quorum: Ignore Aug 21 12:49:32 ld1 stonith-ng[8453]: notice: On loss of CCM Quorum: Ignore Aug 21 12:49:32 ld1 crmd[8457]: notice: Transition 15942 (Complete=4, Pending=0, Fired=0, Skipped=0, Incomplete=0, Source=/var/lib/pacemaker/pengine/pe-input-100.bz2): Complete Aug 21 12:49:32 ld1 crmd[8457]: notice: State transition S_TRANSITION_ENGINE -> S_IDLE Aug 21 12:49:32 ld1 stonith-ng[8453]: notice: On loss of CCM Quorum: Ignore
_______________________________________________ Manage your subscription: https://lists.clusterlabs.org/mailman/listinfo/users ClusterLabs home: https://www.clusterlabs.org/