Re: [ClusterLabs] Can't See Why This Cluster Failed Over
Eric Robinson writes: >> crm configure show xml c_clust19 > > Here is what I am entering using crmsh (version 2.0-1): > > > colocation c_clust19 inf: [ p_mysql_057 p_mysql_092 p_mysql_187 ] > p_vip_clust19 p_fs_clust19 p_lv_on_drbd0 ms_drbd0:Master > order o_clust19 inf: ms_drbd0:promote p_lv_on_drbd0 p_fs_clust19 > p_vip_clust19 [ p_mysql_057 p_mysql_092 p_mysql_187 ] > > > After I save it, I get no errors, but it converts it to this... > > > colocation c_clust19 inf: [ p_mysql_057 p_mysql_092 p_mysql_187 ] ( > p_vip_clust19:Master p_fs_clust19:Master p_lv_on_drbd0:Master ) ( > ms_drbd0:Master ) > order o_clust19 inf: ms_drbd0:promote ( p_lv_on_drbd0:start > p_fs_clust19:start p_vip_clust19:start ) [ p_mysql_057 p_mysql_092 > p_mysql_187 ] > > This looks incorrect to me. > > Here is the xml that it generates. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > The resources in set c_clust19-1 should start sequentially, starting with > p_lv_on_drbd0 and ending with p_vip_clust19. I also don't understand why > p_lv_on_drbd0 and p_vip_clust19 are getting the Master designation. Hi, Yeah, that does indeed look like a bug.. One thing that is confusing and may be one reason why things get split in an unexpected way is because as you can see, the role attribute is applied per resource set, while it looks like it applies per resource in the crmsh syntax. So the shell does some complex logic to "split" sets based on role assignment. Cheers, Kristoffer > > -- > Eric Robinson > > -- // Kristoffer Grönlund // kgronl...@suse.com ___ Users mailing list: Users@clusterlabs.org http://lists.clusterlabs.org/mailman/listinfo/users Project Home: http://www.clusterlabs.org Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf Bugs: http://bugs.clusterlabs.org
Re: [ClusterLabs] Can't See Why This Cluster Failed Over
> crm configure show xml c_clust19 Here is what I am entering using crmsh (version 2.0-1): colocation c_clust19 inf: [ p_mysql_057 p_mysql_092 p_mysql_187 ] p_vip_clust19 p_fs_clust19 p_lv_on_drbd0 ms_drbd0:Master order o_clust19 inf: ms_drbd0:promote p_lv_on_drbd0 p_fs_clust19 p_vip_clust19 [ p_mysql_057 p_mysql_092 p_mysql_187 ] After I save it, I get no errors, but it converts it to this... colocation c_clust19 inf: [ p_mysql_057 p_mysql_092 p_mysql_187 ] ( p_vip_clust19:Master p_fs_clust19:Master p_lv_on_drbd0:Master ) ( ms_drbd0:Master ) order o_clust19 inf: ms_drbd0:promote ( p_lv_on_drbd0:start p_fs_clust19:start p_vip_clust19:start ) [ p_mysql_057 p_mysql_092 p_mysql_187 ] This looks incorrect to me. Here is the xml that it generates. The resources in set c_clust19-1 should start sequentially, starting with p_lv_on_drbd0 and ending with p_vip_clust19. I also don't understand why p_lv_on_drbd0 and p_vip_clust19 are getting the Master designation. -- Eric Robinson ___ Users mailing list: Users@clusterlabs.org http://lists.clusterlabs.org/mailman/listinfo/users Project Home: http://www.clusterlabs.org Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf Bugs: http://bugs.clusterlabs.org
Re: [ClusterLabs] Can't See Why This Cluster Failed Over
Eric Robinson writes: > Here's the config. I don't know why the CRM put in the parenthesis where it > did. That's not the way I typed it. I usually have all my mysql instances > between parenthesis and everything else outside. [ ...] > colocation c_clust19 inf: ( p_mysql_057 p_mysql_092 p_mysql_187 p_mysql_213 > p_mysql_250 p_mysql_289 p_mysql_312 p_vip_clust19 p_mysql_702 p_mysql_743 > p_mysql_745 p_mysql_746 p_fs_clust19 p_lv_on_drbd0 ) ( ms_drbd0:Master ) > colocation c_clust20 inf: p_vip_clust20 p_fs_clust20 p_lv_on_drbd1 > ms_drbd1:Master > order o_clust19 inf: ms_drbd0:promote ( p_lv_on_drbd0:start ) ( p_fs_clust19 > p_vip_clust19 ) ( p_mysql_057 p_mysql_092 p_mysql_187 p_mysql_213 p_mysql_250 > p_mysql_289 p_mysql_312 p_mysql_702 p_mysql_743 p_mysql_745 p_mysql_746 ) This might be a bug in crmsh: What was the expression you intended to write, and which version of crmsh do you have? You can see the resulting XML that crmsh generates and then re-parses into the line syntax using crm configure show xml c_clust19 Cheers, Kristoffer -- // Kristoffer Grönlund // kgronl...@suse.com ___ Users mailing list: Users@clusterlabs.org http://lists.clusterlabs.org/mailman/listinfo/users Project Home: http://www.clusterlabs.org Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf Bugs: http://bugs.clusterlabs.org
Re: [ClusterLabs] Can't See Why This Cluster Failed Over
>> Apr 03 08:50:40 [22767] ha14a crmd: notice: process_lrm_event: >> ha14a-p_mysql_746_start_0:148 [ starting mysqld... \ngave up waiting\n ] > This is where your trouble starts. p_mysql_746 fails to start; the init > script returned exit code 1, and showed the output in the above line. A second set of eyes does wonders. I totally missed that! > I can't see the full configuration, but I'm guessing the colocation > added with the new resource means other resources now depend on it, so > they get stopped. Perhaps you got the sense of the colocation backwards, > i.e. you wanted the new resource to be placed with the old ones, rather > than vice versa? I think I may have something wrong in the config. My goal is: VIP --> depends on --> Filesystem --> depends on --> LVM --> depends on --> drbd p_mysql_057 --> depends on --> VIP p_mysql_092 --> depends on --> VIP p_mysql_187 --> depends on --> VIP etc... Moving VIP, Filesystem, LVP, or drbd moves the whole stack. Each MySQL instance should be able to stop or start without impacting the others. Here's the config. I don't know why the CRM put in the parenthesis where it did. That's not the way I typed it. I usually have all my mysql instances between parenthesis and everything else outside. node ha14a node ha14b primitive p_drbd0 ocf:linbit:drbd \ params drbd_resource=ha01_mysql \ op monitor interval=31s role=Slave \ op monitor interval=30s role=Master primitive p_drbd1 ocf:linbit:drbd \ params drbd_resource=ha02_mysql \ op monitor interval=29s role=Slave \ op monitor interval=28s role=Master primitive p_fs_clust19 Filesystem \ params device="/dev/vg_on_drbd0/lv_on_drbd0" directory="/ha01_mysql" fstype=ext4 options=noatime \ meta target-role=Started primitive p_fs_clust20 Filesystem \ params device="/dev/vg_on_drbd1/lv_on_drbd1" directory="/ha02_mysql" fstype=ext4 options=noatime primitive p_lv_on_drbd0 LVM \ params volgrpname=vg_on_drbd0 \ meta target-role=Started primitive p_lv_on_drbd1 LVM \ params volgrpname=vg_on_drbd1 primitive p_mysql_057 lsb:mysql_057 primitive p_mysql_092 lsb:mysql_092 primitive p_mysql_187 lsb:mysql_187 primitive p_mysql_213 lsb:mysql_213 primitive p_mysql_250 lsb:mysql_250 primitive p_mysql_289 lsb:mysql_289 primitive p_mysql_312 lsb:mysql_312 primitive p_mysql_702 lsb:mysql_702 primitive p_mysql_743 lsb:mysql_743 primitive p_mysql_745 lsb:mysql_745 primitive p_mysql_746 lsb:mysql_746 primitive p_vip_clust19 IPaddr2 \ params ip=192.168.9.106 cidr_netmask=32 \ op monitor interval=30s \ meta target-role=Started primitive p_vip_clust20 IPaddr2 \ params ip=192.168.9.107 cidr_netmask=32 \ op monitor interval=29s ms ms_drbd0 p_drbd0 \ meta master-max=1 master-node-max=1 clone-max=2 clone-node-max=1 notify=true target-role=Master ms ms_drbd1 p_drbd1 \ meta master-max=1 master-node-max=1 clone-max=2 clone-node-max=1 notify=true target-role=Master colocation c_clust19 inf: ( p_mysql_057 p_mysql_092 p_mysql_187 p_mysql_213 p_mysql_250 p_mysql_289 p_mysql_312 p_vip_clust19 p_mysql_702 p_mysql_743 p_mysql_745 p_mysql_746 p_fs_clust19 p_lv_on_drbd0 ) ( ms_drbd0:Master ) colocation c_clust20 inf: p_vip_clust20 p_fs_clust20 p_lv_on_drbd1 ms_drbd1:Master order o_clust19 inf: ms_drbd0:promote ( p_lv_on_drbd0:start ) ( p_fs_clust19 p_vip_clust19 ) ( p_mysql_057 p_mysql_092 p_mysql_187 p_mysql_213 p_mysql_250 p_mysql_289 p_mysql_312 p_mysql_702 p_mysql_743 p_mysql_745 p_mysql_746 ) order o_clust20 inf: ms_drbd1:promote p_lv_on_drbd1 p_fs_clust20 p_vip_clust20 property cib-bootstrap-options: \ dc-version=1.1.11-97629de \ cluster-infrastructure="classic openais (with plugin)" \ expected-quorum-votes=2 \ no-quorum-policy=ignore \ stonith-enabled=false \ maintenance-mode=false \ last-lrm-refresh=1484558149 ___ Users mailing list: Users@clusterlabs.org http://lists.clusterlabs.org/mailman/listinfo/users Project Home: http://www.clusterlabs.org Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf Bugs: http://bugs.clusterlabs.org
Re: [ClusterLabs] Can't See Why This Cluster Failed Over
On 04/07/2017 12:58 PM, Eric Robinson wrote: > Somebody want to look at this log and tell me why the cluster failed over? > All we did was add a new resource. We've done it many times before without > any problems. > > -- > > Apr 03 08:50:30 [22762] ha14acib: info: cib_process_request: > Forwarding cib_apply_diff operation for section 'all' to master > (origin=local/cibadmin/2) > Apr 03 08:50:30 [22762] ha14acib: info: cib_perform_op: Diff: > --- 0.605.2 2 > Apr 03 08:50:30 [22762] ha14acib: info: cib_perform_op: Diff: > +++ 0.607.0 65654c97e62cd549f22f777a5290fe3a > Apr 03 08:50:30 [22762] ha14acib: info: cib_perform_op: + > /cib: @epoch=607, @num_updates=0 > Apr 03 08:50:30 [22762] ha14acib: info: cib_perform_op: ++ > /cib/configuration/resources: type="mysql_745"/> > Apr 03 08:50:30 [22762] ha14acib: info: cib_perform_op: ++ > /cib/configuration/resources: type="mysql_746"/> > Apr 03 08:50:30 [22762] ha14acib: info: cib_perform_op: ++ > /cib/configuration/constraints/rsc_colocation[@id='c_clust19']/resource_set[@id='c_clust19-0']: > > Apr 03 08:50:30 [22762] ha14acib: info: cib_perform_op: ++ > /cib/configuration/constraints/rsc_colocation[@id='c_clust19']/resource_set[@id='c_clust19-0']: > > Apr 03 08:50:30 [22762] ha14acib: info: cib_perform_op: ++ > /cib/configuration/constraints/rsc_order[@id='o_clust19']/resource_set[@id='o_clust19-3']: > > Apr 03 08:50:30 [22762] ha14acib: info: cib_perform_op: ++ > /cib/configuration/constraints/rsc_order[@id='o_clust19']/resource_set[@id='o_clust19-3']: > > Apr 03 08:50:30 [22762] ha14acib: info: cib_process_request: > Completed cib_apply_diff operation for section 'all': OK (rc=0, > origin=ha14a/cibadmin/2, version=0.607.0) > Apr 03 08:50:30 [22762] ha14acib: info: write_cib_contents: > Archived previous version as /var/lib/pacemaker/cib/cib-36.raw > Apr 03 08:50:30 [22762] ha14acib: info: write_cib_contents: > Wrote version 0.607.0 of the CIB to disk (digest: > 1afdb9e480f870a095aa9e39719d29c4) > Apr 03 08:50:30 [22762] ha14acib: info: retrieveCib: > Reading cluster configuration from: /var/lib/pacemaker/cib/cib.DkIgSs > (digest: /var/lib/pacemaker/cib/cib.hPwa66) > Apr 03 08:50:30 [22764] ha14a lrmd: info: > process_lrmd_get_rsc_info: Resource 'p_mysql_745' not found (17 active > resources) > Apr 03 08:50:30 [22764] ha14a lrmd: info: > process_lrmd_rsc_register: Added 'p_mysql_745' to the rsc list (18 active > resources) > Apr 03 08:50:30 [22767] ha14a crmd: info: do_lrm_rsc_op: > Performing key=10:7484:7:91ef4b03-8769-47a1-a364-060569c46e52 > op=p_mysql_745_monitor_0 > Apr 03 08:50:30 [22764] ha14a lrmd: info: > process_lrmd_get_rsc_info: Resource 'p_mysql_746' not found (18 active > resources) > Apr 03 08:50:30 [22764] ha14a lrmd: info: > process_lrmd_rsc_register: Added 'p_mysql_746' to the rsc list (19 active > resources) > Apr 03 08:50:30 [22767] ha14a crmd: info: do_lrm_rsc_op: > Performing key=11:7484:7:91ef4b03-8769-47a1-a364-060569c46e52 > op=p_mysql_746_monitor_0 > Apr 03 08:50:30 [22762] ha14acib: info: cib_perform_op: Diff: > --- 0.607.0 2 > Apr 03 08:50:30 [22762] ha14acib: info: cib_perform_op: Diff: > +++ 0.607.1 (null) > Apr 03 08:50:30 [22762] ha14acib: info: cib_perform_op: + > /cib: @num_updates=1 > Apr 03 08:50:30 [22762] ha14acib: info: cib_perform_op: ++ > /cib/status/node_state[@id='ha14b']/lrm[@id='ha14b']/lrm_resources: > > Apr 03 08:50:30 [22762] ha14acib: info: cib_perform_op: ++ > > operation="monitor" crm-debug-origin="do_update_resource" > crm_feature_set="3.0.9" > transition-key="13:7484:7:91ef4b03-8769-47a1-a364-060569c46e52" > transition-magic="0:7;13:7484:7:91ef4b03-8769-47a1-a364-060569c46e52" > call-id="142" rc-code="7" op-status="0" interval="0" last-run="1491234630" las > Apr 03 08:50:30 [22762] ha14acib: info: cib_perform_op: ++ > > > Apr 03 08:50:30 [22762] ha14acib: info: cib_process_request: > Completed cib_modify operation for section status: OK (rc=0, > origin=ha14b/crmd/7665, version=0.607.1) > Apr 03 08:50:30 [22762] ha14acib: info: cib_perform_op: Diff: > --- 0.607.1 2 > Apr 03 08:50:30 [22762] ha14acib: info: cib_perform_op: Diff: > +++ 0.607.2 (null) > Apr 03 08:50:30 [22762] ha14acib: info: cib_perform_op: + > /cib: @num_updates=2 > Apr 03 08:50:30 [22762] ha14acib: info: cib_perform_op: ++ > /cib/status/node_s
Re: [ClusterLabs] Can't See Why This Cluster Failed Over
Somebody want to look at this log and tell me why the cluster failed over? All we did was add a new resource. We've done it many times before without any problems. -- Apr 03 08:50:30 [22762] ha14acib: info: cib_process_request: Forwarding cib_apply_diff operation for section 'all' to master (origin=local/cibadmin/2) Apr 03 08:50:30 [22762] ha14acib: info: cib_perform_op: Diff: --- 0.605.2 2 Apr 03 08:50:30 [22762] ha14acib: info: cib_perform_op: Diff: +++ 0.607.0 65654c97e62cd549f22f777a5290fe3a Apr 03 08:50:30 [22762] ha14acib: info: cib_perform_op: + /cib: @epoch=607, @num_updates=0 Apr 03 08:50:30 [22762] ha14acib: info: cib_perform_op: ++ /cib/configuration/resources: Apr 03 08:50:30 [22762] ha14acib: info: cib_perform_op: ++ /cib/configuration/resources: Apr 03 08:50:30 [22762] ha14acib: info: cib_perform_op: ++ /cib/configuration/constraints/rsc_colocation[@id='c_clust19']/resource_set[@id='c_clust19-0']: Apr 03 08:50:30 [22762] ha14acib: info: cib_perform_op: ++ /cib/configuration/constraints/rsc_colocation[@id='c_clust19']/resource_set[@id='c_clust19-0']: Apr 03 08:50:30 [22762] ha14acib: info: cib_perform_op: ++ /cib/configuration/constraints/rsc_order[@id='o_clust19']/resource_set[@id='o_clust19-3']: Apr 03 08:50:30 [22762] ha14acib: info: cib_perform_op: ++ /cib/configuration/constraints/rsc_order[@id='o_clust19']/resource_set[@id='o_clust19-3']: Apr 03 08:50:30 [22762] ha14acib: info: cib_process_request: Completed cib_apply_diff operation for section 'all': OK (rc=0, origin=ha14a/cibadmin/2, version=0.607.0) Apr 03 08:50:30 [22762] ha14acib: info: write_cib_contents: Archived previous version as /var/lib/pacemaker/cib/cib-36.raw Apr 03 08:50:30 [22762] ha14acib: info: write_cib_contents: Wrote version 0.607.0 of the CIB to disk (digest: 1afdb9e480f870a095aa9e39719d29c4) Apr 03 08:50:30 [22762] ha14acib: info: retrieveCib:Reading cluster configuration from: /var/lib/pacemaker/cib/cib.DkIgSs (digest: /var/lib/pacemaker/cib/cib.hPwa66) Apr 03 08:50:30 [22764] ha14a lrmd: info: process_lrmd_get_rsc_info: Resource 'p_mysql_745' not found (17 active resources) Apr 03 08:50:30 [22764] ha14a lrmd: info: process_lrmd_rsc_register: Added 'p_mysql_745' to the rsc list (18 active resources) Apr 03 08:50:30 [22767] ha14a crmd: info: do_lrm_rsc_op: Performing key=10:7484:7:91ef4b03-8769-47a1-a364-060569c46e52 op=p_mysql_745_monitor_0 Apr 03 08:50:30 [22764] ha14a lrmd: info: process_lrmd_get_rsc_info: Resource 'p_mysql_746' not found (18 active resources) Apr 03 08:50:30 [22764] ha14a lrmd: info: process_lrmd_rsc_register: Added 'p_mysql_746' to the rsc list (19 active resources) Apr 03 08:50:30 [22767] ha14a crmd: info: do_lrm_rsc_op: Performing key=11:7484:7:91ef4b03-8769-47a1-a364-060569c46e52 op=p_mysql_746_monitor_0 Apr 03 08:50:30 [22762] ha14acib: info: cib_perform_op: Diff: --- 0.607.0 2 Apr 03 08:50:30 [22762] ha14acib: info: cib_perform_op: Diff: +++ 0.607.1 (null) Apr 03 08:50:30 [22762] ha14acib: info: cib_perform_op: + /cib: @num_updates=1 Apr 03 08:50:30 [22762] ha14acib: info: cib_perform_op: ++ /cib/status/node_state[@id='ha14b']/lrm[@id='ha14b']/lrm_resources: Apr 03 08:50:30 [22762] ha14acib: info: cib_perform_op: ++ Apr 03 08:50:30 [22762] ha14acib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=ha14b/crmd/7665, version=0.607.1) Apr 03 08:50:30 [22762] ha14acib: info: cib_perform_op: Diff: --- 0.607.1 2 Apr 03 08:50:30 [22762] ha14acib: info: cib_perform_op: Diff: +++ 0.607.2 (null) Apr 03 08:50:30 [22762] ha14acib: info: cib_perform_op: + /cib: @num_updates=2 Apr 03 08:50:30 [22762] ha14acib: info: cib_perform_op: ++ /cib/status/node_state[@id='ha14b']/lrm[@id='ha14b']/lrm_resources: Apr 03 08:50:30 [22762] ha14acib: info: cib_perform_op: ++ Apr 03 08:50:30 [22762] ha14acib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=ha14b/crmd/7666, version=0.607.2) Apr 03 08:50:30 [22767] ha14a crmd: notice: process_lrm_event: Operation p_mysql_745_monitor_0: not running (node=ha14a, call=142, rc=7, cib-update=88, confirmed=true) Apr 03 08:50:30 [22767] ha14a crmd: notice: process_lrm_event: ha14a-p_mysql_745_monitor_0:142 [ not started\n ] Apr 03 08:50:30 [22762] ha1
[ClusterLabs] Can't See Why This Cluster Failed Over
Somebody want to look at this log and tell me why the cluster failed over? All we did was add a new resource. We've done it many times before without any problems. -- Apr 03 08:50:30 [22762] ha14acib: info: cib_process_request: Forwarding cib_apply_diff operation for section 'all' to master (origin=local/cibadmin/2) Apr 03 08:50:30 [22762] ha14acib: info: cib_perform_op: Diff: --- 0.605.2 2 Apr 03 08:50:30 [22762] ha14acib: info: cib_perform_op: Diff: +++ 0.607.0 65654c97e62cd549f22f777a5290fe3a Apr 03 08:50:30 [22762] ha14acib: info: cib_perform_op: + /cib: @epoch=607, @num_updates=0 Apr 03 08:50:30 [22762] ha14acib: info: cib_perform_op: ++ /cib/configuration/resources: Apr 03 08:50:30 [22762] ha14acib: info: cib_perform_op: ++ /cib/configuration/resources: Apr 03 08:50:30 [22762] ha14acib: info: cib_perform_op: ++ /cib/configuration/constraints/rsc_colocation[@id='c_clust19']/resource_set[@id='c_clust19-0']: Apr 03 08:50:30 [22762] ha14acib: info: cib_perform_op: ++ /cib/configuration/constraints/rsc_colocation[@id='c_clust19']/resource_set[@id='c_clust19-0']: Apr 03 08:50:30 [22762] ha14acib: info: cib_perform_op: ++ /cib/configuration/constraints/rsc_order[@id='o_clust19']/resource_set[@id='o_clust19-3']: Apr 03 08:50:30 [22762] ha14acib: info: cib_perform_op: ++ /cib/configuration/constraints/rsc_order[@id='o_clust19']/resource_set[@id='o_clust19-3']: Apr 03 08:50:30 [22762] ha14acib: info: cib_process_request: Completed cib_apply_diff operation for section 'all': OK (rc=0, origin=ha14a/cibadmin/2, version=0.607.0) Apr 03 08:50:30 [22762] ha14acib: info: write_cib_contents: Archived previous version as /var/lib/pacemaker/cib/cib-36.raw Apr 03 08:50:30 [22762] ha14acib: info: write_cib_contents: Wrote version 0.607.0 of the CIB to disk (digest: 1afdb9e480f870a095aa9e39719d29c4) Apr 03 08:50:30 [22762] ha14acib: info: retrieveCib:Reading cluster configuration from: /var/lib/pacemaker/cib/cib.DkIgSs (digest: /var/lib/pacemaker/cib/cib.hPwa66) Apr 03 08:50:30 [22764] ha14a lrmd: info: process_lrmd_get_rsc_info: Resource 'p_mysql_745' not found (17 active resources) Apr 03 08:50:30 [22764] ha14a lrmd: info: process_lrmd_rsc_register: Added 'p_mysql_745' to the rsc list (18 active resources) Apr 03 08:50:30 [22767] ha14a crmd: info: do_lrm_rsc_op: Performing key=10:7484:7:91ef4b03-8769-47a1-a364-060569c46e52 op=p_mysql_745_monitor_0 Apr 03 08:50:30 [22764] ha14a lrmd: info: process_lrmd_get_rsc_info: Resource 'p_mysql_746' not found (18 active resources) Apr 03 08:50:30 [22764] ha14a lrmd: info: process_lrmd_rsc_register: Added 'p_mysql_746' to the rsc list (19 active resources) Apr 03 08:50:30 [22767] ha14a crmd: info: do_lrm_rsc_op: Performing key=11:7484:7:91ef4b03-8769-47a1-a364-060569c46e52 op=p_mysql_746_monitor_0 Apr 03 08:50:30 [22762] ha14acib: info: cib_perform_op: Diff: --- 0.607.0 2 Apr 03 08:50:30 [22762] ha14acib: info: cib_perform_op: Diff: +++ 0.607.1 (null) Apr 03 08:50:30 [22762] ha14acib: info: cib_perform_op: + /cib: @num_updates=1 Apr 03 08:50:30 [22762] ha14acib: info: cib_perform_op: ++ /cib/status/node_state[@id='ha14b']/lrm[@id='ha14b']/lrm_resources: Apr 03 08:50:30 [22762] ha14acib: info: cib_perform_op: ++ Apr 03 08:50:30 [22762] ha14acib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=ha14b/crmd/7665, version=0.607.1) Apr 03 08:50:30 [22762] ha14acib: info: cib_perform_op: Diff: --- 0.607.1 2 Apr 03 08:50:30 [22762] ha14acib: info: cib_perform_op: Diff: +++ 0.607.2 (null) Apr 03 08:50:30 [22762] ha14acib: info: cib_perform_op: + /cib: @num_updates=2 Apr 03 08:50:30 [22762] ha14acib: info: cib_perform_op: ++ /cib/status/node_state[@id='ha14b']/lrm[@id='ha14b']/lrm_resources: Apr 03 08:50:30 [22762] ha14acib: info: cib_perform_op: ++ Apr 03 08:50:30 [22762] ha14acib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=ha14b/crmd/7666, version=0.607.2) Apr 03 08:50:30 [22767] ha14a crmd: notice: process_lrm_event: Operation p_mysql_745_monitor_0: not running (node=ha14a, call=142, rc=7, cib-update=88, confirmed=true) Apr 03 08:50:30 [22767] ha14a crmd: notice: process_lrm_event: ha14a-p_mysql_745_monitor_0:142 [ not started\n ] Apr 03 08:50:30 [22762] ha1
