Hi I am running a 2 node cluster with this config on centos 6.6
Master/Slave Set: foo-master [foo] Masters: [ kamet ] Slaves: [ orana ] fence-uc-orana (stonith:fence_ilo4): Started kamet fence-uc-kamet (stonith:fence_ilo4): Started orana C-3 (ocf::pw:IPaddr): Started kamet C-FLT (ocf::pw:IPaddr): Started kamet C-FLT2 (ocf::pw:IPaddr): Started kamet E-3 (ocf::pw:IPaddr): Started kamet MGMT-FLT (ocf::pw:IPaddr): Started kamet M-FLT (ocf::pw:IPaddr): Started kamet M-FLT2 (ocf::pw:IPaddr): Started kamet S-FLT (ocf::pw:IPaddr): Started kamet S-FLT2 (ocf::pw:IPaddr): Started kamet where i have a multi-state resource foo being run in master/slave mode and IPaddr RA is just modified IPAddr2 RA. Additionally i have a collocation constraint for the IP addr to be collocated with the master. I have additionally configured fencing and when i plug out the redundancy interface fencing gets triggered correctly. However once the fenced node(kamet) is rejoining i see all my floating IP resources are deleted and system looks to be in this state. Also if i log into kamet i see that the floating ip addresses are actually available. Master/Slave Set: foo-master [foo] Masters: [ orana ] Slaves: [ kamet ] fence-uc-orana (stonith:fence_ilo4): Started orana fence-uc-kamet (stonith:fence_ilo4): Started orana CIB state post fencing of kamet. <cib admin_epoch="0" cib-last-written="Wed Jan 13 19:28:22 2016" crm_feature_set="3.0.9" epoch="72" num_updates="7" validate-with="pacemaker-2.0" have-quorum="1" dc-uuid="orana"> <configuration> <crm_config> <cluster_property_set id="cib-bootstrap-options"> <nvpair id="cib-bootstrap-options-no-quorum-policy" name="no-quorum-policy" value="ignore"/> <nvpair id="cib-bootstrap-options-cluster-recheck-interval" name="cluster-recheck-interval" value="30s"/> <nvpair id="cib-bootstrap-options-stonith-enabled" name="stonith-enabled" value="true"/> <nvpair id="cib-bootstrap-options-dc-version" name="dc-version" value="1.1.11-97629de"/> <nvpair id="cib-bootstrap-options-cluster-infrastructure" name="cluster-infrastructure" value="cman"/> <nvpair id="cib-bootstrap-options-last-lrm-refresh" name="last-lrm-refresh" value="1452665061"/> </cluster_property_set> </crm_config> <nodes> <node id="kamet" uname="kamet"/> <node id="orana" uname="orana"/> </nodes> <resources> <master id="foo-master"> <primitive class="ocf" id="foo" provider="pw" type="uc"> <instance_attributes id="foo-instance_attributes"> <nvpair id="foo-instance_attributes-state" name="state" value="/var/run/uc/role"/> </instance_attributes> <operations> <op id="foo-start-interval-0s" interval="0s" name="start" on-fail="restart" timeout="100s"/> <op id="foo-monitor-interval-10s-role-Master" interval="10s" name="monitor" on-fail="restart" role="Master" timeout="100s"/> <op id="foo-monitor-interval-11s-role-Slave" interval="11s" name="monitor" on-fail="restart" role="Slave" timeout="100s"/> <op id="foo-promote-interval-0s" interval="0s" name="promote" on-fail="restart" timeout="100s"/> <op id="foo-demote-interval-0s" interval="0s" name="demote" on-fail="restart" timeout="100s"/> <op id="foo-stop-interval-0s" interval="0s" name="stop" on-fail="restart" timeout="100s"/> </operations> </primitive> <meta_attributes id="foo-master-meta_attributes"> <nvpair id="foo-master-meta_attributes-master-max" name="master-max" value="1"/> <nvpair id="foo-master-meta_attributes-master-node-max" name="master-node-max" value="1"/> <nvpair id="foo-master-meta_attributes-clone-max" name="clone-max" value="2"/> <nvpair id="foo-master-meta_attributes-clone-node-max" name="clone-node-max" value="1"/> <nvpair id="foo-master-meta_attributes-notify" name="notify" value="true"/> <nvpair id="foo-master-meta_attributes-ordered" name="ordered" value="true"/> </meta_attributes> </master> <primitive class="stonith" id="fence-uc-orana" type="fence_ilo4"> <instance_attributes id="fence-uc-orana-instance_attributes"> <nvpair id="fence-uc-orana-instance_attributes-login" name="login" value="parallel"/> <nvpair id="fence-uc-orana-instance_attributes-passwd" name="passwd" value="wireless"/> <nvpair id="fence-uc-orana-instance_attributes-ipaddr" name="ipaddr" value="10.11.10.30"/> <nvpair id="fence-uc-orana-instance_attributes-pcmk_host_list" name="pcmk_host_list" value="orana"/> <nvpair id="fence-uc-orana-instance_attributes-action" name="action" value="off"/> <nvpair id="fence-uc-orana-instance_attributes-lanplus" name="lanplus" value="1"/> <nvpair id="fence-uc-orana-instance_attributes-delay" name="delay" value="10"/> </instance_attributes> <operations> <op id="fence-uc-orana-start-interval-0s" interval="0s" name="start" on-fail="restart"/> <op id="fence-uc-orana-monitor-interval-5s" interval="5s" name="monitor" on-fail="restart"/> </operations> <meta_attributes id="fence-uc-orana-meta_attributes"> <nvpair id="fence-uc-orana-meta_attributes-failure-timeout" name="failure-timeout" value="3s"/> </meta_attributes> </primitive> <primitive class="stonith" id="fence-uc-kamet" type="fence_ilo4"> <instance_attributes id="fence-uc-kamet-instance_attributes"> <nvpair id="fence-uc-kamet-instance_attributes-login" name="login" value="parallel"/> <nvpair id="fence-uc-kamet-instance_attributes-passwd" name="passwd" value="wireless"/> <nvpair id="fence-uc-kamet-instance_attributes-ipaddr" name="ipaddr" value="10.11.10.21"/> <nvpair id="fence-uc-kamet-instance_attributes-pcmk_host_list" name="pcmk_host_list" value="kamet"/> <nvpair id="fence-uc-kamet-instance_attributes-action" name="action" value="off"/> <nvpair id="fence-uc-kamet-instance_attributes-lanplus" name="lanplus" value="1"/> <nvpair id="fence-uc-kamet-instance_attributes-delay" name="delay" value="0"/> </instance_attributes> <operations> <op id="fence-uc-kamet-start-interval-0s" interval="0s" name="start" on-fail="restart"/> <op id="fence-uc-kamet-monitor-interval-5s" interval="5s" name="monitor" on-fail="restart"/> </operations> <meta_attributes id="fence-uc-kamet-meta_attributes"/> </primitive> <primitive class="ocf" id="C-3" provider="pw" type="IPaddr"> <instance_attributes id="C-3-instance_attributes"> <nvpair id="C-3-instance_attributes-ip" name="ip" value="10.41.1.22"/> <nvpair id="C-3-instance_attributes-nic" name="nic" value="v40"/> <nvpair id="C-3-instance_attributes-cidr_netmask" name="cidr_netmask" value="32"/> <nvpair id="C-3-instance_attributes-iflabel" name="iflabel" value="C-3"/> </instance_attributes> <operations> <op id="C-3-start-timeout-20s" interval="0s" name="start" timeout="20s"/> <op id="C-3-stop-timeout-20s" interval="0s" name="stop" timeout="20s"/> <op id="C-3-monitor-interval-200ms" interval="200ms" name="monitor"/> </operations> <meta_attributes id="C-3-meta_attributes"> <nvpair id="C-3-meta_attributes-failure-timeout" name="failure-timeout" value="3s"/> <nvpair id="C-3-meta_attributes-migration-threshold" name="migration-threshold" value="2"/> </meta_attributes> </primitive> <primitive class="ocf" id="C-FLT" provider="pw" type="IPaddr"> <instance_attributes id="C-FLT-instance_attributes"> <nvpair id="C-FLT-instance_attributes-ip" name="ip" value="10.41.0.108"/> <nvpair id="C-FLT-instance_attributes-nic" name="nic" value="v40"/> <nvpair id="C-FLT-instance_attributes-cidr_netmask" name="cidr_netmask" value="32"/> <nvpair id="C-FLT-instance_attributes-iflabel" name="iflabel" value="C-FLT"/> </instance_attributes> <operations> <op id="C-FLT-start-timeout-20s" interval="0s" name="start" timeout="20s"/> <op id="C-FLT-stop-timeout-20s" interval="0s" name="stop" timeout="20s"/> <op id="C-FLT-monitor-interval-200ms" interval="200ms" name="monitor"/> </operations> <meta_attributes id="C-FLT-meta_attributes"> <nvpair id="C-FLT-meta_attributes-failure-timeout" name="failure-timeout" value="3s"/> <nvpair id="C-FLT-meta_attributes-migration-threshold" name="migration-threshold" value="2"/> </meta_attributes> </primitive> <primitive class="ocf" id="C-FLT2" provider="pw" type="IPaddr"> <instance_attributes id="C-FLT2-instance_attributes"> <nvpair id="C-FLT2-instance_attributes-ip" name="ip" value="10.41.1.15"/> <nvpair id="C-FLT2-instance_attributes-nic" name="nic" value="v40"/> <nvpair id="C-FLT2-instance_attributes-cidr_netmask" name="cidr_netmask" value="32"/> <nvpair id="C-FLT2-instance_attributes-iflabel" name="iflabel" value="C-FLT2"/> </instance_attributes> <operations> <op id="C-FLT2-start-timeout-20s" interval="0s" name="start" timeout="20s"/> <op id="C-FLT2-stop-timeout-20s" interval="0s" name="stop" timeout="20s"/> <op id="C-FLT2-monitor-interval-200ms" interval="200ms" name="monitor"/> </operations> <meta_attributes id="C-FLT2-meta_attributes"> <nvpair id="C-FLT2-meta_attributes-failure-timeout" name="failure-timeout" value="3s"/> <nvpair id="C-FLT2-meta_attributes-migration-threshold" name="migration-threshold" value="2"/> </meta_attributes> </primitive> <primitive class="ocf" id="E-3" provider="pw" type="IPaddr"> <instance_attributes id="E-3-instance_attributes"> <nvpair id="E-3-instance_attributes-ip" name="ip" value="10.21.1.22"/> <nvpair id="E-3-instance_attributes-nic" name="nic" value="v20"/> <nvpair id="E-3-instance_attributes-cidr_netmask" name="cidr_netmask" value="32"/> <nvpair id="E-3-instance_attributes-iflabel" name="iflabel" value="E-3"/> </instance_attributes> <operations> <op id="E-3-start-timeout-20s" interval="0s" name="start" timeout="20s"/> <op id="E-3-stop-timeout-20s" interval="0s" name="stop" timeout="20s"/> <op id="E-3-monitor-interval-200ms" interval="200ms" name="monitor"/> </operations> <meta_attributes id="E-3-meta_attributes"> <nvpair id="E-3-meta_attributes-failure-timeout" name="failure-timeout" value="3s"/> <nvpair id="E-3-meta_attributes-migration-threshold" name="migration-threshold" value="2"/> </meta_attributes> </primitive> <primitive class="ocf" id="MGMT-FLT" provider="pw" type="IPaddr"> <instance_attributes id="MGMT-FLT-instance_attributes"> <nvpair id="MGMT-FLT-instance_attributes-ip" name="ip" value="10.61.0.227"/> <nvpair id="MGMT-FLT-instance_attributes-nic" name="nic" value="eth0"/> <nvpair id="MGMT-FLT-instance_attributes-cidr_netmask" name="cidr_netmask" value="32"/> <nvpair id="MGMT-FLT-instance_attributes-iflabel" name="iflabel" value="MGMT-FLT"/> </instance_attributes> <operations> <op id="MGMT-FLT-start-timeout-20s" interval="0s" name="start" timeout="20s"/> <op id="MGMT-FLT-stop-timeout-20s" interval="0s" name="stop" timeout="20s"/> <op id="MGMT-FLT-monitor-interval-200ms" interval="200ms" name="monitor"/> </operations> <meta_attributes id="MGMT-FLT-meta_attributes"> <nvpair id="MGMT-FLT-meta_attributes-failure-timeout" name="failure-timeout" value="3s"/> <nvpair id="MGMT-FLT-meta_attributes-migration-threshold" name="migration-threshold" value="2"/> </meta_attributes> </primitive> <primitive class="ocf" id="M-FLT" provider="pw" type="IPaddr"> <instance_attributes id="M-FLT-instance_attributes"> <nvpair id="M-FLT-instance_attributes-ip" name="ip" value="10.21.0.108"/> <nvpair id="M-FLT-instance_attributes-nic" name="nic" value="v20"/> <nvpair id="M-FLT-instance_attributes-cidr_netmask" name="cidr_netmask" value="32"/> <nvpair id="M-FLT-instance_attributes-iflabel" name="iflabel" value="M-FLT"/> </instance_attributes> <operations> <op id="M-FLT-start-timeout-20s" interval="0s" name="start" timeout="20s"/> <op id="M-FLT-stop-timeout-20s" interval="0s" name="stop" timeout="20s"/> <op id="M-FLT-monitor-interval-200ms" interval="200ms" name="monitor"/> </operations> <meta_attributes id="M-FLT-meta_attributes"> <nvpair id="M-FLT-meta_attributes-failure-timeout" name="failure-timeout" value="3s"/> <nvpair id="M-FLT-meta_attributes-migration-threshold" name="migration-threshold" value="2"/> </meta_attributes> </primitive> <primitive class="ocf" id="M-FLT2" provider="pw" type="IPaddr"> <instance_attributes id="M-FLT2-instance_attributes"> <nvpair id="M-FLT2-instance_attributes-ip" name="ip" value="10.21.1.15"/> <nvpair id="M-FLT2-instance_attributes-nic" name="nic" value="v20"/> <nvpair id="M-FLT2-instance_attributes-cidr_netmask" name="cidr_netmask" value="32"/> <nvpair id="M-FLT2-instance_attributes-iflabel" name="iflabel" value="M-FLT2"/> </instance_attributes> <operations> <op id="M-FLT2-start-timeout-20s" interval="0s" name="start" timeout="20s"/> <op id="M-FLT2-stop-timeout-20s" interval="0s" name="stop" timeout="20s"/> <op id="M-FLT2-monitor-interval-200ms" interval="200ms" name="monitor"/> </operations> <meta_attributes id="M-FLT2-meta_attributes"> <nvpair id="M-FLT2-meta_attributes-failure-timeout" name="failure-timeout" value="3s"/> <nvpair id="M-FLT2-meta_attributes-migration-threshold" name="migration-threshold" value="2"/> </meta_attributes> </primitive> <primitive class="ocf" id="S-FLT" provider="pw" type="IPaddr"> <instance_attributes id="S-FLT-instance_attributes"> <nvpair id="S-FLT-instance_attributes-ip" name="ip" value="10.31.0.108"/> <nvpair id="S-FLT-instance_attributes-nic" name="nic" value="v30"/> <nvpair id="S-FLT-instance_attributes-cidr_netmask" name="cidr_netmask" value="32"/> <nvpair id="S-FLT-instance_attributes-iflabel" name="iflabel" value="S-FLT"/> </instance_attributes> <operations> <op id="S-FLT-start-timeout-20s" interval="0s" name="start" timeout="20s"/> <op id="S-FLT-stop-timeout-20s" interval="0s" name="stop" timeout="20s"/> <op id="S-FLT-monitor-interval-200ms" interval="200ms" name="monitor"/> </operations> <meta_attributes id="S-FLT-meta_attributes"> <nvpair id="S-FLT-meta_attributes-failure-timeout" name="failure-timeout" value="3s"/> <nvpair id="S-FLT-meta_attributes-migration-threshold" name="migration-threshold" value="2"/> </meta_attributes> </primitive> <primitive class="ocf" id="S-FLT2" provider="pw" type="IPaddr"> <instance_attributes id="S-FLT2-instance_attributes"> <nvpair id="S-FLT2-instance_attributes-ip" name="ip" value="10.31.1.15"/> <nvpair id="S-FLT2-instance_attributes-nic" name="nic" value="v30"/> <nvpair id="S-FLT2-instance_attributes-cidr_netmask" name="cidr_netmask" value="32"/> <nvpair id="S-FLT2-instance_attributes-iflabel" name="iflabel" value="S-FLT2"/> </instance_attributes> <operations> <op id="S-FLT2-start-timeout-20s" interval="0s" name="start" timeout="20s"/> <op id="S-FLT2-stop-timeout-20s" interval="0s" name="stop" timeout="20s"/> <op id="S-FLT2-monitor-interval-200ms" interval="200ms" name="monitor"/> </operations> <meta_attributes id="S-FLT2-meta_attributes"> <nvpair id="S-FLT2-meta_attributes-failure-timeout" name="failure-timeout" value="3s"/> <nvpair id="S-FLT2-meta_attributes-migration-threshold" name="migration-threshold" value="2"/> </meta_attributes> </primitive> </resources> <constraints> <rsc_colocation id="colocation-C-3-foo-master-INFINITY" rsc="C-3" rsc-role="Started" score="INFINITY" with-rsc="foo-master" with-rsc-role="Master"/> <rsc_order first="C-3" first-action="start" id="order-C-3-foo-master-mandatory" then="foo-master" then-action="promote"/> <rsc_colocation id="colocation-C-FLT-foo-master-INFINITY" rsc="C-FLT" rsc-role="Started" score="INFINITY" with-rsc="foo-master" with-rsc-role="Master"/> <rsc_order first="C-FLT" first-action="start" id="order-C-FLT-foo-master-mandatory" then="foo-master" then-action="promote"/> <rsc_colocation id="colocation-C-FLT2-foo-master-INFINITY" rsc="C-FLT2" rsc-role="Started" score="INFINITY" with-rsc="foo-master" with-rsc-role="Master"/> <rsc_order first="C-FLT2" first-action="start" id="order-C-FLT2-foo-master-mandatory" then="foo-master" then-action="promote"/> <rsc_colocation id="colocation-E-3-foo-master-INFINITY" rsc="E-3" rsc-role="Started" score="INFINITY" with-rsc="foo-master" with-rsc-role="Master"/> <rsc_order first="E-3" first-action="start" id="order-E-3-foo-master-mandatory" then="foo-master" then-action="promote"/> <rsc_colocation id="colocation-MGMT-FLT-foo-master-INFINITY" rsc="MGMT-FLT" rsc-role="Started" score="INFINITY" with-rsc="foo-master" with-rsc-role="Master"/> <rsc_order first="MGMT-FLT" first-action="start" id="order-MGMT-FLT-foo-master-mandatory" then="foo-master" then-action="promote"/> <rsc_colocation id="colocation-M-FLT-foo-master-INFINITY" rsc="M-FLT" rsc-role="Started" score="INFINITY" with-rsc="foo-master" with-rsc-role="Master"/> <rsc_order first="M-FLT" first-action="start" id="order-M-FLT-foo-master-mandatory" then="foo-master" then-action="promote"/> <rsc_colocation id="colocation-M-FLT2-foo-master-INFINITY" rsc="M-FLT2" rsc-role="Started" score="INFINITY" with-rsc="foo-master" with-rsc-role="Master"/> <rsc_order first="M-FLT2" first-action="start" id="order-M-FLT2-foo-master-mandatory" then="foo-master" then-action="promote"/> <rsc_colocation id="colocation-S-FLT-foo-master-INFINITY" rsc="S-FLT" rsc-role="Started" score="INFINITY" with-rsc="foo-master" with-rsc-role="Master"/> <rsc_order first="S-FLT" first-action="start" id="order-S-FLT-foo-master-mandatory" then="foo-master" then-action="promote"/> <rsc_colocation id="colocation-S-FLT2-foo-master-INFINITY" rsc="S-FLT2" rsc-role="Started" score="INFINITY" with-rsc="foo-master" with-rsc-role="Master"/> <rsc_order first="S-FLT2" first-action="start" id="order-S-FLT2-foo-master-mandatory" then="foo-master" then-action="promote"/> <rsc_colocation id="colocation-fence-uc-orana-foo-master-INFINITY" rsc="fence-uc-orana" rsc-role="Started" score="INFINITY" with-rsc="foo-master" with-rsc-role="Master"/> <rsc_colocation id="colocation-fence-uc-kamet-foo-master-INFINITY" rsc="fence-uc-kamet" rsc-role="Started" score="INFINITY" with-rsc="foo-master" with-rsc-role="Master"/> <rsc_order first="fence-uc-kamet" first-action="start" id="order-fence-uc-kamet-foo-master-mandatory" then="foo-master" then-action="promote"/> <rsc_order first="fence-uc-orana" first-action="start" id="order-fence-uc-orana-foo-master-mandatory" then="foo-master" then-action="promote"/> </constraints> </configuration> <status> <node_state id="kamet" uname="kamet" in_ccm="false" crmd="offline" crm-debug-origin="send_stonith_update" join="down" expected="down"/> <node_state id="orana" uname="orana" in_ccm="true" crmd="online" crm-debug-origin="do_update_resource" join="member" expected="member"> <lrm id="orana"> <lrm_resources> <lrm_resource id="fence-uc-kamet" type="fence_ilo4" class="stonith"> <lrm_rsc_op id="fence-uc-kamet_last_0" operation_key="fence-uc-kamet_start_0" operation="start" crm-debug-origin="do_update_resource" crm_feature_set="3.0.9" transition-key="13:2304:0:70de24fe-0f17-47ec-a193-a2cdd9f9e5c7" transition-magic="0:0;13:2304:0:70de24fe-0f17-47ec-a193-a2cdd9f9e5c7" call-id="131" rc-code="0" op-status="0" interval="0" last-run="1452731302" last-rc-change="1452731302" exec-time="80" queue-time="0" op-digest="93013a04ac64e00066a4f7b080b31515" on_node="orana"/> <lrm_rsc_op id="fence-uc-kamet_monitor_5000" operation_key="fence-uc-kamet_monitor_5000" operation="monitor" crm-debug-origin="do_update_resource" crm_feature_set="3.0.9" transition-key="1:2304:0:70de24fe-0f17-47ec-a193-a2cdd9f9e5c7" transition-magic="0:0;1:2304:0:70de24fe-0f17-47ec-a193-a2cdd9f9e5c7" call-id="132" rc-code="0" op-status="0" interval="5000" last-rc-change="1452731303" exec-time="128" queue-time="0" op-digest="e884a8e46b98e8563278b8117eb4db82" on_node="orana"/> </lrm_resource> <lrm_resource id="foo" type="uc" class="ocf" provider="pw"> <lrm_rsc_op id="foo_last_0" operation_key="foo_promote_0" operation="promote" crm-debug-origin="do_update_resource" crm_feature_set="3.0.9" transition-key="19:2304:0:70de24fe-0f17-47ec-a193-a2cdd9f9e5c7" transition-magic="0:0;19:2304:0:70de24fe-0f17-47ec-a193-a2cdd9f9e5c7" call-id="133" rc-code="0" op-status="0" interval="0" last-run="1452731303" last-rc-change="1452731303" exec-time="51" queue-time="0" op-digest="96d68f528fd950fa93acae8f44e75df5" on_node="orana"/> <lrm_rsc_op id="foo_monitor_10000" operation_key="foo_monitor_10000" operation="monitor" crm-debug-origin="do_update_resource" crm_feature_set="3.0.9" transition-key="20:2304:8:70de24fe-0f17-47ec-a193-a2cdd9f9e5c7" transition-magic="0:8;20:2304:8:70de24fe-0f17-47ec-a193-a2cdd9f9e5c7" call-id="135" rc-code="8" op-status="0" interval="10000" last-rc-change="1452731303" exec-time="39" queue-time="1" op-digest="f4e31338d1a8837389f1948c9c05d8a8" on_node="orana"/> </lrm_resource> <lrm_resource id="C-3" type="IPaddr" class="ocf" provider="pw"> <lrm_rsc_op id="C-3_last_0" operation_key="C-3_start_0" operation="start" crm-debug-origin="do_update_resource" crm_feature_set="3.0.9" transition-key="36:2301:0:70de24fe-0f17-47ec-a193-a2cdd9f9e5c7" transition-magic="0:0;36:2301:0:70de24fe-0f17-47ec-a193-a2cdd9f9e5c7" call-id="96" rc-code="0" op-status="0" interval="0" last-run="1452731294" last-rc-change="1452731294" exec-time="123" queue-time="0" op-digest="8dd2987e2ec7dfb4c2955839593b8e9f" on_node="orana"/> <lrm_rsc_op id="C-3_monitor_200" operation_key="C-3_monitor_200" operation="monitor" crm-debug-origin="do_update_resource" crm_feature_set="3.0.9" transition-key="37:2301:0:70de24fe-0f17-47ec-a193-a2cdd9f9e5c7" transition-magic="0:0;37:2301:0:70de24fe-0f17-47ec-a193-a2cdd9f9e5c7" call-id="113" rc-code="0" op-status="0" interval="200" last-rc-change="1452731295" exec-time="93" queue-time="0" op-digest="8ede7902b002a044cd67ffc05a799bdc" on_node="orana"/> </lrm_resource> <lrm_resource id="C-FLT" type="IPaddr" class="ocf" provider="pw"> <lrm_rsc_op id="C-FLT_last_0" operation_key="C-FLT_start_0" operation="start" crm-debug-origin="do_update_resource" crm_feature_set="3.0.9" transition-key="38:2301:0:70de24fe-0f17-47ec-a193-a2cdd9f9e5c7" transition-magic="0:0;38:2301:0:70de24fe-0f17-47ec-a193-a2cdd9f9e5c7" call-id="97" rc-code="0" op-status="0" interval="0" last-run="1452731294" last-rc-change="1452731294" exec-time="111" queue-time="0" op-digest="789f55f594a2d2a911e2c1b9a41e103d" on_node="orana"/> <lrm_rsc_op id="C-FLT_monitor_200" operation_key="C-FLT_monitor_200" operation="monitor" crm-debug-origin="do_update_resource" crm_feature_set="3.0.9" transition-key="39:2301:0:70de24fe-0f17-47ec-a193-a2cdd9f9e5c7" transition-magic="0:0;39:2301:0:70de24fe-0f17-47ec-a193-a2cdd9f9e5c7" call-id="108" rc-code="0" op-status="0" interval="200" last-rc-change="1452731295" exec-time="75" queue-time="0" op-digest="b9db07a00f66e57a44226b503d43889f" on_node="orana"/> </lrm_resource> <lrm_resource id="C-FLT2" type="IPaddr" class="ocf" provider="pw"> <lrm_rsc_op id="C-FLT2_last_0" operation_key="C-FLT2_start_0" operation="start" crm-debug-origin="do_update_resource" crm_feature_set="3.0.9" transition-key="40:2301:0:70de24fe-0f17-47ec-a193-a2cdd9f9e5c7" transition-magic="0:0;40:2301:0:70de24fe-0f17-47ec-a193-a2cdd9f9e5c7" call-id="98" rc-code="0" op-status="0" interval="0" last-run="1452731294" last-rc-change="1452731294" exec-time="113" queue-time="0" op-digest="236ab7fb7ace80f0eebd8edf4bd9224b" on_node="orana"/> <lrm_rsc_op id="C-FLT2_monitor_200" operation_key="C-FLT2_monitor_200" operation="monitor" crm-debug-origin="do_update_resource" crm_feature_set="3.0.9" transition-key="41:2301:0:70de24fe-0f17-47ec-a193-a2cdd9f9e5c7" transition-magic="0:0;41:2301:0:70de24fe-0f17-47ec-a193-a2cdd9f9e5c7" call-id="109" rc-code="0" op-status="0" interval="200" last-rc-change="1452731295" exec-time="89" queue-time="0" op-digest="b6c51881a4e04213406b712d950fe34c" on_node="orana"/> </lrm_resource> <lrm_resource id="E-3" type="IPaddr" class="ocf" provider="pw"> <lrm_rsc_op id="E-3_last_0" operation_key="E-3_start_0" operation="start" crm-debug-origin="do_update_resource" crm_feature_set="3.0.9" transition-key="42:2301:0:70de24fe-0f17-47ec-a193-a2cdd9f9e5c7" transition-magic="0:0;42:2301:0:70de24fe-0f17-47ec-a193-a2cdd9f9e5c7" call-id="99" rc-code="0" op-status="0" interval="0" last-run="1452731294" last-rc-change="1452731294" exec-time="126" queue-time="0" op-digest="f3631c868f7539a7fb24804bfccd48f8" on_node="orana"/> <lrm_rsc_op id="E-3_monitor_200" operation_key="E-3_monitor_200" operation="monitor" crm-debug-origin="do_update_resource" crm_feature_set="3.0.9" transition-key="43:2301:0:70de24fe-0f17-47ec-a193-a2cdd9f9e5c7" transition-magic="0:0;43:2301:0:70de24fe-0f17-47ec-a193-a2cdd9f9e5c7" call-id="114" rc-code="0" op-status="0" interval="200" last-rc-change="1452731295" exec-time="75" queue-time="0" op-digest="a99e7c4d084a719699ad5bd04237e8d2" on_node="orana"/> </lrm_resource> <lrm_resource id="MGMT-FLT" type="IPaddr" class="ocf" provider="pw"> <lrm_rsc_op id="MGMT-FLT_last_0" operation_key="MGMT-FLT_start_0" operation="start" crm-debug-origin="do_update_resource" crm_feature_set="3.0.9" transition-key="44:2301:0:70de24fe-0f17-47ec-a193-a2cdd9f9e5c7" transition-magic="0:0;44:2301:0:70de24fe-0f17-47ec-a193-a2cdd9f9e5c7" call-id="100" rc-code="0" op-status="0" interval="0" last-run="1452731294" last-rc-change="1452731294" exec-time="114" queue-time="0" op-digest="a78c07f01d90a62d8f139658bc7ac3c9" on_node="orana"/> <lrm_rsc_op id="MGMT-FLT_monitor_200" operation_key="MGMT-FLT_monitor_200" operation="monitor" crm-debug-origin="do_update_resource" crm_feature_set="3.0.9" transition-key="45:2301:0:70de24fe-0f17-47ec-a193-a2cdd9f9e5c7" transition-magic="0:0;45:2301:0:70de24fe-0f17-47ec-a193-a2cdd9f9e5c7" call-id="110" rc-code="0" op-status="0" interval="200" last-rc-change="1452731295" exec-time="94" queue-time="0" op-digest="72c18513de4f6b9105d664dd90d265e8" on_node="orana"/> </lrm_resource> <lrm_resource id="M-FLT" type="IPaddr" class="ocf" provider="pw"> <lrm_rsc_op id="M-FLT_last_0" operation_key="M-FLT_start_0" operation="start" crm-debug-origin="do_update_resource" crm_feature_set="3.0.9" transition-key="46:2301:0:70de24fe-0f17-47ec-a193-a2cdd9f9e5c7" transition-magic="0:0;46:2301:0:70de24fe-0f17-47ec-a193-a2cdd9f9e5c7" call-id="101" rc-code="0" op-status="0" interval="0" last-run="1452731294" last-rc-change="1452731294" exec-time="122" queue-time="0" op-digest="77947e7e39ef2e8073312201deb6d7e7" on_node="orana"/> <lrm_rsc_op id="M-FLT_monitor_200" operation_key="M-FLT_monitor_200" operation="monitor" crm-debug-origin="do_update_resource" crm_feature_set="3.0.9" transition-key="47:2301:0:70de24fe-0f17-47ec-a193-a2cdd9f9e5c7" transition-magic="0:0;47:2301:0:70de24fe-0f17-47ec-a193-a2cdd9f9e5c7" call-id="115" rc-code="0" op-status="0" interval="200" last-rc-change="1452731295" exec-time="80" queue-time="0" op-digest="a61b31bfa2661c96d41eed752b748f8a" on_node="orana"/> </lrm_resource> <lrm_resource id="M-FLT2" type="IPaddr" class="ocf" provider="pw"> <lrm_rsc_op id="M-FLT2_last_0" operation_key="M-FLT2_start_0" operation="start" crm-debug-origin="do_update_resource" crm_feature_set="3.0.9" transition-key="48:2301:0:70de24fe-0f17-47ec-a193-a2cdd9f9e5c7" transition-magic="0:0;48:2301:0:70de24fe-0f17-47ec-a193-a2cdd9f9e5c7" call-id="102" rc-code="0" op-status="0" interval="0" last-run="1452731294" last-rc-change="1452731294" exec-time="112" queue-time="0" op-digest="72fcbee579c9168ec7a8b329c0b1e8cc" on_node="orana"/> <lrm_rsc_op id="M-FLT2_monitor_200" operation_key="M-FLT2_monitor_200" operation="monitor" crm-debug-origin="do_update_resource" crm_feature_set="3.0.9" transition-key="49:2301:0:70de24fe-0f17-47ec-a193-a2cdd9f9e5c7" transition-magic="0:0;49:2301:0:70de24fe-0f17-47ec-a193-a2cdd9f9e5c7" call-id="111" rc-code="0" op-status="0" interval="200" last-rc-change="1452731295" exec-time="89" queue-time="0" op-digest="8ee1fce96d23a284b277c3b8fdee8b95" on_node="orana"/> </lrm_resource> <lrm_resource id="S-FLT" type="IPaddr" class="ocf" provider="pw"> <lrm_rsc_op id="S-FLT_last_0" operation_key="S-FLT_start_0" operation="start" crm-debug-origin="do_update_resource" crm_feature_set="3.0.9" transition-key="50:2301:0:70de24fe-0f17-47ec-a193-a2cdd9f9e5c7" transition-magic="0:0;50:2301:0:70de24fe-0f17-47ec-a193-a2cdd9f9e5c7" call-id="103" rc-code="0" op-status="0" interval="0" last-run="1452731294" last-rc-change="1452731294" exec-time="123" queue-time="0" op-digest="6d479331421af055b010c182b992957c" on_node="orana"/> <lrm_rsc_op id="S-FLT_monitor_200" operation_key="S-FLT_monitor_200" operation="monitor" crm-debug-origin="do_update_resource" crm_feature_set="3.0.9" transition-key="51:2301:0:70de24fe-0f17-47ec-a193-a2cdd9f9e5c7" transition-magic="0:0;51:2301:0:70de24fe-0f17-47ec-a193-a2cdd9f9e5c7" call-id="116" rc-code="0" op-status="0" interval="200" last-rc-change="1452731295" exec-time="91" queue-time="0" op-digest="a88069f185532a1f62424b120846ff71" on_node="orana"/> </lrm_resource> <lrm_resource id="S-FLT2" type="IPaddr" class="ocf" provider="pw"> <lrm_rsc_op id="S-FLT2_last_0" operation_key="S-FLT2_start_0" operation="start" crm-debug-origin="do_update_resource" crm_feature_set="3.0.9" transition-key="52:2301:0:70de24fe-0f17-47ec-a193-a2cdd9f9e5c7" transition-magic="0:0;52:2301:0:70de24fe-0f17-47ec-a193-a2cdd9f9e5c7" call-id="104" rc-code="0" op-status="0" interval="0" last-run="1452731294" last-rc-change="1452731294" exec-time="112" queue-time="0" op-digest="d0aa64d1a80785f83fca3ed7a7b58527" on_node="orana"/> <lrm_rsc_op id="S-FLT2_monitor_200" operation_key="S-FLT2_monitor_200" operation="monitor" crm-debug-origin="do_update_resource" crm_feature_set="3.0.9" transition-key="53:2301:0:70de24fe-0f17-47ec-a193-a2cdd9f9e5c7" transition-magic="0:0;53:2301:0:70de24fe-0f17-47ec-a193-a2cdd9f9e5c7" call-id="112" rc-code="0" op-status="0" interval="200" last-rc-change="1452731295" exec-time="92" queue-time="0" op-digest="5843ced8789230e395f5cb1f81e27e92" on_node="orana"/> </lrm_resource> <lrm_resource id="fence-uc-orana" type="fence_ilo4" class="stonith"> <lrm_rsc_op id="fence-uc-orana_last_0" operation_key="fence-uc-orana_start_0" operation="start" crm-debug-origin="do_update_resource" crm_feature_set="3.0.9" transition-key="14:2303:0:70de24fe-0f17-47ec-a193-a2cdd9f9e5c7" transition-magic="0:0;14:2303:0:70de24fe-0f17-47ec-a193-a2cdd9f9e5c7" call-id="123" rc-code="0" op-status="0" interval="0" last-run="1452731302" last-rc-change="1452731302" exec-time="108" queue-time="0" op-digest="363cbd5d9ec840db46e5e25e2f8b6652" on_node="orana"/> <lrm_rsc_op id="fence-uc-orana_monitor_5000" operation_key="fence-uc-orana_monitor_5000" operation="monitor" crm-debug-origin="do_update_resource" crm_feature_set="3.0.9" transition-key="12:2303:0:70de24fe-0f17-47ec-a193-a2cdd9f9e5c7" transition-magic="0:0;12:2303:0:70de24fe-0f17-47ec-a193-a2cdd9f9e5c7" call-id="124" rc-code="0" op-status="0" interval="5000" last-rc-change="1452731302" exec-time="94" queue-time="0" op-digest="8024392825e4a1e81b7f628e6dee69a6" on_node="orana"/> </lrm_resource> </lrm_resources> </lrm> <transient_attributes id="orana"> <instance_attributes id="status-orana"> <nvpair id="status-orana-shutdown" name="shutdown" value="0"/> <nvpair id="status-orana-probe_complete" name="probe_complete" value="true"/> <nvpair id="status-orana-master-foo" name="master-foo" value="10"/> </instance_attributes> </transient_attributes> </node_state> </status> </cib> Attaching full corosync.log from orana. Mentioning the interesting parts in the log here. Jan 13 19:32:44 corosync [TOTEM ] A processor joined or left the membership and a new membership was formed. Jan 13 19:32:44 corosync [QUORUM] Members[2]: 1 2 Jan 13 19:32:44 corosync [QUORUM] Members[2]: 1 2 Jan 13 19:32:44 [4296] orana crmd: info: cman_event_callback: Membership 7044: quorum retained Jan 13 19:32:44 [4296] orana crmd: notice: crm_update_peer_state: cman_event_callback: Node kamet[2] - state is now member (was lost) Jan 13 19:32:44 [4296] orana crmd: info: peer_update_callback: kamet is now member (was lost) Jan 13 19:32:44 [4291] orana cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/crmd/2482) Jan 13 19:32:44 [4296] orana crmd: info: crm_cs_flush: Sent 0 CPG messages (1 remaining, last=122): Try again (6) Jan 13 19:32:44 [4291] orana cib: info: crm_cs_flush: Sent 0 CPG messages (1 remaining, last=240): Try again (6) Jan 13 19:32:44 [4296] orana crmd: info: cman_event_callback: Membership 7044: quorum retained Jan 13 19:32:44 [4291] orana cib: info: cib_process_request: Forwarding cib_modify operation for section nodes to master (origin=local/crmd/2483) Jan 13 19:32:44 [4291] orana cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/crmd/2484) Jan 13 19:32:44 [4291] orana cib: info: cib_process_request: Forwarding cib_modify operation for section nodes to master (origin=local/crmd/2485) Jan 13 19:32:44 [4291] orana cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/crmd/2486) Jan 13 19:32:44 corosync [CPG ] chosen downlist: sender r(0) ip(7.7.7.1) ; members(old:1 left:0) Jan 13 19:32:44 corosync [MAIN ] Completed service synchronization, ready to provide service. Jan 13 19:32:44 [4291] orana cib: info: crm_cs_flush: Sent 5 CPG messages (0 remaining, last=245): OK (1) Jan 13 19:32:44 [4296] orana crmd: info: crm_cs_flush: Sent 2 CPG messages (0 remaining, last=124): OK (1) Jan 13 19:32:44 [4291] orana cib: info: cib_perform_op: Diff: --- 0.72.7 2 Jan 13 19:32:44 [4291] orana cib: info: cib_perform_op: Diff: +++ 0.72.8 (null) Jan 13 19:32:44 [4291] orana cib: info: cib_perform_op: + /cib: @num_updates=8 Jan 13 19:32:44 [4291] orana cib: info: cib_perform_op: + /cib/status/node_state[@id='kamet']: @crm-debug-origin=peer_update_callback Jan 13 19:32:44 [4291] orana cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=orana/crmd/2482, version=0.72.8) Jan 13 19:32:44 [4291] orana cib: info: cib_process_request: Completed cib_modify operation for section nodes: OK (rc=0, origin=orana/crmd/2483, version=0.72.8) Jan 13 19:32:44 [4291] orana cib: info: cib_perform_op: Diff: --- 0.72.8 2 Jan 13 19:32:44 [4291] orana cib: info: cib_perform_op: Diff: +++ 0.72.9 (null) Jan 13 19:32:44 [4291] orana cib: info: cib_perform_op: + /cib: @num_updates=9 Jan 13 19:32:44 [4291] orana cib: info: cib_perform_op: + /cib/status/node_state[@id='kamet']: @in_ccm=true, @crm-debug-origin=post_cache_update Jan 13 19:32:44 [4291] orana cib: info: cib_perform_op: + /cib/status/node_state[@id='orana']: @crm-debug-origin=post_cache_update Jan 13 19:32:44 [4291] orana cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=orana/crmd/2484, version=0.72.9) Jan 13 19:32:44 [4291] orana cib: info: cib_process_request: Completed cib_modify operation for section nodes: OK (rc=0, origin=orana/crmd/2485, version=0.72.9) Jan 13 19:32:44 [4291] orana cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=orana/crmd/2486, version=0.72.9) Jan 13 19:32:49 [4291] orana cib: info: cib_process_ping: Reporting our current digest to orana: 9cc493f69fca04421704855b29514829 for 0.72.9 (0x147cad0 0) Jan 13 19:32:53 [4296] orana crmd: info: crm_timer_popped: PEngine Recheck Timer (I_PE_CALC) just popped (30000ms) Jan 13 19:32:53 [4296] orana crmd: notice: do_state_transition: State transition S_IDLE -> S_POLICY_ENGINE [ input=I_PE_CALC cause=C_TIMER_POPPED origin=crm_timer_popped ] Jan 13 19:32:53 [4296] orana crmd: info: do_state_transition: Progressed to state S_POLICY_ENGINE after C_TIMER_POPPED Jan 13 19:32:53 [4295] orana pengine: notice: unpack_config: On loss of CCM Quorum: Ignore Jan 13 19:32:53 [4295] orana pengine: info: determine_online_status_fencing: - Node kamet is not ready to run resources Jan 13 19:32:53 [4295] orana pengine: info: determine_online_status: Node kamet is pending Jan 13 19:32:53 [4295] orana pengine: info: determine_online_status_fencing: Node orana is active Jan 13 19:32:53 [4295] orana pengine: info: determine_online_status: Node orana is online Jan 13 19:32:53 [4295] orana pengine: info: clone_print: Master/Slave Set: foo-master [foo] Jan 13 19:32:53 [4295] orana pengine: info: short_print: Masters: [ orana ] Jan 13 19:32:53 [4295] orana pengine: info: short_print: Stopped: [ kamet ] Jan 13 19:32:53 [4295] orana pengine: info: native_print: fence-uc-orana (stonith:fence_ilo4): Started orana Jan 13 19:32:53 [4295] orana pengine: info: native_print: fence-uc-kamet (stonith:fence_ilo4): Started orana Jan 13 19:32:53 [4295] orana pengine: info: native_print: CWS-3G (ocf::pw:IPaddr): Started orana Jan 13 19:32:53 [4295] orana pengine: info: native_print: CWS-FLT (ocf::pw:IPaddr): Started orana Jan 13 19:32:53 [4295] orana pengine: info: native_print: CWS-FLT2 (ocf::pw:IPaddr): Started orana Jan 13 19:32:53 [4295] orana pengine: info: native_print: EPC-3G (ocf::pw:IPaddr): Started orana Jan 13 19:32:53 [4295] orana pengine: info: native_print: MGMT-FLT (ocf::pw:IPaddr): Started orana Jan 13 19:32:53 [4295] orana pengine: info: native_print: MME-FLT (ocf::pw:IPaddr): Started orana Jan 13 19:32:53 [4295] orana pengine: info: native_print: MME-FLT2 (ocf::pw:IPaddr): Started orana Jan 13 19:32:53 [4295] orana pengine: info: native_print: SGW-FLT (ocf::pw:IPaddr): Started orana Jan 13 19:32:53 [4295] orana pengine: info: native_print: SGW-FLT2 (ocf::pw:IPaddr): Started orana Jan 13 19:32:53 [4295] orana pengine: info: native_color: Resource foo:1 cannot run anywhere Jan 13 19:32:53 [4295] orana pengine: info: master_color: Promoting foo:0 (Master orana) Jan 13 19:32:53 [4295] orana pengine: info: master_color: foo-master: Promoted 1 instances of a possible 1 to master Jan 13 19:32:53 [4295] orana pengine: info: probe_resources: Action probe_complete-kamet on kamet is unrunnable (pending) Jan 13 19:32:53 [4295] orana pengine: warning: custom_action: Action foo:0_monitor_0 on kamet is unrunnable (pending) Jan 13 19:32:53 [4295] orana pengine: warning: custom_action: Action fence-uc-orana_monitor_0 on kamet is unrunnable (pending) Jan 13 19:32:53 [4295] orana pengine: warning: custom_action: Action fence-uc-kamet_monitor_0 on kamet is unrunnable (pending) Jan 13 19:32:53 [4295] orana pengine: warning: custom_action: Action CWS-3G_monitor_0 on kamet is unrunnable (pending) Jan 13 19:32:53 [4295] orana pengine: warning: custom_action: Action CWS-FLT_monitor_0 on kamet is unrunnable (pending) Jan 13 19:32:53 [4295] orana pengine: warning: custom_action: Action CWS-FLT2_monitor_0 on kamet is unrunnable (pending) Jan 13 19:32:53 [4295] orana pengine: warning: custom_action: Action EPC-3G_monitor_0 on kamet is unrunnable (pending) Jan 13 19:32:53 [4295] orana pengine: warning: custom_action: Action MGMT-FLT_monitor_0 on kamet is unrunnable (pending) Jan 13 19:32:53 [4295] orana pengine: warning: custom_action: Action MME-FLT_monitor_0 on kamet is unrunnable (pending) Jan 13 19:32:53 [4295] orana pengine: warning: custom_action: Action MME-FLT2_monitor_0 on kamet is unrunnable (pending) Jan 13 19:32:53 [4295] orana pengine: warning: custom_action: Action SGW-FLT_monitor_0 on kamet is unrunnable (pending) Jan 13 19:32:53 [4295] orana pengine: warning: custom_action: Action SGW-FLT2_monitor_0 on kamet is unrunnable (pending) Jan 13 19:32:53 [4295] orana pengine: info: LogActions: Leave foo:0 (Master orana) Jan 13 19:32:53 [4295] orana pengine: info: LogActions: Leave foo:1 (Stopped) Jan 13 19:32:53 [4295] orana pengine: info: LogActions: Leave fence-uc-orana (Started orana) Jan 13 19:32:53 [4295] orana pengine: info: LogActions: Leave fence-uc-kamet (Started orana) Jan 13 19:32:53 [4295] orana pengine: info: LogActions: Leave CWS-3G (Started orana) Jan 13 19:32:53 [4295] orana pengine: info: LogActions: Leave CWS-FLT (Started orana) Jan 13 19:32:53 [4295] orana pengine: info: LogActions: Leave CWS-FLT2 (Started orana) Jan 13 19:32:53 [4295] orana pengine: info: LogActions: Leave EPC-3G (Started orana) Jan 13 19:32:53 [4295] orana pengine: info: LogActions: Leave MGMT-FLT (Started orana) Jan 13 19:32:53 [4295] orana pengine: info: LogActions: Leave MME-FLT (Started orana) Jan 13 19:32:53 [4295] orana pengine: info: LogActions: Leave MME-FLT2 (Started orana) Jan 13 19:32:53 [4295] orana pengine: info: LogActions: Leave SGW-FLT (Started orana) Jan 13 19:32:53 [4295] orana pengine: info: LogActions: Leave SGW-FLT2 (Started orana) Jan 13 19:32:53 [4295] orana pengine: notice: process_pe_message: Calculated Transition 2313: /var/lib/pacemaker/pengine/pe-input-1448.bz2 Jan 13 19:32:53 [4296] orana crmd: info: do_state_transition: State transition S_POLICY_ENGINE -> S_TRANSITION_ENGINE [ input=I_PE_SUCCESS cause=C_IPC_MESSAGE origin=handle_response ] Jan 13 19:32:53 [4296] orana crmd: info: do_te_invoke: Processing graph 2313 (ref=pe_calc-dc-1452731573-2544) derived from /var/lib/pacemaker/pengine/pe-input-1448.bz2 Jan 13 19:32:53 [4296] orana crmd: notice: run_graph: Transition 2313 (Complete=0, Pending=0, Fired=0, Skipped=0, Incomplete=0, Source=/var/lib/pacemaker/pengine/pe-input-1448.bz2): Complete Jan 13 19:32:53 [4296] orana crmd: info: do_log: FSA: Input I_TE_SUCCESS from notify_crmd() received in state S_TRANSITION_ENGINE Jan 13 19:32:53 [4296] orana crmd: notice: do_state_transition: State transition S_TRANSITION_ENGINE -> S_IDLE [ input=I_TE_SUCCESS cause=C_FSA_INTERNAL origin=notify_crmd ] Jan 13 19:32:59 [4292] orana stonith-ng: info: pcmk_cpg_membership: Joined[2.0] stonith-ng.2 Jan 13 19:32:59 [4292] orana stonith-ng: info: pcmk_cpg_membership: Member[2.0] stonith-ng.1 Jan 13 19:32:59 [4292] orana stonith-ng: info: pcmk_cpg_membership: Member[2.1] stonith-ng.2 Jan 13 19:32:59 [4292] orana stonith-ng: info: crm_update_peer_proc: pcmk_cpg_membership: Node kamet[2] - corosync-cpg is now online Jan 13 19:32:59 [4291] orana cib: info: pcmk_cpg_membership: Joined[2.0] cib.2 Jan 13 19:32:59 [4291] orana cib: info: pcmk_cpg_membership: Member[2.0] cib.1 Jan 13 19:32:59 [4291] orana cib: info: pcmk_cpg_membership: Member[2.1] cib.2 Jan 13 19:32:59 [4291] orana cib: info: crm_update_peer_proc: pcmk_cpg_membership: Node kamet[2] - corosync-cpg is now online Jan 13 19:33:00 [4291] orana cib: info: cib_perform_op: Diff: --- 0.72.9 2 Jan 13 19:33:00 [4291] orana cib: info: cib_perform_op: Diff: +++ 0.73.0 (null) Jan 13 19:33:00 [4291] orana cib: info: cib_perform_op: -- /cib/configuration/crm_config/cluster_property_set[@id='cib-bootstrap-options']/nvpair[@id='cib-bootstrap-options-cluster-recheck-interval'] Jan 13 19:33:00 [4291] orana cib: info: cib_perform_op: -- /cib/configuration/crm_config/cluster_property_set[@id='cib-bootstrap-options']/nvpair[@id='cib-bootstrap-options-stonith-enabled'] Jan 13 19:33:00 [4291] orana cib: info: cib_perform_op: -- /cib/configuration/crm_config/cluster_property_set[@id='cib-bootstrap-options']/nvpair[@id='cib-bootstrap-options-dc-version'] Jan 13 19:33:00 [4291] orana cib: info: cib_perform_op: -- /cib/configuration/crm_config/cluster_property_set[@id='cib-bootstrap-options']/nvpair[@id='cib-bootstrap-options-cluster-infrastructure'] Jan 13 19:33:00 [4291] orana cib: info: cib_perform_op: -- /cib/configuration/crm_config/cluster_property_set[@id='cib-bootstrap-options']/nvpair[@id='cib-bootstrap-options-last-lrm-refresh'] Jan 13 19:33:00 [4291] orana cib: info: cib_perform_op: + /cib: @epoch=73, @num_updates=0 Jan 13 19:33:00 [4291] orana cib: info: cib_process_request: Completed cib_replace operation for section configuration: OK (rc=0, origin=kamet/cibadmin/2, version=0.73.0) Jan 13 19:33:00 [4296] orana crmd: info: abort_transition_graph: Transition aborted by deletion of nvpair[@id='cib-bootstrap-options-cluster-recheck-interval']: Non-status change (cib=0.73.0, source=te_update_diff:383, path=/cib/configuration/crm_config/cluster_property_set[@id='cib-bootstrap-options']/nvpair[@id='cib-bootstrap-options-cluster-recheck-interval'], 1) Jan 13 19:33:00 [4296] orana crmd: notice: do_state_transition: State transition S_IDLE -> S_POLICY_ENGINE [ input=I_PE_CALC cause=C_FSA_INTERNAL origin=abort_transition_graph ] Jan 13 19:33:00 [4291] orana cib: info: write_cib_contents: Archived previous version as /var/lib/pacemaker/cib/cib-83.raw Jan 13 19:33:00 [4291] orana cib: info: write_cib_contents: Wrote version 0.73.0 of the CIB to disk (digest: 3e63c3f9772e422777137db356f7e41d) Jan 13 19:33:00 [4291] orana cib: info: retrieveCib: Reading cluster configuration from: /var/lib/pacemaker/cib/cib.K8qdX0 (digest: /var/lib/pacemaker/cib/cib.3INvqH) Jan 13 19:33:00 [4295] orana pengine: notice: unpack_config: On loss of CCM Quorum: Ignore Jan 13 19:33:00 [4295] orana pengine: info: determine_online_status_fencing: - Node kamet is not ready to run resources Jan 13 19:33:00 [4295] orana pengine: info: determine_online_status: Node kamet is pending Jan 13 19:33:00 [4295] orana pengine: info: determine_online_status_fencing: Node orana is active Jan 13 19:33:00 [4295] orana pengine: info: determine_online_status: Node orana is online Jan 13 19:33:00 [4295] orana pengine: info: clone_print: Master/Slave Set: foo-master [foo] Jan 13 19:33:00 [4295] orana pengine: info: short_print: Masters: [ orana ] Jan 13 19:33:00 [4295] orana pengine: info: short_print: Stopped: [ kamet ] Jan 13 19:33:00 [4295] orana pengine: info: native_print: fence-uc-orana (stonith:fence_ilo4): Started orana Jan 13 19:33:00 [4295] orana pengine: info: native_print: fence-uc-kamet (stonith:fence_ilo4): Started orana Jan 13 19:33:00 [4295] orana pengine: info: native_print: CWS-3G (ocf::pw:IPaddr): Started orana Jan 13 19:33:00 [4295] orana pengine: info: native_print: CWS-FLT (ocf::pw:IPaddr): Started orana Jan 13 19:33:00 [4295] orana pengine: info: native_print: CWS-FLT2 (ocf::pw:IPaddr): Started orana Jan 13 19:33:00 [4295] orana pengine: info: native_print: EPC-3G (ocf::pw:IPaddr): Started orana Jan 13 19:33:00 [4295] orana pengine: info: native_print: MGMT-FLT (ocf::pw:IPaddr): Started orana Jan 13 19:33:00 [4295] orana pengine: info: native_print: MME-FLT (ocf::pw:IPaddr): Started orana Jan 13 19:33:00 [4295] orana pengine: info: native_print: MME-FLT2 (ocf::pw:IPaddr): Started orana Jan 13 19:33:00 [4295] orana pengine: info: native_print: SGW-FLT (ocf::pw:IPaddr): Started orana Jan 13 19:33:00 [4295] orana pengine: info: native_print: SGW-FLT2 (ocf::pw:IPaddr): Started orana Jan 13 19:33:00 [4295] orana pengine: info: native_color: Resource foo:1 cannot run anywhere Jan 13 19:33:00 [4295] orana pengine: info: master_color: Promoting foo:0 (Master orana) Jan 13 19:33:00 [4295] orana pengine: info: master_color: foo-master: Promoted 1 instances of a possible 1 to master Jan 13 19:33:00 [4295] orana pengine: info: probe_resources: Action probe_complete-kamet on kamet is unrunnable (pending) Jan 13 19:33:00 [4295] orana pengine: warning: custom_action: Action foo:0_monitor_0 on kamet is unrunnable (pending) Jan 13 19:33:00 [4295] orana pengine: warning: custom_action: Action fence-uc-orana_monitor_0 on kamet is unrunnable (pending) Jan 13 19:33:00 [4295] orana pengine: warning: custom_action: Action fence-uc-kamet_monitor_0 on kamet is unrunnable (pending) Jan 13 19:33:00 [4295] orana pengine: warning: custom_action: Action CWS-3G_monitor_0 on kamet is unrunnable (pending) Jan 13 19:33:00 [4295] orana pengine: warning: custom_action: Action CWS-FLT_monitor_0 on kamet is unrunnable (pending) Jan 13 19:33:00 [4295] orana pengine: warning: custom_action: Action CWS-FLT2_monitor_0 on kamet is unrunnable (pending) Jan 13 19:33:00 [4295] orana pengine: warning: custom_action: Action EPC-3G_monitor_0 on kamet is unrunnable (pending) Jan 13 19:33:00 [4295] orana pengine: warning: custom_action: Action MGMT-FLT_monitor_0 on kamet is unrunnable (pending) Jan 13 19:33:00 [4295] orana pengine: warning: custom_action: Action MME-FLT_monitor_0 on kamet is unrunnable (pending) Jan 13 19:33:00 [4295] orana pengine: warning: custom_action: Action MME-FLT2_monitor_0 on kamet is unrunnable (pending) Jan 13 19:33:00 [4295] orana pengine: warning: custom_action: Action SGW-FLT_monitor_0 on kamet is unrunnable (pending) Jan 13 19:33:00 [4295] orana pengine: warning: custom_action: Action SGW-FLT2_monitor_0 on kamet is unrunnable (pending) Jan 13 19:33:00 [4295] orana pengine: info: LogActions: Leave foo:0 (Master orana) Jan 13 19:33:00 [4295] orana pengine: info: LogActions: Leave foo:1 (Stopped) Jan 13 19:33:00 [4295] orana pengine: info: LogActions: Leave fence-uc-orana (Started orana) Jan 13 19:33:00 [4295] orana pengine: info: LogActions: Leave fence-uc-kamet (Started orana) Jan 13 19:33:00 [4295] orana pengine: info: LogActions: Leave CWS-3G (Started orana) Jan 13 19:33:00 [4295] orana pengine: info: LogActions: Leave CWS-FLT (Started orana) Jan 13 19:33:00 [4295] orana pengine: info: LogActions: Leave CWS-FLT2 (Started orana) Jan 13 19:33:00 [4295] orana pengine: info: LogActions: Leave EPC-3G (Started orana) Jan 13 19:33:00 [4295] orana pengine: info: LogActions: Leave MGMT-FLT (Started orana) Jan 13 19:33:00 [4295] orana pengine: info: LogActions: Leave MME-FLT (Started orana) Jan 13 19:33:00 [4295] orana pengine: info: LogActions: Leave MME-FLT2 (Started orana) Jan 13 19:33:00 [4295] orana pengine: info: LogActions: Leave SGW-FLT (Started orana) Jan 13 19:33:00 [4295] orana pengine: info: LogActions: Leave SGW-FLT2 (Started orana) Jan 13 19:33:00 [4296] orana crmd: info: do_state_transition: State transition S_POLICY_ENGINE -> S_TRANSITION_ENGINE [ input=I_PE_SUCCESS cause=C_IPC_MESSAGE origin=handle_response ] Jan 13 19:33:00 [4296] orana crmd: info: do_te_invoke: Processing graph 2314 (ref=pe_calc-dc-1452731580-2545) derived from /var/lib/pacemaker/pengine/pe-input-1449.bz2 Jan 13 19:33:00 [4296] orana crmd: notice: run_graph: Transition 2314 (Complete=0, Pending=0, Fired=0, Skipped=0, Incomplete=0, Source=/var/lib/pacemaker/pengine/pe-input-1449.bz2): Complete Jan 13 19:33:00 [4296] orana crmd: info: do_log: FSA: Input I_TE_SUCCESS from notify_crmd() received in state S_TRANSITION_ENGINE Jan 13 19:33:00 [4296] orana crmd: notice: do_state_transition: State transition S_TRANSITION_ENGINE -> S_IDLE [ input=I_TE_SUCCESS cause=C_FSA_INTERNAL origin=notify_crmd ] Jan 13 19:33:00 [4295] orana pengine: notice: process_pe_message: Calculated Transition 2314: /var/lib/pacemaker/pengine/pe-input-1449.bz2 Jan 13 19:33:00 [4291] orana cib: info: cib_perform_op: Diff: --- 0.73.0 2 Jan 13 19:33:00 [4291] orana cib: info: cib_perform_op: Diff: +++ 0.74.0 (null) Jan 13 19:33:00 [4291] orana cib: info: cib_perform_op: + /cib: @epoch=74 Jan 13 19:33:00 [4291] orana cib: info: cib_perform_op: ++ /cib/configuration/crm_config/cluster_property_set[@id='cib-bootstrap-options']: <nvpair id="cib-bootstrap-options-cluster-recheck-interval" name="cluster-recheck-interval" value="30s"/> Jan 13 19:33:00 [4291] orana cib: info: cib_process_request: Completed cib_replace operation for section configuration: OK (rc=0, origin=kamet/cibadmin/2, version=0.74.0) Jan 13 19:33:00 [4296] orana crmd: info: abort_transition_graph: Transition aborted by cib-bootstrap-options-cluster-recheck-interval, cluster-recheck-interval=30s: Non-status change (create cib=0.74.0, source=te_update_diff:383, path=/cib/configuration/crm_config/cluster_property_set[@id='cib-bootstrap-options'], 1) Jan 13 19:33:00 [4296] orana crmd: notice: do_state_transition: State transition S_IDLE -> S_POLICY_ENGINE [ input=I_PE_CALC cause=C_FSA_INTERNAL origin=abort_transition_graph ] Jan 13 19:33:00 [4291] orana cib: info: write_cib_contents: Archived previous version as /var/lib/pacemaker/cib/cib-84.raw Jan 13 19:33:00 [4291] orana cib: info: write_cib_contents: Wrote version 0.74.0 of the CIB to disk (digest: ac133341560d33343b2410d8833adfb8) Jan 13 19:33:00 [4291] orana cib: info: retrieveCib: Reading cluster configuration from: /var/lib/pacemaker/cib/cib.yXjrc1 (digest: /var/lib/pacemaker/cib/cib.zRGbVH) Jan 13 19:33:00 [4295] orana pengine: notice: unpack_config: On loss of CCM Quorum: Ignore Jan 13 19:33:00 [4295] orana pengine: info: determine_online_status_fencing: - Node kamet is not ready to run resources Jan 13 19:33:00 [4295] orana pengine: info: determine_online_status: Node kamet is pending Jan 13 19:33:00 [4295] orana pengine: info: determine_online_status_fencing: Node orana is active Jan 13 19:33:00 [4295] orana pengine: info: determine_online_status: Node orana is online Jan 13 19:33:00 [4295] orana pengine: info: clone_print: Master/Slave Set: foo-master [foo] Jan 13 19:33:00 [4295] orana pengine: info: short_print: Masters: [ orana ] Jan 13 19:33:00 [4295] orana pengine: info: short_print: Stopped: [ kamet ] Jan 13 19:33:00 [4295] orana pengine: info: native_print: fence-uc-orana (stonith:fence_ilo4): Started orana Jan 13 19:33:00 [4295] orana pengine: info: native_print: fence-uc-kamet (stonith:fence_ilo4): Started orana Jan 13 19:33:00 [4295] orana pengine: info: native_print: CWS-3G (ocf::pw:IPaddr): Started orana Jan 13 19:33:00 [4295] orana pengine: info: native_print: CWS-FLT (ocf::pw:IPaddr): Started orana Jan 13 19:33:00 [4295] orana pengine: info: native_print: CWS-FLT2 (ocf::pw:IPaddr): Started orana Jan 13 19:33:00 [4295] orana pengine: info: native_print: EPC-3G (ocf::pw:IPaddr): Started orana Jan 13 19:33:00 [4295] orana pengine: info: native_print: MGMT-FLT (ocf::pw:IPaddr): Started orana Jan 13 19:33:00 [4295] orana pengine: info: native_print: MME-FLT (ocf::pw:IPaddr): Started orana Jan 13 19:33:00 [4295] orana pengine: info: native_print: MME-FLT2 (ocf::pw:IPaddr): Started orana Jan 13 19:33:00 [4295] orana pengine: info: native_print: SGW-FLT (ocf::pw:IPaddr): Started orana Jan 13 19:33:00 [4295] orana pengine: info: native_print: SGW-FLT2 (ocf::pw:IPaddr): Started orana Jan 13 19:33:00 [4295] orana pengine: info: native_color: Resource foo:1 cannot run anywhere Jan 13 19:33:00 [4295] orana pengine: info: master_color: Promoting foo:0 (Master orana) Jan 13 19:33:00 [4295] orana pengine: info: master_color: foo-master: Promoted 1 instances of a possible 1 to master Jan 13 19:33:00 [4295] orana pengine: info: probe_resources: Action probe_complete-kamet on kamet is unrunnable (pending) Jan 13 19:33:00 [4295] orana pengine: warning: custom_action: Action foo:0_monitor_0 on kamet is unrunnable (pending) Jan 13 19:33:00 [4295] orana pengine: warning: custom_action: Action fence-uc-orana_monitor_0 on kamet is unrunnable (pending) Jan 13 19:33:00 [4295] orana pengine: warning: custom_action: Action fence-uc-kamet_monitor_0 on kamet is unrunnable (pending) Jan 13 19:33:00 [4295] orana pengine: warning: custom_action: Action CWS-3G_monitor_0 on kamet is unrunnable (pending) Jan 13 19:33:00 [4295] orana pengine: warning: custom_action: Action CWS-FLT_monitor_0 on kamet is unrunnable (pending) Jan 13 19:33:00 [4295] orana pengine: warning: custom_action: Action CWS-FLT2_monitor_0 on kamet is unrunnable (pending) Jan 13 19:33:00 [4295] orana pengine: warning: custom_action: Action EPC-3G_monitor_0 on kamet is unrunnable (pending) Jan 13 19:33:00 [4295] orana pengine: warning: custom_action: Action MGMT-FLT_monitor_0 on kamet is unrunnable (pending) Jan 13 19:33:00 [4295] orana pengine: warning: custom_action: Action MME-FLT_monitor_0 on kamet is unrunnable (pending) Jan 13 19:33:00 [4295] orana pengine: warning: custom_action: Action MME-FLT2_monitor_0 on kamet is unrunnable (pending) Jan 13 19:33:00 [4295] orana pengine: warning: custom_action: Action SGW-FLT_monitor_0 on kamet is unrunnable (pending) Jan 13 19:33:00 [4295] orana pengine: warning: custom_action: Action SGW-FLT2_monitor_0 on kamet is unrunnable (pending) Jan 13 19:33:00 [4295] orana pengine: info: LogActions: Leave foo:0 (Master orana) Jan 13 19:33:00 [4295] orana pengine: info: LogActions: Leave foo:1 (Stopped) Jan 13 19:33:00 [4295] orana pengine: info: LogActions: Leave fence-uc-orana (Started orana) Jan 13 19:33:00 [4295] orana pengine: info: LogActions: Leave fence-uc-kamet (Started orana) Jan 13 19:33:00 [4295] orana pengine: info: LogActions: Leave CWS-3G (Started orana) Jan 13 19:33:00 [4295] orana pengine: info: LogActions: Leave CWS-FLT (Started orana) Jan 13 19:33:00 [4295] orana pengine: info: LogActions: Leave CWS-FLT2 (Started orana) Jan 13 19:33:00 [4295] orana pengine: info: LogActions: Leave EPC-3G (Started orana) Jan 13 19:33:00 [4295] orana pengine: info: LogActions: Leave MGMT-FLT (Started orana) Jan 13 19:33:00 [4295] orana pengine: info: LogActions: Leave MME-FLT (Started orana) Jan 13 19:33:00 [4295] orana pengine: info: LogActions: Leave MME-FLT2 (Started orana) Jan 13 19:33:00 [4295] orana pengine: info: LogActions: Leave SGW-FLT (Started orana) Jan 13 19:33:00 [4295] orana pengine: info: LogActions: Leave SGW-FLT2 (Started orana) Jan 13 19:33:00 [4296] orana crmd: info: do_state_transition: State transition S_POLICY_ENGINE -> S_TRANSITION_ENGINE [ input=I_PE_SUCCESS cause=C_IPC_MESSAGE origin=handle_response ] Jan 13 19:33:00 [4296] orana crmd: info: do_te_invoke: Processing graph 2315 (ref=pe_calc-dc-1452731580-2546) derived from /var/lib/pacemaker/pengine/pe-input-1450.bz2 Jan 13 19:33:00 [4296] orana crmd: notice: run_graph: Transition 2315 (Complete=0, Pending=0, Fired=0, Skipped=0, Incomplete=0, Source=/var/lib/pacemaker/pengine/pe-input-1450.bz2): Complete Jan 13 19:33:00 [4296] orana crmd: info: do_log: FSA: Input I_TE_SUCCESS from notify_crmd() received in state S_TRANSITION_ENGINE Jan 13 19:33:00 [4296] orana crmd: notice: do_state_transition: State transition S_TRANSITION_ENGINE -> S_IDLE [ input=I_TE_SUCCESS cause=C_FSA_INTERNAL origin=notify_crmd ] Jan 13 19:33:00 [4295] orana pengine: notice: process_pe_message: Calculated Transition 2315: /var/lib/pacemaker/pengine/pe-input-1450.bz2 Jan 13 19:33:00 [4296] orana crmd: info: pcmk_cpg_membership: Joined[2.0] crmd.2 Jan 13 19:33:00 [4296] orana crmd: info: pcmk_cpg_membership: Member[2.0] crmd.1 Jan 13 19:33:00 [4296] orana crmd: info: pcmk_cpg_membership: Member[2.1] crmd.2 Jan 13 19:33:00 [4296] orana crmd: info: crm_update_peer_proc: pcmk_cpg_membership: Node kamet[2] - corosync-cpg is now online Jan 13 19:33:00 [4296] orana crmd: info: peer_update_callback: Client kamet/peer now has status [online] (DC=true, changed=4000000) Jan 13 19:33:00 [4296] orana crmd: notice: do_state_transition: State transition S_IDLE -> S_INTEGRATION [ input=I_NODE_JOIN cause=C_FSA_INTERNAL origin=peer_update_callback ] Jan 13 19:33:00 [4296] orana crmd: info: do_dc_join_offer_one: An unknown node joined - (re-)offer to any unconfirmed nodes Jan 13 19:33:00 [4296] orana crmd: info: join_make_offer: Making join offers based on membership 7044 Jan 13 19:33:00 [4296] orana crmd: info: join_make_offer: join-2: Sending offer to kamet Jan 13 19:33:00 [4291] orana cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/crmd/2492) Jan 13 19:33:00 [4296] orana crmd: info: crm_update_peer_join: join_make_offer: Node kamet[2] - join-2 phase 0 -> 1 Jan 13 19:33:00 [4296] orana crmd: info: join_make_offer: Skipping orana: already known 4 Jan 13 19:33:00 [4296] orana crmd: info: abort_transition_graph: Transition aborted: Peer Halt (source=do_te_invoke:158, 1) Jan 13 19:33:00 [4291] orana cib: info: cib_process_request: Completed cib_modify operation for section nodes: OK (rc=0, origin=kamet/crmd/3, version=0.74.0) Jan 13 19:33:00 [4291] orana cib: info: cib_perform_op: Diff: --- 0.74.0 2 Jan 13 19:33:00 [4291] orana cib: info: cib_perform_op: Diff: +++ 0.74.1 (null) Jan 13 19:33:00 [4291] orana cib: info: cib_perform_op: + /cib: @num_updates=1 Jan 13 19:33:00 [4291] orana cib: info: cib_perform_op: + /cib/status/node_state[@id='kamet']: @crmd=online, @crm-debug-origin=peer_update_callback Jan 13 19:33:00 [4291] orana cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=orana/crmd/2492, version=0.74.1) Jan 13 19:33:00 [4291] orana cib: info: cib_process_replace: Replacement 0.4.0 from kamet not applied to 0.74.1: current epoch is greater than the replacement Jan 13 19:33:00 [4291] orana cib: warning: cib_process_request: Completed cib_replace operation for section 'all': Update was older than existing configuration (rc=-205, origin=kamet/cibadmin/2, version=0.74.1) Jan 13 19:33:00 [4291] orana cib: info: cib_perform_op: Diff: --- 0.74.1 2 Jan 13 19:33:00 [4291] orana cib: info: cib_perform_op: Diff: +++ 0.75.0 (null) Jan 13 19:33:00 [4291] orana cib: info: cib_perform_op: -- /cib/configuration/nodes/node[@id='kamet'] Jan 13 19:33:00 [4291] orana cib: info: cib_perform_op: -- /cib/configuration/nodes/node[@id='orana'] Jan 13 19:33:00 [4291] orana cib: info: cib_perform_op: -- /cib/configuration/resources/primitive[@id='fence-uc-orana']/meta_attributes[@id='fence-uc-orana-meta_attributes'] Jan 13 19:33:00 [4291] orana cib: info: cib_perform_op: -- /cib/configuration/resources/primitive[@id='fence-uc-kamet'] Jan 13 19:33:00 [4291] orana cib: info: cib_perform_op: -- /cib/configuration/resources/primitive[@id='CWS-3G'] Jan 13 19:33:00 [4291] orana cib: info: cib_perform_op: -- /cib/configuration/resources/primitive[@id='CWS-FLT'] Jan 13 19:33:00 [4291] orana cib: info: cib_perform_op: -- /cib/configuration/resources/primitive[@id='CWS-FLT2'] Jan 13 19:33:00 [4291] orana cib: info: cib_perform_op: -- /cib/configuration/resources/primitive[@id='EPC-3G'] Jan 13 19:33:00 [4291] orana cib: info: cib_perform_op: -- /cib/configuration/resources/primitive[@id='MGMT-FLT'] Jan 13 19:33:00 [4291] orana cib: info: cib_perform_op: -- /cib/configuration/resources/primitive[@id='MME-FLT'] Jan 13 19:33:00 [4291] orana cib: info: cib_perform_op: -- /cib/configuration/resources/primitive[@id='MME-FLT2'] Jan 13 19:33:00 [4291] orana cib: info: cib_perform_op: -- /cib/configuration/resources/primitive[@id='SGW-FLT'] Jan 13 19:33:00 [4291] orana cib: info: cib_perform_op: -- /cib/configuration/resources/primitive[@id='SGW-FLT2'] Jan 13 19:33:00 [4291] orana cib: info: cib_perform_op: -- /cib/configuration/constraints/rsc_colocation[@id='colocation-CWS-3G-foo-master-INFINITY'] Jan 13 19:33:00 [4291] orana cib: info: cib_perform_op: -- /cib/configuration/constraints/rsc_order[@id='order-CWS-3G-foo-master-mandatory'] Jan 13 19:33:00 [4291] orana cib: info: cib_perform_op: -- /cib/configuration/constraints/rsc_colocation[@id='colocation-CWS-FLT-foo-master-INFINITY'] Jan 13 19:33:00 [4291] orana cib: info: cib_perform_op: -- /cib/configuration/constraints/rsc_order[@id='order-CWS-FLT-foo-master-mandatory'] Jan 13 19:33:00 [4291] orana cib: info: cib_perform_op: -- /cib/configuration/constraints/rsc_colocation[@id='colocation-CWS-FLT2-foo-master-INFINITY'] Jan 13 19:33:00 [4291] orana cib: info: cib_perform_op: -- /cib/configuration/constraints/rsc_order[@id='order-CWS-FLT2-foo-master-mandatory'] Jan 13 19:33:00 [4291] orana cib: info: cib_perform_op: -- /cib/configuration/constraints/rsc_colocation[@id='colocation-EPC-3G-foo-master-INFINITY'] Jan 13 19:33:00 [4291] orana cib: info: cib_perform_op: -- /cib/configuration/constraints/rsc_order[@id='order-EPC-3G-foo-master-mandatory'] Jan 13 19:33:00 [4291] orana cib: info: cib_perform_op: -- /cib/configuration/constraints/rsc_colocation[@id='colocation-MGMT-FLT-foo-master-INFINITY'] Jan 13 19:33:00 [4291] orana cib: info: cib_perform_op: -- /cib/configuration/constraints/rsc_order[@id='order-MGMT-FLT-foo-master-mandatory'] Jan 13 19:33:00 [4291] orana cib: info: cib_perform_op: -- /cib/configuration/constraints/rsc_colocation[@id='colocation-MME-FLT-foo-master-INFINITY'] Jan 13 19:33:00 [4291] orana cib: info: cib_perform_op: -- /cib/configuration/constraints/rsc_order[@id='order-MME-FLT-foo-master-mandatory'] Jan 13 19:33:00 [4291] orana cib: info: cib_perform_op: -- /cib/configuration/constraints/rsc_colocation[@id='colocation-MME-FLT2-foo-master-INFINITY'] Jan 13 19:33:00 [4291] orana cib: info: cib_perform_op: -- /cib/configuration/constraints/rsc_order[@id='order-MME-FLT2-foo-master-mandatory'] Jan 13 19:33:00 [4291] orana cib: info: cib_perform_op: -- /cib/configuration/constraints/rsc_colocation[@id='colocation-SGW-FLT-foo-master-INFINITY'] Jan 13 19:33:00 [4291] orana cib: info: cib_perform_op: -- /cib/configuration/constraints/rsc_order[@id='order-SGW-FLT-foo-master-mandatory'] Jan 13 19:33:00 [4291] orana cib: info: cib_perform_op: -- /cib/configuration/constraints/rsc_colocation[@id='colocation-SGW-FLT2-foo-master-INFINITY'] Jan 13 19:33:00 [4291] orana cib: info: cib_perform_op: -- /cib/configuration/constraints/rsc_order[@id='order-SGW-FLT2-foo-master-mandatory'] Jan 13 19:33:00 [4291] orana cib: info: cib_perform_op: -- /cib/configuration/constraints/rsc_colocation[@id='colocation-fence-uc-orana-foo-master-INFINITY'] Jan 13 19:33:00 [4291] orana cib: info: cib_perform_op: -- /cib/configuration/constraints/rsc_colocation[@id='colocation-fence-uc-kamet-foo-master-INFINITY'] Jan 13 19:33:00 [4291] orana cib: info: cib_perform_op: -- /cib/configuration/constraints/rsc_order[@id='order-fence-uc-kamet-foo-master-mandatory'] Jan 13 19:33:00 [4291] orana cib: info: cib_perform_op: -- /cib/configuration/constraints/rsc_order[@id='order-fence-uc-orana-foo-master-mandatory'] Jan 13 19:33:00 [4291] orana cib: info: cib_perform_op: + /cib: @epoch=75, @num_updates=0 Jan 13 19:33:00 [4291] orana cib: info: cib_perform_op: + /cib/configuration/resources/primitive[@id='fence-uc-orana']/instance_attributes[@id='fence-uc-orana-instance_attributes']/nvpair[@id='fence-uc-orana-instance_attributes-delay']: @value=0 Jan 13 19:33:00 [4291] orana cib: info: cib_process_request: Completed cib_replace operation for section configuration: OK (rc=0, origin=kamet/cibadmin/2, version=0.75.0) Jan 13 19:33:00 [4292] orana stonith-ng: info: stonith_device_remove: Device 'CWS-3G' not found (2 active devices) Jan 13 19:33:00 [4292] orana stonith-ng: info: stonith_device_remove: Device 'CWS-FLT' not found (2 active devices) Jan 13 19:33:00 [4292] orana stonith-ng: info: stonith_device_remove: Device 'CWS-FLT2' not found (2 active devices) Jan 13 19:33:00 [4292] orana stonith-ng: info: stonith_device_remove: Device 'EPC-3G' not found (2 active devices) Jan 13 19:33:00 [4292] orana stonith-ng: info: stonith_device_remove: Device 'MGMT-FLT' not found (2 active devices) Jan 13 19:33:00 [4292] orana stonith-ng: info: stonith_device_remove: Device 'MME-FLT' not found (2 active devices) Jan 13 19:33:00 [4292] orana stonith-ng: info: stonith_device_remove: Device 'MME-FLT2' not found (2 active devices) Jan 13 19:33:00 [4292] orana stonith-ng: info: stonith_device_remove: Device 'SGW-FLT' not found (2 active devices) Jan 13 19:33:00 [4292] orana stonith-ng: info: stonith_device_remove: Device 'SGW-FLT2' not found (2 active devices) Jan 13 19:33:00 [4292] orana stonith-ng: info: update_cib_stonith_devices_v2: Updating device list from the cib: modify nvpair[@id='fence-uc-orana-instance_attributes-delay'] Jan 13 19:33:00 [4292] orana stonith-ng: info: cib_devices_update: Updating devices to version 0.75.0 Jan 13 19:33:00 [4292] orana stonith-ng: notice: unpack_config: On loss of CCM Quorum: Ignore Jan 13 19:33:00 [4292] orana stonith-ng: info: unpack_nodes: Creating a fake local node Jan 13 19:33:00 [4292] orana stonith-ng: info: stonith_device_register: Overwriting an existing entry for fence-uc-orana from the cib Jan 13 19:33:00 [4292] orana stonith-ng: notice: stonith_device_register: Added 'fence-uc-orana' to the device list (2 active devices) Jan 13 19:33:00 [4291] orana cib: info: write_cib_contents: Archived previous version as /var/lib/pacemaker/cib/cib-85.raw Jan 13 19:33:00 [4291] orana cib: info: write_cib_contents: Wrote version 0.75.0 of the CIB to disk (digest: 4fb4a8ef2f8cde3a07fb30eb706e7e9c) Jan 13 19:33:00 [4291] orana cib: info: retrieveCib: Reading cluster configuration from: /var/lib/pacemaker/cib/cib.HyXAc3 (digest: /var/lib/pacemaker/cib/cib.lOa3UL) Jan 13 19:33:01 [4291] orana cib: info: cib_perform_op: Diff: --- 0.75.0 2 Jan 13 19:33:01 [4291] orana cib: info: cib_perform_op: Diff: +++ 0.76.0 (null) Jan 13 19:33:01 [4291] orana cib: info: cib_perform_op: + /cib: @epoch=76 Jan 13 19:33:01 [4291] orana cib: info: cib_perform_op: ++ /cib/configuration/constraints: <rsc_colocation id="colocation-fence-uc-orana-foo-master-INFINITY" rsc="fence-uc-orana" score="INFINITY" with-rsc="foo-master"/> Jan 13 19:33:01 [4291] orana cib: info: cib_process_request: Completed cib_replace operation for section configuration: OK (rc=0, origin=kamet/cibadmin/2, version=0.76.0) Jan 13 19:33:01 [4291] orana cib: info: write_cib_contents: Archived previous version as /var/lib/pacemaker/cib/cib-86.raw Jan 13 19:33:01 [4291] orana cib: info: write_cib_contents: Wrote version 0.76.0 of the CIB to disk (digest: df07ff6cbef5a35891d43b89b9ba4371) Jan 13 19:33:01 [4291] orana cib: info: retrieveCib: Reading cluster configuration from: /var/lib/pacemaker/cib/cib.Pi8ov3 (digest: /var/lib/pacemaker/cib/cib.kYeIwM) Jan 13 19:33:01 [4291] orana cib: info: cib_perform_op: Diff: --- 0.76.0 2 Jan 13 19:33:01 [4291] orana cib: info: cib_perform_op: Diff: +++ 0.77.0 (null) Jan 13 19:33:01 [4291] orana cib: info: cib_perform_op: + /cib: @epoch=77 Jan 13 19:33:01 [4291] orana cib: info: cib_perform_op: ++ /cib/configuration/resources: <primitive class="stonith" id="fence-uc-kamet" type="fence_ilo4"/> Jan 13 19:33:01 [4291] orana cib: info: cib_perform_op: ++ <instance_attributes id="fence-uc-kamet-instance_attributes"> Jan 13 19:33:01 [4291] orana cib: info: cib_perform_op: ++ <nvpair id="fence-uc-kamet-instance_attributes-login" name="login" value="parallel"/> Jan 13 19:33:01 [4291] orana cib: info: cib_perform_op: ++ <nvpair id="fence-uc-kamet-instance_attributes-passwd" name="passwd" value="wireless"/> Jan 13 19:33:01 [4291] orana cib: info: cib_perform_op: ++ <nvpair id="fence-uc-kamet-instance_attributes-ipaddr" name="ipaddr" value="10.11.10.21"/> Jan 13 19:33:01 [4291] orana cib: info: cib_perform_op: ++ <nvpair id="fence-uc-kamet-instance_attributes-pcmk_host_list" name="pcmk_host_list" value="kamet"/> Jan 13 19:33:01 [4291] orana cib: info: cib_perform_op: ++ <nvpair id="fence-uc-kamet-instance_attributes-action" name="action" value="off"/> Jan 13 19:33:01 [4291] orana cib: info: cib_perform_op: ++ <nvpair id="fence-uc-kamet-instance_attributes-lanplus" name="lanplus" value="1"/> Jan 13 19:33:01 [4291] orana cib: info: cib_perform_op: ++ <nvpair id="fence-uc-kamet-instance_attributes-delay" name="delay" value="10"/> Jan 13 19:33:01 [4291] orana cib: info: cib_perform_op: ++ </instance_attributes> Jan 13 19:33:01 [4291] orana cib: info: cib_perform_op: ++ <operations> Jan 13 19:33:01 [4291] orana cib: info: cib_perform_op: ++ <op id="fence-uc-kamet-start-interval-0s" interval="0s" name="start" on-fail="restart"/> Jan 13 19:33:01 [4291] orana cib: info: cib_perform_op: ++ <op id="fence-uc-kamet-monitor-interval-5s" interval="5s" name="monitor" on-fail="restart"/> Jan 13 19:33:01 [4291] orana cib: info: cib_perform_op: ++ </operations> Jan 13 19:33:01 [4291] orana cib: info: cib_perform_op: ++ </primitive> Jan 13 19:33:01 [4291] orana cib: info: cib_process_request: Completed cib_replace operation for section configuration: OK (rc=0, origin=kamet/cibadmin/2, version=0.77.0) Jan 13 19:33:01 [4292] orana stonith-ng: info: update_cib_stonith_devices_v2: Updating device list from the cib: create resources Jan 13 19:33:01 [4292] orana stonith-ng: info: cib_devices_update: Updating devices to version 0.77.0 Jan 13 19:33:01 [4292] orana stonith-ng: notice: unpack_config: On loss of CCM Quorum: Ignore Jan 13 19:33:01 [4292] orana stonith-ng: info: unpack_nodes: Creating a fake local node Jan 13 19:33:01 [4292] orana stonith-ng: info: stonith_device_register: Overwriting an existing entry for fence-uc-kamet from the cib Jan 13 19:33:01 [4292] orana stonith-ng: notice: stonith_device_register: Added 'fence-uc-kamet' to the device list (2 active devices) Jan 13 19:33:01 [4291] orana cib: info: write_cib_contents: Archived previous version as /var/lib/pacemaker/cib/cib-87.raw Jan 13 19:33:01 [4291] orana cib: info: write_cib_contents: Wrote version 0.77.0 of the CIB to disk (digest: 0fef989ac2484c07ebc724551f42a236) Jan 13 19:33:01 [4291] orana cib: info: retrieveCib: Reading cluster configuration from: /var/lib/pacemaker/cib/cib.Nchv43 (digest: /var/lib/pacemaker/cib/cib.oYETEN) Jan 13 19:33:01 [4291] orana cib: info: cib_perform_op: Diff: --- 0.77.0 2 Jan 13 19:33:01 [4291] orana cib: info: cib_perform_op: Diff: +++ 0.78.0 (null) Jan 13 19:33:01 [4291] orana cib: info: cib_perform_op: + /cib: @epoch=78 Jan 13 19:33:01 [4291] orana cib: info: cib_perform_op: ++ /cib/configuration/constraints: <rsc_colocation id="colocation-fence-uc-kamet-foo-master-INFINITY" rsc="fence-uc-kamet" score="INFINITY" with-rsc="foo-master"/> Jan 13 19:33:01 [4291] orana cib: info: cib_process_request: Completed cib_replace operation for section configuration: OK (rc=0, origin=kamet/cibadmin/2, version=0.78.0) Jan 13 19:33:01 [4291] orana cib: info: write_cib_contents: Archived previous version as /var/lib/pacemaker/cib/cib-88.raw Jan 13 19:33:01 [4291] orana cib: info: write_cib_contents: Wrote version 0.78.0 of the CIB to disk (digest: 1381a968f3bd56162444466add6290cd) Jan 13 19:33:01 [4291] orana cib: info: retrieveCib: Reading cluster configuration from: /var/lib/pacemaker/cib/cib.eWlhn4 (digest: /var/lib/pacemaker/cib/cib.PRhpgO) Jan 13 19:33:01 [4291] orana cib: info: cib_perform_op: Diff: --- 0.78.0 2 Jan 13 19:33:01 [4291] orana cib: info: cib_perform_op: Diff: +++ 0.79.0 (null) Jan 13 19:33:01 [4291] orana cib: info: cib_perform_op: + /cib: @epoch=79 Jan 13 19:33:01 [4291] orana cib: info: cib_perform_op: ++ /cib/configuration/crm_config/cluster_property_set[@id='cib-bootstrap-options']: <nvpair id="cib-bootstrap-options-stonith-enabled" name="stonith-enabled" value="true"/> Jan 13 19:33:01 [4291] orana cib: info: cib_process_request: Completed cib_replace operation for section configuration: OK (rc=0, origin=kamet/cibadmin/2, version=0.79.0) Jan 13 19:33:01 [4291] orana cib: info: write_cib_contents: Archived previous version as /var/lib/pacemaker/cib/cib-89.raw Jan 13 19:33:01 [4291] orana cib: info: write_cib_contents: Wrote version 0.79.0 of the CIB to disk (digest: fba8e35c75c1ba34ae39845213394cb6) Jan 13 19:33:01 [4291] orana cib: info: retrieveCib: Reading cluster configuration from: /var/lib/pacemaker/cib/cib.UQzsJ4 (digest: /var/lib/pacemaker/cib/cib.GINNYO) Jan 13 19:33:01 [4296] orana crmd: info: do_dc_join_offer_one: join-2: Processing join_announce request from kamet in state S_INTEGRATION Jan 13 19:33:01 [4296] orana crmd: info: crm_update_peer_join: do_dc_join_offer_one: Node kamet[2] - join-2 phase 1 -> 0 Jan 13 19:33:01 [4296] orana crmd: info: join_make_offer: join-2: Sending offer to kamet Jan 13 19:33:01 [4296] orana crmd: info: crm_update_peer_join: join_make_offer: Node kamet[2] - join-2 phase 0 -> 1 Jan 13 19:33:01 [4296] orana crmd: info: crm_update_peer_join: join_make_offer: Node orana[1] - join-2 phase 4 -> 0 Jan 13 19:33:01 [4296] orana crmd: info: join_make_offer: join-2: Sending offer to orana Jan 13 19:33:01 [4296] orana crmd: info: crm_update_peer_join: join_make_offer: Node orana[1] - join-2 phase 0 -> 1 Jan 13 19:33:01 [4296] orana crmd: info: abort_transition_graph: Transition aborted: Node join (source=do_dc_join_offer_one:236, 1) Jan 13 19:33:01 [4296] orana crmd: info: crm_update_peer_join: do_dc_join_filter_offer: Node orana[1] - join-2 phase 1 -> 2 Jan 13 19:33:02 [4296] orana crmd: info: crm_update_peer_join: do_dc_join_filter_offer: Node kamet[2] - join-2 phase 1 -> 2 Jan 13 19:33:02 [4296] orana crmd: info: crm_update_peer_expected: do_dc_join_filter_offer: Node kamet[2] - expected state is now member (was down) Jan 13 19:33:02 [4296] orana crmd: info: do_state_transition: State transition S_INTEGRATION -> S_FINALIZE_JOIN [ input=I_INTEGRATED cause=C_FSA_INTERNAL origin=check_join_state ] Jan 13 19:33:02 [4296] orana crmd: info: crmd_join_phase_log: join-2: kamet=integrated Jan 13 19:33:02 [4296] orana crmd: info: crmd_join_phase_log: join-2: orana=integrated Jan 13 19:33:02 [4296] orana crmd: info: do_dc_join_finalize: join-2: Syncing our CIB to the rest of the cluster Jan 13 19:33:02 [4296] orana crmd: info: do_state_transition: State transition S_FINALIZE_JOIN -> S_INTEGRATION [ input=I_JOIN_REQUEST cause=C_HA_MESSAGE origin=route_message ] Jan 13 19:33:02 [4296] orana crmd: info: do_dc_join_offer_one: join-2: Processing join_request request from kamet in state S_INTEGRATION Jan 13 19:33:02 [4296] orana crmd: info: crm_update_peer_join: do_dc_join_offer_one: Node kamet[2] - join-2 phase 2 -> 0 Jan 13 19:33:02 [4296] orana crmd: info: join_make_offer: join-2: Sending offer to kamet Jan 13 19:33:02 [4296] orana crmd: info: crm_update_peer_join: join_make_offer: Node kamet[2] - join-2 phase 0 -> 1 Jan 13 19:33:02 [4296] orana crmd: info: crm_update_peer_join: join_make_offer: Node orana[1] - join-2 phase 2 -> 0 Jan 13 19:33:02 [4296] orana crmd: info: join_make_offer: join-2: Sending offer to orana Jan 13 19:33:02 [4296] orana crmd: info: crm_update_peer_join: join_make_offer: Node orana[1] - join-2 phase 0 -> 1 Jan 13 19:33:02 [4296] orana crmd: info: abort_transition_graph: Transition aborted: Node join (source=do_dc_join_offer_one:236, 1) Jan 13 19:33:02 [4296] orana crmd: info: crm_update_peer_join: do_dc_join_filter_offer: Node kamet[2] - join-2 phase 1 -> 2 Jan 13 19:33:02 [4296] orana crmd: info: crm_update_peer_join: do_dc_join_filter_offer: Node orana[1] - join-2 phase 1 -> 2 Jan 13 19:33:02 [4296] orana crmd: info: do_state_transition: State transition S_INTEGRATION -> S_FINALIZE_JOIN [ input=I_INTEGRATED cause=C_FSA_INTERNAL origin=check_join_state ] Jan 13 19:33:02 [4296] orana crmd: info: crmd_join_phase_log: join-2: kamet=integrated Jan 13 19:33:02 [4296] orana crmd: info: crmd_join_phase_log: join-2: orana=integrated Jan 13 19:33:02 [4296] orana crmd: info: do_dc_join_finalize: join-2: Syncing our CIB to the rest of the cluster Jan 13 19:33:02 [4291] orana cib: info: cib_process_replace: Digest matched on replace from orana: 1b71d8dd45f16ee42d3e7ee539ee89e0 Jan 13 19:33:02 [4291] orana cib: info: cib_process_replace: Replaced 0.79.0 with 0.79.0 from orana Jan 13 19:33:02 [4291] orana cib: info: cib_process_request: Completed cib_replace operation for section 'all': OK (rc=0, origin=orana/crmd/2495, version=0.79.0) Jan 13 19:33:02 [4296] orana crmd: info: crm_update_peer_join: finalize_join_for: Node kamet[2] - join-2 phase 2 -> 3 Jan 13 19:33:02 [4296] orana crmd: info: crm_update_peer_join: finalize_join_for: Node orana[1] - join-2 phase 2 -> 3 Jan 13 19:33:02 [4291] orana cib: info: cib_process_replace: Digest matched on replace from orana: 1b71d8dd45f16ee42d3e7ee539ee89e0 Jan 13 19:33:02 [4291] orana cib: info: cib_process_replace: Replaced 0.79.0 with 0.79.0 from orana Jan 13 19:33:02 [4291] orana cib: info: write_cib_contents: Archived previous version as /var/lib/pacemaker/cib/cib-90.raw Jan 13 19:33:02 [4291] orana cib: info: write_cib_contents: Wrote version 0.79.0 of the CIB to disk (digest: 0b3b25d4809db68d92bb5b905e3a4194) Jan 13 19:33:02 [4291] orana cib: info: cib_process_request: Completed cib_replace operation for section 'all': OK (rc=0, origin=orana/crmd/2497, version=0.79.0) Jan 13 19:33:02 [4291] orana cib: info: retrieveCib: Reading cluster configuration from: /var/lib/pacemaker/cib/cib.GELr89 (digest: /var/lib/pacemaker/cib/cib.EElPMZ) Jan 13 19:33:02 [4291] orana cib: info: cib_process_request: Completed cib_delete operation for section //node_state[@uname='kamet']/transient_attributes: OK (rc=0, origin=kamet/crmd/10, version=0.79.0) Jan 13 19:33:02 [4291] orana cib: info: cib_process_request: Forwarding cib_modify operation for section nodes to master (origin=local/crmd/2498) Jan 13 19:33:02 [4291] orana cib: info: cib_process_request: Forwarding cib_modify operation for section nodes to master (origin=local/crmd/2499) Jan 13 19:33:02 [4291] orana cib: info: cib_perform_op: Diff: --- 0.79.0 2 Jan 13 19:33:02 [4291] orana cib: info: cib_perform_op: Diff: +++ 0.79.1 (null) Jan 13 19:33:02 [4291] orana cib: info: cib_perform_op: + /cib: @num_updates=1 Jan 13 19:33:02 [4291] orana cib: info: cib_perform_op: ++ /cib/configuration/nodes: <node id="kamet" uname="kamet"/> Jan 13 19:33:02 [4291] orana cib: info: cib_process_request: Completed cib_modify operation for section nodes: OK (rc=0, origin=orana/crmd/2498, version=0.79.1) Jan 13 19:33:02 [4291] orana cib: info: cib_perform_op: Diff: --- 0.79.1 2 Jan 13 19:33:02 [4291] orana cib: info: cib_perform_op: Diff: +++ 0.79.2 (null) Jan 13 19:33:02 [4291] orana cib: info: cib_perform_op: + /cib: @num_updates=2 Jan 13 19:33:02 [4291] orana cib: info: cib_perform_op: ++ /cib/configuration/nodes: <node id="orana" uname="orana"/> Jan 13 19:33:02 [4291] orana cib: info: cib_process_request: Completed cib_modify operation for section nodes: OK (rc=0, origin=orana/crmd/2499, version=0.79.2) Jan 13 19:33:02 [4291] orana cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/113) Jan 13 19:33:02 [4291] orana cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=orana/attrd/113, version=0.79.2) Jan 13 19:33:02 [4291] orana cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/115) Jan 13 19:33:02 [4291] orana cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=orana/attrd/115, version=0.79.2) Jan 13 19:33:02 [4291] orana cib: info: write_cib_contents: Archived previous version as /var/lib/pacemaker/cib/cib-91.raw Jan 13 19:33:02 [4291] orana cib: info: write_cib_contents: Wrote version 0.79.0 of the CIB to disk (digest: e60d7f0838319a2b10b16a2ed5c7920e) Jan 13 19:33:02 [4291] orana cib: info: retrieveCib: Reading cluster configuration from: /var/lib/pacemaker/cib/cib.0uYLba (digest: /var/lib/pacemaker/cib/cib.6lSoTZ) Jan 13 19:33:02 [4296] orana crmd: info: services_os_action_execute: Managed IPaddr_meta-data_0 process 6178 exited with rc=0 Jan 13 19:33:03 [4296] orana crmd: info: crm_update_peer_join: do_dc_join_ack: Node kamet[2] - join-2 phase 3 -> 4 Jan 13 19:33:03 [4296] orana crmd: info: do_dc_join_ack: join-2: Updating node state to member for kamet Jan 13 19:33:03 [4296] orana crmd: info: erase_status_tag: Deleting xpath: //node_state[@uname='kamet']/lrm Jan 13 19:33:03 [4291] orana cib: info: cib_process_request: Forwarding cib_delete operation for section //node_state[@uname='kamet']/lrm to master (origin=local/crmd/2500) Jan 13 19:33:03 [4291] orana cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/crmd/2501) Jan 13 19:33:03 [4296] orana crmd: info: crm_update_peer_join: do_dc_join_ack: Node orana[1] - join-2 phase 3 -> 4 Jan 13 19:33:03 [4296] orana crmd: info: do_dc_join_ack: join-2: Updating node state to member for orana Jan 13 19:33:03 [4296] orana crmd: info: erase_status_tag: Deleting xpath: //node_state[@uname='orana']/lrm Jan 13 19:33:03 [4291] orana cib: info: cib_process_request: Completed cib_delete operation for section //node_state[@uname='kamet']/lrm: OK (rc=0, origin=orana/crmd/2500, version=0.79.2) Jan 13 19:33:03 [4291] orana cib: info: cib_perform_op: Diff: --- 0.79.2 2 Jan 13 19:33:03 [4291] orana cib: info: cib_perform_op: Diff: +++ 0.79.3 (null) Jan 13 19:33:03 [4291] orana cib: info: cib_perform_op: + /cib: @num_updates=3 Jan 13 19:33:03 [4291] orana cib: info: cib_perform_op: + /cib/status/node_state[@id='kamet']: @crm-debug-origin=do_lrm_query_internal, @join=member, @expected=member Jan 13 19:33:03 [4291] orana cib: info: cib_perform_op: ++ /cib/status/node_state[@id='kamet']: <lrm id="kamet"/> Jan 13 19:33:03 [4291] orana cib: info: cib_perform_op: ++ <lrm_resources/> Jan 13 19:33:03 [4291] orana cib: info: cib_perform_op: ++ </lrm> Jan 13 19:33:03 [4291] orana cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=orana/crmd/2501, version=0.79.3) Jan 13 19:33:03 [4291] orana cib: info: cib_process_request: Forwarding cib_delete operation for section //node_state[@uname='orana']/lrm to master (origin=local/crmd/2502) Jan 13 19:33:03 [4296] orana crmd: info: do_state_transition: State transition S_FINALIZE_JOIN -> S_POLICY_ENGINE [ input=I_FINALIZED cause=C_FSA_INTERNAL origin=check_join_state ] Jan 13 19:33:03 [4296] orana crmd: info: abort_transition_graph: Transition aborted: Peer Cancelled (source=do_te_invoke:151, 1) Jan 13 19:33:03 [4294] orana attrd: notice: attrd_local_callback: Sending full refresh (origin=crmd) Jan 13 19:33:03 [4294] orana attrd: notice: attrd_trigger_update: Sending flush op to all hosts for: master-foo (10) Jan 13 19:33:03 [4291] orana cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/crmd/2503) Jan 13 19:33:03 [4291] orana cib: info: cib_perform_op: Diff: --- 0.79.3 2 Jan 13 19:33:03 [4291] orana cib: info: cib_perform_op: Diff: +++ 0.79.4 (null) Jan 13 19:33:03 [4291] orana cib: info: cib_perform_op: -- /cib/status/node_state[@id='orana']/lrm[@id='orana'] Jan 13 19:33:03 [4291] orana cib: info: cib_perform_op: + /cib: @num_updates=4 Jan 13 19:33:03 [4291] orana cib: info: cib_process_request: Completed cib_delete operation for section //node_state[@uname='orana']/lrm: OK (rc=0, origin=orana/crmd/2502, version=0.79.4) Jan 13 19:33:03 [4296] orana crmd: info: abort_transition_graph: Transition aborted by deletion of lrm[@id='orana']: Resource state removal (cib=0.79.4, source=te_update_diff:429, path=/cib/status/node_state[@id='orana']/lrm[@id='orana'], 1) Jan 13 19:33:03 [4291] orana cib: info: cib_perform_op: Diff: --- 0.79.4 2 Jan 13 19:33:03 [4291] orana cib: info: cib_perform_op: Diff: +++ 0.79.5 (null) Jan 13 19:33:03 [4291] orana cib: info: cib_perform_op: + /cib: @num_updates=5 Jan 13 19:33:03 [4291] orana cib: info: cib_perform_op: + /cib/status/node_state[@id='orana']: @crm-debug-origin=do_lrm_query_internal Jan 13 19:33:03 [4291] orana cib: info: cib_perform_op: ++ /cib/status/node_state[@id='orana']: <lrm id="orana"/> Jan 13 19:33:03 [4291] orana cib: info: cib_perform_op: ++ <lrm_resources> Jan 13 19:33:03 [4291] orana cib: info: cib_perform_op: ++ <lrm_resource id="MGMT-FLT" type="IPaddr" class="ocf" provider="pw"> Jan 13 19:33:03 [4291] orana cib: info: cib_perform_op: ++ <lrm_rsc_op id="MGMT-FLT_last_0" operation_key="MGMT-FLT_start_0" operation="start" crm-debug-origin="build_active_RAs" crm_feature_set="3.0.9" transition-key="44:2301:0:70de24fe-0f17-47ec-a193-a2cdd9f9e5c7" transition-magic="0:0;44:2301:0:70de24fe-0f17-47ec-a193-a2cdd9f9e5c7" call-id="100" rc-code="0" op-status="0" interval="0" last-run="1452731294" last-rc-change="1452731294" exec-time="114" Jan 13 19:33:03 [4291] orana cib: info: cib_perform_op: ++ <lrm_rsc_op id="MGMT-FLT_monitor_200" operation_key="MGMT-FLT_monitor_200" operation="monitor" crm-debug-origin="build_active_RAs" crm_feature_set="3.0.9" transition-key="45:2301:0:70de24fe-0f17-47ec-a193-a2cdd9f9e5c7" transition-magic="0:0;45:2301:0:70de24fe-0f17-47ec-a193-a2cdd9f9e5c7" call-id="110" rc-code="0" op-status="0" interval="200" last-rc-change="1452731295" exec-time="94" queue-tim Jan 13 19:33:03 [4291] orana cib: info: cib_perform_op: ++ </lrm_resource> Jan 13 19:33:03 [4291] orana cib: info: cib_perform_op: ++ <lrm_resource id="SGW-FLT2" type="IPaddr" class="ocf" provider="pw"> Jan 13 19:33:03 [4291] orana cib: info: cib_perform_op: ++ <lrm_rsc_op id="SGW-FLT2_last_0" operation_key="SGW-FLT2_start_0" operation="start" crm-debug-origin="build_active_RAs" crm_feature_set="3.0.9" transition-key="52:2301:0:70de24fe-0f17-47ec-a193-a2cdd9f9e5c7" transition-magic="0:0;52:2301:0:70de24fe-0f17-47ec-a193-a2cdd9f9e5c7" call-id="104" rc-code="0" op-status="0" interval="0" last-run="1452731294" last-rc-change="1452731294" exec-time="112" Jan 13 19:33:03 [4291] orana cib: info: cib_perform_op: ++ <lrm_rsc_op id="SGW-FLT2_monitor_200" operation_key="SGW-FLT2_monitor_200" operation="monitor" crm-debug-origin="build_active_RAs" crm_feature_set="3.0.9" transition-key="53:2301:0:70de24fe-0f17-47ec-a193-a2cdd9f9e5c7" transition-magic="0:0;53:2301:0:70de24fe-0f17-47ec-a193-a2cdd9f9e5c7" call-id="112" rc-code="0" op-status="0" interval="200" last-rc-change="1452731295" exec-time="92" queue-tim Jan 13 19:33:03 [4291] orana cib: info: cib_perform_op: ++ </lrm_resource> Jan 13 19:33:03 [4291] orana cib: info: cib_perform_op: ++ <lrm_resource id="EPC-3G" type="IPaddr" class="ocf" provider="pw"> Jan 13 19:33:03 [4291] orana cib: info: cib_perform_op: ++ <lrm_rsc_op id="EPC-3G_last_0" operation_key="EPC-3G_start_0" operation="start" crm-debug-origin="build_active_RAs" crm_feature_set="3.0.9" transition-key="42:2301:0:70de24fe-0f17-47ec-a193-a2cdd9f9e5c7" transition-magic="0:0;42:2301:0:70de24fe-0f17-47ec-a193-a2cdd9f9e5c7" call-id="99" rc-code="0" op-status="0" interval="0" last-run="1452731294" last-rc-change="1452731294" exec-time="126" queu Jan 13 19:33:03 [4291] orana cib: info: cib_perform_op: ++ <lrm_rsc_op id="EPC-3G_monitor_200" operation_key="EPC-3G_monitor_200" operation="monitor" crm-debug-origin="build_active_RAs" crm_feature_set="3.0.9" transition-key="43:2301:0:70de24fe-0f17-47ec-a193-a2cdd9f9e5c7" transition-magic="0:0;43:2301:0:70de24fe-0f17-47ec-a193-a2cdd9f9e5c7" call-id="114" rc-code="0" op-status="0" interval="200" last-rc-change="1452731295" exec-time="75" queue-time="0 Jan 13 19:33:03 [4291] orana cib: info: cib_perform_op: ++ </lrm_resource> Jan 13 19:33:03 [4291] orana cib: info: cib_perform_op: ++ <lrm_resource id="CWS-3G" type="IPaddr" class="ocf" provider="pw"> Jan 13 19:33:03 [4291] orana cib: info: cib_perform_op: ++ <lrm_rsc_op id="CWS-3G_last_0" operation_key="CWS-3G_start_0" operation="start" crm-debug-origin="build_active_RAs" crm_feature_set="3.0.9" transition-key="36:2301:0:70de24fe-0f17-47ec-a193-a2cdd9f9e5c7" transition-magic="0:0;36:2301:0:70de24fe-0f17-47ec-a193-a2cdd9f9e5c7" call-id="96" rc-code="0" op-status="0" interval="0" last-run="1452731294" last-rc-change="1452731294" exec-time="123" queu Jan 13 19:33:03 [4291] orana cib: info: cib_perform_op: ++ <lrm_rsc_op id="CWS-3G_monitor_200" operation_key="CWS-3G_monitor_200" operation="monitor" crm-debug-origin="build_active_RAs" crm_feature_set="3.0.9" transition-key="37:2301:0:70de24fe-0f17-47ec-a193-a2cdd9f9e5c7" transition-magic="0:0;37:2301:0:70de24fe-0f17-47ec-a193-a2cdd9f9e5c7" call-id="113" rc-code="0" op-status="0" interval="200" last-rc-change="1452731295" exec-time="93" queue-time="0 Jan 13 19:33:03 [4291] orana cib: info: cib_perform_op: ++ </lrm_resource> Jan 13 19:33:03 [4291] orana cib: info: cib_perform_op: ++ <lrm_resource id="fence-uc-kamet" type="fence_ilo4" class="stonith"> Jan 13 19:33:03 [4291] orana cib: info: cib_perform_op: ++ <lrm_rsc_op id="fence-uc-kamet_last_0" operation_key="fence-uc-kamet_start_0" operation="start" crm-debug-origin="build_active_RAs" crm_feature_set="3.0.9" transition-key="13:2304:0:70de24fe-0f17-47ec-a193-a2cdd9f9e5c7" transition-magic="0:0;13:2304:0:70de24fe-0f17-47ec-a193-a2cdd9f9e5c7" call-id="131" rc-code="0" op-status="0" interval="0" last-run="1452731302" last-rc-change="1452731302" exe Jan 13 19:33:03 [4291] orana cib: info: cib_perform_op: ++ <lrm_rsc_op id="fence-uc-kamet_monitor_5000" operation_key="fence-uc-kamet_monitor_5000" operation="monitor" crm-debug-origin="build_active_RAs" crm_feature_set="3.0.9" transition-key="1:2304:0:70de24fe-0f17-47ec-a193-a2cdd9f9e5c7" transition-magic="0:0;1:2304:0:70de24fe-0f17-47ec-a193-a2cdd9f9e5c7" call-id="132" rc-code="0" op-status="0" interval="5000" last-rc-change="1452731303" exec-time=" Jan 13 19:33:03 [4291] orana cib: info: cib_perform_op: ++ </lrm_resource> Jan 13 19:33:03 [4291] orana cib: info: cib_perform_op: ++ <lrm_resource id="SGW-FLT" type="IPaddr" class="ocf" provider="pw"> Jan 13 19:33:03 [4291] orana cib: info: cib_perform_op: ++ <lrm_rsc_op id="SGW-FLT_last_0" operation_key="SGW-FLT_start_0" operation="start" crm-debug-origin="build_active_RAs" crm_feature_set="3.0.9" transition-key="50:2301:0:70de24fe-0f17-47ec-a193-a2cdd9f9e5c7" transition-magic="0:0;50:2301:0:70de24fe-0f17-47ec-a193-a2cdd9f9e5c7" call-id="103" rc-code="0" op-status="0" interval="0" last-run="1452731294" last-rc-change="1452731294" exec-time="123" q Jan 13 19:33:03 [4291] orana cib: info: cib_perform_op: ++ <lrm_rsc_op id="SGW-FLT_monitor_200" operation_key="SGW-FLT_monitor_200" operation="monitor" crm-debug-origin="build_active_RAs" crm_feature_set="3.0.9" transition-key="51:2301:0:70de24fe-0f17-47ec-a193-a2cdd9f9e5c7" transition-magic="0:0;51:2301:0:70de24fe-0f17-47ec-a193-a2cdd9f9e5c7" call-id="116" rc-code="0" op-status="0" interval="200" last-rc-change="1452731295" exec-time="91" queue-time= Jan 13 19:33:03 [4291] orana cib: info: cib_perform_op: ++ </lrm_resource> Jan 13 19:33:03 [4291] orana cib: info: cib_perform_op: ++ <lrm_resource id="foo" type="uc" class="ocf" provider="pw"> Jan 13 19:33:03 [4291] orana cib: info: cib_perform_op: ++ <lrm_rsc_op id="foo_last_0" operation_key="foo_promote_0" operation="promote" crm-debug-origin="build_active_RAs" crm_feature_set="3.0.9" transition-key="19:2304:0:70de24fe-0f17-47ec-a193-a2cdd9f9e5c7" transition-magic="0:0;19:2304:0:70de24fe-0f17-47ec-a193-a2cdd9f9e5c7" call-id="133" rc-code="0" op-status="0" interval="0" last-run="1452731303" last-rc-change="1452731303" exec-time=" Jan 13 19:33:03 [4291] orana cib: info: cib_perform_op: ++ <lrm_rsc_op id="foo_monitor_10000" operation_key="foo_monitor_10000" operation="monitor" crm-debug-origin="build_active_RAs" crm_feature_set="3.0.9" transition-key="20:2304:8:70de24fe-0f17-47ec-a193-a2cdd9f9e5c7" transition-magic="0:8;20:2304:8:70de24fe-0f17-47ec-a193-a2cdd9f9e5c7" call-id="135" rc-code="8" op-status="0" interval="10000" last-rc-change="1452731303" exec-time="39" que Jan 13 19:33:03 [4291] orana cib: info: cib_perform_op: ++ </lrm_resource> Jan 13 19:33:03 [4291] orana cib: info: cib_perform_op: ++ <lrm_resource id="MME-FLT" type="IPaddr" class="ocf" provider="pw"> Jan 13 19:33:03 [4291] orana cib: info: cib_perform_op: ++ <lrm_rsc_op id="MME-FLT_last_0" operation_key="MME-FLT_start_0" operation="start" crm-debug-origin="build_active_RAs" crm_feature_set="3.0.9" transition-key="46:2301:0:70de24fe-0f17-47ec-a193-a2cdd9f9e5c7" transition-magic="0:0;46:2301:0:70de24fe-0f17-47ec-a193-a2cdd9f9e5c7" call-id="101" rc-code="0" op-status="0" interval="0" last-run="1452731294" last-rc-change="1452731294" exec-time="122" q Jan 13 19:33:03 [4291] orana cib: info: cib_perform_op: ++ <lrm_rsc_op id="MME-FLT_monitor_200" operation_key="MME-FLT_monitor_200" operation="monitor" crm-debug-origin="build_active_RAs" crm_feature_set="3.0.9" transition-key="47:2301:0:70de24fe-0f17-47ec-a193-a2cdd9f9e5c7" transition-magic="0:0;47:2301:0:70de24fe-0f17-47ec-a193-a2cdd9f9e5c7" call-id="115" rc-code="0" op-status="0" interval="200" last-rc-change="1452731295" exec-time="80" queue-time= Jan 13 19:33:03 [4291] orana cib: info: cib_perform_op: ++ </lrm_resource> Jan 13 19:33:03 [4291] orana cib: info: cib_perform_op: ++ <lrm_resource id="fence-uc-orana" type="fence_ilo4" class="stonith"> Jan 13 19:33:03 [4291] orana cib: info: cib_perform_op: ++ <lrm_rsc_op id="fence-uc-orana_last_0" operation_key="fence-uc-orana_start_0" operation="start" crm-debug-origin="build_active_RAs" crm_feature_set="3.0.9" transition-key="14:2303:0:70de24fe-0f17-47ec-a193-a2cdd9f9e5c7" transition-magic="0:0;14:2303:0:70de24fe-0f17-47ec-a193-a2cdd9f9e5c7" call-id="123" rc-code="0" op-status="0" interval="0" last-run="1452731302" last-rc-change="1452731302" exe Jan 13 19:33:03 [4291] orana cib: info: cib_perform_op: ++ <lrm_rsc_op id="fence-uc-orana_monitor_5000" operation_key="fence-uc-orana_monitor_5000" operation="monitor" crm-debug-origin="build_active_RAs" crm_feature_set="3.0.9" transition-key="12:2303:0:70de24fe-0f17-47ec-a193-a2cdd9f9e5c7" transition-magic="0:0;12:2303:0:70de24fe-0f17-47ec-a193-a2cdd9f9e5c7" call-id="124" rc-code="0" op-status="0" interval="5000" last-rc-change="1452731302" exec-time Jan 13 19:33:03 [4291] orana cib: info: cib_perform_op: ++ </lrm_resource> Jan 13 19:33:03 [4291] orana cib: info: cib_perform_op: ++ <lrm_resource id="CWS-FLT" type="IPaddr" class="ocf" provider="pw"> Jan 13 19:33:03 [4291] orana cib: info: cib_perform_op: ++ <lrm_rsc_op id="CWS-FLT_last_0" operation_key="CWS-FLT_start_0" operation="start" crm-debug-origin="build_active_RAs" crm_feature_set="3.0.9" transition-key="38:2301:0:70de24fe-0f17-47ec-a193-a2cdd9f9e5c7" transition-magic="0:0;38:2301:0:70de24fe-0f17-47ec-a193-a2cdd9f9e5c7" call-id="97" rc-code="0" op-status="0" interval="0" last-run="1452731294" last-rc-change="1452731294" exec-time="111" qu Jan 13 19:33:03 [4291] orana cib: info: cib_perform_op: ++ <lrm_rsc_op id="CWS-FLT_monitor_200" operation_key="CWS-FLT_monitor_200" operation="monitor" crm-debug-origin="build_active_RAs" crm_feature_set="3.0.9" transition-key="39:2301:0:70de24fe-0f17-47ec-a193-a2cdd9f9e5c7" transition-magic="0:0;39:2301:0:70de24fe-0f17-47ec-a193-a2cdd9f9e5c7" call-id="108" rc-code="0" op-status="0" interval="200" last-rc-change="1452731295" exec-time="75" queue-time= Jan 13 19:33:03 [4291] orana cib: info: cib_perform_op: ++ </lrm_resource> Jan 13 19:33:03 [4291] orana cib: info: cib_perform_op: ++ <lrm_resource id="CWS-FLT2" type="IPaddr" class="ocf" provider="pw"> Jan 13 19:33:03 [4291] orana cib: info: cib_perform_op: ++ <lrm_rsc_op id="CWS-FLT2_last_0" operation_key="CWS-FLT2_start_0" operation="start" crm-debug-origin="build_active_RAs" crm_feature_set="3.0.9" transition-key="40:2301:0:70de24fe-0f17-47ec-a193-a2cdd9f9e5c7" transition-magic="0:0;40:2301:0:70de24fe-0f17-47ec-a193-a2cdd9f9e5c7" call-id="98" rc-code="0" op-status="0" interval="0" last-run="1452731294" last-rc-change="1452731294" exec-time="113" Jan 13 19:33:03 [4291] orana cib: info: cib_perform_op: ++ <lrm_rsc_op id="CWS-FLT2_monitor_200" operation_key="CWS-FLT2_monitor_200" operation="monitor" crm-debug-origin="build_active_RAs" crm_feature_set="3.0.9" transition-key="41:2301:0:70de24fe-0f17-47ec-a193-a2cdd9f9e5c7" transition-magic="0:0;41:2301:0:70de24fe-0f17-47ec-a193-a2cdd9f9e5c7" call-id="109" rc-code="0" op-status="0" interval="200" last-rc-change="1452731295" exec-time="89" queue-tim Jan 13 19:33:03 [4291] orana cib: info: cib_perform_op: ++ </lrm_resource> Jan 13 19:33:03 [4291] orana cib: info: cib_perform_op: ++ <lrm_resource id="MME-FLT2" type="IPaddr" class="ocf" provider="pw"> Jan 13 19:33:03 [4291] orana cib: info: cib_perform_op: ++ <lrm_rsc_op id="MME-FLT2_last_0" operation_key="MME-FLT2_start_0" operation="start" crm-debug-origin="build_active_RAs" crm_feature_set="3.0.9" transition-key="48:2301:0:70de24fe-0f17-47ec-a193-a2cdd9f9e5c7" transition-magic="0:0;48:2301:0:70de24fe-0f17-47ec-a193-a2cdd9f9e5c7" call-id="102" rc-code="0" op-status="0" interval="0" last-run="1452731294" last-rc-change="1452731294" exec-time="112" Jan 13 19:33:03 [4291] orana cib: info: cib_perform_op: ++ <lrm_rsc_op id="MME-FLT2_monitor_200" operation_key="MME-FLT2_monitor_200" operation="monitor" crm-debug-origin="build_active_RAs" crm_feature_set="3.0.9" transition-key="49:2301:0:70de24fe-0f17-47ec-a193-a2cdd9f9e5c7" transition-magic="0:0;49:2301:0:70de24fe-0f17-47ec-a193-a2cdd9f9e5c7" call-id="111" rc-code="0" op-status="0" interval="200" last-rc-change="1452731295" exec-time="89" queue-tim Jan 13 19:33:03 [4291] orana cib: info: cib_perform_op: ++ </lrm_resource> Jan 13 19:33:03 [4291] orana cib: info: cib_perform_op: ++ </lrm_resources> Jan 13 19:33:03 [4291] orana cib: info: cib_perform_op: ++ </lrm> Jan 13 19:33:03 [4291] orana cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=orana/crmd/2503, version=0.79.5) Jan 13 19:33:03 [4291] orana cib: info: cib_process_request: Forwarding cib_modify operation for section nodes to master (origin=local/crmd/2504) Jan 13 19:33:03 [4296] orana crmd: info: abort_transition_graph: Transition aborted: LRM Refresh (source=process_resource_updates:266, 1) Jan 13 19:33:03 [4291] orana cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/crmd/2505) Jan 13 19:33:03 [4291] orana cib: info: cib_process_request: Forwarding cib_modify operation for section cib to master (origin=local/crmd/2506) Jan 13 19:33:03 [4294] orana attrd: notice: attrd_trigger_update: Sending flush op to all hosts for: shutdown (0) Jan 13 19:33:03 [4291] orana cib: info: cib_process_request: Completed cib_modify operation for section nodes: OK (rc=0, origin=orana/crmd/2504, version=0.79.5) Jan 13 19:33:03 [4291] orana cib: info: cib_perform_op: Diff: --- 0.79.5 2 Jan 13 19:33:03 [4291] orana cib: info: cib_perform_op: Diff: +++ 0.79.6 (null) Jan 13 19:33:03 [4291] orana cib: info: cib_perform_op: + /cib: @num_updates=6 Jan 13 19:33:03 [4291] orana cib: info: cib_perform_op: + /cib/status/node_state[@id='kamet']: @crm-debug-origin=do_state_transition Jan 13 19:33:03 [4291] orana cib: info: cib_perform_op: + /cib/status/node_state[@id='orana']: @crm-debug-origin=do_state_transition Jan 13 19:33:03 [4291] orana cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=orana/crmd/2505, version=0.79.6) Jan 13 19:33:03 [4291] orana cib: info: cib_process_request: Completed cib_modify operation for section cib: OK (rc=0, origin=orana/crmd/2506, version=0.79.6) Jan 13 19:33:03 [4291] orana cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/117) Jan 13 19:33:03 [4294] orana attrd: notice: attrd_trigger_update: Sending flush op to all hosts for: probe_complete (true) Jan 13 19:33:03 [4291] orana cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=orana/attrd/117, version=0.79.6) Jan 13 19:33:03 [4291] orana cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/119) Jan 13 19:33:03 [4291] orana cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=orana/attrd/119, version=0.79.6) Jan 13 19:33:03 [4291] orana cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/121) Jan 13 19:33:03 [4291] orana cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=orana/attrd/121, version=0.79.6) Jan 13 19:33:03 [4295] orana pengine: notice: unpack_config: On loss of CCM Quorum: Ignore Jan 13 19:33:03 [4295] orana pengine: info: determine_online_status_fencing: Node kamet is active Jan 13 19:33:03 [4295] orana pengine: info: determine_online_status: Node kamet is online Jan 13 19:33:03 [4295] orana pengine: info: determine_online_status_fencing: Node orana is active Jan 13 19:33:03 [4295] orana pengine: info: determine_online_status: Node orana is online Jan 13 19:33:03 [4295] orana pengine: warning: process_rsc_state: Detected active orphan MGMT-FLT running on orana Jan 13 19:33:03 [4295] orana pengine: warning: process_rsc_state: Detected active orphan SGW-FLT2 running on orana Jan 13 19:33:03 [4295] orana pengine: warning: process_rsc_state: Detected active orphan EPC-3G running on orana Jan 13 19:33:03 [4295] orana pengine: warning: process_rsc_state: Detected active orphan CWS-3G running on orana Jan 13 19:33:03 [4295] orana pengine: warning: process_rsc_state: Detected active orphan SGW-FLT running on orana Jan 13 19:33:03 [4295] orana pengine: warning: process_rsc_state: Detected active orphan MME-FLT running on orana Jan 13 19:33:03 [4295] orana pengine: warning: process_rsc_state: Detected active orphan CWS-FLT running on orana Jan 13 19:33:03 [4295] orana pengine: warning: process_rsc_state: Detected active orphan CWS-FLT2 running on orana Jan 13 19:33:03 [4295] orana pengine: warning: process_rsc_state: Detected active orphan MME-FLT2 running on orana Jan 13 19:33:03 [4295] orana pengine: info: clone_print: Master/Slave Set: foo-master [foo] Jan 13 19:33:03 [4295] orana pengine: info: short_print: Masters: [ orana ] Jan 13 19:33:03 [4295] orana pengine: info: short_print: Stopped: [ kamet ] Jan 13 19:33:03 [4295] orana pengine: info: native_print: fence-uc-orana (stonith:fence_ilo4): Started orana Jan 13 19:33:03 [4295] orana pengine: info: native_print: fence-uc-kamet (stonith:fence_ilo4): Started orana Jan 13 19:33:03 [4295] orana pengine: info: native_print: MGMT-FLT (ocf::pw:IPaddr): ORPHANED Started orana Jan 13 19:33:03 [4295] orana pengine: info: native_print: SGW-FLT2 (ocf::pw:IPaddr): ORPHANED Started orana Jan 13 19:33:03 [4295] orana pengine: info: native_print: EPC-3G (ocf::pw:IPaddr): ORPHANED Started orana Jan 13 19:33:03 [4295] orana pengine: info: native_print: CWS-3G (ocf::pw:IPaddr): ORPHANED Started orana Jan 13 19:33:03 [4295] orana pengine: info: native_print: SGW-FLT (ocf::pw:IPaddr): ORPHANED Started orana Jan 13 19:33:03 [4295] orana pengine: info: native_print: MME-FLT (ocf::pw:IPaddr): ORPHANED Started orana Jan 13 19:33:03 [4295] orana pengine: info: native_print: CWS-FLT (ocf::pw:IPaddr): ORPHANED Started orana Jan 13 19:33:03 [4295] orana pengine: info: native_print: CWS-FLT2 (ocf::pw:IPaddr): ORPHANED Started orana Jan 13 19:33:03 [4295] orana pengine: info: native_print: MME-FLT2 (ocf::pw:IPaddr): ORPHANED Started orana Jan 13 19:33:03 [4295] orana pengine: notice: DeleteRsc: Removing MGMT-FLT from orana Jan 13 19:33:03 [4295] orana pengine: notice: DeleteRsc: Removing SGW-FLT2 from orana Jan 13 19:33:03 [4295] orana pengine: notice: DeleteRsc: Removing EPC-3G from orana Jan 13 19:33:03 [4295] orana pengine: notice: DeleteRsc: Removing CWS-3G from orana Jan 13 19:33:03 [4295] orana pengine: info: check_action_definition: params:reload <parameters login="parallel" passwd="wireless" pcmk_host_list="kamet" lanplus="1" action="off" delay="10" ipaddr="10.11.10.21"/> Jan 13 19:33:03 [4295] orana pengine: info: check_action_definition: Parameters to fence-uc-kamet_start_0 on orana changed: was 93013a04ac64e00066a4f7b080b31515 vs. now a204a24f6a125ccea22f6dc681d020f0 (reload:3.0.9) 0:0;13:2304:0:70de24fe-0f17-47ec-a193-a2cdd9f9e5c7 Jan 13 19:33:03 [4295] orana pengine: info: check_action_definition: params:reload <parameters login="parallel" passwd="wireless" pcmk_host_list="kamet" lanplus="1" action="off" delay="10" ipaddr="10.11.10.21" CRM_meta_timeout="20000"/> Jan 13 19:33:03 [4295] orana pengine: info: check_action_definition: Parameters to fence-uc-kamet_monitor_5000 on orana changed: was e884a8e46b98e8563278b8117eb4db82 vs. now 948f08ea79b4152d6055da386ebacbb3 (reload:3.0.9) 0:0;1:2304:0:70de24fe-0f17-47ec-a193-a2cdd9f9e5c7 Jan 13 19:33:03 [4295] orana pengine: notice: DeleteRsc: Removing SGW-FLT from orana Jan 13 19:33:03 [4295] orana pengine: notice: DeleteRsc: Removing MME-FLT from orana Jan 13 19:33:03 [4295] orana pengine: info: check_action_definition: params:reload <parameters login="parallel" passwd="wireless" pcmk_host_list="orana" lanplus="1" action="off" delay="0" ipaddr="10.11.10.30"/> Jan 13 19:33:03 [4295] orana pengine: info: check_action_definition: Parameters to fence-uc-orana_start_0 on orana changed: was 363cbd5d9ec840db46e5e25e2f8b6652 vs. now 44b3f881263f198e021240ff36365f0f (reload:3.0.9) 0:0;14:2303:0:70de24fe-0f17-47ec-a193-a2cdd9f9e5c7 Jan 13 19:33:03 [4295] orana pengine: info: check_action_definition: params:reload <parameters login="parallel" passwd="wireless" pcmk_host_list="orana" lanplus="1" action="off" delay="0" ipaddr="10.11.10.30" CRM_meta_timeout="20000"/> Jan 13 19:33:03 [4295] orana pengine: info: check_action_definition: Parameters to fence-uc-orana_monitor_5000 on orana changed: was 8024392825e4a1e81b7f628e6dee69a6 vs. now ec6a353c4e45a74e3497360e0581b40c (reload:3.0.9) 0:0;12:2303:0:70de24fe-0f17-47ec-a193-a2cdd9f9e5c7 Jan 13 19:33:03 [4295] orana pengine: notice: DeleteRsc: Removing CWS-FLT from orana Jan 13 19:33:03 [4295] orana pengine: notice: DeleteRsc: Removing CWS-FLT2 from orana Jan 13 19:33:03 [4295] orana pengine: notice: DeleteRsc: Removing MME-FLT2 from orana Jan 13 19:33:03 [4295] orana pengine: info: master_color: Promoting foo:0 (Master orana) Jan 13 19:33:03 [4295] orana pengine: info: master_color: foo-master: Promoted 1 instances of a possible 1 to master Jan 13 19:33:03 [4295] orana pengine: info: native_color: Stopping orphan resource MGMT-FLT Jan 13 19:33:03 [4295] orana pengine: info: native_color: Stopping orphan resource SGW-FLT2 Jan 13 19:33:03 [4295] orana pengine: info: native_color: Stopping orphan resource EPC-3G Jan 13 19:33:03 [4295] orana pengine: info: native_color: Stopping orphan resource CWS-3G Jan 13 19:33:03 [4295] orana pengine: info: native_color: Stopping orphan resource SGW-FLT Jan 13 19:33:03 [4295] orana pengine: info: native_color: Stopping orphan resource MME-FLT Jan 13 19:33:03 [4295] orana pengine: info: native_color: Stopping orphan resource CWS-FLT Jan 13 19:33:03 [4295] orana pengine: info: native_color: Stopping orphan resource CWS-FLT2 Jan 13 19:33:03 [4295] orana pengine: info: native_color: Stopping orphan resource MME-FLT2 Jan 13 19:33:03 [4295] orana pengine: info: RecurringOp: Start recurring monitor (11s) for foo:1 on kamet Jan 13 19:33:03 [4295] orana pengine: info: RecurringOp: Start recurring monitor (11s) for foo:1 on kamet Jan 13 19:33:03 [4295] orana pengine: info: RecurringOp: Start recurring monitor (5s) for fence-uc-orana on orana Jan 13 19:33:03 [4295] orana pengine: info: RecurringOp: Start recurring monitor (5s) for fence-uc-kamet on kamet Jan 13 19:33:03 [4295] orana pengine: info: LogActions: Leave foo:0 (Master orana) Jan 13 19:33:03 [4295] orana pengine: notice: LogActions: Start foo:1 (kamet) Jan 13 19:33:03 [4295] orana pengine: notice: LogActions: Restart fence-uc-orana (Started orana) Jan 13 19:33:03 [4295] orana pengine: notice: LogActions: Move fence-uc-kamet (Started orana -> kamet) Jan 13 19:33:03 [4295] orana pengine: notice: LogActions: Stop MGMT-FLT (orana) Jan 13 19:33:03 [4295] orana pengine: notice: LogActions: Stop SGW-FLT2 (orana) Jan 13 19:33:03 [4295] orana pengine: notice: LogActions: Stop EPC-3G (orana) Jan 13 19:33:03 [4295] orana pengine: notice: LogActions: Stop CWS-3G (orana) Jan 13 19:33:03 [4295] orana pengine: notice: LogActions: Stop SGW-FLT (orana) Jan 13 19:33:03 [4295] orana pengine: notice: LogActions: Stop MME-FLT (orana) Jan 13 19:33:03 [4295] orana pengine: notice: LogActions: Stop CWS-FLT (orana) Jan 13 19:33:03 [4295] orana pengine: notice: LogActions: Stop CWS-FLT2 (orana) Jan 13 19:33:03 [4295] orana pengine: notice: LogActions: Stop MME-FLT2 (orana) Jan 13 19:33:03 [4295] orana pengine: notice: process_pe_message: Calculated Transition 2316: /var/lib/pacemaker/pengine/pe-input-1451.bz2 Jan 13 19:33:03 [4296] orana crmd: info: do_state_transition: State transition S_POLICY_ENGINE -> S_TRANSITION_ENGINE [ input=I_PE_SUCCESS cause=C_IPC_MESSAGE origin=handle_response ] Jan 13 19:33:03 [4296] orana crmd: info: do_te_invoke: Processing graph 2316 (ref=pe_calc-dc-1452731583-2557) derived from /var/lib/pacemaker/pengine/pe-input-1451.bz2 Jan 13 19:33:03 [4296] orana crmd: notice: te_rsc_command: Initiating action 27: monitor foo:1_monitor_0 on kamet Jan 13 19:33:03 [4296] orana crmd: notice: te_rsc_command: Initiating action 28: monitor fence-uc-orana_monitor_0 on kamet Jan 13 19:33:03 [4296] orana crmd: notice: te_rsc_command: Initiating action 29: monitor fence-uc-kamet_monitor_0 on kamet Jan 13 19:33:03 [4296] orana crmd: notice: te_rsc_command: Initiating action 76: notify foo_pre_notify_start_0 on orana (local) Jan 13 19:33:03 [4296] orana crmd: info: do_lrm_rsc_op: Performing key=76:2316:0:70de24fe-0f17-47ec-a193-a2cdd9f9e5c7 op=foo_notify_0 Jan 13 19:33:03 [4293] orana lrmd: info: log_execute: executing - rsc:foo action:notify call_id:136 uc(foo)[7992]: 2016/01/13_19:33:03 INFO: [UC] Wed Jan 13 19:33:03 EST 2016 NOTIFY pre-start master=orana start=kamet Jan 13 19:33:03 [4293] orana lrmd: info: log_finished: finished - rsc:foo action:notify call_id:136 pid:7992 exit-code:0 exec-time:49ms queue-time:0ms Jan 13 19:33:03 [4296] orana crmd: info: match_graph_event: Action foo_notify_0 (76) confirmed on orana (rc=0) Jan 13 19:33:03 [4296] orana crmd: notice: process_lrm_event: Operation foo_notify_0: ok (node=orana, call=136, rc=0, cib-update=0, confirmed=true) Jan 13 19:33:04 [4291] orana cib: info: cib_perform_op: Diff: --- 0.79.6 2 Jan 13 19:33:04 [4291] orana cib: info: cib_perform_op: Diff: +++ 0.79.7 (null) Jan 13 19:33:04 [4291] orana cib: info: cib_perform_op: + /cib: @num_updates=7 Jan 13 19:33:04 [4291] orana cib: info: cib_perform_op: ++ /cib/status/node_state[@id='kamet']: <transient_attributes id="kamet"/> Jan 13 19:33:04 [4291] orana cib: info: cib_perform_op: ++ <instance_attributes id="status-kamet"> Jan 13 19:33:04 [4291] orana cib: info: cib_perform_op: ++ <nvpair id="status-kamet-shutdown" name="shutdown" value="0"/> Jan 13 19:33:04 [4291] orana cib: info: cib_perform_op: ++ </instance_attributes> Jan 13 19:33:04 [4291] orana cib: info: cib_perform_op: ++ </transient_attributes> Jan 13 19:33:04 [4291] orana cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=kamet/attrd/3, version=0.79.7) Jan 13 19:33:04 [4296] orana crmd: notice: abort_transition_graph: Transition aborted by transient_attributes.kamet 'create': Transient attribute change (cib=0.79.7, source=te_update_diff:391, path=/cib/status/node_state[@id='kamet'], 0) Jan 13 19:33:04 [4291] orana cib: info: cib_perform_op: Diff: --- 0.79.7 2 Jan 13 19:33:04 [4291] orana cib: info: cib_perform_op: Diff: +++ 0.79.8 (null) Jan 13 19:33:04 [4291] orana cib: info: cib_perform_op: + /cib: @num_updates=8 Jan 13 19:33:04 [4291] orana cib: info: cib_perform_op: + /cib/status/node_state[@id='kamet']: @crm-debug-origin=do_update_resource Jan 13 19:33:04 [4291] orana cib: info: cib_perform_op: ++ /cib/status/node_state[@id='kamet']/lrm[@id='kamet']/lrm_resources: <lrm_resource id="fence-uc-orana" type="fence_ilo4" class="stonith"/> Jan 13 19:33:04 [4291] orana cib: info: cib_perform_op: ++ <lrm_rsc_op id="fence-uc-orana_last_0" operation_key="fence-uc-orana_monitor_0" operation="monitor" crm-debug-origin="do_update_resource" crm_feature_set="3.0.9" transition-key="28:2316:7:70de24fe-0f17-47ec-a193-a2cdd9f9e5c7" transition-magic="0:7;28:2316:7:70de24fe-0f17-47ec-a193-a2cdd9f9e5c7" call-id="10" rc-code="7" op-status="0" interval="0" last-run="1452731698" Jan 13 19:33:04 [4291] orana cib: info: cib_perform_op: ++ </lrm_resource> Jan 13 19:33:04 [4291] orana cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=kamet/crmd/11, version=0.79.8) Jan 13 19:33:04 [4296] orana crmd: info: match_graph_event: Action fence-uc-orana_monitor_0 (28) confirmed on kamet (rc=0) Jan 13 19:33:04 [4291] orana cib: info: cib_perform_op: Diff: --- 0.79.8 2 Jan 13 19:33:04 [4291] orana cib: info: cib_perform_op: Diff: +++ 0.79.9 (null) Jan 13 19:33:04 [4291] orana cib: info: cib_perform_op: + /cib: @num_updates=9 Jan 13 19:33:04 [4291] orana cib: info: cib_perform_op: ++ /cib/status/node_state[@id='kamet']/lrm[@id='kamet']/lrm_resources: <lrm_resource id="fence-uc-kamet" type="fence_ilo4" class="stonith"/> Jan 13 19:33:04 [4291] orana cib: info: cib_perform_op: ++ <lrm_rsc_op id="fence-uc-kamet_last_0" operation_key="fence-uc-kamet_monitor_0" operation="monitor" crm-debug-origin="do_update_resource" crm_feature_set="3.0.9" transition-key="29:2316:7:70de24fe-0f17-47ec-a193-a2cdd9f9e5c7" transition-magic="0:7;29:2316:7:70de24fe-0f17-47ec-a193-a2cdd9f9e5c7" call-id="14" rc-code="7" op-status="0" interval="0" last-run="1452731698" Jan 13 19:33:04 [4291] orana cib: info: cib_perform_op: ++ </lrm_resource> Jan 13 19:33:04 [4291] orana cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=kamet/crmd/12, version=0.79.9) Jan 13 19:33:04 [4296] orana crmd: info: match_graph_event: Action fence-uc-kamet_monitor_0 (29) confirmed on kamet (rc=0) Jan 13 19:33:04 [4291] orana cib: info: cib_perform_op: Diff: --- 0.79.9 2 Jan 13 19:33:04 [4291] orana cib: info: cib_perform_op: Diff: +++ 0.79.10 (null) Jan 13 19:33:04 [4291] orana cib: info: cib_perform_op: + /cib: @num_updates=10 Jan 13 19:33:04 [4291] orana cib: info: cib_perform_op: ++ /cib/status/node_state[@id='kamet']/lrm[@id='kamet']/lrm_resources: <lrm_resource id="foo" type="uc" class="ocf" provider="pw"/> Jan 13 19:33:04 [4291] orana cib: info: cib_perform_op: ++ <lrm_rsc_op id="foo_last_0" operation_key="foo_monitor_0" operation="monitor" crm-debug-origin="do_update_resource" crm_feature_set="3.0.9" transition-key="27:2316:7:70de24fe-0f17-47ec-a193-a2cdd9f9e5c7" transition-magic="0:7;27:2316:7:70de24fe-0f17-47ec-a193-a2cdd9f9e5c7" call-id="6" rc-code="7" op-status="0" interval="0" last-run="1452731698" last-rc-chan Jan 13 19:33:04 [4291] orana cib: info: cib_perform_op: ++ </lrm_resource> Jan 13 19:33:04 [4291] orana cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=kamet/crmd/13, version=0.79.10) Jan 13 19:33:04 [4296] orana crmd: info: match_graph_event: Action foo_monitor_0 (27) confirmed on kamet (rc=0) Jan 13 19:33:04 [4296] orana crmd: notice: te_rsc_command: Initiating action 26: probe_complete probe_complete-kamet on kamet - no waiting Jan 13 19:33:04 [4296] orana crmd: info: te_rsc_command: Action 26 confirmed - no wait Jan 13 19:33:04 [4296] orana crmd: notice: run_graph: Transition 2316 (Complete=8, Pending=0, Fired=0, Skipped=29, Incomplete=5, Source=/var/lib/pacemaker/pengine/pe-input-1451.bz2): Stopped Jan 13 19:33:04 [4296] orana crmd: info: do_state_transition: State transition S_TRANSITION_ENGINE -> S_POLICY_ENGINE [ input=I_PE_CALC cause=C_FSA_INTERNAL origin=notify_crmd ] Jan 13 19:33:04 [4291] orana cib: info: cib_perform_op: Diff: --- 0.79.10 2 Jan 13 19:33:04 [4291] orana cib: info: cib_perform_op: Diff: +++ 0.79.11 (null) Jan 13 19:33:04 [4291] orana cib: info: cib_perform_op: + /cib: @num_updates=11 Jan 13 19:33:04 [4291] orana cib: info: cib_perform_op: ++ /cib/status/node_state[@id='kamet']/transient_attributes[@id='kamet']/instance_attributes[@id='status-kamet']: <nvpair id="status-kamet-probe_complete" name="probe_complete" value="true"/> Jan 13 19:33:04 [4291] orana cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=kamet/attrd/8, version=0.79.11) Jan 13 19:33:04 [4291] orana cib: info: cib_process_request: Forwarding cib_modify operation for section status to master (origin=local/attrd/123) Jan 13 19:33:04 [4291] orana cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=orana/attrd/123, version=0.79.11) Jan 13 19:33:04 [4296] orana crmd: info: abort_transition_graph: Transition aborted by status-kamet-probe_complete, probe_complete=true: Transient attribute change (create cib=0.79.11, source=te_update_diff:391, path=/cib/status/node_state[@id='kamet']/transient_attributes[@id='kamet']/instance_attributes[@id='status-kamet'], 1) Jan 13 19:33:04 [4295] orana pengine: notice: unpack_config: On loss of CCM Quorum: Ignore Jan 13 19:33:04 [4295] orana pengine: info: determine_online_status_fencing: Node kamet is active Jan 13 19:33:04 [4295] orana pengine: info: determine_online_status: Node kamet is online Jan 13 19:33:04 [4295] orana pengine: info: determine_online_status_fencing: Node orana is active Jan 13 19:33:04 [4295] orana pengine: info: determine_online_status: Node orana is online Jan 13 19:33:04 [4295] orana pengine: warning: process_rsc_state: Detected active orphan MGMT-FLT running on orana Jan 13 19:33:04 [4295] orana pengine: warning: process_rsc_state: Detected active orphan SGW-FLT2 running on orana Jan 13 19:33:04 [4295] orana pengine: warning: process_rsc_state: Detected active orphan EPC-3G running on orana Jan 13 19:33:04 [4295] orana pengine: warning: process_rsc_state: Detected active orphan CWS-3G running on orana Jan 13 19:33:04 [4295] orana pengine: warning: process_rsc_state: Detected active orphan SGW-FLT running on orana Jan 13 19:33:04 [4295] orana pengine: warning: process_rsc_state: Detected active orphan MME-FLT running on orana Jan 13 19:33:04 [4295] orana pengine: warning: process_rsc_state: Detected active orphan CWS-FLT running on orana Jan 13 19:33:04 [4295] orana pengine: warning: process_rsc_state: Detected active orphan CWS-FLT2 running on orana Jan 13 19:33:04 [4295] orana pengine: warning: process_rsc_state: Detected active orphan MME-FLT2 running on orana Jan 13 19:33:04 [4295] orana pengine: info: clone_print: Master/Slave Set: foo-master [foo] Jan 13 19:33:04 [4295] orana pengine: info: short_print: Masters: [ orana ] Jan 13 19:33:04 [4295] orana pengine: info: short_print: Stopped: [ kamet ] Jan 13 19:33:04 [4295] orana pengine: info: native_print: fence-uc-orana (stonith:fence_ilo4): Started orana Jan 13 19:33:04 [4295] orana pengine: info: native_print: fence-uc-kamet (stonith:fence_ilo4): Started orana Jan 13 19:33:04 [4295] orana pengine: info: native_print: MGMT-FLT (ocf::pw:IPaddr): ORPHANED Started orana Jan 13 19:33:04 [4295] orana pengine: info: native_print: SGW-FLT2 (ocf::pw:IPaddr): ORPHANED Started orana Jan 13 19:33:04 [4295] orana pengine: info: native_print: EPC-3G (ocf::pw:IPaddr): ORPHANED Started orana Jan 13 19:33:04 [4295] orana pengine: info: native_print: CWS-3G (ocf::pw:IPaddr): ORPHANED Started orana Jan 13 19:33:04 [4295] orana pengine: info: native_print: SGW-FLT (ocf::pw:IPaddr): ORPHANED Started orana Jan 13 19:33:04 [4295] orana pengine: info: native_print: MME-FLT (ocf::pw:IPaddr): ORPHANED Started orana Jan 13 19:33:04 [4295] orana pengine: info: native_print: CWS-FLT (ocf::pw:IPaddr): ORPHANED Started orana Jan 13 19:33:04 [4295] orana pengine: info: native_print: CWS-FLT2 (ocf::pw:IPaddr): ORPHANED Started orana Jan 13 19:33:04 [4295] orana pengine: info: native_print: MME-FLT2 (ocf::pw:IPaddr): ORPHANED Started orana Jan 13 19:33:04 [4295] orana pengine: notice: DeleteRsc: Removing MGMT-FLT from orana Jan 13 19:33:04 [4295] orana pengine: notice: DeleteRsc: Removing SGW-FLT2 from orana Jan 13 19:33:04 [4295] orana pengine: notice: DeleteRsc: Removing EPC-3G from orana Jan 13 19:33:04 [4295] orana pengine: notice: DeleteRsc: Removing CWS-3G from orana Jan 13 19:33:04 [4295] orana pengine: info: check_action_definition: params:reload <parameters login="parallel" passwd="wireless" pcmk_host_list="kamet" lanplus="1" action="off" delay="10" ipaddr="10.11.10.21"/> Jan 13 19:33:04 [4295] orana pengine: info: check_action_definition: Parameters to fence-uc-kamet_start_0 on orana changed: was 93013a04ac64e00066a4f7b080b31515 vs. now a204a24f6a125ccea22f6dc681d020f0 (reload:3.0.9) 0:0;13:2304:0:70de24fe-0f17-47ec-a193-a2cdd9f9e5c7 Jan 13 19:33:04 [4295] orana pengine: info: check_action_definition: params:reload <parameters login="parallel" passwd="wireless" pcmk_host_list="kamet" lanplus="1" action="off" delay="10" ipaddr="10.11.10.21" CRM_meta_timeout="20000"/> Jan 13 19:33:04 [4295] orana pengine: info: check_action_definition: Parameters to fence-uc-kamet_monitor_5000 on orana changed: was e884a8e46b98e8563278b8117eb4db82 vs. now 948f08ea79b4152d6055da386ebacbb3 (reload:3.0.9) 0:0;1:2304:0:70de24fe-0f17-47ec-a193-a2cdd9f9e5c7 Jan 13 19:33:04 [4295] orana pengine: notice: DeleteRsc: Removing SGW-FLT from orana Jan 13 19:33:04 [4295] orana pengine: notice: DeleteRsc: Removing MME-FLT from orana Jan 13 19:33:04 [4295] orana pengine: info: check_action_definition: params:reload <parameters login="parallel" passwd="wireless" pcmk_host_list="orana" lanplus="1" action="off" delay="0" ipaddr="10.11.10.30"/> Jan 13 19:33:04 [4295] orana pengine: info: check_action_definition: Parameters to fence-uc-orana_start_0 on orana changed: was 363cbd5d9ec840db46e5e25e2f8b6652 vs. now 44b3f881263f198e021240ff36365f0f (reload:3.0.9) 0:0;14:2303:0:70de24fe-0f17-47ec-a193-a2cdd9f9e5c7 Jan 13 19:33:04 [4295] orana pengine: info: check_action_definition: params:reload <parameters login="parallel" passwd="wireless" pcmk_host_list="orana" lanplus="1" action="off" delay="0" ipaddr="10.11.10.30" CRM_meta_timeout="20000"/> Jan 13 19:33:04 [4295] orana pengine: info: check_action_definition: Parameters to fence-uc-orana_monitor_5000 on orana changed: was 8024392825e4a1e81b7f628e6dee69a6 vs. now ec6a353c4e45a74e3497360e0581b40c (reload:3.0.9) 0:0;12:2303:0:70de24fe-0f17-47ec-a193-a2cdd9f9e5c7 Jan 13 19:33:04 [4295] orana pengine: notice: DeleteRsc: Removing CWS-FLT from orana Jan 13 19:33:04 [4295] orana pengine: notice: DeleteRsc: Removing CWS-FLT2 from orana Jan 13 19:33:04 [4295] orana pengine: notice: DeleteRsc: Removing MME-FLT2 from orana Jan 13 19:33:04 [4295] orana pengine: info: master_color: Promoting foo:0 (Master orana) Jan 13 19:33:04 [4295] orana pengine: info: master_color: foo-master: Promoted 1 instances of a possible 1 to master Jan 13 19:33:04 [4295] orana pengine: info: native_color: Stopping orphan resource MGMT-FLT Jan 13 19:33:04 [4295] orana pengine: info: native_color: Stopping orphan resource SGW-FLT2 Jan 13 19:33:04 [4295] orana pengine: info: native_color: Stopping orphan resource EPC-3G Jan 13 19:33:04 [4295] orana pengine: info: native_color: Stopping orphan resource CWS-3G Jan 13 19:33:04 [4295] orana pengine: info: native_color: Stopping orphan resource SGW-FLT Jan 13 19:33:04 [4295] orana pengine: info: native_color: Stopping orphan resource MME-FLT Jan 13 19:33:04 [4295] orana pengine: info: native_color: Stopping orphan resource CWS-FLT Jan 13 19:33:04 [4295] orana pengine: info: native_color: Stopping orphan resource CWS-FLT2 Jan 13 19:33:04 [4295] orana pengine: info: native_color: Stopping orphan resource MME-FLT2 Jan 13 19:33:04 [4295] orana pengine: info: RecurringOp: Start recurring monitor (11s) for foo:1 on kamet Jan 13 19:33:04 [4295] orana pengine: info: RecurringOp: Start recurring monitor (11s) for foo:1 on kamet Jan 13 19:33:04 [4295] orana pengine: info: RecurringOp: Start recurring monitor (5s) for fence-uc-orana on orana Jan 13 19:33:04 [4295] orana pengine: info: RecurringOp: Start recurring monitor (5s) for fence-uc-kamet on kamet Jan 13 19:33:04 [4295] orana pengine: info: LogActions: Leave foo:0 (Master orana) Jan 13 19:33:04 [4295] orana pengine: notice: LogActions: Start foo:1 (kamet) Jan 13 19:33:04 [4295] orana pengine: notice: LogActions: Restart fence-uc-orana (Started orana) Jan 13 19:33:04 [4295] orana pengine: notice: LogActions: Move fence-uc-kamet (Started orana -> kamet) Jan 13 19:33:04 [4295] orana pengine: notice: LogActions: Stop MGMT-FLT (orana) Jan 13 19:33:04 [4295] orana pengine: notice: LogActions: Stop SGW-FLT2 (orana) Jan 13 19:33:04 [4295] orana pengine: notice: LogActions: Stop EPC-3G (orana) Jan 13 19:33:04 [4295] orana pengine: notice: LogActions: Stop CWS-3G (orana) Jan 13 19:33:04 [4295] orana pengine: notice: LogActions: Stop SGW-FLT (orana) Jan 13 19:33:04 [4295] orana pengine: notice: LogActions: Stop MME-FLT (orana) Jan 13 19:33:04 [4295] orana pengine: notice: LogActions: Stop CWS-FLT (orana) Jan 13 19:33:04 [4295] orana pengine: notice: LogActions: Stop CWS-FLT2 (orana) Jan 13 19:33:04 [4295] orana pengine: notice: LogActions: Stop MME-FLT2 (orana) Jan 13 19:33:04 [4295] orana pengine: notice: process_pe_message: Calculated Transition 2317: /var/lib/pacemaker/pengine/pe-input-1452.bz2 Jan 13 19:33:04 [4296] orana crmd: info: handle_response: pe_calc calculation pe_calc-dc-1452731584-2564 is obsolete Jan 13 19:33:04 [4295] orana pengine: notice: unpack_config: On loss of CCM Quorum: Ignore Jan 13 19:33:04 [4295] orana pengine: info: determine_online_status_fencing: Node kamet is active Jan 13 19:33:04 [4295] orana pengine: info: determine_online_status: Node kamet is online Jan 13 19:33:04 [4295] orana pengine: info: determine_online_status_fencing: Node orana is active Jan 13 19:33:04 [4295] orana pengine: info: determine_online_status: Node orana is online Jan 13 19:33:04 [4295] orana pengine: warning: process_rsc_state: Detected active orphan MGMT-FLT running on orana Jan 13 19:33:04 [4295] orana pengine: warning: process_rsc_state: Detected active orphan SGW-FLT2 running on orana Jan 13 19:33:04 [4295] orana pengine: warning: process_rsc_state: Detected active orphan EPC-3G running on orana Jan 13 19:33:04 [4295] orana pengine: warning: process_rsc_state: Detected active orphan CWS-3G running on orana Jan 13 19:33:04 [4295] orana pengine: warning: process_rsc_state: Detected active orphan SGW-FLT running on orana Jan 13 19:33:04 [4295] orana pengine: warning: process_rsc_state: Detected active orphan MME-FLT running on orana Jan 13 19:33:04 [4295] orana pengine: warning: process_rsc_state: Detected active orphan CWS-FLT running on orana Jan 13 19:33:04 [4295] orana pengine: warning: process_rsc_state: Detected active orphan CWS-FLT2 running on orana Jan 13 19:33:04 [4295] orana pengine: warning: process_rsc_state: Detected active orphan MME-FLT2 running on orana Jan 13 19:33:04 [4295] orana pengine: info: clone_print: Master/Slave Set: foo-master [foo] Jan 13 19:33:04 [4295] orana pengine: info: short_print: Masters: [ orana ] Jan 13 19:33:04 [4295] orana pengine: info: short_print: Stopped: [ kamet ] Jan 13 19:33:04 [4295] orana pengine: info: native_print: fence-uc-orana (stonith:fence_ilo4): Started orana Jan 13 19:33:04 [4295] orana pengine: info: native_print: fence-uc-kamet (stonith:fence_ilo4): Started orana Jan 13 19:33:04 [4295] orana pengine: info: native_print: MGMT-FLT (ocf::pw:IPaddr): ORPHANED Started orana Jan 13 19:33:04 [4295] orana pengine: info: native_print: SGW-FLT2 (ocf::pw:IPaddr): ORPHANED Started orana Jan 13 19:33:04 [4295] orana pengine: info: native_print: EPC-3G (ocf::pw:IPaddr): ORPHANED Started orana Jan 13 19:33:04 [4295] orana pengine: info: native_print: CWS-3G (ocf::pw:IPaddr): ORPHANED Started orana Jan 13 19:33:04 [4295] orana pengine: info: native_print: SGW-FLT (ocf::pw:IPaddr): ORPHANED Started orana Jan 13 19:33:04 [4295] orana pengine: info: native_print: MME-FLT (ocf::pw:IPaddr): ORPHANED Started orana Jan 13 19:33:04 [4295] orana pengine: info: native_print: CWS-FLT (ocf::pw:IPaddr): ORPHANED Started orana Jan 13 19:33:04 [4295] orana pengine: info: native_print: CWS-FLT2 (ocf::pw:IPaddr): ORPHANED Started orana Jan 13 19:33:04 [4295] orana pengine: info: native_print: MME-FLT2 (ocf::pw:IPaddr): ORPHANED Started orana Jan 13 19:33:04 [4295] orana pengine: notice: DeleteRsc: Removing MGMT-FLT from orana Jan 13 19:33:04 [4295] orana pengine: notice: DeleteRsc: Removing SGW-FLT2 from orana Jan 13 19:33:04 [4295] orana pengine: notice: DeleteRsc: Removing EPC-3G from orana Jan 13 19:33:04 [4295] orana pengine: notice: DeleteRsc: Removing CWS-3G from orana Jan 13 19:33:04 [4295] orana pengine: info: check_action_definition: params:reload <parameters login="parallel" passwd="wireless" pcmk_host_list="kamet" lanplus="1" action="off" delay="10" ipaddr="10.11.10.21"/> Jan 13 19:33:04 [4295] orana pengine: info: check_action_definition: Parameters to fence-uc-kamet_start_0 on orana changed: was 93013a04ac64e00066a4f7b080b31515 vs. now a204a24f6a125ccea22f6dc681d020f0 (reload:3.0.9) 0:0;13:2304:0:70de24fe-0f17-47ec-a193-a2cdd9f9e5c7 Jan 13 19:33:04 [4295] orana pengine: info: check_action_definition: params:reload <parameters login="parallel" passwd="wireless" pcmk_host_list="kamet" lanplus="1" action="off" delay="10" ipaddr="10.11.10.21" CRM_meta_timeout="20000"/> Jan 13 19:33:04 [4295] orana pengine: info: check_action_definition: Parameters to fence-uc-kamet_monitor_5000 on orana changed: was e884a8e46b98e8563278b8117eb4db82 vs. now 948f08ea79b4152d6055da386ebacbb3 (reload:3.0.9) 0:0;1:2304:0:70de24fe-0f17-47ec-a193-a2cdd9f9e5c7 Jan 13 19:33:04 [4295] orana pengine: notice: DeleteRsc: Removing SGW-FLT from orana Jan 13 19:33:04 [4295] orana pengine: notice: DeleteRsc: Removing MME-FLT from orana Jan 13 19:33:04 [4295] orana pengine: info: check_action_definition: params:reload <parameters login="parallel" passwd="wireless" pcmk_host_list="orana" lanplus="1" action="off" delay="0" ipaddr="10.11.10.30"/> Jan 13 19:33:04 [4295] orana pengine: info: check_action_definition: Parameters to fence-uc-orana_start_0 on orana changed: was 363cbd5d9ec840db46e5e25e2f8b6652 vs. now 44b3f881263f198e021240ff36365f0f (reload:3.0.9) 0:0;14:2303:0:70de24fe-0f17-47ec-a193-a2cdd9f9e5c7 Jan 13 19:33:04 [4295] orana pengine: info: check_action_definition: params:reload <parameters login="parallel" passwd="wireless" pcmk_host_list="orana" lanplus="1" action="off" delay="0" ipaddr="10.11.10.30" CRM_meta_timeout="20000"/> Jan 13 19:33:04 [4295] orana pengine: info: check_action_definition: Parameters to fence-uc-orana_monitor_5000 on orana changed: was 8024392825e4a1e81b7f628e6dee69a6 vs. now ec6a353c4e45a74e3497360e0581b40c (reload:3.0.9) 0:0;12:2303:0:70de24fe-0f17-47ec-a193-a2cdd9f9e5c7 Jan 13 19:33:04 [4295] orana pengine: notice: DeleteRsc: Removing CWS-FLT from orana Jan 13 19:33:04 [4295] orana pengine: notice: DeleteRsc: Removing CWS-FLT2 from orana Jan 13 19:33:04 [4295] orana pengine: notice: DeleteRsc: Removing MME-FLT2 from orana Jan 13 19:33:04 [4295] orana pengine: info: master_color: Promoting foo:0 (Master orana) Jan 13 19:33:04 [4295] orana pengine: info: master_color: foo-master: Promoted 1 instances of a possible 1 to master Jan 13 19:33:04 [4295] orana pengine: info: native_color: Stopping orphan resource MGMT-FLT Jan 13 19:33:04 [4295] orana pengine: info: native_color: Stopping orphan resource SGW-FLT2 Jan 13 19:33:04 [4295] orana pengine: info: native_color: Stopping orphan resource EPC-3G Jan 13 19:33:04 [4295] orana pengine: info: native_color: Stopping orphan resource CWS-3G Jan 13 19:33:04 [4295] orana pengine: info: native_color: Stopping orphan resource SGW-FLT Jan 13 19:33:04 [4295] orana pengine: info: native_color: Stopping orphan resource MME-FLT Jan 13 19:33:04 [4295] orana pengine: info: native_color: Stopping orphan resource CWS-FLT Jan 13 19:33:04 [4295] orana pengine: info: native_color: Stopping orphan resource CWS-FLT2 Jan 13 19:33:04 [4295] orana pengine: info: native_color: Stopping orphan resource MME-FLT2 Jan 13 19:33:04 [4295] orana pengine: info: RecurringOp: Start recurring monitor (11s) for foo:1 on kamet Jan 13 19:33:04 [4295] orana pengine: info: RecurringOp: Start recurring monitor (11s) for foo:1 on kamet Jan 13 19:33:04 [4295] orana pengine: info: RecurringOp: Start recurring monitor (5s) for fence-uc-orana on orana Jan 13 19:33:04 [4295] orana pengine: info: RecurringOp: Start recurring monitor (5s) for fence-uc-kamet on kamet Jan 13 19:33:04 [4295] orana pengine: info: LogActions: Leave foo:0 (Master orana) Jan 13 19:33:04 [4295] orana pengine: notice: LogActions: Start foo:1 (kamet) Jan 13 19:33:04 [4295] orana pengine: notice: LogActions: Restart fence-uc-orana (Started orana) Jan 13 19:33:04 [4295] orana pengine: notice: LogActions: Move fence-uc-kamet (Started orana -> kamet) Jan 13 19:33:04 [4295] orana pengine: notice: LogActions: Stop MGMT-FLT (orana) Jan 13 19:33:04 [4295] orana pengine: notice: LogActions: Stop SGW-FLT2 (orana) Jan 13 19:33:04 [4295] orana pengine: notice: LogActions: Stop EPC-3G (orana) Jan 13 19:33:04 [4295] orana pengine: notice: LogActions: Stop CWS-3G (orana) Jan 13 19:33:04 [4295] orana pengine: notice: LogActions: Stop SGW-FLT (orana) Jan 13 19:33:04 [4295] orana pengine: notice: LogActions: Stop MME-FLT (orana) Jan 13 19:33:04 [4295] orana pengine: notice: LogActions: Stop CWS-FLT (orana) Jan 13 19:33:04 [4295] orana pengine: notice: LogActions: Stop CWS-FLT2 (orana) Jan 13 19:33:04 [4295] orana pengine: notice: LogActions: Stop MME-FLT2 (orana)Thanks in advance. Regards Arjun _______________________________________________ Users mailing list: Users@clusterlabs.org http://clusterlabs.org/mailman/listinfo/users Project Home: http://www.clusterlabs.org Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf Bugs: http://bugs.clusterlabs.org