Re: [Pacemaker] Cannot create more than 27 multistate resources
On 9 Jul 2014, at 6:49 pm, K Mehta wrote: > Hi, > > [root@vsanqa11 ~]# rpm -qa | grep pcs ; rpm -qa | grep pace ; rpm -qa | grep > libqb; rpm -qa | grep coro; rpm -qa | grep cman > pcs-0.9.90-2.el6.centos.2.noarch > pacemaker-cli-1.1.10-14.el6_5.3.x86_64 > pacemaker-libs-1.1.10-14.el6_5.3.x86_64 > pacemaker-1.1.10-14.el6_5.3.x86_64 > pacemaker-cluster-libs-1.1.10-14.el6_5.3.x86_64 > libqb-devel-0.16.0-2.el6.x86_64 > libqb-0.16.0-2.el6.x86_64 > corosynclib-1.4.1-17.el6_5.1.x86_64 > corosync-1.4.1-17.el6_5.1.x86_64 > cman-3.0.12.1-59.el6_5.2.x86_64 > [root@vsanqa11 ~]# uname -a > Linux vsanqa11 2.6.32-279.el6.x86_64 #1 SMP Fri Jun 22 12:19:21 UTC 2012 > x86_64 x86_64 x86_64 GNU/Linux > [root@vsanqa11 ~]# cat /etc/redhat-release > CentOS release 6.3 (Final) > > > Created 27 resources > > [root@vsanqa11 ~]# pcs status > Cluster name: vsanqa11_12 > Last updated: Wed Jul 9 01:30:05 2014 > Last change: Wed Jul 9 01:24:12 2014 via cibadmin on vsanqa11 > Stack: cman > Current DC: vsanqa11 - partition with quorum > Version: 1.1.10-14.el6_5.3-368c726 > 2 Nodes configured > 54 Resources configured > > > Online: [ vsanqa11 vsanqa12 ] > > Full list of resources: > > Master/Slave Set: ms-0feba438-0f16-4de5-bf18-9d576cf4dd26 > [vha-0feba438-0f16-4de5-bf18-9d576cf4dd26] > Masters: [ vsanqa12 ] > Slaves: [ vsanqa11 ] > Master/Slave Set: ms-329ad1bd-2f9c-483d-a052-270731aefd70 > [vha-329ad1bd-2f9c-483d-a052-270731aefd70] > Masters: [ vsanqa12 ] > Slaves: [ vsanqa11 ] > Master/Slave Set: ms-b5b9e8dc-87c9-4229-b425-870d1bc1f107 > [vha-b5b9e8dc-87c9-4229-b425-870d1bc1f107] > Masters: [ vsanqa12 ] > Slaves: [ vsanqa11 ] > Master/Slave Set: ms-56112cc8-4c56-4454-84ea-ba9b7b82dde7 > [vha-56112cc8-4c56-4454-84ea-ba9b7b82dde7] > Masters: [ vsanqa12 ] > Slaves: [ vsanqa11 ] > Master/Slave Set: ms-20fd5ed6-8e72-4a6b-9b1a-73bd96e1f253 > [vha-20fd5ed6-8e72-4a6b-9b1a-73bd96e1f253] > Masters: [ vsanqa12 ] > Slaves: [ vsanqa11 ] > Master/Slave Set: ms-77af4d7e-d78c-4799-a24a-7536d225 > [vha-77af4d7e-d78c-4799-a24a-7536d225] > Masters: [ vsanqa12 ] > Slaves: [ vsanqa11 ] > Master/Slave Set: ms-913f6e7e-932f-4c3f-9e7f-a70c4153b4c7 > [vha-913f6e7e-932f-4c3f-9e7f-a70c4153b4c7] > Masters: [ vsanqa12 ] > Slaves: [ vsanqa11 ] > Master/Slave Set: ms-3d3ae48c-1955-4c64-b951-f2a2621a70b7 > [vha-3d3ae48c-1955-4c64-b951-f2a2621a70b7] > Masters: [ vsanqa12 ] > Slaves: [ vsanqa11 ] > Master/Slave Set: ms-a8101da1-636f-483b-90a9-9a18fd5a5793 > [vha-a8101da1-636f-483b-90a9-9a18fd5a5793] > Masters: [ vsanqa12 ] > Slaves: [ vsanqa11 ] > Master/Slave Set: ms-290e0fdd-19c6-4b40-8046-0fa2c94a7320 > [vha-290e0fdd-19c6-4b40-8046-0fa2c94a7320] > Masters: [ vsanqa12 ] > Slaves: [ vsanqa11 ] > Master/Slave Set: ms-8341368b-6b6a-4bf0-bb0a-38b32ffec2f4 > [vha-8341368b-6b6a-4bf0-bb0a-38b32ffec2f4] > Masters: [ vsanqa12 ] > Slaves: [ vsanqa11 ] > Master/Slave Set: ms-6695f479-0d21-4388-9540-a440c32e0944 > [vha-6695f479-0d21-4388-9540-a440c32e0944] > Masters: [ vsanqa12 ] > Slaves: [ vsanqa11 ] > Master/Slave Set: ms-85fe517e-9a2b-4ac1-b6e1-e4da57b49969 > [vha-85fe517e-9a2b-4ac1-b6e1-e4da57b49969] > Masters: [ vsanqa12 ] > Slaves: [ vsanqa11 ] > Master/Slave Set: ms-5b9adf5c-3bb5-4353-8c5f-a3e1508f7c4a > [vha-5b9adf5c-3bb5-4353-8c5f-a3e1508f7c4a] > Masters: [ vsanqa12 ] > Slaves: [ vsanqa11 ] > Master/Slave Set: ms-57b4b7a1-d3bb-4b94-b8dc-8a84e4a3de03 > [vha-57b4b7a1-d3bb-4b94-b8dc-8a84e4a3de03] > Masters: [ vsanqa12 ] > Slaves: [ vsanqa11 ] > Master/Slave Set: ms-b4fbac14-ef19-4861-93e5-14574e101484 > [vha-b4fbac14-ef19-4861-93e5-14574e101484] > Masters: [ vsanqa12 ] > Slaves: [ vsanqa11 ] > Master/Slave Set: ms-ec4836d3-b768-4abd-83bf-3a61143346ce > [vha-ec4836d3-b768-4abd-83bf-3a61143346ce] > Masters: [ vsanqa12 ] > Slaves: [ vsanqa11 ] > Master/Slave Set: ms-28d6db1a-bdba-4fc0-b74c-ef74548cf714 > [vha-28d6db1a-bdba-4fc0-b74c-ef74548cf714] > Masters: [ vsanqa12 ] > Slaves: [ vsanqa11 ] > Master/Slave Set: ms-c6cfd538-17f4-4c4b-b259-731e2cac75f3 > [vha-c6cfd538-17f4-4c4b-b259-731e2cac75f3] > Masters: [ vsanqa12 ] > Slaves: [ vsanqa11 ] > Master/Slave Set: ms-06d19988-7549-4882-b780-fd598714ec7f > [vha-06d19988-7549-4882-b780-fd598714ec7f] > Masters: [ vsanqa12 ] > Slaves: [ vsanqa11 ] > Master/Slave Set: ms-ec105b52-42b1-42f5-931f-249fbc2f16c4 > [vha-ec105b52-42b1-42f5-931f-249fbc2f16c4] > Masters: [ vsanqa12 ] > Slaves: [ vsanqa11 ] > Master/Slave Set: ms-6cc9a97b-9714-4cab-bf52-103c46b8593f > [vha-6cc9a97b-9714-4cab-bf52-103c46b8593f] > Masters: [ vsanqa12 ] > Slaves: [ vsanqa11 ] > Master/Slave Set: ms-b80b4a69-67e2-43bd-8eba-f6c18f78d706 > [vha-b80b4a69-67e2-43bd-8eba-f6c18f78d706] > Masters: [ vsanqa12 ] > Slaves: [ vsanqa11 ] > Master/Slave Set: ms-24817d3c-98e0-4408
Re: [Pacemaker] Cannot create more than 27 multistate resources
Didnt see any buffer size suggesstion in syslog Changed buffer size from 20k to 200k and rebooted both systems Did the following after reboot: [root@vsanqa11 kiran]# cat /etc/sysconfig/pacemaker | grep PCMK_ipc # PCMK_ipc_type=shared-mem|socket|posix|sysv export PCMK_ipc_buffer=204800 [root@vsanqa11 kiran]# echo $PCMK_ipc_buffer 204800 [root@vsanqa12 ~]# cat /etc/sysconfig/pacemaker | grep PCMK_ipc # PCMK_ipc_type=shared-mem|socket|posix|sysv export PCMK_ipc_buffer=204800 [root@vsanqa12 ~]# echo $PCMK_ipc_buffer 204800 Same error seen on creation of master Jul 9 23:20:11 vsanqa11 cibadmin[9347]: notice: crm_log_args: Invoked: /usr/sbin/cibadmin -c -R --xml-text #012 #012#012#012#012#012 #012#012#012#012#012 interval=30s timeout=100s op monitor role="Master" interval=30s timeout=100s > > > > > > pcs status output includes > > vha-3de5ab16-9917-4b90-93d2-7b04fc71879c > (ocf::heartbeat:vgc-cm-agent.ocf): Started vsanqa11 > > > > > > [root@vsanqa11 ~]# pcs resource master > ms-3de5ab16-9917-4b90-93d2-7b04fc71879c > vha-3de5ab16-9917-4b90-93d2-7b04fc71879c meta clone-max=2 > globally-unique=false target-role=started > > Error: unable to locate command: /usr/sbin/cibadmin > > An odd error, but I'm pretty sure you're hitting IPC limits. > Check out /etc/sysconfig/pacemaker and look for 'ipc' (then make the > number much bigger :) > The logs will probably indicate a good value to choose. > > Two extra notes > - since you're not using systemd, prefix the line with 'export ' > - you'll need to set the same values in your shell environment > > > > > > > > Deleted one of the resources that was created successfully and reran > above command to create master resource > ms-3de5ab16-9917-4b90-93d2-7b04fc71879c and it worked. > > > > Tried this multiple times and each time saw the same behavior > > > > Attached is the debug output for the above command. This output contains > string "unable to locate command: /usr/sbin/cibadmin" > > > > Regards, > > kiran > > > > > > ___ > > Pacemaker mailing list: Pacemaker@oss.clusterlabs.org > > http://oss.clusterlabs.org/mailman/listinfo/pacemaker > > > > Project Home: http://www.clusterlabs.org > > Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf > > Bugs: http://bugs.clusterlabs.org > > > ___ > Pacemaker mailing list: Pacemaker@oss.clusterlabs.org > http://oss.clusterlabs.org/mailman/listinfo/pacemaker > > Project Home: http://www.clusterlabs.org > Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf > Bugs: http://bugs.clusterlabs.org > > ___ Pacemaker mailing list: Pacemaker@oss.clusterlabs.org http://oss.clusterlabs.org/mailman/listinfo/pacemaker Project Home: http://www.clusterlabs.org Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf Bugs: http://bugs.clusterlabs.org
Re: [Pacemaker] Cannot create more than 27 multistate resources
Can you run crm_report for the period covered by your test and attach the result please? On 10 Jul 2014, at 4:32 pm, K Mehta wrote: > Didnt see any buffer size suggesstion in syslog > Changed buffer size from 20k to 200k and rebooted both systems > > Did the following after reboot: > > [root@vsanqa11 kiran]# cat /etc/sysconfig/pacemaker | grep PCMK_ipc > # PCMK_ipc_type=shared-mem|socket|posix|sysv > export PCMK_ipc_buffer=204800 > [root@vsanqa11 kiran]# echo $PCMK_ipc_buffer > 204800 > > [root@vsanqa12 ~]# cat /etc/sysconfig/pacemaker | grep PCMK_ipc > # PCMK_ipc_type=shared-mem|socket|posix|sysv > export PCMK_ipc_buffer=204800 > [root@vsanqa12 ~]# echo $PCMK_ipc_buffer > 204800 > > > > Same error seen on creation of master > Jul 9 23:20:11 vsanqa11 cibadmin[9347]: notice: crm_log_args: Invoked: > /usr/sbin/cibadmin -c -R --xml-text #012 id="cib-bootstrap-options">#012 id="cib-bootstrap-options-dc-version" name="dc-version" > value="1.1.10-14.el6_5.3-368c726"/>#012 id="cib-bootstrap-options-cluster-infrastructure" > name="cluster-infrastructure" value="cman"/>#012#012 id="cib-bootstrap-options-no-quorum-policy" name="no-quorum-policy" > value="ignore"/>#012 name="stoni > Jul 9 23:20:12 vsanqa11 cibadmin[9352]: notice: crm_log_args: Invoked: > /usr/sbin/cibadmin -c -R --xml-text #012 id="cib-bootstrap-options">#012 id="cib-bootstrap-options-dc-version" name="dc-version" > value="1.1.10-14.el6_5.3-368c726"/>#012 id="cib-bootstrap-options-cluster-infrastructure" > name="cluster-infrastructure" value="cman"/>#012#012 id="cib-bootstrap-options-stonith-enabled" name="stonith-enabled" > value="false"/>#012 name="no-quor > Jul 9 23:20:12 vsanqa11 cibadmin[9466]: notice: crm_log_args: Invoked: > /usr/sbin/cibadmin -o resources -C -X id="vha-256ac0ea-c7bd-4c86-89e3-8b934c6b7281" provider="heartbeat" > type="vgc-cm-agent.ocf"> id="vha-256ac0ea-c7bd-4c86-89e3-8b934c6b7281-instance_attributes"> id="vha-256ac0ea-c7bd-4c86-89e3-8b934c6b7281-instance_attributes-cluster_uuid" > name="cluster_uuid" > value="256ac0ea-c7bd-4c86-89e3-8b934c6b7281"/> id="vha-256ac0ea-c7bd-4c86-89e3-8b934c6b7281-mon > Error: unable to locate command: /usr/sbin/cibadmin > > > > > > On Thu, Jul 10, 2014 at 3:40 AM, Andrew Beekhof wrote: > > On 9 Jul 2014, at 6:49 pm, K Mehta wrote: > > > Hi, > > > > [root@vsanqa11 ~]# rpm -qa | grep pcs ; rpm -qa | grep pace ; rpm -qa | > > grep libqb; rpm -qa | grep coro; rpm -qa | grep cman > > pcs-0.9.90-2.el6.centos.2.noarch > > pacemaker-cli-1.1.10-14.el6_5.3.x86_64 > > pacemaker-libs-1.1.10-14.el6_5.3.x86_64 > > pacemaker-1.1.10-14.el6_5.3.x86_64 > > pacemaker-cluster-libs-1.1.10-14.el6_5.3.x86_64 > > libqb-devel-0.16.0-2.el6.x86_64 > > libqb-0.16.0-2.el6.x86_64 > > corosynclib-1.4.1-17.el6_5.1.x86_64 > > corosync-1.4.1-17.el6_5.1.x86_64 > > cman-3.0.12.1-59.el6_5.2.x86_64 > > [root@vsanqa11 ~]# uname -a > > Linux vsanqa11 2.6.32-279.el6.x86_64 #1 SMP Fri Jun 22 12:19:21 UTC 2012 > > x86_64 x86_64 x86_64 GNU/Linux > > [root@vsanqa11 ~]# cat /etc/redhat-release > > CentOS release 6.3 (Final) > > > > > > Created 27 resources > > > > [root@vsanqa11 ~]# pcs status > > Cluster name: vsanqa11_12 > > Last updated: Wed Jul 9 01:30:05 2014 > > Last change: Wed Jul 9 01:24:12 2014 via cibadmin on vsanqa11 > > Stack: cman > > Current DC: vsanqa11 - partition with quorum > > Version: 1.1.10-14.el6_5.3-368c726 > > 2 Nodes configured > > 54 Resources configured > > > > > > Online: [ vsanqa11 vsanqa12 ] > > > > Full list of resources: > > > > Master/Slave Set: ms-0feba438-0f16-4de5-bf18-9d576cf4dd26 > > [vha-0feba438-0f16-4de5-bf18-9d576cf4dd26] > > Masters: [ vsanqa12 ] > > Slaves: [ vsanqa11 ] > > Master/Slave Set: ms-329ad1bd-2f9c-483d-a052-270731aefd70 > > [vha-329ad1bd-2f9c-483d-a052-270731aefd70] > > Masters: [ vsanqa12 ] > > Slaves: [ vsanqa11 ] > > Master/Slave Set: ms-b5b9e8dc-87c9-4229-b425-870d1bc1f107 > > [vha-b5b9e8dc-87c9-4229-b425-870d1bc1f107] > > Masters: [ vsanqa12 ] > > Slaves: [ vsanqa11 ] > > Master/Slave Set: ms-56112cc8-4c56-4454-84ea-ba9b7b82dde7 > > [vha-56112cc8-4c56-4454-84ea-ba9b7b82dde7] > > Masters: [ vsanqa12 ] > > Slaves: [ vsanqa11 ] > > Master/Slave Set: ms-20fd5ed6-8e72-4a6b-9b1a-73bd96e1f253 > > [vha-20fd5ed6-8e72-4a6b-9b1a-73bd96e1f253] > > Masters: [ vsanqa12 ] > > Slaves: [ vsanqa11 ] > > Master/Slave Set: ms-77af4d7e-d78c-4799-a24a-7536d225 > > [vha-77af4d7e-d78c-4799-a24a-7536d225] > > Masters: [ vsanqa12 ] > > Slaves: [ vsanqa11 ] > > Master/Slave Set: ms-913f6e7e-932f-4c3f-9e7f-a70c4153b4c7 > > [vha-913f6e7e-932f-4c3f-9e7f-a70c4153b4c7] > > Masters: [ vsanqa12 ] > > Slaves: [ vsanqa11 ] > > Master/Slave Set: ms-3d3ae48c-1955-4c64-b951-f2a2621a70b7 > > [vha-3d3ae48c-1955-4c64-b951-f2a2621a70b7] > > Masters: [ vsanqa12 ] > > Slaves: [ vsanqa11 ] > > Master/Slave Set: ms-a8101da1-636f-483b-90a
Re: [Pacemaker] Cannot create more than 27 multistate resources
Any update on this ? On Sat, Jul 12, 2014 at 11:55 PM, K Mehta wrote: > Andrew, > Attached is the report. > > Regards, > Kiran > > > On Fri, Jul 11, 2014 at 4:26 AM, Andrew Beekhof > wrote: > >> Can you run crm_report for the period covered by your test and attach the >> result please? >> >> On 10 Jul 2014, at 4:32 pm, K Mehta wrote: >> >> > Didnt see any buffer size suggesstion in syslog >> > Changed buffer size from 20k to 200k and rebooted both systems >> > >> > Did the following after reboot: >> > >> > [root@vsanqa11 kiran]# cat /etc/sysconfig/pacemaker | grep PCMK_ipc >> > # PCMK_ipc_type=shared-mem|socket|posix|sysv >> > export PCMK_ipc_buffer=204800 >> > [root@vsanqa11 kiran]# echo $PCMK_ipc_buffer >> > 204800 >> > >> > [root@vsanqa12 ~]# cat /etc/sysconfig/pacemaker | grep PCMK_ipc >> > # PCMK_ipc_type=shared-mem|socket|posix|sysv >> > export PCMK_ipc_buffer=204800 >> > [root@vsanqa12 ~]# echo $PCMK_ipc_buffer >> > 204800 >> > >> > >> > >> > Same error seen on creation of master >> > Jul 9 23:20:11 vsanqa11 cibadmin[9347]: notice: crm_log_args: >> Invoked: /usr/sbin/cibadmin -c -R --xml-text #012 >> #012> id="cib-bootstrap-options-dc-version" name="dc-version" >> value="1.1.10-14.el6_5.3-368c726"/>#012> id="cib-bootstrap-options-cluster-infrastructure" >> name="cluster-infrastructure" value="cman"/>#012#012> id="cib-bootstrap-options-no-quorum-policy" name="no-quorum-policy" >> value="ignore"/>#012 > name="stoni >> > Jul 9 23:20:12 vsanqa11 cibadmin[9352]: notice: crm_log_args: >> Invoked: /usr/sbin/cibadmin -c -R --xml-text #012 >> #012> id="cib-bootstrap-options-dc-version" name="dc-version" >> value="1.1.10-14.el6_5.3-368c726"/>#012> id="cib-bootstrap-options-cluster-infrastructure" >> name="cluster-infrastructure" value="cman"/>#012#012> id="cib-bootstrap-options-stonith-enabled" name="stonith-enabled" >> value="false"/>#012 > name="no-quor >> > Jul 9 23:20:12 vsanqa11 cibadmin[9466]: notice: crm_log_args: >> Invoked: /usr/sbin/cibadmin -o resources -C -X > id="vha-256ac0ea-c7bd-4c86-89e3-8b934c6b7281" provider="heartbeat" >> type="vgc-cm-agent.ocf">> id="vha-256ac0ea-c7bd-4c86-89e3-8b934c6b7281-instance_attributes">> id="vha-256ac0ea-c7bd-4c86-89e3-8b934c6b7281-instance_attributes-cluster_uuid" >> name="cluster_uuid" >> value="256ac0ea-c7bd-4c86-89e3-8b934c6b7281"/>> id="vha-256ac0ea-c7bd-4c86-89e3-8b934c6b7281-mon >> > Error: unable to locate command: /usr/sbin/cibadmin >> > >> > >> > >> > >> > >> > On Thu, Jul 10, 2014 at 3:40 AM, Andrew Beekhof >> wrote: >> > >> > On 9 Jul 2014, at 6:49 pm, K Mehta wrote: >> > >> > > Hi, >> > > >> > > [root@vsanqa11 ~]# rpm -qa | grep pcs ; rpm -qa | grep pace ; rpm >> -qa | grep libqb; rpm -qa | grep coro; rpm -qa | grep cman >> > > pcs-0.9.90-2.el6.centos.2.noarch >> > > pacemaker-cli-1.1.10-14.el6_5.3.x86_64 >> > > pacemaker-libs-1.1.10-14.el6_5.3.x86_64 >> > > pacemaker-1.1.10-14.el6_5.3.x86_64 >> > > pacemaker-cluster-libs-1.1.10-14.el6_5.3.x86_64 >> > > libqb-devel-0.16.0-2.el6.x86_64 >> > > libqb-0.16.0-2.el6.x86_64 >> > > corosynclib-1.4.1-17.el6_5.1.x86_64 >> > > corosync-1.4.1-17.el6_5.1.x86_64 >> > > cman-3.0.12.1-59.el6_5.2.x86_64 >> > > [root@vsanqa11 ~]# uname -a >> > > Linux vsanqa11 2.6.32-279.el6.x86_64 #1 SMP Fri Jun 22 12:19:21 UTC >> 2012 x86_64 x86_64 x86_64 GNU/Linux >> > > [root@vsanqa11 ~]# cat /etc/redhat-release >> > > CentOS release 6.3 (Final) >> > > >> > > >> > > Created 27 resources >> > > >> > > [root@vsanqa11 ~]# pcs status >> > > Cluster name: vsanqa11_12 >> > > Last updated: Wed Jul 9 01:30:05 2014 >> > > Last change: Wed Jul 9 01:24:12 2014 via cibadmin on vsanqa11 >> > > Stack: cman >> > > Current DC: vsanqa11 - partition with quorum >> > > Version: 1.1.10-14.el6_5.3-368c726 >> > > 2 Nodes configured >> > > 54 Resources configured >> > > >> > > >> > > Online: [ vsanqa11 vsanqa12 ] >> > > >> > > Full list of resources: >> > > >> > > Master/Slave Set: ms-0feba438-0f16-4de5-bf18-9d576cf4dd26 >> [vha-0feba438-0f16-4de5-bf18-9d576cf4dd26] >> > > Masters: [ vsanqa12 ] >> > > Slaves: [ vsanqa11 ] >> > > Master/Slave Set: ms-329ad1bd-2f9c-483d-a052-270731aefd70 >> [vha-329ad1bd-2f9c-483d-a052-270731aefd70] >> > > Masters: [ vsanqa12 ] >> > > Slaves: [ vsanqa11 ] >> > > Master/Slave Set: ms-b5b9e8dc-87c9-4229-b425-870d1bc1f107 >> [vha-b5b9e8dc-87c9-4229-b425-870d1bc1f107] >> > > Masters: [ vsanqa12 ] >> > > Slaves: [ vsanqa11 ] >> > > Master/Slave Set: ms-56112cc8-4c56-4454-84ea-ba9b7b82dde7 >> [vha-56112cc8-4c56-4454-84ea-ba9b7b82dde7] >> > > Masters: [ vsanqa12 ] >> > > Slaves: [ vsanqa11 ] >> > > Master/Slave Set: ms-20fd5ed6-8e72-4a6b-9b1a-73bd96e1f253 >> [vha-20fd5ed6-8e72-4a6b-9b1a-73bd96e1f253] >> > > Masters: [ vsanqa12 ] >> > > Slaves: [ vsanqa11 ] >> > > Master/Slave Set: ms-77af4d7e-d78c-4799-a24a-7536d225 >> [vha-77af4d7e-d78c-4799-a24a-7536d225] >> > > Masters: [ vsanqa12 ] >> > > Slave
Re: [Pacemaker] Cannot create more than 27 multistate resources
On 16 Jul 2014, at 12:53 pm, K Mehta wrote: > Any update on this ? I don't appear to have the email below. Can you resend the report to my personal address please? > > > On Sat, Jul 12, 2014 at 11:55 PM, K Mehta wrote: > Andrew, > Attached is the report. > > Regards, > Kiran > > > On Fri, Jul 11, 2014 at 4:26 AM, Andrew Beekhof wrote: > Can you run crm_report for the period covered by your test and attach the > result please? > > On 10 Jul 2014, at 4:32 pm, K Mehta wrote: > > > Didnt see any buffer size suggesstion in syslog > > Changed buffer size from 20k to 200k and rebooted both systems > > > > Did the following after reboot: > > > > [root@vsanqa11 kiran]# cat /etc/sysconfig/pacemaker | grep PCMK_ipc > > # PCMK_ipc_type=shared-mem|socket|posix|sysv > > export PCMK_ipc_buffer=204800 > > [root@vsanqa11 kiran]# echo $PCMK_ipc_buffer > > 204800 > > > > [root@vsanqa12 ~]# cat /etc/sysconfig/pacemaker | grep PCMK_ipc > > # PCMK_ipc_type=shared-mem|socket|posix|sysv > > export PCMK_ipc_buffer=204800 > > [root@vsanqa12 ~]# echo $PCMK_ipc_buffer > > 204800 > > > > > > > > Same error seen on creation of master > > Jul 9 23:20:11 vsanqa11 cibadmin[9347]: notice: crm_log_args: Invoked: > > /usr/sbin/cibadmin -c -R --xml-text #012 > id="cib-bootstrap-options">#012 > id="cib-bootstrap-options-dc-version" name="dc-version" > > value="1.1.10-14.el6_5.3-368c726"/>#012 > id="cib-bootstrap-options-cluster-infrastructure" > > name="cluster-infrastructure" value="cman"/>#012#012 > id="cib-bootstrap-options-no-quorum-policy" name="no-quorum-policy" > > value="ignore"/>#012 > name="stoni > > Jul 9 23:20:12 vsanqa11 cibadmin[9352]: notice: crm_log_args: Invoked: > > /usr/sbin/cibadmin -c -R --xml-text #012 > id="cib-bootstrap-options">#012 > id="cib-bootstrap-options-dc-version" name="dc-version" > > value="1.1.10-14.el6_5.3-368c726"/>#012 > id="cib-bootstrap-options-cluster-infrastructure" > > name="cluster-infrastructure" value="cman"/>#012#012 > id="cib-bootstrap-options-stonith-enabled" name="stonith-enabled" > > value="false"/>#012 > name="no-quor > > Jul 9 23:20:12 vsanqa11 cibadmin[9466]: notice: crm_log_args: Invoked: > > /usr/sbin/cibadmin -o resources -C -X > id="vha-256ac0ea-c7bd-4c86-89e3-8b934c6b7281" provider="heartbeat" > > type="vgc-cm-agent.ocf"> > id="vha-256ac0ea-c7bd-4c86-89e3-8b934c6b7281-instance_attributes"> > id="vha-256ac0ea-c7bd-4c86-89e3-8b934c6b7281-instance_attributes-cluster_uuid" > > name="cluster_uuid" > > value="256ac0ea-c7bd-4c86-89e3-8b934c6b7281"/> > id="vha-256ac0ea-c7bd-4c86-89e3-8b934c6b7281-mon > > Error: unable to locate command: /usr/sbin/cibadmin > > > > > > > > > > > > On Thu, Jul 10, 2014 at 3:40 AM, Andrew Beekhof wrote: > > > > On 9 Jul 2014, at 6:49 pm, K Mehta wrote: > > > > > Hi, > > > > > > [root@vsanqa11 ~]# rpm -qa | grep pcs ; rpm -qa | grep pace ; rpm -qa | > > > grep libqb; rpm -qa | grep coro; rpm -qa | grep cman > > > pcs-0.9.90-2.el6.centos.2.noarch > > > pacemaker-cli-1.1.10-14.el6_5.3.x86_64 > > > pacemaker-libs-1.1.10-14.el6_5.3.x86_64 > > > pacemaker-1.1.10-14.el6_5.3.x86_64 > > > pacemaker-cluster-libs-1.1.10-14.el6_5.3.x86_64 > > > libqb-devel-0.16.0-2.el6.x86_64 > > > libqb-0.16.0-2.el6.x86_64 > > > corosynclib-1.4.1-17.el6_5.1.x86_64 > > > corosync-1.4.1-17.el6_5.1.x86_64 > > > cman-3.0.12.1-59.el6_5.2.x86_64 > > > [root@vsanqa11 ~]# uname -a > > > Linux vsanqa11 2.6.32-279.el6.x86_64 #1 SMP Fri Jun 22 12:19:21 UTC 2012 > > > x86_64 x86_64 x86_64 GNU/Linux > > > [root@vsanqa11 ~]# cat /etc/redhat-release > > > CentOS release 6.3 (Final) > > > > > > > > > Created 27 resources > > > > > > [root@vsanqa11 ~]# pcs status > > > Cluster name: vsanqa11_12 > > > Last updated: Wed Jul 9 01:30:05 2014 > > > Last change: Wed Jul 9 01:24:12 2014 via cibadmin on vsanqa11 > > > Stack: cman > > > Current DC: vsanqa11 - partition with quorum > > > Version: 1.1.10-14.el6_5.3-368c726 > > > 2 Nodes configured > > > 54 Resources configured > > > > > > > > > Online: [ vsanqa11 vsanqa12 ] > > > > > > Full list of resources: > > > > > > Master/Slave Set: ms-0feba438-0f16-4de5-bf18-9d576cf4dd26 > > > [vha-0feba438-0f16-4de5-bf18-9d576cf4dd26] > > > Masters: [ vsanqa12 ] > > > Slaves: [ vsanqa11 ] > > > Master/Slave Set: ms-329ad1bd-2f9c-483d-a052-270731aefd70 > > > [vha-329ad1bd-2f9c-483d-a052-270731aefd70] > > > Masters: [ vsanqa12 ] > > > Slaves: [ vsanqa11 ] > > > Master/Slave Set: ms-b5b9e8dc-87c9-4229-b425-870d1bc1f107 > > > [vha-b5b9e8dc-87c9-4229-b425-870d1bc1f107] > > > Masters: [ vsanqa12 ] > > > Slaves: [ vsanqa11 ] > > > Master/Slave Set: ms-56112cc8-4c56-4454-84ea-ba9b7b82dde7 > > > [vha-56112cc8-4c56-4454-84ea-ba9b7b82dde7] > > > Masters: [ vsanqa12 ] > > > Slaves: [ vsanqa11 ] > > > Master/Slave Set: ms-20fd5ed6-8e72-4a6b-9b1a-73bd96e1f253 > > > [vha-20fd5ed6-8e72-4a6b-9b1a-73bd96e1f253] > > > Masters: [ vsanqa12 ] > > > S
Re: [Pacemaker] Cannot create more than 27 multistate resources
Chris, Does the error below mean anything to you? This seems to be happening once the CIB reaches a certain size, but is on the client side and possibly before the pacemaker tools are invoked. On 9 Jul 2014, at 6:49 pm, K Mehta wrote: > [root@vsanqa11 ~]# pcs resource create > vha-3de5ab16-9917-4b90-93d2-7b04fc71879c ocf:heartbeat:vgc-cm-agent.ocf > cluster_uuid=3de5ab16-9917-4b90-93d2-7b04fc71879c op monitor role="Master" > interval=30s timeout=100s op monitor role="Master" interval=30s timeout=100s > > > pcs status output includes > vha-3de5ab16-9917-4b90-93d2-7b04fc71879c > (ocf::heartbeat:vgc-cm-agent.ocf): Started vsanqa11 > > > [root@vsanqa11 ~]# pcs resource master > ms-3de5ab16-9917-4b90-93d2-7b04fc71879c > vha-3de5ab16-9917-4b90-93d2-7b04fc71879c meta clone-max=2 > globally-unique=false target-role=started > Error: unable to locate command: /usr/sbin/cibadmin > Looking in the logs, I see: Jul 12 11:18:24 vsanqa11 cibadmin[7966]: notice: crm_log_args: Invoked: /usr/sbin/cibadmin -c -R --xml-text #012 #012#012#012#012#012 signature.asc Description: Message signed with OpenPGP using GPGMail ___ Pacemaker mailing list: Pacemaker@oss.clusterlabs.org http://oss.clusterlabs.org/mailman/listinfo/pacemaker Project Home: http://www.clusterlabs.org Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf Bugs: http://bugs.clusterlabs.org
Re: [Pacemaker] Cannot create more than 27 multistate resources
On 07/21/2014 05:22 AM, Andrew Beekhof wrote: Chris, Does the error below mean anything to you? This seems to be happening once the CIB reaches a certain size, but is on the client side and possibly before the pacemaker tools are invoked. I grabbed your debug file and did some tests and it looks like the issue is caused by earlier version of pcs (0.9.90 is affected) which try to pass the entire cib on the command line to cibadmin. This has been fixed upstream (and should be built in the next release of RHEL/CentOS). As a workaround, you can use the upstream sources here: https://github.com/feist/pcs (just run pcs from the directory that is cloned). Thanks! Chris On 9 Jul 2014, at 6:49 pm, K Mehta wrote: [root@vsanqa11 ~]# pcs resource create vha-3de5ab16-9917-4b90-93d2-7b04fc71879c ocf:heartbeat:vgc-cm-agent.ocf cluster_uuid=3de5ab16-9917-4b90-93d2-7b04fc71879c op monitor role="Master" interval=30s timeout=100s op monitor role="Master" interval=30s timeout=100s pcs status output includes vha-3de5ab16-9917-4b90-93d2-7b04fc71879c (ocf::heartbeat:vgc-cm-agent.ocf): Started vsanqa11 [root@vsanqa11 ~]# pcs resource master ms-3de5ab16-9917-4b90-93d2-7b04fc71879c vha-3de5ab16-9917-4b90-93d2-7b04fc71879c meta clone-max=2 globally-unique=false target-role=started Error: unable to locate command: /usr/sbin/cibadmin Looking in the logs, I see: Jul 12 11:18:24 vsanqa11 cibadmin[7966]: notice: crm_log_args: Invoked: /usr/sbin/cibadmin -c -R --xml-text #012 #012#012#012#012#012 ___ Pacemaker mailing list: Pacemaker@oss.clusterlabs.org http://oss.clusterlabs.org/mailman/listinfo/pacemaker Project Home: http://www.clusterlabs.org Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf Bugs: http://bugs.clusterlabs.org
Re: [Pacemaker] Cannot create more than 27 multistate resources
I have run into this issue with greater than 55 resources when using pcs-0.9.90-2.el6.centos.2.noarch. The other workaround is to run the pcs command with --debug. Take the whole XML text that pcs is trying to run cibadmin with and place it into a text file. Now convert the cibadmin command from --xml-text (-X) to --xml-file (-x) and use the XML file you just created. Fixed. Colin -Original Message- From: Chris Feist [mailto:cfe...@redhat.com] Sent: Monday, July 21, 2014 11:14 AM To: K Mehta Cc: The Pacemaker cluster resource manager Subject: Re: [Pacemaker] Cannot create more than 27 multistate resources On 07/21/2014 05:22 AM, Andrew Beekhof wrote: > Chris, > > Does the error below mean anything to you? > This seems to be happening once the CIB reaches a certain size, but is on the > client side and possibly before the pacemaker tools are invoked. I grabbed your debug file and did some tests and it looks like the issue is caused by earlier version of pcs (0.9.90 is affected) which try to pass the entire cib on the command line to cibadmin. This has been fixed upstream (and should be built in the next release of RHEL/CentOS). As a workaround, you can use the upstream sources here: https://github.com/feist/pcs (just run pcs from the directory that is cloned). Thanks! Chris > > On 9 Jul 2014, at 6:49 pm, K Mehta wrote: > >> [root@vsanqa11 ~]# pcs resource create >> vha-3de5ab16-9917-4b90-93d2-7b04fc71879c >> ocf:heartbeat:vgc-cm-agent.ocf >> cluster_uuid=3de5ab16-9917-4b90-93d2-7b04fc71879c op monitor >> role="Master" interval=30s timeout=100s op monitor role="Master" >> interval=30s timeout=100s >> >> >> pcs status output includes >> vha-3de5ab16-9917-4b90-93d2-7b04fc71879c >> (ocf::heartbeat:vgc-cm-agent.ocf): Started vsanqa11 >> >> >> [root@vsanqa11 ~]# pcs resource master >> ms-3de5ab16-9917-4b90-93d2-7b04fc71879c >> vha-3de5ab16-9917-4b90-93d2-7b04fc71879c meta clone-max=2 >> globally-unique=false target-role=started >> Error: unable to locate command: /usr/sbin/cibadmin >> > > > Looking in the logs, I see: > > Jul 12 11:18:24 vsanqa11 cibadmin[7966]: notice: crm_log_args: Invoked: > /usr/sbin/cibadmin -c -R --xml-text #012 id="cib-bootstrap-options">#012 id="cib-bootstrap-options-dc-version" name="dc-version" > value="1.1.10-14.el6_5.2-368c726"/>#012 id="cib-bootstrap-options-cluster-infrastructure" > name="cluster-infrastructure" value="cman"/>#012#012 id="cib-bootstrap-options-no-quorum-policy" name="no-quorum-policy" > value="ignore"/>#012 name="stoni > > But am I right in thinking that this that doesn't look like the result of a > pcs command? > > Kiran: Can you give us more information on the other commands you're running? > ___ Pacemaker mailing list: Pacemaker@oss.clusterlabs.org http://oss.clusterlabs.org/mailman/listinfo/pacemaker Project Home: http://www.clusterlabs.org Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf Bugs: http://bugs.clusterlabs.org ___ Pacemaker mailing list: Pacemaker@oss.clusterlabs.org http://oss.clusterlabs.org/mailman/listinfo/pacemaker Project Home: http://www.clusterlabs.org Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf Bugs: http://bugs.clusterlabs.org
Re: [Pacemaker] Cannot create more than 27 multistate resources
This error is thrown when i try to create master resource.
[root@vsanqa11 kiran]# pcs resource master
ms-f3611cc5-a68f-4c69-ab35-a9b226473e5d
vha-f3611cc5-a68f-4c69-ab35-a9b226473e5d meta clone-max=2
globally-unique=false target-role=started
Error: unable to locate command: /usr/sbin/cibadmin
[root@vsanqa11 kiran]# pcs resource master
ms-f3611cc5-a68f-4c69-ab35-a9b226473e5d
vha-f3611cc5-a68f-4c69-ab35-a9b226473e5d meta clone-max=2
globally-unique=false target-role=started
Error: unable to locate command: /usr/sbin/cibadmin
[root@vsanqa11 kiran]# pcs resource master
ms-f3611cc5-a68f-4c69-ab35-a9b226473e5d
vha-f3611cc5-a68f-4c69-ab35-a9b226473e5d meta clone-max=2
globally-unique=false target-role=started
Error: unable to locate command: /usr/sbin/cibadmin
[root@vsanqa11 kiran]# pcs resource master
ms-f3611cc5-a68f-4c69-ab35-a9b226473e5d
vha-f3611cc5-a68f-4c69-ab35-a9b226473e5d meta clone-max=2
globally-unique=false target-role=started
Error: unable to locate command: /usr/sbin/cibadmin
[root@vsanqa11 kiran]# pcs resource master
ms-f3611cc5-a68f-4c69-ab35-a9b226473e5d
vha-f3611cc5-a68f-4c69-ab35-a9b226473e5d meta clone-max=2
globally-unique=false target-role=started
Error: unable to locate command: /usr/sbin/cibadmin
[root@vsanqa11 kiran]# pcs status | grep
"vha-f3611cc5-a68f-4c69-ab35-a9b226473e5d"
vha-f3611cc5-a68f-4c69-ab35-a9b226473e5d
(ocf::heartbeat:vgc-cm-agent.ocf): Started vsanqa11
[root@vsanqa11 kiran]# pcs resource master
ms-f3611cc5-a68f-4c69-ab35-a9b226473e5d
vha-f3611cc5-a68f-4c69-ab35-a9b226473e5d meta clone-max=2
globally-unique=false target-role=started
Error: unable to locate command: /usr/sbin/cibadmin
Creation of these resources is automated using a script. Here is snippet of
the script:
pcs cluster cib $CLUSTER_CREATE_LOG || exit 1
pcs -f $CLUSTER_CREATE_LOG property set stonith-enabled=false ||
exit 1
pcs -f $CLUSTER_CREATE_LOG property set no-quorum-policy=ignore ||
exit 1
#syntax for following command is different across pcs 9.26 and 9.90
pcs -f $CLUSTER_CREATE_LOG resource defaults
resource-stickiness=100 > /dev/null 2>&1
if [ $? -ne 0 ]; then
pcs -f $CLUSTER_CREATE_LOG resource rsc defaults
resource-stickiness=100 || exit 1
fi
pcs -f $CLUSTER_CREATE_LOG resource create vha-$uuid
ocf:heartbeat:vgc-cm-agent.ocf\
cluster_uuid=$uuid \
op monitor role="Master" interval=30s timeout=100s\
op monitor role="Slave" interval=31s timeout=100s || exit 1
pcs -f $CLUSTER_CREATE_LOG resource master ms-${uuid} vha-${uuid}
meta clone-max=2 \
globally-unique=false target-role=started || exit 1
pcs -f $CLUSTER_CREATE_LOG constraint location vha-${uuid}
prefers $node1 || exit 1
pcs -f $CLUSTER_CREATE_LOG constraint location vha-${uuid}
prefers $node2 || exit 1
pcs -f $CLUSTER_CREATE_LOG constraint location ms-${uuid} prefers
$node1 || exit 1
pcs -f $CLUSTER_CREATE_LOG constraint location ms-${uuid} prefers
$node2 || exit 1
#syntax for following command is different across pcs 9.26 and 9.90
pcs cluster cib-push $CLUSTER_CREATE_LOG > /dev/null 2>&1
if [ $? -ne 0 ]; then
pcs cluster push cib $CLUSTER_CREATE_LOG
fi
On Mon, Jul 21, 2014 at 3:52 PM, Andrew Beekhof wrote:
> Chris,
>
> Does the error below mean anything to you?
> This seems to be happening once the CIB reaches a certain size, but is on
> the client side and possibly before the pacemaker tools are invoked.
>
> On 9 Jul 2014, at 6:49 pm, K Mehta wrote:
>
> > [root@vsanqa11 ~]# pcs resource create
> vha-3de5ab16-9917-4b90-93d2-7b04fc71879c ocf:heartbeat:vgc-cm-agent.ocf
> cluster_uuid=3de5ab16-9917-4b90-93d2-7b04fc71879c op monitor role="Master"
> interval=30s timeout=100s op monitor role="Master" interval=30s timeout=100s
> >
> >
> > pcs status output includes
> > vha-3de5ab16-9917-4b90-93d2-7b04fc71879c
> (ocf::heartbeat:vgc-cm-agent.ocf): Started vsanqa11
> >
> >
> > [root@vsanqa11 ~]# pcs resource master
> ms-3de5ab16-9917-4b90-93d2-7b04fc71879c
> vha-3de5ab16-9917-4b90-93d2-7b04fc71879c meta clone-max=2
> globally-unique=false target-role=started
> > Error: unable to locate command: /usr/sbin/cibadmin
> >
>
>
> Looking in the logs, I see:
>
> Jul 12 11:18:24 vsanqa11 cibadmin[7966]: notice: crm_log_args: Invoked:
> /usr/sbin/cibadmin -c -R --xml-text #012 id="cib-bootstrap-options">#012 id="cib-bootstrap-options-dc-version" name="dc-version"
> value="1.1.10-14.el6_5.2-368c726"/>#012 id="cib-bootstrap-options-cluster-infrastructure"
> name="cluster-infrastructure" value="cman"/>#012#012 id="cib-bootstrap-options-no-quorum-policy" name="no-quorum-policy"
> value="ignore"/>#012 name="stoni
>
> But am I right in thinking that this that doesn't look like the result of
> a pcs command?
>
> Kiran: Can you give us more information on the other commands you'r
Re: [Pacemaker] Cannot create more than 27 multistate resources
Following command was failing pcs resource master ms-f3611cc5-a68f-4c69-ab35-a9b226473e5d vha-f3611cc5-a68f-4c69-ab35-a9b226473e5d meta clone-max=2 globally-unique=false target-role=started So i did the following pcs resource master ms-f3611cc5-a68f-4c69-ab35-a9b226473e5d vha-f3611cc5-a68f-4c69-ab35-a9b226473e5d meta clone-max=2 globally-unique=false target-role=started --debug > xmlfile #removed first three lines which are not part of xml from the xmlfile generated cibadmin -Mc --xml-file=xmlfile But pcs status did not show the resource for which pcs resource command had failed Also tried cibadmin --R --xml-file=xmlfile But even then didnt find the resource in output of pcs status 1. Are these right set of commands ? 2. Which version of pcs has fix for this issue ? regards, kiran On Mon, Jul 21, 2014 at 10:21 PM, Colin Mason wrote: > Yes, I had 10 (20) Master/Slave resources, 5 (10) clone resources, 24 > simple resources for a total of 54 when I started running into this error. > I could not create another Master/Slave resource with the exact same error. > > > > Error: unable to locate command: /usr/sbin/cibadmin > > > > You are simply hitting a limit of the amount of characters you can use on > one command because cib.xml is getting large. Use pcs --debug to dump the > XML to screen, paste it into a file and run cibadmin --xml-file directly > for a workaround. Works great. > > > > Colin > > > > > > *From:* K Mehta [mailto:kiranmehta1...@gmail.com] > *Sent:* Monday, July 21, 2014 12:28 PM > *To:* Colin Mason > > *Subject:* Re: [Pacemaker] Cannot create more than 27 multistate resources > > > > Colin, > > In your setup do you have multi state resources or simple > resources ? > > In my case, I get error in creation of master after successful > creation of 27 multi state resources. > > > > [root@vsanqa11 kiran]# pcs status > > Cluster name: vsanqa11_12 > > Last updated: Mon Jul 21 09:25:36 2014 > > Last change: Mon Jul 21 09:17:25 2014 via cibadmin on vsanqa11 > > Stack: cman > > Current DC: vsanqa12 - partition with quorum > > Version: 1.1.10-14.el6_5.2-368c726 > > 2 Nodes configured > > 55 Resources configured **<<<<< (27 multi state resources on two node > cluster = 54 resources + 1 resource whose master is to be created) > > > > After this master resource cannot be created. > > > > > > > > On Mon, Jul 21, 2014 at 8:51 PM, Colin Mason > wrote: > > I have run into this issue with greater than 55 resources when using > pcs-0.9.90-2.el6.centos.2.noarch. > > The other workaround is to run the pcs command with --debug. Take the > whole XML text that pcs is trying to run cibadmin with and place it into a > text file. Now convert the cibadmin command from --xml-text (-X) to > --xml-file (-x) and use the XML file you just created. Fixed. > > Colin > > > -Original Message- > From: Chris Feist [mailto:cfe...@redhat.com] > Sent: Monday, July 21, 2014 11:14 AM > To: K Mehta > Cc: The Pacemaker cluster resource manager > Subject: Re: [Pacemaker] Cannot create more than 27 multistate resources > > On 07/21/2014 05:22 AM, Andrew Beekhof wrote: > > Chris, > > > > Does the error below mean anything to you? > > This seems to be happening once the CIB reaches a certain size, but is > on the client side and possibly before the pacemaker tools are invoked. > > I grabbed your debug file and did some tests and it looks like the issue > is caused by earlier version of pcs (0.9.90 is affected) which try to pass > the entire cib on the command line to cibadmin. This has been fixed > upstream (and should be built in the next release of RHEL/CentOS). > > As a workaround, you can use the upstream sources here: > https://github.com/feist/pcs (just run pcs from the directory that is > cloned). > > Thanks! > Chris > > > > > On 9 Jul 2014, at 6:49 pm, K Mehta wrote: > > > >> [root@vsanqa11 ~]# pcs resource create > >> vha-3de5ab16-9917-4b90-93d2-7b04fc71879c > >> ocf:heartbeat:vgc-cm-agent.ocf > >> cluster_uuid=3de5ab16-9917-4b90-93d2-7b04fc71879c op monitor > >> role="Master" interval=30s timeout=100s op monitor role="Master" > >> interval=30s timeout=100s > >> > >> > >> pcs status output includes > >> vha-3de5ab16-9917-4b90-93d2-7b04fc71879c > (ocf::heartbeat:vgc-cm-agent.ocf): Started vsanqa11 > >> > >> > >> [root@vsanqa11 ~]# pcs resource master > >> ms-3de5ab16-9917-4b90-93d2-7b04fc71879c > >> vha-3de5ab16-9917-4b90-93d2-7b04fc71879c
Re: [Pacemaker] Cannot create more than 27 multistate resources
Hi Kiran,
the --debug flag makes pcs print info about all runned external commands
and their output. Pcs has to read the CIB first in order to update it so
your xmlfile contains the original CIB at the beginning. You need to
find the updated CIB xml in the file (it starts on the line that looks
like this: 'Running: /usr/sbin/cibadmin --replace -o configuration -V
-X') and delete everything else. Then use 'cibadmin -R
--xml-file=xmlfile' command for uploading the CIB to the cluster.
The issues with large CIB have been fixed in pcs 0.9.116.
Regards,
Tomas
Dne 11.9.2014 v 07:24 K Mehta napsal(a):
Following command was failing
pcs resource master ms-f3611cc5-a68f-4c69-ab35-a9b226473e5d
vha-f3611cc5-a68f-4c69-ab35-a9b226473e5d meta clone-max=2
globally-unique=false target-role=started
So i did the following
pcs resource master ms-f3611cc5-a68f-4c69-ab35-a9b226473e5d
vha-f3611cc5-a68f-4c69-ab35-a9b226473e5d meta clone-max=2
globally-unique=false target-role=started --debug > xmlfile
#removed first three lines which are not part of xml from the xmlfile
generated
cibadmin -Mc --xml-file=xmlfile
But pcs status did not show the resource for which pcs resource command
had failed
Also tried
cibadmin --R --xml-file=xmlfile
But even then didnt find the resource in output of pcs status
1. Are these right set of commands ?
2. Which version of pcs has fix for this issue ?
regards,
kiran
On Mon, Jul 21, 2014 at 10:21 PM, Colin Mason
mailto:cma...@frontiernetworks.ca>> wrote:
Yes, I had 10 (20) Master/Slave resources, 5 (10) clone resources,
24 simple resources for a total of 54 when I started running into
this error. I could not create another Master/Slave resource with
the exact same error.
__ __
Error: unable to locate command: /usr/sbin/cibadmin
__ __
You are simply hitting a limit of the amount of characters you can
use on one command because cib.xml is getting large. Use pcs --debug
to dump the XML to screen, paste it into a file and run cibadmin
--xml-file directly for a workaround. Works great.
__ __
Colin
__ __
__ __
*From:*K Mehta [mailto:kiranmehta1...@gmail.com
<mailto:kiranmehta1...@gmail.com>]
*Sent:* Monday, July 21, 2014 12:28 PM
*To:* Colin Mason
*Subject:* Re: [Pacemaker] Cannot create more than 27 multistate
resources
__ __
Colin,
In your setup do you have multi state resources or simple
resources ?
In my case, I get error in creation of master after
successful creation of 27 multi state resources.
__ __
[root@vsanqa11 kiran]# pcs status
Cluster name: vsanqa11_12
Last updated: Mon Jul 21 09:25:36 2014
Last change: Mon Jul 21 09:17:25 2014 via cibadmin on vsanqa11
Stack: cman
Current DC: vsanqa12 - partition with quorum
Version: 1.1.10-14.el6_5.2-368c726
2 Nodes configured
55 Resources configured **<<<<< (27 multi state resources on two
node cluster = 54 resources + 1 resource whose master is to be
created)
__ __
After this master resource cannot be created.
__ __
__ __
__ __
On Mon, Jul 21, 2014 at 8:51 PM, Colin Mason
mailto:cma...@frontiernetworks.ca>>
wrote:
I have run into this issue with greater than 55 resources when using
pcs-0.9.90-2.el6.centos.2.noarch.
The other workaround is to run the pcs command with --debug. Take
the whole XML text that pcs is trying to run cibadmin with and place
it into a text file. Now convert the cibadmin command from
--xml-text (-X) to --xml-file (-x) and use the XML file you just
created. Fixed.
Colin
-Original Message-
From: Chris Feist [mailto:cfe...@redhat.com <mailto:cfe...@redhat.com>]
Sent: Monday, July 21, 2014 11:14 AM
To: K Mehta
Cc: The Pacemaker cluster resource manager
Subject: Re: [Pacemaker] Cannot create more than 27 multistate resources
On 07/21/2014 05:22 AM, Andrew Beekhof wrote:
> Chris,
>
> Does the error below mean anything to you?
> This seems to be happening once the CIB reaches a certain size,
but is on the client side and possibly before the pacemaker tools
are invoked.
I grabbed your debug file and did some tests and it looks like the
issue is caused by earlier version of pcs (0.9.90 is affected) which
try to pass the entire cib on the command line to cibadmin. This
has been fixed upstream (and should be built in the next release of
RHEL/CentOS).
As a workaround, you can use the upstream sources here:
https://github.com/feist/pcs (just run pcs from the directory that
is cloned).
Thanks!
Chris
>
> On 9 Jul 2014, at 6:49 pm, K Mehta mailto:kiranme
