Hi, this is a known issue, see https://issues.apache.org/jira/browse/QPID-3483.
Some background: corosync has not authenticated qpidd process to access its CPG methods. Without CMAN in use, the access is allowed by /etc/corosync/uidgid.d/qpidd file. However, CMAN configuration (cluster.conf) simply ignores the file at all, not providing an option to set it up within the conf file. Workaround: run qpidd under root (as root user has access to corosync CPG). Kind regards, Pavel Moravec ----- Original Message ----- > From: "David Zhou" <dayda...@gmail.com> > To: users@qpid.apache.org > Sent: Thursday, December 8, 2011 8:33:44 AM > Subject: Qpid clustering with cman > > (nabble link: > http://apache-qpid-users.2158936.n2.nabble.com/Qpid-clustering-with-cman-td7073352.html > ) > > Hi all, > > I set up a 2-nodes Qpid cluster (0.12) which is synchronized by > corosync > (1.4.2). > The cluster works fine except running with cman to avoid split-brain. > It's said that settings of corosync.conf would be replaced by > cluster.conf, > but from the following trial and error it seems that corosync doesn't > fetch > settings from cluster.conf. > I wonder there are problems with my configurations. Could you please > help > me out? Thank you!! > > First here apply the configuration sample from Qpid > wiki<https://cwiki.apache.org/qpid/configuring-qpidd-with-cluster-manager.html> > : > > --cluster.conf------------------------------------------ > <?xml version="1.0" ?> > <cluster name="David" config_version="1"> > <cman two_node="1" expected_votes="1"> > </cman> > <clusternodes> > <clusternode name="c1q1" nodeid="1"> > </clusternode> > <clusternode name="c1q2" nodeid="2"> > </clusternode> > </clusternodes> > <rm> > <failoverdomains> > <failoverdomain name="only_broker1" nofailback="0" ordered="0" > restricted="1"> > <failoverdomainnode name="c1q1" priority="1"/> > </failoverdomain> > <failoverdomain name="only_broker2" nofailback="0" ordered="0" > restricted="1"> > <failoverdomainnode name="c1q2" priority="1"/> > </failoverdomain> > </failoverdomains> > <resources> > <script name="qpidd" file="/etc/init.d/qpidd" /> > </resources> > <service name="qpidd_broker1" domain="only_broker1"> > <script ref="qpidd" /> > </service> > <service name="qpidd_broker2" domain="only_broker2"> > <script ref="qpidd" /> > </service> > </rm> > </cluster> > ------------------------------------------------------- > > => Starting cman OK, but starting qpidd get the following error: > > --log------------------------------------------------- > Dec 7 19:02:45 c1q2 qpidd[6893]: notice Journal "TplStore": Created > Dec 7 19:02:45 c1q2 qpidd[6893]: notice Store module initialized; > store-dir=/var/spool/qpid > Dec 7 19:02:45 c1q2 qpidd[6893]: notice Initializing CPG > Dec 7 19:02:45 c1q2 corosync[6572]: [IPC ] Invalid IPC > credentials. > Dec 7 19:02:45 c1q2 qpidd[6893]: critical Unexpected error: Failed > to > initialize CPG.: security (100) > ------------------------------------------------------- > > I think it's because the bindnetaddr is not set in cluster.conf > (Without > cman, corosync + qpidd works fine), so I add the syntax found at > here<https://bugzilla.redhat.com/show_bug.cgi?id=613880> > > --cluster.conf------------------------------------------ > <?xml version="1.0" ?> > <cluster name="David" config_version="1"> > <cman two_node="1" expected_votes="1"> > </cman> > <totem> > <interface ringnumber="0" bindnetaddr="192.168.0.0" > mcastaddr="226.94.1.1" mcastport="5405" /> > </totem> > <clusternodes> > <clusternode name="c1q1" nodeid="1"> > </clusternode> > <clusternode name="c1q2" nodeid="2"> > </clusternode> > </clusternodes> > <rm> > <failoverdomains> > <failoverdomain name="only_broker1" nofailback="0" ordered="0" > restricted="1"> > <failoverdomainnode name="c1q1" priority="1"/> > </failoverdomain> > <failoverdomain name="only_broker2" nofailback="0" ordered="0" > restricted="1"> > <failoverdomainnode name="c1q2" priority="1"/> > </failoverdomain> > </failoverdomains> > <resources> > <script name="qpidd" file="/etc/init.d/qpidd" /> > </resources> > <service name="qpidd_broker1" domain="only_broker1"> > <script ref="qpidd" /> > </service> > <service name="qpidd_broker2" domain="only_broker2"> > <script ref="qpidd" /> > </service> > </rm> > </cluster> > ------------------------------------------------------- > > The result is not OK. corosync finds no multicast address specified. > > --log-------------------------------------------------- > Dec 8 11:05:08 c1q2 corosync[7465]: [MAIN ] Corosync Cluster > Engine ('1.4.2'): started and ready to provide service. > Dec 8 11:05:08 c1q2 corosync[7465]: [MAIN ] Corosync built-in > features: nss > Dec 8 11:05:08 c1q2 corosync[7465]: [MAIN ] Successfully read > config from /etc/cluster/cluster.conf > Dec 8 11:05:08 c1q2 corosync[7465]: [MAIN ] Successfully parsed > cman config > Dec 8 11:05:08 c1q2 corosync[7465]: [MAIN ] Successfully > configured openais services to load > Dec 8 11:05:08 c1q2 corosync[7465]: [MAIN ] parse error in > config: > No multicast address specified > Dec 8 11:05:08 c1q2 corosync[7465]: [MAIN ] Corosync Cluster > Engine exiting with status 8 at main.c:1708. > -------------------------------------------------------- > > Then I change the settings according to the *Multicast* section > described > here <http://sourceware.org/cluster/doc/usage.txt> > > --cluster.conf------------------------------------------ > <?xml version="1.0" ?> > <cluster name="David" config_version="1"> > <cman two_node="1" expected_votes="1"> > <multicast addr="226.94.1.1"/> > </cman> > <totem> > <interface bindnetaddr="192.168.0.0" /> > </totem> > <clusternodes> > <clusternode name="c1q1" nodeid="1"> > <multicast addr="226.94.1.1" interface="eth0"/> > </clusternode> > <clusternode name="c1q2" nodeid="2"> > <multicast addr="226.94.1.1" interface="eth0"/> > </clusternode> > </clusternodes> > <fencedevices> > </fencedevices> > <rm> > <failoverdomains> > <failoverdomain name="only_broker1" nofailback="0" ordered="0" > restricted="1"> > <failoverdomainnode name="c1q1" priority="1"/> > </failoverdomain> > <failoverdomain name="only_broker2" nofailback="0" ordered="0" > restricted="1"> > <failoverdomainnode name="c1q2" priority="1"/> > </failoverdomain> > </failoverdomains> > <resources> > <script name="qpidd" file="/etc/init.d/qpidd" /> > </resources> > <service name="qpidd_broker1" domain="only_broker1"> > <script ref="qpidd" /> > </service> > <service name="qpidd_broker2" domain="only_broker2"> > <script ref="qpidd" /> > </service> > </rm> > </cluster> > ------------------------------------------------------- > > It still not OK as the following shows: > > --log-------------------------------------------------- > Dec 8 11:51:23 c1q2 corosync[7936]: [MAIN ] Corosync Cluster > Engine ('1.4.2'): started and ready to provide service. > Dec 8 11:51:23 c1q2 corosync[7936]: [MAIN ] Corosync built-in > features: nss > Dec 8 11:51:23 c1q2 corosync[7936]: [MAIN ] Successfully read > config from /etc/cluster/cluster.conf > Dec 8 11:51:23 c1q2 corosync[7936]: [MAIN ] Successfully parsed > cman config > Dec 8 11:51:23 c1q2 corosync[7936]: [MAIN ] Successfully > configured openais services to load > Dec 8 11:51:23 c1q2 corosync[7936]: [MAIN ] parse error in > config: > No multicast address specified > Dec 8 11:51:23 c1q2 corosync[7936]: [MAIN ] Corosync Cluster > Engine exiting with status 8 at main.c:1708. > -------------------------------------------------------- > > --console---------------------------------------------- > root@c1q2:/etc/cluster# service cman start > Starting cluster: > Checking Network Manager... [ OK ] > Global setup... [ OK ] > Loading kernel modules... [ OK ] > Mounting configfs... [ OK ] > Starting cman... tempfile:11: element multicast: Relax-NG validity > error : Element clusternode has extra content: multicast > tempfile:10: element clusternode: Relax-NG validity error : Element > clusternodes has extra content: clusternode > Configuration fails to validate > corosync died: Error, reason code is 8 > [FAILED] > ------------------------------------------------------- > > > Thank you for you help!! > > > -- > View this message in context: > http://apache-qpid-users.2158936.n2.nabble.com/Qpid-clustering-with-cman-tp7073493p7073493.html > Sent from the Apache Qpid users mailing list archive at Nabble.com. --------------------------------------------------------------------- Apache Qpid - AMQP Messaging Implementation Project: http://qpid.apache.org Use/Interact: mailto:users-subscr...@qpid.apache.org
