[Pacemaker] Corosync using unicast instead of multicast
Hi all, I'm having an issue with a setup using the following: cluster-glue-1.0.6-1.6.el5.x86_64.rpm cluster-glue-libs-1.0.6-1.6.el5.x86_64.rpm corosync-1.2.7-1.1.el5.x86_64.rpm corosynclib-1.2.7-1.1.el5.x86_64.rpm drbd83-8.3.2-6.el5_3.x86_64.rpm kmod-drbd83-8.3.2-6.el5_3.x86_64.rpm openais-1.1.3-1.6.el5.x86_64.rpm openaislib-1.1.3-1.6.el5.x86_64.rpm pacemaker-1.0.9.1-1.el5.x86_64.rpm pacemaker-libs-1.0.9.1-1.el5.x86_64.rpm resource-agents-1.0.3-2.el5.x86_64.rpm This is a two-node HA cluster, with the nodes interconnected via bonded interfaces through the switch. The issue is that I have no control of the switch itself, can't do anything about that, and from what I understand the environment doesn't allow enabling multicast on the switch. In this situation, how can I have the setup functional (with redundant rings, rrp_mode: active) without using multicast. I've seen that individual network sockets are formed between nodes, unicast sockets, as well as the multicast sockets. I'm interested in knowing how will the lack of multicast affect the redundant rings, connectivity, failover, etc. I've also seen this page https://lists.linux-foundation.org/pipermail/openais/2010-October/015271.html And here it states using UDPU transport mode avoids using multicast or broadcast, but it's a patch, is this integrated in any of the newer versions of corosync? Thank you in advance. Regards, Dan -- Dan FRINCU Systems Engineer CCNA, RHCE Streamwide Romania ___ Pacemaker mailing list: Pacemaker@oss.clusterlabs.org http://oss.clusterlabs.org/mailman/listinfo/pacemaker Project Home: http://www.clusterlabs.org Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf Bugs: http://developerbugs.linux-foundation.org/enter_bug.cgi?product=Pacemaker
Re: [Pacemaker] Corosync using unicast instead of multicast
This question should be on the openais list, however, I happen to know the answer. To get up and running quickly you can configure broadcast with the version you have. Corosync can distinguish separate clusters with the multicast address and port that become payload to the messages. The patch you referred to can be applied to the top of tree for corosync or you can wait for a new release 1.3.0 planned for the end of November. Alan On Thu, Nov 4, 2010 at 1:02 AM, Dan Frincu wrote: > Hi all, > > I'm having an issue with a setup using the following: > cluster-glue-1.0.6-1.6.el5.x86_64.rpm > cluster-glue-libs-1.0.6-1.6.el5.x86_64.rpm > corosync-1.2.7-1.1.el5.x86_64.rpm > corosynclib-1.2.7-1.1.el5.x86_64.rpm > drbd83-8.3.2-6.el5_3.x86_64.rpm > kmod-drbd83-8.3.2-6.el5_3.x86_64.rpm > openais-1.1.3-1.6.el5.x86_64.rpm > openaislib-1.1.3-1.6.el5.x86_64.rpm > pacemaker-1.0.9.1-1.el5.x86_64.rpm > pacemaker-libs-1.0.9.1-1.el5.x86_64.rpm > resource-agents-1.0.3-2.el5.x86_64.rpm > > This is a two-node HA cluster, with the nodes interconnected via bonded > interfaces through the switch. The issue is that I have no control of the > switch itself, can't do anything about that, and from what I understand the > environment doesn't allow enabling multicast on the switch. In this > situation, how can I have the setup functional (with redundant rings, > rrp_mode: active) without using multicast. > > I've seen that individual network sockets are formed between nodes, unicast > sockets, as well as the multicast sockets. I'm interested in knowing how > will the lack of multicast affect the redundant rings, connectivity, > failover, etc. > > I've also seen this page > https://lists.linux-foundation.org/pipermail/openais/2010-October/015271.html > And here it states using UDPU transport mode avoids using multicast or > broadcast, but it's a patch, is this integrated in any of the newer versions > of corosync? > > Thank you in advance. > > Regards, > > Dan > > -- > Dan FRINCU > Systems Engineer > CCNA, RHCE > Streamwide Romania > > ___ > Pacemaker mailing list: Pacemaker@oss.clusterlabs.org > http://oss.clusterlabs.org/mailman/listinfo/pacemaker > > Project Home: http://www.clusterlabs.org > Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf > Bugs: > http://developerbugs.linux-foundation.org/enter_bug.cgi?product=Pacemaker > > ___ Pacemaker mailing list: Pacemaker@oss.clusterlabs.org http://oss.clusterlabs.org/mailman/listinfo/pacemaker Project Home: http://www.clusterlabs.org Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf Bugs: http://developerbugs.linux-foundation.org/enter_bug.cgi?product=Pacemaker
Re: [Pacemaker] Corosync using unicast instead of multicast
Hi, Alan Jones wrote: This question should be on the openais list, however, I happen to know the answer. To get up and running quickly you can configure broadcast with the version you have. I've done that already, however I was a little concerned as to what Steven Dake said on the openais mailing list about using broadcast "Broadcast and redundant ring probably don't work to well together.". I've also done some testing and saw that the broadcast address used is 255.255.255.255, regardless of what the bindnetaddr network address is, and quite frankly, I was hoping to see a directed broadcast address. This wasn't the case, therefore I wonder whether this was the issue that Steven was referring to, because by using the 255.255.255.255 as a broadcast address, there is the slight chance that some application running in the same network might send a broadcast packet using the same port as configured on the cluster. How would the cluster react to that, would it ignore the packet, would it wreak havoc? Regards, Dan That's my main concern right now. Corosync can distinguish separate clusters with the multicast address and port that become payload to the messages. The patch you referred to can be applied to the top of tree for corosync or you can wait for a new release 1.3.0 planned for the end of November. Alan On Thu, Nov 4, 2010 at 1:02 AM, Dan Frincu wrote: Hi all, I'm having an issue with a setup using the following: cluster-glue-1.0.6-1.6.el5.x86_64.rpm cluster-glue-libs-1.0.6-1.6.el5.x86_64.rpm corosync-1.2.7-1.1.el5.x86_64.rpm corosynclib-1.2.7-1.1.el5.x86_64.rpm drbd83-8.3.2-6.el5_3.x86_64.rpm kmod-drbd83-8.3.2-6.el5_3.x86_64.rpm openais-1.1.3-1.6.el5.x86_64.rpm openaislib-1.1.3-1.6.el5.x86_64.rpm pacemaker-1.0.9.1-1.el5.x86_64.rpm pacemaker-libs-1.0.9.1-1.el5.x86_64.rpm resource-agents-1.0.3-2.el5.x86_64.rpm This is a two-node HA cluster, with the nodes interconnected via bonded interfaces through the switch. The issue is that I have no control of the switch itself, can't do anything about that, and from what I understand the environment doesn't allow enabling multicast on the switch. In this situation, how can I have the setup functional (with redundant rings, rrp_mode: active) without using multicast. I've seen that individual network sockets are formed between nodes, unicast sockets, as well as the multicast sockets. I'm interested in knowing how will the lack of multicast affect the redundant rings, connectivity, failover, etc. I've also seen this page https://lists.linux-foundation.org/pipermail/openais/2010-October/015271.html And here it states using UDPU transport mode avoids using multicast or broadcast, but it's a patch, is this integrated in any of the newer versions of corosync? Thank you in advance. Regards, Dan -- Dan FRINCU Systems Engineer CCNA, RHCE Streamwide Romania ___ Pacemaker mailing list: Pacemaker@oss.clusterlabs.org http://oss.clusterlabs.org/mailman/listinfo/pacemaker Project Home: http://www.clusterlabs.org Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf Bugs: http://developerbugs.linux-foundation.org/enter_bug.cgi?product=Pacemaker ___ Pacemaker mailing list: Pacemaker@oss.clusterlabs.org http://oss.clusterlabs.org/mailman/listinfo/pacemaker Project Home: http://www.clusterlabs.org Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf Bugs: http://developerbugs.linux-foundation.org/enter_bug.cgi?product=Pacemaker -- Dan FRINCU Systems Engineer CCNA, RHCE Streamwide Romania ___ Pacemaker mailing list: Pacemaker@oss.clusterlabs.org http://oss.clusterlabs.org/mailman/listinfo/pacemaker Project Home: http://www.clusterlabs.org Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf Bugs: http://developerbugs.linux-foundation.org/enter_bug.cgi?product=Pacemaker
Re: [Pacemaker] Corosync using unicast instead of multicast
On 11/05/2010 01:30 AM, Dan Frincu wrote: Hi, Alan Jones wrote: This question should be on the openais list, however, I happen to know the answer. To get up and running quickly you can configure broadcast with the version you have. I've done that already, however I was a little concerned as to what Steven Dake said on the openais mailing list about using broadcast "Broadcast and redundant ring probably don't work to well together.". I've also done some testing and saw that the broadcast address used is 255.255.255.255, regardless of what the bindnetaddr network address is, and quite frankly, I was hoping to see a directed broadcast address. This wasn't the case, therefore I wonder whether this was the issue that Steven was referring to, because by using the 255.255.255.255 as a broadcast address, there is the slight chance that some application running in the same network might send a broadcast packet using the same This can happen with multicast or unicast modes as well. If a third party application communicates on the multicast/port combo or unicast port of a cluster node, there is conflict. With encryption, corosync encrypts and authenticates all packets, ignoring packets without a proper signature. The signatures are difficult to spoof. Without encryption, bad things happen in this condition. For more details, read "SECURITY" file in our source distribution. port as configured on the cluster. How would the cluster react to that, would it ignore the packet, would it wreak havoc? Regards, Dan That's my main concern right now. Corosync can distinguish separate clusters with the multicast address and port that become payload to the messages. The patch you referred to can be applied to the top of tree for corosync or you can wait for a new release 1.3.0 planned for the end of November. Alan On Thu, Nov 4, 2010 at 1:02 AM, Dan Frincu wrote: Hi all, I'm having an issue with a setup using the following: cluster-glue-1.0.6-1.6.el5.x86_64.rpm cluster-glue-libs-1.0.6-1.6.el5.x86_64.rpm corosync-1.2.7-1.1.el5.x86_64.rpm corosynclib-1.2.7-1.1.el5.x86_64.rpm drbd83-8.3.2-6.el5_3.x86_64.rpm kmod-drbd83-8.3.2-6.el5_3.x86_64.rpm openais-1.1.3-1.6.el5.x86_64.rpm openaislib-1.1.3-1.6.el5.x86_64.rpm pacemaker-1.0.9.1-1.el5.x86_64.rpm pacemaker-libs-1.0.9.1-1.el5.x86_64.rpm resource-agents-1.0.3-2.el5.x86_64.rpm This is a two-node HA cluster, with the nodes interconnected via bonded interfaces through the switch. The issue is that I have no control of the switch itself, can't do anything about that, and from what I understand the environment doesn't allow enabling multicast on the switch. In this situation, how can I have the setup functional (with redundant rings, rrp_mode: active) without using multicast. I've seen that individual network sockets are formed between nodes, unicast sockets, as well as the multicast sockets. I'm interested in knowing how will the lack of multicast affect the redundant rings, connectivity, failover, etc. I've also seen this page https://lists.linux-foundation.org/pipermail/openais/2010-October/015271.html And here it states using UDPU transport mode avoids using multicast or broadcast, but it's a patch, is this integrated in any of the newer versions of corosync? Thank you in advance. Regards, Dan -- Dan FRINCU Systems Engineer CCNA, RHCE Streamwide Romania ___ Pacemaker mailing list:Pacemaker@oss.clusterlabs.org http://oss.clusterlabs.org/mailman/listinfo/pacemaker Project Home:http://www.clusterlabs.org Getting started:http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf Bugs: http://developerbugs.linux-foundation.org/enter_bug.cgi?product=Pacemaker ___ Pacemaker mailing list:Pacemaker@oss.clusterlabs.org http://oss.clusterlabs.org/mailman/listinfo/pacemaker Project Home:http://www.clusterlabs.org Getting started:http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf Bugs:http://developerbugs.linux-foundation.org/enter_bug.cgi?product=Pacemaker -- Dan FRINCU Systems Engineer CCNA, RHCE Streamwide Romania ___ Pacemaker mailing list: Pacemaker@oss.clusterlabs.org http://oss.clusterlabs.org/mailman/listinfo/pacemaker Project Home: http://www.clusterlabs.org Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf Bugs: http://developerbugs.linux-foundation.org/enter_bug.cgi?product=Pacemaker ___ Pacemaker mailing list: Pacemaker@oss.clusterlabs.org http://oss.clusterlabs.org/mailman/listinfo/pacemaker Project Home: http://www.clusterlabs.org Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf Bugs: http://developerbugs.linux-foundation.org/enter_bug.cgi?product=Pacemaker
Re: [Pacemaker] Corosync using unicast instead of multicast
Hi, Steven Dake wrote: On 11/05/2010 01:30 AM, Dan Frincu wrote: Hi, Alan Jones wrote: This question should be on the openais list, however, I happen to know the answer. To get up and running quickly you can configure broadcast with the version you have. I've done that already, however I was a little concerned as to what Steven Dake said on the openais mailing list about using broadcast "Broadcast and redundant ring probably don't work to well together.". I've also done some testing and saw that the broadcast address used is 255.255.255.255, regardless of what the bindnetaddr network address is, and quite frankly, I was hoping to see a directed broadcast address. This wasn't the case, therefore I wonder whether this was the issue that Steven was referring to, because by using the 255.255.255.255 as a broadcast address, there is the slight chance that some application running in the same network might send a broadcast packet using the same This can happen with multicast or unicast modes as well. If a third party application communicates on the multicast/port combo or unicast port of a cluster node, there is conflict. With encryption, corosync encrypts and authenticates all packets, ignoring packets without a proper signature. The signatures are difficult to spoof. Without encryption, bad things happen in this condition. For more details, read "SECURITY" file in our source distribution. OK, I read the SECURITY file, a lot of overhead is added, I understand the reasons why it does it this way, not going to go into the details right now. Basically enabling encryption ensures that any traffic going between the nodes is both encrypted and authenticated, so rogue messages that happen to reach the exact network socket will be discarded. I'll come back to this a little bit later. Then again, I have this sentence in my head that I can't seem to get rid of "Broadcast and redundant ring probably don't work to well together, broadcast and redundant ring probably don't work to well together" and also I read "OpenAIS now provides broadcast network communication in addition to multicast. This functionality is considered Technology Preview for standalone usage of OpenAIS", therefore I'm a little bit more concerned. Can you shed some light on this please? Two questions: 1) What do you mean by "Broadcast and redundant ring probably don't work to well together"? 2) Is using Corosync's broadcast feature instead of multicast stable enough to be used in production systems? Thank you in advance. Best regards, Dan port as configured on the cluster. How would the cluster react to that, would it ignore the packet, would it wreak havoc? Regards, Dan That's my main concern right now. Corosync can distinguish separate clusters with the multicast address and port that become payload to the messages. The patch you referred to can be applied to the top of tree for corosync or you can wait for a new release 1.3.0 planned for the end of November. Alan On Thu, Nov 4, 2010 at 1:02 AM, Dan Frincu wrote: Hi all, I'm having an issue with a setup using the following: cluster-glue-1.0.6-1.6.el5.x86_64.rpm cluster-glue-libs-1.0.6-1.6.el5.x86_64.rpm corosync-1.2.7-1.1.el5.x86_64.rpm corosynclib-1.2.7-1.1.el5.x86_64.rpm drbd83-8.3.2-6.el5_3.x86_64.rpm kmod-drbd83-8.3.2-6.el5_3.x86_64.rpm openais-1.1.3-1.6.el5.x86_64.rpm openaislib-1.1.3-1.6.el5.x86_64.rpm pacemaker-1.0.9.1-1.el5.x86_64.rpm pacemaker-libs-1.0.9.1-1.el5.x86_64.rpm resource-agents-1.0.3-2.el5.x86_64.rpm This is a two-node HA cluster, with the nodes interconnected via bonded interfaces through the switch. The issue is that I have no control of the switch itself, can't do anything about that, and from what I understand the environment doesn't allow enabling multicast on the switch. In this situation, how can I have the setup functional (with redundant rings, rrp_mode: active) without using multicast. I've seen that individual network sockets are formed between nodes, unicast sockets, as well as the multicast sockets. I'm interested in knowing how will the lack of multicast affect the redundant rings, connectivity, failover, etc. I've also seen this page https://lists.linux-foundation.org/pipermail/openais/2010-October/015271.html And here it states using UDPU transport mode avoids using multicast or broadcast, but it's a patch, is this integrated in any of the newer versions of corosync? Thank you in advance. Regards, Dan -- Dan FRINCU Systems Engineer CCNA, RHCE Streamwide Romania ___ Pacemaker mailing list:Pacemaker@oss.clusterlabs.org http://oss.clusterlabs.org/mailman/listinfo/pacemaker Project Home:http://www.clusterlabs.org Getting started:http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf Bugs: http://developerbugs.linux-foundation.org/enter_bug.cgi?product=Pacemaker ___ Pacemaker mailing list:Pac
Re: [Pacemaker] Corosync using unicast instead of multicast
On 11/08/2010 05:50 AM, Dan Frincu wrote: Hi, Steven Dake wrote: On 11/05/2010 01:30 AM, Dan Frincu wrote: Hi, Alan Jones wrote: This question should be on the openais list, however, I happen to know the answer. To get up and running quickly you can configure broadcast with the version you have. I've done that already, however I was a little concerned as to what Steven Dake said on the openais mailing list about using broadcast "Broadcast and redundant ring probably don't work to well together.". I've also done some testing and saw that the broadcast address used is 255.255.255.255, regardless of what the bindnetaddr network address is, and quite frankly, I was hoping to see a directed broadcast address. This wasn't the case, therefore I wonder whether this was the issue that Steven was referring to, because by using the 255.255.255.255 as a broadcast address, there is the slight chance that some application running in the same network might send a broadcast packet using the same This can happen with multicast or unicast modes as well. If a third party application communicates on the multicast/port combo or unicast port of a cluster node, there is conflict. With encryption, corosync encrypts and authenticates all packets, ignoring packets without a proper signature. The signatures are difficult to spoof. Without encryption, bad things happen in this condition. For more details, read "SECURITY" file in our source distribution. OK, I read the SECURITY file, a lot of overhead is added, I understand the reasons why it does it this way, not going to go into the details right now. Basically enabling encryption ensures that any traffic going between the nodes is both encrypted and authenticated, so rogue messages that happen to reach the exact network socket will be discarded. I'll come back to this a little bit later. Then again, I have this sentence in my head that I can't seem to get rid of "Broadcast and redundant ring probably don't work to well together, broadcast and redundant ring probably don't work to well together" and also I read "OpenAIS now provides broadcast network communication in addition to multicast. This functionality is considered Technology Preview for standalone usage of OpenAIS", therefore I'm a little bit more concerned. Can you shed some light on this please? Two questions: 1) What do you mean by "Broadcast and redundant ring probably don't work to well together"? broadcast requires a specific port to run on. As a result, the ports should be different for each interface. I have not done any specific testing on broadcast with redundant ring - you would probably be the first. 2) Is using Corosync's broadcast feature instead of multicast stable enough to be used in production systems? Personally I'd wait for 2.0 for this feature and use bonding for the moment. Thank you in advance. Best regards, Dan port as configured on the cluster. How would the cluster react to that, would it ignore the packet, would it wreak havoc? Regards, Dan That's my main concern right now. Corosync can distinguish separate clusters with the multicast address and port that become payload to the messages. The patch you referred to can be applied to the top of tree for corosync or you can wait for a new release 1.3.0 planned for the end of November. Alan On Thu, Nov 4, 2010 at 1:02 AM, Dan Frincu wrote: Hi all, I'm having an issue with a setup using the following: cluster-glue-1.0.6-1.6.el5.x86_64.rpm cluster-glue-libs-1.0.6-1.6.el5.x86_64.rpm corosync-1.2.7-1.1.el5.x86_64.rpm corosynclib-1.2.7-1.1.el5.x86_64.rpm drbd83-8.3.2-6.el5_3.x86_64.rpm kmod-drbd83-8.3.2-6.el5_3.x86_64.rpm openais-1.1.3-1.6.el5.x86_64.rpm openaislib-1.1.3-1.6.el5.x86_64.rpm pacemaker-1.0.9.1-1.el5.x86_64.rpm pacemaker-libs-1.0.9.1-1.el5.x86_64.rpm resource-agents-1.0.3-2.el5.x86_64.rpm This is a two-node HA cluster, with the nodes interconnected via bonded interfaces through the switch. The issue is that I have no control of the switch itself, can't do anything about that, and from what I understand the environment doesn't allow enabling multicast on the switch. In this situation, how can I have the setup functional (with redundant rings, rrp_mode: active) without using multicast. I've seen that individual network sockets are formed between nodes, unicast sockets, as well as the multicast sockets. I'm interested in knowing how will the lack of multicast affect the redundant rings, connectivity, failover, etc. I've also seen this page https://lists.linux-foundation.org/pipermail/openais/2010-October/015271.html And here it states using UDPU transport mode avoids using multicast or broadcast, but it's a patch, is this integrated in any of the newer versions of corosync? Thank you in advance. Regards, Dan -- Dan FRINCU Systems Engineer CCNA, RHCE Streamwide Romania ___ Pacemaker mailing list:Pacemaker@oss.clusterlabs.org http://oss.clusterlabs.o
