Re: [Pacemaker] Routing-Ressources on a 2-Node-Cluster
On 23/04/2013, at 6:05 PM, T. wrote: > Hi Devin, > > thank you very much for your answer. > >> If you insist on trying to do this with just the Linux-HA cluster, >> I don't have any suggestions as to how you should proceed. > I know that the "construct" we are building is quite complicated. > > The problem is, that the active network (10.20.10.x) is too small to > cover both locations (in real we only have a /26 subnet available) and > we can not change/move this (public) addresses to an other range. > > In addition a NAT-translation over a router is not possible, the servers > have to be accessible directly via their public ip-address, that has to > be the cluster-ip. > > So we have to deal with two different networks in the two locations > (10.20.11.x/10.20.12.x) and create an "overlay" for the current > ip-addresses :-( Did you try corosync's udpu feature? It's basically the same as ucast from ha.cf but I don't recall if that feature made it into 6.4 > > > The current status is, that I added a modified the > heartbeat-Route2-script that also allows the "metric" as parameter and > with this it works as expected. > > But I am really fighting with the new corosync/pacemaker, for me the old > heartbeat (1) was much easier and gave me all functionality I needed. So > I make progress, but quite/too slow ... > -- > To Answer please replace "invalid" with "de" ! > Zum Antworten bitte "invalid" durch "de" ersetzen ! > > > Chau y hasta luego, > > Thorolf > > > ___ > Pacemaker mailing list: Pacemaker@oss.clusterlabs.org > http://oss.clusterlabs.org/mailman/listinfo/pacemaker > > Project Home: http://www.clusterlabs.org > Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf > Bugs: http://bugs.clusterlabs.org ___ Pacemaker mailing list: Pacemaker@oss.clusterlabs.org http://oss.clusterlabs.org/mailman/listinfo/pacemaker Project Home: http://www.clusterlabs.org Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf Bugs: http://bugs.clusterlabs.org
Re: [Pacemaker] Routing-Ressources on a 2-Node-Cluster
Hi Devin, thank you very much for your answer. > If you insist on trying to do this with just the Linux-HA cluster, > I don't have any suggestions as to how you should proceed. I know that the "construct" we are building is quite complicated. The problem is, that the active network (10.20.10.x) is too small to cover both locations (in real we only have a /26 subnet available) and we can not change/move this (public) addresses to an other range. In addition a NAT-translation over a router is not possible, the servers have to be accessible directly via their public ip-address, that has to be the cluster-ip. So we have to deal with two different networks in the two locations (10.20.11.x/10.20.12.x) and create an "overlay" for the current ip-addresses :-( The current status is, that I added a modified the heartbeat-Route2-script that also allows the "metric" as parameter and with this it works as expected. But I am really fighting with the new corosync/pacemaker, for me the old heartbeat (1) was much easier and gave me all functionality I needed. So I make progress, but quite/too slow ... -- To Answer please replace "invalid" with "de" ! Zum Antworten bitte "invalid" durch "de" ersetzen ! Chau y hasta luego, Thorolf ___ Pacemaker mailing list: Pacemaker@oss.clusterlabs.org http://oss.clusterlabs.org/mailman/listinfo/pacemaker Project Home: http://www.clusterlabs.org Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf Bugs: http://bugs.clusterlabs.org
Re: [Pacemaker] Routing-Ressources on a 2-Node-Cluster
David Coulson wrote: > Your configuration seems to have way too many moving parts and since you are > making routing changes when the nodes become primary it is difficult to > ensure that it will actually work based upon the monitoring you are doing > when it is passive. > > Not 100% sure what you're trying to accomplish, but when a config is this > complex it's usually not ideal. I would concur with this observation. To the OP: You're trying to achieve two orthogonal types of availability: Server and network. Trying to do this on a single cluster is far too complex with the setup you've described. Assuming you can get it working at all, it sounds overly fragile. Assuming that you can't get ahold of your own statics, have your own ASN, and have the statics routed by more than one ISP, I'd suggest breaking it out into two pieces: (1) Deal with the network connectivity by a pair of highly-available firewalls (OpenBSD with carp et al works very well for this). You can listen for connections coming from both ISPs and route them to the Linux HA-cluster (you can configure things so that the return packets go out the correct ISP link). Depending on your situation you *may* be able to configure non-return outbound traffic to use both ISP links as well. (2) Build a more normal Linux HA-cluster NAT'd behind those firewalls. That way the Linux cluster has only one (redundant) default route and it's configuration, which is already complex, doesn't become unmanageable. I have this type of configuration active in a few locations and it works well. It also has the advantage that other non-HA resources on the network also benefit from the redundant firewalls / ISP links. If you insist on trying to do this with just the Linux-HA cluster, I don't have any suggestions as to how you should proceed. Devin ___ Pacemaker mailing list: Pacemaker@oss.clusterlabs.org http://oss.clusterlabs.org/mailman/listinfo/pacemaker Project Home: http://www.clusterlabs.org Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf Bugs: http://bugs.clusterlabs.org
Re: [Pacemaker] Routing-Ressources on a 2-Node-Cluster
Hi David, > Why can't both your cluster nodes have 10.20.10.1 as their default > route all the time? because the different locations have different networks, routers, IP-addresses etc. But to be always reachable from the outside, they need one common IP-address, that is valid on the active node. And only for the active node the 10.20.10.1 is the way to go out, the passive node does even not have any connectivity to the outside. There is just a "little hole" for getting updates and for the pingd-resource needed for the HA-functionality. This is done by a proxy and/or by routes and it is working as expected. > Your configuration seems to have way too many moving parts and since you > are making routing changes when the nodes become primary it is difficult Well, beside the servers that have to run on the active node, the only "moving part" is the cluster-ip, it's default route and an additional route. So I do not see too much complexity? > Not 100% sure what you're trying to accomplish, but when a config is > this complex it's usually not ideal. That might be correct, but this is not the problem I have. So any other hint what I should do to get the "metric" working for the route-resources? Thanks a lot, -- Chau y hasta luego, Thorolf ___ Pacemaker mailing list: Pacemaker@oss.clusterlabs.org http://oss.clusterlabs.org/mailman/listinfo/pacemaker Project Home: http://www.clusterlabs.org Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf Bugs: http://bugs.clusterlabs.org
Re: [Pacemaker] Routing-Ressources on a 2-Node-Cluster
On 4/19/13 5:48 AM, T. wrote: When a server gets active, it will get the cluster-ip "10.20.10.70" and the default route to "10.20.10.1". Why can't both your cluster nodes have 10.20.10.1 as their default route all the time? Your configuration seems to have way too many moving parts and since you are making routing changes when the nodes become primary it is difficult to ensure that it will actually work based upon the monitoring you are doing when it is passive. Not 100% sure what you're trying to accomplish, but when a config is this complex it's usually not ideal. David ___ Pacemaker mailing list: Pacemaker@oss.clusterlabs.org http://oss.clusterlabs.org/mailman/listinfo/pacemaker Project Home: http://www.clusterlabs.org Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf Bugs: http://bugs.clusterlabs.org
Re: [Pacemaker] Routing-Ressources on a 2-Node-Cluster
Hi, > Because the nodes are located in different networks, each node needs > it's own route-ressource, that is only valid if the node is passive and > will be removed when this node goes active and gets the default route > for the cluster-ip. I did not found any solution for this, so I'm going "the other way". I added static routes with a high metric to each server: route add 1.2.3.4 via 10.20.11.1 metric 20 and route add 1.2.3.4 via 10.20.11.1 metric 20 When a server gets active, it will get the cluster-ip "10.20.10.70" and the default route to "10.20.10.1". In addition, the active node will get a new route via the cluster-ip with a lower metric, that overwrites the above static route: route add 1.2.3.4 via 10.20.10.1 metric 10 I tested this manually and it works as expected. There is just a "small" Problem: the heartbeat-route resource does not allow defining the metric :-( So should I change the "/usr/lib/ocf/resource.d/heartbeat/Route", should I create a new start-script or is there any other solution? Thanks a lot for your help, -- To Answer please replace "invalid" with "de" ! Zum Antworten bitte "invalid" durch "de" ersetzen ! Chau y hasta luego, Thorolf ___ Pacemaker mailing list: Pacemaker@oss.clusterlabs.org http://oss.clusterlabs.org/mailman/listinfo/pacemaker Project Home: http://www.clusterlabs.org Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf Bugs: http://bugs.clusterlabs.org
Re: [Pacemaker] Routing-Ressources on a 2-Node-Cluster
Hi David, thank you very much for your answer. > Are they really on different networks? What is the net mask? Yes I'm "quite" sure, they are connectect via different switches to different carriers to different networks. It is the preparation for location redundancy. > ClusterIP won't work unless they are both on the same VLAN. The switches have VLANs configured, one for each own network and an other for the cluster-ip network. I have not done the switch and router configuration, but am I sure that it works as expected and also the cluster-ip with it's default route works for manual failover on both nodes. > Maybe you could post your routing configuration, since what you are > explaining is hard to visualise. OK, I try it with some ASCII-art :-) Switch +---+ .11.70++ .11.1 +-+ | NodeA |---| VLAN A |-| RouterA | +---+ ++ +-+ | | ...| | | ...| | .10.70++ .10.1 +-+ 1.2.3.4 +---+ +---| VLAN C |-| RouterC |---| PingD | | ++ +-+ +---+ = | ...| . | | ...| . +---+ .12.70++ .12.1 +-+ . | NodeB |---| VLAN B |-| RouterB |...+ +---+ ++ +-+ [route add 1.2.3.4 via 10.20.12.1] This are the current routes, while the default route on NodeA was set by corosync/pacemaker when it changed to active, I had to add the route on NodeB manually when it falls back to passive to reach the external ping-node (ip route add 1.2.3.4 via 10.20.12.1 dev bond0): For NodeA (active): default via 10.20.10.1 dev bond0 For NodeB (passive): 1.2.3.4 via 10.20.12.1 dev bond0 or (manually on NodeA: ip route add 1.2.3.4 via 10.20.11.1 dev bond0): For NodeA (passive): 1.2.3.4 via 10.20.12.1 dev bond0 For NodeB (active): default via 10.20.10.1 dev bond0 What I need is, that the routes I set manually on the passive node, are handled by corosync/pacemaker too. For the .11.0/24 and .12.0/24-networks there is no default route defined. Thanks again for any help. -- To Answer please replace "invalid" with "de" ! Zum Antworten bitte "invalid" durch "de" ersetzen ! Chau y hasta luego, Thorolf ___ Pacemaker mailing list: Pacemaker@oss.clusterlabs.org http://oss.clusterlabs.org/mailman/listinfo/pacemaker Project Home: http://www.clusterlabs.org Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf Bugs: http://bugs.clusterlabs.org
Re: [Pacemaker] Routing-Ressources on a 2-Node-Cluster
On Apr 15, 2013, at 1:59 PM, "T." wrote: > > > For the access-network I use a different NIC, the nodes are in different > networks, NodeA has 10.20.11.70, NodeB has 10.20.12.70 and I have > configured a cluster-ip, the active node gets, (10.20.10.70). Are they really on different networks? What is the net mask? ClusterIP won't work unless they are both on the same VLAN. Maybe you could post your routing configuration, since what you are explaining is hard to visualise. David ___ Pacemaker mailing list: Pacemaker@oss.clusterlabs.org http://oss.clusterlabs.org/mailman/listinfo/pacemaker Project Home: http://www.clusterlabs.org Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf Bugs: http://bugs.clusterlabs.org
[Pacemaker] Routing-Ressources on a 2-Node-Cluster
Hi everybody, I have a CentOS 6.4-Cluster with corosync 1.4.1, pacemaker 1.1.8 and crmsh 1.2.5. The crmsh I reinstalled from a different repo, because I don't like pcs very much - no idea why it was removed in the pacemaker 1.1.8-build for RHEL/CentOS :-( The setup is not that difficult, anyway I do not know how to solve my problem. I have two nodes, NodeA and NodeB, directly connected via an own NIC for the cluster-communication and drbd. For the access-network I use a different NIC, the nodes are in different networks, NodeA has 10.20.11.70, NodeB has 10.20.12.70 and I have configured a cluster-ip, the active node gets, (10.20.10.70). I addition I have a HA-ping-cluster, that I use as cloned pingd-ressource on both cluster-nodes. The active node can reach this pingd-ressource via the cluster-ip and it's default route, but for the passive node I have to set a special route, that the system is able to reach it. Because the nodes are located in different networks, each node needs it's own route-ressource, that is only valid if the node is passive and will be removed when this node goes active and gets the default route for the cluster-ip. How do I have to configure this via CRM? Thanks a lot for your hints, -- To Answer please replace "invalid" with "de" ! Zum Antworten bitte "invalid" durch "de" ersetzen ! Chau y hasta luego, Thorolf ___ Pacemaker mailing list: Pacemaker@oss.clusterlabs.org http://oss.clusterlabs.org/mailman/listinfo/pacemaker Project Home: http://www.clusterlabs.org Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf Bugs: http://bugs.clusterlabs.org
