On Wed, May 9, 2018 at 9:02 AM, Numan Siddique <nusid...@redhat.com> wrote:
> > > On Wed, May 9, 2018 at 9:02 PM, Han Zhou <zhou...@gmail.com> wrote: > >> Hi Numan, >> >> Thanks you so much for the detailed answer! Please see my comments inline. >> >> On Wed, May 9, 2018 at 7:41 AM, Numan Siddique <nusid...@redhat.com> >> wrote: >> >>> Hi Han, >>> >>> Please see below for inline comments >>> >>> On Wed, May 9, 2018 at 5:17 AM, Han Zhou <zhou...@gmail.com> wrote: >>> >>>> Hi Babu/Numan, >>>> >>>> I have a question regarding OVN pacemaker OCF script. >>>> I see in the script MASTER_IP is used to start the active DB and >>>> standby DBs will use that IP to sync from. >>>> >>>> In the Documentation/topics/integration.rst it is also mentioned: >>>> >>>> `master_ip` is the IP address on which the active database server is >>>> expected to be listening, the slave node uses it to connect to the master >>>> node. >>>> >>>> However, since active node will change after failover, I wonder if we >>>> should provide all the IPs of each nodes, and let pacemaker to decide which >>>> IP is the master IP to be used, dynamically. >>>> >>> >>> >>> >>>> I see in the documentation it is mentioned about using the IPAddr2 >>>> resource for virtual IP. Does it indicate that we should use the virtual IP >>>> as the master IP? >>>> >>> >>> That is true. If the master ip is not virtual ip, then we will not be >>> able to figure out which is the master node. We need to configure >>> networking-ovn and ovn-controller to point to the right master node so that >>> they can do write transactions on the DB. >>> >>> Below is how we have configured pacemaker OVN HA dbs in tripleo >>> openstack deployment >>> >>> - Tripleo deployment creates many virtual IPs (using IPAddr2) and these >>> IP addresses are frontend IPs for keystone and all other openstack API >>> services and haproxy is used to load balance the traffic (the deployment >>> will mostly have 3 controllers and all the openstack API services will be >>> running on each node). >>> >>> - We choose one of the IPaddr2 virtual ip and we set a colocation >>> constraint when creating the OVN pacemaker HA db resource i.e we ask >>> pacemaker to promote the ovsdb-servers running in the node configured with >>> the virtual ip (i.e master_ip). Pacemaker will call the promote action [1] >>> on the node where master ip is configured. >>> >>> - tripleo configures "ovn_nb_connection=tcp:VIP:6641" and " >>> ovn_sb_connection=tcp:VIP:6642" in neutron.conf and runs "ovs-vsctl set >>> open . external_ids:ovn-remote=tcp:VIP:6642" on all the nodes where >>> ovn-controller service is started. >>> >>> - Suppose the master ip node goes down for some reason. Pacemaker >>> detects this and moves the virtual ip IPAddr2 resource to another node and >>> promotes the ovsdb-servers running on that node to master. This way, the >>> neutron-servers and ovn-controlloers can still talk to the same IP without >>> even noticing that other node becoming master. >>> >>> >>> >>> Since tripleo was using the IPaddr2 model, we thought this would be the >>> better way to have a master/slave HA for ovsdb-servers. >>> >>> However, this may not work in all scenarios, since the virtual IP works >>>> only if it can be routed to all nodes, e.g. when all nodes are on the same >>>> subnet. >>>> >>> >>> You mean you want to create a pacemaker cluster with nodes belonging to >>> different subnets ? I had a chat with the pacemaker folks and this is >>> possible. You can also create a IPAddr2 resource. Pacemaker doesn't put any >>> restrictions. But you need to solve the reachability of that ip from all >>> the networks/nodes. >>> >> >> Yes, and this is why we can't use IPAddr2 due to the reachability >> problem. (Not in same L2, no BGP, etc.) >> >> >>> In those cases the IPAddr2 virtual IP won't work. In those cases, for >>>> the clients to access the DB, we can use Load-Balancer VIP. But the problem >>>> is still how to set the master_ip and how to make the standby to connect to >>>> the new active after failover. >>>> >>> >>> I am a bit confused here. Your setup will still have the pacemaker >>> cluster right ? Are you talking about having OVN db servers active/passive >>> setup on a non pacemaker cluster setup ? If so, I don't think the OVN OCF >>> script can be used and you have to solve it differently. Correct me if I am >>> wrong here. >>> >>> >> You mentioned above "However, since active node will change after >>> failover, I wonder if we should provide all the IPs of each nodes, and let >>> pacemaker to decide which IP is the master IP to be used, dynamically". >>> >>> We can definitely add this support. Whenever pacemaker promotes a node, >>> other nodes come to know about it and OVN OCF script can configure the >>> ovsdb-servers on the slave nodes to connect to the new master. But how will >>> you configure the neutron-server and ovn-controllers to talk to the new >>> master ? >>> Are you planning to use load balancer IP for this purpose ? What if the >>> load balancer ip resolves to a standby server ? >>> >> >> We still have pacemaker to manage the cluster HA, but just don't use >> IPAddr2 for VIP. To solve the VIP problem, we use physical/soft >> load-balancer. The VIP is on LB rather than bound on the ovn central node >> interface. There is no problem for client, but a little problem on the OCF >> script. Since the OCF script relies on the master IP to start the active >> OVSDB, but the master IP (now LB VIP) is not attached on the node >> interface, this will fail. Now that you explained the usage of master IP, I >> think a small change can solve this problem: don't use master IP when >> starting the active OVSDB service, i.e. listen on 0.0.0.0. >> > > This would require changes in OVN OCF script right ? Probably it should be > enhanced such that the existing approach isn't broken (may be with a new > ocf param). > > Yes, and the change should be small. We will submit a patch for this. > > >> For standby OVSDBs, they will continue using master IP to sync from >> active. The standby should not listen on any port (or just on different >> port from the active if they have to), so that the LB health-check can >> figure out the active member and point the VIP/master IP correctly to the >> active one. >> >> In addition, how do you configure northd for NB/SB DB? I think both >> master-ip/vip and unix socket should work, but they are different. If using >> master-ip/vip, northd can be active on any one of the nodes, not co-locate >> with NB/SB DBs, and ovsdb named lock ensures only one is active. However, >> it seems we can also use unix socket to always connect to local NB/SB. >> Since NB/SB is managed as a single pacemaker resource, they failover >> together, so we can consider ovn-northd part of the bundle (but not managed >> by pacemaker). This way, although all northds are running, but only the one >> on the active NB/SB node matters, and ovsdb named lock is irrelevant here. >> Any thoughts/experience on this? >> > > In the case of openstack we let pacemaker manage it via the OCF script. So > we set manage_northd=yes. ovn-northd will be started only on master node. > Other option i think of is starting ovn-northd on the desired nodes (like > how you would start ovn-controllers) and point ovn-northd to use > tcp:LB_IP:6641/6642 (just like how neutron-server and ovn-controllers are > configured). The ovsdb named lock would make sure that only one is active. > > OK. I think what I mentioned that using unix socket instead of vip for northd to access works ONLY IF northd is managed by pacemaker (otherwise when northd fails on the master node, the other northds can't take over). Again, thanks for explain :)
_______________________________________________ discuss mailing list disc...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
