Public bug reported: when a system is brought up using vxlan which has no interfaces added to br-ex bridge, if an ha router is created, the tunnel between the network nodes is created, however, when a external gateways is set for the router the vxlan tunnels between the network nodes is dropped.
how to reproduce: bring up an openstack system using three nodes: one "controller" running q-vpn, q-agt, q-svc, and the other necessary services (keystone, mysql, etc...) , One network node running q-l3, and q-agt, and ONE compute node, running the usual nova stuff (n-cpu, etc..) and the q-l3 and q-agt The q-l3 agent on the network node and controllers should be se to agent_mode=dvr_snat, the l3 agent on the compute node should be set to agent_mode=dvr Delete all resources (routers, networks, subnets, etc...) so the system is cmpletey clean Add the following commands to create the necessary networks. make sure that there is not interfaces in the br-ex bridges of any of the nodes: (ovs-vsctl show should sho br-ex bridge with no interfaces) neutron net-create public --router:exernal neutron subnet-create public 123.0.0.0/24 --disable-dhcp #create the ha router: neutron router-create harouter --ha=True --distributed=False #check that the ha network has been created: neutron net-list #check that the tunnels between the controller and network nodes are up: execute "sudo ovs-vsctl show " on the nodes. # since the controller and network node have an instance of the ha router running they will connect to each other to provide # the ha network via tunnels. # it will look something like: # Bridge br-tun # fail_mode: secure # Port patch-int # Interface patch-int # type: patch # options: {peer=patch-tun} # Port br-tun # Interface br-tun # type: internal # Port "vxlan-650000b1" # Interface "vxlan-650000b1" # type: vxlan # options: {df_default="true", in_key=flow, local_ip="101.0.0.178", out_key=flow, remote_ip="101.0.0.177"} # Bridge br-ex # ... # next set the external gateway for the router: neutron router-gateway-set harouter public # at this point the vxlan tunnels above will drop (disappear from the ovs-vsctl show output) # # remove the external gateay from the router: neutron router-gateway-clear harouter # it might be necessary to execute the "neutron router-gateway-set routerha public / neutron router-gateway-clear harouter" a few times to see the problem. More testing suggests that adding an interface to br-ex (by hand) prevents the problem from happening. in other words if before you do anything you do something like "sudo ovs-vsctl add-pr br-ex dummy-interface" to each of the nodes, the problem will not manifest. This would suggest that the problem is only present when br-ex has no interfaces in it to begin with. That case is very unlikey since most of the time br-ex will have an interface attached to it (the one connecting to the outside world external interface/nice). However it could present problems for ci testing if no interface is added to br-ex. This was last tested no master branch commit: commit 43710925db0523dfbf0cdabbf2352db4304c6163 Merge: 397fc4d 2759362 Author: Jenkins <jenk...@review.openstack.org> Date: Wed Sep 2 11:20:18 2015 +0000 Merge "Remove duplicated codes in two test cases" attaching local.conf files which can be used to setup system. change ip addresses respectively ** Affects: neutron Importance: Undecided Status: New ** Tags: ha neutron routing ** Attachment added: "compute node local.conf" https://bugs.launchpad.net/bugs/1491961/+attachment/4456949/+files/compute_node_only.conf -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to neutron. https://bugs.launchpad.net/bugs/1491961 Title: On systems with no ports in br-ex, ha network is dropped when external gateway is set for ha router Status in neutron: New Bug description: when a system is brought up using vxlan which has no interfaces added to br-ex bridge, if an ha router is created, the tunnel between the network nodes is created, however, when a external gateways is set for the router the vxlan tunnels between the network nodes is dropped. how to reproduce: bring up an openstack system using three nodes: one "controller" running q-vpn, q-agt, q-svc, and the other necessary services (keystone, mysql, etc...) , One network node running q-l3, and q-agt, and ONE compute node, running the usual nova stuff (n-cpu, etc..) and the q-l3 and q-agt The q-l3 agent on the network node and controllers should be se to agent_mode=dvr_snat, the l3 agent on the compute node should be set to agent_mode=dvr Delete all resources (routers, networks, subnets, etc...) so the system is cmpletey clean Add the following commands to create the necessary networks. make sure that there is not interfaces in the br-ex bridges of any of the nodes: (ovs-vsctl show should sho br-ex bridge with no interfaces) neutron net-create public --router:exernal neutron subnet-create public 123.0.0.0/24 --disable-dhcp #create the ha router: neutron router-create harouter --ha=True --distributed=False #check that the ha network has been created: neutron net-list #check that the tunnels between the controller and network nodes are up: execute "sudo ovs-vsctl show " on the nodes. # since the controller and network node have an instance of the ha router running they will connect to each other to provide # the ha network via tunnels. # it will look something like: # Bridge br-tun # fail_mode: secure # Port patch-int # Interface patch-int # type: patch # options: {peer=patch-tun} # Port br-tun # Interface br-tun # type: internal # Port "vxlan-650000b1" # Interface "vxlan-650000b1" # type: vxlan # options: {df_default="true", in_key=flow, local_ip="101.0.0.178", out_key=flow, remote_ip="101.0.0.177"} # Bridge br-ex # ... # next set the external gateway for the router: neutron router-gateway-set harouter public # at this point the vxlan tunnels above will drop (disappear from the ovs-vsctl show output) # # remove the external gateay from the router: neutron router-gateway-clear harouter # it might be necessary to execute the "neutron router-gateway-set routerha public / neutron router-gateway-clear harouter" a few times to see the problem. More testing suggests that adding an interface to br-ex (by hand) prevents the problem from happening. in other words if before you do anything you do something like "sudo ovs-vsctl add-pr br-ex dummy-interface" to each of the nodes, the problem will not manifest. This would suggest that the problem is only present when br-ex has no interfaces in it to begin with. That case is very unlikey since most of the time br-ex will have an interface attached to it (the one connecting to the outside world external interface/nice). However it could present problems for ci testing if no interface is added to br-ex. This was last tested no master branch commit: commit 43710925db0523dfbf0cdabbf2352db4304c6163 Merge: 397fc4d 2759362 Author: Jenkins <jenk...@review.openstack.org> Date: Wed Sep 2 11:20:18 2015 +0000 Merge "Remove duplicated codes in two test cases" attaching local.conf files which can be used to setup system. change ip addresses respectively To manage notifications about this bug go to: https://bugs.launchpad.net/neutron/+bug/1491961/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp