Public bug reported: We did a 4 node bare metal OpenStack Queens install. After setting up networking and adding eth0 to br-ex and restarting network service we cannot ping from qrouter to external floating IP network. Below layout of the 4 node setup and our ovs db info.
This was a fresh install using PackStack script modify to prep all nodes except the storage node. CentOS 7 OpenStack Queens release static hostname: controller01 Icon name: computer Machine ID: 0f62242dd7f04961b2fa642777708526 Boot ID: 1bf746fe751f4e58902431573696f31e Operating System: CentOS Linux 7 (Core) CPE OS Name: cpe:/o:centos:centos:7 Kernel: Linux 3.10.0-957.5.1.el7.x86_64 Architecture: x86-64 node 1 controller/network node 2 compute01 node 3 compute02 node 4 cinder storage [root@controller01 neutron(keystone_admin)]# neutron-server --version neutron-server 12.0.5 root@controller01 neutron(keystone_admin)]# ovs-vsctl show 96de914b-630f-4014-b738-e149ee385b15 Manager "ptcp:6640:127.0.0.1" is_connected: true Bridge "br-eth1" Controller "tcp:127.0.0.1:6633" is_connected: true fail_mode: secure Port "br-eth1" Interface "br-eth1" type: internal Port "eth1" Interface "eth1" Port "phy-br-eth1" Interface "phy-br-eth1" type: patch options: {peer="int-br-eth1"} Bridge br-int Controller "tcp:127.0.0.1:6633" is_connected: true fail_mode: secure Port "int-br-eth1" Interface "int-br-eth1" type: patch options: {peer="phy-br-eth1"} Port patch-tun Interface patch-tun type: patch options: {peer=patch-int} Port "tap5d460e96-f2" tag: 1 Interface "tap5d460e96-f2" type: internal Port br-int Interface br-int type: internal Port "qg-96178c89-7a" tag: 1 Interface "qg-96178c89-7a" type: internal Port "qr-232080af-bb" tag: 2 Interface "qr-232080af-bb" type: internal Port "tap31ad97cd-15" tag: 2 Interface "tap31ad97cd-15" type: internal Bridge br-ex Port "eth0" Interface "eth0" Port br-ex Interface br-ex type: internal Bridge br-tun Controller "tcp:127.0.0.1:6633" is_connected: true fail_mode: secure Port patch-int Interface patch-int type: patch options: {peer=patch-tun} Port br-tun Interface br-tun type: internal Port "vxlan-c0a8015c" Interface "vxlan-c0a8015c" type: vxlan options: {df_default="true", in_key=flow, local_ip="192.168.1.90", out_key=flow, remote_ip="192.168.1.92"} Port "vxlan-c0a8015b" Interface "vxlan-c0a8015b" type: vxlan options: {df_default="true", in_key=flow, local_ip="192.168.1.90", out_key=flow, remote_ip="192.168.1.91"} ovs_version: "2.9.0" floating IP network = 192.168.30.0/24 moment interface network on all nodes = 192.168.1.0/24 tenant network = 10.10.1.0/24 [root@controller01 neutron(keystone_admin)]# ip netns exec qrouter-c2d1460b-3585-4d37-a782-0ae4a713738b route -n Kernel IP routing table Destination Gateway Genmask Flags Metric Ref Use Iface 0.0.0.0 192.168.1.1 0.0.0.0 UG 0 0 0 qg-96178c89-7a 10.10.1.0 0.0.0.0 255.255.255.0 U 0 0 0 qr-232080af-bb 192.168.1.1 0.0.0.0 255.255.255.255 UH 0 0 0 qg-96178c89-7a 192.168.30.0 0.0.0.0 255.255.255.0 U 0 0 0 qg-96178c89-7a [root@controller01 neutron(keystone_admin)]# openstack server list +--------------------------------------+-------------+--------+----------------------------------------------+--------------+---------+ | ID | Name | Status | Networks | Image | Flavor | +--------------------------------------+-------------+--------+----------------------------------------------+--------------+---------+ | e6e950c0-6efd-4b9d-913c-5736016e6a2a | Test-Cirros | ACTIVE | kubernetes-network=10.10.1.14, 192.168.30.57 | cirros image | m1.tiny | +--------------------------------------+-------------+--------+----------------------------------------------+--------------+---------+ What's interesting, is that from the qrouter-c2d1460b-3585-4d37-a782-0ae4a713738b we can ping the floating IP of the Test-Cirros VM, however we just cannot ping out side to any 192.168.1.0/24 IP. [root@controller01 neutron(keystone_admin)]# ip netns exec qrouter-c2d1460b-3585-4d37-a782-0ae4a713738b ping -c4 192.168.30.57 PING 192.168.30.57 (192.168.30.57) 56(84) bytes of data. 64 bytes from 192.168.30.57: icmp_seq=1 ttl=64 time=2.02 ms 64 bytes from 192.168.30.57: icmp_seq=2 ttl=64 time=0.500 ms 64 bytes from 192.168.30.57: icmp_seq=3 ttl=64 time=0.528 ms 64 bytes from 192.168.30.57: icmp_seq=4 ttl=64 time=0.583 ms ** Affects: neutron Importance: Undecided Status: New -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to neutron. https://bugs.launchpad.net/bugs/1818383 Title: neutron not allowing access to external network Status in neutron: New Bug description: We did a 4 node bare metal OpenStack Queens install. After setting up networking and adding eth0 to br-ex and restarting network service we cannot ping from qrouter to external floating IP network. Below layout of the 4 node setup and our ovs db info. This was a fresh install using PackStack script modify to prep all nodes except the storage node. CentOS 7 OpenStack Queens release static hostname: controller01 Icon name: computer Machine ID: 0f62242dd7f04961b2fa642777708526 Boot ID: 1bf746fe751f4e58902431573696f31e Operating System: CentOS Linux 7 (Core) CPE OS Name: cpe:/o:centos:centos:7 Kernel: Linux 3.10.0-957.5.1.el7.x86_64 Architecture: x86-64 node 1 controller/network node 2 compute01 node 3 compute02 node 4 cinder storage [root@controller01 neutron(keystone_admin)]# neutron-server --version neutron-server 12.0.5 root@controller01 neutron(keystone_admin)]# ovs-vsctl show 96de914b-630f-4014-b738-e149ee385b15 Manager "ptcp:6640:127.0.0.1" is_connected: true Bridge "br-eth1" Controller "tcp:127.0.0.1:6633" is_connected: true fail_mode: secure Port "br-eth1" Interface "br-eth1" type: internal Port "eth1" Interface "eth1" Port "phy-br-eth1" Interface "phy-br-eth1" type: patch options: {peer="int-br-eth1"} Bridge br-int Controller "tcp:127.0.0.1:6633" is_connected: true fail_mode: secure Port "int-br-eth1" Interface "int-br-eth1" type: patch options: {peer="phy-br-eth1"} Port patch-tun Interface patch-tun type: patch options: {peer=patch-int} Port "tap5d460e96-f2" tag: 1 Interface "tap5d460e96-f2" type: internal Port br-int Interface br-int type: internal Port "qg-96178c89-7a" tag: 1 Interface "qg-96178c89-7a" type: internal Port "qr-232080af-bb" tag: 2 Interface "qr-232080af-bb" type: internal Port "tap31ad97cd-15" tag: 2 Interface "tap31ad97cd-15" type: internal Bridge br-ex Port "eth0" Interface "eth0" Port br-ex Interface br-ex type: internal Bridge br-tun Controller "tcp:127.0.0.1:6633" is_connected: true fail_mode: secure Port patch-int Interface patch-int type: patch options: {peer=patch-tun} Port br-tun Interface br-tun type: internal Port "vxlan-c0a8015c" Interface "vxlan-c0a8015c" type: vxlan options: {df_default="true", in_key=flow, local_ip="192.168.1.90", out_key=flow, remote_ip="192.168.1.92"} Port "vxlan-c0a8015b" Interface "vxlan-c0a8015b" type: vxlan options: {df_default="true", in_key=flow, local_ip="192.168.1.90", out_key=flow, remote_ip="192.168.1.91"} ovs_version: "2.9.0" floating IP network = 192.168.30.0/24 moment interface network on all nodes = 192.168.1.0/24 tenant network = 10.10.1.0/24 [root@controller01 neutron(keystone_admin)]# ip netns exec qrouter-c2d1460b-3585-4d37-a782-0ae4a713738b route -n Kernel IP routing table Destination Gateway Genmask Flags Metric Ref Use Iface 0.0.0.0 192.168.1.1 0.0.0.0 UG 0 0 0 qg-96178c89-7a 10.10.1.0 0.0.0.0 255.255.255.0 U 0 0 0 qr-232080af-bb 192.168.1.1 0.0.0.0 255.255.255.255 UH 0 0 0 qg-96178c89-7a 192.168.30.0 0.0.0.0 255.255.255.0 U 0 0 0 qg-96178c89-7a [root@controller01 neutron(keystone_admin)]# openstack server list +--------------------------------------+-------------+--------+----------------------------------------------+--------------+---------+ | ID | Name | Status | Networks | Image | Flavor | +--------------------------------------+-------------+--------+----------------------------------------------+--------------+---------+ | e6e950c0-6efd-4b9d-913c-5736016e6a2a | Test-Cirros | ACTIVE | kubernetes-network=10.10.1.14, 192.168.30.57 | cirros image | m1.tiny | +--------------------------------------+-------------+--------+----------------------------------------------+--------------+---------+ What's interesting, is that from the qrouter-c2d1460b-3585-4d37-a782-0ae4a713738b we can ping the floating IP of the Test-Cirros VM, however we just cannot ping out side to any 192.168.1.0/24 IP. [root@controller01 neutron(keystone_admin)]# ip netns exec qrouter-c2d1460b-3585-4d37-a782-0ae4a713738b ping -c4 192.168.30.57 PING 192.168.30.57 (192.168.30.57) 56(84) bytes of data. 64 bytes from 192.168.30.57: icmp_seq=1 ttl=64 time=2.02 ms 64 bytes from 192.168.30.57: icmp_seq=2 ttl=64 time=0.500 ms 64 bytes from 192.168.30.57: icmp_seq=3 ttl=64 time=0.528 ms 64 bytes from 192.168.30.57: icmp_seq=4 ttl=64 time=0.583 ms To manage notifications about this bug go to: https://bugs.launchpad.net/neutron/+bug/1818383/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp