Public bug reported: [Summary] the container can not be pinged via name space, after 860 tenants/networks/container created
[Topo] 1 controller, 2 network nodes, 6 compute nodes, all in ubuntu 14.04 (openstack version is 2015.1.2, linux kernel version is 3.19.0-31) root@ah:~# uname -a Linux ah.container13 3.19.0-31-generic #36~14.04.1-Ubuntu SMP Thu Oct 8 10:21:08 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux root@ah:~# root@ah:~# dpkg -l | grep neutron ii neutron-common 1:2015.1.2-0ubuntu1~cloud0 all Neutron is a virtual network service for Openstack - common ii neutron-plugin-ml2 1:2015.1.2-0ubuntu1~cloud0 all Neutron is a virtual network service for Openstack - ML2 plugin ii neutron-server 1:2015.1.2-0ubuntu1~cloud0 all Neutron is a virtual network service for Openstack - server ii python-neutron 1:2015.1.2-0ubuntu1~cloud0 all Neutron is a virtual network service for Openstack - Python library ii python-neutron-fwaas 2015.1.2-0ubuntu2~cloud0 all Firewall-as-a-Service driver for OpenStack Neutron ii python-neutronclient 1:2.3.11-0ubuntu1.2~cloud0 all client - Neutron is a virtual network service for Openstack root@ah:~# [Description and expect result] the container should be pinged via name space [Reproduceable or not] reproducible intermittently when large number of tenant/network/instance configed [Recreate Steps] 1)use script to create: 860 tenants, 1 network/router in each tenant, 1 cirros container in each network, all containers are associate to FIP 2)create one more tenant, 1 network/contaier in the tenant, the container can be in Active state, but can not be pinged via name space >>>>ISSUE [Configration] config files on controller/network/compute are attached [logs] instance can be in Active state: oot@ah:~# nova --os-tenant-id 73731bbaf2db48f89a067604e3556e05 list +--------------------------------------+-----------------------------+--------+------------+-------------+----------------------------------------------------+ | ID | Name | Status | Task State | Power State | Networks | +--------------------------------------+-----------------------------+--------+------------+-------------+----------------------------------------------------+ | d5ba18d5-aaf9-4ed6-9a2b-71d2b2f10bae | mexico_test_new_2_1_net1_vm | ACTIVE | - | Running | mexico_test_new_2_1_net1=10.10.32.3, 172.168.6.211 | +--------------------------------------+-----------------------------+--------+------------+-------------+----------------------------------------------------+ root@ah:~# keystone tenant-list | grep test_new_2_1 | 73731bbaf2db48f89a067604e3556e05 | mexico_test_new_2_1 | True | root@ah:~# neutron net-list | grep exico_test_new_2_1_net1 | a935642d-b56c-4a87-83c5-755f01bf0814 | mexico_test_new_2_1_net1 | bed0330f-e0ea-4bcc-bc75-96766dad32a7 10.10.32.0/24 | root@ah:~# on network node: root@ah:~# ip netns | grep a935642d-b56c-4a87-83c5-755f01bf0814 qdhcp-a935642d-b56c-4a87-83c5-755f01bf0814 root@ah:~# ip netns exec qdhcp-a935642d-b56c-4a87-83c5-755f01bf0814 ping 10.10.32.3 PING 10.10.32.3 (10.10.32.3) 56(84) bytes of data. >From 10.10.32.2 icmp_seq=1 Destination Host Unreachable >From 10.10.32.2 icmp_seq=2 Destination Host Unreachable >From 10.10.32.2 icmp_seq=3 Destination Host Unreachable >From 10.10.32.2 icmp_seq=4 Destination Host Unreachable >>>ISSUE [Root cause anlyze or debug inf] high load on controller and network node [Attachment] log files on controller/network/compute are attached ** Affects: neutron Importance: Undecided Status: New ** Attachment added: "log files and config files on controller/network/compute are attached" https://bugs.launchpad.net/bugs/1513048/+attachment/4512713/+files/log_config.rar -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to neutron. https://bugs.launchpad.net/bugs/1513048 Title: the container can not be pinged via name space, after 860 tenants/networks/container created Status in neutron: New Bug description: [Summary] the container can not be pinged via name space, after 860 tenants/networks/container created [Topo] 1 controller, 2 network nodes, 6 compute nodes, all in ubuntu 14.04 (openstack version is 2015.1.2, linux kernel version is 3.19.0-31) root@ah:~# uname -a Linux ah.container13 3.19.0-31-generic #36~14.04.1-Ubuntu SMP Thu Oct 8 10:21:08 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux root@ah:~# root@ah:~# dpkg -l | grep neutron ii neutron-common 1:2015.1.2-0ubuntu1~cloud0 all Neutron is a virtual network service for Openstack - common ii neutron-plugin-ml2 1:2015.1.2-0ubuntu1~cloud0 all Neutron is a virtual network service for Openstack - ML2 plugin ii neutron-server 1:2015.1.2-0ubuntu1~cloud0 all Neutron is a virtual network service for Openstack - server ii python-neutron 1:2015.1.2-0ubuntu1~cloud0 all Neutron is a virtual network service for Openstack - Python library ii python-neutron-fwaas 2015.1.2-0ubuntu2~cloud0 all Firewall-as-a-Service driver for OpenStack Neutron ii python-neutronclient 1:2.3.11-0ubuntu1.2~cloud0 all client - Neutron is a virtual network service for Openstack root@ah:~# [Description and expect result] the container should be pinged via name space [Reproduceable or not] reproducible intermittently when large number of tenant/network/instance configed [Recreate Steps] 1)use script to create: 860 tenants, 1 network/router in each tenant, 1 cirros container in each network, all containers are associate to FIP 2)create one more tenant, 1 network/contaier in the tenant, the container can be in Active state, but can not be pinged via name space >>>>ISSUE [Configration] config files on controller/network/compute are attached [logs] instance can be in Active state: oot@ah:~# nova --os-tenant-id 73731bbaf2db48f89a067604e3556e05 list +--------------------------------------+-----------------------------+--------+------------+-------------+----------------------------------------------------+ | ID | Name | Status | Task State | Power State | Networks | +--------------------------------------+-----------------------------+--------+------------+-------------+----------------------------------------------------+ | d5ba18d5-aaf9-4ed6-9a2b-71d2b2f10bae | mexico_test_new_2_1_net1_vm | ACTIVE | - | Running | mexico_test_new_2_1_net1=10.10.32.3, 172.168.6.211 | +--------------------------------------+-----------------------------+--------+------------+-------------+----------------------------------------------------+ root@ah:~# keystone tenant-list | grep test_new_2_1 | 73731bbaf2db48f89a067604e3556e05 | mexico_test_new_2_1 | True | root@ah:~# neutron net-list | grep exico_test_new_2_1_net1 | a935642d-b56c-4a87-83c5-755f01bf0814 | mexico_test_new_2_1_net1 | bed0330f-e0ea-4bcc-bc75-96766dad32a7 10.10.32.0/24 | root@ah:~# on network node: root@ah:~# ip netns | grep a935642d-b56c-4a87-83c5-755f01bf0814 qdhcp-a935642d-b56c-4a87-83c5-755f01bf0814 root@ah:~# ip netns exec qdhcp-a935642d-b56c-4a87-83c5-755f01bf0814 ping 10.10.32.3 PING 10.10.32.3 (10.10.32.3) 56(84) bytes of data. From 10.10.32.2 icmp_seq=1 Destination Host Unreachable From 10.10.32.2 icmp_seq=2 Destination Host Unreachable From 10.10.32.2 icmp_seq=3 Destination Host Unreachable From 10.10.32.2 icmp_seq=4 Destination Host Unreachable >>>ISSUE [Root cause anlyze or debug inf] high load on controller and network node [Attachment] log files on controller/network/compute are attached To manage notifications about this bug go to: https://bugs.launchpad.net/neutron/+bug/1513048/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp