Public bug reported:
Description of problem:
When doing mass deletion (more then 64 VM's parallely), some VM's got stuck on
ERROR state with the error:
"Connection to neutron failed: Maximum attempts reached", "code": 500,
"created": "2014-06-10T21:34:38Z"
Version-Release number of selected component (if applicable):
openstack-neutron-openvswitch-2014.1-26.el7ost.noarch
python-neutronclient-2.3.4-2.el7ost.noarch
python-neutron-2014.1-26.el7ost.noarch
openstack-neutron-ml2-2014.1-26.el7ost.noarch
openstack-neutron-2014.1-26.el7ost.noarch
How reproducible:
100%
Steps to Reproduce:
1. Setup an environment with a lot of ACTIVE VM's (more then 64)
2. Run mass deletion
for each in `nova list | grep ACTIVE | cut -d"|" -f3` ; do nova delete
$each & done
Actual results:
[root@cougar16 ~(keystone_stress1)]$ nova list
+--------------------------------------+------------+--------+------------+-------------+-----------------------+
| ID | Name | Status | Task State |
Power State | Networks |
+--------------------------------------+------------+--------+------------+-------------+-----------------------+
| 0731ab43-99da-409a-99b9-627287b0a80a | stress1-42 | ERROR | deleting |
Running | private1=192.168.1.61 |
+--------------------------------------+------------+--------+------------+-------------+-----------------------+
[root@cougar16 ~(keystone_stress1)]$ nova show stress1-42
+--------------------------------------+-----------------------------------------------------------------------------------------------------------------------+
| Property | Value
|
+--------------------------------------+-----------------------------------------------------------------------------------------------------------------------+
| OS-DCF:diskConfig | MANUAL
|
| OS-EXT-AZ:availability_zone | nova
|
| OS-EXT-STS:power_state | 1
|
| OS-EXT-STS:task_state | deleting
|
| OS-EXT-STS:vm_state | error
|
| OS-SRV-USG:launched_at | 2014-06-10T21:28:53.000000
|
| OS-SRV-USG:terminated_at | -
|
| accessIPv4 |
|
| accessIPv6 |
|
| config_drive |
|
| created | 2014-06-10T21:27:33Z
|
| fault | {"message": "Connection to neutron
failed: Maximum attempts reached", "code": 500, "created":
"2014-06-10T21:34:38Z"} |
| flavor | mini (0)
|
| hostId |
0c295b885647eb08a3c04a15eb86f9746430dd635c5f8c6291315508
|
| id | 0731ab43-99da-409a-99b9-627287b0a80a
|
| image | cirros
(ae31ea8c-c5ca-4ca1-9662-3545304d8e79)
|
| key_name | -
|
| metadata | {}
|
| name | stress1-42
|
| os-extended-volumes:volumes_attached | []
|
| private1 network | 192.168.1.61
|
| security_groups | default
|
| status | ERROR
|
| tenant_id | 1f160feb483943249381963bc9e87eee
|
| updated | 2014-06-10T21:34:21Z
|
| user_id | 9630e22ff0bc410aaee6bb217b660fee
|
+--------------------------------------+-----------------------------------------------------------------------------------------------------------------------+
Expected results:
no previously ACTIVE VM's available
Additional info:
1. [root@cougar16 ~(keystone_stress1)]$ nova flavor-show mini
+----------------------------+-------+
| Property | Value |
+----------------------------+-------+
| OS-FLV-DISABLED:disabled | False |
| OS-FLV-EXT-DATA:ephemeral | 0 |
| disk | 0 |
| extra_specs | {} |
| id | 0 |
| name | mini |
| os-flavor-access:is_public | True |
| ram | 64 |
| rxtx_factor | 1.0 |
| swap | |
| vcpus | 1 |
+----------------------------+-------+
2. [root@cougar16 ~(keystone_stress1)]$ nova image-show cirros
+----------------------+--------------------------------------+
| Property | Value |
+----------------------+--------------------------------------+
| OS-EXT-IMG-SIZE:size | 13147648 |
| created | 2014-06-10T18:03:36Z |
| id | ae31ea8c-c5ca-4ca1-9662-3545304d8e79 |
| minDisk | 0 |
| minRam | 0 |
| name | cirros |
| progress | 100 |
| status | ACTIVE |
| updated | 2014-06-10T18:03:44Z |
+----------------------+--------------------------------------+
3. Environment has 2 api workers and 2 rpc workers
** Affects: neutron
Importance: Undecided
Status: New
--
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1328700
Title:
On mass deletion - some VM's stuck on ERROR due to connection failed
to neutron
Status in OpenStack Neutron (virtual network service):
New
Bug description:
Description of problem:
When doing mass deletion (more then 64 VM's parallely), some VM's got stuck
on ERROR state with the error:
"Connection to neutron failed: Maximum attempts reached", "code": 500,
"created": "2014-06-10T21:34:38Z"
Version-Release number of selected component (if applicable):
openstack-neutron-openvswitch-2014.1-26.el7ost.noarch
python-neutronclient-2.3.4-2.el7ost.noarch
python-neutron-2014.1-26.el7ost.noarch
openstack-neutron-ml2-2014.1-26.el7ost.noarch
openstack-neutron-2014.1-26.el7ost.noarch
How reproducible:
100%
Steps to Reproduce:
1. Setup an environment with a lot of ACTIVE VM's (more then 64)
2. Run mass deletion
for each in `nova list | grep ACTIVE | cut -d"|" -f3` ; do nova delete
$each & done
Actual results:
[root@cougar16 ~(keystone_stress1)]$ nova list
+--------------------------------------+------------+--------+------------+-------------+-----------------------+
| ID | Name | Status | Task State |
Power State | Networks |
+--------------------------------------+------------+--------+------------+-------------+-----------------------+
| 0731ab43-99da-409a-99b9-627287b0a80a | stress1-42 | ERROR | deleting |
Running | private1=192.168.1.61 |
+--------------------------------------+------------+--------+------------+-------------+-----------------------+
[root@cougar16 ~(keystone_stress1)]$ nova show stress1-42
+--------------------------------------+-----------------------------------------------------------------------------------------------------------------------+
| Property | Value
|
+--------------------------------------+-----------------------------------------------------------------------------------------------------------------------+
| OS-DCF:diskConfig | MANUAL
|
| OS-EXT-AZ:availability_zone | nova
|
| OS-EXT-STS:power_state | 1
|
| OS-EXT-STS:task_state | deleting
|
| OS-EXT-STS:vm_state | error
|
| OS-SRV-USG:launched_at | 2014-06-10T21:28:53.000000
|
| OS-SRV-USG:terminated_at | -
|
| accessIPv4 |
|
| accessIPv6 |
|
| config_drive |
|
| created | 2014-06-10T21:27:33Z
|
| fault | {"message": "Connection to neutron
failed: Maximum attempts reached", "code": 500, "created":
"2014-06-10T21:34:38Z"} |
| flavor | mini (0)
|
| hostId |
0c295b885647eb08a3c04a15eb86f9746430dd635c5f8c6291315508
|
| id | 0731ab43-99da-409a-99b9-627287b0a80a
|
| image | cirros
(ae31ea8c-c5ca-4ca1-9662-3545304d8e79)
|
| key_name | -
|
| metadata | {}
|
| name | stress1-42
|
| os-extended-volumes:volumes_attached | []
|
| private1 network | 192.168.1.61
|
| security_groups | default
|
| status | ERROR
|
| tenant_id | 1f160feb483943249381963bc9e87eee
|
| updated | 2014-06-10T21:34:21Z
|
| user_id | 9630e22ff0bc410aaee6bb217b660fee
|
+--------------------------------------+-----------------------------------------------------------------------------------------------------------------------+
Expected results:
no previously ACTIVE VM's available
Additional info:
1. [root@cougar16 ~(keystone_stress1)]$ nova flavor-show mini
+----------------------------+-------+
| Property | Value |
+----------------------------+-------+
| OS-FLV-DISABLED:disabled | False |
| OS-FLV-EXT-DATA:ephemeral | 0 |
| disk | 0 |
| extra_specs | {} |
| id | 0 |
| name | mini |
| os-flavor-access:is_public | True |
| ram | 64 |
| rxtx_factor | 1.0 |
| swap | |
| vcpus | 1 |
+----------------------------+-------+
2. [root@cougar16 ~(keystone_stress1)]$ nova image-show cirros
+----------------------+--------------------------------------+
| Property | Value |
+----------------------+--------------------------------------+
| OS-EXT-IMG-SIZE:size | 13147648 |
| created | 2014-06-10T18:03:36Z |
| id | ae31ea8c-c5ca-4ca1-9662-3545304d8e79 |
| minDisk | 0 |
| minRam | 0 |
| name | cirros |
| progress | 100 |
| status | ACTIVE |
| updated | 2014-06-10T18:03:44Z |
+----------------------+--------------------------------------+
3. Environment has 2 api workers and 2 rpc workers
To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/1328700/+subscriptions
--
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help : https://help.launchpad.net/ListHelp
