Public bug reported: Description =========== when instance in block_device_mapping task state, can't force-delete it by nova force-delete command.
Steps to reproduce ================== * Boot an instance, as soon as instance task state changed to block_device_mapping, run nova force-delete INSTANCE_UUID command, get some error: # nova force-delete 08ed7e71-4470-4746-ae65-d17c659f29c9 /usr/lib/python2.7/site-packages/novaclient/client.py:278: UserWarning: The 'tenant_id' argument is deprecated in Ocata and its use may result in errors in future releases. As 'project_id' is provided, the 'tenant_id' argument will be ignored. warnings.warn(msg) ERROR (ClientException): Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible. <class 'nova.exception.InstanceInvalidState'> (HTTP 500) (Request-ID: req-b185ecd7-5f0d-441e-8263-59af8b66aae7) Expected result =============== force delete instance sucessfully Actual result ============= get some error and force delete instance failed Environment =========== # rpm -qa | grep nova openstack-nova-api-15.0.0-1.el7.noarch openstack-nova-console-15.0.0-1.el7.noarch openstack-nova-common-15.0.0-1.el7.noarch openstack-nova-conductor-15.0.0-1.el7.noarch openstack-nova-novncproxy-15.0.0-1.el7.noarch puppet-nova-10.3.0-1.el7.noarch python-nova-15.0.0-1.el7.noarch openstack-nova-cert-15.0.0-1.el7.noarch openstack-nova-placement-api-15.0.0-1.el7.noarch openstack-nova-compute-15.0.0-1.el7.noarch openstack-nova-scheduler-15.0.0-1.el7.noarch python2-novaclient-7.1.0-1.el7.noarch Logs & Configs ============== 2017-07-18 15:47:22.159 36543 DEBUG nova.api.openstack.wsgi [req-b185ecd7-5f0d-441e-8263-59af8b66aae7 - - - - -] Action: 'action', calling method: <bound method DeferredDeleteController._force_delete of <nova.api.openstack.compute.deferred_delete.DeferredDeleteController object at 0x83f4c10>>, body: {"forceDelete": null} _process_stack /usr/lib/python2.7/site-packages/nova/api/openstack/wsgi.py:623 2017-07-18 15:47:22.159 36543 DEBUG nova.compute.api [req-b185ecd7-5f0d-441e-8263-59af8b66aae7 - - - - -] [instance: 08ed7e71-4470-4746-ae65-d17c659f29c9] Fetching instance by UUID get /usr/lib/python2.7/site-packages/nova/compute/api.py:2313 2017-07-18 15:47:22.172 36543 DEBUG oslo_db.sqlalchemy.engines [req-b185ecd7-5f0d-441e-8263-59af8b66aae7 - - - - -] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION _check_effective_sql_mode /usr/lib/python2.7/site-packages/oslo_db/sqlalchemy/engines.py:261 2017-07-18 15:47:22.212 36543 ERROR nova.api.openstack.extensions [req-b185ecd7-5f0d-441e-8263-59af8b66aae7 - - - - -] Unexpected exception in API method 2017-07-18 15:47:22.212 36543 ERROR nova.api.openstack.extensions Traceback (most recent call last): 2017-07-18 15:47:22.212 36543 ERROR nova.api.openstack.extensions File "/usr/lib/python2.7/site-packages/nova/api/openstack/extensions.py", line 338, in wrapped 2017-07-18 15:47:22.212 36543 ERROR nova.api.openstack.extensions return f(*args, **kwargs) 2017-07-18 15:47:22.212 36543 ERROR nova.api.openstack.extensions File "/usr/lib/python2.7/site-packages/nova/api/openstack/compute/deferred_delete.py", line 64, in _force_delete 2017-07-18 15:47:22.212 36543 ERROR nova.api.openstack.extensions self.compute_api.force_delete(context, instance) 2017-07-18 15:47:22.212 36543 ERROR nova.api.openstack.extensions File "/usr/lib/python2.7/site-packages/nova/compute/api.py", line 170, in inner 2017-07-18 15:47:22.212 36543 ERROR nova.api.openstack.extensions return function(self, context, instance, *args, **kwargs) 2017-07-18 15:47:22.212 36543 ERROR nova.api.openstack.extensions File "/usr/lib/python2.7/site-packages/nova/compute/api.py", line 143, in inner 2017-07-18 15:47:22.212 36543 ERROR nova.api.openstack.extensions method=f.__name__) 2017-07-18 15:47:22.212 36543 ERROR nova.api.openstack.extensions InstanceInvalidState: Instance 08ed7e71-4470-4746-ae65-d17c659f29c9 in task_state block_device_mapping. Cannot force_delete while the instance is in this state. 2017-07-18 15:47:22.212 36543 ERROR nova.api.openstack.extensions 2017-07-18 15:47:22.219 36543 INFO nova.api.openstack.wsgi [req-b185ecd7-5f0d-441e-8263-59af8b66aae7 - - - - -] HTTP exception thrown: Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible. <class 'nova.exception.InstanceInvalidState'> 2017-07-18 15:47:22.220 36543 DEBUG nova.api.openstack.wsgi [req-b185ecd7-5f0d-441e-8263-59af8b66aae7 - - - - -] Returning 500 to user: Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible. <class 'nova.exception.InstanceInvalidState'> __call__ /usr/lib/python2.7/site-packages/nova/api/openstack/wsgi.py:1039 2017-07-18 15:47:22.221 36543 INFO nova.osapi_compute.wsgi.server [req-b185ecd7-5f0d-441e-8263-59af8b66aae7 - - - - -] 192.168.2.145 "POST /v2.1/227c2369e7e9475f93c3ec7f07082937/servers/08ed7e71-4470-4746-ae65-d17c659f29c9/action HTTP/1.1" status: 500 len: 578 time: 0.0697980 ** Affects: nova Importance: Undecided Status: New -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1704945 Title: nova force-delete can't delete instance in vm_state block_device_mapping Status in OpenStack Compute (nova): New Bug description: Description =========== when instance in block_device_mapping task state, can't force-delete it by nova force-delete command. Steps to reproduce ================== * Boot an instance, as soon as instance task state changed to block_device_mapping, run nova force-delete INSTANCE_UUID command, get some error: # nova force-delete 08ed7e71-4470-4746-ae65-d17c659f29c9 /usr/lib/python2.7/site-packages/novaclient/client.py:278: UserWarning: The 'tenant_id' argument is deprecated in Ocata and its use may result in errors in future releases. As 'project_id' is provided, the 'tenant_id' argument will be ignored. warnings.warn(msg) ERROR (ClientException): Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible. <class 'nova.exception.InstanceInvalidState'> (HTTP 500) (Request-ID: req-b185ecd7-5f0d-441e-8263-59af8b66aae7) Expected result =============== force delete instance sucessfully Actual result ============= get some error and force delete instance failed Environment =========== # rpm -qa | grep nova openstack-nova-api-15.0.0-1.el7.noarch openstack-nova-console-15.0.0-1.el7.noarch openstack-nova-common-15.0.0-1.el7.noarch openstack-nova-conductor-15.0.0-1.el7.noarch openstack-nova-novncproxy-15.0.0-1.el7.noarch puppet-nova-10.3.0-1.el7.noarch python-nova-15.0.0-1.el7.noarch openstack-nova-cert-15.0.0-1.el7.noarch openstack-nova-placement-api-15.0.0-1.el7.noarch openstack-nova-compute-15.0.0-1.el7.noarch openstack-nova-scheduler-15.0.0-1.el7.noarch python2-novaclient-7.1.0-1.el7.noarch Logs & Configs ============== 2017-07-18 15:47:22.159 36543 DEBUG nova.api.openstack.wsgi [req-b185ecd7-5f0d-441e-8263-59af8b66aae7 - - - - -] Action: 'action', calling method: <bound method DeferredDeleteController._force_delete of <nova.api.openstack.compute.deferred_delete.DeferredDeleteController object at 0x83f4c10>>, body: {"forceDelete": null} _process_stack /usr/lib/python2.7/site-packages/nova/api/openstack/wsgi.py:623 2017-07-18 15:47:22.159 36543 DEBUG nova.compute.api [req-b185ecd7-5f0d-441e-8263-59af8b66aae7 - - - - -] [instance: 08ed7e71-4470-4746-ae65-d17c659f29c9] Fetching instance by UUID get /usr/lib/python2.7/site-packages/nova/compute/api.py:2313 2017-07-18 15:47:22.172 36543 DEBUG oslo_db.sqlalchemy.engines [req-b185ecd7-5f0d-441e-8263-59af8b66aae7 - - - - -] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION _check_effective_sql_mode /usr/lib/python2.7/site-packages/oslo_db/sqlalchemy/engines.py:261 2017-07-18 15:47:22.212 36543 ERROR nova.api.openstack.extensions [req-b185ecd7-5f0d-441e-8263-59af8b66aae7 - - - - -] Unexpected exception in API method 2017-07-18 15:47:22.212 36543 ERROR nova.api.openstack.extensions Traceback (most recent call last): 2017-07-18 15:47:22.212 36543 ERROR nova.api.openstack.extensions File "/usr/lib/python2.7/site-packages/nova/api/openstack/extensions.py", line 338, in wrapped 2017-07-18 15:47:22.212 36543 ERROR nova.api.openstack.extensions return f(*args, **kwargs) 2017-07-18 15:47:22.212 36543 ERROR nova.api.openstack.extensions File "/usr/lib/python2.7/site-packages/nova/api/openstack/compute/deferred_delete.py", line 64, in _force_delete 2017-07-18 15:47:22.212 36543 ERROR nova.api.openstack.extensions self.compute_api.force_delete(context, instance) 2017-07-18 15:47:22.212 36543 ERROR nova.api.openstack.extensions File "/usr/lib/python2.7/site-packages/nova/compute/api.py", line 170, in inner 2017-07-18 15:47:22.212 36543 ERROR nova.api.openstack.extensions return function(self, context, instance, *args, **kwargs) 2017-07-18 15:47:22.212 36543 ERROR nova.api.openstack.extensions File "/usr/lib/python2.7/site-packages/nova/compute/api.py", line 143, in inner 2017-07-18 15:47:22.212 36543 ERROR nova.api.openstack.extensions method=f.__name__) 2017-07-18 15:47:22.212 36543 ERROR nova.api.openstack.extensions InstanceInvalidState: Instance 08ed7e71-4470-4746-ae65-d17c659f29c9 in task_state block_device_mapping. Cannot force_delete while the instance is in this state. 2017-07-18 15:47:22.212 36543 ERROR nova.api.openstack.extensions 2017-07-18 15:47:22.219 36543 INFO nova.api.openstack.wsgi [req-b185ecd7-5f0d-441e-8263-59af8b66aae7 - - - - -] HTTP exception thrown: Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible. <class 'nova.exception.InstanceInvalidState'> 2017-07-18 15:47:22.220 36543 DEBUG nova.api.openstack.wsgi [req-b185ecd7-5f0d-441e-8263-59af8b66aae7 - - - - -] Returning 500 to user: Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible. <class 'nova.exception.InstanceInvalidState'> __call__ /usr/lib/python2.7/site-packages/nova/api/openstack/wsgi.py:1039 2017-07-18 15:47:22.221 36543 INFO nova.osapi_compute.wsgi.server [req-b185ecd7-5f0d-441e-8263-59af8b66aae7 - - - - -] 192.168.2.145 "POST /v2.1/227c2369e7e9475f93c3ec7f07082937/servers/08ed7e71-4470-4746-ae65-d17c659f29c9/action HTTP/1.1" status: 500 len: 578 time: 0.0697980 To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1704945/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
