Public bug reported: When RPC between services fails, because of a bug in the RPC service, network faults etc., the server gets stuck in a task, and never exits it.
For example, when powering up a server, nova-api sets the task_state of the server to 'powering-on', and then sends nova-compute an RPC message to nova-compute. nova-compute is the one responsible for setting the task_state back to NULL after the server was powered-up, but if the RPC message fails to reach nova-compute, the server is forever stuck in 'powering-on'. This can prevent further API operations on the VM and leave a good VMs in non-operable state. The user can usually bypass this manually by running a 'reset-state'. To reproduce: 1. Make RPC messages hang 2. Issue a 'server start' request 3. After request, do a 'server show' - the server is stuck in 'powering-on' forever. This issue was previously reported to the mailing list. Please see this: http://lists.openstack.org/pipermail/openstack-dev/2016-April/092239.html and this http://lists.openstack.org/pipermail/openstack-dev/2016-April/092240.html ** Affects: nova Importance: Undecided Status: New ** Tags: compute -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1571175 Title: RPC failure makes server stuck in a task Status in OpenStack Compute (nova): New Bug description: When RPC between services fails, because of a bug in the RPC service, network faults etc., the server gets stuck in a task, and never exits it. For example, when powering up a server, nova-api sets the task_state of the server to 'powering-on', and then sends nova-compute an RPC message to nova-compute. nova-compute is the one responsible for setting the task_state back to NULL after the server was powered-up, but if the RPC message fails to reach nova-compute, the server is forever stuck in 'powering-on'. This can prevent further API operations on the VM and leave a good VMs in non-operable state. The user can usually bypass this manually by running a 'reset-state'. To reproduce: 1. Make RPC messages hang 2. Issue a 'server start' request 3. After request, do a 'server show' - the server is stuck in 'powering-on' forever. This issue was previously reported to the mailing list. Please see this: http://lists.openstack.org/pipermail/openstack-dev/2016-April/092239.html and this http://lists.openstack.org/pipermail/openstack-dev/2016-April/092240.html To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1571175/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
