Related fix for oslo.service: https://review.openstack.org/#/c/190175/
** Changed in: oslo.service Status: Confirmed => Fix Released -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1382390 Title: nova-api should shutdown gracefully Status in OpenStack Compute (Nova): In Progress Status in The Oslo library incubator: Confirmed Status in Library for running OpenStack services: Fix Released Bug description: In IceHouse, An awesome feature got implemented: graceful shutdown nova service, which can make sure in-process RPC request got done before kill the process. But nova-api not support graceful shutdown now, which can cause problem when do upgrading. For example, when a request to create an instance was in-progress, kill the nova-api may lead to quota not sync or odd database records. Especially in large-scale development, there are hundreds of request in a second, kill the nova-api will interrupt lots in-process greenlet. In nova/wsgi.py, when stoping WSGI service, we first shrink the greenlet pool size to 0, then kill the eventlet wsgi server. The work around is quick and easy: wait for all greenlets in the pool to finish, then kill wsgi server. The code looks like below: diff --git a/nova/wsgi.py b/nova/wsgi.py index ba52872..3c89297 100644 --- a/nova/wsgi.py +++ b/nova/wsgi.py @@ -212,6 +212,9 @@ class Server(object): if self._server is not None: # Resize pool to stop new requests from being processed self._pool.resize(0) + num = self._pool.running() + LOG.info(_("Waiting WSGI server to finish %d requests." % num)) + self._pool.waitall() self._server.kill() def wait(self): To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1382390/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp