Public bug reported: When doing volume-attachemnts, nova-api does an RPC-call (with a long_rpc_timeout) into nova-compute to reserve_block_device_name(). This takes a lock on the instance. If a volume-attachment was in progress, which also takes the instance-lock, nova-api's RPC-call needs to wait.
Having RPC-calls in nova-api, that can take a long time, will block the process handling the request. If a project does a lot of volume- attachments (e.g. for a k8s workload > 10 attachments per instance), this could starve out other users of nova-api by occupying all available processes. When running nova-api with eventlet, a small number of processes can handle a lot of requests in parallel and some blocking rpc-calls don't matter too much. When switching to uWSGI, the number of processes would have to be increased drastically to accommodate for that - unless it's possible to map those requests to threads and use a high number of threads instead. What's the recommended way to run nova-api on uWSGI to handle this? Low number of processes with high number of threads to mimic eventlet? ** Affects: nova Importance: Undecided Status: New -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1930406 Title: parallel volume-attachment requests might starve out nova-api for others Status in OpenStack Compute (nova): New Bug description: When doing volume-attachemnts, nova-api does an RPC-call (with a long_rpc_timeout) into nova-compute to reserve_block_device_name(). This takes a lock on the instance. If a volume-attachment was in progress, which also takes the instance-lock, nova-api's RPC-call needs to wait. Having RPC-calls in nova-api, that can take a long time, will block the process handling the request. If a project does a lot of volume- attachments (e.g. for a k8s workload > 10 attachments per instance), this could starve out other users of nova-api by occupying all available processes. When running nova-api with eventlet, a small number of processes can handle a lot of requests in parallel and some blocking rpc-calls don't matter too much. When switching to uWSGI, the number of processes would have to be increased drastically to accommodate for that - unless it's possible to map those requests to threads and use a high number of threads instead. What's the recommended way to run nova-api on uWSGI to handle this? Low number of processes with high number of threads to mimic eventlet? To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1930406/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp