Public bug reported:
When doing volume-attachemnts, nova-api does an RPC-call (with a
long_rpc_timeout) into nova-compute to reserve_block_device_name(). This
takes a lock on the instance. If a volume-attachment was in progress,
which also takes the instance-lock, nova-api's RPC-call needs to wait.
Having RPC-calls in nova-api, that can take a long time, will block the
process handling the request. If a project does a lot of volume-
attachments (e.g. for a k8s workload > 10 attachments per instance),
this could starve out other users of nova-api by occupying all available
processes.
When running nova-api with eventlet, a small number of processes can
handle a lot of requests in parallel and some blocking rpc-calls don't
matter too much.
When switching to uWSGI, the number of processes would have to be
increased drastically to accommodate for that - unless it's possible to
map those requests to threads and use a high number of threads instead.
What's the recommended way to run nova-api on uWSGI to handle this? Low
number of processes with high number of threads to mimic eventlet?
** Affects: nova
Importance: Undecided
Status: New
--
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1930406
Title:
parallel volume-attachment requests might starve out nova-api for
others
Status in OpenStack Compute (nova):
New
Bug description:
When doing volume-attachemnts, nova-api does an RPC-call (with a
long_rpc_timeout) into nova-compute to reserve_block_device_name().
This takes a lock on the instance. If a volume-attachment was in
progress, which also takes the instance-lock, nova-api's RPC-call
needs to wait.
Having RPC-calls in nova-api, that can take a long time, will block
the process handling the request. If a project does a lot of volume-
attachments (e.g. for a k8s workload > 10 attachments per instance),
this could starve out other users of nova-api by occupying all
available processes.
When running nova-api with eventlet, a small number of processes can
handle a lot of requests in parallel and some blocking rpc-calls don't
matter too much.
When switching to uWSGI, the number of processes would have to be
increased drastically to accommodate for that - unless it's possible
to map those requests to threads and use a high number of threads
instead.
What's the recommended way to run nova-api on uWSGI to handle this?
Low number of processes with high number of threads to mimic eventlet?
To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1930406/+subscriptions
--
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help : https://help.launchpad.net/ListHelp
