Reviewed: https://review.openstack.org/386656
Committed:
https://git.openstack.org/cgit/openstack/oslo.messaging/commit/?id=a6c193f3eba62cdcbfe04d0fa93e95352bcfb1c3
Submitter: Jenkins
Branch: master
commit a6c193f3eba62cdcbfe04d0fa93e95352bcfb1c3
Author: John Eckersberg <jecke...@redhat.com>
Date: Fri Oct 14 11:02:47 2016 -0400
rabbit: Avoid busy loop on epoll_wait with heartbeat+eventlet
Calling threading.Event.wait() when using eventlet results in a busy
loop calling epoll_wait, because the Python 2.x
threading.Condition.wait() implementation busy-waits by calling
sleep() with very small values (0.0005..0.05s). Because sleep() is
monkey-patched by eventlet, this results in many very short timers
being added to the eventlet hub, and forces eventlet to constantly
epoll_wait looking for new data unecessarily.
This utilizes a new Event from eventletutils which conditionalizes the
event primitive depending on whether or not eventlet is being used.
If it is, eventlet.event.Event is used instead of threading.Event.
The eventlet.event.Event implementation does not suffer from the same
busy-wait sleep problem. If eventlet is not used, the previous
behavior is retained.
Change-Id: I5c211092d282e724d1c87ce4d06b6c44b592e764
Depends-On: Id33c9f8c17102ba1fe24c12b053c336b6d265501
Closes-bug: #1518430
** Changed in: oslo.messaging
Status: In Progress => Fix Released
--
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1518430
Title:
liberty: ~busy loop on epoll_wait being called with zero timeout
Status in OpenStack Compute (nova):
Invalid
Status in oslo.messaging:
Fix Released
Status in nova package in Ubuntu:
Invalid
Status in python-oslo.messaging package in Ubuntu:
New
Bug description:
Context: openstack juju/maas deploy using 1510 charms release
on trusty, with:
openstack-origin: "cloud:trusty-liberty"
source: "cloud:trusty-updates/liberty
* Several openstack nova- and neutron- services, at least:
nova-compute, neutron-server, nova-conductor,
neutron-openvswitch-agent,neutron-vpn-agent
show almost busy looping on epoll_wait() calls, with zero timeout set
most frequently.
- nova-compute (chose it b/cos single proc'd) strace and ltrace captures:
http://paste.ubuntu.com/13371248/ (ltrace, strace)
As comparison, this is how it looks on a kilo deploy:
- http://paste.ubuntu.com/13371635/
* 'top' sample from a nova-cloud-controller unit from
this completely idle stack:
http://paste.ubuntu.com/13371809/
FYI *not* seeing this behavior on keystone, glance, cinder,
ceilometer-api.
As this issue is present on several components, it likely comes
from common libraries (oslo concurrency?), fyi filed the bug to
nova itself as a starting point for debugging.
Note: The description in the following bug gives a good overview of
the issue and points to a possible fix for oslo.messaging:
https://bugs.launchpad.net/mos/+bug/1380220
To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1518430/+subscriptions
--
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help : https://help.launchpad.net/ListHelp
