** Also affects: nova (Ubuntu Groovy) Importance: High Status: Fix Released
** Also affects: glance (Ubuntu Groovy) Importance: High Status: Fix Released ** Also affects: cinder (Ubuntu Groovy) Importance: High Status: Triaged ** Also affects: neutron (Ubuntu Groovy) Importance: High Status: Triaged ** Also affects: nova (Ubuntu Focal) Importance: Undecided Status: New ** Also affects: glance (Ubuntu Focal) Importance: Undecided Status: New ** Also affects: cinder (Ubuntu Focal) Importance: Undecided Status: New ** Also affects: neutron (Ubuntu Focal) Importance: Undecided Status: New ** Changed in: neutron (Ubuntu Focal) Importance: Undecided => High ** Changed in: neutron (Ubuntu Focal) Status: New => Triaged ** Changed in: cinder (Ubuntu Focal) Status: New => Fix Released ** Changed in: cinder (Ubuntu Focal) Importance: Undecided => High ** Changed in: cinder (Ubuntu Focal) Status: Fix Released => Triaged ** Changed in: glance (Ubuntu Focal) Status: New => Fix Released ** Changed in: nova (Ubuntu Focal) Status: New => Fix Released -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to Glance. https://bugs.launchpad.net/bugs/1863021 Title: [SRU] eventlet monkey patch results in assert len(_active) == 1 AssertionError Status in Cinder: In Progress Status in Glance: Fix Released Status in neutron: In Progress Status in OpenStack Compute (nova): Fix Released Status in cinder package in Ubuntu: Triaged Status in glance package in Ubuntu: Fix Released Status in neutron package in Ubuntu: Triaged Status in nova package in Ubuntu: Fix Released Status in cinder source package in Focal: Triaged Status in glance source package in Focal: Fix Released Status in neutron source package in Focal: Triaged Status in nova source package in Focal: Fix Released Status in cinder source package in Groovy: Triaged Status in glance source package in Groovy: Fix Released Status in neutron source package in Groovy: Triaged Status in nova source package in Groovy: Fix Released Bug description: [Impact] This appears to be the same issue documented here: https://github.com/eventlet/eventlet/issues/592 However I seem to only hit this with python3.8. Basically nova, glance, neutron, and cinder services fail with: Exception ignored in: <function _after_fork at 0x7f1100d075e0> Traceback (most recent call last): File "/usr/lib/python3.8/threading.py", line 1454, in _after_fork assert len(_active) == 1 AssertionError: Exception ignored in: <function _after_fork at 0x7f1100d075e0> Traceback (most recent call last): File "/usr/lib/python3.8/threading.py", line 1454, in _after_fork assert len(_active) == 1 AssertionError: Patching nova/monkey_patch.py with the following appears to fix this: diff --git a/nova/monkey_patch.py b/nova/monkey_patch.py index a07ff91dac..bb7252c643 100644 --- a/nova/monkey_patch.py +++ b/nova/monkey_patch.py @@ -59,6 +59,9 @@ def _monkey_patch(): else: eventlet.monkey_patch() + import __original_module_threading + import threading + __original_module_threading.current_thread.__globals__['_active'] = threading._active # NOTE(rpodolyaka): import oslo_service first, so that it makes eventlet # hub use a monotonic clock to avoid issues with drifts of system time (see Similar patches to glance/cmd/api.py, glance/cmd/scrubber.py and glance/cmd/registry.py appears to fix it for glance. [Test Case] 'systemctl status <service>' should not show the failure reported in [Impact]. [Regression Potential] Should be low. The fix provided is a well-known solution to the problem. Openstack deployment + tempest regression testing will be performed to help reduce regression potential. To manage notifications about this bug go to: https://bugs.launchpad.net/cinder/+bug/1863021/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp