[Yahoo-eng-team] [Bug 1914886] Re: Trunk bridges aren't removed
** Also affects: neutron Importance: Undecided Status: New ** Changed in: neutron Assignee: (unassigned) => Rodolfo Alonso (rodolfo-alonso-hernandez) -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to neutron. https://bugs.launchpad.net/bugs/1914886 Title: Trunk bridges aren't removed Status in neutron: New Status in os-vif: Triaged Bug description: Recently I found out on the devstack that when I have VM with trunk port and some subports connected to it, trunk bridge isn't deleted when VM is migrated to the another host. I think that the same thing will happen when instance will be simply deleted as vif object in case of trunk port (when ML2/ovs is used on Neutron's side) is objects.vif.VIFOpenVSwitch and _unplug method for this type of vif object don't tries to delete bridge at all. To manage notifications about this bug go to: https://bugs.launchpad.net/neutron/+bug/1914886/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1837200] Re: Deleted images info should be obfuscated - OSSN-0075
I believe this was fixed with https://review.opendev.org/c/openstack/glance/+/579507 See also https://specs.openstack.org/openstack/glance-specs/specs/rocky/implemented/glance/mitigate-ossn-0075.html ** Changed in: glance Status: In Progress => Fix Released -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to Glance. https://bugs.launchpad.net/bugs/1837200 Title: Deleted images info should be obfuscated - OSSN-0075 Status in Glance: Fix Released Bug description: Because OSSN-0075 the Cloud Operator may choose to never purge the "images" table. But, regulations/policy may require that deleted data is not kept. For this case the deleted image records need to be obfuscated (except the image id). To manage notifications about this bug go to: https://bugs.launchpad.net/glance/+bug/1837200/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1874705] Re: Websso fails when HTTP_REFERRER that horizon is unable to connect to gets used
** Also affects: keystone Importance: Undecided Status: New -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Identity (keystone). https://bugs.launchpad.net/bugs/1874705 Title: Websso fails when HTTP_REFERRER that horizon is unable to connect to gets used Status in OpenStack Dashboard (Horizon): In Progress Status in OpenStack Identity (keystone): New Bug description: I am currently having an issue where a request to Horizon's websso endpoint fails to respond in time as the token validation request fails to connect between Horizon and Keystone. (On Openstack Train) I am trying to login to Horizon using an external identity provider. I have set the WEBSSO_KEYSTONE_URL to keystones external facing endpoint as the IDP is on an external network. The POST request to https://horizon_ip/auth/websso/ that includes a keystone token for validation in its params is failing. This request routes to the horizon view 'websso' (https://opendev.org/openstack/horizon/src/branch/master/openstack_auth/views.py#L165) The token authentication request to keystone in this view uses the requests HTTP_REFERRER when available as the keystone endpoint to use. The previous request was to keystone on its external endpoint (as used by the external identity provider) to its route 'auth/OS-FEDERATION/websso/openid', and therefore the HTTP_REFERRER for this POST request is the external keystone endpoint. Our Openstack services have minimal external connectivity for security reasons. So in our setup the horizon service is unable to make connections to the external keystone endpoint. Therefore in the horizon apache logs I see: Unable to establish connection to https://keystone_external_ip:5000/v3/auth/tokens Which eventually leads to a time out. As this is request between Horizon and Keystone ideally for us it should be using the internal endpoint. I've had a go at setting the auth_url to be settings.OPENSTACK_KEYSTONE_URL and this lets me login successfully. I am unsure as to why the HTTP_REFERRER gets used in preference over the settings.OPENSTACK_KEYSTONE_URL for this request? I propose either: 1. Removing the use of HTTP_REFERRER in favor of settings.OPENSTACK_KEYSTONE_URL. 2. Providing a setting to toggle between using the HTTP_REFERRER or settings.OPENSTACK_KEYSTONE_URL to build the auth request with. Original commit in django_openstack_auth for websso view: https://github.com/openstack/django_openstack_auth/commit/302f422568a32b513ffbb3089ba799a4416df108 To manage notifications about this bug go to: https://bugs.launchpad.net/horizon/+bug/1874705/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1934912] [NEW] Router update fails for ports with allowed_address_pairs containg IP range in CIDR notation
Public bug reported:
With https://review.opendev.org/c/openstack/neutron/+/792791 neutron build from
branch `stable/train` fails to update routers with ports containing an
`allowed_address_pair` containing an IP address range in CIDR notation, i.e.:
```
openstack port show 135515bf-6cdf-45d7-affa-c775d2a43ce1 -f value -c
allowed_address_pairs
[{'mac_address': 'fa:16:3e:1e:c4:f1', 'ip_address': '192.168.0.0/16'}]
```
I could not find definitive information on wether this is an allowed
value for allowed_address_pairs, but at least the openstack/magnum
project makes use of this.
Once the above is set neutron-l3-agent logs errors shown in
http://paste.openstack.org/show/807237/ and connection to all resources
behind the router stop.
Steps to reproduce:
Set up openstack environment with neutron build from git branch stable/train
with OVS, DVR and router HA in a multinode deployment on ubuntu bionic.
Create a test environment:
openstack network create test
openstack subnet create --network test --subnet-range 10.0.0.0/24 test
openstack router create --ha --distributed test
openstack router set --external-gateway test
openstack router add subnet test test
openstack server create --image --flavor m1.small --security-group
--network test test
openstack security group create icmp
openstack security group rule create --protocol icmp --ingress icmp
openstack server add security group test icmp
openstack floating ip create
openstack server add floating ip test
ping
openstack port set --allowed-address ip-address=192.168.0.0/16
ping
Observe loss of ping after setting allowed_address_pairs.
Revert https://review.opendev.org/c/openstack/neutron/+/792791 and redeploy
neutron
ping
Observe reestablishment of the connection.
Please let me know if you need any other information
** Affects: neutron
Importance: Undecided
Status: New
--
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1934912
Title:
Router update fails for ports with allowed_address_pairs containg IP
range in CIDR notation
Status in neutron:
New
Bug description:
With https://review.opendev.org/c/openstack/neutron/+/792791 neutron build
from branch `stable/train` fails to update routers with ports containing an
`allowed_address_pair` containing an IP address range in CIDR notation, i.e.:
```
openstack port show 135515bf-6cdf-45d7-affa-c775d2a43ce1 -f value -c
allowed_address_pairs
[{'mac_address': 'fa:16:3e:1e:c4:f1', 'ip_address': '192.168.0.0/16'}]
```
I could not find definitive information on wether this is an allowed
value for allowed_address_pairs, but at least the openstack/magnum
project makes use of this.
Once the above is set neutron-l3-agent logs errors shown in
http://paste.openstack.org/show/807237/ and connection to all
resources behind the router stop.
Steps to reproduce:
Set up openstack environment with neutron build from git branch stable/train
with OVS, DVR and router HA in a multinode deployment on ubuntu bionic.
Create a test environment:
openstack network create test
openstack subnet create --network test --subnet-range 10.0.0.0/24 test
openstack router create --ha --distributed test
openstack router set --external-gateway test
openstack router add subnet test test
openstack server create --image --flavor m1.small
--security-group --network test test
openstack security group create icmp
openstack security group rule create --protocol icmp --ingress icmp
openstack server add security group test icmp
openstack floating ip create
openstack server add floating ip test
ping
openstack port set --allowed-address ip-address=192.168.0.0/16
ping
Observe loss of ping after setting allowed_address_pairs.
Revert https://review.opendev.org/c/openstack/neutron/+/792791 and redeploy
neutron
ping
Observe reestablishment of the connection.
Please let me know if you need any other information
To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/1934912/+subscriptions
--
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1934915] [NEW] [OVN Octavia Provider] Investigate tempest failures
Public bug reported: There are two tempest tests that are failing a significant amount of time, causing us to not be able to merge code into the ovn-octavia- provider repo: Class: octavia_tempest_plugin.tests.scenario.v2.test_traffic_ops.TrafficOperationsScenarioTest Tests: test_source_ip_port_tcp_traffic test_source_ip_port_udp_traffic I plan on disabling them while I investigate the failures, filing this bug so I keep track of things and for adding any debugging notes. ** Affects: neutron Importance: High Assignee: Brian Haley (brian-haley) Status: New ** Tags: ovn-octavia-provider -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to neutron. https://bugs.launchpad.net/bugs/1934915 Title: [OVN Octavia Provider] Investigate tempest failures Status in neutron: New Bug description: There are two tempest tests that are failing a significant amount of time, causing us to not be able to merge code into the ovn-octavia- provider repo: Class: octavia_tempest_plugin.tests.scenario.v2.test_traffic_ops.TrafficOperationsScenarioTest Tests: test_source_ip_port_tcp_traffic test_source_ip_port_udp_traffic I plan on disabling them while I investigate the failures, filing this bug so I keep track of things and for adding any debugging notes. To manage notifications about this bug go to: https://bugs.launchpad.net/neutron/+bug/1934915/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1930448] Re: 'VolumeNotFound' exception is not handled
Reviewed: https://review.opendev.org/c/openstack/nova/+/794006
Committed:
https://opendev.org/openstack/nova/commit/9cdecc81fb8729160693c244d8adf124eed8b9b2
Submitter: "Zuul (22348)"
Branch:master
commit 9cdecc81fb8729160693c244d8adf124eed8b9b2
Author: Stephen Finucane
Date: Tue Jun 1 17:21:58 2021 +0100
api: Handle invalid volume UUIDs during spawn
If a user requests an invalid volume UUID when creating an instance,
a 'VolumeNotFound' exception will be raised. This is not currently
handled. Correct this.
Change-Id: I6137dc1b6b51321fee1c080bf4b85197b19bf223
Signed-off-by: Stephen Finucane
Closes-Bug: #1930448
** Changed in: nova
Status: In Progress => Fix Released
--
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1930448
Title:
'VolumeNotFound' exception is not handled
Status in OpenStack Compute (nova):
Fix Released
Bug description:
Attempting to attach an additional volume (i.e. not boot from volume)
using an invalid ID currently results in a HTTP 500 error. This error
should be handled and a HTTP 4xx error returned instead.
$ openstack server create \
--flavor m1.tiny --image cirros-0.5.1-x86_64-disk --network private \
--block-device
source_type=volume,uuid=44d317a3-6183-4063-868b-aa0728576f5f,destination_type=volume,delete_on_termination=true
\
--wait test-server
Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/
and attach the Nova API log if possible.
(HTTP 500) (Request-ID:
req-7fe03627-c4ce-4f4b-9d5c-3abd6b88d3e3)
where '44d317a3-6183-4063-868b-aa0728576f5f' is not an UUID
corresponding to a valid volume.
A full traceback from nova-compute is provided below.
Jun 01 15:05:14 devstack-ubuntu2004 devstack@n-api.service[1658]: ERROR
nova.api.openstack.wsgi [None req-7fe03627-c4ce-4f4b-9d5c-3abd6b88d3e3 demo
admin] Unexpected exception in API method: nova.exception.VolumeNotFound:
Volume 44d317a3-6183-4063-868b-aa0728576f5f could not be found.
Jun 01 15:05:14 devstack-ubuntu2004 devstack@n-api.service[1658]: ERROR
nova.api.openstack.wsgi Traceback (most recent call last):
Jun 01 15:05:14 devstack-ubuntu2004 devstack@n-api.service[1658]: ERROR
nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line
432, in wrapper
Jun 01 15:05:14 devstack-ubuntu2004 devstack@n-api.service[1658]: ERROR
nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs)
Jun 01 15:05:14 devstack-ubuntu2004 devstack@n-api.service[1658]: ERROR
nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line
498, in get
Jun 01 15:05:14 devstack-ubuntu2004 devstack@n-api.service[1658]: ERROR
nova.api.openstack.wsgi item = cinderclient(
Jun 01 15:05:14 devstack-ubuntu2004 devstack@n-api.service[1658]: ERROR
nova.api.openstack.wsgi File
"/usr/local/lib/python3.8/dist-packages/cinderclient/v2/volumes.py", line 281,
in get
Jun 01 15:05:14 devstack-ubuntu2004 devstack@n-api.service[1658]: ERROR
nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id,
"volume")
Jun 01 15:05:14 devstack-ubuntu2004 devstack@n-api.service[1658]: ERROR
nova.api.openstack.wsgi File
"/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 293, in _get
Jun 01 15:05:14 devstack-ubuntu2004 devstack@n-api.service[1658]: ERROR
nova.api.openstack.wsgi resp, body = self.api.client.get(url)
Jun 01 15:05:14 devstack-ubuntu2004 devstack@n-api.service[1658]: ERROR
nova.api.openstack.wsgi File
"/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in
get
Jun 01 15:05:14 devstack-ubuntu2004 devstack@n-api.service[1658]: ERROR
nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs)
Jun 01 15:05:14 devstack-ubuntu2004 devstack@n-api.service[1658]: ERROR
nova.api.openstack.wsgi File
"/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 206, in
_cs_request
Jun 01 15:05:14 devstack-ubuntu2004 devstack@n-api.service[1658]: ERROR
nova.api.openstack.wsgi return self.request(url, method, **kwargs)
Jun 01 15:05:14 devstack-ubuntu2004 devstack@n-api.service[1658]: ERROR
nova.api.openstack.wsgi File
"/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 192, in
request
Jun 01 15:05:14 devstack-ubuntu2004 devstack@n-api.service[1658]: ERROR
nova.api.openstack.wsgi raise exceptions.from_response(resp, body)
Jun 01 15:05:14 devstack-ubuntu2004 devstack@n-api.service[1658]: ERROR
nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume
44d317a3-6183-4063-868b-aa0728576f5f could not be found. (HTTP 404)
(Request-ID: req-9fcc1abe-1212-45f5-ac76-60fdecd22506)
Jun 01 15:05:14 devstack-ubuntu2004 devstack@n-api.service[1658]: ERROR
nova.api.openstack
[Yahoo-eng-team] [Bug 1933401] Re: [OVN]The type of ovn controller is not recognized as a gateway agent
Reviewed: https://review.opendev.org/c/openstack/neutron/+/797796 Committed: https://opendev.org/openstack/neutron/commit/1f3762b0fb8258e2f77b600df8ab96f20d65f5b8 Submitter: "Zuul (22348)" Branch:master commit 1f3762b0fb8258e2f77b600df8ab96f20d65f5b8 Author: zhouhenglc Date: Thu Jun 24 10:41:15 2021 +0800 [OVN] "ControllerAgent" should accept Chassis and Chassis_Private Because we support OVN version with and without Chassis_Private, "ControllerAgent" should accept both type of registers. In Chassis, the "external_ids" are stored as a main attribute while in Chassis_Private are stored on the chassis reference list. Closes-bug: #1933401 Change-Id: I4575ca06ccba3537a8fd2347b837231e0643e64c ** Changed in: neutron Status: In Progress => Fix Released -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to neutron. https://bugs.launchpad.net/bugs/1933401 Title: [OVN]The type of ovn controller is not recognized as a gateway agent Status in neutron: Fix Released Bug description: neutron use ovn as mechanism_driver ovn version is 21.03 I have a gateway node, and set this node as gateway node use "ovs- vsctl set open_vswitch . external_ids:ovn-cms-options=enable-chassis- as-gw", restart ovn-controller, but I use "openstack network agent list" to see agents, agent type still "OVN Controller agnet" not "OVN Controller Gateway agent" To manage notifications about this bug go to: https://bugs.launchpad.net/neutron/+bug/1933401/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1934930] [NEW] [ovn] Multiple servers can try to create neutron_pg_drop at the same time
Public bug reported: Even though we use may_exist=True to create the neutron_pg_drop Port_Group, it's possible that when another server creates it before us that we don't get the update before we check if it exists before exiting. ** Affects: neutron Importance: Undecided Status: In Progress -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to neutron. https://bugs.launchpad.net/bugs/1934930 Title: [ovn] Multiple servers can try to create neutron_pg_drop at the same time Status in neutron: In Progress Bug description: Even though we use may_exist=True to create the neutron_pg_drop Port_Group, it's possible that when another server creates it before us that we don't get the update before we check if it exists before exiting. To manage notifications about this bug go to: https://bugs.launchpad.net/neutron/+bug/1934930/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1821755] Re: live migration break the anti-affinity policy of server group simultaneously
** Also affects: nova/ussuri Importance: Undecided Status: New ** Also affects: nova/train Importance: Undecided Status: New ** Changed in: nova/ussuri Status: New => Fix Committed -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1821755 Title: live migration break the anti-affinity policy of server group simultaneously Status in OpenStack Compute (nova): Fix Released Status in OpenStack Compute (nova) train series: New Status in OpenStack Compute (nova) ussuri series: Fix Committed Status in OpenStack Compute (nova) victoria series: Fix Committed Status in OpenStack Compute (nova) wallaby series: Fix Committed Bug description: Description === If we live migrate two instance simultaneously, the instances will break the instance group policy. Steps to reproduce == OpenStack env with three compute nodes(node1, node2 and node3). Then we create two VMs(vm1, vm2) with the anti-affinity policy. At last, we live migrate two VMs simultaneously. Before live-migration, the VMs are located as followed: node1 -> vm1 node2 -> vm2 node3 * nova live-migration vm1 * nova live-migration vm2 Expected result === Fail to live migrate vm1 and vm2. Actual result = node1 node2 node3 -> vm1,vm2 Environment === master branch of openstack As described above, the live migration could not check the in-progress live-migration and just select the host by scheduler filter. So that they are migrated to the same host. To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1821755/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1369008] Re: filter on image name should not use exact name
The change has been abandoned six years ago, and would probably require a (lite-)spec. I'm closing this, feel free to reopen if you believe this should be revisited. ** Changed in: glance Status: In Progress => Won't Fix -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to Glance. https://bugs.launchpad.net/bugs/1369008 Title: filter on image name should not use exact name Status in Glance: Won't Fix Bug description: Horizon passes name filter into glanceclient is like name=cirros. cirros is the partial name of the cirros-0.3.2-x86_64-uec-kernel image. glance should use the partial name to do search instead of using it as exact name matching. The original bug reported in horizon like the following...after more digging, I moved the bug into glanceclient. "at Admin-> Images The filter on Image Name has to be exact the same as the image name. If I have the image names like the followings, it could be hard and not user friendly: Fedora-x86_64-20-20140618-sda cirros-0.3.2-x86_64-uec-kernel Think the image name filter should do the same way as instances table's instance name filter which uses contains, not the exact match." To manage notifications about this bug go to: https://bugs.launchpad.net/glance/+bug/1369008/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1934937] [NEW] Heartbeat in pthreads in nova-wallaby crashes with greenlet error
Public bug reported:
When performing a heartbeat to rabbit (inside a nova-compute process),
there is a greenlet error which causes a hard crash.
I'm not exactly sure what details are relevant, but can provide more
info if there's something that will be useful!
This is on RHEL7 (essentially... somewhat custom image based on it)
Log snippet:
```
2021-07-07 19:34:52,686 DEBUG [oslo.messaging._drivers.impl_rabbit]
/opt/openstack/venv/nova-23.0.20031033070/lib/python3.8/site-packages/oslo_messaging/_drivers/impl_rabbit.py:__init__:608
[279fc413-9d7c-4fad-89e8-8de308658947] Connecting to AMQP server on
127.0.0.1:5671
2021-07-07 19:34:52,699 DEBUG [amqp.connection.Connection.heartbeat_tick]
/opt/openstack/venv/nova-23.0.20031033070/lib/python3.8/site-packages/amqp/connection.py:heartbeat_tick:726
heartbeat_tick : for connection 79f7cf4331b34cb0a2e3608281076773
2021-07-07 19:34:52,699 DEBUG [amqp.connection.Connection.heartbeat_tick]
/opt/openstack/venv/nova-23.0.20031033070/lib/python3.8/site-packages/amqp/connection.py:heartbeat_tick:740
heartbeat_tick : Prev sent/recv: None/None, now - 6/6, monotonic -
9634.717472491, last_heartbeat_sent - 9634.717470288, heartbeat int. - 60 for
connection 79f7cf4331b34cb0a2e3608281076773
2021-07-07 19:34:52,700 DEBUG [amqp.connection.Connection.heartbeat_tick]
/opt/openstack/venv/nova-23.0.20031033070/lib/python3.8/site-packages/amqp/connection.py:heartbeat_tick:726
heartbeat_tick : for connection 79f7cf4331b34cb0a2e3608281076773
2021-07-07 19:34:52,701 DEBUG [amqp.connection.Connection.heartbeat_tick]
/opt/openstack/venv/nova-23.0.20031033070/lib/python3.8/site-packages/amqp/connection.py:heartbeat_tick:740
heartbeat_tick : Prev sent/recv: 6/6, now - 6/6, monotonic - 9634.719438155,
last_heartbeat_sent - 9634.717470288, heartbeat int. - 60 for connection
79f7cf4331b34cb0a2e3608281076773
2021-07-07 19:34:52,718 DEBUG [amqp]
/opt/openstack/venv/nova-23.0.20031033070/lib/python3.8/site-packages/amqp/connection.py:_on_start:382
Start from server, version: 0.9, properties: {'capabilities':
{'publisher_confirms': True, 'exchange_exchange_bindings': True, 'basic.nack':
True, 'consumer_cancel_notify': True, 'connection.blocked': True,
'consumer_priorities': True, 'authentication_failure_close': True,
'per_consumer_qos': True, 'direct_reply_to': True}, 'cluster_name':
'rabbit_5672@fedb460a.openstack', 'copyright': 'Copyright (c) 2007-2020 VMware,
Inc. or its affiliates.', 'information': 'Licensed under the MPL 1.1. Website:
https://rabbitmq.com', 'platform': 'Erlang/OTP 23.0.2', 'product': 'RabbitMQ',
'version': '3.8.5'}, mechanisms: [b'PLAIN', b'AMQPLAIN', b'EXTERNAL'], locales:
['en_US']
2021-07-07 19:34:52,719 DEBUG [amqp]
/opt/openstack/venv/nova-23.0.20031033070/lib/python3.8/site-packages/amqp/channel.py:__init__:104
using channel_id: 1
2021-07-07 19:34:52,720 DEBUG [amqp]
/opt/openstack/venv/nova-23.0.20031033070/lib/python3.8/site-packages/amqp/channel.py:_on_open_ok:444
Channel open
2021-07-07 19:34:52,721 DEBUG [amqp.connection.Connection.heartbeat_tick]
/opt/openstack/venv/nova-23.0.20031033070/lib/python3.8/site-packages/amqp/connection.py:heartbeat_tick:726
heartbeat_tick : for connection c0299792d20e42a2b0a17d037d7d3058
Traceback (most recent call last):
File
"/opt/openstack/venv/nova-23.0.20031033070/lib/python3.8/site-packages/eventlet/hubs/hub.py",
line 476, in fire_timers
timer()
File
"/opt/openstack/venv/nova-23.0.20031033070/lib/python3.8/site-packages/eventlet/hubs/timer.py",
line 59, in __call__
cb(*args, **kw)
File
"/opt/openstack/venv/nova-23.0.20031033070/lib/python3.8/site-packages/eventlet/semaphore.py",
line 152, in _do_acquire
waiter.switch()
greenlet.error: cannot switch to a different thread
```
Versions:
```
oslo.messaging==12.7.1
nova==23.0.2 (packaged locally from stable/wallaby as of July 3, 2021)
```
** Affects: nova
Importance: Undecided
Status: New
** Affects: oslo.messaging
Importance: Undecided
Status: New
** Also affects: nova
Importance: Undecided
Status: New
--
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1934937
Title:
Heartbeat in pthreads in nova-wallaby crashes with greenlet error
Status in OpenStack Compute (nova):
New
Status in oslo.messaging:
New
Bug description:
When performing a heartbeat to rabbit (inside a nova-compute process),
there is a greenlet error which causes a hard crash.
I'm not exactly sure what details are relevant, but can provide more
info if there's something that will be useful!
This is on RHEL7 (essentially... somewhat custom image based on it)
Log snippet:
```
2021-07-07 19:34:52,686 DEBUG [oslo.messaging._drivers.impl_rabbit]
/opt/openstack/venv/nova-23.0.20031033070/lib/python3.8/site-packages/oslo_messaging/_drivers/impl_rabbit.py:__init__:608
[279fc413-9d7c-4fad-89e8-8de3
[Yahoo-eng-team] [Bug 1934512] Re: Remove SG RPC "use_enhanced_rpc" check
Reviewed: https://review.opendev.org/c/openstack/neutron/+/799446 Committed: https://opendev.org/openstack/neutron/commit/f637a1f60e4f38c519184cc9ce7a0dea857a95b0 Submitter: "Zuul (22348)" Branch:master commit f637a1f60e4f38c519184cc9ce7a0dea857a95b0 Author: Rodolfo Alonso Hernandez Date: Mon Jul 5 09:24:04 2021 + Remove SG RPC "use_enhanced_rpc" check. It's been a long time since [1] was implemented. Enhanced RPC is now supported by default. Closes-Bug: #1934512 [1]https://review.opendev.org/c/openstack/neutron/+/111876 Change-Id: I80c3076b9545be55b11858c4422402dd5ae1a68e ** Changed in: neutron Status: In Progress => Fix Released -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to neutron. https://bugs.launchpad.net/bugs/1934512 Title: Remove SG RPC "use_enhanced_rpc" check Status in neutron: Fix Released Bug description: This check was implemented during the SG RPC refactor in [1], seven years ago. It is not needed anymore. [1]https://review.opendev.org/c/openstack/neutron/+/111876 To manage notifications about this bug go to: https://bugs.launchpad.net/neutron/+bug/1934512/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1934948] [NEW] [RFE] refactor of L3 resources update procedure
Public bug reported: In the L3 meeting 2021-06-30, I mentioned this topic. https://meetings.opendev.org/meetings/neutron_l3/2021/neutron_l3.2021-06-30-14.00.log.html#l-28 Current L3 resources (floating IPs, router interface, router external gateway) processing procedure is a bit heavy, and sometimes waste of times. For instance, if one floating IP is bind to a port under one router, the steps are: 1. floating IP updated 2. notify L3 agent that this floating IPs router is updated 3. L3 agent sync the router info 4. L3 agent reprocess all router related resources The alternative is we can use resource cache for router related resources, then the procedure for floating IP update can be changed to: 1. floating IP updated. 2. OVO object update event send out. 3. L3 agents which has the related router resident will do the processing of floating IP only, no sync of full router info, no processing of all router resources again! This can be a huge performace improvement for L3 related resources processing. Thoughts? ** Affects: neutron Importance: Undecided Status: New -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to neutron. https://bugs.launchpad.net/bugs/1934948 Title: [RFE] refactor of L3 resources update procedure Status in neutron: New Bug description: In the L3 meeting 2021-06-30, I mentioned this topic. https://meetings.opendev.org/meetings/neutron_l3/2021/neutron_l3.2021-06-30-14.00.log.html#l-28 Current L3 resources (floating IPs, router interface, router external gateway) processing procedure is a bit heavy, and sometimes waste of times. For instance, if one floating IP is bind to a port under one router, the steps are: 1. floating IP updated 2. notify L3 agent that this floating IPs router is updated 3. L3 agent sync the router info 4. L3 agent reprocess all router related resources The alternative is we can use resource cache for router related resources, then the procedure for floating IP update can be changed to: 1. floating IP updated. 2. OVO object update event send out. 3. L3 agents which has the related router resident will do the processing of floating IP only, no sync of full router info, no processing of all router resources again! This can be a huge performace improvement for L3 related resources processing. Thoughts? To manage notifications about this bug go to: https://bugs.launchpad.net/neutron/+bug/1934948/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1934957] [NEW] [sriov] Unable to change the VF state for i350 interface
Public bug reported:
When sriov-nic-agent configures VF state, the exception is as follows:
2021-07-08 06:15:47.773 34 DEBUG oslo.privsep.daemon [-] privsep: Exception
during request[139820149013392]: Operation not supported on interface eno4,
namespace None. _process_cmd
/usr/local/lib/python3.6/site-packages/oslo_privsep/daemon.py:490
Traceback (most recent call last):
File
"/usr/local/lib/python3.6/site-packages/neutron/privileged/agent/linux/ip_lib.py",
line 263, in _run_iproute_link
return ip.link(command, index=idx, **kwargs)
File "/usr/local/lib/python3.6/site-packages/pyroute2/iproute/linux.py", line
1360, in link
msg_flags=msg_flags)
File "/usr/local/lib/python3.6/site-packages/pyroute2/netlink/nlsocket.py",
line 376, in nlm_request
return tuple(self._genlm_request(*argv, **kwarg))
File "/usr/local/lib/python3.6/site-packages/pyroute2/netlink/nlsocket.py",
line 869, in nlm_request
callback=callback):
File "/usr/local/lib/python3.6/site-packages/pyroute2/netlink/nlsocket.py",
line 379, in get
return tuple(self._genlm_get(*argv, **kwarg))
File "/usr/local/lib/python3.6/site-packages/pyroute2/netlink/nlsocket.py",
line 704, in get
raise msg['header']['error']
pyroute2.netlink.exceptions.NetlinkError: (95, 'Operation not supported')
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/lib/python3.6/site-packages/oslo_privsep/daemon.py", line
485, in _process_cmd
ret = func(*f_args, **f_kwargs)
File "/usr/local/lib/python3.6/site-packages/oslo_privsep/priv_context.py",
line 249, in _wrap
return func(*args, **kwargs)
File
"/usr/local/lib/python3.6/site-packages/neutron/privileged/agent/linux/ip_lib.py",
line 403, in set_link_vf_feature
return _run_iproute_link("set", device, namespace=namespace, vf=vf_config)
File
"/usr/local/lib/python3.6/site-packages/neutron/privileged/agent/linux/ip_lib.py",
line 265, in _run_iproute_link
_translate_ip_device_exception(e, device, namespace)
File
"/usr/local/lib/python3.6/site-packages/neutron/privileged/agent/linux/ip_lib.py",
line 237, in _translate_ip_device_exception
namespace=namespace)
neutron.privileged.agent.linux.ip_lib.InterfaceOperationNotSupported: Operation
not supported on interface eno4, namespace None.
2021-07-08 06:15:47.773 34 DEBUG oslo.privsep.daemon [-] privsep:
reply[139820149013392]: (5,
'neutron.privileged.agent.linux.ip_lib.InterfaceOperationNotSupported',
('Operation not supported on interface eno4, namespace None.',)) _call_back
/usr/local/lib/python3.6/site-packages/oslo_privsep/daemon.py:511
2021-07-08 06:15:47.774 24 WARNING
neutron.plugins.ml2.drivers.mech_sriov.agent.sriov_nic_agent
[req-661d08fb-983f-4632-9eb4-91585a557753 - - - - -] Device fa:16:3e:66:e4:91
does not support state change:
neutron.privileged.agent.linux.ip_lib.InterfaceOperationNotSupported: Operation
not supported on interface eno4, namespace None.
But the vm network traffic is no problem. We use i350 interface， and I
found these discuss about i350[1][2]. This exception is not impact for
vm traffic, maybe we can ignore it when interface is i350.
[1]https://sourceforge.net/p/e1000/bugs/653/
[2]https://community.intel.com/t5/Ethernet-Products/On-SRIOV-interface-I350-unable-to-change-the-VF-state-from-auto/td-p/704769
version:
neutron-sriov-nic-agent version 17.1.3
** Affects: neutron
Importance: Undecided
Status: New
--
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1934957
Title:
[sriov] Unable to change the VF state for i350 interface
Status in neutron:
New
Bug description:
When sriov-nic-agent configures VF state, the exception is as follows:
2021-07-08 06:15:47.773 34 DEBUG oslo.privsep.daemon [-] privsep: Exception
during request[139820149013392]: Operation not supported on interface eno4,
namespace None. _process_cmd
/usr/local/lib/python3.6/site-packages/oslo_privsep/daemon.py:490
Traceback (most recent call last):
File
"/usr/local/lib/python3.6/site-packages/neutron/privileged/agent/linux/ip_lib.py",
line 263, in _run_iproute_link
return ip.link(command, index=idx, **kwargs)
File "/usr/local/lib/python3.6/site-packages/pyroute2/iproute/linux.py",
line 1360, in link
msg_flags=msg_flags)
File "/usr/local/lib/python3.6/site-packages/pyroute2/netlink/nlsocket.py",
line 376, in nlm_request
return tuple(self._genlm_request(*argv, **kwarg))
File "/usr/local/lib/python3.6/site-packages/pyroute2/netlink/nlsocket.py",
line 869, in nlm_request
callback=callback):
File "/usr/local/lib/python3.6/site-packages/pyroute2/netlink/nlsocket.py",
line 379, in get
return tuple(self._genlm_get(*argv, **kwarg))
File "/usr/local/lib/python3.6/site-packages/pyroute2/netlink/nlsocket.py",
line 704, in get
raise msg['heade
