Public bug reported: As of this change in Rocky:
https://review.opendev.org/#/c/585339/ When refreshing the vif model for an instance, e.g. when we get a network-changed event with a specific port ID: https://logs.opendev.org/26/674326/2/experimental/nova-osprofiler- redis/899a204/controller/logs/screen-n-cpu.txt.gz#_Aug_02_18_35_50_884613 Aug 02 18:35:50.884613 ubuntu-bionic-vexxhost-sjc1-0009723918 nova- compute[20428]: DEBUG nova.network.neutronv2.api [req-1a4c2dbf-df86-4044 -a59f-f751a53c5ea6 req-b0e1e2f7-d126-4e55-909d-4803816ca80f service nova] [instance: 5bbe0419-fbeb-4667-8c56-785fdc1d0a62] Refreshing network info cache for port 252040d6-4469-46ec-88c3-85e599a43104 {{(pid=20428) _get_instance_nw_info /opt/stack/nova/nova/network/neutronv2/api.py:1756}} We get the network for the port multiple times, first here: https://github.com/openstack/nova/blob/600ecf3d9a5116d040cd18023ff270b91b06247d/nova/network/neutronv2/api.py#L2966 https://github.com/openstack/nova/blob/600ecf3d9a5116d040cd18023ff270b91b06247d/nova/network/neutronv2/api.py#L392 And then we pass that list of 1 network dict to _build_vif_model here: https://github.com/openstack/nova/blob/600ecf3d9a5116d040cd18023ff270b91b06247d/nova/network/neutronv2/api.py#L2850 and pass it to _nw_info_build_network here: https://github.com/openstack/nova/blob/600ecf3d9a5116d040cd18023ff270b91b06247d/nova/network/neutronv2/api.py#L2883 Which then calls _get_physnet_tunneled_info which gets the network again here: https://github.com/openstack/nova/blob/600ecf3d9a5116d040cd18023ff270b91b06247d/nova/network/neutronv2/api.py#L1904 and/or here: https://github.com/openstack/nova/blob/600ecf3d9a5116d040cd18023ff270b91b06247d/nova/network/neutronv2/api.py#L1928 Furthermore, when we're doing forced _heal_instance_info_cache (stein+) we'll refresh the vif model for all ports that are currently attached to the server: https://github.com/openstack/nova/blob/600ecf3d9a5116d040cd18023ff270b91b06247d/nova/network/neutronv2/api.py#L3015 And rebuild the vif model per port here: https://github.com/openstack/nova/blob/600ecf3d9a5116d040cd18023ff270b91b06247d/nova/network/neutronv2/api.py#L3027 If there is more than one port on the same network attached to the server, we'll be calling show_network for each port even though we're getting the same data when those ports are on the same network. I noticed this while checking some osprofiler results and noticed the network-changed event on the port-targeted refresh took a relatively long time: https://logs.opendev.org/26/674326/2/experimental/nova-osprofiler- redis/899a204/osprofiler-traces/trace-fc50ca23-a6c2-474a- ac07-e61e706eb27d.html.gz ** Affects: nova Importance: Medium Assignee: Matt Riedemann (mriedem) Status: Triaged ** Affects: nova/rocky Importance: Medium Status: Confirmed ** Affects: nova/stein Importance: Medium Status: Confirmed ** Tags: neutron performance ** Changed in: nova Status: New => Triaged ** Changed in: nova Importance: Undecided => Medium ** Also affects: nova/rocky Importance: Undecided Status: New ** Also affects: nova/stein Importance: Undecided Status: New ** Changed in: nova/rocky Status: New => Confirmed ** Changed in: nova/stein Importance: Undecided => Medium ** Changed in: nova/stein Status: New => Confirmed ** Changed in: nova/rocky Importance: Undecided => Medium ** Changed in: nova Assignee: (unassigned) => Matt Riedemann (mriedem) -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1838817 Title: neutron: refreshing vif model for a server gets the same network multiple times Status in OpenStack Compute (nova): Triaged Status in OpenStack Compute (nova) rocky series: Confirmed Status in OpenStack Compute (nova) stein series: Confirmed Bug description: As of this change in Rocky: https://review.opendev.org/#/c/585339/ When refreshing the vif model for an instance, e.g. when we get a network-changed event with a specific port ID: https://logs.opendev.org/26/674326/2/experimental/nova-osprofiler- redis/899a204/controller/logs/screen-n-cpu.txt.gz#_Aug_02_18_35_50_884613 Aug 02 18:35:50.884613 ubuntu-bionic-vexxhost-sjc1-0009723918 nova- compute[20428]: DEBUG nova.network.neutronv2.api [req-1a4c2dbf- df86-4044-a59f-f751a53c5ea6 req-b0e1e2f7-d126-4e55-909d-4803816ca80f service nova] [instance: 5bbe0419-fbeb-4667-8c56-785fdc1d0a62] Refreshing network info cache for port 252040d6-4469-46ec- 88c3-85e599a43104 {{(pid=20428) _get_instance_nw_info /opt/stack/nova/nova/network/neutronv2/api.py:1756}} We get the network for the port multiple times, first here: https://github.com/openstack/nova/blob/600ecf3d9a5116d040cd18023ff270b91b06247d/nova/network/neutronv2/api.py#L2966 https://github.com/openstack/nova/blob/600ecf3d9a5116d040cd18023ff270b91b06247d/nova/network/neutronv2/api.py#L392 And then we pass that list of 1 network dict to _build_vif_model here: https://github.com/openstack/nova/blob/600ecf3d9a5116d040cd18023ff270b91b06247d/nova/network/neutronv2/api.py#L2850 and pass it to _nw_info_build_network here: https://github.com/openstack/nova/blob/600ecf3d9a5116d040cd18023ff270b91b06247d/nova/network/neutronv2/api.py#L2883 Which then calls _get_physnet_tunneled_info which gets the network again here: https://github.com/openstack/nova/blob/600ecf3d9a5116d040cd18023ff270b91b06247d/nova/network/neutronv2/api.py#L1904 and/or here: https://github.com/openstack/nova/blob/600ecf3d9a5116d040cd18023ff270b91b06247d/nova/network/neutronv2/api.py#L1928 Furthermore, when we're doing forced _heal_instance_info_cache (stein+) we'll refresh the vif model for all ports that are currently attached to the server: https://github.com/openstack/nova/blob/600ecf3d9a5116d040cd18023ff270b91b06247d/nova/network/neutronv2/api.py#L3015 And rebuild the vif model per port here: https://github.com/openstack/nova/blob/600ecf3d9a5116d040cd18023ff270b91b06247d/nova/network/neutronv2/api.py#L3027 If there is more than one port on the same network attached to the server, we'll be calling show_network for each port even though we're getting the same data when those ports are on the same network. I noticed this while checking some osprofiler results and noticed the network-changed event on the port-targeted refresh took a relatively long time: https://logs.opendev.org/26/674326/2/experimental/nova-osprofiler- redis/899a204/osprofiler-traces/trace-fc50ca23-a6c2-474a- ac07-e61e706eb27d.html.gz To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1838817/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp