Public bug reported: Description =========== Nova update the MAC of the direct-physical port based on the MAC of the PF selected during the initial boot of the VM. But Nova does not update the MAC when the VM is migrated to another compute and therefore using another PF.
Steps to reproduce ================== Needs a multi node devstack with available SRIOV PFs. stack@master0:~$ openstack hypervisor list +----+---------------------+-----------------+-----------+-------+ | ID | Hypervisor Hostname | Hypervisor Type | Host IP | State | +----+---------------------+-----------------+-----------+-------+ | 1 | master0 | QEMU | 10.1.0.21 | up | | 2 | node0 | QEMU | 10.1.0.22 | up | +----+---------------------+-----------------+-----------+-------+ stack@master0:~$ mysql -D nova_cell1 -e "select status, address, parent_addr, dev_type, compute_node_id, product_id, instance_uuid from pci_devices;" +-----------+--------------+--------------+----------+-----------------+------------+---------------+ | status | address | parent_addr | dev_type | compute_node_id | product_id | instance_uuid | +-----------+--------------+--------------+----------+-----------------+------------+---------------+ | available | 0000:81:00.0 | NULL | type-PF | 1 | 154d | NULL | | available | 0000:81:00.1 | NULL | type-PF | 1 | 154d | NULL | | available | 0000:81:10.0 | 0000:81:00.0 | type-VF | 1 | 10ed | NULL | | available | 0000:81:10.2 | 0000:81:00.0 | type-VF | 1 | 10ed | NULL | | available | 0000:81:10.4 | 0000:81:00.0 | type-VF | 1 | 10ed | NULL | | available | 0000:81:10.6 | 0000:81:00.0 | type-VF | 1 | 10ed | NULL | | available | 0000:81:00.0 | NULL | type-PF | 2 | 154d | NULL | | available | 0000:81:00.1 | NULL | type-PF | 2 | 154d | NULL | | available | 0000:81:10.0 | 0000:81:00.0 | type-VF | 2 | 10ed | NULL | | available | 0000:81:10.2 | 0000:81:00.0 | type-VF | 2 | 10ed | NULL | | available | 0000:81:10.4 | 0000:81:00.0 | type-VF | 2 | 10ed | NULL | | available | 0000:81:10.6 | 0000:81:00.0 | type-VF | 2 | 10ed | NULL | +-----------+--------------+--------------+----------+-----------------+------------+---------------+ These are the PF MAC's stack@master0:~$ ip a | grep b4:96:91:34 link/ether b4:96:91:34:f4:34 brd ff:ff:ff:ff:ff:ff link/ether b4:96:91:34:f4:36 brd ff:ff:ff:ff:ff:ff stack@node0:~/nova$ ip a | grep b4:96:91:34 link/ether b4:96:91:34:ed:d4 brd ff:ff:ff:ff:ff:ff link/ether b4:96:91:34:ed:d6 brd ff:ff:ff:ff:ff:ff 1) create a port with vnic_type=directy-physical stack@master0:~$ openstack port show port-direct-physical +-------------------------+---------------------------------------------------------------------------+ | Field | Value | +-------------------------+---------------------------------------------------------------------------+ | admin_state_up | UP | | allowed_address_pairs | | | binding_host_id | | | binding_profile | physical_network='physnet0' | | binding_vif_details | | | binding_vif_type | unbound | | binding_vnic_type | direct-physical | | created_at | 2021-08-19T12:02:06Z | | data_plane_status | None | | description | | | device_id | | | device_owner | | | device_profile | None | | dns_assignment | None | | dns_domain | None | | dns_name | None | | extra_dhcp_opts | | | fixed_ips | ip_address='10.0.4.227', subnet_id='a3960124-77c3-4127-8a91-9ae99ed552ea' | | id | 864ee6f2-2e72-47b6-9917-6b8e931afc3e | | ip_allocation | None | | mac_address | fa:16:3e:71:fe:26 | | name | port-direct-physical | | network_id | 96d669e1-0c1e-474e-aad6-57cd9e026980 | | numa_affinity_policy | None | | port_security_enabled | True | | project_id | 493cf4fe555d4ea0b50bdd46a8edf5c2 | | propagate_uplink_status | None | | qos_network_policy_id | None | | qos_policy_id | None | | resource_request | None | | revision_number | 10 | | security_group_ids | 16150c45-2217-4e73-8e4d-ca946cbc21cc | | status | DOWN | | tags | | | trunk_details | None | | updated_at | 2021-09-01T13:12:42Z | +-------------------------+---------------------------------------------------------------------------+ 2) boot a VM with that port stack@master0:~$ openstack --os-compute-api-version 2.80 server create --image cirros-0.5.2-x86_64-disk --flavor c1 --nic port-id=port-direct- physical vm1 --wait +-------------------------------------+-----------------------------------------------------------------------------------------------------------------------+ | Field | Value | +-------------------------------------+-----------------------------------------------------------------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | node0 | | OS-EXT-SRV-ATTR:hostname | vm1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | node0 | | OS-EXT-SRV-ATTR:instance_name | instance-000000ff | | OS-EXT-SRV-ATTR:kernel_id | | | OS-EXT-SRV-ATTR:launch_index | 0 | | OS-EXT-SRV-ATTR:ramdisk_id | | | OS-EXT-SRV-ATTR:reservation_id | r-ou2kcw3c | | OS-EXT-SRV-ATTR:root_device_name | /dev/vda | | OS-EXT-SRV-ATTR:user_data | None | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2021-09-01T13:23:26.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | net0=10.0.4.227 | | adminPass | B5Zje4m3vbq6 | | config_drive | | | created | 2021-09-01T13:23:23Z | | description | None | | flavor | disk='1', ephemeral='0', extra_specs.hw_rng:allowed='True', original_name='cirros256', ram='256', swap='0', vcpus='1' | | hostId | 512757d0f0cb65d56c6cabcc54b317bbc1c73dbbe8eb603d9452bc59 | | host_status | UP | | id | 4707526d-6980-4af7-9476-d85e3f6aec4f | | image | cirros-0.5.2-x86_64-disk (1193639c-3c6a-4d3c-9c66-ce1ab741a9d3) | | key_name | None | | locked | False | | locked_reason | None | | name | vm1 | | progress | 0 | | project_id | 493cf4fe555d4ea0b50bdd46a8edf5c2 | | properties | | | security_groups | name='default' | | server_groups | [] | | status | ACTIVE | | tags | | | trusted_image_certificates | None | | updated | 2021-09-01T13:23:26Z | | user_id | e08f3a84c1484d6ebdde1faf24fd0373 | | volumes_attached | | +-------------------------------------+-----------------------------------------------------------------------------------------------------------------------+ 3) Observer the the VM is booted on node0 and check that the MAC of the port is updated to the mac of one of the PF on node0 stack@master0:~$ openstack port show port-direct-physical | grep mac | mac_address | b4:96:91:34:ed:d6 | 4) Cold migrate the VM stack@master0:~$ openstack server migrate vm1 stack@master0:~$ openstack server migration confirm vm1 5) Observe that the VM is now running on master0 but MAC of the port is not changed stack@master0:~$ openstack server show vm1 | grep hypervisor | OS-EXT-SRV-ATTR:hypervisor_hostname | master0 | stack@master0:~$ openstack port show port-direct-physical | grep mac | mac_address | b4:96:91:34:ed:d6 | Expected behavior ================= The MAC of the neutron port is updated according the the MAC of the PF on master0 Environment =========== multinode devstack on recent master (nova on 033af94179) ** Affects: nova Importance: Undecided Status: New ** Tags: compute sriov ** Tags added: sriov ** Tags added: comp ** Tags removed: comp ** Tags added: compute -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1942329 Title: MAC address of direct-physical port is not updated during migration Status in OpenStack Compute (nova): New Bug description: Description =========== Nova update the MAC of the direct-physical port based on the MAC of the PF selected during the initial boot of the VM. But Nova does not update the MAC when the VM is migrated to another compute and therefore using another PF. Steps to reproduce ================== Needs a multi node devstack with available SRIOV PFs. stack@master0:~$ openstack hypervisor list +----+---------------------+-----------------+-----------+-------+ | ID | Hypervisor Hostname | Hypervisor Type | Host IP | State | +----+---------------------+-----------------+-----------+-------+ | 1 | master0 | QEMU | 10.1.0.21 | up | | 2 | node0 | QEMU | 10.1.0.22 | up | +----+---------------------+-----------------+-----------+-------+ stack@master0:~$ mysql -D nova_cell1 -e "select status, address, parent_addr, dev_type, compute_node_id, product_id, instance_uuid from pci_devices;" +-----------+--------------+--------------+----------+-----------------+------------+---------------+ | status | address | parent_addr | dev_type | compute_node_id | product_id | instance_uuid | +-----------+--------------+--------------+----------+-----------------+------------+---------------+ | available | 0000:81:00.0 | NULL | type-PF | 1 | 154d | NULL | | available | 0000:81:00.1 | NULL | type-PF | 1 | 154d | NULL | | available | 0000:81:10.0 | 0000:81:00.0 | type-VF | 1 | 10ed | NULL | | available | 0000:81:10.2 | 0000:81:00.0 | type-VF | 1 | 10ed | NULL | | available | 0000:81:10.4 | 0000:81:00.0 | type-VF | 1 | 10ed | NULL | | available | 0000:81:10.6 | 0000:81:00.0 | type-VF | 1 | 10ed | NULL | | available | 0000:81:00.0 | NULL | type-PF | 2 | 154d | NULL | | available | 0000:81:00.1 | NULL | type-PF | 2 | 154d | NULL | | available | 0000:81:10.0 | 0000:81:00.0 | type-VF | 2 | 10ed | NULL | | available | 0000:81:10.2 | 0000:81:00.0 | type-VF | 2 | 10ed | NULL | | available | 0000:81:10.4 | 0000:81:00.0 | type-VF | 2 | 10ed | NULL | | available | 0000:81:10.6 | 0000:81:00.0 | type-VF | 2 | 10ed | NULL | +-----------+--------------+--------------+----------+-----------------+------------+---------------+ These are the PF MAC's stack@master0:~$ ip a | grep b4:96:91:34 link/ether b4:96:91:34:f4:34 brd ff:ff:ff:ff:ff:ff link/ether b4:96:91:34:f4:36 brd ff:ff:ff:ff:ff:ff stack@node0:~/nova$ ip a | grep b4:96:91:34 link/ether b4:96:91:34:ed:d4 brd ff:ff:ff:ff:ff:ff link/ether b4:96:91:34:ed:d6 brd ff:ff:ff:ff:ff:ff 1) create a port with vnic_type=directy-physical stack@master0:~$ openstack port show port-direct-physical +-------------------------+---------------------------------------------------------------------------+ | Field | Value | +-------------------------+---------------------------------------------------------------------------+ | admin_state_up | UP | | allowed_address_pairs | | | binding_host_id | | | binding_profile | physical_network='physnet0' | | binding_vif_details | | | binding_vif_type | unbound | | binding_vnic_type | direct-physical | | created_at | 2021-08-19T12:02:06Z | | data_plane_status | None | | description | | | device_id | | | device_owner | | | device_profile | None | | dns_assignment | None | | dns_domain | None | | dns_name | None | | extra_dhcp_opts | | | fixed_ips | ip_address='10.0.4.227', subnet_id='a3960124-77c3-4127-8a91-9ae99ed552ea' | | id | 864ee6f2-2e72-47b6-9917-6b8e931afc3e | | ip_allocation | None | | mac_address | fa:16:3e:71:fe:26 | | name | port-direct-physical | | network_id | 96d669e1-0c1e-474e-aad6-57cd9e026980 | | numa_affinity_policy | None | | port_security_enabled | True | | project_id | 493cf4fe555d4ea0b50bdd46a8edf5c2 | | propagate_uplink_status | None | | qos_network_policy_id | None | | qos_policy_id | None | | resource_request | None | | revision_number | 10 | | security_group_ids | 16150c45-2217-4e73-8e4d-ca946cbc21cc | | status | DOWN | | tags | | | trunk_details | None | | updated_at | 2021-09-01T13:12:42Z | +-------------------------+---------------------------------------------------------------------------+ 2) boot a VM with that port stack@master0:~$ openstack --os-compute-api-version 2.80 server create --image cirros-0.5.2-x86_64-disk --flavor c1 --nic port-id=port- direct-physical vm1 --wait +-------------------------------------+-----------------------------------------------------------------------------------------------------------------------+ | Field | Value | +-------------------------------------+-----------------------------------------------------------------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | node0 | | OS-EXT-SRV-ATTR:hostname | vm1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | node0 | | OS-EXT-SRV-ATTR:instance_name | instance-000000ff | | OS-EXT-SRV-ATTR:kernel_id | | | OS-EXT-SRV-ATTR:launch_index | 0 | | OS-EXT-SRV-ATTR:ramdisk_id | | | OS-EXT-SRV-ATTR:reservation_id | r-ou2kcw3c | | OS-EXT-SRV-ATTR:root_device_name | /dev/vda | | OS-EXT-SRV-ATTR:user_data | None | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2021-09-01T13:23:26.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | net0=10.0.4.227 | | adminPass | B5Zje4m3vbq6 | | config_drive | | | created | 2021-09-01T13:23:23Z | | description | None | | flavor | disk='1', ephemeral='0', extra_specs.hw_rng:allowed='True', original_name='cirros256', ram='256', swap='0', vcpus='1' | | hostId | 512757d0f0cb65d56c6cabcc54b317bbc1c73dbbe8eb603d9452bc59 | | host_status | UP | | id | 4707526d-6980-4af7-9476-d85e3f6aec4f | | image | cirros-0.5.2-x86_64-disk (1193639c-3c6a-4d3c-9c66-ce1ab741a9d3) | | key_name | None | | locked | False | | locked_reason | None | | name | vm1 | | progress | 0 | | project_id | 493cf4fe555d4ea0b50bdd46a8edf5c2 | | properties | | | security_groups | name='default' | | server_groups | [] | | status | ACTIVE | | tags | | | trusted_image_certificates | None | | updated | 2021-09-01T13:23:26Z | | user_id | e08f3a84c1484d6ebdde1faf24fd0373 | | volumes_attached | | +-------------------------------------+-----------------------------------------------------------------------------------------------------------------------+ 3) Observer the the VM is booted on node0 and check that the MAC of the port is updated to the mac of one of the PF on node0 stack@master0:~$ openstack port show port-direct-physical | grep mac | mac_address | b4:96:91:34:ed:d6 | 4) Cold migrate the VM stack@master0:~$ openstack server migrate vm1 stack@master0:~$ openstack server migration confirm vm1 5) Observe that the VM is now running on master0 but MAC of the port is not changed stack@master0:~$ openstack server show vm1 | grep hypervisor | OS-EXT-SRV-ATTR:hypervisor_hostname | master0 | stack@master0:~$ openstack port show port-direct-physical | grep mac | mac_address | b4:96:91:34:ed:d6 | Expected behavior ================= The MAC of the neutron port is updated according the the MAC of the PF on master0 Environment =========== multinode devstack on recent master (nova on 033af94179) To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1942329/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp