Public bug reported:
Description
===========
It seems that the main failure happens due to the incorrect NUMA filtering in
the pci allocation mechanism. The allocation is being done according to the
instance NUMA topology, however, this is not always correct. Specifically in
the case when a user selects hw:numa_nodes=1, which would mean that VM will
take resources from just one numa node and not from a specific one.
Steps to reproduce
==================
Create nova flavor with NUMA awareness, CPU pinning, Huge pages, etc:
# nova flavor-create prefer_pin_1 auto 2048 20 1
# nova flavor-key prefer_pin_1 set hw:numa_nodes=1
# nova flavor-key prefer_pin_1 set hw:mem_page_size=1048576
# nova flavor-key prefer_pin_1 set hw:numa_mempolicy=strict
# nova flavor-key prefer_pin_1 set hw:cpu_policy=dedicated
# nova flavor-key prefer_pin_1 set hw:cpu_thread_policy=prefer
Then instantiate VMs with direct-physical neutron ports:
neutron port-create nfv_sriov --binding:vnic-type direct-physical --name pf1
nova boot pf1 --flavor prefer_pin_1 --image centos_udev --nic
port-id=a0fe88f6-07cc-4c70-b702-1915e36ed728
neutron port-create nfv_sriov --binding:vnic-type direct-physical --name pf2
nova boot pf2 --flavor prefer_pin_1 --image centos_udev --nic
port-id=b96de3ec-ef94-428b-96bc-dc46623a2427
Third VM instantiation failed. Our environment has got 4 NICs configured
to be allocated. However, with a regular flavor (m1.normal), the
instantiation works:
neutron port-create nfv_sriov --binding:vnic-type direct-physical --name pf3
nova boot pf3 --flavor 2 --image centos_udev --nic
port-id=52caacfe-0324-42bd-84ad-9a54d80e8fbe
neutron port-create nfv_sriov --binding:vnic-type direct-physical --name pf4
nova boot pf4 --flavor 2 --image centos_udev --nic
port-id=7335a9a6-82d0-4595-bb88-754678db56ef
Expected result
===============
PCI passthrough (PFs and VFs) should work in an environment with
NUMATopologyFilter enable
Actual result
=============
Checking availability of NICs with NUMATopologyFilter is not working.
Environment
===========
1 controller + 1 compute.
OpenStack Mitaka
Logs & Configs
==============
See attachment
** Affects: nova
Importance: Undecided
Status: New
** Attachment added: "sosreport-nfv100.hi.inet-20160609134718.tar.xz"
https://bugs.launchpad.net/bugs/1590746/+attachment/4680374/+files/sosreport-nfv100.hi.inet-20160609134718.tar.xz
--
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1590746
Title:
SRIOV PF/VF allocation fails with NUMA aware flavor
Status in OpenStack Compute (nova):
New
Bug description:
Description
===========
It seems that the main failure happens due to the incorrect NUMA filtering in
the pci allocation mechanism. The allocation is being done according to the
instance NUMA topology, however, this is not always correct. Specifically in
the case when a user selects hw:numa_nodes=1, which would mean that VM will
take resources from just one numa node and not from a specific one.
Steps to reproduce
==================
Create nova flavor with NUMA awareness, CPU pinning, Huge pages, etc:
# nova flavor-create prefer_pin_1 auto 2048 20 1
# nova flavor-key prefer_pin_1 set hw:numa_nodes=1
# nova flavor-key prefer_pin_1 set hw:mem_page_size=1048576
# nova flavor-key prefer_pin_1 set hw:numa_mempolicy=strict
# nova flavor-key prefer_pin_1 set hw:cpu_policy=dedicated
# nova flavor-key prefer_pin_1 set hw:cpu_thread_policy=prefer
Then instantiate VMs with direct-physical neutron ports:
neutron port-create nfv_sriov --binding:vnic-type direct-physical --name pf1
nova boot pf1 --flavor prefer_pin_1 --image centos_udev --nic
port-id=a0fe88f6-07cc-4c70-b702-1915e36ed728
neutron port-create nfv_sriov --binding:vnic-type direct-physical --name pf2
nova boot pf2 --flavor prefer_pin_1 --image centos_udev --nic
port-id=b96de3ec-ef94-428b-96bc-dc46623a2427
Third VM instantiation failed. Our environment has got 4 NICs
configured to be allocated. However, with a regular flavor
(m1.normal), the instantiation works:
neutron port-create nfv_sriov --binding:vnic-type direct-physical --name pf3
nova boot pf3 --flavor 2 --image centos_udev --nic
port-id=52caacfe-0324-42bd-84ad-9a54d80e8fbe
neutron port-create nfv_sriov --binding:vnic-type direct-physical --name pf4
nova boot pf4 --flavor 2 --image centos_udev --nic
port-id=7335a9a6-82d0-4595-bb88-754678db56ef
Expected result
===============
PCI passthrough (PFs and VFs) should work in an environment with
NUMATopologyFilter enable
Actual result
=============
Checking availability of NICs with NUMATopologyFilter is not working.
Environment
===========
1 controller + 1 compute.
OpenStack Mitaka
Logs & Configs
==============
See attachment
To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1590746/+subscriptions
--
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help : https://help.launchpad.net/ListHelp
