On 09/28/2017 11:37 AM, Sahid Orentino Ferdjaoui wrote:
Please consider the support of MDEV for the /pci framework which
provides support for vGPUs [0].
Accordingly to the discussion [1]
With this first implementation which could be used as a skeleton for
implementing PCI Devices in Resource Tracker
I'm not entirely sure what you're referring to above as "implementing
PCI devices in Resource Tracker". Could you elaborate? The resource
tracker already embeds a PciManager object that manages PCI devices, as
you know. Perhaps you meant "implement PCI devices as Resource Providers"?
we provide support for
attaching vGPUs to guests. And also to provide affinity per NUMA
nodes. An other important point is that that implementation can take
advantage of the ongoing specs like PCI NUMA policies.
* The Implementation [0]
[PATCH 01/13] pci: update PciDevice object field 'address' to accept
[PATCH 02/13] pci: add for PciDevice object new field mdev
[PATCH 03/13] pci: generalize object unit-tests for different
[PATCH 04/13] pci: add support for mdev device type request
[PATCH 05/13] pci: generalize stats unit-tests for different
[PATCH 06/13] pci: add support for mdev devices type devspec
[PATCH 07/13] pci: add support for resource pool stats of mdev
[PATCH 08/13] pci: make manager to accept handling mdev devices
In this serie of patches we are generalizing the PCI framework to
handle MDEV devices. We arguing it's a lot of patches but most of them
are small and the logic behind is basically to make it understand two
new fields MDEV_PF and MDEV_VF.
That's not really "generalizing the PCI framework to handle MDEV
devices" :) More like it's just changing the /pci module to understand a
different device management API, but ok.
[PATCH 09/13] libvirt: update PCI node device to report mdev devices
[PATCH 10/13] libvirt: report mdev resources
[PATCH 11/13] libvirt: add support to start vm with using mdev (vGPU)
In this serie of patches we make libvirt driver support, as usually,
return resources and attach devices returned by the pci manager. This
part can be reused for Resource Provider.
Perhaps, but the idea behind the resource providers framework is to
treat devices as generic things. Placement doesn't need to know about
the particular device attachment status.
[PATCH 12/13] functional: rework fakelibvirt host pci devices
[PATCH 13/13] libvirt: resuse SRIOV funtional tests for MDEV devices
Here we reuse 100/100 of the functional tests used for SR-IOV
devices. Again here, this part can be reused for Resource Provider.
Probably not, but I'll take a look :)
For the record, I have zero confidence in any existing "functional"
tests for NUMA, SR-IOV, CPU pinning, huge pages, and the like.
Unfortunately, due to the fact that these features often require
hardware that either the upstream community CI lacks or that depends on
libraries, drivers and kernel versions that really aren't available to
non-bleeding edge users (or users with very deep pockets).
* The Usage
There are no difference between SR-IOV and MDEV, from operators point
of view who knows how to expose SR-IOV devices in Nova, they already
know how to expose MDEV devices (vGPUs).
Operators will be able to expose MDEV devices in the same manner as
they expose SR-IOV:
1/ Configure whitelist devices
['{"vendor_id":"10de"}']
2/ Create aliases
[{"vendor_id":"10de", "name":"vGPU"}]
3/ Configure the flavor
openstack flavor set --property "pci_passthrough:alias"="vGPU:1"
* Limitations
The mdev does not provide 'product_id' but 'mdev_type' which should be
considered to exactly identify which resource users can request e.g:
nvidia-10. To provide that support we have to add a new field
'mdev_type' so aliases could be something like:
{"vendor_id":"10de", mdev_type="nvidia-10" "name":"alias-nvidia-10"}
{"vendor_id":"10de", mdev_type="nvidia-11" "name":"alias-nvidia-11"}
I do have plan to add but first I need to have support from upstream
to continue that work.
As mentioned in IRC and the previous ML discussion, my focus is on the
nested resource providers work and reviews, along with the other two
top-priority scheduler items (move operations and alternate hosts).
I'll do my best to look at your patch series, but please note it's lower
priority than a number of other items.
One thing that would be very useful, Sahid, if you could get with Eric
Fried (efried) on IRC and discuss with him the "generic device
management" system that was discussed at the PTG. It's likely that the
/pci module is going to be overhauled in Rocky and it would be good to
have the mdev device management API requirements included in that
discussion.
Best,
-jay
Best,
-jay
[0]
https://review.openstack.org/#/q/status:open+project:openstack/nova+branch:master+topic:pci-mdev-support
[1]
http://lists.openstack.org/pipermail/openstack-dev/2017-September/122591.html
__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
