> On 5 Jun 2018, at 14:56, Eric Fried <[email protected]> wrote: > > To summarize: cyborg could model things nested-wise, but there would be > no way to schedule them yet. > > Couple of clarifications inline. > > On 06/05/2018 08:29 AM, Jay Pipes wrote: >> On 06/05/2018 08:50 AM, Stephen Finucane wrote: >>> I thought nested resource providers were already supported by >>> placement? To the best of my knowledge, what is /not/ supported is >>> virt drivers using these to report NUMA topologies but I doubt that >>> affects you. The placement guys will need to weigh in on this as I >>> could be missing something but it sounds like you can start using this >>> functionality right now. >> >> To be clear, this is what placement and nova *currently* support with >> regards to nested resource providers: >> >> 1) When creating a resource provider in placement, you can specify a >> parent_provider_uuid and thus create trees of providers. This was >> placement API microversion 1.14. Also included in this microversion was >> support for displaying the parent and root provider UUID for resource >> providers. >> >> 2) The nova "scheduler report client" (terrible name, it's mostly just >> the placement client at this point) understands how to call placement >> API 1.14 and create resource providers with a parent provider. >> >> 3) The nova scheduler report client uses a ProviderTree object [1] to >> cache information about the hierarchy of providers that it knows about. >> For nova-compute workers managing hypervisors, that means the >> ProviderTree object contained in the report client is rooted in a >> resource provider that represents the compute node itself (the >> hypervisor). For nova-compute workers managing baremetal, that means the >> ProviderTree object contains many root providers, each representing an >> Ironic baremetal node. >> >> 4) The placement API's GET /allocation_candidates endpoint now >> understands the concept of granular request groups [2]. Granular request >> groups are only relevant when a user wants to specify that child >> providers in a provider tree should be used to satisfy part of an >> overall scheduling request. However, this support is yet incomplete -- >> see #5 below. > > Granular request groups are also usable/useful when sharing providers > are in play. That functionality is complete on both the placement side > and the report client side (see below). > >> The following parts of the nested resource providers modeling are *NOT* >> yet complete, however: >> >> 5) GET /allocation_candidates does not currently return *results* when >> granular request groups are specified. So, while the placement service >> understands the *request* for granular groups, it doesn't yet have the >> ability to constrain the returned candidates appropriately. Tetsuro is >> actively working on this functionality in this patch series: >> >> https://review.openstack.org/#/q/status:open+project:openstack/nova+branch:master+topic:bp/nested-resource-providers-allocation-candidates >> >> >> 6) The virt drivers need to implement the update_provider_tree() >> interface [3] and construct the tree of resource providers along with >> appropriate inventory records for each child provider in the tree. Both >> libvirt and XenAPI virt drivers have patch series up that begin to take >> advantage of the nested provider modeling. However, a number of concerns >> [4] about in-place nova-compute upgrades when moving from a single >> resource provider to a nested provider tree model were raised, and we >> have begun brainstorming how to handle the migration of existing data in >> the single-provider model to the nested provider model. [5] We are >> blocking any reviews on patch series that modify the local provider >> modeling until these migration concerns are fully resolved. >> >> 7) The scheduler does not currently pass granular request groups to >> placement. > > The code is in place to do this [6] - so the scheduler *will* pass > granular request groups to placement if your flavor specifies them. As > noted above, such flavors will be limited to exploiting sharing > providers until Tetsuro's series merges. But no further code work is > required on the scheduler side. > > [6] https://review.openstack.org/#/c/515811/ > >> Once #5 and #6 are resolved, and once the migration/upgrade >> path is resolved, clearly we will need to have the scheduler start >> making requests to placement that represent the granular request groups >> and have the scheduler pass the resulting allocation candidates to its >> filters and weighers. >> >> Hope this helps highlight where we currently are and the work still left >> to do (in Rocky) on nested resource providers. >> >> Best, >> -jay >> >> >> [1] >> https://github.com/openstack/nova/blob/master/nova/compute/provider_tree.py >> >> [2] >> https://specs.openstack.org/openstack/nova-specs/specs/queens/approved/granular-resource-requests.html >> >> >> [3] >> https://github.com/openstack/nova/blob/f902e0d5d87fb05207e4a7aca73d185775d43df2/nova/virt/driver.py#L833 >> >> >> [4] http://lists.openstack.org/pipermail/openstack-dev/2018-May/130783.html >> >> [5] https://etherpad.openstack.org/p/placement-making-the-(up)grade >> >> __________________________________________________________________________ >> OpenStack Development Mailing List (not for usage questions) >> Unsubscribe: [email protected]?subject:unsubscribe >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > __________________________________________________________________________ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: [email protected]?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
__________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: [email protected]?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
