On 10/01/2018 09:01 AM, Jim Rollenhagen wrote:
On Mon, Oct 1, 2018 at 8:03 AM Jay Pipes <jaypi...@gmail.com
<mailto:jaypi...@gmail.com>> wrote:
On 10/01/2018 04:36 AM, John Garbutt wrote:
> On Fri, 28 Sep 2018 at 00:46, Jay Pipes <jaypi...@gmail.com
<mailto:jaypi...@gmail.com>
> <mailto:jaypi...@gmail.com <mailto:jaypi...@gmail.com>>> wrote:
>
> On 09/27/2018 06:23 PM, Matt Riedemann wrote:
> > On 9/27/2018 3:02 PM, Jay Pipes wrote:
> >> A great example of this would be the proposed "deploy
template"
> from
> >> [2]. This is nothing more than abusing the placement
traits API in
> >> order to allow passthrough of instance configuration data
from the
> >> nova flavor extra spec directly into the nodes.instance_info
> field in
> >> the Ironic database. It's a hack that is abusing the entire
> concept of
> >> the placement traits concept, IMHO.
> >>
> >> We should have a way *in Nova* of allowing instance
configuration
> >> key/value information to be passed through to the virt
driver's
> >> spawn() method, much the same way we provide for
user_data that
> gets
> >> exposed after boot to the guest instance via configdrive
or the
> >> metadata service API. What this deploy template thing is
is just a
> >> hack to get around the fact that nova doesn't have a
basic way of
> >> passing through some collated instance configuration
key/value
> >> information, which is a darn shame and I'm really kind of
> annoyed with
> >> myself for not noticing this sooner. :(
> >
> > We talked about this in Dublin through right? We said a good
> thing to do
> > would be to have some kind of template/profile/config/whatever
> stored
> > off in glare where schema could be registered on that
thing, and
> then
> > you pass a handle (ID reference) to that to nova when
creating the
> > (baremetal) server, nova pulls it down from glare and hands it
> off to
> > the virt driver. It's just that no one is doing that work.
>
> No, nobody is doing that work.
>
> I will if need be if it means not hacking the placement API
to serve
> this purpose (for which it wasn't intended).
>
>
> Going back to the point Mark Goddard made, there are two things here:
>
> 1) Picking the correct resource provider
> 2) Telling Ironic to transform the picked node in some way
>
> Today we allow the use Capabilities for both.
>
> I am suggesting we move to using Traits only for (1), leaving (2) in
> place for now, while we decide what to do (i.e. future of "deploy
> template" concept).
>
> It feels like Ironic's plan to define the "deploy templates" in
Ironic
> should replace the dependency on Glare for this use case, largely
> because the definition of the deploy template (in my mind) is very
> heavily related to inspector and driver properties, etc. Mark is
looking
> at moving that forward at the moment.
That won't do anything about the flavor explosion problem, though,
right
John?
Does nova still plan to allow passing additional desired traits into the
server create request?
I (we?) was kind of banking on that to solve the Baskin Robbins thing.
That's precisely what I've been looking into. From what I can tell,
Ironic was planning on using these CUSTOM_DEPLOY_TEMPLATE_XXX traits in
two ways:
1) To tell Nova what scheduling constraints the instance needed -- e.g.
"hey Nova, make sure I land on a node that supports UEFI boot mode
because my boot image relies on that".
2) As a convenient (because it would require no changes to Nova) way of
passing instance pre-spawn configuration data to the Ironic virt driver
-- e.g. pass the entire set of traits that are in the RequestSpec's
flavor and image extra specs to Ironic before calling the Ironic node
provision API.
#1 is fine IMHO, since it (mostly) represents a "capability" that the
resource provider (the Ironic baremetal node) must support in order for
the instance to successfully boot.
#2 is a problem, though, because it *doesn't* represent a capability. In
fact, it can represent any and all sorts of key/value, JSON/dict or
other information and this information is not intended to be passed to
the placement/scheduler service as a constraint. It is this data, also,
that tends to create the flavor explosion problem because it means that
Ironic deployers need to create dozens of flavors that vary only
slightly based on a user's desired deployment configuration.
So, deployers end up needing to create dozens of flavors varying only
slightly by things like RAID level or some pre-defined disk partitioning
layout. For example, the deployer might have dozens of flavors that look
like this:
- 12CPU_128G_RAID10_DRIVE_LAYOUT_X
- 12CPU_128G_RAID5_DRIVE_LAYOUT_X
- 12CPU_128G_RAID01_DRIVE_LAYOUT_X
- 12CPU_128G_RAID10_DRIVE_LAYOUT_Y
- 12CPU_128G_RAID5_DRIVE_LAYOUT_Y
- 12CPU_128G_RAID01_DRIVE_LAYOUT_Y
which is the flavor explosion problem we talk about.
Add into that mix the desire to have certain functionality "activated"
-- like "this Ironic node supports UEFI boot mode but I need to twiddle
some foo before I can provision using UEFI boot mode" -- and things just
get ugly.
I suppose the long-term solution I would like to see is a clean
separation of #1 from #2, with traits being used as-is for the *required
capabilities* (i.e. does this node support hardware RAID?) and separate
command-line arguments for #2, with those command-line arguments being
references to Glance metadef items. That way, Glance metadefs can be
used for validating the data supplied to the command line instead of
Nova needing to add that data validation logic.
So, the long-term command line solution might look something like this:
# list the disk partitioning schemes from glance
openstack metadefs list --object disk_partitioning_scheme
# Choose the disk partitioning scheme you want...
# let's say that this disk partitioning scheme is a JSON
# document that looks like this (the schema for this document
# would belong to the metadef schema in Glance)
{
"type": "disk_partitioning_scheme",
"version": 123,
"object": {
"raid" : {
"level": 5
},
"partitioning_table_format": "gpt",
"partitions": {
"/dev/sda": {
"format": "ext4",
"label": "os",
"size": "256MB",
"mountpoint": "/"
},
"/dev/sdb": {
"format": "ext4",
"label": "home",
"size": "2TB",
"mountpoint": "/home"
}...
}
},
"placement": {
"traits": {
"required": [
"STORAGE_DISK_SSD",
"STORAGE_RAID_HARDWARE",
"BOOT_MODE_UEFI"
]
}
}
}
openstack server create --flavor METAL_12CPU_128G \
--image $IMAGE_UUID \
--config-data $PARTITIONING_SCHEME_UUID
# nova would see the "placement:traits" collection in the config-data
# object and would merge those trait constraints into the constraints it
# constructs from the flavor extra specs and image metadata, passing
# those constraints directly to placement for scheduling filters.
# Note that this is virtually identical to the solution we developed
# with the Neutron team to put a "network:capabilities" section in the
# port binding profile to indicate the required traits for a port...
# nova would pass the config-data object to the virt driver
# to do with as it wants.
One alternative could also have Ironic folks separate disk partitioning
from RAID configuration, making separate disk_partitioning_scheme and
raid_config metadef items in Glance, decoupling the RAID configuration
from the disk partitioning configuration. The user could then specify
different RAID config and disk partitioning on the command line, like so:
openstack server create --flavor METAL_12CPU_128G \
--image $IMAGE_UUID \
--config-data $PARTIONING_SCHEME_UUID \
--config-data $RAID_CONFIG_UUID
Anyway, these are the thoughts I'm typing up in the mentioned spec...
-jay
__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
