[openstack-dev] [ironic] using ironic as a replacement for existing datacenter baremetal provisioning
Hi ironic folks, As I'm trying to explore how GoDaddy can use ironic I've created the following in an attempt to document some of my concerns, and I'm wondering if you folks could help myself identity ongoing work to solve these (or alternatives?) List of concerns with ironic: 1.)Nova <-> ironic interactions are generally seem terrible? -How to accept raid config and partitioning(?) from end users? Seems to not a yet agreed upon method between nova/ironic. -How to run multiple conductors/nova-computes? Right now as far as I can tell all of ironic front-end by a single nova-compute, which I will have to manage via a cluster technology between two or mode nodes. Because of this and the way host-agregates work I am unable to expose fault domains for ironic instances (all of ironic can only be under a single AZ (the az that is assigned to the nova-compute node)). Unless I create multiple nova-compute servers and manage multiple independent ironic setups. This makes on-boarding/query of hardware capacity painful. - Nova appears to be forcing a we are "compute" as long as "compute" is VMs, means that we will have a baremetal flavor explosion (ie the mismatch between baremetal and VM). - This is a feeling I got from the ironic-nova cross project meeting in Austin. General exmaple goes back to raid config above. I can configure a single piece of hardware many different ways, but to fit into nova's world view I need to have many different flavors exposed to end-user. In this way many flavors can map back to a single piece of hardware with just a lsightly different configuration applied. So how am I suppose to do a single server with 6 drives as either: Raid 1 + Raid 5, Raid 5, Raid 10, Raid 6, or JBOD. Seems like I would need to pre-mark out servers that were going to be a specific raid level. Which means that I need to start managing additional sub-pools of hardware to just deal with how the end users wants the raid configured, this is pretty much a non-starter for us. I have not really heard of whats being done on this specific front. 2.) Inspector: - IPA service doesn't gather port/switching information - Inspection service doesn't process port/switching information, which means that it wont add it to ironic. Which makes managing network swinging of the host a non-starter. As I would inspect the host – then modify the ironic record to add the details about what port/switch the server is connected to from a different source. At that point why wouldn't I just onboard everything through the API? - Doesn't grab hardware disk configurations, If the server has multiple raids (r1 + r5) only reports boot raid disk capacity. - Inspection is geared towards using a different network and dnsmasq infrastructure than what is in use for ironic/neutron. Which also means that in order to not conflict with dhcp requests for servers in ironic I need to use different networks. Which also means I now need to handle swinging server ports between different networks. 3.) IPA image: - Default build stuff is pinned to extremly old versions due to gate failure issues. So I can not work without a fork for onboard of servers due to the fact that IPMI modules aren't built for the kernel, so inspection can never match the node against ironic. Seems like currently functionality here is MVP for gate to work and to deploy images. But if you need to do firmware, bios-config, any other hardware specific features you are pretty much going to need to roll your own IPA image and IPA modules to do standard provisioning tasks. 4.) Conductor: - Serial-over-lan consoles require a unique port on the conductor server (I have seen purposes to try and fix this?), this is painful to manage with large numbers of servers. - SOL consoles aren't restarted when conductor is restarted (I think this might be fixed in newer versions of ironic?), again if end users aren't suppose to consume ironic api's directly - this is painful to handle. - Its very easy to get a node to fall off the staemachine rails (reboot a server while an image is being deployed to it), the only way I have seen to be able to fix this is to update the DB directly. - As far as I can tell shell-in-a- box, SOL consoles aren't support via nova – so how are end users suppose to consume the shell-in-box console? - I have BMC that need specific configuration (some require SOL on com2, others on com1) this makes it pretty much impossible without per box overrides against the conductor hardcoded templates. - Additionally it would be nice to default to having a provisioning kernel/image that was set as a single config option with per server overrides – rather than on each server. If we ever change the IPA image – that means at scale we would need to update thousands of ironic nodes. What is ironic doing to monitor the hardware for failures? I assume the answer here is nothing and that we will nee
Re: [openstack-dev] [ironic] using ironic as a replacement for existing datacenter baremetal provisioning
Excerpts from Kris G. Lindgren's message of 2016-06-06 20:44:26 +: > Hi ironic folks, > As I'm trying to explore how GoDaddy can use ironic I've created the > following in an attempt to document some of my concerns, and I'm wondering if > you folks could help myself identity ongoing work to solve these (or > alternatives?) Hi Kris. I've been using Ironic in various forms for a while, and I can answer a few of these things. > List of concerns with ironic: > > 1.)Nova <-> ironic interactions are generally seem terrible? I don't know if I'd call it terrible, but there's friction. Things that are unchangable on hardware are just software configs in vms (like mac addresses, overlays, etc), and things that make no sense in VMs are pretty standard on servers (trunked vlans, bonding, etc). One way we've gotten around it is by using Ironic standalone via Bifrost[1]. This deploys Ironic in wide open auth mode on 127.0.0.1, and includes playbooks to build config drives and deploy images in a fairly rudimentary way without Nova. I call this the "better than Cobbler" way of getting a toe into the Ironic waters. [1] https://github.com/openstack/bifrost > -How to accept raid config and partitioning(?) from end users? Seems to not > a yet agreed upon method between nova/ironic. AFAIK accepting it from the users just isn't solved. Administrators do have custom ramdisks that they boot to pre-configure RAID during enrollment. >-How to run multiple conductors/nova-computes? Right now as far as I can > tell all of ironic front-end by a single nova-compute, which I will have to > manage via a cluster technology between two or mode nodes. Because of this > and the way host-agregates work I am unable to expose fault domains for > ironic instances (all of ironic can only be under a single AZ (the az that is > assigned to the nova-compute node)). Unless I create multiple nova-compute > servers and manage multiple independent ironic setups. This makes > on-boarding/query of hardware capacity painful. The nova-compute does almost nothing. It really just talks to the scheduler to tell it what's going on in Ironic. If it dies, deploys won't stop. You can run many many conductors and spread load and fault tolerance among them easily. I think for multiple AZs though, you're right, there's no way to expose that. Perhaps it can be done with cells, which I think Rackspace's OnMetal uses (but I'll let them refute or confirm that). Seems like the virt driver could be taught to be AZ-aware and some metadata in the server record could allow AZs to go through to Ironic. > - Nova appears to be forcing a we are "compute" as long as "compute" is > VMs, means that we will have a baremetal flavor explosion (ie the mismatch > between baremetal and VM). > - This is a feeling I got from the ironic-nova cross project meeting in > Austin. General exmaple goes back to raid config above. I can configure a > single piece of hardware many different ways, but to fit into nova's world > view I need to have many different flavors exposed to end-user. In this way > many flavors can map back to a single piece of hardware with just a lsightly > different configuration applied. So how am I suppose to do a single server > with 6 drives as either: Raid 1 + Raid 5, Raid 5, Raid 10, Raid 6, or JBOD. > Seems like I would need to pre-mark out servers that were going to be a > specific raid level. Which means that I need to start managing additional > sub-pools of hardware to just deal with how the end users wants the raid > configured, this is pretty much a non-starter for us. I have not really > heard of whats being done on this specific front. You got that right. Perhaps people are comfortable with this limitation. It is at least simple. > > 2.) Inspector: > - IPA service doesn't gather port/switching information > - Inspection service doesn't process port/switching information, which > means that it wont add it to ironic. Which makes managing network swinging > of the host a non-starter. As I would inspect the host – then modify the > ironic record to add the details about what port/switch the server is > connected to from a different source. At that point why wouldn't I just > onboard everything through the API? > - Doesn't grab hardware disk configurations, If the server has multiple > raids (r1 + r5) only reports boot raid disk capacity. > - Inspection is geared towards using a different network and dnsmasq > infrastructure than what is in use for ironic/neutron. Which also means that > in order to not conflict with dhcp requests for servers in ironic I need to > use different networks. Which also means I now need to handle swinging > server ports between different networks. > > 3.) IPA image: > - Default build stuff is pinned to extremly old versions due to gate > failure issues. So I can not work without a fork for onboard of servers due > to the fact that IPMI modules a
Re: [openstack-dev] [ironic] using ironic as a replacement for existing datacenter baremetal provisioning
On 06/06/2016 01:44 PM, Kris G. Lindgren wrote: > Hi ironic folks, > As I'm trying to explore how GoDaddy can use ironic I've created the following > in an attempt to document some of my concerns, and I'm wondering if you folks > could help myself identity ongoing work to solve these (or alternatives?) > List of concerns with ironic: Hi Kris, There is a lot of ongoing work in and around the Ironic project. Thanks for diving in and for sharing your concerns; you're not alone. I'll respond to each group of concerns, as some of these appear quite similar to each other and align with stuff we're already doing. Hopefully I can provide some helpful background to where the project is at today. > > 1.)Nova <-> ironic interactions are generally seem terrible? These two projects are coming at the task of managing "compute" with significantly different situations and we've been working, for the last ~2 years, to build a framework that can provide both virtual and physical resources through one API. It's not a simple task, and we have a lot more to do. > -How to accept raid config and partitioning(?) from end users? Seems to not > a > yet agreed upon method between nova/ironic. Nova expresses partitioning in a very limited way on the flavor. You get root, swap, and ephemeral partitions -- and that's it. Ironic honors those today, but they're pinned on the flavor definition, eg. by the cloud admin (or whoever can define the flavor. If your users need more complex partitioning, they could create additional partitions after the instance is created. This limitation within Ironic exists, in part, because the projects' goal is to provide hardware through the OpenStack Compute API -- which doesn't express arbitrary partitionability. (If you're interested, there is a lengthier and more political discussion about whether the cloud should support "pets" and whether arbitrary partitioning is needed for "cattle".) RAID configuration isn't something that Nova allows their users to choose today - it doesn't fit in the Nova model of "compute", and there is, to my knowledge, nothing in the Nova API to allow its input. We've discussed this a little bit, but so far settled on leaving it up to the cloud admin to set this in Ironic. There has been discussion with the Cinder community over ways to express volume spanning and mirroring, but apply it to a machines' local disks, but these discussions didn't result in any traction. There's also been discussion of ways we could do ad-hoc changes in RAID level, based on flavor metadata, during the provisioning process (rather than ahead of time) but no code has been done for this yet, AFAIK. So, where does that leave us? With the "explosion of flavors" that you described. It may not be ideal, but that is the common ground we've reached. >-How to run multiple conductors/nova-computes? Right now as far as I can > tell all of ironic front-end by a single nova-compute, which I will have to > manage via a cluster technology between two or mode nodes. Because of this > and > the way host-agregates work I am unable to expose fault domains for ironic > instances (all of ironic can only be under a single AZ (the az that is > assigned > to the nova-compute node)). Unless I create multiple nova-compute servers and > manage multiple independent ironic setups. This makes on-boarding/query of > hardware capacity painful. Yep. It's not ideal, and the community is very well aware of, and actively working on, this limitation. It also may not be as bad as you may think. The nova-compute process doesn't do very much, and tests show it handling some thousands of ironic nodes fairly well in parallel. Standard active-passive management of that process should suffice. A lot of design work has been done to come up with a joint solution by folks on both the Ironic and Nova teams. http://specs.openstack.org/openstack/nova-specs/specs/newton/approved/ironic-multiple-compute-hosts.html As a side note, it's possible (though not tested, recommended, or well documented) to run more than one nova-compute. See https://github.com/openstack/ironic/blob/master/ironic/nova/compute/manager.py > - Nova appears to be forcing a we are "compute" as long as "compute" is VMs, > means that we will have a baremetal flavor explosion (ie the mismatch between > baremetal and VM). > - This is a feeling I got from the ironic-nova cross project meeting in > Austin. General exmaple goes back to raid config above. I can configure a > single piece of hardware many different ways, but to fit into nova's world > view > I need to have many different flavors exposed to end-user. In this way many > flavors can map back to a single piece of hardware with just a lsightly > different configuration applied. So how am I suppose to do a single server > with > 6 drives as either: Raid 1 + Raid 5, Raid 5, Raid 10, Raid 6, or JBOD. Seems > like I would need to pre-mark out servers that were going to be a specific >
Re: [openstack-dev] [ironic] using ironic as a replacement for existing datacenter baremetal provisioning
Thanks for getting to this before me, Deva. Saved me some typing. :) A little more color inline. On Mon, Jun 06, 2016 at 05:01:04PM -0700, Devananda van der Veen wrote: > > On 06/06/2016 01:44 PM, Kris G. Lindgren wrote: > > Hi ironic folks, > > As I'm trying to explore how GoDaddy can use ironic I've created the > > following > > in an attempt to document some of my concerns, and I'm wondering if you > > folks > > could help myself identity ongoing work to solve these (or alternatives?) > > List of concerns with ironic: > > Hi Kris, > > There is a lot of ongoing work in and around the Ironic project. Thanks for > diving in and for sharing your concerns; you're not alone. > > I'll respond to each group of concerns, as some of these appear quite similar > to > each other and align with stuff we're already doing. Hopefully I can provide > some helpful background to where the project is at today. > > > > > 1.)Nova <-> ironic interactions are generally seem terrible? > > These two projects are coming at the task of managing "compute" with > significantly different situations and we've been working, for the last ~2 > years, to build a framework that can provide both virtual and physical > resources > through one API. It's not a simple task, and we have a lot more to do. > > > > -How to accept raid config and partitioning(?) from end users? Seems to > > not a > > yet agreed upon method between nova/ironic. > > Nova expresses partitioning in a very limited way on the flavor. You get root, > swap, and ephemeral partitions -- and that's it. Ironic honors those today, > but > they're pinned on the flavor definition, eg. by the cloud admin (or whoever > can > define the flavor. > > If your users need more complex partitioning, they could create additional > partitions after the instance is created. This limitation within Ironic > exists, > in part, because the projects' goal is to provide hardware through the > OpenStack > Compute API -- which doesn't express arbitrary partitionability. (If you're > interested, there is a lengthier and more political discussion about whether > the > cloud should support "pets" and whether arbitrary partitioning is needed for > "cattle".) > > > RAID configuration isn't something that Nova allows their users to choose > today > - it doesn't fit in the Nova model of "compute", and there is, to my > knowledge, > nothing in the Nova API to allow its input. We've discussed this a little bit, > but so far settled on leaving it up to the cloud admin to set this in Ironic. > > There has been discussion with the Cinder community over ways to express > volume > spanning and mirroring, but apply it to a machines' local disks, but these > discussions didn't result in any traction. > > There's also been discussion of ways we could do ad-hoc changes in RAID level, > based on flavor metadata, during the provisioning process (rather than ahead > of > time) but no code has been done for this yet, AFAIK. > > So, where does that leave us? With the "explosion of flavors" that you > described. It may not be ideal, but that is the common ground we've reached. > > >-How to run multiple conductors/nova-computes? Right now as far as I > > can > > tell all of ironic front-end by a single nova-compute, which I will have to > > manage via a cluster technology between two or mode nodes. Because of this > > and > > the way host-agregates work I am unable to expose fault domains for ironic > > instances (all of ironic can only be under a single AZ (the az that is > > assigned > > to the nova-compute node)). Unless I create multiple nova-compute servers > > and > > manage multiple independent ironic setups. This makes on-boarding/query of > > hardware capacity painful. > > Yep. It's not ideal, and the community is very well aware of, and actively > working on, this limitation. It also may not be as bad as you may think. The > nova-compute process doesn't do very much, and tests show it handling some > thousands of ironic nodes fairly well in parallel. Standard active-passive > management of that process should suffice. > > A lot of design work has been done to come up with a joint solution by folks > on > both the Ironic and Nova teams. > http://specs.openstack.org/openstack/nova-specs/specs/newton/approved/ironic-multiple-compute-hosts.html It's important to point out here that we're re-working how this works, but it's still one of our highest priorities: https://review.openstack.org/#/c/320016/ > > As a side note, it's possible (though not tested, recommended, or well > documented) to run more than one nova-compute. See > https://github.com/openstack/ironic/blob/master/ironic/nova/compute/manager.py > > > - Nova appears to be forcing a we are "compute" as long as "compute" is > > VMs, > > means that we will have a baremetal flavor explosion (ie the mismatch > > between > > baremetal and VM). > > - This is a feeling I got from the ironic-nova cross pro
Re: [openstack-dev] [ironic] using ironic as a replacement for existing datacenter baremetal provisioning
On 06/07/2016 02:01 AM, Devananda van der Veen wrote: On 06/06/2016 01:44 PM, Kris G. Lindgren wrote: Hi ironic folks, As I'm trying to explore how GoDaddy can use ironic I've created the following in an attempt to document some of my concerns, and I'm wondering if you folks could help myself identity ongoing work to solve these (or alternatives?) List of concerns with ironic: Hi Kris, There is a lot of ongoing work in and around the Ironic project. Thanks for diving in and for sharing your concerns; you're not alone. I'll respond to each group of concerns, as some of these appear quite similar to each other and align with stuff we're already doing. Hopefully I can provide some helpful background to where the project is at today. 1.)Nova <-> ironic interactions are generally seem terrible? These two projects are coming at the task of managing "compute" with significantly different situations and we've been working, for the last ~2 years, to build a framework that can provide both virtual and physical resources through one API. It's not a simple task, and we have a lot more to do. -How to accept raid config and partitioning(?) from end users? Seems to not a yet agreed upon method between nova/ironic. Nova expresses partitioning in a very limited way on the flavor. You get root, swap, and ephemeral partitions -- and that's it. Ironic honors those today, but they're pinned on the flavor definition, eg. by the cloud admin (or whoever can define the flavor. If your users need more complex partitioning, they could create additional partitions after the instance is created. This limitation within Ironic exists, in part, because the projects' goal is to provide hardware through the OpenStack Compute API -- which doesn't express arbitrary partitionability. (If you're interested, there is a lengthier and more political discussion about whether the cloud should support "pets" and whether arbitrary partitioning is needed for "cattle".) RAID configuration isn't something that Nova allows their users to choose today - it doesn't fit in the Nova model of "compute", and there is, to my knowledge, nothing in the Nova API to allow its input. We've discussed this a little bit, but so far settled on leaving it up to the cloud admin to set this in Ironic. There has been discussion with the Cinder community over ways to express volume spanning and mirroring, but apply it to a machines' local disks, but these discussions didn't result in any traction. There's also been discussion of ways we could do ad-hoc changes in RAID level, based on flavor metadata, during the provisioning process (rather than ahead of time) but no code has been done for this yet, AFAIK. I'm still pretty interested in it, because I agree with anything said above about building RAID ahead-of-time not being convenient. I don't quite understand how such a feature would look like, we might add it as a topic for midcycle. So, where does that leave us? With the "explosion of flavors" that you described. It may not be ideal, but that is the common ground we've reached. -How to run multiple conductors/nova-computes? Right now as far as I can tell all of ironic front-end by a single nova-compute, which I will have to manage via a cluster technology between two or mode nodes. Because of this and the way host-agregates work I am unable to expose fault domains for ironic instances (all of ironic can only be under a single AZ (the az that is assigned to the nova-compute node)). Unless I create multiple nova-compute servers and manage multiple independent ironic setups. This makes on-boarding/query of hardware capacity painful. Yep. It's not ideal, and the community is very well aware of, and actively working on, this limitation. It also may not be as bad as you may think. The nova-compute process doesn't do very much, and tests show it handling some thousands of ironic nodes fairly well in parallel. Standard active-passive management of that process should suffice. A lot of design work has been done to come up with a joint solution by folks on both the Ironic and Nova teams. http://specs.openstack.org/openstack/nova-specs/specs/newton/approved/ironic-multiple-compute-hosts.html As a side note, it's possible (though not tested, recommended, or well documented) to run more than one nova-compute. See https://github.com/openstack/ironic/blob/master/ironic/nova/compute/manager.py - Nova appears to be forcing a we are "compute" as long as "compute" is VMs, means that we will have a baremetal flavor explosion (ie the mismatch between baremetal and VM). - This is a feeling I got from the ironic-nova cross project meeting in Austin. General exmaple goes back to raid config above. I can configure a single piece of hardware many different ways, but to fit into nova's world view I need to have many different flavors exposed to end-user. In this way many flavors can map back to a single piece of hardware with just a lsightly
Re: [openstack-dev] [ironic] using ironic as a replacement for existing datacenter baremetal provisioning
Clint Byrum wrote: Excerpts from Kris G. Lindgren's message of 2016-06-06 20:44:26 +: Hi ironic folks, As I'm trying to explore how GoDaddy can use ironic I've created the following in an attempt to document some of my concerns, and I'm wondering if you folks could help myself identity ongoing work to solve these (or alternatives?) Hi Kris. I've been using Ironic in various forms for a while, and I can answer a few of these things. List of concerns with ironic: 1.)Nova<-> ironic interactions are generally seem terrible? I don't know if I'd call it terrible, but there's friction. Things that are unchangable on hardware are just software configs in vms (like mac addresses, overlays, etc), and things that make no sense in VMs are pretty standard on servers (trunked vlans, bonding, etc). One way we've gotten around it is by using Ironic standalone via Bifrost[1]. This deploys Ironic in wide open auth mode on 127.0.0.1, and includes playbooks to build config drives and deploy images in a fairly rudimentary way without Nova. I call this the "better than Cobbler" way of getting a toe into the Ironic waters. [1] https://github.com/openstack/bifrost Out of curiosity, why ansible vs turning https://github.com/openstack/nova/blob/master/nova/virt/ironic/driver.py (or something like it) into a tiny-wsgi-app (pick useful name here) that has its own REST api (that looks pretty similar to the public functions in that driver file)? That seems almost easier than building a bunch of ansible scripts that appear (at a glance) to do similar things; and u get the benefit of using a actual programming language vs a half-programming-ansible-yaml-language... A realization I'm having is that I'm really not a fan of using ansible as a half-programming-ansible-yaml-language, which it seems like people start to try to do after a while (because at some point you need something like if statements, then things like [1] get created), no offense to the authors, but I guess this is my personal preference (it's also one of the reasons taskflow directly is a lib. in python, because , people don't need to learn a new language). [1] https://github.com/openstack/bifrost/blob/master/playbooks/roles/ironic-enroll-dynamic/tasks/main.yml -How to accept raid config and partitioning(?) from end users? Seems to not a yet agreed upon method between nova/ironic. AFAIK accepting it from the users just isn't solved. Administrators do have custom ramdisks that they boot to pre-configure RAID during enrollment. -How to run multiple conductors/nova-computes? Right now as far as I can tell all of ironic front-end by a single nova-compute, which I will have to manage via a cluster technology between two or mode nodes. Because of this and the way host-agregates work I am unable to expose fault domains for ironic instances (all of ironic can only be under a single AZ (the az that is assigned to the nova-compute node)). Unless I create multiple nova-compute servers and manage multiple independent ironic setups. This makes on-boarding/query of hardware capacity painful. The nova-compute does almost nothing. It really just talks to the scheduler to tell it what's going on in Ironic. If it dies, deploys won't stop. You can run many many conductors and spread load and fault tolerance among them easily. I think for multiple AZs though, you're right, there's no way to expose that. Perhaps it can be done with cells, which I think Rackspace's OnMetal uses (but I'll let them refute or confirm that). Seems like the virt driver could be taught to be AZ-aware and some metadata in the server record could allow AZs to go through to Ironic. - Nova appears to be forcing a we are "compute" as long as "compute" is VMs, means that we will have a baremetal flavor explosion (ie the mismatch between baremetal and VM). - This is a feeling I got from the ironic-nova cross project meeting in Austin. General exmaple goes back to raid config above. I can configure a single piece of hardware many different ways, but to fit into nova's world view I need to have many different flavors exposed to end-user. In this way many flavors can map back to a single piece of hardware with just a lsightly different configuration applied. So how am I suppose to do a single server with 6 drives as either: Raid 1 + Raid 5, Raid 5, Raid 10, Raid 6, or JBOD. Seems like I would need to pre-mark out servers that were going to be a specific raid level. Which means that I need to start managing additional sub-pools of hardware to just deal with how the end users wants the raid configured, this is pretty much a non-starter for us. I have not really heard of whats being done on this specific front. You got that right. Perhaps people are comfortable with this limitation. It is at least simple. 2.) Inspector: - IPA service doesn't gather port/switching information - Inspection service doesn't process port/swi
Re: [openstack-dev] [ironic] using ironic as a replacement for existing datacenter baremetal provisioning
Excerpts from Joshua Harlow's message of 2016-06-07 08:46:28 -0700: > Clint Byrum wrote: > > Excerpts from Kris G. Lindgren's message of 2016-06-06 20:44:26 +: > >> Hi ironic folks, > >> As I'm trying to explore how GoDaddy can use ironic I've created the > >> following in an attempt to document some of my concerns, and I'm wondering > >> if you folks could help myself identity ongoing work to solve these (or > >> alternatives?) > > > > Hi Kris. I've been using Ironic in various forms for a while, and I can > > answer a few of these things. > > > >> List of concerns with ironic: > >> > >> 1.)Nova<-> ironic interactions are generally seem terrible? > > > > I don't know if I'd call it terrible, but there's friction. Things that > > are unchangable on hardware are just software configs in vms (like mac > > addresses, overlays, etc), and things that make no sense in VMs are > > pretty standard on servers (trunked vlans, bonding, etc). > > > > One way we've gotten around it is by using Ironic standalone via > > Bifrost[1]. This deploys Ironic in wide open auth mode on 127.0.0.1, > > and includes playbooks to build config drives and deploy images in a > > fairly rudimentary way without Nova. > > > > I call this the "better than Cobbler" way of getting a toe into the > > Ironic waters. > > > > [1] https://github.com/openstack/bifrost > > Out of curiosity, why ansible vs turning > https://github.com/openstack/nova/blob/master/nova/virt/ironic/driver.py > (or something like it) into a tiny-wsgi-app (pick useful name here) that > has its own REST api (that looks pretty similar to the public functions > in that driver file)? That's an interesting idea. I think a reason Bifrost doesn't just import nova virt drivers is that they're likely _not_ a supported public API (despite not having _'s at the front). Also, a lot of the reason Bifrost exists is to enable users to get the benefits of all the baremetal abstraction work done in Ironic without having to fully embrace all of OpenStack's core. So while you could get a little bit of the stuff from nova (like config drive building), you'd still need to handle network address assignment, image management, etc. etc., and pretty soon you start having to run a tiny glance and a tiny neutron. The Bifrost way is the opposite: I just want a tiny Ironic, and _nothing_ else. > > That seems almost easier than building a bunch of ansible scripts that > appear (at a glance) to do similar things; and u get the benefit of > using a actual programming language vs a > half-programming-ansible-yaml-language... > > A realization I'm having is that I'm really not a fan of using ansible > as a half-programming-ansible-yaml-language, which it seems like people > start to try to do after a while (because at some point you need > something like if statements, then things like [1] get created), no > offense to the authors, but I guess this is my personal preference (it's > also one of the reasons taskflow directly is a lib. in python, because > , people don't need to learn a new language). > We use python in Ansible all the time: http://git.openstack.org/cgit/openstack/bifrost/tree/playbooks/library The reason to use Ansible is that it has already implemented all of the idempotency and error handling and UI needs that one might need for running workflows. I've tried multiple times to understand taskflow, and to me, Ansible is the anti-taskflow. It's easy to pick up, easy to read the workflows, doesn't require deep surgery on your code to use (just execute ansible-playbook), and is full of modules to support nearly anything your deployment may need. __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [ironic] using ironic as a replacement for existing datacenter baremetal provisioning
Clint Byrum wrote: Excerpts from Joshua Harlow's message of 2016-06-07 08:46:28 -0700: Clint Byrum wrote: Excerpts from Kris G. Lindgren's message of 2016-06-06 20:44:26 +: Hi ironic folks, As I'm trying to explore how GoDaddy can use ironic I've created the following in an attempt to document some of my concerns, and I'm wondering if you folks could help myself identity ongoing work to solve these (or alternatives?) Hi Kris. I've been using Ironic in various forms for a while, and I can answer a few of these things. List of concerns with ironic: 1.)Nova<-> ironic interactions are generally seem terrible? I don't know if I'd call it terrible, but there's friction. Things that are unchangable on hardware are just software configs in vms (like mac addresses, overlays, etc), and things that make no sense in VMs are pretty standard on servers (trunked vlans, bonding, etc). One way we've gotten around it is by using Ironic standalone via Bifrost[1]. This deploys Ironic in wide open auth mode on 127.0.0.1, and includes playbooks to build config drives and deploy images in a fairly rudimentary way without Nova. I call this the "better than Cobbler" way of getting a toe into the Ironic waters. [1] https://github.com/openstack/bifrost Out of curiosity, why ansible vs turning https://github.com/openstack/nova/blob/master/nova/virt/ironic/driver.py (or something like it) into a tiny-wsgi-app (pick useful name here) that has its own REST api (that looks pretty similar to the public functions in that driver file)? That's an interesting idea. I think a reason Bifrost doesn't just import nova virt drivers is that they're likely _not_ a supported public API (despite not having _'s at the front). Also, a lot of the reason Bifrost exists is to enable users to get the benefits of all the baremetal abstraction work done in Ironic without having to fully embrace all of OpenStack's core. So while you could get a little bit of the stuff from nova (like config drive building), you'd still need to handle network address assignment, image management, etc. etc., and pretty soon you start having to run a tiny glance and a tiny neutron. The Bifrost way is the opposite: I just want a tiny Ironic, and _nothing_ else. Ya, I'm just thinking that at a certain point That seems almost easier than building a bunch of ansible scripts that appear (at a glance) to do similar things; and u get the benefit of using a actual programming language vs a half-programming-ansible-yaml-language... A realization I'm having is that I'm really not a fan of using ansible as a half-programming-ansible-yaml-language, which it seems like people start to try to do after a while (because at some point you need something like if statements, then things like [1] get created), no offense to the authors, but I guess this is my personal preference (it's also one of the reasons taskflow directly is a lib. in python, because , people don't need to learn a new language). We use python in Ansible all the time: http://git.openstack.org/cgit/openstack/bifrost/tree/playbooks/library The reason to use Ansible is that it has already implemented all of the idempotency and error handling and UI needs that one might need for running workflows. I've tried multiple times to understand taskflow, and to me, Ansible is the anti-taskflow. It's easy to pick up, easy to read the workflows, doesn't require deep surgery on your code to use (just execute ansible-playbook), and is full of modules to support nearly anything your deployment may need. Actually they are pretty similar (to a degree), taskflow is pretty much the same/similar thing ansible is using internally, a graph structure (last time I checked) that gets ran in parallel or in serial using a executor concept[1]. Said surgery is only required if you want a deep integration, nothing is stopping folks from using taskflow in the same manner as running a bunch of task == similar to ansible style (taskflow also doesn't need to have its own module concepts as pypi modules primarily just work because it's python). But ya, anyway... can't win over everyone ;) [1] https://github.com/ansible/ansible/tree/devel/lib/ansible/executor __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [ironic] using ironic as a replacement for existing datacenter baremetal provisioning
Joshua Harlow wrote: Clint Byrum wrote: Excerpts from Joshua Harlow's message of 2016-06-07 08:46:28 -0700: Clint Byrum wrote: Excerpts from Kris G. Lindgren's message of 2016-06-06 20:44:26 +: Hi ironic folks, As I'm trying to explore how GoDaddy can use ironic I've created the following in an attempt to document some of my concerns, and I'm wondering if you folks could help myself identity ongoing work to solve these (or alternatives?) Hi Kris. I've been using Ironic in various forms for a while, and I can answer a few of these things. List of concerns with ironic: 1.)Nova<-> ironic interactions are generally seem terrible? I don't know if I'd call it terrible, but there's friction. Things that are unchangable on hardware are just software configs in vms (like mac addresses, overlays, etc), and things that make no sense in VMs are pretty standard on servers (trunked vlans, bonding, etc). One way we've gotten around it is by using Ironic standalone via Bifrost[1]. This deploys Ironic in wide open auth mode on 127.0.0.1, and includes playbooks to build config drives and deploy images in a fairly rudimentary way without Nova. I call this the "better than Cobbler" way of getting a toe into the Ironic waters. [1] https://github.com/openstack/bifrost Out of curiosity, why ansible vs turning https://github.com/openstack/nova/blob/master/nova/virt/ironic/driver.py (or something like it) into a tiny-wsgi-app (pick useful name here) that has its own REST api (that looks pretty similar to the public functions in that driver file)? That's an interesting idea. I think a reason Bifrost doesn't just import nova virt drivers is that they're likely _not_ a supported public API (despite not having _'s at the front). Also, a lot of the reason Bifrost exists is to enable users to get the benefits of all the baremetal abstraction work done in Ironic without having to fully embrace all of OpenStack's core. So while you could get a little bit of the stuff from nova (like config drive building), you'd still need to handle network address assignment, image management, etc. etc., and pretty soon you start having to run a tiny glance and a tiny neutron. The Bifrost way is the opposite: I just want a tiny Ironic, and _nothing_ else. Ya, I'm just thinking that at a certain point Oops forgot to fill this out, was just thinking that at a certain point it might be easier to figure out how to extract that API (meh, if its public or private) and just have someone make an executive decision around ironic being a stand-alone thing or not (and a capable stand-alone thing, not a sorta-standalone-thing). That seems almost easier than building a bunch of ansible scripts that appear (at a glance) to do similar things; and u get the benefit of using a actual programming language vs a half-programming-ansible-yaml-language... A realization I'm having is that I'm really not a fan of using ansible as a half-programming-ansible-yaml-language, which it seems like people start to try to do after a while (because at some point you need something like if statements, then things like [1] get created), no offense to the authors, but I guess this is my personal preference (it's also one of the reasons taskflow directly is a lib. in python, because , people don't need to learn a new language). We use python in Ansible all the time: http://git.openstack.org/cgit/openstack/bifrost/tree/playbooks/library The reason to use Ansible is that it has already implemented all of the idempotency and error handling and UI needs that one might need for running workflows. I've tried multiple times to understand taskflow, and to me, Ansible is the anti-taskflow. It's easy to pick up, easy to read the workflows, doesn't require deep surgery on your code to use (just execute ansible-playbook), and is full of modules to support nearly anything your deployment may need. Actually they are pretty similar (to a degree), taskflow is pretty much the same/similar thing ansible is using internally, a graph structure (last time I checked) that gets ran in parallel or in serial using a executor concept[1]. Said surgery is only required if you want a deep integration, nothing is stopping folks from using taskflow in the same manner as running a bunch of task == similar to ansible style (taskflow also doesn't need to have its own module concepts as pypi modules primarily just work because it's python). But ya, anyway... can't win over everyone ;) [1] https://github.com/ansible/ansible/tree/devel/lib/ansible/executor __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: o
Re: [openstack-dev] [ironic] using ironic as a replacement for existing datacenter baremetal provisioning
On 06/07/2016 09:55 AM, Joshua Harlow wrote: > Joshua Harlow wrote: >> Clint Byrum wrote: >>> Excerpts from Joshua Harlow's message of 2016-06-07 08:46:28 -0700: Clint Byrum wrote: > Excerpts from Kris G. Lindgren's message of 2016-06-06 20:44:26 +: >> Hi ironic folks, >> As I'm trying to explore how GoDaddy can use ironic I've created >> the following in an attempt to document some of my concerns, and >> I'm wondering if you folks could help myself identity ongoing work >> to solve these (or alternatives?) > Hi Kris. I've been using Ironic in various forms for a while, and I can > answer a few of these things. > >> List of concerns with ironic: >> >> 1.)Nova<-> ironic interactions are generally seem terrible? > I don't know if I'd call it terrible, but there's friction. Things that > are unchangable on hardware are just software configs in vms (like mac > addresses, overlays, etc), and things that make no sense in VMs are > pretty standard on servers (trunked vlans, bonding, etc). > > One way we've gotten around it is by using Ironic standalone via > Bifrost[1]. This deploys Ironic in wide open auth mode on 127.0.0.1, > and includes playbooks to build config drives and deploy images in a > fairly rudimentary way without Nova. > > I call this the "better than Cobbler" way of getting a toe into the > Ironic waters. > > [1] https://github.com/openstack/bifrost Out of curiosity, why ansible vs turning https://github.com/openstack/nova/blob/master/nova/virt/ironic/driver.py (or something like it) into a tiny-wsgi-app (pick useful name here) that has its own REST api (that looks pretty similar to the public functions in that driver file)? >>> >>> That's an interesting idea. I think a reason Bifrost doesn't just import >>> nova virt drivers is that they're likely _not_ a supported public API >>> (despite not having _'s at the front). Also, a lot of the reason Bifrost >>> exists is to enable users to get the benefits of all the baremetal >>> abstraction work done in Ironic without having to fully embrace all of >>> OpenStack's core. So while you could get a little bit of the stuff from >>> nova (like config drive building), you'd still need to handle network >>> address assignment, image management, etc. etc., and pretty soon you >>> start having to run a tiny glance and a tiny neutron. The Bifrost way >>> is the opposite: I just want a tiny Ironic, and _nothing_ else. >>> >> >> Ya, I'm just thinking that at a certain point > You've got two statements in here, which I'm going to reply to separately: > Oops forgot to fill this out, was just thinking that at a certain point it > might > be easier to figure out how to extract that API (meh, if its public or > private) The nova-core team has repeatedly stated that they do not have plans to support the nova virt driver API as a stable or externally-consumable python API. Changing that would affect a lot more than Ironic (eg. defcore). A change like that is not just about what is easier for developers, but also what is better for the community. > and just have someone make an executive decision around ironic being a > stand-alone thing or not (and a capable stand-alone thing, not a > sorta-standalone-thing). We already decided to support Ironic as a stand-alone service. So, could you clarify what you mean when you call it a "sorta-standalone-thing"? In what ways do you think it's *not* functional? Do you have specific recommendations on what we can improve, based on experience using either Ironic or Bifrost? -Devananda __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [ironic] using ironic as a replacement for existing datacenter baremetal provisioning
Devananda van der Veen wrote: On 06/07/2016 09:55 AM, Joshua Harlow wrote: Joshua Harlow wrote: Clint Byrum wrote: Excerpts from Joshua Harlow's message of 2016-06-07 08:46:28 -0700: Clint Byrum wrote: Excerpts from Kris G. Lindgren's message of 2016-06-06 20:44:26 +: Hi ironic folks, As I'm trying to explore how GoDaddy can use ironic I've created the following in an attempt to document some of my concerns, and I'm wondering if you folks could help myself identity ongoing work to solve these (or alternatives?) Hi Kris. I've been using Ironic in various forms for a while, and I can answer a few of these things. List of concerns with ironic: 1.)Nova<-> ironic interactions are generally seem terrible? I don't know if I'd call it terrible, but there's friction. Things that are unchangable on hardware are just software configs in vms (like mac addresses, overlays, etc), and things that make no sense in VMs are pretty standard on servers (trunked vlans, bonding, etc). One way we've gotten around it is by using Ironic standalone via Bifrost[1]. This deploys Ironic in wide open auth mode on 127.0.0.1, and includes playbooks to build config drives and deploy images in a fairly rudimentary way without Nova. I call this the "better than Cobbler" way of getting a toe into the Ironic waters. [1] https://github.com/openstack/bifrost Out of curiosity, why ansible vs turning https://github.com/openstack/nova/blob/master/nova/virt/ironic/driver.py (or something like it) into a tiny-wsgi-app (pick useful name here) that has its own REST api (that looks pretty similar to the public functions in that driver file)? That's an interesting idea. I think a reason Bifrost doesn't just import nova virt drivers is that they're likely _not_ a supported public API (despite not having _'s at the front). Also, a lot of the reason Bifrost exists is to enable users to get the benefits of all the baremetal abstraction work done in Ironic without having to fully embrace all of OpenStack's core. So while you could get a little bit of the stuff from nova (like config drive building), you'd still need to handle network address assignment, image management, etc. etc., and pretty soon you start having to run a tiny glance and a tiny neutron. The Bifrost way is the opposite: I just want a tiny Ironic, and _nothing_ else. Ya, I'm just thinking that at a certain point You've got two statements in here, which I'm going to reply to separately: Oops forgot to fill this out, was just thinking that at a certain point it might be easier to figure out how to extract that API (meh, if its public or private) The nova-core team has repeatedly stated that they do not have plans to support the nova virt driver API as a stable or externally-consumable python API. Changing that would affect a lot more than Ironic (eg. defcore). A change like that is not just about what is easier for developers, but also what is better for the community. Right, I'm starting to come to the belief that what is better for the community is to change this; because from what I can tell (from my view of the world) that tying all the things to nova has really been detrimental (to a degree) to the whole progression of the cloud as a whole. It's an opinionated thing to say, yes I understand that, but there becomes a point where I feel we need to re-evaluate what people really care about from openstack (because I start to believe that treating the whole thing as a single product, well that went out the window a long time ago, with the creation of the big-tent by the TC, with the creation of mesos, k8s and others by other companies not in openstack...); and really what's left after that is a bunch of services that to survive (as a useful set of services) must accept that there is more than just openstack in the wider world (ie, kubernetes, mesos, the-next-best-thing...) and if we don't start embracing those other communities (and no that doesn't mean be an `integration engine` on-top or around them) then we are pretty much obsoleting ourselves. Ya, I know this is a hot (and touchy) topic, and probably other people don't agree, that's ok... I don't mind the flames, ha ( spicy). and just have someone make an executive decision around ironic being a stand-alone thing or not (and a capable stand-alone thing, not a sorta-standalone-thing). We already decided to support Ironic as a stand-alone service. So, could you clarify what you mean when you call it a "sorta-standalone-thing"? In what ways do you think it's *not* functional? Do you have specific recommendations on what we can improve, based on experience using either Ironic or Bifrost? I'll work on this list, as some folks that are trying start to try to connect ironic (IMHO without nova, because well kubernetes is enough like nova that there isn't a need for 2-layers of nova-like-systems at that point) into kubernetes as a 'resource provider'. I'm sure attempting to do that
Re: [openstack-dev] [ironic] using ironic as a replacement for existing datacenter baremetal provisioning
On Tue, Jun 7, 2016, at 02:34 PM, Joshua Harlow wrote: > Devananda van der Veen wrote: > > On 06/07/2016 09:55 AM, Joshua Harlow wrote: > >> Joshua Harlow wrote: > >>> Clint Byrum wrote: > Excerpts from Joshua Harlow's message of 2016-06-07 08:46:28 -0700: > > Clint Byrum wrote: > >> Excerpts from Kris G. Lindgren's message of 2016-06-06 20:44:26 +: > >>> Hi ironic folks, > >>> As I'm trying to explore how GoDaddy can use ironic I've created > >>> the following in an attempt to document some of my concerns, and > >>> I'm wondering if you folks could help myself identity ongoing work > >>> to solve these (or alternatives?) > >> Hi Kris. I've been using Ironic in various forms for a while, and I can > >> answer a few of these things. > >> > >>> List of concerns with ironic: > >>> > >>> 1.)Nova<-> ironic interactions are generally seem terrible? > >> I don't know if I'd call it terrible, but there's friction. Things that > >> are unchangable on hardware are just software configs in vms (like mac > >> addresses, overlays, etc), and things that make no sense in VMs are > >> pretty standard on servers (trunked vlans, bonding, etc). > >> > >> One way we've gotten around it is by using Ironic standalone via > >> Bifrost[1]. This deploys Ironic in wide open auth mode on 127.0.0.1, > >> and includes playbooks to build config drives and deploy images in a > >> fairly rudimentary way without Nova. > >> > >> I call this the "better than Cobbler" way of getting a toe into the > >> Ironic waters. > >> > >> [1] https://github.com/openstack/bifrost > > Out of curiosity, why ansible vs turning > > https://github.com/openstack/nova/blob/master/nova/virt/ironic/driver.py > > (or something like it) into a tiny-wsgi-app (pick useful name here) that > > has its own REST api (that looks pretty similar to the public functions > > in that driver file)? > That's an interesting idea. I think a reason Bifrost doesn't just import > nova virt drivers is that they're likely _not_ a supported public API > (despite not having _'s at the front). Also, a lot of the reason Bifrost > exists is to enable users to get the benefits of all the baremetal > abstraction work done in Ironic without having to fully embrace all of > OpenStack's core. So while you could get a little bit of the stuff from > nova (like config drive building), you'd still need to handle network > address assignment, image management, etc. etc., and pretty soon you > start having to run a tiny glance and a tiny neutron. The Bifrost way > is the opposite: I just want a tiny Ironic, and _nothing_ else. > > >>> Ya, I'm just thinking that at a certain point > > > > You've got two statements in here, which I'm going to reply to separately: > > > >> Oops forgot to fill this out, was just thinking that at a certain point it > >> might > >> be easier to figure out how to extract that API (meh, if its public or > >> private) > > > > The nova-core team has repeatedly stated that they do not have plans to > > support > > the nova virt driver API as a stable or externally-consumable python API. > > Changing that would affect a lot more than Ironic (eg. defcore). A change > > like > > that is not just about what is easier for developers, but also what is > > better > > for the community. > > > > Right, I'm starting to come to the belief that what is better for the > community is to change this; because from what I can tell (from my view > of the world) that tying all the things to nova has really been > detrimental (to a degree) to the whole progression of the cloud as a > whole. When I have heard the statement made about the virt driver API it has simply been that it is not stable and there are no plans at this point to make it stable. My own opinion is that it is sometimes painful to use from within Nova and I do not wish to expose that pain to others. There are changes I would like to see made before it could be considered externally consumable. > > It's an opinionated thing to say, yes I understand that, but there > becomes a point where I feel we need to re-evaluate what people really > care about from openstack (because I start to believe that treating the > whole thing as a single product, well that went out the window a long > time ago, with the creation of the big-tent by the TC, with the creation > of mesos, k8s and others by other companies not in openstack...); and > really what's left after that is a bunch of services that to survive (as > a useful set of services) must accept that there is more than just > openstack in the wider world (ie, kubernetes, mesos, > the-next-best-thing...) and if we don't start embracing those other > communities (and no that doesn't mean be an `integration engine` on-top > or around them) then we are pretty much obsoleting ourselv
Re: [openstack-dev] [ironic] using ironic as a replacement for existing datacenter baremetal provisioning
Replying to a digest so sorry for the copy and pastes…. >> There's also been discussion of ways we could do ad-hoc changes in RAID >> level, >> based on flavor metadata, during the provisioning process (rather than ahead >> of >> time) but no code has been done for this yet, AFAIK. > > I'm still pretty interested in it, because I agree with anything said > above about building RAID ahead-of-time not being convenient. I don't > quite understand how such a feature would look like, we might add it as > a topic for midcycle. This sounds like an interesting/acceptable way to handle this problem to me. Update the node to set the desired raid state from the flavor. >> - Inspection is geared towards using a different network and dnsmasq >> infrastructure than what is in use for ironic/neutron. Which also means >> that in >> order to not conflict with dhcp requests for servers in ironic I need to use >> different networks. Which also means I now need to handle swinging server >> ports >> between different networks. > > Inspector is designed to respond only to requests for nodes in the inspection > phase, so that it *doesn't* conflict with provisioning of nodes by Ironic. > I've > been using the same network for inspection and provisioning without issue -- > so > I'm not sure what problem you're encountering here. So I was mainly thinking about the use case of using inspector to onboard unknown hosts into ironic (though I see didn't mention that). So in a datacenter we are always on boarding servers. Right now we boot a linux agent that "inventories" the box and adds it to our management system as a node to be able to be consumed by a build request. My understanding is that inspector supports this as of Mitaka. However, the install guide for inspection states that you need to install its own dnsmasq instance for inspection. To me this implies that this is suppose to be a separate network. As if I have 2 dhcp servers running on the same L2 network I am going to get races between the 2 dhcp servers for normal provisioning activities. Especially if one dhcp server is configured to respond to everything (so that it can onboard unknown hardware) and the other only to specific hosts(ironic/neutron). The only way that wouldn't be an issue is if both inspector and ironic/neutron are using the same dhcp servers. Or am I missing something? ___ Kris Lindgren Senior Linux Systems Engineer GoDaddy __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [ironic] using ironic as a replacement for existing datacenter baremetal provisioning
On 07/06/2016 23:59, "Kris G. Lindgren" mailto:klindg...@godaddy.com>> wrote: Replying to a digest so sorry for the copy and pastes >> There's also been discussion of ways we could do ad-hoc changes in RAID >> level, >> based on flavor metadata, during the provisioning process (rather than ahead >> of >> time) but no code has been done for this yet, AFAIK. > > I'm still pretty interested in it, because I agree with anything said > above about building RAID ahead-of-time not being convenient. I don't > quite understand how such a feature would look like, we might add it as > a topic for midcycle. This sounds like an interesting/acceptable way to handle this problem to me. Update the node to set the desired raid state from the flavor. >> - Inspection is geared towards using a different network and dnsmasq >> infrastructure than what is in use for ironic/neutron. Which also means >> that in >> order to not conflict with dhcp requests for servers in ironic I need to use >> different networks. Which also means I now need to handle swinging server >> ports >> between different networks. >> Inspector is designed to respond only to requests for nodes in the inspection > phase, so that it *doesn't* conflict with provisioning of nodes by Ironic. > I've > been using the same network for inspection and provisioning without issue -- > so > I'm not sure what problem you're encountering here. So I was mainly thinking about the use case of using inspector to onboard unknown hosts into ironic (though I see didn't mention that). So in a datacenter we are always on boarding servers. Right now we boot a linux agent that "inventories" the box and adds it to our management system as a node to be able to be consumed by a build request. My understanding is that inspector supports this as of Mitaka. However, the install guide for inspection states that you need to install its own dnsmasq instance for inspection. To me this implies that this is suppose to be a separate network. As if I have 2 dhcp servers running on the same L2 network I am going to get races between the 2 dhcp servers for normal provisioning activities. Especially if one dhcp server is configured to respond to everything (so that it can onboard unknown hardware) and the other only to specific hosts(ironic/neutron). The only way that wouldn't be an issue is if both inspector and ironic/neutron are using the same dhcp servers. Or am I missing something? Ironic inspector handles this by managing Iptables as a firewall service to white/black list nodes, only allowing nodes that should be talking to the Inspector's dnsmasq instance to be served by it. This avoids any DHCP races between Ironic and Inspector. ___ Kris Lindgren Senior Linux Systems Engineer GoDaddy __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [ironic] using ironic as a replacement for existing datacenter baremetal provisioning
On Wed, Jun 08, 2016 at 10:39:20AM +, Sam Betts (sambetts) wrote: > > > On 07/06/2016 23:59, "Kris G. Lindgren" > mailto:klindg...@godaddy.com>> wrote: > > Replying to a digest so sorry for the copy and pastes > > > >> There's also been discussion of ways we could do ad-hoc changes in RAID > >> level, > >> based on flavor metadata, during the provisioning process (rather than > >> ahead of > >> time) but no code has been done for this yet, AFAIK. > > > > I'm still pretty interested in it, because I agree with anything said > > above about building RAID ahead-of-time not being convenient. I don't > > quite understand how such a feature would look like, we might add it as > > a topic for midcycle. > > This sounds like an interesting/acceptable way to handle this problem to me. > Update the node to set the desired raid state from the flavor. > > >> - Inspection is geared towards using a different network and dnsmasq > > >> infrastructure than what is in use for ironic/neutron. Which also means > >> that in > >> order to not conflict with dhcp requests for servers in ironic I need to > >> use > >> different networks. Which also means I now need to handle swinging server > >> ports > >> between different networks. > >> Inspector is designed to respond only to requests for nodes in the > >> inspection > > phase, so that it *doesn't* conflict with provisioning of nodes by Ironic. > > I've > > been using the same network for inspection and provisioning without issue > > -- so > > I'm not sure what problem you're encountering here. > > So I was mainly thinking about the use case of using inspector to onboard > unknown hosts into ironic (though I see didn't mention that). So in a > datacenter we are always on boarding servers. Right now we boot a linux > agent that "inventories" the box and adds it to our management system as a > node to be able to be consumed by a build request. My understanding is that > inspector supports this as of Mitaka. However, the install guide for > inspection states that you need to install its own dnsmasq instance for > inspection. To me this implies that this is suppose to be a separate > network. As if I have 2 dhcp servers running on the same L2 network I am > going to get races between the 2 dhcp servers for normal provisioning > activities. Especially if one dhcp server is configured to respond to > everything (so that it can onboard unknown hardware) and the other only to > specific hosts(ironic/neutron). The only way that wouldn't be an issue is if > both inspector and ironic/neutron are using th e same dhcp servers. Or am I missing something? > > Ironic inspector handles this by managing Iptables as a firewall service to > white/black list nodes, only allowing nodes that should be talking to the > Inspector's dnsmasq instance to be served by it. This avoids any DHCP races > between Ironic and Inspector. To add to this, the primary reason this is done is because Neutron doesn't allow wildcard DHCP (there's a spec up for this, but it isn't moving quickly). // jim > > > ___ > Kris Lindgren > Senior Linux Systems Engineer > GoDaddy > __ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [ironic] using ironic as a replacement for existing datacenter baremetal provisioning
> >1.)Nova<-> ironic interactions are generally seem terrible? > I don't know if I'd call it terrible, but there's friction. Things that > are unchangable on hardware are just software configs in vms (like mac > addresses, overlays, etc), and things that make no sense in VMs are > pretty standard on servers (trunked vlans, bonding, etc). > > One way we've gotten around it is by using Ironic standalone via > Bifrost[1]. This deploys Ironic in wide open auth mode on 127.0.0.1, > and includes playbooks to build config drives and deploy images in a > fairly rudimentary way without Nova. > > I call this the "better than Cobbler" way of getting a toe into the > Ironic waters. > > [1] https://github.com/openstack/bifrost > >>>Out of curiosity, why ansible vs turning > >>>https://github.com/openstack/nova/blob/master/nova/virt/ironic/driver.py > >>>(or something like it) into a tiny-wsgi-app (pick useful name here) that > >>>has its own REST api (that looks pretty similar to the public functions > >>>in that driver file)? > >> > >>That's an interesting idea. I think a reason Bifrost doesn't just import > >>nova virt drivers is that they're likely _not_ a supported public API > >>(despite not having _'s at the front). Also, a lot of the reason Bifrost > >>exists is to enable users to get the benefits of all the baremetal > >>abstraction work done in Ironic without having to fully embrace all of > >>OpenStack's core. So while you could get a little bit of the stuff from > >>nova (like config drive building), you'd still need to handle network > >>address assignment, image management, etc. etc., and pretty soon you > >>start having to run a tiny glance and a tiny neutron. The Bifrost way > >>is the opposite: I just want a tiny Ironic, and _nothing_ else. > >> > > > >Ya, I'm just thinking that at a certain point > > Oops forgot to fill this out, was just thinking that at a certain point it > might be easier to figure out how to extract that API (meh, if its public or > private) and just have someone make an executive decision around ironic > being a stand-alone thing or not (and a capable stand-alone thing, not a > sorta-standalone-thing). So, I've been thinking about this quite a bit. We've also talked about doing a v2 API (as evil as that may be) in Ironic here and there. We've had lots of lessons learned from the v1 API, mostly that our API is absolutely terrible for humans. I'd love to fix that (whether that requires a v2 API or not is unclear, so don't focus on that). I've noticed that people keep talking about the Nova driver API not being public/stable/whatever in this thread - let's ignore that and think bigger. So, there's two large use cases for ironic that we support today: * Ironic as a backend to nova. Operators still need to interact with the Ironic API for management, troubleshooting, and fixing issues that computers do not handle today. * Ironic standalone - by this I mean ironic without nova. The primary deployment method here is using Bifrost, and I also call it the "better than cobbler" case. I'm not sure if people are using this without bifrost, or with other non-nova services, today. Users in this model, as I understand things, do not interact with the Ironic API directly (except maybe for troubleshooting). There's other use cases I would like to support: * Ironic standalone, without Bifrost. I would love for a deployer to be able to stand up Ironic as an end-user facing API, probably with Keystone, maybe with Neutron/Glance/Swift if needed. This would require a ton of discussion and work (e.g. ironic has no concept of tenants/projects today, we might want a scheduler, a concept of an instance, etc) and would be a very long road. The ideal solution to this is to break out the Compute API and scheduler to be separate from Nova, but that's an even longer road, so let's pretend I didn't say that and not devolve this thread into that conversation (yet). * Ironic as a backend to other things. Josh pointed out kubernetes somewhere, I'd love to be an official backend there. Heat today goes through Nova to get an ironic instance, it seems reasonable to have heat talk directly to ironic. Things like that. The amount of work here might depend on the application using ironic (e.g. I think k8s has it's own scheduler, heat does not, right?). So all that said, I think there is one big step we can take in the short-term that works for all of these use cases: make our API better. Make it simpler. Take a bunch of the logic in the Nova driver, and put it in our API instead. spawn() becomes /v1/nodes/foo/deploy or something, etc (I won't let us bikeshed those specifics in this thread). Just doing that allows us to remove a bunch of code from a number of places (nova, bifrost, shade, tempest(?)) and make those simpler. It allows direct API users to more easily deploy things, making one API call instead of
Re: [openstack-dev] [ironic] using ironic as a replacement for existing datacenter baremetal provisioning
On Thu, Jun 09, 2016 at 06:17:56PM +0100, Lucas Alvares Gomes wrote: > Hi, > > Thanks for writing it down Jim. > > > So, I've been thinking about this quite a bit. We've also talked about > > doing a v2 API (as evil as that may be) in Ironic here and there. We've > > had lots of lessons learned from the v1 API, mostly that our API is > > absolutely terrible for humans. I'd love to fix that (whether that > > requires a v2 API or not is unclear, so don't focus on that). > > > > I've noticed that people keep talking about the Nova driver API > > not being public/stable/whatever in this thread - let's ignore that and > > think bigger. > > > > So, there's two large use cases for ironic that we support today: > > > > * Ironic as a backend to nova. Operators still need to interact with the > > Ironic API for management, troubleshooting, and fixing issues that > > computers do not handle today. > > > > * Ironic standalone - by this I mean ironic without nova. The primary > > deployment method here is using Bifrost, and I also call it the > > "better than cobbler" case. I'm not sure if people are using this > > without bifrost, or with other non-nova services, today. Users in this > > model, as I understand things, do not interact with the Ironic API > > directly (except maybe for troubleshooting). > > > > There's other use cases I would like to support: > > > > * Ironic standalone, without Bifrost. I would love for a deployer to be > > able to stand up Ironic as an end-user facing API, probably with > > Keystone, maybe with Neutron/Glance/Swift if needed. This would > > require a ton of discussion and work (e.g. ironic has no concept of > > tenants/projects today, we might want a scheduler, a concept of an > > instance, etc) and would be a very long road. The ideal solution to > > this is to break out the Compute API and scheduler to be separate from > > Nova, but that's an even longer road, so let's pretend I didn't say > > that and not devolve this thread into that conversation (yet). > > > > * Ironic as a backend to other things. Josh pointed out kubernetes > > somewhere, I'd love to be an official backend there. Heat today goes > > through Nova to get an ironic instance, it seems reasonable to have > > heat talk directly to ironic. Things like that. The amount of work > > here might depend on the application using ironic (e.g. I think k8s > > has it's own scheduler, heat does not, right?). > > > > So all that said, I think there is one big step we can take in the > > short-term that works for all of these use cases: make our API better. > > Make it simpler. Take a bunch of the logic in the Nova driver, and put > > it in our API instead. spawn() becomes /v1/nodes/foo/deploy or > > something, etc (I won't let us bikeshed those specifics in this thread). > > Just doing that allows us to remove a bunch of code from a number of > > places (nova, bifrost, shade, tempest(?)) and make those simpler. It > > allows direct API users to more easily deploy things, making one API > > call instead of a bunch (we could even create Neutron ports and such for > > them). It allows k8s and friends to write less code. Oh, let's also stop > > directly exposing state machine transitions as API actions, that's > > crazy, kthx. > > > > I think this is what Josh is trying to get at, except maybe with a > > separate API service in between, which doesn't sound very desirable to > > me. > > > > Thoughts on this? > > > > Additionally, in the somewhat-short term, I'd like us to try to > > enumerate the major use cases we're trying to solve, and make those use > > cases ridiculously simple to deploy. Ironic is quickly becoming a > > tangled mess of configuration options and tweaking surrounding services > > (nova, neutron) to deploy it. Once it's figured out, it works very well. > > However, it's incredibly difficult to figure out how to get there. > > > > Ultimately, I'd like someone that wants to deploy ironic in a common use > > case, with off-the-shelf hardware, to be able to get a POC up and > > running in a matter of hours, not days or weeks. > > > > Who's in? :) > > > > I agree in general with the idea but I think it needs a tad more > context. We need to remember that Ironic (ex-Nova Baremetal) was > created to fill a gap in OpenStack that was missing for TripleO > project to get off the ground. That was the problem being solved and > these aspects are reflected in the ReST API: Being admin-only, not > "human-friendly" (standalone came later), etc... Sorry, I didn't mean to slag on people here. In fact, I tried to come up with a way to say "no offense" but couldn't figure the words out. Ironic did start with a very specific use case, it's come a super long way, and you all did what you had to do to get things going. For that I'm forever indebted to you. :) > > * Ironic as a backend to other things. Josh pointed out kubernetes > > somewhere, I'd love to be an official backend there. Heat
Re: [openstack-dev] [ironic] using ironic as a replacement for existing datacenter baremetal provisioning
Hi, Thanks for writing it down Jim. > So, I've been thinking about this quite a bit. We've also talked about > doing a v2 API (as evil as that may be) in Ironic here and there. We've > had lots of lessons learned from the v1 API, mostly that our API is > absolutely terrible for humans. I'd love to fix that (whether that > requires a v2 API or not is unclear, so don't focus on that). > > I've noticed that people keep talking about the Nova driver API > not being public/stable/whatever in this thread - let's ignore that and > think bigger. > > So, there's two large use cases for ironic that we support today: > > * Ironic as a backend to nova. Operators still need to interact with the > Ironic API for management, troubleshooting, and fixing issues that > computers do not handle today. > > * Ironic standalone - by this I mean ironic without nova. The primary > deployment method here is using Bifrost, and I also call it the > "better than cobbler" case. I'm not sure if people are using this > without bifrost, or with other non-nova services, today. Users in this > model, as I understand things, do not interact with the Ironic API > directly (except maybe for troubleshooting). > > There's other use cases I would like to support: > > * Ironic standalone, without Bifrost. I would love for a deployer to be > able to stand up Ironic as an end-user facing API, probably with > Keystone, maybe with Neutron/Glance/Swift if needed. This would > require a ton of discussion and work (e.g. ironic has no concept of > tenants/projects today, we might want a scheduler, a concept of an > instance, etc) and would be a very long road. The ideal solution to > this is to break out the Compute API and scheduler to be separate from > Nova, but that's an even longer road, so let's pretend I didn't say > that and not devolve this thread into that conversation (yet). > > * Ironic as a backend to other things. Josh pointed out kubernetes > somewhere, I'd love to be an official backend there. Heat today goes > through Nova to get an ironic instance, it seems reasonable to have > heat talk directly to ironic. Things like that. The amount of work > here might depend on the application using ironic (e.g. I think k8s > has it's own scheduler, heat does not, right?). > > So all that said, I think there is one big step we can take in the > short-term that works for all of these use cases: make our API better. > Make it simpler. Take a bunch of the logic in the Nova driver, and put > it in our API instead. spawn() becomes /v1/nodes/foo/deploy or > something, etc (I won't let us bikeshed those specifics in this thread). > Just doing that allows us to remove a bunch of code from a number of > places (nova, bifrost, shade, tempest(?)) and make those simpler. It > allows direct API users to more easily deploy things, making one API > call instead of a bunch (we could even create Neutron ports and such for > them). It allows k8s and friends to write less code. Oh, let's also stop > directly exposing state machine transitions as API actions, that's > crazy, kthx. > > I think this is what Josh is trying to get at, except maybe with a > separate API service in between, which doesn't sound very desirable to > me. > > Thoughts on this? > > Additionally, in the somewhat-short term, I'd like us to try to > enumerate the major use cases we're trying to solve, and make those use > cases ridiculously simple to deploy. Ironic is quickly becoming a > tangled mess of configuration options and tweaking surrounding services > (nova, neutron) to deploy it. Once it's figured out, it works very well. > However, it's incredibly difficult to figure out how to get there. > > Ultimately, I'd like someone that wants to deploy ironic in a common use > case, with off-the-shelf hardware, to be able to get a POC up and > running in a matter of hours, not days or weeks. > > Who's in? :) > I agree in general with the idea but I think it needs a tad more context. We need to remember that Ironic (ex-Nova Baremetal) was created to fill a gap in OpenStack that was missing for TripleO project to get off the ground. That was the problem being solved and these aspects are reflected in the ReST API: Being admin-only, not "human-friendly" (standalone came later), etc... > * Ironic as a backend to other things. Josh pointed out kubernetes > somewhere, I'd love to be an official backend there. Heat today goes > through Nova to get an ironic instance, it seems reasonable to have > heat talk directly to ironic. Things like that. The amount of work > here might depend on the application using ironic (e.g. I think k8s > has it's own scheduler, heat does not, right?). There was an attempt to do that before in heat, but they were refused at the time because it didn't fit the context above [0]. That wasn't the goal/scope of the project. Now we have v1 is (almost) 3 years and during this time Ironic evolved a _lot_, it does covers way more us
Re: [openstack-dev] [ironic] using ironic as a replacement for existing datacenter baremetal provisioning
On Thu, Jun 09, 2016 at 06:17:56PM +0100, Lucas Alvares Gomes wrote: > Hi, > > Thanks for writing it down Jim. > > > So, I've been thinking about this quite a bit. We've also talked about > > doing a v2 API (as evil as that may be) in Ironic here and there. We've > > had lots of lessons learned from the v1 API, mostly that our API is > > absolutely terrible for humans. I'd love to fix that (whether that > > requires a v2 API or not is unclear, so don't focus on that). > > > > I've noticed that people keep talking about the Nova driver API > > not being public/stable/whatever in this thread - let's ignore that and > > think bigger. > > > > So, there's two large use cases for ironic that we support today: > > > > * Ironic as a backend to nova. Operators still need to interact with the > > Ironic API for management, troubleshooting, and fixing issues that > > computers do not handle today. > > > > * Ironic standalone - by this I mean ironic without nova. The primary > > deployment method here is using Bifrost, and I also call it the > > "better than cobbler" case. I'm not sure if people are using this > > without bifrost, or with other non-nova services, today. Users in this > > model, as I understand things, do not interact with the Ironic API > > directly (except maybe for troubleshooting). > > > > There's other use cases I would like to support: > > > > * Ironic standalone, without Bifrost. I would love for a deployer to be > > able to stand up Ironic as an end-user facing API, probably with > > Keystone, maybe with Neutron/Glance/Swift if needed. This would > > require a ton of discussion and work (e.g. ironic has no concept of > > tenants/projects today, we might want a scheduler, a concept of an > > instance, etc) and would be a very long road. The ideal solution to > > this is to break out the Compute API and scheduler to be separate from > > Nova, but that's an even longer road, so let's pretend I didn't say > > that and not devolve this thread into that conversation (yet). > > > > * Ironic as a backend to other things. Josh pointed out kubernetes > > somewhere, I'd love to be an official backend there. Heat today goes > > through Nova to get an ironic instance, it seems reasonable to have > > heat talk directly to ironic. Things like that. The amount of work > > here might depend on the application using ironic (e.g. I think k8s > > has it's own scheduler, heat does not, right?). > > > > So all that said, I think there is one big step we can take in the > > short-term that works for all of these use cases: make our API better. > > Make it simpler. Take a bunch of the logic in the Nova driver, and put > > it in our API instead. spawn() becomes /v1/nodes/foo/deploy or > > something, etc (I won't let us bikeshed those specifics in this thread). > > Just doing that allows us to remove a bunch of code from a number of > > places (nova, bifrost, shade, tempest(?)) and make those simpler. It > > allows direct API users to more easily deploy things, making one API > > call instead of a bunch (we could even create Neutron ports and such for > > them). It allows k8s and friends to write less code. Oh, let's also stop > > directly exposing state machine transitions as API actions, that's > > crazy, kthx. > > > > I think this is what Josh is trying to get at, except maybe with a > > separate API service in between, which doesn't sound very desirable to > > me. > > > > Thoughts on this? > > > > Additionally, in the somewhat-short term, I'd like us to try to > > enumerate the major use cases we're trying to solve, and make those use > > cases ridiculously simple to deploy. Ironic is quickly becoming a > > tangled mess of configuration options and tweaking surrounding services > > (nova, neutron) to deploy it. Once it's figured out, it works very well. > > However, it's incredibly difficult to figure out how to get there. > > > > Ultimately, I'd like someone that wants to deploy ironic in a common use > > case, with off-the-shelf hardware, to be able to get a POC up and > > running in a matter of hours, not days or weeks. > > > > Who's in? :) > > > > I agree in general with the idea but I think it needs a tad more > context. We need to remember that Ironic (ex-Nova Baremetal) was > created to fill a gap in OpenStack that was missing for TripleO > project to get off the ground. That was the problem being solved and > these aspects are reflected in the ReST API: Being admin-only, not > "human-friendly" (standalone came later), etc... > > > * Ironic as a backend to other things. Josh pointed out kubernetes > > somewhere, I'd love to be an official backend there. Heat today goes > > through Nova to get an ironic instance, it seems reasonable to have > > heat talk directly to ironic. Things like that. The amount of work > > here might depend on the application using ironic (e.g. I think k8s > > has it's own scheduler, heat does not, right?). > > There was an attempt
Re: [openstack-dev] [ironic] using ironic as a replacement for existing datacenter baremetal provisioning
On 06/07/2016 02:34 PM, Joshua Harlow wrote: I'll work on this list, as some folks that are trying start to try to connect ironic (IMHO without nova, because well kubernetes is enough like nova that there isn't a need for 2-layers of nova-like-systems at that point) into kubernetes as a 'resource provider'. I'm sure attempting to do that will expose a bunch of specific recommendations soon enough. Good luck with that. I think you'll find that all of the things that the Nova community has spent the last 4 years working on for the "enterprise" pet VMs, supporting NFV/network I/O-sensitive workloads with NUMA affinity and L2 networking constraints, and non-12-factor-app-from-the-start things are going to want to be addressed in k8s [1]. I won't at all be surprised to see the k8s community three years from now be where Nova is right now with regards to supporting these terrible stateful applications. k8s and mesos are all nice and hot right now, as is anything container-related. Just because something is trendy and hot, however, does not suddenly make the problem domains that more traditional stateful applications find themselves in magically disappear. Best, -jay [1] https://github.com/kubernetes/kubernetes/issues/260 __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [ironic] using ironic as a replacement for existing datacenter baremetal provisioning
Jay Pipes wrote: On 06/07/2016 02:34 PM, Joshua Harlow wrote: I'll work on this list, as some folks that are trying start to try to connect ironic (IMHO without nova, because well kubernetes is enough like nova that there isn't a need for 2-layers of nova-like-systems at that point) into kubernetes as a 'resource provider'. I'm sure attempting to do that will expose a bunch of specific recommendations soon enough. Good luck with that. Thank you very much for the blessings sir jay ;) I think you'll find that all of the things that the Nova community has spent the last 4 years working on for the "enterprise" pet VMs, supporting NFV/network I/O-sensitive workloads with NUMA affinity and L2 networking constraints, and non-12-factor-app-from-the-start things are going to want to be addressed in k8s [1]. I won't at all be surprised to see the k8s community three years from now be where Nova is right now with regards to supporting these terrible stateful applications. Ya, there will always be this 'temptation' and I know it's hard to resist, there is always a temptress in the room, this is always one of those :-P k8s and mesos are all nice and hot right now, as is anything container-related. Just because something is trendy and hot, however, does not suddenly make the problem domains that more traditional stateful applications find themselves in magically disappear. Best, -jay [1] https://github.com/kubernetes/kubernetes/issues/260 __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [ironic] using ironic as a replacement for existing datacenter baremetal provisioning
Hi, >> I agree in general with the idea but I think it needs a tad more >> context. We need to remember that Ironic (ex-Nova Baremetal) was >> created to fill a gap in OpenStack that was missing for TripleO >> project to get off the ground. That was the problem being solved and >> these aspects are reflected in the ReST API: Being admin-only, not >> "human-friendly" (standalone came later), etc... > > Sorry, I didn't mean to slag on people here. In fact, I tried to come up > with a way to say "no offense" but couldn't figure the words out. Ironic > did start with a very specific use case, it's come a super long way, and > you all did what you had to do to get things going. For that I'm forever > indebted to you. :) > No offense taken at all, I also do think that the current API is absolutely terrible for humans! I just wanted to point out that it wasn't actually architected for it, plus, IIRC nobody in the project at the time had much - if any - experience designing ReST APIs. > ++, I do agree we could make a v2 faster than shoehorning things into > v1. The "evil" part of my comment is around removing v1 in the future, > actually. No matter the project, it's a long hard road, and will take > years to do (and even then some tools will likely be left old and > broken). > Yeah on that perspective it's evil indeed :-/ Cheers, Lucas __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [ironic] using ironic as a replacement for existing datacenter baremetal provisioning
Jim Rollenhagen wrote: 1.)Nova<-> ironic interactions are generally seem terrible? I don't know if I'd call it terrible, but there's friction. Things that are unchangable on hardware are just software configs in vms (like mac addresses, overlays, etc), and things that make no sense in VMs are pretty standard on servers (trunked vlans, bonding, etc). One way we've gotten around it is by using Ironic standalone via Bifrost[1]. This deploys Ironic in wide open auth mode on 127.0.0.1, and includes playbooks to build config drives and deploy images in a fairly rudimentary way without Nova. I call this the "better than Cobbler" way of getting a toe into the Ironic waters. [1] https://github.com/openstack/bifrost Out of curiosity, why ansible vs turning https://github.com/openstack/nova/blob/master/nova/virt/ironic/driver.py (or something like it) into a tiny-wsgi-app (pick useful name here) that has its own REST api (that looks pretty similar to the public functions in that driver file)? That's an interesting idea. I think a reason Bifrost doesn't just import nova virt drivers is that they're likely _not_ a supported public API (despite not having _'s at the front). Also, a lot of the reason Bifrost exists is to enable users to get the benefits of all the baremetal abstraction work done in Ironic without having to fully embrace all of OpenStack's core. So while you could get a little bit of the stuff from nova (like config drive building), you'd still need to handle network address assignment, image management, etc. etc., and pretty soon you start having to run a tiny glance and a tiny neutron. The Bifrost way is the opposite: I just want a tiny Ironic, and _nothing_ else. Ya, I'm just thinking that at a certain point Oops forgot to fill this out, was just thinking that at a certain point it might be easier to figure out how to extract that API (meh, if its public or private) and just have someone make an executive decision around ironic being a stand-alone thing or not (and a capable stand-alone thing, not a sorta-standalone-thing). So, I've been thinking about this quite a bit. We've also talked about doing a v2 API (as evil as that may be) in Ironic here and there. We've had lots of lessons learned from the v1 API, mostly that our API is absolutely terrible for humans. I'd love to fix that (whether that requires a v2 API or not is unclear, so don't focus on that). I've noticed that people keep talking about the Nova driver API not being public/stable/whatever in this thread - let's ignore that and think bigger. So, there's two large use cases for ironic that we support today: * Ironic as a backend to nova. Operators still need to interact with the Ironic API for management, troubleshooting, and fixing issues that computers do not handle today. * Ironic standalone - by this I mean ironic without nova. The primary deployment method here is using Bifrost, and I also call it the "better than cobbler" case. I'm not sure if people are using this without bifrost, or with other non-nova services, today. Users in this model, as I understand things, do not interact with the Ironic API directly (except maybe for troubleshooting). There's other use cases I would like to support: * Ironic standalone, without Bifrost. I would love for a deployer to be able to stand up Ironic as an end-user facing API, probably with Keystone, maybe with Neutron/Glance/Swift if needed. This would require a ton of discussion and work (e.g. ironic has no concept of tenants/projects today, we might want a scheduler, a concept of an instance, etc) and would be a very long road. The ideal solution to this is to break out the Compute API and scheduler to be separate from Nova, but that's an even longer road, so let's pretend I didn't say that and not devolve this thread into that conversation (yet). That'd be nice, that is more of what I was thinking when I think of 'standalone' in that it's standalone from the aspect of not needing to go through another compute layer [nova] to get to the bottom compute layer [ironic], whereas bifrost (from my small understanding of it) also discards the rest of the openstack services (and therefore also discarding the non-zero amount of functionality they provide). * Ironic as a backend to other things. Josh pointed out kubernetes somewhere, I'd love to be an official backend there. Heat today goes through Nova to get an ironic instance, it seems reasonable to have heat talk directly to ironic. Things like that. The amount of work here might depend on the application using ironic (e.g. I think k8s has it's own scheduler, heat does not, right?). Correct, I'm a big fan of this, and I think openstack as a community needs more of it... We as a community IMHO need to embrace and be best buddies (pick other word here) with other projects that are outside of openstack that are in what I would call the 'cloud family' because i
