Re: [openstack-dev] Shovel (RackHD/OpenStack)
Heck, Joseph wrote: Thanks for the links Jeremy! I'm still reading through what exactly "bigtent" means, not sure I grok the placement for littler/ancillary things like this effort, but the the links are hugely helpful! Hey Joe, welcome back :P Rather than defining OpenStack around the "Integrated Release", and having integrated, incubated and stackforge things, we now define OpenStack based on a mission and way to do development, and have only one category of official things. If RackHD helps with the OpenStack Mission and is developed in the OpenStack Way, then it can definitely apply to become "an OpenStack project". It's still a bit far away from that state though... the first step if you want to go in that direction would be to host it under OpenStack dev infrastructure. Cheers :) -- Thierry Carrez (ttx) __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Shovel (RackHD/OpenStack)
On Wed, Jan 13, 2016 at 09:56:57PM +, Heck, Joseph wrote: > Hey Jay! (yeah, I’m here and lurking in the corners, albeit with a > different email at the moment) > > Yep - RackHD was created by a company that was acquired by EMC to attack > the lowest-level of hardware automation. EMC was interesting in pushing > that into Open Source, and surprisingly I was completely game for that > project :-) There’s all sorts of PR around that project that I won’t > bother replaying here, but if anyone’s interested, I’d be happy to share > more details. > > There was immediately interest in how this could work with OpenStack, and > as a plugin/driver to Ironic was the obvious play. We took a couple > different options of possible attacks, and decided to leverage something > that would both show off the underlying hardware introspection which > wasn’t obviously visible (or arguably perhaps relevant) from the Ironic > style APIs (the horizon plugin) as well as be leverage by Ironic to do > hardware provisioning using those APIs. > > Andre (who was key in doing this effort inside EMC) was interested in > helping manage it and is bringing it here to introduce folks to the fact > that we’ve done this work, and that it will be submitted it into > incubation with OpenStack. So yep - we totally want to contribute it to > the Ironic efforts. Andre and Jim are coordinating on that effort (hi > jroll, nice to meet you) and it was Jim that suggested that Andre let the > community know here that we’ve started this effort. Hi. :) So, to be clear, I haven't been working with Andre on this, except to help him figure out how to create an OpenStack project, and to suggest he email this list. From what I know, the things RackHD/shovel offer that Ironic (Inspector) doesn't have is additional SEL monitoring, as well as the capability to register a second ironic node as a failover for another node (I haven't investigated how this actually works). I do share Jay's concern - why are these separate projects, rather than contributing to ironic (inspector) itself? Why would a user want to use both ironic *and* RackHD? From the RackHD docs: "RackHD is focused on being the lowest level of automation that interrogates agnostic hardware and provisions machines with operating systems." And the Ironic mission statement: "To produce an OpenStack service and associated libraries capable of managing and provisioning physical machines, and to do this in a security-aware and fault-tolerant manner." So, I'm not sure I see much difference in the goals, which makes me wonder if ironic and RackHD are truly complementary (as shovel implies) or if they are actually aiming to do the same thing. I'd love to see RackHD folks contributing to Ironic. Would it possible for EMC to work on contributing RackHD features that Ironic lacks into Ironic, rather than building a bridge between the two? // jim > > Anyway, I’m lurking here again - but Andre is doing to real lifting :-) > > -joe > > On 1/13/16, 1:22 PM, "Mooney, Sean K" <sean.k.moo...@intel.com> wrote: > > > > > > >> -Original Message- > >> From: Jay Pipes [mailto:jaypi...@gmail.com] > >> Sent: Wednesday, January 13, 2016 8:53 PM > >> To: openstack-dev@lists.openstack.org > >> Subject: Re: [openstack-dev] Shovel (RackHD/OpenStack) > >> > >> On 01/13/2016 03:28 PM, Keedy, Andre wrote: > >> > Hi All, I'm pleased to announce a new application called 'Shovel 'that > >> > is now available in a public repository on GitHub > >> > (https://github.com/keedya/Shovel). Shovel is a server with a set of > >> > APIs that wraps around RackHD/Ironic's existing APIs allowing users to > >> > find Baremetal Compute nodes that are dynamically discovered by RackHD > >> > and register them with Ironic. Shovel also uses the SEL pollers > >> > service in RackHD to monitor compute nodes and logs errors from SEL > >> > into the Ironic Database. Shovel includes a graphical interface using > >> Swagger UI. > >> > > >> > Also provided is a Shovel Horizon plugin to interface with the Shovel > >> > service that is available in a public repository on GitHub > >> > (https://github.com/keedya/shovel-horizon-plugin). The Plugin adds a > >> > new Panel to the admin Dashboard called rackhd that displays a table > >> > of all the Baremetal systems discovered by RackHD. It also allows the > >> > user to see the node catalog in a nice table view, register/unregister > >> > node in Ironic, display node SEL and enable/register a failover node. > >> > > >> > I invite you to take a look at Shovel an
Re: [openstack-dev] Shovel (RackHD/OpenStack)
They’re definitely overlapping, but RackHD wasn’t created, and isn’t meant, to be specific to OpenStack, but as a more general purpose need for low level automation of hardware. Just like there’s Cobbler, Razor, and Hanlon out there - RackHD isn’t aiming to be OpenStack, just agnostic, and with some different features and functionality than what Ironic is trying to do. That said, we do think they could work well together, which was the point of putting together this effort and submitting it to the community for potential inclusion within OpenStack, in my mind in the same fashion that you have a Cisco or Juniper driver set of Neutron (yes, I still think of it as Quantum). I suggested that Andre reach out to you guys to see how best to accommodate that. We’ve started the process to set it up for incubation - is that still the best route to take? Or more specifically to the general community as well to to Ironic folks - in what way can we provide the means to make this most available to the OpenStack Community at large? I haven’t been following the community and the norms closely for a couple years - Stackforge appears to have dissipated in favor of projects in incubation to share things like this. Is that accurate? -joe On 1/15/16, 10:15 AM, "Jim Rollenhagen" <j...@jimrollenhagen.com> wrote: >On Wed, Jan 13, 2016 at 09:56:57PM +, Heck, Joseph wrote: >> Hey Jay! (yeah, I’m here and lurking in the corners, albeit with a >> different email at the moment) >> >> Yep - RackHD was created by a company that was acquired by EMC to attack >> the lowest-level of hardware automation. EMC was interesting in pushing >> that into Open Source, and surprisingly I was completely game for that >> project :-) There’s all sorts of PR around that project that I won’t >> bother replaying here, but if anyone’s interested, I’d be happy to share >> more details. >> >> There was immediately interest in how this could work with OpenStack, >>and >> as a plugin/driver to Ironic was the obvious play. We took a couple >> different options of possible attacks, and decided to leverage something >> that would both show off the underlying hardware introspection which >> wasn’t obviously visible (or arguably perhaps relevant) from the Ironic >> style APIs (the horizon plugin) as well as be leverage by Ironic to do >> hardware provisioning using those APIs. >> >> Andre (who was key in doing this effort inside EMC) was interested in >> helping manage it and is bringing it here to introduce folks to the fact >> that we’ve done this work, and that it will be submitted it into >> incubation with OpenStack. So yep - we totally want to contribute it to >> the Ironic efforts. Andre and Jim are coordinating on that effort (hi >> jroll, nice to meet you) and it was Jim that suggested that Andre let >>the >> community know here that we’ve started this effort. > >Hi. :) > >So, to be clear, I haven't been working with Andre on this, except to >help him figure out how to create an OpenStack project, and to suggest >he email this list. > >From what I know, the things RackHD/shovel offer that Ironic (Inspector) >doesn't have is additional SEL monitoring, as well as the capability to >register a second ironic node as a failover for another node (I haven't >investigated how this actually works). > >I do share Jay's concern - why are these separate projects, rather than >contributing to ironic (inspector) itself? Why would a user want to use >both ironic *and* RackHD? > >From the RackHD docs: >"RackHD is focused on being the lowest level of automation that >interrogates agnostic hardware and provisions machines with operating >systems." > >And the Ironic mission statement: >"To produce an OpenStack service and associated libraries capable of >managing and provisioning physical machines, and to do this in a >security-aware and fault-tolerant manner." > >So, I'm not sure I see much difference in the goals, which makes me >wonder if ironic and RackHD are truly complementary (as shovel implies) >or if they are actually aiming to do the same thing. > >I'd love to see RackHD folks contributing to Ironic. Would it possible >for EMC to work on contributing RackHD features that Ironic lacks into >Ironic, rather than building a bridge between the two? > >// jim > >> >> Anyway, I’m lurking here again - but Andre is doing to real lifting :-) >> >> -joe >> >> On 1/13/16, 1:22 PM, "Mooney, Sean K" <sean.k.moo...@intel.com> wrote: >> >> > >> > >> >> -Original Message- >> >> From: Jay Pipes [mailto:jaypi...@gmail.com] >> >> Sent: Wednesda
Re: [openstack-dev] Shovel (RackHD/OpenStack)
On 2016-01-15 18:44:42 + (+), Heck, Joseph wrote: [...] > I haven’t been following the community and the norms closely for a couple > years - Stackforge appears to have dissipated in favor of projects in > incubation to share things like this. Is that accurate? The idea of "incubating" before becoming an official project basically went away with the advent of the "Big Tent"[1]. Also the _term_ "StackForge" isn't dead (yet[2] anyway), but we have unofficial^WStackForge repos just share the same Git repository namespace[3] with official ones now for ease of management and to reduce the amount of disruptive renaming we used to have to accommodate moving between namespaces. [1] http://governance.openstack.org/resolutions/20141202-project-structure-reform-spec.html [2] https://review.openstack.org/265352 [3] http://governance.openstack.org/resolutions/20150615-stackforge-retirement.html -- Jeremy Stanley __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Shovel (RackHD/OpenStack)
Yep! Thanks Jim! ___ From: Jim Rollenhagen <j...@jimrollenhagen.com<mailto:j...@jimrollenhagen.com>> Sent: Friday, January 15, 2016 5:48 PM Subject: Re: [openstack-dev] Shovel (RackHD/OpenStack) To: OpenStack Development Mailing List (not for usage questions) <openstack-dev@lists.openstack.org<mailto:openstack-dev@lists.openstack.org>> On Fri, Jan 15, 2016 at 06:44:42PM +, Heck, Joseph wrote: > They’re definitely overlapping, but RackHD wasn’t created, and isn’t > meant, to be specific to OpenStack, but as a more general purpose need for > low level automation of hardware. Just like there’s Cobbler, Razor, and > Hanlon out there - RackHD isn’t aiming to be OpenStack, just agnostic, and > with some different features and functionality than what Ironic is trying > to do. Right, I agree. I just don't see why a deployer would want to stand up ironic (and shovel) if they already had RackHD running, or vice versa. There's a pretty small feature gap, and it seems like features Ironic would like to have but doesn't yet. > That said, we do think they could work well together, which was the point > of putting together this effort and submitting it to the community for > potential inclusion within OpenStack, in my mind in the same fashion that > you have a Cisco or Juniper driver set of Neutron (yes, I still think of > it as Quantum). I suggested that Andre reach out to you guys to see how > best to accommodate that. > > We’ve started the process to set it up for incubation - is that still the > best route to take? Or more specifically to the general community as well > to to Ironic folks - in what way can we provide the means to make this > most available to the OpenStack Community at large? > > I haven’t been following the community and the norms closely for a couple > years - Stackforge appears to have dissipated in favor of projects in > incubation to share things like this. Is that accurate? Right, like Jeremy said, you'd now just create an unofficial OpenStack project, as described here: http://docs.openstack.org/infra/manual/creators.html Then you'd be able to use the OpenStack infrastructure, like CI systems and Launchpad. The governance step is the part that makes it "official", and you could skip that for now. Later, if we do find that this is something that should be managed by the Ironic team, we'd add it to the Ironic project in the governance repo, making it an official OpenStack project. Note that you could also start your own project team (similar to the ironic project team) with its own mission statement and set of code repositories. This is roughly the equivalent of the old "incubation" thing OpenStack used to do. Does that help? // jim > > -joe > > On 1/15/16, 10:15 AM, "Jim Rollenhagen" > <j...@jimrollenhagen.com<mailto:j...@jimrollenhagen.com>> wrote: > >On Wed, Jan 13, 2016 at 09:56:57PM +, Heck, Joseph wrote: > >> Hey Jay! (yeah, I’m here and lurking in the corners, albeit with a > >> different email at the moment) > >> > >> Yep - RackHD was created by a company that was acquired by EMC to attack > >> the lowest-level of hardware automation. EMC was interesting in pushing > >> that into Open Source, and surprisingly I was completely game for that > >> project :-) There’s all sorts of PR around that project that I won’t > >> bother replaying here, but if anyone’s interested, I’d be happy to share > >> more details. > >> > >> There was immediately interest in how this could work with OpenStack, > >>and > >> as a plugin/driver to Ironic was the obvious play. We took a couple > >> different options of possible attacks, and decided to leverage something > >> that would both show off the underlying hardware introspection which > >> wasn’t obviously visible (or arguably perhaps relevant) from the Ironic > >> style APIs (the horizon plugin) as well as be leverage by Ironic to do > >> hardware provisioning using those APIs. > >> > >> Andre (who was key in doing this effort inside EMC) was interested in > >> helping manage it and is bringing it here to introduce folks to the fact > >> that we’ve done this work, and that it will be submitted it into > >> incubation with OpenStack. So yep - we totally want to contribute it to > >> the Ironic efforts. Andre and Jim are coordinating on that effort (hi > >> jroll, nice to meet you) and it was Jim that suggested that Andre let > >>the > >> community know here that we’ve started this effort. > > > >Hi. :) > > > >So, to be clear, I haven't been working with Andre on this, except to > >help him figure
Re: [openstack-dev] Shovel (RackHD/OpenStack)
Thanks for the links Jeremy! I'm still reading through what exactly "bigtent" means, not sure I grok the placement for littler/ancillary things like this effort, but the the links are hugely helpful! - joe _ From: Jeremy Stanley <fu...@yuggoth.org<mailto:fu...@yuggoth.org>> Sent: Friday, January 15, 2016 11:44 AM Subject: Re: [openstack-dev] Shovel (RackHD/OpenStack) To: OpenStack Development Mailing List (not for usage questions) <openstack-dev@lists.openstack.org<mailto:openstack-dev@lists.openstack.org>> On 2016-01-15 18:44:42 + (+), Heck, Joseph wrote: [...] > I haven’t been following the community and the norms closely for a couple > years - Stackforge appears to have dissipated in favor of projects in > incubation to share things like this. Is that accurate? The idea of "incubating" before becoming an official project basically went away with the advent of the "Big Tent"[1]. Also the _term_ "StackForge" isn't dead (yet[2] anyway), but we have unofficial^WStackForge repos just share the same Git repository namespace[3] with official ones now for ease of management and to reduce the amount of disruptive renaming we used to have to accommodate moving between namespaces. [1] http://governance.openstack.org/resolutions/20141202-project-structure-reform-spec.html [2] https://review.openstack.org/265352 [3] http://governance.openstack.org/resolutions/20150615-stackforge-retirement.html -- Jeremy Stanley __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org<mailto:openstack-dev-requ...@lists.openstack.org>?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Shovel (RackHD/OpenStack)
> -Original Message- > From: Jay Pipes [mailto:jaypi...@gmail.com] > Sent: Wednesday, January 13, 2016 8:53 PM > To: openstack-dev@lists.openstack.org > Subject: Re: [openstack-dev] Shovel (RackHD/OpenStack) > > On 01/13/2016 03:28 PM, Keedy, Andre wrote: > > Hi All, I'm pleased to announce a new application called 'Shovel 'that > > is now available in a public repository on GitHub > > (https://github.com/keedya/Shovel). Shovel is a server with a set of > > APIs that wraps around RackHD/Ironic's existing APIs allowing users to > > find Baremetal Compute nodes that are dynamically discovered by RackHD > > and register them with Ironic. Shovel also uses the SEL pollers > > service in RackHD to monitor compute nodes and logs errors from SEL > > into the Ironic Database. Shovel includes a graphical interface using > Swagger UI. > > > > Also provided is a Shovel Horizon plugin to interface with the Shovel > > service that is available in a public repository on GitHub > > (https://github.com/keedya/shovel-horizon-plugin). The Plugin adds a > > new Panel to the admin Dashboard called rackhd that displays a table > > of all the Baremetal systems discovered by RackHD. It also allows the > > user to see the node catalog in a nice table view, register/unregister > > node in Ironic, display node SEL and enable/register a failover node. > > > > I invite you to take a look at Shovel and Shovel horizon plugin that > > is available to the public on GitHub. > > Would EMC be interested in contributing to the OpenStack Ironic project > around hardware discovery and automated registration of hardware? It > would be nice to have a single community pulling in the same direction. > It looks to me that RackHD is only a few months old. Was there a > particular reason that EMC decided to start a new open source project > for doing hardware management instead of contributing to the OpenStack > Ironic project? > > It was a bit surprising to me actually, to see Joe Heck, who used to be > a very active contributor in OpenStack, started the RackHD project. > > Also, just FYI, "Shovel" is a RabbitMQ thing: > > https://www.rabbitmq.com/shovel.html > > Might be worth looking into a rename of your project to avoid confusion, > but that's just a suggestion. Its also a python library for converting function into tasks invokable >From the commandline however it has not had a release in the past year so Development may not be ongoing. https://github.com/seomoz/shovel https://pypi.python.org/pypi/shovel > > Best, > -jay > > > > __ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: OpenStack-dev- > requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] Shovel (RackHD/OpenStack)
Hi All, I'm pleased to announce a new application called 'Shovel 'that is now available in a public repository on GitHub (https://github.com/keedya/Shovel). Shovel is a server with a set of APIs that wraps around RackHD/Ironic's existing APIs allowing users to find Baremetal Compute nodes that are dynamically discovered by RackHD and register them with Ironic. Shovel also uses the SEL pollers service in RackHD to monitor compute nodes and logs errors from SEL into the Ironic Database. Shovel includes a graphical interface using Swagger UI. Also provided is a Shovel Horizon plugin to interface with the Shovel service that is available in a public repository on GitHub (https://github.com/keedya/shovel-horizon-plugin). The Plugin adds a new Panel to the admin Dashboard called rackhd that displays a table of all the Baremetal systems discovered by RackHD. It also allows the user to see the node catalog in a nice table view, register/unregister node in Ironic, display node SEL and enable/register a failover node. I invite you to take a look at Shovel and Shovel horizon plugin that is available to the public on GitHub. Thanks Andre __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Shovel (RackHD/OpenStack)
On 01/13/2016 03:28 PM, Keedy, Andre wrote: Hi All, I’m pleased to announce a new application called ‘Shovel ‘that is now available in a public repository on GitHub (https://github.com/keedya/Shovel). Shovel is a server with a set of APIs that wraps around RackHD/Ironic’s existing APIs allowing users to find Baremetal Compute nodes that are dynamically discovered by RackHD and register them with Ironic. Shovel also uses the SEL pollers service in RackHD to monitor compute nodes and logs errors from SEL into the Ironic Database. Shovel includes a graphical interface using Swagger UI. Also provided is a Shovel Horizon plugin to interface with the Shovel service that is available in a public repository on GitHub (https://github.com/keedya/shovel-horizon-plugin). The Plugin adds a new Panel to the admin Dashboard called rackhd that displays a table of all the Baremetal systems discovered by RackHD. It also allows the user to see the node catalog in a nice table view, register/unregister node in Ironic, display node SEL and enable/register a failover node. I invite you to take a look at Shovel and Shovel horizon plugin that is available to the public on GitHub. Would EMC be interested in contributing to the OpenStack Ironic project around hardware discovery and automated registration of hardware? It would be nice to have a single community pulling in the same direction. It looks to me that RackHD is only a few months old. Was there a particular reason that EMC decided to start a new open source project for doing hardware management instead of contributing to the OpenStack Ironic project? It was a bit surprising to me actually, to see Joe Heck, who used to be a very active contributor in OpenStack, started the RackHD project. Also, just FYI, "Shovel" is a RabbitMQ thing: https://www.rabbitmq.com/shovel.html Might be worth looking into a rename of your project to avoid confusion, but that's just a suggestion. Best, -jay __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
