[openstack-dev] [heat] [scheduler] Bringing things together for Icehouse

2013-09-15 Thread Mike Spreitzer
I've read up on recent goings-on in the scheduler subgroup, and have some thoughts to contribute. But first I must admit that I am still a newbie to OpenStack, and still am missing some important clues. One thing that mystifies me is this: I see essentially the same thing, which I have general

Re: [openstack-dev] [heat] [scheduler] Bringing things together for Icehouse

2013-09-15 Thread Gary Kotton
AM To: OpenStack Development Mailing List mailto:openstack-dev@lists.openstack.org>> Subject: [openstack-dev] [heat] [scheduler] Bringing things together for Icehouse I've read up on recent goings-on in the scheduler subgroup, and have some thoughts to contribute. But first I must admit

Re: [openstack-dev] [heat] [scheduler] Bringing things together for Icehouse

2013-09-15 Thread Mike Spreitzer
> From: Gary Kotton > ... > Can you please join us at the up and coming scheduler meeting. That > will give you a chance to bring up the idea's and discuss them with > a larger audience. I will do so on Sep 17. Later meetings still TBD. Regards, Mike___

Re: [openstack-dev] [heat] [scheduler] Bringing things together for Icehouse

2013-09-16 Thread Mike Spreitzer
I have written a brief document, with pictures. See https://docs.google.com/document/d/1hQQGHId-z1A5LOipnBXFhsU3VAMQdSe-UXvL4VPY4ps Regards, Mike___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman

Re: [openstack-dev] [heat] [scheduler] Bringing things together for Icehouse

2013-09-17 Thread Gary Kotton
penstack-dev@lists.openstack.org>> Subject: Re: [openstack-dev] [heat] [scheduler] Bringing things together for Icehouse I have written a brief document, with pictures. See https://docs.google.com/document/d/1hQQGHId-z1A5LOipnBXFhsU3VAMQdSe-UXvL4VPY

Re: [openstack-dev] [heat] [scheduler] Bringing things together for Icehouse

2013-09-17 Thread Mike Spreitzer
Fixed, sorry. From: Gary Kotton To: OpenStack Development Mailing List , Date: 09/17/2013 03:26 AM Subject:Re: [openstack-dev] [heat] [scheduler] Bringing things together for Icehouse Hi, The document is locked. Thanks Gary From: Mike Spreitzer Reply-To: OpenStack

Re: [openstack-dev] [heat] [scheduler] Bringing things together for Icehouse

2013-09-18 Thread Stein, Manuel (Manuel)
_ From: Mike Spreitzer [mailto:mspre...@us.ibm.com] Sent: Dienstag, 17. September 2013 07:00 To: OpenStack Development Mailing List Subject: Re: [openstack-dev] [heat] [scheduler] Bringing things together for Icehouse I have written a brief document, with pictures. Se

Re: [openstack-dev] [heat] [scheduler] Bringing things together for Icehouse

2013-09-18 Thread Mike Spreitzer
Manuel, and others: I am sorry, in the rush at the end of the scheduler meeting a critical fact flew from my mind: the material I distributed beforehand was intended as something I could reference during discussion in the meeting, I did not expect it to fully stand on its own. Indeed, you have

Re: [openstack-dev] [heat] [scheduler] Bringing things together for Icehouse

2013-09-20 Thread Mike Spreitzer
I have written a new outline of my thoughts, you can find it at https://docs.google.com/document/d/1RV_kN2Io4dotxZREGEks9DM0Ih_trFZ-PipVDdzxq_E It is intended to stand up better to independent study. However, it is still just an outline. I am still learning about stuff going on in OpenStack,

Re: [openstack-dev] [heat] [scheduler] Bringing things together for Icehouse

2013-09-20 Thread Christopher Armstrong
Hi Mike, I have a *slightly* better idea of the kind of stuff you're talking about, but I think it would really help if you could include some concrete real-world use cases and describe why a holistic scheduler inside of Heat is necessary for solving them. On Fri, Sep 20, 2013 at 2:13 AM, Mike S

Re: [openstack-dev] [heat] [scheduler] Bringing things together for Icehouse

2013-09-23 Thread Zane Bitter
On 15/09/13 09:19, Mike Spreitzer wrote: But first I must admit that I am still a newbie to OpenStack, and still am missing some important clues. One thing that mystifies me is this: I see essentially the same thing, which I have generally taken to calling holistic scheduling, discussed in two m

Re: [openstack-dev] [heat] [scheduler] Bringing things together for Icehouse

2013-09-23 Thread Keith Bray
I think this picture is relevant to Heat context: https://docs.google.com/drawings/d/1Y_yyIpql5_cdC8116XrBHzn6GfP_g0NHTTG_W4o 0R9U/edit As more and more types of compute (containers, VMs, bare metal) and other resources (geographically dispersed) become available from the cloud with boarder capa

Re: [openstack-dev] [heat] [scheduler] Bringing things together for Icehouse

2013-09-23 Thread Clint Byrum
Excerpts from Keith Bray's message of 2013-09-23 12:22:16 -0700: > > I think this picture is relevant to Heat context: > https://docs.google.com/drawings/d/1Y_yyIpql5_cdC8116XrBHzn6GfP_g0NHTTG_W4o > 0R9U/edit > > > As more and more types of compute (containers, VMs, bare metal) and other > resou

Re: [openstack-dev] [heat] [scheduler] Bringing things together for Icehouse

2013-09-23 Thread Mike Spreitzer
I was not trying to raise issues of geographic dispersion and other higher level structures, I think the issues I am trying to raise are relevant even without them. This is not to deny the importance, or relevance, of higher levels of structure. But I would like to first respond to the discus

Re: [openstack-dev] [heat] [scheduler] Bringing things together for Icehouse

2013-09-23 Thread Mike Spreitzer
Someone earlier asked for greater clarity about infrastructure orchestration, so here is my view. I see two main issues: (1) deciding the order in which to do things, and (2) doing them in an acceptable order. That's an oversimplified wording because, in general, some parallelism is possible.

Re: [openstack-dev] [heat] [scheduler] Bringing things together for Icehouse

2013-09-23 Thread Clint Byrum
Excerpts from Mike Spreitzer's message of 2013-09-23 20:31:32 -0700: > I was not trying to raise issues of geographic dispersion and other higher > level structures, I think the issues I am trying to raise are relevant > even without them. This is not to deny the importance, or relevance, of >

Re: [openstack-dev] [heat] [scheduler] Bringing things together for Icehouse

2013-09-24 Thread Zane Bitter
On 24/09/13 05:31, Mike Spreitzer wrote: I was not trying to raise issues of geographic dispersion and other higher level structures, I think the issues I am trying to raise are relevant even without them. This is not to deny the importance, or relevance, of higher levels of structure. But I wo

Re: [openstack-dev] [heat] [scheduler] Bringing things together for Icehouse

2013-09-24 Thread Debojyoti Dutta
Joining the party late :) I think there have been a lot of interesting ideas around wholistic scheduling over the last few summits. However seems there is no clear agreement on 1) where it should be specified and implemented 2) what the specifications look like - VRT, policies, templates etc etc e

Re: [openstack-dev] [heat] [scheduler] Bringing things together for Icehouse

2013-09-25 Thread Mike Spreitzer
Debo, Yathi: I have read https://docs.google.com/document/d/1IiPI0sfaWb1bdYiMWzAAx0HYR6UqzOan_Utgml5W1HI/edit?pli=1 and most of the referenced materials, and I have a couple of big-picture questions. That document talks about making Nova call out to something that makes the sort of smart deci

Re: [openstack-dev] [heat] [scheduler] Bringing things together for Icehouse

2013-09-27 Thread Debojyoti Dutta
Hi Mike We understand your point, and, academically, we agree to a large extent. We aware of the optimization results for holistic placement etc (and how any realistic formulation is hard). When we look at the scheduler today, it does one thing at a time because it was meant to be pragmatic when i

Re: [openstack-dev] [heat] [scheduler] Bringing things together for Icehouse (now featuring software orchestration)

2013-09-24 Thread Mike Spreitzer
Let me elaborate a little on my thoughts about software orchestration, and respond to the recent mails from Zane and Debo. I have expanded my picture at https://docs.google.com/drawings/d/1Y_yyIpql5_cdC8116XrBHzn6GfP_g0NHTTG_W4o0R9U and added a companion picture at https://docs.google.com/dra

Re: [openstack-dev] [heat] [scheduler] Bringing things together for Icehouse (now featuring software orchestration)

2013-09-24 Thread Clint Byrum
Excerpts from Mike Spreitzer's message of 2013-09-24 22:03:21 -0700: > Let me elaborate a little on my thoughts about software orchestration, and > respond to the recent mails from Zane and Debo. I have expanded my > picture at > https://docs.google.com/drawings/d/1Y_yyIpql5_cdC8116XrBHzn6GfP_g

Re: [openstack-dev] [heat] [scheduler] Bringing things together for Icehouse (now featuring software orchestration)

2013-09-25 Thread Thomas Spatzier
Clint Byrum wrote on 25.09.2013 08:46:57: > From: Clint Byrum > To: openstack-dev , > Date: 25.09.2013 08:48 > Subject: Re: [openstack-dev] [heat] [scheduler] Bringing things > together for Icehouse (now featuring software orchestration) > > Excerpts from Mike Spreitzer&#x

Re: [openstack-dev] [heat] [scheduler] Bringing things together for Icehouse (now featuring software orchestration)

2013-09-25 Thread Mike Spreitzer
Clint wrote: > There is a third stealth-objective that CFN has caused to linger in > Heat. That is "packaging cloud applications". By allowing the 100% > concrete CFN template to stand alone, users can "ship" the template. > > IMO this marrying of software assembly, config, and orchestration is a

Re: [openstack-dev] [heat] [scheduler] Bringing things together for Icehouse (now featuring software orchestration)

2013-09-25 Thread Clint Byrum
Excerpts from Thomas Spatzier's message of 2013-09-25 00:59:44 -0700: > Clint Byrum wrote on 25.09.2013 08:46:57: > > From: Clint Byrum > > To: openstack-dev , > > Date: 25.09.2013 08:48 > > Subject: Re: [openstack-dev] [heat] [scheduler] Bringing things > >

Re: [openstack-dev] [heat] [scheduler] Bringing things together for Icehouse (now featuring software orchestration)

2013-09-25 Thread Thomas Spatzier
Excerpt from Clint's mail on 25.09.2013 22:23:07: > > I think we already have some summit suggestions for discussing HOT, > it would be good to come prepared with some visions for the future > of HOT so that we can hash these things out, so I'd like to see this > discussion continue. Absolutely!

Re: [openstack-dev] [heat] [scheduler] Bringing things together for Icehouse (now featuring software orchestration)

2013-09-26 Thread Steven Hardy
On Wed, Sep 25, 2013 at 11:04:54PM +0200, Thomas Spatzier wrote: > Excerpt from Clint's mail on 25.09.2013 22:23:07: > > > > > I think we already have some summit suggestions for discussing HOT, > > it would be good to come prepared with some visions for the future > > of HOT so that we can hash t

Re: [openstack-dev] [heat] [scheduler] Bringing things together for Icehouse (now featuring software orchestration)

2013-09-27 Thread Zane Bitter
On 25/09/13 07:03, Mike Spreitzer wrote: Zane wrote: > To take the first example, wouldn't your holistic scheduler effectively have > to reserve a compute instance and some directly attached block storage prior > to actually creating them? Have you considered Climate rather than Heat as > an

Re: [openstack-dev] [heat] [scheduler] Bringing things together for Icehouse (now featuring software orchestration)

2013-09-27 Thread Mike Spreitzer
Zane Bitter wrote on 09/27/2013 08:24:49 AM: > Your diagrams clearly show scheduling happening in a separate stage to > (infrastructure) orchestration, which is to say that at the point where > resources are scheduled, their actual creation is in the *future*. > > I am not a Climate expert, bu

Re: [openstack-dev] [heat] [scheduler] Bringing things together for Icehouse (now featuring software orchestration)

2013-09-27 Thread Mike Spreitzer
Sorry, I was a bit too hasty in writing the last part of my last message; I forgot to qualify "software orchestration" to indicate I am speaking only of its preparatory phase. I should have written: Zane Bitter wrote on 09/27/2013 08:24:49 AM: ... > If I understood your remarks correctly, we

Re: [openstack-dev] [heat] [scheduler] Bringing things together for Icehouse (now featuring software orchestration)

2013-09-30 Thread Georgy Okrokvertskhov
Hi, I am working on the OpenStack project Murano which actually had to solve the same problem with software level orchestration. Right now Murano has a DSL language which allows you to define a workflow for a complex service deployment. Murano uses Heat for infrastructure managements and actually

Re: [openstack-dev] [heat] [scheduler] Bringing things together for Icehouse (now featuring software orchestration)

2013-09-30 Thread Clint Byrum
Excerpts from Georgy Okrokvertskhov's message of 2013-09-30 11:44:26 -0700: > Hi, > > I am working on the OpenStack project Murano which actually had to solve > the same problem with software level orchestration. Right now Murano has a > DSL language which allows you to define a workflow for a com

Re: [openstack-dev] [heat] [scheduler] Bringing things together for Icehouse (now featuring software orchestration)

2013-10-01 Thread Robert Collins
On 1 October 2013 19:31, Clint Byrum wrote: > TripleO meets at 2000 UTC every Tuesday. 1900UTC. :) -Rob -- Robert Collins Distinguished Technologist HP Converged Cloud ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.op

Re: [openstack-dev] [heat] [scheduler] Bringing things together for Icehouse (now featuring software orchestration)

2013-10-01 Thread Sylvain Bauza
Hi Mike and Zane, Le 27/09/2013 15:58, Mike Spreitzer a écrit : Zane Bitter wrote on 09/27/2013 08:24:49 AM: > Your diagrams clearly show scheduling happening in a separate stage to > (infrastructure) orchestration, which is to say that at the point where > resources are scheduled, their actua

Re: [openstack-dev] [heat] [scheduler] Bringing things together for Icehouse (now featuring software orchestration)

2013-10-01 Thread Thomas Spatzier
Clint Byrum wrote on 01.10.2013 08:31:44 - Excerpt: > From: Clint Byrum > To: openstack-dev , > Date: 01.10.2013 08:33 > Subject: Re: [openstack-dev] [heat] [scheduler] Bringing things > together for Icehouse (now featuring software orchestration) > > Excerpts from Ge

Re: [openstack-dev] [heat] [scheduler] Bringing things together for Icehouse (now featuring software orchestration)

2013-10-02 Thread Mike Spreitzer
FYI, I have refined my pictures at https://docs.google.com/drawings/d/1Y_yyIpql5_cdC8116XrBHzn6GfP_g0NHTTG_W4o0R9U and https://docs.google.com/drawings/d/1TCfNwzH_NBnx3bNz-GQQ1bRVgBpJdstpu0lH_TONw6g to hopefully make it clearer that I agree with the sentiment that holistic infrastructure sched