Angus Salkeld <[email protected]> wrote on 12.11.2013 23:05:57: > From: Angus Salkeld <[email protected]> > To: [email protected], > Date: 12.11.2013 23:09 > Subject: Re: [openstack-dev] [Mistral] really simple workflow for > Heat configuration tasks > > On 12/11/13 13:04 +0100, Thomas Spatzier wrote: > >Hi Angus, > > > >that is an interesting idea. Since you mentioned the software config > >proposal in the beginning as a related item, I guess you are trying to > >solve some software config related issues with Mistral. So a few questions, > >looking at this purely from a software config perspect: > > > >Are you thinking about doing the infrastructure orchestration (VMs, > >volumes, network etc) with Heat's current capabilities and then let the > >complete software orchestration be handled by Mistral tasks? I.e. bootstrap > >the workers on each VM and have the definition of when which agent does > >something defined in a flow? > > Well we either add an api to heat to do "install_config" or we use > a service that is designed to do tasks. Clint convienced me quite > easily that "install/apply_config" is just a task. > > > > >If yes, is there a way for passing data around - e.g. output produced by > >one software config step is input for another software config step? > > > >Again, if my above assumption is true, couldn't there be problems when we > >having two ways of doing orchestration, when the software layer thing would > >take the Heat engine out of some processing and take away some control? Or > >are you thinking about using Mistral as a general mechanism for task > >execution in Heat, which would then probably resolve the conflict? > > > At this point we really do not need a "flow", just a task concept > from Mistral. Prehaps ways of grouping them and targeting them > for a particular server. > > I'd see the config_deployer resource posting a task to Mistral > and we have an agent in the server that can consume tasks and > pass them to sub-agents that understand the particular format.
Ok, makes sense to me. And I don't see a conflict with the software config proposal, but this is one of the "implementation details" we said need to be figured out :-) > > If we do this then Heat is in charge of the orchestration and > there are not two workflows fighting for control. I do agree > that there should just be one. > > I think once Mistral is more mature we can decide whether to pass > full workflow control over to it, but for now the task functionality > is all we need. (and a time based one would be neat too btw). > > -Angus > > >Regards, > >Thomas > > > >Angus Salkeld <[email protected]> wrote on 12.11.2013 02:15:15: > >> From: Angus Salkeld <[email protected]> > >> To: [email protected], > >> Date: 12.11.2013 02:18 > >> Subject: [openstack-dev] [Mistral] really simple workflow for Heat > >> configuration tasks > >> > >> Hi all > >> > >> I think some of you were at the Software Config session at summit, > >> but I'll link the ideas that were discussed: > >> https://wiki.openstack.org/wiki/Heat/Blueprints/hot-software-config-WIP > >> https://wiki.openstack.org/wiki/Heat/Blueprints/hot-software-config > >> > >> To me the basics of it are: > >> 1. we need an entity/resource to place the configuration (in Heat) > >> 2. we need a resource to install the configuration > >> (basically a task in Mistral) > >> > >> > >> A big issue to me is the conflict between heat's taskflow and the new > >> external one. What I mean by conflict is that it will become tricky > >> to manage two parallel taskflow instances in one stack. > >> > >> This could be solved by: > >> 1: totally using mistral (only use mistral workflow) > >> 2: use a very simple model of just asking mistral to run tasks (no > >> workflow) this allows us to use heat's workflow > >> but mistral's task runner. > >> > >> Given that mistral has no real implementation yet, "2" would seem > >> reasonable to me. (I think Heat developers are open to "1" when > >> Mistral is more mature.) > >> > >> How could we use Mistral for config installation? > >> ------------------------------------------------- > >> 1. We have a resource type in Heat that creates tasks in a Mistral > >> workflow (manual workflow). > >> 2. Heat pre-configures the server to have a Mistral worker > >> installed. > >> 3. the Mistral worker pulls tasks from the workflow and passes them > >> to an agent that can run it. (the normal security issues jump up > >> here - giving access to the taskflow from a guest). > >> > >> To do this we need an api that can add tasks to a workflow dynamically. > >> like this: > >> - create a simple workflow > >> - create and run task A [run on server X] > >> - create and run task B [run on server Y] > >> - create and run task C [run on server X] > >> > >> (note: the task is run and completes before the next is added if there > >> is a dependancy, if tasks can be run in parallel then we add > >multiple > >> tasks) > >> > >> The api could be something like: > >> CRUD <mistral>/workflows/ > >> CRUD <mistral>/workflows/<wf>/tasks > >> > >> > >> One thing that I am not sure of is how a server(worker) would know if a > >task > >> was for it or not. > >> - perhaps we have a capability property of the task that we can use > >> (capablitiy[server] = <server-id>) or actually specify the worker we > >> want. > >> > >> I think this would be a good starting point for Mistral as it is a > >> very simple but concrete starting point. Also if this is not done in > >> Mistral we will have to add this in Heat (lets rather have it where > >> it should be). This will also give us a chance to have confidence > >> with Mistral before trying to do more complex workflows. > >> > >> If you (Heat and Mistral developers) are open to this we can discuss > >> what needs to be done. I am willing to help with implementation. > >> > >> Thanks > >> -Angus > >> > >> > >> > >> > >> _______________________________________________ > >> OpenStack-dev mailing list > >> [email protected] > >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > >> > > > > > >_______________________________________________ > >OpenStack-dev mailing list > >[email protected] > >http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > _______________________________________________ > OpenStack-dev mailing list > [email protected] > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > _______________________________________________ OpenStack-dev mailing list [email protected] http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
