On Tue, 2016-01-12 at 12:52 +0100, Jiri Tomasek wrote: > On 01/11/2016 04:51 PM, Dan Prince wrote: > > Background info: > > > > We've got a problem in TripleO at the moment where many of our > > workflows can be driven by the command line only. This causes some > > problems for those trying to build a UI around the workflows in > > that > > they have to duplicate deployment logic in potentially multiple > > places. > > There are specs up for review which outline how we might solve this > > problem by building what is called TripleO API [1]. > > > > Late last year I began experimenting with an OpenStack service > > called > > Mistral which contains a generic workflow API. Mistral supports > > defining workflows in YAML and then creating, managing, and > > executing > > them via an OpenStack API. Initially the effort was focused around > > the > > idea of creating a workflow in Mistral which could supplant our > > "baremetal introspection" workflow which currently lives in python- > > tripleoclient. I create a video presentation which outlines this > > effort > > [2]. This particular workflow seemed to fit nicely within the > > Mistral > > tooling. > > > > ---- > > > > More recently I've turned my attention to what it might look like > > if we > > were to use Mistral as a replacement for the TripleO API entirely. > > This > > brings forth the question of would TripleO be better off building > > out > > its own API... or would relying on existing OpenStack APIs be a > > better > > solution? > > > > Some things I like about the Mistral solution: > > > > - The API already exists and is generic. > > > > - Mistral already supports interacting with many of the OpenStack > > API's > > we require [3]. Integration with keystone is baked in. Adding > > support > > for new clients seems straightforward (I've had no issues in adding > > support for ironic, inspector, and swift actions). > > > > - Mistral actions are pluggable. We could fairly easily wrap some > > of > > our more complex workflows (perhaps those that aren't easy to > > replicate > > with pure YAML workflows) by creating our own TripleO Mistral > > actions. > > This approach would be similar to creating a custom Heat > > resource... > > something we have avoided with Heat in TripleO but I think it is > > perhaps more reasonable with Mistral and would allow us to again > > build > > out our YAML workflows to drive things. This might allow us to > > build > > off some of the tripleo-common consolidation that is already > > underway > > ... > > > > - We could achieve a "stable API" by simply maintaining input > > parameters for workflows in a stable manner. Or perhaps workflows > > get > > versioned like a normal API would be as well. > > > > - The purist part of me likes Mistral quite a bit. It fits nicely > > with > > the deploy OpenStack with OpenStack. I sort of feel like if we have > > to > > build our own API in TripleO part of this vision has failed and > > could > > even be seen as a massive technical debt which would likely be hard > > to > > build a community around outside of TripleO. > > > > - Some of the proposed validations could perhaps be implemented as > > new > > Mistral actions as well. I'm not convinced we require TripleO API > > just > > to support a validations mechanism yet. Perhaps validations seem > > hard > > because we are simply trying to do them in the wrong places anyway? > > (like for example perhaps we should validate network connectivity > > at > > inspection time rather than during provisioning). > > > > - Power users might find a workflow built around a Mistral API more > > easy to interact with and expand upon. Perhaps this ends up being > > something that gets submitted as a patchset back to the TripleO > > that we > > accept into our upstream "stock" workflow sets. > > > > ---- > > > > Last week we landed the last patches [4] to our undercloud to > > enable > > installing Mistral by simply setting: enable_mistral = true in > > undercloud.conf. NOTE: you'll need to be using a recent trunk repo > > from > > Delorean so that you have the recently added Mistral packages for > > this > > to work. Although the feature is disable by default this should > > enable > > those wishing to tinker with Mistral as a new TripleO undercloud > > service an easy path forwards. > > > > [1] https://review.openstack.org/#/c/230432 > > [2] https://www.youtube.com/watch?v=bnAT37O-sdw > > [3] http://git.openstack.org/cgit/openstack/mistral/tree/mistral/ac > > tion > > s/openstack/mapping.json > > [4] https://etherpad.openstack.org/p/tripleo-undercloud-workflow > > > > > > ___________________________________________________________________ > > _______ > > OpenStack Development Mailing List (not for usage questions) > > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsu > > bscribe > > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > Hi, I have a few questions: > > Is Mistral action able to access/manipulate local files? E.g. access > the > templates installed at undercloud's > /usr/share/openstack-tripleo-heat-templates?
Yes. I'm actually working some prototype Mistral actions for tripleo- common which will do just this to dovetail into a deployment workflow. (I had to take care of a few heatclient things first though). > > I Mistral action able to call either OpenStack service python client > or > OpenStack service API directly? The Mistral actions themselves would be written in Python. So I think the answer here is yes... we could easily make use of existing OpenStack python clients, or go at the API's ourselves. > > What is the response from the Mistral action in the workflow? Lets > say > we'd use Mistral to get a list of available environments (we do this > in > tripleo-common now) So we call Mistral API to trigger a workflow > that > has single action which gets the list of environments. Is mistral > able > to provide this list as a response, or it is able just to trigger a > workflow? Yes, the response from individual actions is quite flexible and can really be just about anything we want so long at it is serializable I think. Probably best to look directly at the Mistral action base class for the best answer here I think though: http://git.openstack.org/cgit/openstack/mistral/tree/mistral/actions/ba se.py#n50 The action result could then be processed by YAQL in a Mistral workflow and then used in subsequent workflow items or treated as an output to the workflow itself. > > In similar manner, is Mistral able to provide a way to get workflow > in > progress data? E.g. We have a Mistral workflow for nodes > introspection. > This workflow contains several actions calling to ironic and > ironic-inspector apis. GUI calls Mistral API to trigger the workflow > and > now it needs to have a way to track the progress of that workflow. > How > would this be achieved? I've been running 'mistral execution-list' and then you can watch (poll) for the relevant execution items to finish running. Sure, polling isn't great but I'd say lets start with this perhaps. > I think forcing GUI to poll the APIs that are > called in the actions and try to implement logic that estimates what > state the workflow is to report about it is not a valid solution. We > need the workflow API (Mistral) to provide a lets say web sockets > connection and push the status of actions along with relevant data, > so > GUI can listen to those. I don't think Mistral has a websockets implementation. I think Zaqar does though, and I think perhaps one way we could go about this sort of notification might be to integrate our workflows with a Zaqar queue or something. GUI would listen to a Zaqar queue for example... and the workflow (as it executes) would post things to a specific queue. Perhaps this is opt-in, only if a Zaqar queue is provided to a given workflow. FWIW, integrating Mistral w/ Zaqar actions would likely be quite easy. Alternately we could look at what it would take to add websocket support to Mistral directly. > > I am about to implement your nodes workflow in the GUI to test how it > works. > > Jirka > > _____________________________________________________________________ > _____ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubs > cribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev