Hi Mistral folks,
Congrats on getting the 0.0.2 release out. I had a look at Renat's
screencast and the examples, and I wanted to share some feedback based
on my experience with Heat. Y'all will have to judge for yourselves to
what extent this experience is applicable to Mistral. (Assume that
everything I know about it was covered in the screencast and you won't
be far wrong.)
The first thing that struck me looking at
https://github.com/stackforge/mistral-extra/tree/master/examples/create_vm
is that I have to teach Mistral how to talk to Nova. I can't overstate
how surprising this is as a user, because Mistral is supposed to want to
become a part of OpenStack. It should know how to talk to Nova! There is
actually an existing DSL for interacting with OpenStack[1], and here's
what the equivalent operation looks like:
os server create $server_name --image $image_id --flavor $flavor_id
--nic net-id=$network_id
Note that this is approximately exactly 96.875% shorter (or 3200%
shorter, if you're in advertising).
This approach reminds me a bit of TOSCA, in the way that it requires you
to define every node type before you use it. (Even TOSCA is moving away
from this by developing a Simple Profile that includes the most common
ones in the box - an approach I assume/hope you're considering also.)
The stated reason for this is that they want TOSCA templates to run on
any cloud regardless of its underlying features (rather than take a
lowest-common-denominator approach, as other attempts at hybrid clouds
have done). Contrast that with Heat, which is unapologetically an
orchestration system *for OpenStack*.
I note from the screencast that Mistral's stated mission is to:
Provide a mechanism to define and execute
tasks and workflows *in OpenStack clouds*
(My emphasis.) IMO the design doesn't reflect the mission. You need to
decide whether you are trying to build the OpenStack workflow DSL or the
workflow DSL to end all workflow DSLs.
That problem could be solved by including built-in definitions for core
OpenStack service in a similar way to std.* (i.e. take the TOSCA Simple
Profile approach), but I'm actually not sure that goes far enough. The
lesson of Heat is that we do best when we orchestrate *only* OpenStack APIs.
For example, when we started working on Heat, there was no autoscaling
in OpenStack so we implemented it ourselves inside Heat. Two years
later, there's still no autoscaling in OpenStack other than what we
implemented, and we've been struggling for a year to try to split Heat's
implementation out into a separate API so that everyone can use it.
Looking at things like std.email, I feel a similar way about them.
OpenStack is missing something equivalent to SNS, where a message on a
queue can trigger an email or another type of notification, and a lot of
projects are going to eventually need something like that. It would be
really unfortunate if all of them went out and invented it
independently. It's much better to implement such things as their own
building blocks that can be combined together in complex ways rather
than adding that complexity to a bunch of services.
Such a notification service could even be extended to do std.http-like
ReST calls, although personally the whole idea of OpenStack services
calling out to arbitrary HTTP APIs makes me extremely uncomfortable.
Much better IMO to just post messages to queues and let the receiver
(long) poll for it.
So I would favour a DSL that is *much* simpler, and replaces all of
std.* with functions that call OpenStack APIs, and only OpenStack APIs,
including the API for posting messages to Marconi queues, which would be
the method of communication to the outside world. (If the latter part
sounds a bit like SWF, it's for a good reason, but the fact that it
would allow access directly to all of the OpenStack APIs before
resorting to an SDK makes it much more powerful, as well as providing a
solid justification for why this should be part of OpenStack.)
The ideal way to get support for all of the possible OpenStack APIs
would be to do it by introspection on python-openstackclient. That means
you'd only have to do the work once and it will stay up to date. This
would avoid the problem we have in Heat, where we have to implement each
resource type separately. (This is the source of a great deal of Heat's
value to users - the existence of tested resource plugins - but also the
thing that stops us from iterating the code quicker.)
I'm also unsure that it's a good idea for things like timers to be set
up inside the DSL. I would prefer that the DSL just define workflows and
export entry points to them. Then have various ways to trigger them:
from the API manually, from a message to a Marconi queue, from a timer,
&c. The latter two you'd set up through the Mistral API. If a user
wanted a single document that set up one or more workflows and their
triggers, a Heat template would do that job.
I can see that your goal is to make a system that works with any
existing application without changes. I think this is not so important
as you think; the lesson of AWS is that developers will happily write
their applications to use your service if you make it simple enough for
them to understand. In a year's time will anybody think twice about
spinning up a container to poll a message queue and proxy it into ReST
calls, if that's what they need to do to interface to some
legacy/outside code?
As I said at the beginning, you are the experts and you'll have to
decide for yourselves how much of this feedback is relevant to Mistral.
You certainly know a bunch of things that I don't about
workflow-as-a-service. (I am, of course, interested in being
re-educated!) But I hope that some of our experience on the Heat project
might be helpful to you.
cheers,
Zane.
[1] http://docs.openstack.org/developer/python-openstackclient/
_______________________________________________
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
