I fall very much in the same mentality as Ben. I'm +1 to all of his
points, with a few comments inline.
On 01/22/2016 12:24 PM, Ben Nemec wrote:
So I haven't weighed in on this yet, in part because I was on vacation
when it was first proposed and missed a lot of the initial discussion,
and also because I wanted to take some time to order my thoughts on it.
Also because my initial reaction...was not conducive to calm and
rational discussion. ;-)
The tldr is that I don't like it. To explain why, I'm going to make a
list (everyone loves lists, right? Top $NUMBER reasons we should stop
expecting other people to write our API for us):
1) We've been down this road before. Except last time it was with Heat.
I'm being somewhat tongue-in-cheek here, but expecting a general
service to provide us a user-friendly API for our specific use case just
doesn't make sense to me.
I think it's important to think about outside integrations here. The
current model is to tell other clients to manipulate Heat environments
and understand how to parse/inspect templates*. Now it will be to
understand/parse/manipulate Mistral workflows. Neither of those are
conducive to the types of UI wireframes we've proposed in the past, much
less friendly to completely outside integrators.
* I realize some of that inspection is moving into Heat, but it's still
at the mechanical template level rather than providing insight into how
to actually use them.
2) The TripleO API is not a workflow API. I also largely missed this
discussion, but the TripleO API is a _Deployment_ API. In some cases
there also happens to be a workflow going on behind the scenes, but
honestly that's not something I want our users to have to care about.
I'm glad Ben mentioned this, because I always viewed the workflow
aspects as a subset of what actually needs to be done.
3) It ties us 100% to a given implementation. If Mistral proves to be a
poor choice for some reason, or insufficient for a particular use case,
we have no alternative. If we have an API and decide to change our
implementation, nobody has to know or care. This is kind of the whole
point of having an API - it shields users from all the nasty
implementation details under the surface.
I strongly agree with this one. It's not even generic speculation; we've
told people in the past to deal with Heat templates and now we're
telling them to deal with workflows. We already have a history of the
backend changing and an API would give us much more flexibility (and
less annoyed users).
4) It raises the bar even further for both new deployers and developers.
You already need to have a pretty firm grasp of Puppet and Heat
templates to understand how our stuff works, not to mention a decent
understanding of quite a number of OpenStack services.
This presents a big chicken and egg problem for people new to OpenStack.
It's great that we're based on OpenStack and that allows people to peek
under the hood and do some tinkering, but it can't be required for
everyone. A lot of our deployers are going to have little to no
OpenStack experience, and TripleO is already a daunting task for those
people (hell, it's daunting for people who _are_ experienced).
5) What does reimplementing all of our tested, well-understood Python
into a new YAML format gain us? This is maybe the biggest thing I'm
missing from this whole discussion. We lose a bunch of things (ease of
transition from other Python projects, excellent existing testing
framework, etc.), but what are we actually gaining other than the
ability to say that we use N + 1 OpenStack services? Because we're way
past the point where "It's OpenStack deploying OpenStack" is sufficient
reason for people to pay attention to us. We need less "Ooh, neat" and
more "Ooh, that's easy to use and works well." It's still not clear to
me that Mistral helps in any way with the latter.
6) On the testing note, how do we test these workflows? Do we know what
happens when step X fails? How do we test that they handle it properly
in an automated and repeatable way? In Python these are largely easy
questions to answer: unit tests. How do you unit test YAML? This is a
big reason I'm not even crazy about having Mistral on the back end of a
TripleO API. We'd be going from code that we can test and prove works
in a variety of scenarios, to YAML that is tested and proven to work in
exactly the three scenarios we run in CI. This is basically the same
situation we had with tripleo-incubator, and it was bad there too.
I dunno. Maybe I'm too late to this party to have any impact on the
discussion, but I very much do not like the direction we're going and I
would be remiss if I didn't at least point out my concerns with it.
-Ben
On 01/13/2016 03:41 AM, Tzu-Mainn Chen wrote:
Hey all,
I realize now from the title of the other TripleO/Mistral thread [1] that
the discussion there may have gotten confused. I think using Mistral for
TripleO processes that are obviously workflows - stack deployment, node
registration - makes perfect sense. That thread is exploring practicalities
for doing that, and I think that's great work.
What I inappropriately started to address in that thread was a somewhat
orthogonal point that Dan asked in his original email, namely:
"what it might look like if we were to use Mistral as a replacement for the
TripleO API entirely"
I'd like to create this thread to talk about that; more of a 'should we'
than 'can we'. And to do that, I want to indulge in a thought exercise
stemming from an IRC discussion with Dan and others. All, please correct me
if I've misstated anything.
The IRC discussion revolved around one use case: deploying a Heat stack
directly from a Swift container. With an updated patch, the Heat CLI can
support this functionality natively. Then we don't need a TripleO API; we
can use Mistral to access that functionality, and we're done, with no need
for additional code within TripleO. And, as I understand it, that's the
true motivation for using Mistral instead of a TripleO API: avoiding custom
code within TripleO.
That's definitely a worthy goal... except from my perspective, the story
doesn't quite end there. A GUI needs additional functionality, which boils
down to: understanding the Heat deployment templates in order to provide
options for a user; and persisting those options within a Heat environment
file.
Right away I think we hit a problem. Where does the code for 'understanding
options' go? Much of that understanding comes from the capabilities map
in tripleo-heat-templates [2]; it would make sense to me that responsibility
for that would fall to a TripleO library.
Still, perhaps we can limit the amount of TripleO code. So to give API
access to 'getDeploymentOptions', we can create a Mistral workflow.
Retrieve Heat templates from Swift -> Parse capabilities map
Which is fine-ish, except from an architectural perspective
'getDeploymentOptions' violates the abstraction layer between storage and
business logic, a problem that is compounded because 'getDeploymentOptions'
is not the only functionality that accesses the Heat templates and needs
exposure through an API. And, as has been discussed on a separate TripleO
thread, we're not even sure Swift is sufficient for our needs; one possible
consideration right now is allowing deployment from templates stored in
multiple places, such as the file system or git.
Are we going to have duplicate 'getDeploymentOptions' workflows for each
storage mechanism? If we consolidate the storage code within a TripleO
library, do we really need a *workflow* to call a single function? Is a
thin TripleO API that contains no additional business logic really so bad
at that point?
My gut reaction is to say that proposing Mistral in place of a TripleO API
is to look at the engineering concerns from the wrong direction. The
Mistral alternative comes from a desire to limit custom TripleO code at all
costs. I think that is an extremely dangerous attitude that leads to
compromises and workarounds that will quickly lead to a shaky code base
full of design flaws that make it difficult to implement or extend any
functionality cleanly.
I think the correct attitude is to simply look at the problem we're
trying to solve and find the correct architecture. For these get/set
methods that the API needs, it's pretty simple: storage -> some logic ->
a REST API. Adding a workflow engine on top of that is unneeded, and I
believe that means it's an incorrect solution.
Thanks,
Tzu-Mainn Chen
[1] http://lists.openstack.org/pipermail/openstack-dev/2016-January/083757.html
[2]
https://github.com/openstack/tripleo-heat-templates/blob/master/capabilities_map.yaml
__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
