----- Original Message ----- > On Wed, 2016-01-13 at 04:41 -0500, Tzu-Mainn Chen wrote: > > Hey all, > > > > I realize now from the title of the other TripleO/Mistral thread [1] > > that > > the discussion there may have gotten confused. I think using Mistral > > for > > TripleO processes that are obviously workflows - stack deployment, > > node > > registration - makes perfect sense. That thread is exploring > > practicalities > > for doing that, and I think that's great work. > > > > What I inappropriately started to address in that thread was a > > somewhat > > orthogonal point that Dan asked in his original email, namely: > > > > "what it might look like if we were to use Mistral as a replacement > > for the > > TripleO API entirely" > > > > I'd like to create this thread to talk about that; more of a 'should > > we' > > than 'can we'. And to do that, I want to indulge in a thought > > exercise > > stemming from an IRC discussion with Dan and others. All, please > > correct me > > if I've misstated anything. > > > > The IRC discussion revolved around one use case: deploying a Heat > > stack > > directly from a Swift container. With an updated patch, the Heat CLI > > can > > support this functionality natively. Then we don't need a TripleO > > API; we > > can use Mistral to access that functionality, and we're done, with no > > need > > for additional code within TripleO. And, as I understand it, that's > > the > > true motivation for using Mistral instead of a TripleO API: avoiding > > custom > > code within TripleO. > > The True motivation for investigating Mistral was to counter the > assertion that we needed to build our own REST API for workflows in the > TripleO API spec. This: > > "We need a REST API that supports the overcloud deployment workflow." > > https://review.openstack.org/#/c/230432/13/specs/mitaka/tripleo-overclo > ud-deployment-api.rst > > In doing that I was trying to wrap some of the existing code in > tripleo-common in Mistral actions and it occurred to me that some > things (like the ability to deploy heat templates from Swift) might > benefit others more if we put them in the respective client libraries > (like heatclient) instead of carrying them in tripleo-common where they > only benefit us. Especially since in the heatclient case it already had > a --template-object option to begin with (https://bugs.launchpad.net/py > thon-heatclient/+bug/1532326) > > This goes towards a similar pattern we follow with other components > like Puppet which is that instead of trying to create our own > functionality in puppet-tripleo we are trying to as much as possible > add the features we need to the individual modules for each project > (puppet-nova, puppet-swift, etc). > > In other words avoiding custom code within TripleO is just good > practice in general, not something that is specific to the Mistral vs > TripleO API discussion. When someone looks at TripleO as a project I > would very much like them to admire our architecture... not what we've > had to build to support it. As an example I'm very glad to see TripleO > be a bit more friendly to config management tooling... rather than > trying to build our own version (os-apply-config, etc.). And adding > code to the right place usually works out better for everyone and can > help build up the larger OpenStack community too. > > > > > That's definitely a worthy goal... except from my perspective, the > > story > > doesn't quite end there. A GUI needs additional functionality, which > > boils > > down to: understanding the Heat deployment templates in order to > > provide > > options for a user; and persisting those options within a Heat > > environment > > file. > > TripleO API was previously described as what would be our "workflows > API" for TripleO. A places for common workflows that are used for the > CLI, and UI together (one code path for all!). > > What you describe here sounds like a different sort of idea which is a > GUI helper sort of API. FWIW I think it is totally fine for a GUI to > maintain its own API for caching reasons etc if the members of that > team find it to be a requirement. Please don't feel that any of the > workflow API comparisons block requirements for things to build a GUI > though. > > > > > > Right away I think we hit a problem. Where does the code for > > 'understanding > > options' go? Much of that understanding comes from the capabilities > > map > > in tripleo-heat-templates [2]; it would make sense to me that > > responsibility > > for that would fall to a TripleO library. > > > > Still, perhaps we can limit the amount of TripleO code. So to give > > API > > access to 'getDeploymentOptions', we can create a Mistral workflow. > > > > Retrieve Heat templates from Swift -> Parse capabilities map > > I have no issues with this workflow if it helps the UI team to create > say a cached copy of the capabilities map. > > Similar to the above heatclient discussion I would ask why put this > into a TripleO library if we don't have to. Could this code perhaps be > made into a more generic Heat feature? If not immediately, then perhaps > eventually? What I mean is... it would be totally fine to write this > code in say TripleO common with an eye towards moving it eventually to > someplace better like Heat API (proper) or heatclient perhaps. > > Regardless of where the code for parsing the capabilities map lives > though I think it would be a fine use of a Mistral workflow. > > > > > Which is fine-ish, except from an architectural perspective > > 'getDeploymentOptions' violates the abstraction layer between storage > > and > > business logic, a problem that is compounded because > > 'getDeploymentOptions' > > is not the only functionality that accesses the Heat templates and > > needs > > exposure through an API. And, as has been discussed on a separate > > TripleO > > thread, we're not even sure Swift is sufficient for our needs; one > > possible > > consideration right now is allowing deployment from templates stored > > in > > multiple places, such as the file system or git. > > > > Are we going to have duplicate 'getDeploymentOptions' workflows for > > each > > storage mechanism? If we consolidate the storage code within a > > TripleO > > library, do we really need a *workflow* to call a single > > function? Is a > > thin TripleO API that contains no additional business logic really so > > bad > > at that point? > > Do we really need our TripleO libaries to support multiple storage > backends at all right now. As a cloud deployer I can store my heat > templates wherever I like... git, svn, cvs... whatever. When I'm ready > to deploy I would then use python-tripleoclient or the UI to "upload" > my local copy of the templates into Swift. At this point I could > either use the UI or python-tripleoclient to drive the rest of the > deployment to completion. The fact that the TripleO workflows use Swift > for storage is actually abstracted from me such that I don't really > care much. > > > > > My gut reaction is to say that proposing Mistral in place of a > > TripleO API > > is to look at the engineering concerns from the wrong direction. The > > Mistral alternative comes from a desire to limit custom TripleO code > > at all > > costs. I think that is an extremely dangerous attitude that leads to > > compromises and workarounds that will quickly lead to a shaky code > > base > > full of design flaws that make it difficult to implement or extend > > any > > functionality cleanly. > > Similar to what I said above limiting code in TripleO isn't necessarily > the primary desire here I think. It is more of a question of can we use > a generic workflow API like Mistral, or should we go and build our own. > > Whether more or less of the code lives in a TripleO common library is > something we can debate and take a step at a time I think. > > > > > I think the correct attitude is to simply look at the problem we're > > trying to solve and find the correct architecture. For these get/set > > methods that the API needs, it's pretty simple: storage -> some logic > > -> > > a REST API. Adding a workflow engine on top of that is unneeded, and > > I > > believe that means it's an incorrect solution. > > Totally agree on finding the correct architecture. What you suggest as > a "dangerous attitude" is actually just trying to ask the question that > to me have not yet been asked. Trying out tools and solutions we may > not have tried yet. (FWIW I found the dangerous attitude bit to be a > bit un-constructive to this conversation and would prefer to leave that > verbiage elsewhere) > > If you feel strongly that this particular case of parsing the > capabilities map and managing config settings is not suitably > implemented with a generic workflow tool then perhaps we implement both > and see how it works out. Continue work on creating TripleO API... > > I would however like to perhaps see us scale back selling TripleO API > as our generic workflow API. Perhaps there still is a niche where it > helps with some things and I've got no issue with pursuing this idea. I > will likely continue to ask the questions about putting code into the > right place (like could some of this be implemented as a Heat API > feature instead). Perhaps this just comes in time after we've refined > our deployment workflows a bit to accomidate but UI and CLI. >
That seems more than fair (and I apologize for the 'dangerous attitude' comment, which I did not intend to be construed as an attack). I think I'll give the API spec one more polish and then re-submit, with the hope that the discussion here brings us much closer to consensus as to the path forward. Thanks! Mainn > > Dan > > > > > > > Thanks, > > Tzu-Mainn Chen > > > > > > > > [1] http://lists.openstack.org/pipermail/openstack-dev/2016-January/0 > > 83757.html > > [2] https://github.com/openstack/tripleo-heat-templates/blob/master/c > > apabilities_map.yaml > > > > _____________________________________________________________________ > > _____ > > OpenStack Development Mailing List (not for usage questions) > > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubs > > cribe > > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > __________________________________________________________________________ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > __________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev