Re: [openstack-dev] [Murano] Need a new DSL for Murano
I think this is the great new service which will accompany OpenStack Solum similarly as Bosh is accompanying Cloud Foundry and CloudForms is accompanying OpenShift. I wouldn't call it the Application Catalog but the Service Catalog, because of the primary focus on the Service Life-cycle management (in opposite to application lifecycle management which is focused on code-to-binary, execution, remote debugging and log grabbing etc). In order to make Murano the real Service Catalog, it should support (over DSL) run-time events processing such as service scaling (up/out/in/down), healing, replication, live-migration etc. In addition, it should support a template creation which could be used by Solum similar to Heroku BuildPack. I would happy to hear your opinion on how you envision the Murano's roadmap. Thanks, Dmitry ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Murano] Need a new DSL for Murano
Hi Dmitry, I agree with you on this vision, however we have to think more on the terminology: Service Catalog in OpenStack relates to Keystone (where by Service we mean Openstack's infrastructure-level services). I understand your concerns on runtime lifecycle vs code-to-binary lifecycle though - it is absolutely valid, and we do not want to have any overlap with Solum in this matter. -- Regards, Alexander Tivelkov On Mon, Feb 24, 2014 at 1:47 PM, Dmitry mey...@gmail.com wrote: I think this is the great new service which will accompany OpenStack Solum similarly as Bosh is accompanying Cloud Foundry and CloudForms is accompanying OpenShift. I wouldn't call it the Application Catalog but the Service Catalog, because of the primary focus on the Service Life-cycle management (in opposite to application lifecycle management which is focused on code-to-binary, execution, remote debugging and log grabbing etc). In order to make Murano the real Service Catalog, it should support (over DSL) run-time events processing such as service scaling (up/out/in/down), healing, replication, live-migration etc. In addition, it should support a template creation which could be used by Solum similar to Heroku BuildPack. I would happy to hear your opinion on how you envision the Murano's roadmap. Thanks, Dmitry ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Murano] Need a new DSL for Murano
My observation has been that Murano has changed from a Windows focused Deployment Service to a Metadata Application Catalog Workflow thing (I fully admit this may be an invalid observation). It's unclear to me what OpenStack pain/use-cases is to be solved by complex object composition, description of data types, contracts... Murano is intended to provide high level definition of Applications which will include description of specific application requirements (like input parameters, external dependencies) as well as low level scripts, heat snippets and workflows which will be required to manage application. Object composition is required to address application diversity. We expect to be an integration layer for numerous different application from different OS and areas like WebServices, BigData, SAP, Enterprise specific apps. In order to decrease amount of work for Application author we will provide a way to reuse existing applications by extending them via object composition. Inside Murano we provide a library of workflows and requirements for general application classes. This library has workflows for instance creation, networking and app deployments. This will help application author to write only application specific stuff. For example if I need to publish my web service based on Tomcat I will create a new application class which has tomcat as a parent and then I will add my application specific parameters like App URL and will add deployment script which will download App war file and put it to Tomcat app directory. All other stuff will be done automatically by existing workflows for tomcat application. You can imagine this as a nested Heat template inclusion but with ability to override and\or extend it. The ability to add a workflow actually looks like a dynamic Heat resource type generation as you can specify actions and associated workflows. These actions can be triggered by external events to provide an ability of application life cycle management. Thanks Georgy On Mon, Feb 17, 2014 at 4:41 PM, Keith Bray keith.b...@rackspace.comwrote: Can someone elaborate further on the things that Murano is intended to solve within the OpenStack ecosystem? My observation has been that Murano has changed from a Windows focused Deployment Service to a Metadata Application Catalog Workflow thing (I fully admit this may be an invalid observation). It's unclear to me what OpenStack pain/use-cases is to be solved by complex object composition, description of data types, contracts... Your thoughts would be much appreciated. Thanks, -Keith From: Renat Akhmerov rakhme...@mirantis.com Reply-To: OpenStack Development Mailing List (not for usage questions) openstack-dev@lists.openstack.org Date: Monday, February 17, 2014 1:33 AM To: OpenStack Development Mailing List (not for usage questions) openstack-dev@lists.openstack.org Subject: Re: [openstack-dev] [Murano] Need a new DSL for Murano Clint, We're collaborating with Murano. We may need to do it in a way that others could see it though. There are several things here: - Murano doesn't really have a workflow engine similar to Mistral's. People get confused with that but it's just a legacy terminology, I think Murano folks were going to rename this component to be more precise about it. - Mistral DSL doesn't seem to be a good option for solving tasks that Murano is intended to solve. Specifically I mean things like complex object composition, description of data types, contracts and so on. Like Alex and Stan mentioned Murano DSL tends to grow into a full programming language . - Most likely Mistral will be used in Murano for implementation, at least we see where it would be valuable. But Mistral is not so matured yet, we need to keep working hard and be patient :) Anyway, we keep thinking on how to make both languages look similar or at least the possibility to use them seamlessly, if needed (call Mistral workflows from Murano DSL or vice versa). Renat Akhmerov @ Mirantis Inc. On 16 Feb 2014, at 05:48, Clint Byrum cl...@fewbar.com wrote: Excerpts from Alexander Tivelkov's message of 2014-02-14 18:17:10 -0800: Hi folks, Murano matures, and we are getting more and more feedback from our early adopters. The overall reception is very positive, but at the same time there are some complaints as well. By now the most significant complaint is is hard to write workflows for application deployment and maintenance. Current version of workflow definition markup really have some design drawbacks which limit its potential adoption. They are caused by the fact that it was never intended for use for Application Catalog use-cases. Just curious, is there any reason you're not collaborating on Mistral for this rather than both having a workflow engine? ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org
Re: [openstack-dev] [Murano] Need a new DSL for Murano
Can someone elaborate further on the things that Murano is intended to solve within the OpenStack ecosystem? My observation has been that Murano has changed from a Windows focused Deployment Service to a Metadata Application Catalog Workflow thing (I fully admit this may be an invalid observation). It's unclear to me what OpenStack pain/use-cases is to be solved by complex object composition, description of data types, contracts... Your thoughts would be much appreciated. Thanks, -Keith From: Renat Akhmerov rakhme...@mirantis.commailto:rakhme...@mirantis.com Reply-To: OpenStack Development Mailing List (not for usage questions) openstack-dev@lists.openstack.orgmailto:openstack-dev@lists.openstack.org Date: Monday, February 17, 2014 1:33 AM To: OpenStack Development Mailing List (not for usage questions) openstack-dev@lists.openstack.orgmailto:openstack-dev@lists.openstack.org Subject: Re: [openstack-dev] [Murano] Need a new DSL for Murano Clint, We're collaborating with Murano. We may need to do it in a way that others could see it though. There are several things here: * Murano doesn’t really have a “workflow engine” similar to Mistral’s. People get confused with that but it’s just a legacy terminology, I think Murano folks were going to rename this component to be more precise about it. * Mistral DSL doesn’t seem to be a good option for solving tasks that Murano is intended to solve. Specifically I mean things like complex object composition, description of data types, contracts and so on. Like Alex and Stan mentioned Murano DSL tends to grow into a full programming language. * Most likely Mistral will be used in Murano for implementation, at least we see where it would be valuable. But Mistral is not so matured yet, we need to keep working hard and be patient :) Anyway, we keep thinking on how to make both languages look similar or at least the possibility to use them seamlessly, if needed (call Mistral workflows from Murano DSL or vice versa). Renat Akhmerov @ Mirantis Inc. On 16 Feb 2014, at 05:48, Clint Byrum cl...@fewbar.commailto:cl...@fewbar.com wrote: Excerpts from Alexander Tivelkov's message of 2014-02-14 18:17:10 -0800: Hi folks, Murano matures, and we are getting more and more feedback from our early adopters. The overall reception is very positive, but at the same time there are some complaints as well. By now the most significant complaint is is hard to write workflows for application deployment and maintenance. Current version of workflow definition markup really have some design drawbacks which limit its potential adoption. They are caused by the fact that it was never intended for use for Application Catalog use-cases. Just curious, is there any reason you're not collaborating on Mistral for this rather than both having a workflow engine? ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.orgmailto:OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Murano] Need a new DSL for Murano
Keith, I believe it is the same place Marketplace has in AWS or OpenShift Application Gallery to Red Hat OpenShift. The idea was of having free/open app catalog that cloud admins can fill with application metadata describing those apps and then end users construct environments from those applications in user friendly UI. It is not ready recipes like Heat templates are that you only have to fill several parameters and the whole stack created for you but a constructor (think LEGO) for applications. Lets take a normal non-tech user (who cannot program/write JSON/YAML/whatever templates etc.) who wants to install Drupal on his OpenStack account. With Heat it would require finding HOT template for Drupal, answer several questions and click one button in Thermal. Then new Heat stack would be created with typically one VM for Drupal and another for MySQL. With Murano UI user finds Drupal in App Catalog and puts it onto environment. Now if he clicks deploy at this point the result would be the same as in Heat. But he also can edit what was generated. For example UI shows that Drupal requires a database that can be MySQL or PostgreSQL. Instead of vanilla MySQL user can go to App Catalog and peak Galera MySQL or PostgreSQL or something else that is compatible with those 2 DBMS. Or he can reuse database server that already exist in his environment. Drupal is hosted on web server so again user can replace Apache with Ngnix or host Drupal on some existing web server. The same goes for VM instances - user may choose to have both database and Drupal on the same VM. Now if you take all the components involved with even simple app like Drupal (Drupal, web server, PHP module for web server, database, VMs, operating systems, instance flavors) there are hundreds of combinations that can be constructed. With the Heat it would require hundreds of HOT templates (even if we forget that Heat cannot refer to resources located in another stack). And if user later decides to add load balancer to his environment to balance Drupal instances he is in trouble because load balancer brings another set of combinations (different LB implementations, underlying operating system, flavor) and things become unmanageable. So Heat is good for DevOps that can write HOT templates for exact task with all the control possible. And Murano is for the rest of us. It is something cloud providers can expose to users that do not know JSON, shell, puppet etc but want to install complex applications and still have control on it. On Tue, Feb 18, 2014 at 4:41 AM, Keith Bray keith.b...@rackspace.comwrote: Can someone elaborate further on the things that Murano is intended to solve within the OpenStack ecosystem? My observation has been that Murano has changed from a Windows focused Deployment Service to a Metadata Application Catalog Workflow thing (I fully admit this may be an invalid observation). It's unclear to me what OpenStack pain/use-cases is to be solved by complex object composition, description of data types, contracts... Your thoughts would be much appreciated. Thanks, -Keith From: Renat Akhmerov rakhme...@mirantis.com Reply-To: OpenStack Development Mailing List (not for usage questions) openstack-dev@lists.openstack.org Date: Monday, February 17, 2014 1:33 AM To: OpenStack Development Mailing List (not for usage questions) openstack-dev@lists.openstack.org Subject: Re: [openstack-dev] [Murano] Need a new DSL for Murano Clint, We're collaborating with Murano. We may need to do it in a way that others could see it though. There are several things here: - Murano doesn't really have a workflow engine similar to Mistral's. People get confused with that but it's just a legacy terminology, I think Murano folks were going to rename this component to be more precise about it. - Mistral DSL doesn't seem to be a good option for solving tasks that Murano is intended to solve. Specifically I mean things like complex object composition, description of data types, contracts and so on. Like Alex and Stan mentioned Murano DSL tends to grow into a full programming language . - Most likely Mistral will be used in Murano for implementation, at least we see where it would be valuable. But Mistral is not so matured yet, we need to keep working hard and be patient :) Anyway, we keep thinking on how to make both languages look similar or at least the possibility to use them seamlessly, if needed (call Mistral workflows from Murano DSL or vice versa). Renat Akhmerov @ Mirantis Inc. On 16 Feb 2014, at 05:48, Clint Byrum cl...@fewbar.com wrote: Excerpts from Alexander Tivelkov's message of 2014-02-14 18:17:10 -0800: Hi folks, Murano matures, and we are getting more and more feedback from our early adopters. The overall reception is very positive, but at the same time there are some complaints as well. By now the most significant complaint is is hard
Re: [openstack-dev] [Murano] Need a new DSL for Murano
Clint, We're collaborating with Murano. We may need to do it in a way that others could see it though. There are several things here: Murano doesn’t really have a “workflow engine” similar to Mistral’s. People get confused with that but it’s just a legacy terminology, I think Murano folks were going to rename this component to be more precise about it. Mistral DSL doesn’t seem to be a good option for solving tasks that Murano is intended to solve. Specifically I mean things like complex object composition, description of data types, contracts and so on. Like Alex and Stan mentioned Murano DSL tends to grow into a full programming language. Most likely Mistral will be used in Murano for implementation, at least we see where it would be valuable. But Mistral is not so matured yet, we need to keep working hard and be patient :) Anyway, we keep thinking on how to make both languages look similar or at least the possibility to use them seamlessly, if needed (call Mistral workflows from Murano DSL or vice versa). Renat Akhmerov @ Mirantis Inc. On 16 Feb 2014, at 05:48, Clint Byrum cl...@fewbar.com wrote: Excerpts from Alexander Tivelkov's message of 2014-02-14 18:17:10 -0800: Hi folks, Murano matures, and we are getting more and more feedback from our early adopters. The overall reception is very positive, but at the same time there are some complaints as well. By now the most significant complaint is is hard to write workflows for application deployment and maintenance. Current version of workflow definition markup really have some design drawbacks which limit its potential adoption. They are caused by the fact that it was never intended for use for Application Catalog use-cases. Just curious, is there any reason you're not collaborating on Mistral for this rather than both having a workflow engine? ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Murano] Need a new DSL for Murano
Hi Joshua, Thank you very much for you feedback! This is really a great question, and it was the first question we've asked ourselves when we started thinking about this new design. We've considered both options: to have our own syntax (python-like, java-like, something-else-like) or to use YAML. We've chosen the latest, and here are the reasons. The most important moment: this is not a general-purpose language, but a domain-specific one. And Murano's domain is manipulating with complex nested data-structures, which are usually represented by YAML. Murano's environments are in YAML. Murano's VM-side commands are wrapped into YAML. The primary building blocks of Murano - Heat templates - are written on YAML. Actually, at a simplified level murano's workflows can be thought of as algorithms that just generate yaml fragments. So, it would be beneficial for Murano to manipulate with YAML-constructs at the DSL level. If we use YAML notation, yaml-blocks become first-class citizens in the language, while in a regular python-like language they would be just formatted-strings. For example, look at this code snippet which generates a heat template to deploy an instance: http://paste.openstack.org/show/65725/ As you may see, the code on lines 7-11 is a Heat-template, seamlessly embedded inside Murano's workflow code. It has the variables right inside the template, and the Murano engine will substitute them with a user-specified (or workflow-computed) data Another reason for YAML: the notation is very easy to extend: you'll just have to add some new predefined key and a handler for its value: the format will not be broken, so older code will run out of the box, and even the newer code will most probably run fine on the older parser (the unknown sections will simply be ignored). This will allow us to extend the language without making any breaking-changes. The regular languages do not provide this flexibility: they will have problems if detecting unrecognised lexems or constructs. Last but not least: the perception. You are absolutely right when you say that this is actually a full programming language. However, we don't want to rush all its capabilities to unprepared developers. If some developer does not want any complexity, they may think about it as about some fancy configuration markup language: a declarative, heat-template-like header followed by a sequence of simple actions. And only if needed the power comes at your service: variable assignments, flow control, flexible data contracts, complex compositions, inheritance trees.. I can imagine a lot of scary stuff here J But at the same time, YAML's indent-based syntax will look familiar to python developers. Yes, everything comes at cost, and yaml may seem a bit bulky at the first glance. But I believe that people will get used to it soon enough, and the benefits are really important. I hope this answers your concern. We'll come up with more examples and ideas: this thing has just emerged, nothing is set in stone yet, I am actively seeking for feedback and ideas. So thanks a loot for your question, I really appreciate it. -- Regards, Alexander Tivelkov On Fri, Feb 14, 2014 at 6:41 PM, Joshua Harlow harlo...@yahoo-inc.comwrote: An honest question, U are mentioning what appears to be the basis for a full programming language (variables, calling other workflows - similar to functions) but then u mention this is being stuffed into yaml. Why? It appears like u might as well spend the effort and define a grammar and simplistic language that is not stuffed inside yaml. Shoving one into yaml syntax seems like it gets u none of the benefits of syntax checking, parsing, validation (highlighting...) and all the pain of yaml. Something doesn't seem right about the approach of creating languages inside the yaml format (in a way it becomes like xsl, yet xsl at least has a spec and is well defined). My 2 cents Sent from my really tiny device... On Feb 14, 2014, at 7:22 PM, Alexander Tivelkov ativel...@mirantis.com wrote: Hi folks, Murano matures, and we are getting more and more feedback from our early adopters. The overall reception is very positive, but at the same time there are some complaints as well. By now the most significant complaint is is hard to write workflows for application deployment and maintenance. Current version of workflow definition markup really have some design drawbacks which limit its potential adoption. They are caused by the fact that it was never intended for use for Application Catalog use-cases. I'll briefly touch these drawbacks first: 1. Murano's workflow engine is actually a state machine, however the workflow markup does not explicitly define the states and transitions. 2. There is no data isolation within any environment, which causes both potential security vulnerabilities and unpredictable workflow behaviours. 3. There is no easy way to reuse the
Re: [openstack-dev] [Murano] Need a new DSL for Murano
Just to add my 2 cents. 1. YAML is already familiar to OpenStack developers as Heat and others use it. So at least the syntax (not to mess with semantics) doesn't have to be learned. 2. YAML parser is very flexible an can be extended with additional types or constructs like Key: filename.yaml to include content of other file etc. Murano DSL may operate on already deserialized (parsed) data leaving yaml files access to hosting operation. Thus the engine itself can be independent from how and where the YAML files are stored. This is very good for App Catalog that stores its data in Glance. Also it always possible to serialize it to some other format (XML, JSON, whatever) if it required for some purpose 3. YAML declarations can be processed by Murano dashboard, 3-rd party software and various tooling for Murano. If it wasn't YAML but some handcrafted syntax then we would have to provide them with embeddable parser for that syntax 4. Implementing parser for full-blown language is not a trivial task to say the least. It would require much more time for development and greatly increase probability to shoot ourself in the foot. And we really don't want to have like a year between Murano versions. On Sat, Feb 15, 2014 at 12:18 PM, Alexander Tivelkov ativel...@mirantis.com wrote: Hi Joshua, Thank you very much for you feedback! This is really a great question, and it was the first question we've asked ourselves when we started thinking about this new design. We've considered both options: to have our own syntax (python-like, java-like, something-else-like) or to use YAML. We've chosen the latest, and here are the reasons. The most important moment: this is not a general-purpose language, but a domain-specific one. And Murano's domain is manipulating with complex nested data-structures, which are usually represented by YAML. Murano's environments are in YAML. Murano's VM-side commands are wrapped into YAML. The primary building blocks of Murano - Heat templates - are written on YAML. Actually, at a simplified level murano's workflows can be thought of as algorithms that just generate yaml fragments. So, it would be beneficial for Murano to manipulate with YAML-constructs at the DSL level. If we use YAML notation, yaml-blocks become first-class citizens in the language, while in a regular python-like language they would be just formatted-strings. For example, look at this code snippet which generates a heat template to deploy an instance: http://paste.openstack.org/show/65725/ As you may see, the code on lines 7-11 is a Heat-template, seamlessly embedded inside Murano's workflow code. It has the variables right inside the template, and the Murano engine will substitute them with a user-specified (or workflow-computed) data Another reason for YAML: the notation is very easy to extend: you'll just have to add some new predefined key and a handler for its value: the format will not be broken, so older code will run out of the box, and even the newer code will most probably run fine on the older parser (the unknown sections will simply be ignored). This will allow us to extend the language without making any breaking-changes. The regular languages do not provide this flexibility: they will have problems if detecting unrecognised lexems or constructs. Last but not least: the perception. You are absolutely right when you say that this is actually a full programming language. However, we don't want to rush all its capabilities to unprepared developers. If some developer does not want any complexity, they may think about it as about some fancy configuration markup language: a declarative, heat-template-like header followed by a sequence of simple actions. And only if needed the power comes at your service: variable assignments, flow control, flexible data contracts, complex compositions, inheritance trees.. I can imagine a lot of scary stuff here J But at the same time, YAML's indent-based syntax will look familiar to python developers. Yes, everything comes at cost, and yaml may seem a bit bulky at the first glance. But I believe that people will get used to it soon enough, and the benefits are really important. I hope this answers your concern. We'll come up with more examples and ideas: this thing has just emerged, nothing is set in stone yet, I am actively seeking for feedback and ideas. So thanks a loot for your question, I really appreciate it. -- Regards, Alexander Tivelkov On Fri, Feb 14, 2014 at 6:41 PM, Joshua Harlow harlo...@yahoo-inc.comwrote: An honest question, U are mentioning what appears to be the basis for a full programming language (variables, calling other workflows - similar to functions) but then u mention this is being stuffed into yaml. Why? It appears like u might as well spend the effort and define a grammar and simplistic language that is not stuffed inside yaml. Shoving one into yaml syntax seems like it
Re: [openstack-dev] [Murano] Need a new DSL for Murano
Excerpts from Alexander Tivelkov's message of 2014-02-14 18:17:10 -0800: Hi folks, Murano matures, and we are getting more and more feedback from our early adopters. The overall reception is very positive, but at the same time there are some complaints as well. By now the most significant complaint is is hard to write workflows for application deployment and maintenance. Current version of workflow definition markup really have some design drawbacks which limit its potential adoption. They are caused by the fact that it was never intended for use for Application Catalog use-cases. Just curious, is there any reason you're not collaborating on Mistral for this rather than both having a workflow engine? ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [Murano] Need a new DSL for Murano
Hi folks, Murano matures, and we are getting more and more feedback from our early adopters. The overall reception is very positive, but at the same time there are some complaints as well. By now the most significant complaint is is hard to write workflows for application deployment and maintenance. Current version of workflow definition markup really have some design drawbacks which limit its potential adoption. They are caused by the fact that it was never intended for use for Application Catalog use-cases. I'll briefly touch these drawbacks first: 1. Murano's workflow engine is actually a state machine, however the workflow markup does not explicitly define the states and transitions. 2. There is no data isolation within any environment, which causes both potential security vulnerabilities and unpredictable workflow behaviours. 3. There is no easy way to reuse the workflows and their related procedures between several applications. 4. The markup uses JSONPath, which relies on Python's 'eval' function. This is insecure and has to be avoided. 5. 5. The workflow markup is XML-based, which is not a common practice in the OpenStack community. So, it turns out that we have to design and implement a new workflow definition notation, which will not have any of the issues mentioned above. At the same time, it should still allow to fully specify the configuration of any third-party Application, its dependencies with other Applications and define specific actions which are required for Application deployment, configuration and life cycle management. This new notation should allow to do the following: - List all the required configuration parameters and dependencies for a given application - Validate user input and match it to the defined parameters - Define specific deployment actions and their execution order - Define behaviors to handle the events of changes in application's environment Also, it should satisfy the following requirements: - Minimize the amount of configuration for common application parts, i.e. reuse existing configuration parts and add only difference specific to the application. - Allow to use different deployment tools with using the same markup constructs. i.e. provide a high-level abstraction on the underlying tools (heat, shell, chef, puppet etc) - For security reasons it should NOT allow to execute arbitrary operations - i.e. should allow to run only predefined set of meaningful configuration actions. So, I would suggest to introduce a simple and domain specific notation which would satisfy these needs: - Application dependencies and configuration properties are defined declaratively, in a way similar to how it is done in Heat templates. - Each property has special constraints and rules, allowing to validate the input and applications relationship within the environment. - The workflows are defined in imperative way: as a sequence of actions or method calls. This may include assigning data variables or calling the workflows of other applications. - All of these may be packaged in a YAML format. The example may look something like this [1] The final version may become a bit more complicated, but as the starting point this should look fine. I suggest to cover this in more details on our next IRC meeting on Tuesday. Any feedback or suggestions are appreciated. [1] https://etherpad.openstack.org/p/murano-new-dsl-example -- Regards, Alexander Tivelkov ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Murano] Need a new DSL for Murano
An honest question, U are mentioning what appears to be the basis for a full programming language (variables, calling other workflows - similar to functions) but then u mention this is being stuffed into yaml. Why? It appears like u might as well spend the effort and define a grammar and simplistic language that is not stuffed inside yaml. Shoving one into yaml syntax seems like it gets u none of the benefits of syntax checking, parsing, validation (highlighting...) and all the pain of yaml. Something doesn't seem right about the approach of creating languages inside the yaml format (in a way it becomes like xsl, yet xsl at least has a spec and is well defined). My 2 cents Sent from my really tiny device... On Feb 14, 2014, at 7:22 PM, Alexander Tivelkov ativel...@mirantis.commailto:ativel...@mirantis.com wrote: Hi folks, Murano matures, and we are getting more and more feedback from our early adopters. The overall reception is very positive, but at the same time there are some complaints as well. By now the most significant complaint is is hard to write workflows for application deployment and maintenance. Current version of workflow definition markup really have some design drawbacks which limit its potential adoption. They are caused by the fact that it was never intended for use for Application Catalog use-cases. I'll briefly touch these drawbacks first: 1. Murano's workflow engine is actually a state machine, however the workflow markup does not explicitly define the states and transitions. 2. There is no data isolation within any environment, which causes both potential security vulnerabilities and unpredictable workflow behaviours. 3. There is no easy way to reuse the workflows and their related procedures between several applications. 4. The markup uses JSONPath, which relies on Python’s 'eval' function. This is insecure and has to be avoided. 5. 5. The workflow markup is XML-based, which is not a common practice in the OpenStack community. So, it turns out that we have to design and implement a new workflow definition notation, which will not have any of the issues mentioned above. At the same time, it should still allow to fully specify the configuration of any third-party Application, its dependencies with other Applications and define specific actions which are required for Application deployment, configuration and life cycle management. This new notation should allow to do the following: * List all the required configuration parameters and dependencies for a given application * Validate user input and match it to the defined parameters * Define specific deployment actions and their execution order * Define behaviors to handle the events of changes in application’s environment Also, it should satisfy the following requirements: * Minimize the amount of configuration for common application parts, i.e. reuse existing configuration parts and add only difference specific to the application. * Allow to use different deployment tools with using the same markup constructs. i.e. provide a high-level abstraction on the underlying tools (heat, shell, chef, puppet etc) * For security reasons it should NOT allow to execute arbitrary operations - i.e. should allow to run only predefined set of meaningful configuration actions. So, I would suggest to introduce a simple and domain specific notation which would satisfy these needs: * Application dependencies and configuration properties are defined declaratively, in a way similar to how it is done in Heat templates. * Each property has special constraints and rules, allowing to validate the input and applications relationship within the environment. * The workflows are defined in imperative way: as a sequence of actions or method calls. This may include assigning data variables or calling the workflows of other applications. * All of these may be packaged in a YAML format. The example may look something like this [1] The final version may become a bit more complicated, but as the starting point this should look fine. I suggest to cover this in more details on our next IRC meeting on Tuesday. Any feedback or suggestions are appreciated. [1] https://etherpad.openstack.org/p/murano-new-dsl-example -- Regards, Alexander Tivelkov ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.orgmailto:OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
