[openstack-dev] [Mistral] Global Context and Execution Environment

[W Chan]

Nikolay,

Regarding whether the execution environment BP is the same as this global
context BP, I think the difference is in the scope of the variables.  The
global context that I'm proposing is provided to the workflow at execution
and is only relevant to this execution.  For example, some contextual
information about this specific workflow execution (i.e. a reference to a
record in an external system related such as a service ticket ID or CMDB
record ID).  The values do not necessary carry across multiple executions.
But as I understand, the execution environment configuration is a set of
reusable configuration that can be shared across multiple workflow
executions.  The fact where action parameters are specified explicitly over
and over again is a common problem in the DSL.

Winson
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Mistral] Global Context and Execution Environment

[Renat Akhmerov]

Winson, ok, I got the idea.

Just a crazy idea that came to my mind. What if we just mark some of the input 
parameters as “global”? For example,

wf:
  type: direct
  input:
- param1
- param2: global

One way or another we’re talking about different scopes. I see the following 
possible scopes:

* local - default scope, only current workflow tasks can see it
* global - all entities can see it: this workflow itself (its tasks), its 
nested workflows and actions
* workflow - only this workflow and actions called from this workflow can see it

However, if we follow that path we would need to change how Mistral validates 
workflow input parameters. Currently, if we pass something into workflow it 
must be declared as an input parameter. In case of “global” scope and nested 
workflows this mechanism is too primitive because a nested workflow may get 
something that it doesn’t expect. So it may not be that straightforward.

Thoughts?

Just in case, I’ll repeat related BPs from another thread:

* https://blueprints.launchpad.net/mistral/+spec/mistral-execution-environment 

* https://blueprints.launchpad.net/mistral/+spec/mistral-global-context 

* https://blueprints.launchpad.net/mistral/+spec/mistral-workflow-constants 

* https://blueprints.launchpad.net/mistral/+spec/mistral-default-input-values 


Renat Akhmerov
@ Mirantis Inc.



> On 10 Dec 2014, at 13:12, W Chan  wrote:
> 
> Nikolay,
> 
> Regarding whether the execution environment BP is the same as this global 
> context BP, I think the difference is in the scope of the variables.  The 
> global context that I'm proposing is provided to the workflow at execution 
> and is only relevant to this execution.  For example, some contextual 
> information about this specific workflow execution (i.e. a reference to a 
> record in an external system related such as a service ticket ID or CMDB 
> record ID).  The values do not necessary carry across multiple executions.  
> But as I understand, the execution environment configuration is a set of 
> reusable configuration that can be shared across multiple workflow 
> executions.  The fact where action parameters are specified explicitly over 
> and over again is a common problem in the DSL.
> 
> Winson
> 
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Mistral] Global Context and Execution Environment

[W Chan]

Renat, Dmitri,

On supplying the global context into the workflow execution...

In addition to Renat's proposal, I have a few here.

1) Pass them implicitly in start_workflow as another kwargs in the
**params.  But on thinking, we should probably make global context
explicitly defined in the WF spec.  Passing them implicitly may be hard to
follow during troubleshooting where the value comes from by looking at the
WF spec.  Plus there will be cases where WF authors want it explicitly
defined. Still debating here...

inputs = {...}
globals = {...}
start_workflow('my_workflow', inputs, globals=globals)

2) Instead of adding to the WF spec, what if we change the scope in
existing input params?  For example, inputs defined in the top workflow by
default is visible to all subflows (pass down to workflow task on
run_workflow) and tasks (passed to action on execution).

3) Add to the WF spec

workflow:
type: direct
global:
- global1
- global2
input:
- input1
- input2

Winson
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Mistral] Global Context and Execution Environment

[Dmitri Zimine]

Winson, Lakshmi, Renat: 

It looked good and I began to write down the summary: 
https://etherpad.openstack.org/p/mistral-global-context

But than realized that it’s not safe to assume from the action, that the global 
context will be supplied as part of API call. 
Check it out in the etherpad.

What problems are we trying to solve: 
1) reduce passing the same parameters over and over from parent to child
2) “automatically” make a parameter accessible to most actions without typing 
it all over (like auth token) 

Can #1 be solved by passing input to subworkflows automatically
Can #2 be solved somehow else? Default passing of arbitrary parameters to 
action seems like breaking abstraction.

Thoughts? need to brainstorm further….

DZ> 


On Dec 12, 2014, at 12:54 PM, W Chan  wrote:

> Renat, Dmitri,
> 
> On supplying the global context into the workflow execution...
> 
> In addition to Renat's proposal, I have a few here.
> 
> 1) Pass them implicitly in start_workflow as another kwargs in the **params.  
> But on thinking, we should probably make global context explicitly defined in 
> the WF spec.  Passing them implicitly may be hard to follow during 
> troubleshooting where the value comes from by looking at the WF spec.  Plus 
> there will be cases where WF authors want it explicitly defined. Still 
> debating here...
> 
> inputs = {...}
> globals = {...}
> start_workflow('my_workflow', inputs, globals=globals)
> 
> 2) Instead of adding to the WF spec, what if we change the scope in existing 
> input params?  For example, inputs defined in the top workflow by default is 
> visible to all subflows (pass down to workflow task on run_workflow) and 
> tasks (passed to action on execution).
> 
> 3) Add to the WF spec
> 
> workflow:
> type: direct
> global:
> - global1
> - global2
> input:
> - input1
> - input2
> 
> Winson
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Mistral] Global Context and Execution Environment

[Renat Akhmerov]

Hi,

> It looked good and I began to write down the summary: 
> https://etherpad.openstack.org/p/mistral-global-context 
> 
Thanks, I left my comments in there.

> What problems are we trying to solve: 
> 1) reduce passing the same parameters over and over from parent to child
> 2) “automatically” make a parameter accessible to most actions without typing 
> it all over (like auth token) 

I agree that it’s one of the angle from what we’re looking at the problem. 
However, IMO, it’s wider than just these two points. My perspective is that we 
are, first of all, discussing workflow variables’ scoping (see my previous 
email in this thread). So I would rather focus on that. Let’s list all the 
scopes that would make sense, their semantics and use cases where each of them 
could solve particular usability problems (I’m saying “usability problems” 
because it’s really all about usability only).

The reason I’m trying to discuss all this from this point of view is because I 
think we should try to be more formal on things like that. 

> Can #1 be solved by passing input to subworkflows automatically

No, it can’t. “input” is something that gets validated upon workflow execution 
(which happens now) and can’t be arbitrarily passed all the way through because 
of that. If we introduce something like “global” scope then we can always pass 
variables of this scope down to nested workflows using a separate mechanism 
(i.e. different parameter of start_workflow() method). 

> Can #2 be solved somehow else? Default passing of arbitrary parameters to 
> action seems like breaking abstraction

Yes, unless explicitly specified I would not give actions more than they need. 
Encapsulation has been proven to be a good thing.

> Thoughts? need to brainstorm further….

Just once again, I appeal to talk about scopes, their semantics and use cases 
purely from workflow language (DSL) and API standpoint because I’m afraid 
otherwise we could bury ourselves under a pile of minor technical details. 
Specification first, implementation second.

Thanks

Renat Akhmerov
@ Mirantis Inc.


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev