On 11/10/13 14:25 -0400, Lakshminaraya Renganarayana wrote:
Excellent discussion on various issues around orchestration and
coordination -- thanks to you all, in particular to Clint, Angus, Stan,
Thomas, Joshua, Zane, Steve ...
After reading the discussions, I am finding the following themes emerging
(please feel free to correct/add):
1. Most of the building blocks needed for effective coordination and
orchestration are already in Heat/HOT.
2. Heat would like to view software configuration as a resource (type) with
related providers + plugins
3. There is scope for communication/synchronization mechanisms that would
complement the wait-conditions and signals
I would like to propose a simple abstraction that would complement the
current wait-conditions and signals. My proposal is based our experience
with supporting such an abstraction on our DSL and also on an extension of
Heat. In a nut-shell, this abstraction is a global data space (visible
across resources, stacks) from which resources can read and write their
inputs / outputs PLUS the semantics that reads will block until the read
values are available and writes are non-blocking. We used ZooKeeper to
implement this global data space and the blocking-read/non-blocking-writes
semantics. But, these could be implemented using several other mechanisms
and I believe the techniques currently used by Heat for meta-data service
can be used here.
I would like to make clear that I am not proposing a replacement for
wait-conditions and signals. I am hoping that wait-conditions and signals
would be used by power-users (concurrent/distributed programming experts)
and the proposed abstraction would be used by folks (like me) who do not
want to reason about concurrency and related problems. Also, the proposed
global data-space with blocking reads and non-blocking writes is not a new
idea (google tuple-spaces, linda) and it has been proven in other domains
such as coordination languages to improve the level of abstraction and
productivity.
The benefits of the proposed abstraction are:
G1. Support finer granularity of dependences
G2. Allow Heat to reason/analyze about these dependences so that it can
order resource creations/management
G3. Avoid classic synchronization problems such as dead-locks and race
conditions
G4 *Conjecture* : Capture most of the coordination use cases (including
those required for software configuration / orchestration).
Here is more detailed description: Let us say that we can use either
pre-defined or custom resource types to define resources at arbitrary
levels of granularity. This can be easily supported and I guess is already
possible in current version of Heat/HOT. Given this, the proposed
abstraction has two parts: (1) an interface style specification a
resource's inputs and outputs and (2) a global name/data space. The
interface specification which would capture
We do specify the resources inputs (properties) and outputs
(attributes). I am not sure what is new here?
- INPUTS: all the attributes that are consumed/used/read by that resource
(currently, we have Ref, GetAttrs that can give this implicitly)
- OUTPUTS: all the attributes that are produced/written by that resource (I
do not know if this write-set is currently well-defined for a resource. I
think some of them are implicitly defined by Heat on particular resource
types.)
- Global name-space and data-space : all the values produced and consumed
(INPUTS/OUTPUTS) are described using a names that are fully qualified
(XXX.stack_name.resource_name.property_name). The data values associated
with these names are stored in a global data-space. Reads are blocking,
i.e., reading a value will block the execution resource/thread until the
value is available. Writes are non-blocking, i.e., any thread can write a
value and the write will succeed immediately.
I don't believe this would give us any new behaviour.
The ability to define resources at arbitrary levels of granularity together
with the explicit specification of INPUTS/OUTPUTS allows us to reap the
benefits G1 and G2 outlined above. Note that the ability to reason about
the inputs/outputs of each resource and the induced dependencies will also
allow Heat to detect dead-locks via dependence cycles (benefit G3). This is
already done today in Heat for Refs, GetAttr on base-resources, but the
proposal is to extend the same to arbitrary attributes for any resource.
How are TemplateResources and NestedStacks any different? To my
knowledge this is aleady the case.
The blocking-read and non-blocking writes further structures the
specification to avoid deadlocks and race conditions (benefit G3).
Have you experienced deadlocks with heat? I have never seen this...
As for G4, the conjecture, I can only give as evidence our experience with
using our DSL with the proposed abstraction to deploy a few reasonably
large applications :-)
I would like to know your comments and suggestions. Also, if there is
interest I can write a Blueprint / proposal with more details and
use-cases.
We already have this functionality. Whenever you use a Ref or Attr
from another resource we become dependant on it. This feels like
replacing an existing system with a completely different one with
very little benefit.
To me what is missing to better support complex software configuration
is :
- better integrating with existing configuration tools (puppet, chef,
salt, ansible, etc). (resource types)
- integration with a ssh key store (Barbican) so the tools above can
do non-userdata related configuration
-Angus
Thanks,
LN
Clint Byrum <cl...@fewbar.com> wrote on 10/11/2013 12:40:19 PM:
From: Clint Byrum <cl...@fewbar.com>
To: openstack-dev <openstack-dev@lists.openstack.org>
Date: 10/11/2013 12:43 PM
Subject: Re: [openstack-dev] [Heat] HOT Software orchestration
proposal for workflows
Excerpts from Stan Lagun's message of 2013-10-11 07:22:37 -0700:
> Hello,
>
> Thanks Angus, Clint, I've got your design.
>
> It seems that Murano can built on top of that. With service metadata
> knowledge Murano can generate HOT templates with set of interdependent
> configs.
> Here is what will be needed:
>
> 1. Ability to implement support for custom software configuration tool
> (type: OS::SoftwareConfig::MuranoAgent)
These are really syntactic sugar, but you can implement them as resource
plugins for users who want Murano resources. In the absence of this,
just putting things in the free form metadata area of the resource that
the MuranoAgent can interpret would suffice.
> 2. Ability to provide arbitrary input values for the config
We already have that, there's a free-form json document called "metadata"
attached to every resource. Or maybe I missed what you mean here. The
new capability that is in the works that will make that better is to
have multiple reusable metadata blocks referenced on one instance.
> 3. Ability to return arbitrary (JSON-compatible) data structure from
config
> application and use attributes of that structure as an input for other
> configs
Note that I'd like to see more use cases specified for this ability. The
random string generator that Steve Baker has put up should handle most
cases where you just need passwords. Generated key sharing might best
be deferred to something like Barbican which does a lot more than Heat
to try and keep your secrets safe.
> 4. Ability to provide config body that is an input to Murano Agent of
> arbitrary size
Isn't this the same as 2?
> 5. Work well with large graph of configs with a lot of dependencies.
> Independent configs on different VMs should be applied in parallel.
>
Yes this does look good. For dependent configs, the exsiting wait
condition can be used.
> Does it confirm to your plans?
>
I think it confirms that we're heading toward consensus on where to draw
the software config vs. infrastructure orchestration line. That is very
exciting. :)
_______________________________________________
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
_______________________________________________
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
_______________________________________________
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
