Good stuff. Obviously a "fancy include" was the wrong metaphor. The
lifecycles are linked though. When a create the referencing service (aria
service create), ARIA will match the other service template, and run create
service on it, and then continue on. Uninstall of the referencer will
destroy the referenced service. After such an instantiation, would listing
all services produce two (perhaps with some parent indicator) or one? From
what I've seen, reqs and caps don't have the equivalent of a lifecycle
interface associated. Maybe I missed it. It is implicit in the
orchestrator to match stuff up at service creation time, I'm assuming, and
using plugins somehow? The sort of flagship scenario is matching cloud
compute resources without specifying an image. Since discovering whether a
particular cloud has a given capability, I guessing a plugin for that cloud
has to search cloud inventory and perhaps quota to provide info for the
match. Since plugins (at least the Cloudify variety) tend to be triggered
due to operations that they connect to lifecycle methods, and there is no
"materialize_capabilities" interface that I noticed, some other kind of
magic must be performed. Maybe this is an orchestrator-specific detail
outside of TOSCA.
DeWayne
On Fri, Aug 11, 2017 at 10:57 AM, Tal Liron <t...@cloudify.co> wrote:
> OK, let me try to organize this differently. Three potential conceptions
> here:
>
> 1) A "fancy include," as you say. All that would happen here is that the
> TOSCA parser would automatically find the service template to include from
> some kind of repository of recognized service templates, and just include
> that. The language in the TOSCA spec suggests that this is not the
> intention: it is something that happens at the "orchestrator," not the
> "parser."
>
> 2) Static service composition. This happens not at the parsing stage, but
> rather the instantiation stage, where reqs-and-caps happens. I think this
> is what is intended: substitution mapping is specifically about mapping
> reqs-and-caps. And this is also where things like scalability and placement
> happen: think of a requirement matching a capability, but that capability
> not having enough capacity. So, the node might be able to scale out: in the
> case of substitution, this would mean duplicating an entire service
> instance. My understanding is that this is what is intended by TOSCA, and
> it's entirely within the scope of what we can do. We've recently just
> refactored the instantiation phase into a new "topology" module where
> exactly all this logic is concentrated. So think of it this way -- ARIA has
> three parts: "parser," "topology manager," and "workflow engine"
> ("orchestrator").
>
> (I think I might have confused some thing here a bit when mentioning a
> "logical proxy node." I do not mean an actual piece of proxy software
> running on a machine somewhere. I mean just a data point in the
> ARIA-generated topology that can be used as a barrier of sorts when
> constructing the task graph -- because the task graph follows
> relationships, the edges of the topology. It could be that we discover in
> our POC that this is not needed, because actually the substitution node is
> already part of our topology data model and we might be able to easily take
> that into account when generating the task graph.)
>
> 3) Live service composition. I think this is a fantasy of some people: that
> ARIA would be able to take existing, "running" service chains and run
> workflows with them. I do think this is a solvable problem, but not via
> substitution mapping per se. A solution could involve deployment of a proxy
> service (which could actually be encapsulated in a substitution mapping,
> but doesn't have to be), or configuring specialized virtual ports via an
> SDN controller, or via logical proxy nodes created via an inspection tool,
> etc. I cannot believe that there is a one-size-fits-all solution to this
> problem. The dream of NFV might be to have everything connect to each other
> like LEGO blocks, but there are far too many protocols, configuration, and
> security standards. I think point #2, what I called "static service
> composition," is a realistic compromise and within the scope of what TOSCA
> promises.
>
>
> On Fri, Aug 11, 2017 at 12:08 PM, DeWayne Filppi <dewa...@cloudify.co>
> wrote:
>
> > To my eyes, the spec doesn't speak of runtime service substituion, but
> > parsetime template composition. IOW, substitution mapping is a fancy
> > "include", or the equivalent of an interface definition. Is it
> understood
> > by the ARIA team that this includes proxying of running services? IOW,
> if
> > my template requires a database service that my template does *not* want
> to
> > control the lifecycle of, I can "substitution map" an instance of a
> > template (i.e. a running service)? This would be a lovely feature, but
> > it's not really a "substitution map", rather more of a "service proxy"
> (as
> > implemented as a plugin in Cloudify). Just trying to clarify. Maybe
> the
> > community thinks that "substitution map" as something that occurs beyond
> > parsetime, or should.
> >
> > On Fri, Aug 11, 2017 at 9:52 AM, Tal Liron <t...@cloudify.co> wrote:
> >
> > > Well, DJ, it's all just opinion at this stage and feedback is welcome!
> > >
> > > Option 1:
> > > > To look at satisfying nodes present in a substituting
> service,
> > > > Have these nodes part of the newly created service and remove the
> > > > substituting service(nodes with different ID's. Also we are very much
> > in
> > > > favor of UUID )
> > > > With this approach I guess the substituting service should
> not
> > > > have any associated workflows running. If at all an workflow
> execution
> > is
> > > > already triggered I hope this service will not be considered for
> > > > substitution.
> > > > I hope this is the correct approach when we are looking at a
> > > > service for the substitution
> > > >
> > >
> > > Yes, this is a good idea. It would be easy to discover this according
> to
> > > the stored node state -- just make sure that all nodes are in a stable
> > > state before composition.
> > >
> > > This leads to a general issue: before composition, the substituting
> > > services must be validated in some way before composition begins.
> > >
> > > Also, let's all start using TOSCA terminology here: the containing
> > service
> > > is called the "top-level service," and the contained services are
> called
> > > "substituting services."
> > >
> > > Also, we keep using trivial examples, but it's definitely possible for
> a
> > > top-level service to require several substituting services at the same
> > > time. I can definitely see such things happening in NFV with even
> simple
> > > service chains. Basically every VNF could be a substituting service.
> > >
> > > So, actually one of the validations would be to make sure you do not
> > create
> > > circular composition: if the top-level also has substitution mappings,
> > you
> > > need to make sure that one of the substituting ones doesn't require it.
> > :)
> > > Not a very likely situation, but it would lead to failure.
> > >
> > >
> > >
> > > > Option 2:
> > > > While creating a service look at the req-caps at the
> > > > service-template level and create a service including the nodes
> > provided
> > > by
> > > > the substituting service-template. With this approach there would not
> > be
> > > > any service created from the service-template which is providing
> the
> > > > substitution functionality. The service-template would remain the
> same
> > > but
> > > > only the service would be added with extra nodes.
> > > >
> > > > Are you considering both option 1 & 2 for the implementation ? If not
> > > > which one do you feel which take priority. I see option 2 at this
> stage
> > > > could be the best possible approach
> > > > Also could you please let me know a tentative time for this feature
> to
> > be
> > > > available?
> > > >
> > >
> > > I think both options 1 and 2 make sense and are useful, and actually
> one
> > is
> > > a subset of the other.
> > >
> > > With option #2 (substituting a service template), it means new nodes
> are
> > > instantiated and the composed service would include all nodes. So, an
> > > "install" workflow would install everything at once. In this case we do
> > > need to fix the lifecycle workflows to be "boundary aware," so that
> > > workflows of substituting service nodes are part of their own task
> > graph. I
> > > think that possibly using a logical proxy node in between might solve
> > this
> > > situation automatically.
> > >
> > > With option #1 (substituting a service) the substituting nodes might
> > > already be installed. Or not (they might have been instantiated but
> still
> > > not installed). So, the lifecycle workflows should only work on the
> nodes
> > > that have not yet been installed.
> > >
> > > The point is that we just need to beef up the lifecycle workflows to
> > > properly work with boundaries and with a mix of nodes in different
> > states.
> > > If they can do that, then they can handle any kind of service
> > composition,
> > > whether it's option #1 or option #2.
> > >
> > > I don't think we can provide a timeline yet, but I will say that we are
> > in
> > > the research phase and may have a POC soon. Avia is in charge of this
> > JIRA,
> > > so I will let him chime in with the current state of things.
> > >
> >
>
