OK, let me try to organize this differently. Three potential conceptions
here:
1) A "fancy include," as you say. All that would happen here is that the
TOSCA parser would automatically find the service template to include from
some kind of repository of recognized service templates, and just include
that. The language in the TOSCA spec suggests that this is not the
intention: it is something that happens at the "orchestrator," not the
"parser."
2) Static service composition. This happens not at the parsing stage, but
rather the instantiation stage, where reqs-and-caps happens. I think this
is what is intended: substitution mapping is specifically about mapping
reqs-and-caps. And this is also where things like scalability and placement
happen: think of a requirement matching a capability, but that capability
not having enough capacity. So, the node might be able to scale out: in the
case of substitution, this would mean duplicating an entire service
instance. My understanding is that this is what is intended by TOSCA, and
it's entirely within the scope of what we can do. We've recently just
refactored the instantiation phase into a new "topology" module where
exactly all this logic is concentrated. So think of it this way -- ARIA has
three parts: "parser," "topology manager," and "workflow engine"
("orchestrator").
(I think I might have confused some thing here a bit when mentioning a
"logical proxy node." I do not mean an actual piece of proxy software
running on a machine somewhere. I mean just a data point in the
ARIA-generated topology that can be used as a barrier of sorts when
constructing the task graph -- because the task graph follows
relationships, the edges of the topology. It could be that we discover in
our POC that this is not needed, because actually the substitution node is
already part of our topology data model and we might be able to easily take
that into account when generating the task graph.)
3) Live service composition. I think this is a fantasy of some people: that
ARIA would be able to take existing, "running" service chains and run
workflows with them. I do think this is a solvable problem, but not via
substitution mapping per se. A solution could involve deployment of a proxy
service (which could actually be encapsulated in a substitution mapping,
but doesn't have to be), or configuring specialized virtual ports via an
SDN controller, or via logical proxy nodes created via an inspection tool,
etc. I cannot believe that there is a one-size-fits-all solution to this
problem. The dream of NFV might be to have everything connect to each other
like LEGO blocks, but there are far too many protocols, configuration, and
security standards. I think point #2, what I called "static service
composition," is a realistic compromise and within the scope of what TOSCA
promises.
On Fri, Aug 11, 2017 at 12:08 PM, DeWayne Filppi <dewa...@cloudify.co>
wrote:
> To my eyes, the spec doesn't speak of runtime service substituion, but
> parsetime template composition. IOW, substitution mapping is a fancy
> "include", or the equivalent of an interface definition. Is it understood
> by the ARIA team that this includes proxying of running services? IOW, if
> my template requires a database service that my template does *not* want to
> control the lifecycle of, I can "substitution map" an instance of a
> template (i.e. a running service)? This would be a lovely feature, but
> it's not really a "substitution map", rather more of a "service proxy" (as
> implemented as a plugin in Cloudify). Just trying to clarify. Maybe the
> community thinks that "substitution map" as something that occurs beyond
> parsetime, or should.
>
> On Fri, Aug 11, 2017 at 9:52 AM, Tal Liron <t...@cloudify.co> wrote:
>
> > Well, DJ, it's all just opinion at this stage and feedback is welcome!
> >
> > Option 1:
> > > To look at satisfying nodes present in a substituting service,
> > > Have these nodes part of the newly created service and remove the
> > > substituting service(nodes with different ID's. Also we are very much
> in
> > > favor of UUID )
> > > With this approach I guess the substituting service should not
> > > have any associated workflows running. If at all an workflow execution
> is
> > > already triggered I hope this service will not be considered for
> > > substitution.
> > > I hope this is the correct approach when we are looking at a
> > > service for the substitution
> > >
> >
> > Yes, this is a good idea. It would be easy to discover this according to
> > the stored node state -- just make sure that all nodes are in a stable
> > state before composition.
> >
> > This leads to a general issue: before composition, the substituting
> > services must be validated in some way before composition begins.
> >
> > Also, let's all start using TOSCA terminology here: the containing
> service
> > is called the "top-level service," and the contained services are called
> > "substituting services."
> >
> > Also, we keep using trivial examples, but it's definitely possible for a
> > top-level service to require several substituting services at the same
> > time. I can definitely see such things happening in NFV with even simple
> > service chains. Basically every VNF could be a substituting service.
> >
> > So, actually one of the validations would be to make sure you do not
> create
> > circular composition: if the top-level also has substitution mappings,
> you
> > need to make sure that one of the substituting ones doesn't require it.
> :)
> > Not a very likely situation, but it would lead to failure.
> >
> >
> >
> > > Option 2:
> > > While creating a service look at the req-caps at the
> > > service-template level and create a service including the nodes
> provided
> > by
> > > the substituting service-template. With this approach there would not
> be
> > > any service created from the service-template which is providing the
> > > substitution functionality. The service-template would remain the same
> > but
> > > only the service would be added with extra nodes.
> > >
> > > Are you considering both option 1 & 2 for the implementation ? If not
> > > which one do you feel which take priority. I see option 2 at this stage
> > > could be the best possible approach
> > > Also could you please let me know a tentative time for this feature to
> be
> > > available?
> > >
> >
> > I think both options 1 and 2 make sense and are useful, and actually one
> is
> > a subset of the other.
> >
> > With option #2 (substituting a service template), it means new nodes are
> > instantiated and the composed service would include all nodes. So, an
> > "install" workflow would install everything at once. In this case we do
> > need to fix the lifecycle workflows to be "boundary aware," so that
> > workflows of substituting service nodes are part of their own task
> graph. I
> > think that possibly using a logical proxy node in between might solve
> this
> > situation automatically.
> >
> > With option #1 (substituting a service) the substituting nodes might
> > already be installed. Or not (they might have been instantiated but still
> > not installed). So, the lifecycle workflows should only work on the nodes
> > that have not yet been installed.
> >
> > The point is that we just need to beef up the lifecycle workflows to
> > properly work with boundaries and with a mix of nodes in different
> states.
> > If they can do that, then they can handle any kind of service
> composition,
> > whether it's option #1 or option #2.
> >
> > I don't think we can provide a timeline yet, but I will say that we are
> in
> > the research phase and may have a POC soon. Avia is in charge of this
> JIRA,
> > so I will let him chime in with the current state of things.
> >
>
