What about the case, though, where you want not to override the launch but
add 'mixin' behaviour?
I guess this is where Alex's 'run-before' etc. comes in (I'm not so keen on
the "initd" style of notation).
For run-before etc. to work you'd need to be able to have names for each
member of a sequence of tasks, but you practically have that above anyway,
if you make the effectors a list of key/action pairs, i.e. instead of
effectors:
start:
035-pre-launch-get-semaphore: { acquire-semaphore: { scope:
$brooklyn:parent(), name: "node-launch" } }
040-launch: { ssh: "service cassandra start" }
045-confirm-service-up: { wait: { sensor: service.inCluster, timeout:
20m } }
050-finish-release-semaphore: semaphore-release
you have
effectors:
start:
- pre-launch-get-semaphore: { acquire-semaphore: { scope:
$brooklyn:parent(), name: "node-launch" } }
- launch: { ssh: "service cassandra start" }
- confirm-service-up: { wait: { sensor: service.inCluster, timeout:
20m } }
- finish-release-semaphore: semaphore-release
and then a child "mixes-in" via:
effectors:
start:
my-pre-launch:
task: { acquire-semaphore: ... }
run-before: launch
But I think this still needs a lot of thought, especially if you are going
to have an entity inheriting from a whole hierarchy of super entities -
whose "launch" are you talking about above? How is this big graph of tasks
for each entity going to be ordered?
On Wed, 11 Jan 2017 at 16:20 Sam Corbett <[email protected]>
wrote:
> +1 to Alex's core suggestion. It would be a powerful extension to
> Brooklyn. We should not force blueprint authors into arbitrary
> structures that were only a good fit for someone else's scenario in the
> past.
>
> To lob another suggestion into the mix, I imagine writing:
>
> effectors:
> # default named phases, not needed normally.
> start: [provision, install, customise, launch]
> # launch phase overridden to wrap latter half with mutex handling
> launch:
> - ssh-task-1
> - mutexed
> subtasks:
> - ssh-task-2
> - wait-for-running
>
> This implies that each of the phases of start is another effector and
> that they are configurable objects. The `mutexed` task cleans up
> properly when its subtasks fail.
>
> Sam
>
>
> On 11/01/2017 15:55, Svetoslav Neykov wrote:
> > Alex,
> >
> > I like the task based approach and agree it's something we need to push
> for. I'm not convinced that locking-like behaviour should be represented as
> tasks though. It brings procedural style flavour to concurrency - for
> example in your original example there's a separate tasks for acquiring and
> releasing the lock. They could easily get misplaced, overwritten,
> forgotten, etc. Not clear how releasing it works in case of errors.
> > What I'd prefer is more declarative approach where the tasks are grouped
> and the locking requirements are applied on the group. At worst referencing
> a start task and optionally an end task (with the default of the parent
> task existing).
> >
> > Plugging into the lifecycle of entities has other use-cases. No matter
> how an entity is defined - whether using the current monolithic START
> effector or one composited of smaller tasks - there's no way to be notified
> when its tasks get executed. Some concrete examples - the
> "SystemServiceEnricher" which looks for the "launch" task and can be
> applied on any "SoftwareProcessEnity"; An entity which needs to do some
> cleanup based on the shutdown of another entity (DNS blueprints); latching
> at arbitrary points during entity lifecycle; etc.
> >
> > One of the alternatives I mention in the original email (Effector
> execution notifications) is a step in the "task based" approach direction.
> It's still in Java, but we are splitting the monolith effectors into
> smaller building blocks, which could in future get reused in YAML.
> >
> > So to summarise - +1 for tasks as building blocks, but still need
> visibility into the executing blocks.
> >
> >> PS - as a near-term option if needed we could extend SoftwareProcess
> LATCH to do something special if the config/sensor it is given is a
> "Semaphore" type
> > What do you think the behaviour should be here - releasing the semaphore
> after the corresponding step completes or at the end of the wrapping
> effector? I think this is defined by the blueprint authors. And making this
> configurable adds even more complexity. Instead could invest in developing
> the above functionality.
> >
> > Svet.
> >
> >
> >
> >> On 11.01.2017 г., at 16:25, Alex Heneveld <
> [email protected]> wrote:
> >>
> >>
> >> svet, all,
> >>
> >> lots of good points.
> >>
> >> the idea of "lifecycle phases" in our software process entities has
> grown to be something of a monster, in my opinion. they started as a small
> set of conventions but they've grown to the point where it's the primary
> hook for yaml and people are wanting "pre.pre.install".
> >>
> >> it's (a) misguided since for any value of N, N phases is too few and
> (b) opaque coming from a java superclass.
> >>
> >> a lot of things rely on the current SoftwareProcess so not saying we
> kill it, but for the community it would be much healthier (in terms of
> consumers) to allow multiple strategies and especially focus on *the
> reusability of tasks in YAML* -- eg "provision" or "install template files"
> or "create run dir" -- so people can write rich blueprints that don't
> require magic lifecycle phases from superclasses.
> >>
> >> the "init.d" numbered approach svet cites is one way these are wired
> together, with extensions (child types) able to insert new numbered steps
> or override with the number and label. but that would be one strategy,
> there might be simpler ones that are just a list, or other sequencing
> strategies like precondition/action/postcondition or runs-before/runs-after
> (where to extend something, you can say `my-pre-customize: { do: something,
> run-before: launch, run-after: customize }`).
> >>
> >> we're not sure exactly how that would look but wrt mutex logic the idea
> that adjuncts plug into the entity lifecycles feels wrong, like it's
> pushing for more standardisation in lifecycle phases where things can plug
> in. whereas with a task approach we can have effector definitions being
> explicit about synchronization, which i think is better. and if they want
> to make that configurable/pluggable they can do this and be explicit about
> how that is done (for instance it might take a config param, even a config
> param (or child or relation) which is an entity and call an effector on
> that if set).
> >>
> >> concretely in the examples i'm saying instead of ENRICHER APPROACH
> >>
> >> memberSpec:
> >> $brooklyn:entitySpec:
> >> type: cluster-member
> >> brooklyn.enrichers:
> >> - type:
> org.apache.brooklyn.enricher.stock.AquirePermissionToProceed
> >> brooklyn.config:
> >> stage: post.provisioning
> >> # lifecycle stage "post.provisioning" is part of
> cluster-member and
> >> # the enricher above understands those stages
> >>
> >> we concentrate on a way to define tasks and extend them, so that we
> could instead have a TASK APPROACH:
> >>
> >> memberSpec:
> >> $brooklyn:entitySpec:
> >> type: cluster-member
> >> effectors:
> >> start:
> >> 035-pre-launch-get-semaphore: { acquire-semaphore: ... }
> >> # assume 040-launch is defined in the parent "start"
> yaml defn
> >> # using a new hypothetical "initd" yaml-friendly task
> factory,
> >> # acquire-semaphore is a straightforward task;
> >> # scope can come from parent/ancestor task,
> >> # also wanted to ensure mutex is not kept on errors
> >>
> >> or
> >>
> >> memberSpec:
> >> $brooklyn:entitySpec:
> >> type: cluster-member
> >> effectors:
> >> start:
> >> my-pre-launch:
> >> task: { acquire-semaphore: ... }
> >> run-before: launch
> >> run-after: customize
> >> # launch and customize defined in the parent "start"
> yaml defn,
> >> # using a new hypothetical "ordered-labels"
> yaml-friendly task factory;
> >> # acquire-semaphore and scope are as in the "initd"
> example
> >>
> >>
> >> both approaches need a little bit of time to get your head around the
> new concepts but the latter approach is much more powerful and general.
> there's a lot TBD but the point i'm making is that *if we make tasks easier
> to work with in yaml, it becomes more natural to express concurrency
> control as tasks*.
> >>
> >>
> >> PS - as a near-term option if needed we could extend SoftwareProcess
> LATCH to do something special if the config/sensor it is given is a
> "Semaphore" type
> >>
> >> best
> >> alex
> >>
>
>
