Thanks everyone for your opinions. I still think it's valuable to have the "tasks callback" mechanism, but will pitch it again in another context in the future. For now I settled on extending the latch functionality in SoftwareProcess to limit the parallelism on the immediate task it's guarding. The PR is available at https://github.com/apache/brooklyn-server/pull/520 <https://github.com/apache/brooklyn-server/pull/520> and any feedback is welcome. Longer term we'll be able to express this better with the tasks based approach discussed below.
Svet. > On 12.01.2017 г., at 11:49, Aled Sage <[email protected]> wrote: > > Hi all, > > Great discussion - really like the direction this is going of using tasks in > yaml blueprints. > > However, it feels like we've launched into discussing a complex use-case > (concurrency control) without having first discussed what yaml blueprints > would look like for simpler tasks (*). I suggest we discuss that in another > proposal thread, and get agreement of what the YAML should look like. Let's > focus on the yaml blueprint author - i.e. reach agreement on some yaml > examples, without worrying too much about how it's implemented under the > covers. > > I'd like the examples to show an entire blueprint - including defining > effectors, sensors, feeds, etc. > > --- > > Svet, for the "callback behaviour for effectors" I think that would be a > powerful/advanced feature. If multiple callbacks were registered (e.g. in the > super-type and in the sub-type, when extending existing blueprints), the > order the callbacks are called could get tricky for blueprint authors. Maybe > we live with that. Maybe the task-based approach will give a more elegant way > to control it, making the callbacks redundant. > > If it helps to solve your immediate problem cleanly, I'm fine with us adding > support for it. But I worry that we'd want to deprecate it shortly after > task-based yaml is supported. > > Aled > > (*) Sorry if I've forgotten about some previous discussion on the mailing > list - if I have, then please point us at it again! > > > > On 11/01/2017 17:45, Svetoslav Neykov wrote: >> Sam I think that's a great approach to composing tasks and mixing them with >> semaphores. >> >>> sem --id brooklyn --fg "$REMOVE_OLD_CMD ; $ADD_NEW_CMD" >> Alex, I like that, gives you much cleaner semantics than flock. >> >>> effectors: >>> start: >>> my-pre-launch: >>> task: { acquire-semaphore: ... } >>> run-before: launch >> >> Geoff, I have some reservations re init.d style naming as well. Mostly due >> to being too fragile when having multiple layers of blueprints. I'm going to >> suggest one more alternative, somehow similar to before/after. Instead of >> declaring when to run a task (before/after another task), just override the >> parent task, do the work you need to and reference the parent task >> explicitly. >> >> effectors: >> start: >> launch: [my-custom-launch, $super.launch] >> my-custom-launch: >> - ssh: .... >> >> Anyway this is getting sidetracked. I agree that the task based approach is >> a nice place to be longer term. Might go with extending the latches shorter >> term - need to think some more about it. >> >> What do people think about introducing the callback behaviour for effectors >> though? I see it as an orthogonal concept to what's possible with composing >> tasks. It's a middle ground for implementing the semaphores where the >> latches are too limited and "semaphores as tasks" are still not here. Will >> make life easier for some more complex scenarios where one needs to sync >> with/influence other entities (examples in previous emails). >> >> Svet. >> >> >> >>> On 11.01.2017 г., at 18:50, Alex Heneveld <[email protected]> >>> wrote: >>> >>> >>> Svet- >>> >>> On 11/01/2017 15:55, Svetoslav Neykov wrote: >>>>> PS - as a near-term option if needed we could extend SoftwareProcess >>>>> LATCH to do something special if the config/sensor it is given is a >>>>> "Semaphore" type >>>> What do you think the behaviour should be here - releasing the semaphore >>>> after the corresponding step completes or at the end of the wrapping >>>> effector? I think this is defined by the blueprint authors. And making >>>> this configurable adds even more complexity. Instead could invest in >>>> developing the above functionality. >>> I'd probably release the latch sempahore right after the phase but I agree >>> it's arbitrary and don't like that. Only suggested it if we need something >>> very quick as it's isolated and probably fairly easy in the code. >>> >>> Note code exists and is run implicitly I think during the Install step >>> (otherwise things like yum complain), along with I think the class is >>> called SemaphoreWithOwner. >>> >>> I'd LOVE to invest in the task-based functionality. Code-wise it's not >>> that far away with the type registry and YOML, but it needs a couple more >>> people to become familiar with it! >>> >>> Lastly I meant to say -- someone mentioned flock but the bash *sem* command >>> (aka parallel) is awesome though not as well known as it should be. This >>> is an example used in zookeeper: >>> >>> >>> sudo yum install parallel >>> >>> REMOVE_OLD_CMD="sed -i /server.*=/d zoo.cfg" >>> ADD_NEW_CMD="cat >> zoo.cfg << EOF >>> server.0=NEW >>> server.1=NEW >>> EOF >>> " >>> >>> sem --id brooklyn --fg "$REMOVE_OLD_CMD ; $ADD_NEW_CMD" >>> >>> >>> It runs the commands in the quotes on the last line while acquiring the >>> semaphore with the given name "brooklyn" on the machine. You can configure >>> semaphore counts and timeouts too. >>> >>> Best >>> Alex >>> >
