On 6/17/2014 7:04 AM, Eoghan Glynn wrote: > Folks, > > A question for the taskflow ninjas. > > Any thoughts on best practice WRT $subject? > > Specifically I have in mind this ceilometer review[1] which adopts > the approach of using very fine-grained tasks (at the level of an > individual alarm evaluation) combined with short-term assignments > to individual workers. > > But I'm also thinking of future potential usage of taskflow within > ceilometer, to support partitioning of work over a scaled-out array > of central agents. > > Does taskflow also naturally support a model whereby more chunky > tasks (possibly including ongoing periodic work) are assigned to > workers in a stickier fashion, such that re-balancing of workload > can easily be triggered when a change is detected in the pool of > available workers?
I don't think taskflow today is really focused on load balancing of tasks. Something like gearman [1] might be better suited in the near term? My understanding is that taskflow is really focused on in-process tasks (with retry, restart, etc) and later will support distributed tasks. But my data could be stale too. (jharlow?) Even still, the decision of smaller tasks vs. chunky ones really comes down to how much work you want to re-do if there is a failure. I've seen some uses of taskflow where the breakdown of tasks seemed artificially small. Meaning, the overhead of going back to the library on an undo/rewind is greater than the undo itself. -S [1] http://gearman.org/ _______________________________________________ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev