Generally to avoid this I've used the Synapse micro service bus. It
also allows the creation of an unlimited number of queues, allowing
higher priority tasks to "jump the queue". ' Backpressure' is
precisely what message buses avoid in distributed computing.
One of my never-have-time-for projects is to port Synapse to Pharo.
SST has a 'start a slave on another node and route to it' methodology
but it's hella complex, especially in terms of distributed garbage
collection etc. For real-time systems SST is great, not really
necessary to get into that kind of complexity otherwise though.
AndrewOn Tue, 2018-04-24 at 15:31 +0000, Santiago Bragagnolo wrote:
>
>
> On Tue, 24 Apr 2018 at 16:18 Holger Freyther <hol...@freyther.de>
> wrote:
> >
> >
> > > On 24. Apr 2018, at 20:16, Santiago Bragagnolo
> > o...@gmail.com> wrote:
> > >
> > > Hi Holger!
> > > I respond in bold
> >
> > hehe. And in the reply I am back to non rich text. Let me see if I
> > quote it correctly.
> hahahaha, non rich?, how come? I will still bolding hoping that if
> you need to ensure the content you will check in a rich text client
> :D
>
> >
> >
> > >
> > >
> > >
> > > On Tue, 24 Apr 2018 at 12:00 Holger Freyther <hol...@freyther.de>
> > wrote:
> > > Hey!
> > >
> >
> >
> > > I wondered if somebody thought of remote task execution?
> > >
> > > *If you mean something else, I would need more information :).
> >
> >
> >
> >
> > > When you do [ action ] schedule / [ action ] future, both created
> > tasks are scheduled into the default runner. The default runner is
> > a working pool with a default 'poolSizeMax' on 4, meaning, limit 4
> > processes working over the tasks. (this is a dynamic configuration,
> > you can change it by
> > > TKTConfiguration runner poolMaxSize: 20. )
> >
> > Yes. But with more work than the workers can handle the queue will
> > grow. Which means the (median/max) latency of the system will
> > monotonically increase.. to the point of the entire system failing
> > (tasks handled after the external deadlines expired, effectively no
> > work being done).
>
> Normally the worker pool adjust to the minimal needed workers (there
> is a watch dog checking how much idle processes are there, or more
> workers are needed, and ensuring to spawn or stop process regarding
> to the state).
> So, the number poolMaxSize is just a maximal limit. This limit should
> be set for ensuring that the tasks that are running concurrently are
> not incurring into too much resource consumption or into too much
> overhead leading to kind of trashing.
> I am not really friend of setting only a number for such a complex
> problematic, but so far is the only approach I found that it does not
> lead to a complex design. If you have better ideas to discuss on this
> subject, i am completely open. (the same to deal with priorities by
> general system understanding rather than absolute numbers)
>
>
> >
> > For network connected systems I like to think in terms of "back
> > pressure" (not read more from the socket than the image can handle,
> > eventually leading to the TCP window shrinking) and one way of
> > doing it is to have bounded queues (and/or sleep when scheduling
> > work).
> >
> > I can see multiple parts of a solution (and they have different
> > benefits and issues):
> >
> > * Be able to attach a deadline to a task (e.g. see context.Context
> > in go)
> > * Be able to have a "blocking until queue is less than X elements"
> > schedule (but that is difficult as one task might be scheduled
> > during the >>#value of another task).
> >
> I wouldn't mind to have a second type of queue with this behaviour,
> with a mean of configuration for setting one or other queue with it's
> specific management encapsulated.
>
> Personally, in my domains of usage ( crawling, querying and in
> sensor/actuator) i personally wouldn't use it. But I suppose you have
> a better domain for this case. It would be good to discuss it to have
> a better understanding of the need.
>
> >
> >
> > > Are there ideas how to add remote task scheduling? Maybe use
> > Seamless for it?
> > > Since you speak about seamless here, i suppose two different
> > images, doesn't matter where.
> > > It's not a bad idea to go seamless, but i did not go through the
> > first restriction of remote executions (if the remote image can or
> > not execute the task and if both images share the same semantic for
> > the execution), then i did not yet checked on which communication
> > platform to use for it
> >
> > Right it would need to be homogeneous images (and care taken that
> > the external interface remains similar enough).
> I would like to understand a bit better what you are trying to do.
> I have the hunch that you are looking for a multiple images solution,
> for load balance in between images. TaskIT is mean to plan tasks into
> process, regarding to the local image needs. You seems to need
> something for planning tasks of a general system, beyond one image,
> and maybe taking in care a process / network topology.
>
> If it's more that side, we should discuss in what extensions we can
> do into taskit to be suitable of usage in this case, but surely I
> would be inclined to do a higher level framework or even middleware
> that uses taskit, than to add all those complexities in taskit. The
> good news is that i may be needing something similar, so I will be
> able to help there.
>
>
> >
> >
> > > Have workers connect to the scheduler? Other ideas?
> > > what do you mean by connection to the scheduler? The workers we
> > use do not know their pools, if that is what you mean.
> >
> > Let's assume scheduling a task is simple (read something from a
> > socket) but the computation is expensive (database look-up, math,
> > etc). Hopefully one will reach the point where one image can
> > schedule more tasks than a single worker image can handle. At that
> > point it could be neat to scale by just starting another image. By
> > inverting the launch order (workers connect to the scheduler)
> > scaling can become more easy.
> hahaha, now i re-read this paragraph and is not a hunch anymore. You
> are looking for multiple images :).
>
>
>
> > holger
> >
> >
> >
> >
