Hi -
>
> When exactly is the case that a ContainerRouter should put a blocking
> activation to a queue for stealing? Since a) it is not spawning containers
> and b) it is not parsing request/response bodies, can we say this would
> only happen when a ContainerRouter maxes out its incoming request
handling?
>
That's exactly the idea! The work-stealing queue will only be used if the
Router where to request landed cannot serve the demand right now. For
example, if it maxed out the slots it has for a certain action (all
containers are working to their full extent) it requests more resources and
puts the request-token on the work-stealing queue.
So to clarify, ContainerRouter "load" (which can trigger use of queue) is
mostly (only?) based on:
* the number of Container references
* the number of outstanding inbound HTTP requests, e.g. when lots of requests
can be routed to the same container
* the number of outstand outbound HTTP requests to remote action containers
(assume all are remote)
It is unclear the order of magnitude considered for "maxed out slots", since
container refs should be simple (like ip+port, action metadata, activation
count, and warm state), inbound connection handling is basically a http server,
and outbound is a connection pool per action container (let's presume
connection reuse for the moment).
I think it will certainly need testing to determine these and to be
configurable in any case, for each of these separate stats.. Is there anything
else that affects the load for ContainerRouter?
That request-token will then be taken by any Router that has free capacity
for that action (note: this is not simple with kafka, but might be simpler
with other MQ technologies). Since new resources have been requested, it is
guaranteed that one Router will eventually become free.
Is "requests resources" here requesting new action containers, which it won't
be able to process itself immediately, but should startup + warm and be
provided to "any ContainerRouter"? This makes, sense, just want to clarify that
"resources == containers".
>
> If ContainerManager has enough awareness of ContainerRouters' states, I'm
> not sure where using a queue would be used (for redirecting to other
> ContainerRouters) vs ContainerManager responding with a ContainerRouters
> reference (instead of an action container reference) - I'm not following
> the logic of the edge case in the proposal - there is mention of "which
> controller the request needs to go", but maybe this is a typo and should
> say ContainerRouter?
>
Indeed that's a typo, it should say ContainerRouter.
The ContainerManager only knows which Router has which Container. It does
not know whether the respective Router has capacity on that container (the
capacity metric is very hard to share since it's ever changing).
Hence, in an edge-case where there are less Containers than Routers, the
ContainerManager can hand out references to the Routers it gave Containers
to the Routers that have none. (This is the edge-case described in the
proposal).
I'm not sure why in this case the ContainerManager does not just create a new
container, instead of sending to another Router? If there is some intended
limit on "number of containers for a particular action", that would be a
reason, but given that the ContainerManager cannot know the state of the
existing containers, it seems like sending to another Router which has the
container, but may not be able to use it immediately, may cause failures in
some cases.
The work-stealing queue though is used to rebalance work in case one of the
Routers get overloaded.
Got it.
Thanks
Tyson
