On 21-Mar-2010 Zero Hero wrote:
> Please take this with a grain of salt, I know you've got scads of
> experience.
Experience, not really. Thinking on the matter, yep. Sorry if my post was a
bit abrupt.
> One thing I can see your thinking about is the "durable queue" problem.
My thinking is that "durable" would be something you have to overload the queue
to implement. As you say, it's difficult to solve correctly, and probably also
application-specific.
> One issue is out-of-orderness. If we have a failure going to a particular
> IKC kernel, we don't want the next requests going out and succeeding.
[...]
> I couldn't see in the code where you were accounting for this (if in fact it
> was something you're attacking).
Again, out-of-orderness is application-specific. It could even be
message-specific. As in one class of messages must be in-order, another
class could be dropped on the floor and it doesn't matter.
In my code, you'll see a sort keys %{ $self->{Q} } in __connect. Easily, but
it takes care of in-orderness on the sender side. At least, until perl's scalar
wraps around.
> 1. you pass in an integer for request/reply matching.
> 2. This gets sent with the real request, which is executed on the foreign
> IKC. Some wrapper function sends the integer back with the desired
> results (or are you making a second reply/request just for ACK...?).
Correct on both points. The acknowledge would be handled by IKC::Responder.
Either a small mod to the core IKC or overloading the Responder.
> One is we'll have to set timers on the request/responses to reap them from
> the queue.
You're right, I completely punted on the timeout issue.
> Or perhaps there is a different strategy you're thinking about.
My thinking is that IKC (or it's successor) needs to be flexible enough that
any strategy could be used, on a per-source, per-destination or per-message
basis. The hard/impossible one will be per-message, unless we move to Reflex.
By source, I mean either for the entire local kernel, or per-SENDER session.
By destination, either by dest kernel, session or even destination event.
I now realise that my code is very unclear on some elements : the $config
would be used somehow in every place I put an XXX in comments to decide which
strategy to use for the current request in the current situation.
In fact, that code was written so that I could stop obcessing about it and get
to sleep.
> We want to keep a per-foreign-IKC queue to make sure we preserve
> in-orderness. We need to halt this queue sending if we don't get the ACK
> back. We're not really doing that here.
I implemented a queue per destination, that is per poe://kernel/session.
Each queue tracks if the remote kernel it is interested in is available
({connected} via __connect and __disconnect) This is verified in _default.
> They can complete out-of-order (if we temporarily cut the network for
> example, some will fail and later ones will succeed). So now if we retry,
> we'll have out-of-order execution.
So : REQ1, REQ2, temporary failure blocks REQ1, but REQ2 is successful, ACK2
sent, timeout, REQ1 retransmit, ACK1 sent.
Nasty, when in-order is important. And next to impossible to write a unit
test for. This can be solved at the application level by adding a sequence
number to the request. At the IKC level the REQNO would need to be verified on
the remote side. And the remote side would need to know that in-order is
important for that class of requests. Would the remote side queue REQ2 until
REQ1 comes? We end up with 2 queues, one in the sending kernel, one in the
destination kernel. It's what TCP does.
However, the AMQP model is that there is one queue, exterior to both processes.
The sender places messages on the queue, the receiver polls the queue for new
messages. IKC1 does this too.
-Philip
