The C++ RPC implementation has one, limited, form of backpressure created specifically for Sandstorm sandboxing purposes: setFlowLimit().
https://github.com/capnproto/capnproto/blob/master/c++/src/capnp/rpc.h#L115 This simple approach works well enough to prevent buggy Sandstorm apps from filling up the front-end's memory. It can theoretically lead to deadlock, though, in the case where a recursive call bounces back and forth enough times to fill the limit, then gets stuck waiting. -Kenton On Fri, Sep 1, 2017 at 9:26 AM, Ross Light <r...@zombiezen.com> wrote: > Just wanted to close this thread off: I think I have what I need to > unblock Go RPC improvements. My ramblings on implementation at the end > didn't make much sense and were more complicated than what's needed. Don't > mind me. :) > > Time permitting, I'll try to collect my observations about backpressure in > Cap'n Proto in some sort of sensible documentation. Perhaps this would be > a good candidate for some of the non-normative docs of the RPC spec. I > agree that being able to apply backpressure to a single capability without > blocking the whole connection would be a boon. > > One thing I'm currently curious about in the C++ implementation: does the > RPC system provide any backpressure for sending calls to the remote vat? > AFAICT there's no bound on the EventLoop queue. > > On Wed, Jul 26, 2017 at 10:38 AM Kenton Varda <ken...@cloudflare.com> > wrote: > >> On Wed, Jul 26, 2017 at 9:16 AM, Ross Light <r...@zombiezen.com> wrote: >>> >>> Cap'n Proto extends the model by making request/response pairings >>>> explicit, but it doesn't require that a response be sent before a new >>>> request arrives. >>>> >>> >>> Good point; I'm not arguing for that restriction. I'm fine with this >>> sequence (which conceptually only requires one actor): >>> >>> 1. Alice sends Bob foo1() >>> 2. Bob starts working on foo1() >>> 3. Alice sends Bob foo2(). Bob queues it. >>> 4. Alice sends Bob foo3(). Bob queues it. >>> 5. Bob finishes foo1() and returns foo1()'s response to Alice >>> 6. Bob starts working on foo2() >>> 7. Bob finishes foo2() and returns foo2()'s response to Alice >>> 8. Bob starts working on foo3() >>> 9. Bob finishes foo3() and returns foo3()'s response to Alice >>> >> >> In this example, you're saying Bob can't start working on a new request >> until after sending a response for the last request. That's what I'm saying >> is *not* a constraint imposed by Cap'n Proto. >> >> >>> Here's the harder sequence (which IIUC, C++ permits. *If it doesn't*, >>> then it simplifies everything.): >>> >>> 1. Alice sends Bob foo1() >>> 2. Bob starts working on foo1(). It's going to do something that will >>> take a long time (read as: requires a future), so it acknowledges delivery >>> and keeps going. Bob now has has multiple conceptual actors for the same >>> capability, although I can see how this can be also be thought of as a >>> single actor receiving request messages and sending response messages. >>> 3. Alice sends Bob foo2() >>> 4. Bob starts working on foo2(). >>> 5. foo2() is short, so Bob returns a result to Alice. >>> 6. foo1()'s long task completes. Bob returns foo1()'s result to Alice. >>> >> >> This does not create "multiple conceptual actors". I think you may be >> mixing up actors with threads. The difference between a (conceptual) thread >> and an (conceptual) actor is that a thread follows a call stack (possibly >> crossing objects) while an actor follows an object (sending asynchronous >> messages to other objects). >> >> In step 2, when Bob initiates "something that will take a long time", in >> your threaded approach in Go, he makes a blocking call of some sort. But in >> the actor model, blocking calls aren't allowed. Bob would initiate a >> long-running operation by sending a message. When the operation completes, >> a message is sent back to Bob with the results. In between these messages, >> Bob is free to process other messages. The important thing is that only one >> message handler is executing in Bob at a time, therefore Bob's state does >> not need to be protected by a mutex. However, message handlers cannot block >> -- they always complete immediately. >> >> Concretely speaking, in C++, the implementation of Bob.foo() will call >> some other function that returns a promise, and then foo() will return a >> promise chained off of it. As soon as foo() returns that promise, then a >> new method on Bob can be invoked immediately, without waiting for the >> returned promise to resolve. >> >> This of course suffers from the "function coloring problem" you >> referenced earlier. All Cap'n Proto methods are colored red (asynchronous). >> >> I think what the function coloring analogy misses, though, is that >> permitting functions to block doesn't really avoid the function-coloring >> problem, it only sweeps the problem under the rug. Even in a multi-threaded >> program, it is incredibly important to know which functions might block. >> Because, in a multi-threaded program, you almost certainly don't want to >> call a blocking functions while holding a mutex lock. If you do, you risk >> blocking not only your own thread, but all other threads that might need to >> take that lock. And in the case of bidirectional communication, you risk >> deadlock. >> >> This is, I think, exactly the problem I think you're running into here. >> >> Alternatively, what if making a new call implicitly acknowledged the >>>> current call? This avoids cognitive overhead and probably produces the >>>> desired behavior? >>>> >>> >>> I don't think this is a good idea, since it seems common to want to >>> start off a call (or multiple) before acknowledging delivery. >>> >> >> I guess I meant: *Waiting* on results of a sub call should implicitly >> acknowledge the super call / unblock concurrent calls. So you could *start* >> multiple sub calls while still being protected from concurrent calls, but >> as soon as you *wait* on one, you're no longer protected. >> >> >>> I thought about this a bit more over the last couple of days and I >>> think I have a way out (finally). Right now, operating on the connection >>> acquires a mutex. I think I need to extend this to be a mutex+condition, >>> where the condition is for is-connection-making-call. When the connection >>> makes a call, it marks the is-connection-making-call bit, then plumbs the >>> is-in-a-connection-call info through the Context (think of as thread-local >>> storage, except explicit). When the connection acquires the mutex, >>> non-send-RPC operations will block on the is-connection-making-call bit to >>> be cleared and send-RPC operations will not block. I've examined the >>> send-RPC path and that operation ought to be safe to be called. This would >>> avoid the nasty queue idea that I had. >>> >> >> Sorry, I don't follow this. >> >> -Kenton >> > -- You received this message because you are subscribed to the Google Groups "Cap'n Proto" group. To unsubscribe from this group and stop receiving emails from it, send an email to capnproto+unsubscr...@googlegroups.com. Visit this group at https://groups.google.com/group/capnproto.
