On Wed, Jul 19, 2017 at 2:57 PM, Ross Light <r...@zombiezen.com> wrote:
> Replies inline (with the disclaimer that I'm not Kenton, my only > credentials are that I have stared at this file for a long time): > > On Wed, Jul 19, 2017 at 1:46 PM Thomas Leonard <tal...@gmail.com> wrote: > >> Hi, >> >> I'm trying to write an implementation of the RPC spec (level 1, in >> OCaml). I found a few parts of the spec unclear - could someone clarify >> them for me? >> >> It says: >> >> [ExportId] >> > The exporter chooses an ID before sending a capability over the wire. If >> > the capability is already in the table, the exporter should reuse the >> same ID. >> >> But later: >> >> [CapDescriptor] >> > senderHosted @1 :ExportId; >> > A capability newly exported by the sender. This is the ID of the new >> capability in the >> > sender's export table (receiver's import table). >> >> How can the exporter reuse the same ID, if it has to be newly exported? >> > > That seems like a doc/spec typo. You can always specify an existing > capability. I think the wording should be something like: "A capability > exported by the sender. This may or may not be a new ID in the sender's > export table (receiver's import table)." > Correct. > > >> [Message] >> > This could be e.g. because the sender received an invalid or >> nonsensical >> > message (`isCallersFault` is true) or because the sender had an >> internal error >> > (`isCallersFault` is false). >> >> isCallersFault appears to be deprecated (`obsoleteIsCallersFault` appears >> much later). >> > > Yup, Exception has changed (IMO for the better). Instead of placing blame > on sender or receiver (such distinctions are hard to draw in general), > exceptions are now about what action that caller is advised to take based > on the failure. > Correct. > > [Call.sendResultsTo] >> > When `yourself` is used, the receiver must still send a `Return` for >> the call, but sets the >> > field `resultsSentElsewhere` in that `Return` rather than including the >> results. >> >> When should `resultsSentElsewhere` be returned? Once the result is known? >> Or >> once the first takeFromOtherQuestion collects it? >> > > (I haven't implemented this for Go yet, but want to.) AFAICT > resultsSentElsewhere should be sent once the result is known. > I think the answer here is "it doesn't really matter". When Alice calls Bob.foo(), and Bob tail-calls back to Alice.bar(), Bob sends the Call to bar() with "send to yourself" *immediately* followed by the Return for foo() with "take from other question". Eventually Alice sends a Return for bar(), but Bob doesn't really do anything with this Return, so it actually doesn't matter when it is sent. That said, the C++ implementation appears to wait for bar() to finish before sending the Return. > > >> Can takeFromOtherQuestion be used more than once for a single source >> question? >> > > I would assume that it could be used until Finish message is sent for that > question, much like other question-based data. In practice, every call's > result is held in the answers table until Finish is received. > No, it can only be used once. For languages without garbage collection, it would be annoying for the protocol to specify that some messages can potentially be shared. > > >> > The `Call` for bar'() has `sendResultsTo` set to `yourself`, with the >> value being the >> > question ID originally assigned to the bar() call. >> >> What does "the value" refer to here? `yourself` has type `Void`. >> > >> > Vat B receives the `Return` for bar'() and sends a `Return` for bar(), >> with >> > `receivedFromYourself` set in place of the results. >> >> `receivedFromYourself` does not appear anywhere else in the spec. >> > > I think this whole example is stale and probably needs another draft. > Yeah. I must have had an earlier version where the child call specified its parent, rather than the parent return specifying the child. > >> [Return.releaseParamCaps] >> > If true, all capabilities that were in the params should be considered >> released. >> >> Just to be sure: as if the sender had sent a release message for each one >> with `count=1`? >> > > (I might be wrong on this point, it's been a while since I've looked. The > docs should probably spell this out.) Usually. The list of CapDescriptors > in a Payload could point to the same capability multiple times. A release > message of count=1 per CapDescriptor is a more accurate way of phrasing > this. > Correct. > > >> [Payload] >> Why is it not possible to send exceptions in payloads? Should I export >> each >> broken capability as an export and then immediately send a Resolve for >> each >> one, resolving it to an exception? >> > > Payload is only used for parameters and results. It doesn't make sense > for parameters to be an exception, and results is inside a union where you > could specify an exception that is an alternative. I'm not sure I > understand the use-case where you are sending a broken capability. > Correct that Payload isn't relevant to resolving capabilities. To the original question: Hmm, I guess it would have made sense for CapDescriptor to have an additional variant for an already-broken capability, to avoid the need for an extra Resolve. However, yes, in practice, the C++ implementation will introduce a promise-capability and then immediately send a Resolve resolving it to an exception. > > >> [Resolve] >> > When an export ID sent over the wire (e.g. in a `CapDescriptor`) is >> indicated to be a promise, >> > this indicates that the sender will follow up at some point with a >> `Resolve` message. If the >> > same `promiseId` is sent again before `Resolve`, still only one >> `Resolve` is sent. If the >> > same ID is sent again later _after_ a `Resolve`, it can only be because >> the export's >> > reference count hit zero in the meantime and the ID was re-assigned to >> a new export, therefore >> > this later promise does _not_ correspond to the earlier `Resolve`. >> >> It's not clear to me why it is useful for the receiver to know this. >> Presumably the sender can't reuse an export ID until the receiver >> explicitly releases it anyway. >> Should an implementation keep track of whether a resolve has arrived yet >> and behave differently based on this when it sees an export ID? >> > > It's more specifying that the receiver should not resolve the promise more > than once. I believe in this case that it would be a protocol violation, > in which case the correct behavior would be for the receiver to send an > abort. > The text here is me trying to prove that it's safe for the protocol to specify that only one Resolve message is sent no matter how many times the export ID was introduced. I'm trying to show that there's no race conditions caused by messages travelling in opposite directions passing each other in-flight. Specifically, the rule I'm stating is: After you send a Resolve message for a promise, you *cannot* attempt to reference the same promise again in a subsequent message. The associated export ID is off-limits until it has been released. Once released, it can be reused as normal. Put another way, after a Resolve, the export ID's refcount is only allowed to decrease until it hits zero, and then it can increase again. The reason for this is that if you referenced an already-resolved promise, it's possible that the other end has already released the import and sent a Release message before it receives the new reference. In that case, when it receives the new reference, it will mistakenly believe that it's hearing about an all-new promise, and will expect a new Resolve message, which will never come. Luckily, there's no need to reference a promise again once it has been resolved. It always makes sense to reference the object it resolved to instead. However, to get this right may require some bookkeeping in the implementation. (If not for this rule, then I would have had to make a different rule instead: I would have had to say that if you reference a promise again after resolving it, then you need to send another Resolve message. But that would have just been wasteful.) > > > The sender promises that from this point forth, until `promiseId` is >> released, it shall >> > simply forward all messages to the capability designated by `cap`. >> >> Does something similar apply to Return messages? Might be worth >> mentioning it there too. >> > > I believe so, but I don't know/remember. :( > The comment here is referring to a rule that resolves the Tribble 4-way Race Condition, which relates to embargoes, which I'll discuss below. The rule says that once you've declared that promise P resolves to capability C, then any future message address to promise P shall be forwarded to capability C -- *even if* capability C itself turns out to be a promise which resolves to D. Even after the resolution, messages to P cannot be forwarded directly to D -- they *must* be forwarded to C (which will then forward to D). A similar requirement applies to returns, yes. > [Disembargo] >> > Embargos are used to enforce E-order in the presence of promise >> resolution. That is, if an >> > application makes two calls foo() and bar() on the same capability >> reference, in that order, >> > the calls should be delivered in the order in which they were made. >> But if foo() is called >> > on a promise, and that promise happens to resolve before bar() is >> called, then the two calls >> > may travel different paths over the network, and thus could arrive in >> the wrong order. In >> > this case, the call to `bar()` must be embargoed, and a `Disembargo` >> message must be sent along >> > the same path as `foo()` to ensure that the `Disembargo` arrives after >> `foo()`. >> >> What does "this case" refer to? When exactly is an embargo needed, and >> when not? >> > > If you're implementing level 1 (two-party), then really the only place > where this applies is when you receive a capability that the receiver hosts > as part of a return or resolve after you have made calls on the promised > capability. This implies that the RPC system needs to keep track of which > parts of the answer have had calls made on them. When this occurs, the > receiver gives the application code an embargoed client, and then sends a > Disembargo with senderLoopback set. It releases the embargo once the same > disembargo ID is returned with receiverLoopback set. > > For me, this was the hardest part of the spec to understand. I understand > why it's needed, but it's really hard to grok the implications. > Example: 1. Alice -- in Vat A -- holds a capability C which is a promise pointing towards Vat B. 2. Alice calls foo() on C. This call is sent to Vat B. 3. Vat B informs Vat A that C has resolved, and it points to Carol, who also lives in Vat A. Thus, future calls can be made directly in-process. (However, the call to foo() is still in-flight.) 4. Alice calls bar() on C. Since the promise has resolved, this call can be delivered locally directly to Carol. 5. Vat B reflects the call to foo() back to Carol in Vat A. In a naive implementation, the bar() call will arrive at Carol before the foo() call does, which is wrong. We need to introduce embargoes to prevent this: 1. Alice -- in Vat A -- holds a capability C which is a promise pointing towards Vat B. 2. Alice calls foo() on C. This call is sent to Vat B. 3. Vat B informs Vat A that C has resolved, and it points to Carol, who also lives in Vat A. Thus, future calls can be made directly in-process. (However, the call to foo() is still in-flight.) 3.1 Vat A marks C as pointing to Carol, but embargoed. 3.2 Vat A sends a Disembargo message towards Vat B, addressed to the original promise. (It then releases the promise.) 4. Alice calls bar() on C. Since the promise has resolved, this call can be delivered locally directly to Carol. 4.1 Because C is marked embargoed, the message is held in a queue, not delivered yet. 5. Vat B reflects the call to foo() back to Carol in Vat A. 5.1 Vat A delivers foo() to Carol. 5.2 Vat B reflects the Disembargo back to Carol in Vat A. 5.3 Vat A processes the Disembargo, which releases the embargo on the capability C. 5.4 The bar() call, which was previously embargoed, is now delivered to Carol. > > > There are two particular cases where embargos are important. Consider >> object Alice, in Vat A, >> > who holds a promise P, pointing towards Vat B, that eventually resolves >> to Carol. >> >> Could Carol be another promise here? Should Alice wait until the target >> is fully resolved before doing a disembargo, or do a disembargo for each >> step? >> > > See above explanation. But no, Carol cannot be a promise, since the only > time that an embargo is triggered is once you get back a locally hosted > capability. > Actually Carol could be a promise -- a promise currently hosted in Vat A. (The original promise, which resolved, was hosted in Vat B, but it resolved to another promise, back in Vat A.) This situation leads to the Tribble 4-way Race Condition, which is described in the Disembargo doc comment. The short version is that once Vat B declares to Vat A that the promise has resolved to Carol, then Vat B must always forward all messages addressed to the promise directly to Carol. Even if Vat A later declares that Carol has further resolved to Dave, Vat B cannot start forwarding messages directly to Dave -- it must keep sending them to Carol. > > >> [Accept] >> > This message is also used to pick up a redirected return -- see >> `Return.redirect`. >> >> `redirect` doesn't appear anyway else in this spec. I guess it's >> `Return.sendResultsTo.thirdParty`. >> > > Probably. It's Level 3, so it's invisible to me. :D > Correct. > > >> >> [ Network-specific Parameters] >> > For interaction over the global internet between parties with no other >> prior arrangement, a >> > particular set of bindings for these types is defined elsewhere. >> (TODO(someday): Specify where >> > these common definitions live.) >> >> Do these definitions exist now? >> > > ¯\_(ツ)_/¯ > They do not. :( I agree the docs should be updated here. It'd be great if someone who is not me wanted to make the changes since other people can probably better-identify which bits are confusing than I can... :) -Kenton -- You received this message because you are subscribed to the Google Groups "Cap'n Proto" group. To unsubscribe from this group and stop receiving emails from it, send an email to capnproto+unsubscr...@googlegroups.com. Visit this group at https://groups.google.com/group/capnproto.
