If you're aiming to slot Cap'n Proto into an existing system with existing protocols, you probably want to use Cap'n Proto's serialization layer only, not the RPC layer. In that case it's just a byte blob, so embedding is easy. The RPC layer, OTOH, dictates a lot about the design of your application code, and requires an underlying transport in which messages can be sent in either direction at any time (which often doesn't fit well into existing protocols).
-Kenton On Tue, Jun 13, 2017 at 1:24 PM, Eric Hopper <omnifari...@gmail.com> wrote: > I'm working with some stuff right now in which there is a need for mass > structured data transfer between two processes running on the same machine. > They already have a protocol, and didn't use shared memory out of security > concerns. > > While I'm not in a position to get them to completely rethink the whole > protocol at this time, I can certainly make it a recommendation for the > future. Using memfds with memory sealing (which I will have to read about > in more depth) might well be a good solution in the futrue. > > Unfortunately, there is a need to have some compatibility with a Windows > implementation, but they can just use Cap'n Proto based named pipes or TCP > connections and still be somewhat better off than the TCP-based protocol > they use now. Right now they also move pointers around to point at > structures read in off the wire. Those structures contain no pointers of > course, but still... > > How much of a state machine is Cap'n Protocol internally? Can another > transport be slid in underneath it really easily? How about if that > transport multiplexes between several different senders, each of which is > using Cap'n Proto, but which then have their own frame structure around it > in which those frames may not correspond neatly to protocol units in Cap'n > Proto (i.e. one Cap'n Proto protocol unit might be split between two > frames)? > > > On Tue, Jun 13, 2017 at 12:07 PM, Kenton Varda <ken...@cloudflare.com> > wrote: > >> Indeed, this was a major design goal! >> >> I haven't personally used it this way yet, though I've used it for mmap() >> many times, which has similar properties. >> >> I would like to develop an RPC transport which uses shared memory. It >> would be especially neat if the code could automatically upgrade to shared >> memory whenever it detects that it's communicating over a unix socket. >> >> One interesting hurdle for comms is what happens if you don't trust the >> sending process. If the memory is still shared, they can potentially modify >> the data while you're consuming it. If you make sure to read each bit of >> data no more than once, then in theory no attack is possible -- but I >> haven't yet reviewed the Cap'n Proto library itself to check that the >> pointer validation reads each bit once, which would need to be guaranteed >> before an application could consider relying on this. >> >> Linux's memfds support "sealing" the memory after writing it so that the >> consuming end can be assured that no further changes are occurring. >> However, messages need to be pretty large before this becomes worthwhile -- >> for typical-size messages, an upfront memcpy() will be much faster. >> >> -Kenton >> >> On Tue, Jun 13, 2017 at 11:26 AM, Omnifarious <omnifari...@gmail.com> >> wrote: >> >>> Cap'n Proto seems like it would be ideal for a shared memory >>> communications channel, right down to the relative pointers. Has anybody >>> used it this way? What are the hurdles? >>> >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "Cap'n Proto" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to capnproto+unsubscr...@googlegroups.com. >>> Visit this group at https://groups.google.com/group/capnproto. >>> >> >> > > > -- > Please only send email to this account that you consider public enough to > be published on a public web page. > Eric Hopper -- http://www.omnifarious.org/~hopper/ > > -- > You received this message because you are subscribed to the Google Groups > "Cap'n Proto" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to capnproto+unsubscr...@googlegroups.com. > Visit this group at https://groups.google.com/group/capnproto. > -- You received this message because you are subscribed to the Google Groups "Cap'n Proto" group. To unsubscribe from this group and stop receiving emails from it, send an email to capnproto+unsubscr...@googlegroups.com. Visit this group at https://groups.google.com/group/capnproto.
