Yeah, C++ error messages are the worst. :( You kind of have to get used to knowing that "can't convert" errors are usually because you forgot to kj::mv...
-Kenton On Tue, Nov 30, 2021 at 10:08 AM pepijn de vos <pepijnde...@gmail.com> wrote: > Ah I solved it... by not storing sim in a variable. Or probably kj::mv > would have done the job too. > It's kinda unfortunate the error you get just tells you it's the wrong > type, rather than.. hey you should move the thing. > Onwards!! > Pepijn > > On Tue, Nov 30, 2021 at 4:50 PM pepijn de vos <pepijnde...@gmail.com> > wrote: > >> Hi Kenton, >> >> I got back to this problem and tried implementing the fork idea but ran >> into another silly problem where the C++ templates are smarter than me it >> seems. >> You recommended to not use the EzRPC server, but there is a great lack of >> examples of other ways to go about it. >> >> I found a Rust example that uses the low-level API but of course not >> exactly the C++ one: >> https://github.com/capnproto/capnproto-rust/blob/master/capnp-rpc/examples/calculator/server.rs >> I found some code that's copy-pasted all over using github search that >> seems to use the lower level API: >> https://github.com/Nickan/nickan.github.io/blob/b0eb31c33a3b6720606974f9576e663f3b7852ca/drawio/etc/sandstorm/server.c%2B%2B#L420-L422 >> >> However, I cannot quite get it to work. >> >> kj::AsyncIoContext ctx = kj::setupAsyncIo(); >> auto stream = ctx.lowLevelProvider->wrapSocketFd(clientFd); >> auto network = capnp::TwoPartyVatNetwork(*stream, capnp::rpc::twoparty:: >> Side::CLIENT); >> kj::Own<kj::Filesystem> fs = kj::newDiskFilesystem(); >> const kj::Directory &dir = fs->getCurrent(); >> >> auto sim = kj::heap<SimulatorImpl>(dir); >> auto rpc = capnp::makeRpcServer(network, sim); >> >> So I set up asyncio, wrap my unix socket FD into a stream, use that to >> create a network (client or server??) >> The problem is the call to makeRpcServer. SimulatorImpl is a ::Server >> subclass. >> This seems to match the random copy-pasted example, but gives compile >> errors that it can't convert sim to whatever is expected. >> I tried reading the source code, which tells me it expects a restorer or >> a bootstrap, but the restorer is deprecated, but also what EzRPC seems to >> be using. >> I tried looking at the tests but that appears to do the same thing, just >> pass it an owned instance of the implementation?? >> >> https://github.com/capnproto/capnproto/blob/e5af75ff0513e6d971fa7b48e8108427766e51f9/c%2B%2B/src/capnp/rpc-test.c%2B%2B#L1273-L1279 >> I am at a loss. >> >> Pepijn >> >> >> On Thu, May 6, 2021 at 8:22 PM Kenton Varda <ken...@cloudflare.com> >> wrote: >> >>> I don't recommend using EzRPC. That interface turns out to be too >>> restrictive. >>> >>> To set up a KJ event loop, use kj::setupAsyncIo(). >>> >>> Then to initiate or accept Cap'n Proto RPC connections, use >>> capnp::TwoPartyClient / capnp::TwoPartyServer. >>> >>> For FD passing, create a capnp server object that overrides the virtual >>> method `kj::Maybe<int> Capability::Server::getFd()` to return an FD number. >>> Then on the Client objects pointing to that server, you can call `getFd()` >>> to get that file descriptor -- even from another process, as long as the >>> connection between the two is a unix domain socket. >>> >>> -Kenton >>> >>> On Thu, May 6, 2021 at 1:07 PM pepijn de vos <pepijnde...@gmail.com> >>> wrote: >>> >>>> Thanks for the suggestions. >>>> This is still for the simulation server, so each connection is pretty >>>> expensive anyway, and it's unlikely there will be more in flight than the >>>> machine has cores available. It might even make sense to have a pool as you >>>> suggest, and just reject connections if there are no cores available. >>>> Concern is that you need to supervise these pools in case they crash. With >>>> fork you can just launch a new process. Any pointers to useful functions >>>> for this kinds of hacks would be appreciated. I don't have a clue how I'd >>>> go about obtaining and sending file descriptors over RPC and making new >>>> servers out of them. I should probably start by studying EzRPC and >>>> kj::LowLevelAsyncIoProvider. >>>> >>>> Pepijn >>>> >>>> On Thu, May 6, 2021 at 7:48 PM Kenton Varda <ken...@cloudflare.com> >>>> wrote: >>>> >>>>> Ohhh, sorry, I misread. If you haven't started the KJ event loop until >>>>> after the fork, you are probably OK. >>>>> >>>>> You can definitely construct a KJ socket from a raw socket using >>>>> kj::LowLevelAsyncIoProvider. >>>>> >>>>> But if you're OK with the entire connection being handled in the >>>>> isolated process, there are some other options too: >>>>> - Have a pool of processes that are all waiting to accept from the >>>>> same file descriptor. Each connection will only be accepted by one of the >>>>> processes. Make sure that process doesn't accept a new connection until >>>>> the >>>>> old connection is done. >>>>> - Cap'n Proto supports sending file descriptors between processes in >>>>> RPC messages. So you could have one process that is accepting connections, >>>>> but each time it does, it sends the file descriptor to some other >>>>> processes >>>>> via an RPC. Then that process actually handles the connection. >>>>> >>>>> These options are both more complicated than the simple fork() >>>>> approach you mentioned, so aren't necessarily better. But fork() is pretty >>>>> expensive so it might turn out reusing processes has some performance >>>>> benefit. >>>>> >>>>> -Kenton >>>>> >>>>> On Thu, May 6, 2021 at 12:39 PM pepijn de vos <pepijnde...@gmail.com> >>>>> wrote: >>>>> >>>>>> Hi Kenton, >>>>>> >>>>>> Agree forking an eventloop will lead to disaster, which is why I >>>>>> suggested having a blocking accept loop, and then fork and start the >>>>>> event >>>>>> loop in the new process. But I'm not sure if it's at all possible to >>>>>> convert a raw unix socket to a jk socket and then give it to an eventloop >>>>>> to handle the connection. >>>>>> >>>>>> RPC to a process certainly requires less hacks. >>>>>> >>>>>> Pepijn >>>>>> >>>>>> On Thu, May 6, 2021 at 7:35 PM Kenton Varda <ken...@cloudflare.com> >>>>>> wrote: >>>>>> >>>>>>> Hi Pepijn, >>>>>>> >>>>>>> Yeah, forking won't work well. I don't recommend that. One problem >>>>>>> is that if you fork() with an event loop running, both the parent and >>>>>>> child >>>>>>> processes will end up with the same epoll FD and so will receive each >>>>>>> other's events, which will likely lead to chaos. >>>>>>> >>>>>>> So I recommend spawning a subprocess and talking to it using RPC >>>>>>> over a unix socket. Yes, it adds some overhead, but really not that >>>>>>> much. >>>>>>> >>>>>>> -Kenton >>>>>>> >>>>>>> On Thu, May 6, 2021 at 2:39 AM pepij...@gmail.com < >>>>>>> pepijnde...@gmail.com> wrote: >>>>>>> >>>>>>>> Hey, >>>>>>>> >>>>>>>> I'm writing an API for some shared libraries that are written with >>>>>>>> a "binary" use in mind. >>>>>>>> They use a ton of global state and are not thread safe. >>>>>>>> Of course as a server, multiple users could try to use the server >>>>>>>> at the same time, these libraries can't deal with that. What's more >>>>>>>> they >>>>>>>> also like to segfault sometimes. >>>>>>>> >>>>>>>> So what I'd like to do is have these shared libraries completely in >>>>>>>> their own address space. >>>>>>>> What would be the best way to do that? >>>>>>>> >>>>>>>> One thing I thought could work is have a blocking accept loop, and >>>>>>>> then fork the handling of the connection. Not sure how this would work >>>>>>>> with >>>>>>>> kj. >>>>>>>> >>>>>>>> The other solution would be that you launch a process in the method >>>>>>>> handler that runs its own capnp server, and then proxy calls to it. >>>>>>>> Seems >>>>>>>> like it adds more overhead. >>>>>>>> >>>>>>>> Regards, >>>>>>>> Pepijn >>>>>>>> >>>>>>>> -- >>>>>>>> You received this message because you are subscribed to the Google >>>>>>>> Groups "Cap'n Proto" group. >>>>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>>>> send an email to capnproto+unsubscr...@googlegroups.com. >>>>>>>> To view this discussion on the web visit >>>>>>>> https://groups.google.com/d/msgid/capnproto/99fcf2cf-e6e4-4918-8734-22b7f85cfb85n%40googlegroups.com >>>>>>>> <https://groups.google.com/d/msgid/capnproto/99fcf2cf-e6e4-4918-8734-22b7f85cfb85n%40googlegroups.com?utm_medium=email&utm_source=footer> >>>>>>>> . >>>>>>>> >>>>>>> -- You received this message because you are subscribed to the Google Groups "Cap'n Proto" group. To unsubscribe from this group and stop receiving emails from it, send an email to capnproto+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/capnproto/CAJouXQmwXf51NRjT2pL0x4do%2B8Ur3C7jFiMURYsLmi8SqARkGA%40mail.gmail.com.
