On Sun, Jul 5, 2020 at 1:48 PM Vitali Lovich <vlov...@gmail.com> wrote:
> Yeah, so I'm still working through getting executeSync working. I do wish > executeAsync would still be able to do the right thing when invoked from > off-thread to avoid creating artificial synchronization points, but I can > see it being challenging to accomplish something like that without adding > synchronization into the happy path too. Basically the challenges I'm > having is trouble terminating the event loop of a server. I *think* the > answer I've stumbled upon is to wrap any futures I schedule with a > cancellation object that I then cancel when I go to shut down the thread. > I'm now seeing a crash on the non shutdown path though that I'm trying to > figure out. > Make sure you're using Executor::addRef() to take a strong reference from the calling thread. Then if the receiving thread quits, the calling thread should get a DISCONNECTED exception, but shouldn't crash. FWIW I think we *could* easily have something like execAsync() that doesn't require the caller to be a KJ thread -- but the lambda (and all its owned captures) would be destroyed in the receiving thread, and the calling thread wouldn't get any notification of completion (it would have to implement its own notification if needed). But that's probably fine for most use cases. > I couldn't figure out how to get the message passing going. None of > the I/O facilities in KJ look like they are amenable to having one end not > connected to the event loop that it was created on. If you have any tips on > how to make this work I'd definitely be interested in learning more. It > also wasn't clear from the documentation (which I may have overlooked) if > I/O objects created on 1 thread's loop can have one end living on another > loop and moreover how to adapt them back into a fashion that would be > friendly to not being on an event loop at all. For example, some of the > documentation around capability transfers says it doesn't work on Windows > but then other pieces seem to imply it might (maybe it's only if I'm trying > to . > Well, it's true you can't use newTwoWayPipe() because it returns a pair of KJ objects which can't transfer between threads. But if you create your own OS pipe and then pass the file descriptors (or HANDLEs) to LowLevelAsyncIoProvider to wrap them, then you can certainly wrap one end in the KJ thread and do whatever you want with the other end. Alternatively you could have the KJ thread listen for network connections on a loopback port, and then connect to it from another thread. Windows unfortunately doesn't have socketpair(), which makes these things a lot easier. > Re #3, that I too couldn't figure out. AsyncIoStream is a bit amenable to > that because you can shutdown off-thread since it's a syscall that doesn't > return a promise, but that doesn't work for the other ones that don't have > a way of closing the FD or changing the FD to be blocking so that it can be > used off-thread. > > I did actually do #4 but where I got stuck is that I still ended up stuck > at step 1 where I wasn't sure how to bridge a native KJ thread & a non-KJ > thread. Additionally, various facilities within KJ don't play well with (I > *think*) with having a separate event loop (eg. > kj::getCurrentThreadExecutor() IIRC) & I couldn't get it quite to work well > (+ various facilities like setupIoAsync aren't available & a bunch of I/O > code seemingly needs to be reimplemented from scratch to get parity on > certain things). There's also a lot of KJ re-implemented in the nodeJS > piece in addition to the nodeJS integration so it's a bit hard to at a > glance figure out which is the loop adaption pieces needed & which are just > for bridging with v8 (at least, not without spending a lot of time to fully > groking that file). > Unfortunately, you do in fact have to re-implement AsyncIoStream if you use an alternate event loop, because you have to build it on that alternate event loop's own facilities for async socket I/O. > > If there's any improvements to KJ/capnp you have a semi clear idea on how > to architect to improve this area of the code, I'm happy to discuss it with > you offline & work on a patch set for KJ/capnp. I've noticed it's a > perineal pain point & solving that (& documenting it) would probably sand > off some of the rough edges for new people. Might be a key improvement to > land for 1.0. > > On Sun, Jul 5, 2020 at 10:18 AM Kenton Varda <ken...@cloudflare.com> > wrote: > >> I posted an answer on Stack Overflow. >> >> On Fri, Jul 3, 2020 at 10:47 PM Vitali Lovich <vlov...@gmail.com> wrote: >> >>> I'm trying to integrate cap'n'proto with surrounding threads that don't >>> use KJ event loops. On Android it's easier because I have an event loop to >>> adapt to with ALooper (albeit it's crashing - details in the SO post). >>> >>> For my Windows codebase I don't really have anything in the way of an >>> event loop. Would it be a good idea to create a simple event loop >>> implementation that just has a condition variable + a vector of callbacks >>> to invoke & adapt it in a similar way that I did with the ALooper? Will I >>> need more? >>> >>> More details here: >>> >>> https://stackoverflow.com/questions/62667061/how-to-integrate-capnproto-threads-with-non-capnproto-threads >>> >>> >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "Cap'n Proto" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to capnproto+unsubscr...@googlegroups.com. >>> To view this discussion on the web visit >>> https://groups.google.com/d/msgid/capnproto/e4d62be1-d478-47f4-83bf-6b715237c414n%40googlegroups.com >>> <https://groups.google.com/d/msgid/capnproto/e4d62be1-d478-47f4-83bf-6b715237c414n%40googlegroups.com?utm_medium=email&utm_source=footer> >>> . >>> >> -- You received this message because you are subscribed to the Google Groups "Cap'n Proto" group. To unsubscribe from this group and stop receiving emails from it, send an email to capnproto+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/capnproto/CAJouXQkznOE6Xu52gApkX8pu9hUxr0A3j0%2BV5keqL2HX8C97VQ%40mail.gmail.com.
