Re: [capnproto] Xcode hits SIGUSR1 signal when I use an Executor
Hi Jens, Two options for writing sync-like code that is actually async: - KJ supports "Fibers", which allocates a separate stack that runs in the same thread. When running on that stack, you get a `WaitScope` that you can use to wait on promises. When you wait, the thread switches back to the main stack. - We have some support for C++20 coroutines using co_await/co_return (aka async/await). However, it's currently based on `-fcoroutines-ts` in Clang; it may not work with any other compiler, and this flag is going to go away soon, though we'll likely upgrade to the final version of coroutines before that happens. -Kenton On Wed, Aug 31, 2022 at 12:29 PM Jens Alfke wrote: > > > > On Aug 30, 2022, at 7:37 PM, 'Kenton Varda' via Cap'n Proto < > capnproto@googlegroups.com> wrote: > > > > Hi Jens, > > > > It sounds very obnoxious that your debugger insists on breaking on this > signal. > > I’ve filed a bug report with Apple. > > > With that said, you could try compiling with `-DKJ_USE_PIPE_FOR_WAKEUP`, > which causes it to avoid using signals for this. You need to compile both > KJ itself and your own code that depends on it with this define; if they > don't match you may get undefined behavior. > > For posterity: since I’m building with CMake, I accomplished this by > adding `add_compile_definitions(KJ_USE_PIPE_FOR_WAKEUP)` to my top-level > CMakeLists.txt, above the line `add_subdirectory(vendor/capnproto)`. > > Now my code can talk inter-thread RPC over the fake socket! Yay! > > But of course there’s always another roadblock. The next one I hit is a > fatal exception "expected !loop.running; wait() is not allowed from within > event callbacks.” Apparently code running in an Executor block on the > target thread is not allowed to call Promise.wait()? This is a bummer, as > so far I’ve been lazy and written my RPC client code in blocking style. > Looks like it’s time to fully async-ify it. > > —Jens -- You received this message because you are subscribed to the Google Groups "Cap'n Proto" group. To unsubscribe from this group and stop receiving emails from it, send an email to capnproto+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/capnproto/CAJouXQnMWk2ZROUhaVzr14YnM9yKghdjA1%3Dy7pN94DkkMkWGiQ%40mail.gmail.com.
Re: [capnproto] Xcode hits SIGUSR1 signal when I use an Executor
> On Aug 30, 2022, at 7:37 PM, 'Kenton Varda' via Cap'n Proto > wrote: > > Hi Jens, > > It sounds very obnoxious that your debugger insists on breaking on this > signal. I’ve filed a bug report with Apple. > With that said, you could try compiling with `-DKJ_USE_PIPE_FOR_WAKEUP`, > which causes it to avoid using signals for this. You need to compile both KJ > itself and your own code that depends on it with this define; if they don't > match you may get undefined behavior. For posterity: since I’m building with CMake, I accomplished this by adding `add_compile_definitions(KJ_USE_PIPE_FOR_WAKEUP)` to my top-level CMakeLists.txt, above the line `add_subdirectory(vendor/capnproto)`. Now my code can talk inter-thread RPC over the fake socket! Yay! But of course there’s always another roadblock. The next one I hit is a fatal exception "expected !loop.running; wait() is not allowed from within event callbacks.” Apparently code running in an Executor block on the target thread is not allowed to call Promise.wait()? This is a bummer, as so far I’ve been lazy and written my RPC client code in blocking style. Looks like it’s time to fully async-ify it. —Jens -- You received this message because you are subscribed to the Google Groups "Cap'n Proto" group. To unsubscribe from this group and stop receiving emails from it, send an email to capnproto+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/capnproto/6A1BA18F-C625-4486-8A71-166D2E6A9268%40mooseyard.com.
Re: [capnproto] Xcode hits SIGUSR1 signal when I use an Executor
Hi Jens, Indeed, KJ uses SIGUSR1 to communicate between threads. It sounds very obnoxious that your debugger insists on breaking on this signal. The signal is not a problem, it's a normal part of KJ's operation. With that said, you could try compiling with `-DKJ_USE_PIPE_FOR_WAKEUP`, which causes it to avoid using signals for this. You need to compile both KJ itself and your own code that depends on it with this define; if they don't match you may get undefined behavior. `kj::newTwoWayPipe()` (the global function) is a completely in-memory implementation of TwoWayPipe which is tied to a single thread. If you use `kj::AsyncIoProvider::newTwoWayPipe()` instead, it will create an implementation backed by a socketpair(). However, unfortunately, the KJ AsyncIoStream wrapper objects are still tied to the particular thread that created them. In order to communicate between threads, you will need to manually create a socketpair(), and then use kj::LowLevelAsyncIoProvider::wrapSocketFd() to create the AsyncIoStream wrappers. Each thread would need to use its own LowLevelAsyncIoProvider object to create the AsyncIoStream for its end of the pipe. Sorry, there isn't currently a more-convenient way to do this. -Kenton On Tue, Aug 30, 2022 at 7:38 PM Jens Alfke wrote: > TL;DR: Can the two streams created by kj::newTwoWayPipe() be used on > different threads? It kind of appears not. > > I’ve found a workaround, the LLDB command > process handle --stop false SIGUSR1 > Unfortunately adding it to my .lldbrc file does nothing; I have to enter > it by hand every time I start the process. > > The next roadblock is that my unit tests create a client and a server > object, then connect them together by calling kj::newTwoWayPipe() and > giving one end of the pipe to each. This worked fine in a single thread. > However, now one end (an AsyncIoStream) gets passed into the new background > thread where the client object lives. I get an exception > "expected threadLocalEventLoop == || threadLocalEventLoop == > nullptr; Event armed from different thread than it was created in. You > must use Executor to queue events cross-thread.” > > From this and the backtrace it looks as though when I write to this end of > the pipe, it wants to directly notify the other end, which won’t work > because it’s the wrong thread for that. I was hoping that the streams would > use normal Unix I/O, since the comment about the TwoWayPipe class says > "Typically backed by socketpair() system call”. > > So how do I set up streams to do I/O between threads? > > —Jens > > -- > You received this message because you are subscribed to the Google Groups > "Cap'n Proto" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to capnproto+unsubscr...@googlegroups.com. > To view this discussion on the web visit > https://groups.google.com/d/msgid/capnproto/EA5DCF15-8128-4F65-AE98-708AE1B75D1E%40mooseyard.com > . > -- You received this message because you are subscribed to the Google Groups "Cap'n Proto" group. To unsubscribe from this group and stop receiving emails from it, send an email to capnproto+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/capnproto/CAJouXQnRhZViMA7rYbqZP94iw_EV-szgSLafZSnsZu6%3DKZEsxA%40mail.gmail.com.
Re: [capnproto] Xcode hits SIGUSR1 signal when I use an Executor
TL;DR: Can the two streams created by kj::newTwoWayPipe() be used on different threads? It kind of appears not. I’ve found a workaround, the LLDB command process handle --stop false SIGUSR1 Unfortunately adding it to my .lldbrc file does nothing; I have to enter it by hand every time I start the process. The next roadblock is that my unit tests create a client and a server object, then connect them together by calling kj::newTwoWayPipe() and giving one end of the pipe to each. This worked fine in a single thread. However, now one end (an AsyncIoStream) gets passed into the new background thread where the client object lives. I get an exception "expected threadLocalEventLoop == || threadLocalEventLoop == nullptr; Event armed from different thread than it was created in. You must use Executor to queue events cross-thread.” >From this and the backtrace it looks as though when I write to this end of the >pipe, it wants to directly notify the other end, which won’t work because it’s >the wrong thread for that. I was hoping that the streams would use normal Unix >I/O, since the comment about the TwoWayPipe class says "Typically backed by >socketpair() system call”. So how do I set up streams to do I/O between threads? —Jens -- You received this message because you are subscribed to the Google Groups "Cap'n Proto" group. To unsubscribe from this group and stop receiving emails from it, send an email to capnproto+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/capnproto/EA5DCF15-8128-4F65-AE98-708AE1B75D1E%40mooseyard.com.
