Hi Yonatan, Your understanding is correct. The design takes care of not affecting RPCs in parent process due to FD shutdown in the child's post-fork handlers. Child process will recreate its own connection when a Python object (with a gRPC stub inside) inherited from the parent needs to be used. This should handle the case you described in your post <https://github.com/grpc/grpc/issues/13873>. That was one of the use cases in our mind when deciding how to solve the fork issue. Please give it a try and let us know if you see any issues.
-Srini On Wednesday, October 17, 2018 at 5:26:16 PM UTC-7, Yonatan Zunger wrote: > > > Wow -- I just saw the notes for 16264 > <https://github.com/grpc/grpc/pull/16264>, and that 1.15 now supports > fork() in Python. This is huge and great news! > > I just want to make sure I understand how this change works, and in > particular what the consequences of the shutdown of the core-level gRPC > resources in the child's post-fork handler means. The use case which is > (IMO) most important is where you create some kind of Python object which > has a gRPC stub inside it (e.g., a client object meant to talk to servers), > then fork() (often through use of the core Python multiprocessing library), > and use that object from within the child process as well. (This is > important because the multiprocessing library is the only built-in > parallelization mechanism that doesn't suffer from serialization due to the > GIL) The overhead cost, IIUC, would be essentially a restart of the core > resources, which is roughly equivalent to the cost of closing and reopening > the channel, but *not* of having to reboot all the wrapping objects, like > bigtable clients or whatever. (Which are notoriously slow to start up > because they also want to look up all sorts of metadata from the server > when they boot) > > I could imagine several gotchas here: for example, that the cancellation > of in-flight RPC's by the child during the reboot would also affect RPC's > in flight due to other threads, meaning that the client object has to be > entirely idle during the fork process. > > Am I understanding the new change correctly? What are the intended use > cases that it's meant to unlock? > > Yonatan > -- You received this message because you are subscribed to the Google Groups "grpc.io" group. To unsubscribe from this group and stop receiving emails from it, send an email to grpc-io+unsubscr...@googlegroups.com. To post to this group, send email to grpc-io@googlegroups.com. Visit this group at https://groups.google.com/group/grpc-io. To view this discussion on the web visit https://groups.google.com/d/msgid/grpc-io/d7279410-69ab-4515-aa73-4d9b30cbcc73%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
