I suspect this may be due to long-standing bugs regarding forking a process that has grpc channels. See, e.g. https://github.com/grpc/grpc/issues/18321
On Wed, Feb 26, 2020 at 9:02 AM laxman reddy <[email protected]> wrote: > > Hello Team, > i am using beam for experimenting for my project usecase > i have specific steps for a pipeline. for each step to process i am using do > function in pardo invocation in pipeline > in one of the steps i had implemented multiprocessing implemented > > with out multiprocessing the step is working fine > when i introduce multiprocessing in pardo transform step is blocking execution > > > if use beam supported multiprocessing option vis direct runner number of > workers it is not recognizing my module function > if use multi threading it is again blocking infinitely > > > can you suggest me how to resolve issue with in the pardo do fn > or any other suggestion when multiprocessing > > > > >
