gRPC Python server spawns worker threads. Each incoming request will be routed to one of the worker. The main thread is expected to be blocked via the `wait_for_termination` call.
On the other hand, I have little expertise about HPC cluster. I'm not sure what is blocking here. I wonder if you have access to debuggers like gdb, then you can peek into what function it is executing last. On Friday, October 1, 2021 at 8:29:29 AM UTC-7 lee ming wrote: > > Hi, > > I am running into a really odd issue where the following line (obtained > from hello world's greeter_server.py example) freezes when it is executed > on a HPC cluster. > > server = grpc.server(futures.ThreadPoolExecutor(max_workers=10)) > > The following stack trace could provide more information on where the > greeter_server program gets stuck within the gRPC Python library: > > Thread 204411 (active): "MainThread" > __init__ (grpc/_server.py:958) > create_server (grpc/_server.py:1003) > server (grpc/__init__.py:2064) > serve (greeter_server.py:31) > <module> (greeter_server.py:40) > > (Apologies for the poor formatting) > > The weird thing about this issue is that > > - The hello world example (greeter_server + greeter_client) works fine > locally. > - I attempted to debug by adding the "GRPC_VERBOSITY=DEBUG > GRPC_TRACE=all" flag (and confirmed it works locally), but nothing > gets printed out while running on the HPC cluster. Furthermore, no > exception is thrown, rather the program simply freezes with the above > stack > trace. > > I'm wondering if anyone has run into this issue before. I have spoken to > the administrator of the HPC cluster and as far as I know there are no > specific restrictions from preventing gRPC servers from being hosted. > > Thanks in advance! > > -- You received this message because you are subscribed to the Google Groups "grpc.io" group. To unsubscribe from this group and stop receiving emails from it, send an email to grpc-io+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/grpc-io/ab9b059c-7532-4e63-92a8-b4436a6c0ef8n%40googlegroups.com.
