Re: [grpc-io] Slow gRPC communication with large file in Python

2022-02-08 Thread Josh Parks
Indeed it does seem to have to do with the M1 mac I'm developing on... I just ran it on my windows box and got the following: (ve-pas) C:\Users\Josh\Code\grpc_min>python grpc_client.py 0.3099939823150635s 18506294 photons in 10086

Re: [grpc-io] Slow gRPC communication with large file in Python

2022-02-08 Thread Josh Parks
Hi all, I'm on a Macbook M1Max (perhaps it's the architecture causing the issue?), have 64 GB of RAM as well... Sorry for the error in the requirements, I used `pyreqs` and that seems to mess up... Running using pyenv virtual environment Also, added cprofile outputs for server and client to

Re: [grpc-io] Slow gRPC communication with large file in Python

2022-02-08 Thread 'Lidi Zheng' via grpc.io
Windows tends to have slower "localhost" transport comparing to Linux. Josh, can you share your deployment setup? The answer in SO is also helpful, streaming calls perform better in terms of throughput. On Tuesday, February 8, 2022 at 10:46:03 AM UTC-8 Richard Belleville wrote: > Josh, > > I d

Re: [grpc-io] Slow gRPC communication with large file in Python

2022-02-08 Thread 'Richard Belleville' via grpc.io
Josh, I don't think I'm able to reproduce with your repo. I'm getting something like 0.2s on my desktop: (venv) rbellevi@rbell:~/Dev/tmp/grpc_min$ python3 grpc_client.py 0.28313207626342773s18506294 photons in 1008640 bins 0.14323067665100098s18506294 photons in 1008640 bins (venv) rbelle

[grpc-io] Slow gRPC communication with large file in Python

2022-02-08 Thread Josh Parks
I'm trying to do a large array transfer (10-50MB) over gRPC in python and it's quite slow (5-10 seconds, both client and server on localhost). I've tried both streaming and unary requests, and they both seem to run slowly. For more details/conversation, here's the stackoveflow question: https: