Thanks Michael, syntax = "proto3";
service FileServer { rpc upload(stream Chunk) returns (Reply) {} rpc download(Request) returns (stream Chunk) {} } message Chunk { bytes buffer = 1; } message Request { string name = 1; } message Reply { int32 length = 1; and my client.py def get_file_chunks(filename): with open(filename, 'rb') as f: while True: piece = f.read(CHUNK_SIZE); if len(piece) == 0: return yield chunk_pb2.Chunk(buffer=piece) def save_chunks_to_file(chunks, filename): with open(filename, 'wb') as f: for chunk in chunks: f.write(chunk.buffer) class FileClient: def __init__(self, address): channel = grpc.insecure_channel(address) self.stub = chunk_pb2_grpc.FileServerStub(channel) def upload(self, in_file_name): chunks_generator = get_file_chunks(in_file_name) response = self.stub.upload(chunks_generator) assert response.length == os.path.getsize(in_file_name) def download(self, target_name, out_file_name): response = self.stub.download(chunk_pb2.Request(name=target_name)) save_chunks_to_file(response, out_file_name) if __name__ == '__main__': client = FileClient('localhost:8888') # demo for file uploading in_file_name = sys.argv[1] client.upload(in_file_name) On Tuesday, April 28, 2020 at 12:13:47 AM UTC+5:30, Michael Webster wrote: > > Do you have any snippets you can share from your code? I use python-grpc > in another project for transferring files and I've never had an issue with > files > 2gb, with 1mb chunks (local network operations). > > On Monday, April 27, 2020 at 7:08:50 PM UTC+1, VigneshDhanraj G wrote: >> >> Thanks Mya, >> >> There is no any error on the server and i am getting error if i upload >> more than 2GB file, i am constanly uploading chunk on the storge. >> >> On Monday, April 27, 2020 at 8:42:50 PM UTC+5:30, Mya Pitzeruse wrote: >>> >>> Is there an associated error on the server? The error message suggests a >>> server error with an index out of bounds exception: 2^31. Only way I could >>> see something like that happening is if the file was being buffered on the >>> server side in memory. >>> >>> On Mon, Apr 27, 2020 at 6:56 AM VigneshDhanraj G <vignesh...@gmail.com> >>> wrote: >>> >>>> Hi Team, >>>> >>>> I am not able to upload large files more than 2GB even after chunk. I >>>> have found the file is uploaded still exeception raised? Please help me to >>>> understand the issue here and way to fix this problem. >>>> >>>> Traceback (most recent call last): >>>> File "demo_client.py", line 12, in <module> >>>> client.upload(in_file_name) >>>> File >>>> "/home/vigneshdhanraj/Project/grpc-upload/grpc-file-transfer/src/lib.py", >>>> line 34, in upload >>>> response = self.stub.upload(chunks_generator) >>>> File >>>> "/home/vigneshdhanraj/Project/grpc-upload/myenv/lib/python3.6/site-packages/grpc/_channel.py", >>>> >>>> line 1011, in __call__ >>>> return _end_unary_response_blocking(state, call, False, None) >>>> File >>>> "/home/vigneshdhanraj/Project/grpc-upload/myenv/lib/python3.6/site-packages/grpc/_channel.py", >>>> >>>> line 729, in _end_unary_response_blocking >>>> raise _InactiveRpcError(state) >>>> grpc._channel._InactiveRpcError: <_InactiveRpcError of RPC that >>>> terminated with: >>>> status = StatusCode.UNKNOWN >>>> details = "*Exception calling application: Value out of range: >>>> 2147483648*" >>>> debug_error_string = >>>> "{"created":"@1587649701.194973268","description":"Error received from >>>> peer >>>> ipv6:[::1]:8888","file":"src/core/lib/surface/call.cc","file_line":1056,"grpc_message":"Exception >>>> >>>> calling application: Value out of range: 2147483648","grpc_status":2}" >>>> >>>> Regards, >>>> VigneshDhanraj G >>>> >>>> -- >>>> You received this message because you are subscribed to the Google >>>> Groups "grpc.io" group. >>>> To unsubscribe from this group and stop receiving emails from it, send >>>> an email to grp...@googlegroups.com. >>>> To view this discussion on the web visit >>>> https://groups.google.com/d/msgid/grpc-io/cbc945c2-6d3a-4d2b-a52f-2a6b59a9dfba%40googlegroups.com >>>> >>>> <https://groups.google.com/d/msgid/grpc-io/cbc945c2-6d3a-4d2b-a52f-2a6b59a9dfba%40googlegroups.com?utm_medium=email&utm_source=footer> >>>> . >>>> >>> >>> >>> -- >>> >>> Mya Pitzeruse >>> >>> Principal Software Engineer - Service Infrastructure >>> >>> Gender Pronouns: She, Her, Hers >>> >>> mjp...@indeed.com >>> >>> >>> Indeed - We help people get jobs. >>> >>> Indeed.com <http://www.indeed.com/> >>> >>> Facebook <http://www.facebook.com/indeed> | Twitter >>> <http://www.twitter.com/indeed> | Instagram >>> <http://www.instagram.com/indeedworks> >>> >> -- You received this message because you are subscribed to the Google Groups "grpc.io" group. To unsubscribe from this group and stop receiving emails from it, send an email to grpc-io+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/grpc-io/e3f6c515-71cd-40ca-bafb-b409788cb1fb%40googlegroups.com.