The maximum value of an int32 is 2,147,483,647. Why are you using a signed
type for length in the first place though?
On Mon, Apr 27, 2020 at 12:01 PM VigneshDhanraj G <
vigneshdhanra...@gmail.com> wrote:
> Thanks Michael,
>
> syntax = "proto3";
>
> service FileServer {
> rpc upload(stream Chunk) returns (Reply) {}
> rpc download(Request) returns (stream Chunk) {}
> }
>
> message Chunk {
> bytes buffer = 1;
> }
>
> message Request {
> string name = 1;
> }
>
> message Reply {
> int32 length = 1;
>
> and my client.py
>
>
> def get_file_chunks(filename):
> with open(filename, 'rb') as f:
> while True:
> piece = f.read(CHUNK_SIZE);
> if len(piece) == 0:
> return
> yield chunk_pb2.Chunk(buffer=piece)
>
>
> def save_chunks_to_file(chunks, filename):
> with open(filename, 'wb') as f:
> for chunk in chunks:
> f.write(chunk.buffer)
>
>
> class FileClient:
> def __init__(self, address):
> channel = grpc.insecure_channel(address)
> self.stub = chunk_pb2_grpc.FileServerStub(channel)
>
> def upload(self, in_file_name):
> chunks_generator = get_file_chunks(in_file_name)
> response = self.stub.upload(chunks_generator)
> assert response.length == os.path.getsize(in_file_name)
>
> def download(self, target_name, out_file_name):
> response = self.stub.download(chunk_pb2.Request(name=target_name))
> save_chunks_to_file(response, out_file_name)
>
>
> if __name__ == '__main__':
> client = FileClient('localhost:8888')
>
> # demo for file uploading
> in_file_name = sys.argv[1]
> client.upload(in_file_name)
>
>
>
>
> On Tuesday, April 28, 2020 at 12:13:47 AM UTC+5:30, Michael Webster wrote:
>>
>> Do you have any snippets you can share from your code? I use python-grpc
>> in another project for transferring files and I've never had an issue with
>> files > 2gb, with 1mb chunks (local network operations).
>>
>> On Monday, April 27, 2020 at 7:08:50 PM UTC+1, VigneshDhanraj G wrote:
>>>
>>> Thanks Mya,
>>>
>>> There is no any error on the server and i am getting error if i upload
>>> more than 2GB file, i am constanly uploading chunk on the storge.
>>>
>>> On Monday, April 27, 2020 at 8:42:50 PM UTC+5:30, Mya Pitzeruse wrote:
>>>>
>>>> Is there an associated error on the server? The error message suggests
>>>> a server error with an index out of bounds exception: 2^31. Only way I
>>>> could see something like that happening is if the file was being buffered
>>>> on the server side in memory.
>>>>
>>>> On Mon, Apr 27, 2020 at 6:56 AM VigneshDhanraj G <vignesh...@gmail.com>
>>>> wrote:
>>>>
>>>>> Hi Team,
>>>>>
>>>>> I am not able to upload large files more than 2GB even after chunk. I
>>>>> have found the file is uploaded still exeception raised? Please help me to
>>>>> understand the issue here and way to fix this problem.
>>>>>
>>>>> Traceback (most recent call last):
>>>>> File "demo_client.py", line 12, in <module>
>>>>> client.upload(in_file_name)
>>>>> File
>>>>> "/home/vigneshdhanraj/Project/grpc-upload/grpc-file-transfer/src/lib.py",
>>>>> line 34, in upload
>>>>> response = self.stub.upload(chunks_generator)
>>>>> File
>>>>> "/home/vigneshdhanraj/Project/grpc-upload/myenv/lib/python3.6/site-packages/grpc/_channel.py",
>>>>> line 1011, in __call__
>>>>> return _end_unary_response_blocking(state, call, False, None)
>>>>> File
>>>>> "/home/vigneshdhanraj/Project/grpc-upload/myenv/lib/python3.6/site-packages/grpc/_channel.py",
>>>>> line 729, in _end_unary_response_blocking
>>>>> raise _InactiveRpcError(state)
>>>>> grpc._channel._InactiveRpcError: <_InactiveRpcError of RPC that
>>>>> terminated with:
>>>>> status = StatusCode.UNKNOWN
>>>>> details = "*Exception calling application: Value out of range:
>>>>> 2147483648*"
>>>>> debug_error_string =
>>>>> "{"created":"@1587649701.194973268","description":"Error received from
>>>>> peer
>>>>> ipv6:[::1]:8888","file":"src/core/lib/surface/call.cc","file_line":1056,"grpc_message":"Exception
>>>>> calling application: Value out of range: 2147483648","grpc_status":2}"
>>>>>
>>>>> Regards,
>>>>> VigneshDhanraj G
>>>>>
>>>>> --
>>>>> You received this message because you are subscribed to the Google
>>>>> Groups "grpc.io" group.
>>>>> To unsubscribe from this group and stop receiving emails from it, send
>>>>> an email to grp...@googlegroups.com.
>>>>> To view this discussion on the web visit
>>>>> https://groups.google.com/d/msgid/grpc-io/cbc945c2-6d3a-4d2b-a52f-2a6b59a9dfba%40googlegroups.com
>>>>> <https://groups.google.com/d/msgid/grpc-io/cbc945c2-6d3a-4d2b-a52f-2a6b59a9dfba%40googlegroups.com?utm_medium=email&utm_source=footer>
>>>>> .
>>>>>
>>>>
>>>>
>>>> --
>>>>
>>>> Mya Pitzeruse
>>>>
>>>> Principal Software Engineer - Service Infrastructure
>>>>
>>>> Gender Pronouns: She, Her, Hers
>>>>
>>>> mjp...@indeed.com
>>>>
>>>>
>>>> Indeed - We help people get jobs.
>>>>
>>>> Indeed.com <http://www.indeed.com/>
>>>>
>>>> Facebook <http://www.facebook.com/indeed> | Twitter
>>>> <http://www.twitter.com/indeed> | Instagram
>>>> <http://www.instagram.com/indeedworks>
>>>>
>>> --
> You received this message because you are subscribed to the Google Groups "
> grpc.io" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to grpc-io+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/grpc-io/e3f6c515-71cd-40ca-bafb-b409788cb1fb%40googlegroups.com
> <https://groups.google.com/d/msgid/grpc-io/e3f6c515-71cd-40ca-bafb-b409788cb1fb%40googlegroups.com?utm_medium=email&utm_source=footer>
> .
>
--
You received this message because you are subscribed to the Google Groups
"grpc.io" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to grpc-io+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/grpc-io/CACV55uyS%3DMWkZD2BXRXWk1vw3e2gq%2BT0PdxB3PdKMg0PqLKuPA%40mail.gmail.com.
