Re: [grpc-io] gRPC for large data transfer
FWIW, gRPC can support msg size up to 4GB (minus a few bytes) but you may be limited to 2GB or less due to protobuf library limitations. For example, python protobuf plugin default is 64MB which can be increased. See https://github.com/grpc/grpc/issues/19221. On Tuesday, April 21, 2020 at 7:54:05 AM UTC-7 Dean Hiller wrote: > Having worked at Twitter(heavy thrift!!!) I would say it doesn't really > matter as long as you deliver it in pieces typically. I would not deliver > one HUGE object (HUGE is relative) and instead might call N times to > deliver the whole thing. This has huge advantages in slamming it in many > cases and when done right, servers can remain stateless. My 2 cents is > that 80% of your perf issue will be in 20% of the code and there is a good > chance, it's not the protocol unless you have been optimizing your server > for a while. (Donald Knuth - Premature optimization is the root of all > evil) > > On Mon, Apr 20, 2020 at 7:29 PM Philip wrote: > >> I have the large size of request data to be submitted to remote server >> API. (something like machine learning's feature data). >> Will I use thrift or gRPC (protobuf behind it) to implement it for >> better performance/security etc? >> >> Thank you. >> >> -- >> You received this message because you are subscribed to the Google Groups >> "grpc.io" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to grpc-io+u...@googlegroups.com. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/grpc-io/07228c47-81c2-57bb-f196-7fb4588e68ea%40list.199903.xyz >> . >> > -- You received this message because you are subscribed to the Google Groups "grpc.io" group. To unsubscribe from this group and stop receiving emails from it, send an email to grpc-io+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/grpc-io/9dddba95-dae6-4687-99de-bdc0b5a32773%40googlegroups.com.
Re: [grpc-io] gRPC for large data transfer
Having worked at Twitter(heavy thrift!!!) I would say it doesn't really matter as long as you deliver it in pieces typically. I would not deliver one HUGE object (HUGE is relative) and instead might call N times to deliver the whole thing. This has huge advantages in slamming it in many cases and when done right, servers can remain stateless. My 2 cents is that 80% of your perf issue will be in 20% of the code and there is a good chance, it's not the protocol unless you have been optimizing your server for a while. (Donald Knuth - Premature optimization is the root of all evil) On Mon, Apr 20, 2020 at 7:29 PM Philip wrote: > I have the large size of request data to be submitted to remote server > API. (something like machine learning's feature data). > Will I use thrift or gRPC (protobuf behind it) to implement it for > better performance/security etc? > > Thank you. > > -- > You received this message because you are subscribed to the Google Groups " > grpc.io" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to grpc-io+unsubscr...@googlegroups.com. > To view this discussion on the web visit > https://groups.google.com/d/msgid/grpc-io/07228c47-81c2-57bb-f196-7fb4588e68ea%40list.199903.xyz > . > -- You received this message because you are subscribed to the Google Groups "grpc.io" group. To unsubscribe from this group and stop receiving emails from it, send an email to grpc-io+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/grpc-io/CAFMKK546AUfZsYK9KCpQ%2B%2B6-oOOB5hY-P4Objc6QWbvWEjuw%3DQ%40mail.gmail.com.
[grpc-io] gRPC for large data transfer
I have the large size of request data to be submitted to remote server API. (something like machine learning's feature data). Will I use thrift or gRPC (protobuf behind it) to implement it for better performance/security etc? Thank you. -- You received this message because you are subscribed to the Google Groups "grpc.io" group. To unsubscribe from this group and stop receiving emails from it, send an email to grpc-io+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/grpc-io/07228c47-81c2-57bb-f196-7fb4588e68ea%40list.199903.xyz.
