[grpc-io] Re: long-distance hops - latency doubles when msg exceeds 16kb

['Alon Kaftan' via grpc.io]

HI again
We tcpdump the traffic.
We see the first 16kb frame is sent from the client to the server, but the 
2nd one is not sent until an ack is returned from the server.
In our case, our msg size is 30KB, and round trip latency is ~220ms.
So because of this, call takes ~440ms instead of ~200ms.

We set the InitialWindowSize and InitialConnWindowSize to 1MB, on both 
server and client - no change.
We checked both unary RPCs and Stream RPCs - the same.
We set the WriteBufferSize and ReadBufferSize to zero (write/read directly 
to/from the wire) - 95% latency remained the same, but Avg latency dropped 
by 100ms - not sure why it had this effect.

Again, in all of the conditions above, if we increase to rate of messages 
to more then 2 or 3 per second, the latency drops to 450ms
Looking at http2debug=2 logs, we see it seems that in the higher rate the 
when latency is low, grpc somehow uses a previous opened stream used by 
another RPC to sent the new RPC

Anyone has encountered a similar behaviour. ? 
Anyone that understand GRPC implementation (Go lang) can explain why it 
behaves this why and if there is a way to make it work better ?
Any help would be very much appreciated 
Thanks !


On Wednesday, August 17, 2022 at 4:44:28 PM UTC+3 Alon Kaftan wrote:

> Also, if we run few dummy (small payload) calls per seconds, in parallel 
> to the big payload calls, on the same connection => the latency of the big 
> payload calls is reduced to 250ms as well
> Thoughts ?
>
> On Wednesday, August 17, 2022 at 1:46:20 PM UTC+3 Alon Kaftan wrote:
>
>> Hi
>> we have a go grpc client running in US and a go grpc server on APAC.
>> Both on AWS.
>> on the same connection we make Unary calls
>> when the message size is lower than 16KB, roundtrip latency is 250ms
>> when the message size is crossing the 16KKB, round trip latency jumps to 
>> 450ms
>> when the message size is crossing the 100KB, round trip latency jumps to 
>> 650ms
>>
>> Also, if we increase the rate of the 16KB+ messages from 1/sec to 2/sec 
>> or more, the latency drops to 250ms
>>
>> we ruled out load balancers as we connected the client & server pods 
>> directly and observed the behaviour remains the same
>>
>> Any ideas where to start with this kind of behaviour ?
>>
>> Thanks!
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"grpc.io" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to grpc-io+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/grpc-io/b3e09191-d887-435b-b1e3-d81d34b020b8n%40googlegroups.com.

[grpc-io] Dynamic types using Any in protobuf and go

[Ankur]

1. I have created a generic API using Any field. Clients can define any new 
message in a proto file, and send the data using this API in Any field. The 
client also shares the proto file to the server beforehand.
Input for API:

message Document {
 DocumentMeta meta = 1;
 bytes data = 2;
 google.protobuf.Any details = 3;
}

2. On server-side, I am trying to register the new MessageType from the 
shared .proto file in protoregistry.GlobalTypes. 

tmp_file := filename + "tmp.pb"
cmd := exec.Command("protoc","--descriptor_set_out=" + tmp_file, 
"-I"+src_dir,path.Join(src_dir, 
filename))

protoFile, err := ioutil.ReadFile(tmp_file)
pb_set := new(descriptorpb.FileDescriptorSet)
if err := proto.Unmarshal(protoFile, pb_set); err != nil {
return err
}
pb := pb_set.GetFile()[0]
fd, err := protodesc.NewFile(pb, protoregistry.GlobalFiles)
if err = protoregistry.GlobalFiles.RegisterFile(fd); err != nil {
return err
}
msgType = dynamicpb.NewMessageType(fd.Messages().Get(0))
err = protoregistry.GlobalTypes.RegisterMessage(msgType)

This works fine. Eg: On client side, I created a message named Dog. After 
registering, I print GlobalTypes in server and it lists 
com.test.eventbus.pb.events.Dog

3. From the API input, I convert the Any type to a proto.Message as follows
basic, err := anypb.UnmarshalNew(doc.GetDetails(), proto.UnmarshalOptions{})

UnmarshalNew uses protoregistry.GlobalTypes as default resolver and that is 
why I am registering Dog type in GlobalTypes

4. From the proto.Message type in step 3 (basic), I want to obtain the 
schema of the message. 
msgV1 := proto.MessageV1(basic)
desc, err := desc.LoadMessageDescriptorForMessage(msgV1)

This fails in LoadMessageDescriptorForMessage with error
error interface conversion: *dynamicpb.Message is not desc.protoMessage: 
missing method Descriptor

It seems to me either there is some error in Type registration or 
unmarshalling of actual data. Can someone point out the right way to do 
this?

Regards



-- 
You received this message because you are subscribed to the Google Groups 
"grpc.io" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to grpc-io+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/grpc-io/ad7a12e1-4cc7-435b-9230-6ea3cb7f2a1dn%40googlegroups.com.

[grpc-io] Re: Flutter with GRPC

[Piotr Morgwai Kotarbinski]

I think "full" gRPC is not yet supported in browsers generally due to 
browser platform limitations (this might have changed though: you need to 
verify if this info is up2date). There's a protocol gGRPC-web designed 
exactly to make up for it and it is definitely supported on flutter:  
https://pub.dev/documentation/grpc/latest/grpc_web/grpc_web-library.html

Furthermore, there's also a higher level abstraction wrapper that 
automatically switches between "native" gRPC and gRPC-web depending for 
which platform the build was made:
https://pub.dev/documentation/grpc/latest/grpc_or_grpcweb/grpc_or_grpcweb-library.html

Hope this helps :)

On Wednesday, August 17, 2022 at 8:25:55 PM UTC+7 ricardo.s...@gmail.com 
wrote:

> Hi Google lovers!
> I'm new in all here, and i try to create a example of flutter (on web) 
> with gRPC.
> Unfortunately I notice that gRPC on flutter don't support WEB.
> I'm doing this wrong?
> Why this lib doesn't support WEB?
> https://pub.dev/packages/grpc

-- 
You received this message because you are subscribed to the Google Groups 
"grpc.io" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to grpc-io+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/grpc-io/d78a6ffe-fd4c-4ae6-838e-d7d441fda88dn%40googlegroups.com.

[grpc-io] Help understand DNS resolution between client and Nginx Ingress

[Staniel Yao]

Hello gRPC community,

I am looking for help on understand my grpc client DNS resolution behavior. 
Both service and clients are written in Java and deployed in EKS. 

My architecture is, my grpc client is using a DNS name which could resolve 
to 3 different A records via R53. Then the request will go through NLB -> 
Nginx Ingress Controller (with gRPC enabled) -> my backend. The Nginx is 
able to load balance the request as long as the connection is establish. 

However, since my DNS names could resolve to 3 records, thus 3 different 
NLBs. The DNS could be resolved from 1 NLB to another only during startup, 
or service restart, and any followup switch took long delays (83 mins 
consistently based on my experiment over the last 2 days). I would like to 
find a way to make this interval shorter to make the R53 DNS load balancing 
more balanced.

I read some documents and discussion 
 and it's not helping. I think 
it's because the MAX_CONNECTION_AGE is only affecting the connection 
between Nginx and backend and doesn't affect the client connection to Nginx 
itself. Am I right on this? I also try to tune some Nginx config map value 
such as keep-alive 

 
but it's also not helping.

I also enable debug log on client side but there is no outstanding log 
happened during the switch time. The only thing I noticed is the debug log 
line's ID and IP address changed during the switch point, like this:

*2022-08-21 04:34:38.417 [grpc-default-worker-ELG-1-13] DEBUG 
io.grpc.netty.shaded.io.grpc.netty.NettyClientHandler:214 - [id: id-new, 
L:/x.x.x.x - R:my-service-dns/y.y.y.y:443] INBOUND PING: ack=true 
bytes=1234*

So in summary, I would like to get some suggestion to understand:
1: Why my client switch DNS every other *83* mins? It seems a weird magic 
number as I can not find anything related.
2: What other config could help shorten the interval for triggering DNS 
resolution switch if my client is talking to Nginx Ingress Controller?

Thanks for any suggestion!

-- 
You received this message because you are subscribed to the Google Groups 
"grpc.io" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to grpc-io+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/grpc-io/ef34755e-edae-4e60-9f1a-c71702cf36fen%40googlegroups.com.