So I'm running into some weird issues with configuring keep alive. I have
java based grpc client and server running in same namespace. Each has their
own istio proxy running as side car.
Firstly when I look into tcp dump, I can see PINGS[0] from client to server
but no PING[0] with ACK set back. The connections is not getting reset
however. Even if the rate is less than keepalivepermit time im not seeing
any GO AWAY[0]. I filtred out TCP type and checked all other packets. This
packet was took from all of the interface from the pod like this
kubectl exec -n namespace -i podname -- tcpdump -i any -U -s0 -w - |
wireshark -k -i -
Later i enabled trace logs of io.grpc. In that I can see the below in
client logs
│ {"t":"2025-09-28T05:23:13.521Z","msg":"[id: 0x1ac0fbdb,
L:/10.XX.XX.XX:55332 -
R:grpc-server-dev.svc.cluster.local/172.XX.XX.XX:6868] OUTBOUND PING:
ack=false
bytes=1111","lgr":"io.grpc.netty.shaded.io.grpc.netty.NettyClientHandler","trd":"gr
│
│ pc-default-worker-ELG-1-2","lvl":"DEBUG"} │
│ {"t":"2025-09-28T05:23:13.521Z","msg":"[id: 0x1ac0fbdb,
L:/10.XX.XX.XX:55332 -
R:grpc-serve-dev.svc.cluster.local/172.XX.XX.XX:6868] INBOUND PING:
ack=true
bytes=1111","lgr":"io.grpc.netty.shaded.io.grpc.netty.NettyClientHandler","trd":"grpc
│
│ -default-worker-ELG-1-2","lvl":"DEBUG"}
To my surprise I can see in client logs the INBOUNG and OUTBOUND PING with
its ACK. This means my client is getting ack for its PINGS. I still haven't
found out why i didnt get this in my TCP dump
BUT I dont see the reverse of this in my server logs. It has also io.grpc
trace logs enabled. I don't see the INBOUND PING with ack=false and
OUTBOUND PING with ack=true in my server logs
And I don't see GO_AWAY also if permitkeepalive threshold is greater than
my keepalive time.
This setup completely works fine when I run these two services in my local
machine. Everything works fine. However in my cluster im facing these weird
issues. Any help would be greatly appreciated, I'm breaking my head for a
week now trying to find out what is happening
--
You received this message because you are subscribed to the Google Groups
"grpc.io" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion visit
https://groups.google.com/d/msgid/grpc-io/792b773f-d68d-4032-b77b-7ef7bffd9fd3n%40googlegroups.com.
