Recently I found my HTTP server close connection by RST/ACK instead of FIN,
I've searched for days but haven't find reason. I'm asking for advices for
solve this problem.
1. In which situations the HTTP server will use RST/ACK to close connection
instead of FIN in Go?
2. How can I force the server use FIN, not RST/ACK?
*Appearance*
I have a HTTP server running on a CentOS Linux server. Recently I found
sometimes the HTTP server close the connection with RST/ACK instead of
FIN/ACK, the client side didn't handle the RST/ACK packet in the right way,
it cause the whole service is not stable enough.
*Information maybe important*
Service logic: my service handler will check if there is new data for the
client request, if yes, it will return new data immediately, otherwise it
will sleep 5 seconds then respond a timestamp. I found that when the server
run into the sleep branch, sometimes it will sent RST/ACK instead of
FIN/ACK.
Go version: 1.13
Framework: gin (But I think this is not the reason 'cause gin is too high
level)
Radio: About 15%, seems not relate to the QPS of server.
*TCPDUMP*
I use `gopacket` instead of `tcpdump -r` for better displaying and
understanding.
I use 10.10.10.10 as the service side, and 192.168.123.123 as the client
side.
[SYN] PACKET: 74 bytes, wire length 74 cap length 74 @ 2020-12-29
16:33:57.229551 +0800 CST
- Layer 1 (14 bytes) = Ethernet {Contents=[..14..] Payload=[..60..]
SrcMAC=98:03:9b:84:17:8a DstMAC=7c:d3:0a:69:53:dd EthernetType=IPv4
Length=0}
- Layer 2 (20 bytes) = IPv4 {Contents=[..20..] Payload=[..40..] Version=4
IHL=5 TOS=64 Length=60 Id=16065 Flags=DF FragOffset=0 TTL=64 Protocol=TCP
Checksum=21831 SrcIP=192.168.123.123 DstIP=10.10.10.10 Options=[]
Padding=[]}
- Layer 3 (40 bytes) = TCP {Contents=[..40..] Payload=[] SrcPort=19603
DstPort=8674 Seq=682579372 Ack=0 DataOffset=10 FIN=false SYN=true RST=false
PSH=false ACK=false URG=false ECE=false CWR=false NS=false Window=29200
Checksum=42658 Urgent=0 Options=[..5..] Padding=[]}
[SYN,ACK] PACKET: 74 bytes, wire length 74 cap length 74 @ 2020-12-29
16:33:57.229609 +0800 CST
- Layer 1 (14 bytes) = Ethernet {Contents=[..14..] Payload=[..60..]
SrcMAC=7c:d3:0a:69:53:dd DstMAC=98:03:9b:84:17:8a EthernetType=IPv4
Length=0}
- Layer 2 (20 bytes) = IPv4 {Contents=[..20..] Payload=[..40..] Version=4
IHL=5 TOS=64 Length=60 Id=0 Flags=DF FragOffset=0 TTL=55 Protocol=TCP
Checksum=40200 SrcIP=10.10.10.10 DstIP=192.168.123.123 Options=[]
Padding=[]}
- Layer 3 (40 bytes) = TCP {Contents=[..40..] Payload=[] SrcPort=8674
DstPort=19603 Seq=2588707367 Ack=682579373 DataOffset=10 FIN=false SYN=true
RST=false PSH=false ACK=true URG=false ECE=false CWR=false NS=false
Window=57920 Checksum=25750 Urgent=0 Options=[..5..] Padding=[]}
[ACK] PACKET: 66 bytes, wire length 66 cap length 66 @ 2020-12-29
16:33:57.229621 +0800 CST
- Layer 1 (14 bytes) = Ethernet {Contents=[..14..] Payload=[..52..]
SrcMAC=98:03:9b:84:17:8a DstMAC=7c:d3:0a:69:53:dd EthernetType=IPv4
Length=0}
- Layer 2 (20 bytes) = IPv4 {Contents=[..20..] Payload=[..32..] Version=4
IHL=5 TOS=64 Length=52 Id=16066 Flags=DF FragOffset=0 TTL=64 Protocol=TCP
Checksum=21838 SrcIP=192.168.123.123 DstIP=10.10.10.10 Options=[]
Padding=[]}
- Layer 3 (32 bytes) = TCP {Contents=[..32..] Payload=[] SrcPort=19603
DstPort=8674 Seq=682579373 Ack=2588707368 DataOffset=8 FIN=false SYN=false
RST=false PSH=false ACK=true URG=false ECE=false CWR=false NS=false
Window=115 Checksum=42650 Urgent=0 Options=[TCPOption(NOP:),
TCPOption(NOP:), TCPOption(Timestamps:2729995117/397747075
0xa2b86b6d17b52383)] Padding=[]}
[PSH,ACK] PACKET: 4344 bytes, wire length 4344 cap length 4344 @
2020-12-29 16:33:57.229631 +0800 CST
- Layer 1 (14 bytes) = Ethernet {Contents=[..14..] Payload=[..4330..]
SrcMAC=98:03:9b:84:17:8a DstMAC=7c:d3:0a:69:53:dd EthernetType=IPv4
Length=0}
- Layer 2 (20 bytes) = IPv4 {Contents=[..20..] Payload=[..4310..] Version=4
IHL=5 TOS=64 Length=4330 Id=16067 Flags=DF FragOffset=0 TTL=64 Protocol=TCP
Checksum=17559 SrcIP=192.168.123.123 DstIP=10.10.10.10 Options=[]
Padding=[]}
- Layer 3 (32 bytes) = TCP {Contents=[..32..] Payload=[..4278..]
SrcPort=19603 DstPort=8674 Seq=682579373 Ack=2588707368 DataOffset=8
FIN=false SYN=false RST=false PSH=true ACK=true URG=false ECE=false
CWR=false NS=false Window=115 Checksum=46928 Urgent=0
Options=[TCPOption(NOP:), TCPOption(NOP:),
TCPOption(Timestamps:2729995117/397747075 0xa2b86b6d17b52383)] Padding=[]}
- Layer 4 (4278 bytes) = Payload 4278 byte(s)
POST /xxxxxx
Host: abc.com
Connection: close
Content-Length: 1169
Accept: */*
Accept-Encoding: deflate, gzip
content-type: application/json; charset=utf-8
[SOME JSON DATA]
[ACK] PACKET: 66 bytes, wire length 66 cap length 66 @ 2020-12-29
16:33:57.229687 +0800 CST
- Layer 1 (14 bytes) = Ethernet {Contents=[..14..] Payload=[..52..]
SrcMAC=7c:d3:0a:69:53:dd DstMAC=98:03:9b:84:17:8a EthernetType=IPv4
Length=0}
- Layer 2 (20 bytes) = IPv4 {Contents=[..20..] Payload=[..32..] Version=4
IHL=5 TOS=64 Length=52 Id=47331 Flags=DF FragOffset=0 TTL=55 Protocol=TCP
Checksum=58412 SrcIP=10.10.10.10 DstIP=192.168.123.123 Options=[]
Padding=[]}
- Layer 3 (32 bytes) = TCP {Contents=[..32..] Payload=[] SrcPort=8674
DstPort=19603 Seq=2588707368 Ack=682583651 DataOffset=8 FIN=false SYN=false
RST=false PSH=false ACK=true URG=false ECE=false CWR=false NS=false
Window=260 Checksum=25578 Urgent=0 Options=[TCPOption(NOP:),
TCPOption(NOP:), TCPOption(Timestamps:397747075/2729995117
0x17b52383a2b86b6d)] Padding=[]}
[PSH,ACK] PACKET: 324 bytes, wire length 324 cap length 324 @ 2020-12-29
16:34:02.232296 +0800 CST
- Layer 1 (14 bytes) = Ethernet {Contents=[..14..] Payload=[..310..]
SrcMAC=7c:d3:0a:69:53:dd DstMAC=98:03:9b:84:17:8a EthernetType=IPv4
Length=0}
- Layer 2 (20 bytes) = IPv4 {Contents=[..20..] Payload=[..290..] Version=4
IHL=5 TOS=64 Length=310 Id=47332 Flags=DF FragOffset=0 TTL=55 Protocol=TCP
Checksum=58153 SrcIP=10.10.10.10 DstIP=192.168.123.123 Options=[]
Padding=[]}
- Layer 3 (32 bytes) = TCP {Contents=[..32..] Payload=[..258..]
SrcPort=8674 DstPort=19603 Seq=2588707368 Ack=682583651 DataOffset=8
FIN=false SYN=false RST=false PSH=true ACK=true URG=false ECE=false
CWR=false NS=false Window=260 Checksum=13502 Urgent=0
Options=[TCPOption(NOP:), TCPOption(NOP:),
TCPOption(Timestamps:397752076/2729995117 0x17b5370ca2b86b6d)] Padding=[]}
- Layer 4 (258 bytes) = Payload 258 byte(s)
HTTP/1.1 200 OK
Content-Type: application/json; charset=utf-8
Date: Tue, 29 Dec 2020 08:34:02 GMT
Content-Length: 115
Connection: close
{"errno":0,"errmsg":"success","tipmsg":"OK","request_id":"12345678","data":{"lastsynctime":1609146356}}
[ACK] PACKET: 66 bytes, wire length 66 cap length 66 @ 2020-12-29
16:34:02.232308 +0800 CST
- Layer 1 (14 bytes) = Ethernet {Contents=[..14..] Payload=[..52..]
SrcMAC=98:03:9b:84:17:8a DstMAC=7c:d3:0a:69:53:dd EthernetType=IPv4
Length=0}
- Layer 2 (20 bytes) = IPv4 {Contents=[..20..] Payload=[..32..] Version=4
IHL=5 TOS=64 Length=52 Id=16070 Flags=DF FragOffset=0 TTL=64 Protocol=TCP
Checksum=21834 SrcIP=192.168.123.123 DstIP=10.10.10.10 Options=[]
Padding=[]}
- Layer 3 (32 bytes) = TCP {Contents=[..32..] Payload=[] SrcPort=19603
DstPort=8674 Seq=682583651 Ack=2588707626 DataOffset=8 FIN=false SYN=false
RST=false PSH=false ACK=true URG=false ECE=false CWR=false NS=false
Window=119 Checksum=42650 Urgent=0 Options=[TCPOption(NOP:),
TCPOption(NOP:), TCPOption(Timestamps:2730000120/397752076
0xa2b87ef817b5370c)] Padding=[]}
[RST,ACK] PACKET: 66 bytes, wire length 66 cap length 66 @ 2020-12-29
16:34:02.23231 +0800 CST
- Layer 1 (14 bytes) = Ethernet {Contents=[..14..] Payload=[..52..]
SrcMAC=7c:d3:0a:69:53:dd DstMAC=98:03:9b:84:17:8a EthernetType=IPv4
Length=0}
- Layer 2 (20 bytes) = IPv4 {Contents=[..20..] Payload=[..32..] Version=4
IHL=5 TOS=64 Length=52 Id=47333 Flags=DF FragOffset=0 TTL=55 Protocol=TCP
Checksum=58410 SrcIP=10.10.10.10 DstIP=192.168.123.123 Options=[]
Padding=[]}
- Layer 3 (32 bytes) = TCP {Contents=[..32..] Payload=[] SrcPort=8674
DstPort=19603 Seq=2588707626 Ack=682583651 DataOffset=8 FIN=false SYN=false
RST=true PSH=false ACK=true URG=false ECE=false CWR=false NS=false
Window=260 Checksum=20315 Urgent=0 Options=[TCPOption(NOP:),
TCPOption(NOP:), TCPOption(Timestamps:397752076/2729995117
0x17b5370ca2b86b6d)] Padding=[]}
[RST] PACKET: 60 bytes, wire length 60 cap length 60 @ 2020-12-29
16:34:02.232429 +0800 CST
- Layer 1 (14 bytes) = Ethernet {Contents=[..14..] Payload=[..46..]
SrcMAC=7c:d3:0a:69:53:dd DstMAC=98:03:9b:84:17:8a EthernetType=IPv4
Length=0}
- Layer 2 (20 bytes) = IPv4 {Contents=[..20..] Payload=[..20..] Version=4
IHL=5 TOS=64 Length=40 Id=0 Flags=DF FragOffset=0 TTL=55 Protocol=TCP
Checksum=40220 SrcIP=10.10.10.10 DstIP=192.168.123.123 Options=[]
Padding=[]}
- Layer 3 (20 bytes) = TCP {Contents=[..20..] Payload=[] SrcPort=8674
DstPort=19603 Seq=2588707626 Ack=0 DataOffset=5 FIN=false SYN=false
RST=true PSH=false ACK=false URG=false ECE=false CWR=false NS=false
Window=0 Checksum=30080 Urgent=0 Options=[] Padding=[]}
Can anyone give me some guesses? Thanks!
-- Eddix Chen
--
You received this message because you are subscribed to the Google Groups
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/golang-nuts/77a8961f-33da-48d3-9d19-371c21ff39e2n%40googlegroups.com.
