Non-standard TCP stack processing of packets with unacceptable ACK numbers

[Paul Fiterau Brostean]

Hello,






      TEST HARNESS (CLIENT)                        LINUX SERVER

  1.  -                                          LISTEN (server listen, then 
accepts)

  2.  - --> <SEQ=100><CTL=SYN>               --> SYN-RECEIVED

  3.  - <-- <SEQ=300><ACK=101><CTL=SYN,ACK>  <-- SYN-RECEIVED

  4.  - --> <SEQ=101><ACK=301><CTL=ACK>      --> ESTABLISHED


  6.  - --> <SEQ=101><ACK=99999><CTL=FIN,ACK> --> CLOSING (client sends FIN,ACK 
with not yet sent acknowledgement number)

  7.  - <-- <SEQ=302><ACK=102><CTL=ACK>      <-- CLOSING (ACK is 102 instead of 
101, why?)


  8.  - <-- <SEQ=301><ACK=102><CTL=FIN,ACK>  <-- CLOSING (retransmission, again 
ACK is 102)










Thanks, Paul.

P.S. Our most recent publication on relating to TCP was at CAV (some big 
conference), article link for anyone interested
http://www.sws.cs.ru.nl/publications/papers/fvaan/FJV16/main.pdf . Now we are 
working on applying more advanced techniques that abstract less and capture 
more behavior.
P.S.2 Potentially violated RFC fragment:
   3.  If the connection is in a synchronized state (ESTABLISHED,
    FIN-WAIT-1, FIN-WAIT-2, CLOSE-WAIT, CLOSING, LAST-ACK, TIME-WAIT),
    any unacceptable segment (out of window sequence number or
    unacceptible acknowledgment number) must elicit only an empty
    acknowledgment segment containing the current send-sequence number
    and an acknowledgment indicating the next sequence number expected
    to be received, and *the connection remains in the same state*.

My name is Paul Fiterau, I am a PhD student at Radboud University whose focus for the past few years has been among others to develop and apply inference techniques on TCP stacks in order to obtain nice models, and to verify them if possible using formal methods. We contacted you on something similar 2 years back. The older (3.19 kernel release) Linux TCP stack we analyze exhibits behavior that seems odd to me. The scenario is as follows (all packets have empty payloads, no window scaling, rcv/snd window size should not be a factor): 5. - <-- <SEQ=301><ACK=101><CTL=FIN,ACK> <-- FIN WAIT-1 (server opts to close the data connection calling "close" on the connection socket) ... (silence from CLIENT) Now, note that packet 6 while having the expected sequence number, acknowledges something that wasn't sent by the server. So I would expect the packet to maybe prompt an ACK response from the server, and then be ignored. Yet it is not ignored and actually leads to an increase of the acknowledgement number in the server's retransmission of the FIN,ACK packet. The explanation I found is that the FIN in packet 6 was processed, despite the acknowledgement number being unacceptable. Further experiments indeed show that the server processes this FIN, transitioning to CLOSING, then on receiving an ACK for the FIN it had send in packet 5, the server (or better said connection) transitions from CLOSING to TIME_WAIT (as signaled by netstat). I attached a capture showing the scenario, as well as an equivalent capture for a Windows 10 TCP server, which behaves exactly as I would expect by not increasing the expected sequence number after packet 6, and thus not processing the FIN flag received. I hope someone more knowledgeable can clear this up for me. Is it ok for the server to process the FIN bit in a packet with an unacceptable acknowledgement number? Could this be an inconsistency in the tested stack?

linux.pcapng
Description: Binary data

win.pcapng
Description: Binary data