On Thu, Nov 17, 2005 at 03:21:01PM +0000, Karl O. Pinc wrote: > Let me apologize in advance: when all you've got is a hammer, > everything looks like a nail. I keep harping on the 2MSL TCP > rule -- reuse of source IP/port dest IP/port quad. So, > could be a TCP(ish) issue, although I don't feel entirely > qualified to claim this. > > Seems to me like you could be burning through all the possible > source ports the client wants to use. After that the firewall > sees the TCP violation and does not let the traffic through, > seeing the reuse as a spoof attempt. When the FIN-WAITs expire > then you've got another bunch of "quads" to use and things rip > again for a while until you again run out.
This definitely looks to be happening: $ tcpdump -nr 12345.pcap dst port 12345 and \ '(tcp[tcpflags] & tcp-syn != 0)' \ | awk '{print $3}' |awk -F. '{print $5}' | \ sort |uniq -c |sort -n | tail 1 60234 1 60319 1 60402 1 60460 1 60783 1 60798 1 60965 1 60981 1 60998 4 40856 And, sure enough, source port 40856 is where things go wrong. You can see in the packet capture that when things get to this point, the client sends 3 syns in rapid succession from this source port and the firewall doesn't allow them through. I've seen a case or two where the last of the 3 got through, likely because timers had started to expire. Source port 40856 was used at time X and a second or two later, it gets reused. Now my problem is figuring out how to deal with this situation. I believe the firewall is doing what it should but others may argue it is being too strict. I could also just widen the defaut port range on the clients, but that doesn't strike me as the best solution. Thanks very much for your input! -jon