Hi all,
In the past weeks, I found time to dive in deeper, narrow the problem down, and
fix it. Here is the feedback to you (maybe for future reference).
It is a firewall (more precise: connection tracking) issue after all. Both
routers exchange the active connections with conntrackd, in a SYNC_FTFW mode
(i.e. pushing the connections from the other router directly into the local
connection tracking table) to create a fully redundant router cluster, where
data can flow in and out through both routers.
At least, that was the idea: it turned out that a SYN/ACK was blocked when it
returned on the other router (B) than the one that routed the SYN (router A),
although the connection tracking table of router B showed the connection (as
SYN-SENT). Or, when the SYN/ACK was passed through router A, but when later
during the connection the return traffic was sent trough router B, the
connection would also break.
I will not trouble you with all the gory details, but the fundamental issue
turned out to be the fact that TCP connection tracking marked the SYN/ACK
invalid. This is because the data that was ACKed by the packet was never seen
by router B, and is therefore out of the ACK-window. This level of detail on
the connection tracking info is clearly not exchanged by conntrackd.
This was confirmed by switching on the logging of packets marked invalid by the
connection tracking system:
echo 255 > /proc/sys/net/netfilter/nf_conntrack_log_invalid
This logged lines like
kernel:[15861946.414756] nf_ct_tcp: ACK is over the upper bound (ACKed
data not seen yet) IN= OUT= SRC=…
Grepping the kernel modules for that line, found
/lib/modules/2.6.32-5-686/kernel/net/netfilter/nf_conntrack.ko.
I dove into the source (nf_conntrack_proto_tcp.c) of the kernel module and
found the switch nf_conntrack_tcp_be_liberal that turns off the ACK window
checking.
Typing:
echo 1 > /proc/sys/net/netfilter/nf_conntrack_tcp_be_liberal
on both routers finally fixed my problem. Thanks for your time and help!
--
Best regards,
Reinier Boon
________________________________________
Reinier Boon | Senior software engineer | Telecats bv | KvK Enschede 06069106 |
Tel: +31 53 488 99 26 | Fax: +31 53 488 99 10 | E mail: [email protected]
