We have recently replaced an old PC firewall router running FreeBSD 4
and IPFilter 3.4.20 by a new, faster, machine running FreeBSD 5.4 and
IPIFilter 4.1.13.
The new machine has been having trouble establishing BGP sessions with
the (Cisco) router run by our service provider. We are running Quagga
on the PC.
After much head-scratching and splinter extraction the problems have
been resolved by executing the following command on the new FreeBSD
system:
# sysctl net.inet.ipf.fr_minttl=1
changing the IPFilter parameter "fr_minttl".
This was prompted by notinging that the BGP session, a TCP call to
port 179, was using a normal TTL for packets from the PC to the
Cisco, but the Cisco was using a TTL of "1" for packets *to* the PC.
This is, it seems, quite normal for Cisco BGP implementations when
the destination of the BGP/TCP call is necessarily an immediate
neighbor. I had also noted that the state-table *was* getting an
entry made by my rules when the SYN packet left the PC for the Cisco,
but that the state ended up as "5/3".
In the following line numbers are in verstion 4.1.13.
Reading the code, I noticed that there is a flag, "FI_LOWTTL", that
gets set in "fil.c" at line 2432 if the TTL in a packet is less
than "fr_minttl". Later this gets passed with the "fin" structure
in a call to "fr_checkstate" in "ip_state.c". Eventually, about
line xxxx, "fr_matchsrcdst" gets called with a "cmask" parameter
of "FI_CMP" that includes the "FI_LOWTTL" bit. Reducing the value
of "fr_minttl" to 1 means this bit never gets set on packets and
cannot cause a packet to fail to match a state-table entry.
It is interesting that the test around line 1850 in "ip_state.c"
is not symmetric between the packet and state-table entry. In our
problem, if the BGP call was established from the Cisco (using
TTL=1) to the PC, it normally worked successfully. Calls from the
PC (using a normal TTL) normally did not work successfully; and
combining "tcpdump" with "ktrace" showed packets arriving at the
PC which did not reach the Quagga BGP routing process.
I'm fairly sure there's a problem in this area. What needs the
attention of an expert is the decision about what is the Right
Thing(tm) to do. I do hope an expert or two reads this list...
--
David Pick
