John Haxby wrote:
On 14 January 2011 18:01, Peter Ruprecht <[email protected]
<mailto:[email protected]>> wrote:
Hi everyone,
I think I'm seeing a difference in behavior between RHEL 5 and 6 on
how packets get routed between different subnets on different
network interfaces. Say I have a dual-homed host, with each
interface connected to a different physical class C subnet. The
routing table looks like:
# netstat -rn
Kernel IP routing table
Destination Gateway Genmask Flags MSS Window
irtt Iface
128.138.140.0 0.0.0.0 255.255.255.0 U 0 0
0 eth1
128.138.107.0 0.0.0.0 255.255.255.0 U 0 0
0 eth0
169.254.0.0 0.0.0.0 255.255.0.0 U 0 0
0 eth0
169.254.0.0 0.0.0.0 255.255.0.0 U 0 0
0 eth1
0.0.0.0 128.138.107.1 0.0.0.0 UG 0 0
0 eth0
In RHEL5, if I ping the host's 128.138.140.X address from a machine
on the 128.138.107. subnet, I can use tcpdump to see the icmp
request coming in on eth1, and the reply going out on eth0. The
host is not doing forwarding; that is, there's a 0 in
/proc/sys/net/ipv4/ip_forward.
As someone else pointed out (npanderson at ups.com <http://ups.com>) ,
this is to do with reverse path filtering. More specifically, it's to
do with a change to rp_filter that happened in RHEL6. (I tracked down
the specific commit a little while ago, I can do it again if you're
interested).
It's instructive to compare npanderson's quote of the RHEL5 docs with RHEL6:
-----
rp_filter - INTEGER
0 - No source validation.
1 - Strict mode as defined in RFC3704 Strict Reverse Path
Each incoming packet is tested against the FIB and if the
interface
is not the best reverse path the packet check will fail.
By default failed packets are discarded.
2 - Loose mode as defined in RFC3704 Loose Reverse Path
Each incoming packet's source address is also tested against
the FIB
and if the source address is not reachable via any interface
the packet check will fail.
Current recommended practice in RFC3704 is to enable strict mode
to prevent IP spoofing from DDos attacks. If using asymmetric
routing
or other complicated routing, then loose mode is recommended.
The max value from conf/{all,interface}/rp_filter is used
when doing source validation on the {interface}.
Default value is 0. Note that some distributions enable it
in startup scripts.
-----
Red Hat are (correctly) setting rp_filter to 1, strict mode. In this
case a packet coming in eth0 will have its source address routed out on
the same interface that it came in on (because that's the default
route). However, a packet coming in on eth1 will have it source address
routed out on a different interface to the one it came in on and it will
be discarded. Silently.
This is basically asymmetric routing and is quite possibly not what you
want anyway (it messes up TCP flow control) so there are two ways to fix
this: stick with asymmetric routing and permit it or fix the asymmetric
routing.
The first one is easiest: in /etc/sysctl.conf change rp_filter=1 to
rp_filter=2). You'll need to load that and restart the network. It's
probably easiest to reboot :-) to be sure. I suspect that it was not
restarting enough things that prevented this change from working before.
The second one may be simple as simple as adding those routes that
should go out on eth1 to the routing table or running some routing
daemon. It depends on your network topology, basically. This would be
the preferred solution if it's practicable.
Thanks for the very detailed and clear explanation, and also to Nathan
Anderson for originally pointing me at reverse path filtering. After
setting rp_filter to 2 (rather than 1 or 0) for the relevant interfaces,
the routing on the new RHEL 6 system is working in the same way as on
RHEL 5. I greatly appreciate your help!
-Peter
_______________________________________________
rhelv6-list mailing list
[email protected]
https://www.redhat.com/mailman/listinfo/rhelv6-list
