On 18.06.2014, Stephen Carville wrote: > I set up a CentOS 6.5 box to test ipvsadm. So far I have been unable to > get it to forward connections. When I try to connect, it doesn't write > anything in /var/log/messages to tell me what is happening. Netstat > doesn't see anything listening on the interface IP (I read elsewhere > that is normal) and tshark sees the incoming SYN but there is either a > timeout or a RST. > > Rules right now: > > $ ipvsadm -L > > IP Virtual Server version 1.2.1 (size=4096) > Prot LocalAddress:Port Scheduler Flags > -> RemoteAddress:Port Forward Weight ActiveConn InActConn > TCP 10.212.160.40:4172 lc persistent 360 > -> 10.212.170.162:4172 Route 1 0 0 > -> 10.212.170.163:4172 Route 1 0 0 > > IP forwarding is turned on: > > $ sysctl net.ipv4.ip_forward > net.ipv4.ip_forward = 1
Short answer: switch to kernel 3.6 or newer, turn off rp_filter for the interface receiving the reply packet, and replace rp_filter functionality by more accurate and flexible iptables rules in the FORWARD chain. Long: When you're using direct routing/gatewaying or tunneling, your replies will have the VIP address as the source address of the packet. They're routed via your balancer, who also knows the same IP address to be a local address. Depending on /proc/sys/net/ipv4/conf/*/rp_filter or kernel version, you need to either change a little bit of configuration or apply special kernel patches (forward_shared). If you're running a linux kernel <3.6, your kernel will always drop those packets as being spoofed. If you're turning on net/ipv4/conf/*/log_martians, you'll also see kernel log messages on this. >From the stone age of IPVS, there is a kernel patch introducing a sysctl >switch (forward_shared) to turn off this behaviour for selected interfaces. If >misconfigured (e.g. "turn it on for all interfaces"), this patch will >introduce an option for third parties to spoof packets onto your network >(which is something you don't want to do). http://www.ssi.bg/~ja/ does offer recent patches for about any major linux kernel source from 2.4.19 to 3.15, just in case you're interested. Linux Kernel 3.6 did somehow obsolete the forward_shared-patch by changing the behaviour of rp_filter for the interface receiving the spoofed packet: -if rp_filter is turned on and ip_forward=1, the kernel will most likely drop your reply packets, as they're most likely received by the 'wrong' interface: an interface without a matching entry in the routing table for this IP address. Given the point many linux distros do enable rp_filter per default, you most likely hit that wall. -if rp_filter is turned off (=0) and ip_forward=1, the kernel will gladly forward your "spoofed" reply packets. The exact use of rp_filter on a dedicated load balancer is fairly limited, if at all. It's much better to replace its functionality by dedicated iptables rules and restricting the FORWARD chain. For your configuration, this results in the following configuration changes: $ iptables -P FORWARD DROP $ iptables -A FORWARD -i eth1 -o eth0 --source 10.212.160.40 -j ACCEPT $ sysctl -w proc/sys/net/ipv4/conf/eth1/rp_filter=0 If you're keen on extra-tight security, you may also copy the permissive line and add "-m --mac-source $realserver-mac" for every realserver. This ensures only your real servers may send "spoofed" traffic via your balancer. Best, Anders -- 1&1 Internet AG Expert Systems Architect (IT Operations) Brauerstrasse 50 v://49.721.91374.0 D-76135 Karlsruhe f://49.721.91374.225 Amtsgericht Montabaur HRB 6484 Vorstand: Ralph Dommermuth, Frank Einhellinger, Robert Hoffmann, Andreas Hofmann, Markus Huhn, Hans-Henning Kettler, Uwe Lamnek, Jan Oetjen, Christian Würst Aufsichtsratsvorsitzender: Michael Scheeren _______________________________________________ Please read the documentation before posting - it's available at: http://www.linuxvirtualserver.org/ LinuxVirtualServer.org mailing list - lvs-users@LinuxVirtualServer.org Send requests to lvs-users-requ...@linuxvirtualserver.org or go to http://lists.graemef.net/mailman/listinfo/lvs-users
