On 23.06.2014, Dennis Jacobfeuerborn wrote: > On 23.06.2014 11:57, Anders Henke wrote: > > On 18.06.2014, Stephen Carville wrote: > >> I set up a CentOS 6.5 box to test ipvsadm. So far I have been unable to > >> get it to forward connections. When I try to connect, it doesn't write > >> anything in /var/log/messages to tell me what is happening. Netstat > >> doesn't see anything listening on the interface IP (I read elsewhere > >> that is normal) and tshark sees the incoming SYN but there is either a > >> timeout or a RST. > >> > >> Rules right now: > >> > >> $ ipvsadm -L > >> > >> IP Virtual Server version 1.2.1 (size=4096) > >> Prot LocalAddress:Port Scheduler Flags > >> -> RemoteAddress:Port Forward Weight ActiveConn InActConn > >> TCP 10.212.160.40:4172 lc persistent 360 > >> -> 10.212.170.162:4172 Route 1 0 0 > >> -> 10.212.170.163:4172 Route 1 0 0 > >> > >> IP forwarding is turned on: > >> > >> $ sysctl net.ipv4.ip_forward > >> net.ipv4.ip_forward = 1 > > > > Short answer: switch to kernel 3.6 or newer, turn off rp_filter for the > > interface receiving the reply packet, and replace rp_filter functionality > > by more accurate and flexible iptables rules in the FORWARD chain. > > Since he is running CentOS 6.5 he can simply set > /proc/sys/net/ipv4/conf/<interface>/accept_local to 1 to prevent packets > from being dropped as martians. This was introduced in 2.6.33 but > backported to recent RHEL/CentOS kernels so no need to go to 3.6 or newer. > You still have to set the rp_filter though since this is a different > isssue than the martian packet one.
It's a little bit more complicated :-) In this setup, the VIP is configured on eth0, but the realserver is contacted via eth1. The realserver will most likely to reply via eth1 as well. Upstream network equipment will route incoming traffic for the VIP to eth0, and by seperating the realservers onto a dedicated network, we're removing the need for handling the ARP problem with realserver configuration. As long as no realservers are connected to eth0, the realservers don't need to disable ARP replies at all. Regarding kernel configuration, accept_local is only revelant if rp_filter is set. -if eth1 has both rp_filter=1 and accept_local=1, the kernel checks if the IP address is routed and local to eth1, otherwise it is being dropped. The VIP is configured on eth0, so the packet is being discarded. -If eth1 has rp_filter=2 (loose mode) and accept_local=1, the packet is accepted. Loose mode doesn't check if the incoming interface matches the routing decision, just if "any" route on the host, including a default route, does know where to send this packet may be sent to. So whenever your system does have a default route configured, rp_filter=2 for any interface other than your default interface is almost equivalent to rp_filter=0. The only difference is the handling of addresses local to the balancer, and by configuring accept_local=1 for that interface, one is also removing that single difference. In the end, you don't have any kind of reverse path filtering for that specific interface, but you've just added some check to your kernel which will always return true, but merely "looks" like some added level of security. That's part of the reason why in such a situation, I'm advocating to de-configure rp_filter for that specific interface: you don't gain security, you're just adding a useless check. If you do want to improve security, you need to add the expected or aimed level of security by dedicated iptables forwarding rules. === Alternative option Another option is to configure the VIP to be hosted on eth1, configure all interfaces with rp_filter=1 (default on CentOS 6.5) and additionally configure eth1 with accept_local=1: $ ip address del 10.212.160.40/32 dev eth0 $ ip address add 10.212.160.40/32 dev eth1 $ sysctl -w net.ipv4.conf.all.rp_filter = 1 $ sysctl -w net.ipv4.conf.eth1.accept_local = 1 Due to Linux' weak host model, packets for 10.212.160.40 will be accepted on any interface and sent to the realserver. The realserver replies are received at an interface hosting that IP address, so accept_local will match and forward replies as intended. Downside: most network engineers looking at this configuration instinctively will assume a misconfiguration. It's not that intuitive, as it's relying on the weak host model. However, it's consistent with most realserver configurations and does have the benefits of strict mode reverse path filtering. As there are no forwarding restrictions per default other than the just configured reverse path filtering and we're running on ip_forward=1, one should still take care of the exact forwarding configuration, e.g. by adding iptables rules to the forwarding chain. Otherwise, a malicious host on eth0 may access any host running behind eth1 just by adding the balancer as a router. Specific iptables forwarding rules or selectively enabling forwarding only for eth1 does remove that risk. However, the later one is very specific to the exact realserver network configuration (you need to enable forwarding on eth0 to forward traffic from eth0 to eth1, if your balancer acts as the only router for your realservers). So if one is adding fine-granular forwarding rules for a specific interface anyway, which are tighter than rp_filter for that interface - why care about rp_filter for eth1? Best, Anders -- 1&1 Internet AG Expert Systems Architect (IT Operations) Brauerstrasse 50 v://49.721.91374.0 D-76135 Karlsruhe f://49.721.91374.225 Amtsgericht Montabaur HRB 6484 Vorstand: Ralph Dommermuth, Frank Einhellinger, Robert Hoffmann, Andreas Hofmann, Markus Huhn, Hans-Henning Kettler, Uwe Lamnek, Jan Oetjen, Christian Würst Aufsichtsratsvorsitzender: Michael Scheeren _______________________________________________ Please read the documentation before posting - it's available at: http://www.linuxvirtualserver.org/ LinuxVirtualServer.org mailing list - lvs-users@LinuxVirtualServer.org Send requests to lvs-users-requ...@linuxvirtualserver.org or go to http://lists.graemef.net/mailman/listinfo/lvs-users
