Hi list, this is my first post, so please be gentle.
I'm trying to get a lvs (lab)setup to work with snat.
I've been reading several posts and the mailinglist, and they told that
the snat should work with kernel 2.6.36 and higher.
Therefor I have installed ubuntu 11.04 amd64 which has the following
packages:
loadbalancer-ng ~ # uname -a
Linux loadbalancer-ng 2.6.38-8-server #42-Ubuntu SMP Mon Apr 11 03:49:04
UTC 2011 x86_64 x86_64 x86_64 GNU/Linux
loadbalancer-ng ~ # iptables -V
iptables v1.4.10
loadbalancer-ng ~ # ipvsadm -v
ipvsadm v1.25 2008/5/15 (compiled with popt and IPVS v1.2.1)
The setup is the following:
+--------------------------+
| linux router |
+------------+-------------+ ip 172.16.31.1/24
|
|
+------------+-------------+ ip 172.16.31.10/24
| lvs loadbalancer | VIP: 172.16.31.10:80
+------------+-------------+ ip 172.16.29.10/24
|
|
+------------+-------------+ ip 172.16.29.5/24
| linux realserver | RIP: 172.16.29.5:80
+--------------------------+
loadbalancer-ng ~ # ipvsadm -ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP 172.16.31.10:80 rr
-> 172.16.29.5:80 Masq 1 0 0
When I do the following:
router ~ # wget -O /dev/null http://172.16.31.10:80 -q; echo $?
0
I can see the following traffic on the real:
real www # tcpdump -i eth1 -nn
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth1, link-type EN10MB (Ethernet), capture size 96 bytes
14:10:22.070213 IP 172.16.31.1.60572 > 172.16.29.5.80: Flags [S], seq
531016714, win 5840, options [mss 1460,sackOK,TS val 2116681 ecr
0,nop,wscale 5], length 0
14:10:22.070241 IP 172.16.29.5.80 > 172.16.31.1.60572: Flags [S.], seq
511492588, ack 531016715, win 5792, options [mss 1460,sackOK,TS val
2097489 ecr 2116681,nop,wscale 5], length 0
14:10:22.073096 IP 172.16.31.1.60572 > 172.16.29.5.80: Flags [.], ack 1,
win 183, options [nop,nop,TS val 2116681 ecr 2097489], length 0
14:10:22.073385 IP 172.16.31.1.60572 > 172.16.29.5.80: Flags [P.], seq
1:111, ack 1, win 183, options [nop,nop,TS val 2116681 ecr 2097489],
length 110
14:10:22.073400 IP 172.16.29.5.80 > 172.16.31.1.60572: Flags [.], ack
111, win 181, options [nop,nop,TS val 2097489 ecr 2116681], length 0
14:10:22.073507 IP 172.16.29.5.80 > 172.16.31.1.60572: Flags [P.], seq
1:245, ack 111, win 181, options [nop,nop,TS val 2097489 ecr 2116681],
length 244
14:10:22.073606 IP 172.16.29.5.80 > 172.16.31.1.60572: Flags [F.], seq
245, ack 111, win 181, options [nop,nop,TS val 2097489 ecr 2116681],
length 0
14:10:22.076351 IP 172.16.31.1.60572 > 172.16.29.5.80: Flags [.], ack
245, win 216, options [nop,nop,TS val 2116681 ecr 2097489], length 0
14:10:22.076793 IP 172.16.31.1.60572 > 172.16.29.5.80: Flags [F.], seq
111, ack 246, win 216, options [nop,nop,TS val 2116681 ecr 2097489],
length 0
14:10:22.076830 IP 172.16.29.5.80 > 172.16.31.1.60572: Flags [.], ack
112, win 181, options [nop,nop,TS val 2097489 ecr 2116681], length 0
It is NOT SOURCE NATTED.
When I do the following:
loadbalancer-ng ~ # wget -O /dev/null http://172.16.31.10:80 -q; echo $?
0
real www # tcpdump -i eth1 -nn
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth1, link-type EN10MB (Ethernet), capture size 96 bytes
14:11:16.511561 IP 172.16.29.10.40215 > 172.16.29.5.80: Flags [S], seq
1416440597, win 32792, options [mss 16396,sackOK,TS val 2165715 ecr
0,nop,wscale 5], length 0
14:11:16.511594 IP 172.16.29.5.80 > 172.16.29.10.40215: Flags [S.], seq
1375362407, ack 1416440598, win 5792, options [mss 1460,sackOK,TS val
2102933 ecr 2165715,nop,wscale 5], length 0
14:11:16.512355 IP 172.16.29.10.40215 > 172.16.29.5.80: Flags [.], ack
1, win 1025, options [nop,nop,TS val 2165715 ecr 2102933], length 0
14:11:16.513378 IP 172.16.29.10.40215 > 172.16.29.5.80: Flags [P.], seq
1:111, ack 1, win 1025, options [nop,nop,TS val 2165715 ecr 2102933],
length 110
14:11:16.513402 IP 172.16.29.5.80 > 172.16.29.10.40215: Flags [.], ack
111, win 181, options [nop,nop,TS val 2102933 ecr 2165715], length 0
14:11:16.513538 IP 172.16.29.5.80 > 172.16.29.10.40215: Flags [P.], seq
1:245, ack 111, win 181, options [nop,nop,TS val 2102933 ecr 2165715],
length 244
14:11:16.513590 IP 172.16.29.5.80 > 172.16.29.10.40215: Flags [F.], seq
245, ack 111, win 181, options [nop,nop,TS val 2102933 ecr 2165715],
length 0
14:11:16.514541 IP 172.16.29.10.40215 > 172.16.29.5.80: Flags [.], ack
245, win 1059, options [nop,nop,TS val 2165715 ecr 2102933], length 0
14:11:16.515023 IP 172.16.29.10.40215 > 172.16.29.5.80: Flags [F.], seq
111, ack 246, win 1059, options [nop,nop,TS val 2165715 ecr 2102933],
length 0
14:11:16.515040 IP 172.16.29.5.80 > 172.16.29.10.40215: Flags [.], ack
112, win 181, options [nop,nop,TS val 2102933 ecr 2165715], length 0
It IS SOURCE NATTED!!! What I've noticed is that the POSTROUTING rules
are not hit in the first tcpdump.
Other relevant configuration needed for this setup (correct me if I'm
wrong):
loadbalancer-ng ~ # iptables -vnL POSTROUTING -t nat
Chain POSTROUTING (policy ACCEPT 0 packets, 0 bytes)
pkts bytes target prot opt in out source
destination
1 60 all -- * * 0.0.0.0/0
0.0.0.0/0
6 360 SNAT all -- * * 0.0.0.0/0
0.0.0.0/0 vaddr 172.16.31.10 vport 80 to:172.16.29.10
loadbalancer-ng ~ # sysctl net.ipv4.vs.conntrack
net.ipv4.vs.conntrack = 1
loadbalancer-ng ~ # sysctl -a 2> /dev/null | grep \.forwarding | grep -v
mc_forward
net.ipv4.conf.all.forwarding = 1
net.ipv4.conf.default.forwarding = 1
net.ipv4.conf.lo.forwarding = 1
net.ipv4.conf.eth0.forwarding = 1
net.ipv4.conf.eth1.forwarding = 1
net.ipv4.conf.eth2.forwarding = 1
net.ipv6.conf.all.forwarding = 1
net.ipv6.conf.default.forwarding = 1
net.ipv6.conf.lo.forwarding = 1
net.ipv6.conf.eth0.forwarding = 1
net.ipv6.conf.eth1.forwarding = 1
net.ipv6.conf.eth2.forwarding = 1
loadbalancer-ng ~ # lsmod
Module Size Used by
xt_string 12586 0
xt_mark 12563 0
xt_DSCP 12629 0
xt_dscp 12597 0
xt_multiport 12597 0
xt_hashlimit 17698 0
xt_owner 12498 0
xt_iprange 12541 0
xt_NFQUEUE 12699 0
ipt_addrtype 12599 0
iptable_filter 12810 0
xt_tcpudp 12603 0
ipt_REJECT 12576 0
iptable_nat 13182 1
iptable_mangle 12734 0
ip_tables 27456 3 iptable_filter,iptable_nat,iptable_mangle
xt_CHECKSUM 12549 0
ebtables 30915 0
ip6table_filter 12815 0
ip6_tables 27845 1 ip6table_filter
ipt_LOG 17016 0
xt_conntrack 12728 0
xt_connmark 12755 0
xt_state 12578 0
ipt_MASQUERADE 12759 0
nf_nat 25736 2 iptable_nat,ipt_MASQUERADE
nf_conntrack_ipv4 19640 3 iptable_nat,nf_nat
nf_defrag_ipv4 12729 1 nf_conntrack_ipv4
xt_ipvs 12536 1
x_tables 29545 26
xt_string,xt_mark,xt_DSCP,xt_dscp,xt_multiport,xt_hashlimit,xt_owner,xt_iprange,xt_NFQUEUE,ipt_addrtype,iptable_filter,xt_tcpudp,ipt_REJECT,iptable_nat,iptable_mangle,ip_tables,xt_CHECKSUM,ebtables,ip6table_filter,ip6_tables,ipt_LOG,xt_conntrack,xt_connmark,xt_state,ipt_MASQUERADE,xt_ipvs
ip_vs_rr 12602 1
ip_vs 137211 4 xt_ipvs,ip_vs_rr
nf_conntrack 81956 8
iptable_nat,xt_conntrack,xt_connmark,xt_state,ipt_MASQUERADE,nf_nat,nf_conntrack_ipv4,ip_vs
libcrc32c 12644 1 ip_vs
snd_intel8x0 38272 0
snd_ac97_codec 134270 1 snd_intel8x0
ac97_bus 12730 1 snd_ac97_codec
snd_pcm 96531 2 snd_intel8x0,snd_ac97_codec
snd_timer 29602 1 snd_pcm
psmouse 73535 0
serio_raw 13166 0
snd 67346 4
snd_intel8x0,snd_ac97_codec,snd_pcm,snd_timer
virtio_balloon 13153 0
soundcore 12680 1 snd
snd_page_alloc 18529 2 snd_intel8x0,snd_pcm
i2c_piix4 13303 0
lp 17789 0
parport 46458 1 lp
floppy 74120 0
There's obviously something wrong, but I cannot find it.
This is the loadbalancer configuration:
iptables -t nat -A POSTROUTING -m ipvs --vaddr 172.16.31.10 --vport 80
-j SNAT --to-source 172.16.29.10
ipvsadm -A -t 172.16.31.10:80 -s rr
ipvsadm -a -t 172.16.31.10:80 -r 172.16.29.5:80 -m
I hope someone can tell me why this does not work as expected.
Another question I have: I would like to have a subnet for my VIPS. Is
it necessary to configure all VIPS as alias IP's on the loadbalancer to
get it to work?
_______________________________________________
Please read the documentation before posting - it's available at:
http://www.linuxvirtualserver.org/
LinuxVirtualServer.org mailing list - [email protected]
Send requests to [email protected]
or go to http://lists.graemef.net/mailman/listinfo/lvs-users
