Ok guys, here is what I've done now trying to setup LVS-NAT: LVS server:
root@lvs1:~# ifconfig eth0 Link encap:Ethernet HWaddr 76:bc:30:e1:95:29 inet addr:10.56.213.7 Bcast:10.56.213.103 Mask:255.255.255.192 inet6 addr: fe80::74bc:30ff:fee1:9529/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:40395 errors:0 dropped:0 overruns:0 frame:0 TX packets:115566 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:11154768 (11.1 MB) TX bytes:10420406 (10.4 MB) eth0:2 Link encap:Ethernet HWaddr 76:bc:30:e1:95:29 inet addr:192.168.12.2 Bcast:192.168.12.7 Mask:255.255.255.248 UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 eth0:3 Link encap:Ethernet HWaddr 76:bc:30:e1:95:29 inet addr:192.168.12.1 Bcast:192.168.12.7 Mask:255.255.255.248 UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 eth1 Link encap:Ethernet HWaddr 1a:99:f3:ea:7c:e6 inet addr:192.168.0.1 Bcast:192.168.0.3 Mask:255.255.255.252 inet6 addr: fe80::1899:f3ff:feea:7ce6/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:89880 errors:0 dropped:0 overruns:0 frame:0 TX packets:113137 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:6277612 (6.2 MB) TX bytes:15654801 (15.6 MB) eth1:1 Link encap:Ethernet HWaddr 1a:99:f3:ea:7c:e6 inet addr: **publicIP** Bcast:**broadcast** Mask:255.255.255.248 UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 root@lvs1:~# sysctl -p net.ipv4.ip_forward = 1 net.ipv4.conf.default.rp_filter = 0 net.ipv4.conf.all.send_redirects = 0 net.ipv4.conf.default.accept_source_route = 0 root@lvs1:~# ipvsadm IP Virtual Server version 1.2.1 (size=4096) Prot LocalAddress:Port Scheduler Flags -> RemoteAddress:Port Forward Weight ActiveConn InActConn TCP **mypublicDomain**:http rr -> 192.168.12.4:http Masq 1 0 3 root@lvs1:~# lynx --dump 192.168.12.4 It works! This is the default web page for this server. The web server software is running but no content has been added, yet. One realserver (192.168.12.4): root@ns1:~# ifconfig eth0 Link encap:Ethernet HWaddr be:b2:76:d3:4f:ff inet addr:192.168.12.4 Bcast:192.168.12.7 Mask:255.255.255.248 inet6 addr: fe80::bcb2:76ff:fed3:4fff/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:6757 errors:0 dropped:0 overruns:0 frame:0 TX packets:8980 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:358707 (358.7 KB) TX bytes:2761791 (2.7 MB) /etc/network/interfaces auto eth0 iface eth0 inet static address 192.168.12.4 netmask 255.255.255.248 broadcast 192.168.12.7 gateway 192.168.12.1 root@ns1:~# route -n Kernel IP routing table Destination Gateway Genmask Flags Metric Ref Use Iface 192.168.12.0 0.0.0.0 255.255.255.248 U 0 0 0 eth0 Now testing it trying to access **mypublicIP**:80 ETH1 (PUBLIC IP TCPDUMP) tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on eth1, link-type EN10MB (Ethernet), capture size 65535 bytes 10:50:46.630822 IP 177.54.114.32.52692 > **MYPUBLICIP**.80: Flags [S], seq 2306349141, win 65535, options [mss 1440,nop,wscale 4,nop,nop,TS val 292573515 ecr 0,sackOK,eol], length 0 10:50:46.641358 IP 177.54.114.32.52691 > **MYPUBLICIP**.80: Flags [S], seq 2697243517, win 65535, options [mss 1440,nop,wscale 4,nop,nop,TS val 292573515 ecr 0,sackOK,eol], length 0 10:50:46.890409 IP 177.54.114.32.52693 > **MYPUBLICIP**.80: Flags [S], seq 3700386367, win 65535, options [mss 1440,nop,wscale 4,nop,nop,TS val 292573760 ecr 0,sackOK,eol], length 0 10:50:47.736744 IP 177.54.114.32.52692 > **MYPUBLICIP**.80: Flags [S], seq 2306349141, win 65535, options [mss 1440,nop,wscale 4,nop,nop,TS val 292574604 ecr 0,sackOK,eol], length 0 10:50:47.744056 IP 177.54.114.32.52691 > **MYPUBLICIP**.80: Flags [S], seq 2697243517, win 65535, options [mss 1440,nop,wscale 4,nop,nop,TS val 292574604 ecr 0,sackOK,eol], length 0 10:50:47.946942 IP 177.54.114.32.52693 > **MYPUBLICIP**.80: Flags [S], seq 3700386367, win 65535, options [mss 1440,nop,wscale 4,nop,nop,TS val 292574802 ecr 0,sackOK,eol], length 0 10:50:48.840885 IP 177.54.114.32.52692 > **MYPUBLICIP**.80: Flags [S], seq 2306349141, win 65535, options [mss 1440,nop,wscale 4,nop,nop,TS val 292575697 ecr 0,sackOK,eol], length 0 10:50:48.848524 IP 177.54.114.32.52691 > **MYPUBLICIP**.80: Flags [S], seq 2697243517, win 65535, options [mss 1440,nop,wscale 4,nop,nop,TS val 292575697 ecr 0,sackOK,eol], length 0 10:50:49.050437 IP 177.54.114.32.52693 > **MYPUBLICIP**.80: Flags [S], seq 3700386367, win 65535, options [mss 1440,nop,wscale 4,nop,nop,TS val 292575893 ecr 0,sackOK,eol], length 0 10:50:49.944101 IP 177.54.114.32.52692 > **MYPUBLICIP**.80: Flags [S], seq 2306349141, win 65535, options [mss 1440,nop,wscale 4,nop,nop,TS val 292576786 ecr 0,sackOK,eol], length 0 10:50:49.951483 IP 177.54.114.32.52691 > **MYPUBLICIP**.80: Flags [S], seq 2697243517, win 65535, options [mss 1440,nop,wscale 4,nop,nop,TS val 292576786 ecr 0,sackOK,eol], length 0 10:50:50.153465 IP 177.54.114.32.52693 > **MYPUBLICIP**.80: Flags [S], seq 3700386367, win 65535, options [mss 1440,nop,wscale 4,nop,nop,TS val 292576984 ecr 0,sackOK,eol], length 0 10:50:51.048314 IP 177.54.114.32.52692 > **MYPUBLICIP**.80: Flags [S], seq 2306349141, win 65535, options [mss 1440,nop,wscale 4,nop,nop,TS val 292577876 ecr 0,sackOK,eol], length 0 10:50:51.055642 IP 177.54.114.32.52691 > **MYPUBLICIP**.80: Flags [S], seq 2697243517, win 65535, options [mss 1440,nop,wscale 4,nop,nop,TS val 292577876 ecr 0,sackOK,eol], length 0 10:50:51.258095 IP 177.54.114.32.52693 > **MYPUBLICIP**.80: Flags [S], seq 3700386367, win 65535, options [mss 1440,nop,wscale 4,nop,nop,TS val 292578074 ecr 0,sackOK,eol], length 0 10:50:52.153507 IP 177.54.114.32.52692 > **MYPUBLICIP**.80: Flags [S], seq 2306349141, win 65535, options [mss 1440,nop,wscale 4,nop,nop,TS val 292578964 ecr 0,sackOK,eol], length 0 10:50:52.160489 IP 177.54.114.32.52691 > **MYPUBLICIP**.80: Flags [S], seq 2697243517, win 65535, options [mss 1440,nop,wscale 4,nop,nop,TS val 292578964 ecr 0,sackOK,eol], length 0 10:50:52.367694 IP 177.54.114.32.52693 > **MYPUBLICIP**.80: Flags [S], seq 3700386367, win 65535, options [mss 1440,nop,wscale 4,nop,nop,TS val 292579162 ecr 0,sackOK,eol], length 0 10:50:54.261032 IP 177.54.114.32.52692 > **MYPUBLICIP**.80: Flags [S], seq 2306349141, win 65535, options [mss 1440,nop,wscale 4,nop,nop,TS val 292581049 ecr 0,sackOK,eol], length 0 10:50:54.268266 IP 177.54.114.32.52691 > **MYPUBLICIP**.80: Flags [S], seq 2697243517, win 65535, options [mss 1440,nop,wscale 4,nop,nop,TS val 292581049 ecr 0,sackOK,eol], length 0 10:50:54.470990 IP 177.54.114.32.52693 > **MYPUBLICIP**.80: Flags [S], seq 3700386367, win 65535, options [mss 1440,nop,wscale 4,nop,nop,TS val 292581246 ecr 0,sackOK,eol], length 0 10:50:58.360312 IP 177.54.114.32.52692 > **MYPUBLICIP**.80: Flags [S], seq 2306349141, win 65535, options [mss 1440,sackOK,eol], length 0 10:50:58.367643 IP 177.54.114.32.52691 > **MYPUBLICIP**.80: Flags [S], seq 2697243517, win 65535, options [mss 1440,sackOK,eol], length 0 10:50:58.570472 IP 177.54.114.32.52693 > **MYPUBLICIP**.80: Flags [S], seq 3700386367, win 65535, options [mss 1440,sackOK,eol], length 0 Then It seems ipvsadm redirects packet from eth1 to eth0 correctly and rewrites the packet to match the new destination IP (my realserver): root@lvs1:~# tcpdump -ni eth0:3 not host 224.0.0.18 tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on eth0:3, link-type EN10MB (Ethernet), capture size 65535 bytes 10:48:20.506530 IP 177.54.114.32.52681 > 192.168.12.4.80: Flags [S], seq 1943196812, win 65535, options [mss 1440,nop,wscale 4,nop,nop,TS val 292429109 ecr 0,sackOK,eol], length 0 10:48:20.506569 IP 177.54.114.32.52680 > 192.168.12.4.80: Flags [S], seq 2842631520, win 65535, options [mss 1440,nop,wscale 4,nop,nop,TS val 292429109 ecr 0,sackOK,eol], length 0 10:48:20.759464 IP 177.54.114.32.52682 > 192.168.12.4.80: Flags [S], seq 267664571, win 65535, options [mss 1440,nop,wscale 4,nop,nop,TS val 292429356 ecr 0,sackOK,eol], length 0 10:48:21.518465 IP 177.54.114.32.52681 > 192.168.12.4.80: Flags [S], seq 1943196812, win 65535, options [mss 1440,nop,wscale 4,nop,nop,TS val 292430109 ecr 0,sackOK,eol], length 0 10:48:21.518857 IP 177.54.114.32.52680 > 192.168.12.4.80: Flags [S], seq 2842631520, win 65535, options [mss 1440,nop,wscale 4,nop,nop,TS val 292430109 ecr 0,sackOK,eol], length 0 10:48:21.823556 IP 177.54.114.32.52682 > 192.168.12.4.80: Flags [S], seq 267664571, win 65535, options [mss 1440,nop,wscale 4,nop,nop,TS val 292430406 ecr 0,sackOK,eol], length 0 10:48:22.621495 IP 177.54.114.32.52681 > 192.168.12.4.80: Flags [S], seq 1943196812, win 65535, options [mss 1440,nop,wscale 4,nop,nop,TS val 292431203 ecr 0,sackOK,eol], length 0 10:48:22.622106 IP 177.54.114.32.52680 > 192.168.12.4.80: Flags [S], seq 2842631520, win 65535, options [mss 1440,nop,wscale 4,nop,nop,TS val 292431203 ecr 0,sackOK,eol], length 0 10:48:22.925555 IP 177.54.114.32.52682 > 192.168.12.4.80: Flags [S], seq 267664571, win 65535, options [mss 1440,nop,wscale 4,nop,nop,TS val 292431500 ecr 0,sackOK,eol], length 0 10:48:23.726898 IP 177.54.114.32.52681 > 192.168.12.4.80: Flags [S], seq 1943196812, win 65535, options [mss 1440,nop,wscale 4,nop,nop,TS val 292432295 ecr 0,sackOK,eol], length 0 10:48:23.727515 IP 177.54.114.32.52680 > 192.168.12.4.80: Flags [S], seq 2842631520, win 65535, options [mss 1440,nop,wscale 4,nop,nop,TS val 292432295 ecr 0,sackOK,eol], length 0 10:48:24.030893 IP 177.54.114.32.52682 > 192.168.12.4.80: Flags [S], seq 267664571, win 65535, options [mss 1440,nop,wscale 4,nop,nop,TS val 292432592 ecr 0,sackOK,eol], length 0 10:48:24.831695 IP 177.54.114.32.52681 > 192.168.12.4.80: Flags [S], seq 1943196812, win 65535, options [mss 1440,nop,wscale 4,nop,nop,TS val 292433387 ecr 0,sackOK,eol], length 0 10:48:24.831850 IP 177.54.114.32.52680 > 192.168.12.4.80: Flags [S], seq 2842631520, win 65535, options [mss 1440,nop,wscale 4,nop,nop,TS val 292433387 ecr 0,sackOK,eol], length 0 10:48:25.132910 IP 177.54.114.32.52682 > 192.168.12.4.80: Flags [S], seq 267664571, win 65535, options [mss 1440,nop,wscale 4,nop,nop,TS val 292433685 ecr 0,sackOK,eol], length 0 10:48:25.509307 ARP, Request who-has 192.168.12.4 tell 192.168.12.2, length 28 10:48:25.510077 ARP, Reply 192.168.12.4 is-at be:b2:76:d3:4f:ff, length 28 10:48:25.934629 IP 177.54.114.32.52680 > 192.168.12.4.80: Flags [S], seq 2842631520, win 65535, options [mss 1440,nop,wscale 4,nop,nop,TS val 292434474 ecr 0,sackOK,eol], length 0 10:48:25.934664 IP 177.54.114.32.52681 > 192.168.12.4.80: Flags [S], seq 1943196812, win 65535, options [mss 1440,nop,wscale 4,nop,nop,TS val 292434474 ecr 0,sackOK,eol], length 0 10:48:26.235733 IP 177.54.114.32.52682 > 192.168.12.4.80: Flags [S], seq 267664571, win 65535, options [mss 1440,nop,wscale 4,nop,nop,TS val 292434773 ecr 0,sackOK,eol], length 0 10:48:28.040339 IP 177.54.114.32.52681 > 192.168.12.4.80: Flags [S], seq 1943196812, win 65535, options [mss 1440,nop,wscale 4,nop,nop,TS val 292436562 ecr 0,sackOK,eol], length 0 10:48:28.040803 IP 177.54.114.32.52680 > 192.168.12.4.80: Flags [S], seq 2842631520, win 65535, options [mss 1440,nop,wscale 4,nop,nop,TS val 292436562 ecr 0,sackOK,eol], length 0 10:48:28.341964 IP 177.54.114.32.52682 > 192.168.12.4.80: Flags [S], seq 267664571, win 65535, options [mss 1440,nop,wscale 4,nop,nop,TS val 292436858 ecr 0,sackOK,eol], length 0 10:48:32.177348 IP 177.54.114.32.52681 > 192.168.12.4.80: Flags [S], seq 1943196812, win 65535, options [mss 1440,sackOK,eol], length 0 10:48:32.177658 IP 177.54.114.32.52680 > 192.168.12.4.80: Flags [S], seq 2842631520, win 65535, options [mss 1440,sackOK,eol], length 0 10:48:32.479714 IP 177.54.114.32.52682 > 192.168.12.4.80: Flags [S], seq 267664571, win 65535, options [mss 1440,sackOK,eol], length 0 10:48:40.464706 IP 177.54.114.32.52681 > 192.168.12.4.80: Flags [S], seq 1943196812, win 65535, options [mss 1440,sackOK,eol], length 0 10:48:40.464740 IP 177.54.114.32.52680 > 192.168.12.4.80: Flags [S], seq 2842631520, win 65535, options [mss 1440,sackOK,eol], length 0 10:48:40.566405 IP 177.54.114.32.52682 > 192.168.12.4.80: Flags [S], seq 267664571, win 65535, options [mss 1440,sackOK,eol], length 0 Then I have at the realserver (192.168.12.4): tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on eth0, link-type EN10MB (Ethernet), capture size 65535 bytes 10:53:31.819480 IP 177.54.114.32.52711 > 192.168.12.4.80: Flags [S], seq 329952672, win 65535, options [mss 1440,nop,wscale 4,nop,nop,TS val 292739115 ecr 0,sackOK,eol], length 0 10:53:31.820101 IP 177.54.114.32.52712 > 192.168.12.4.80: Flags [S], seq 2821848538, win 65535, options [mss 1440,nop,wscale 4,nop,nop,TS val 292739115 ecr 0,sackOK,eol], length 0 10:53:32.072647 IP 177.54.114.32.52713 > 192.168.12.4.80: Flags [S], seq 3482892199, win 65535, options [mss 1440,nop,wscale 4,nop,nop,TS val 292739362 ecr 0,sackOK,eol], length 0 10:53:32.868727 IP 177.54.114.32.52712 > 192.168.12.4.80: Flags [S], seq 2821848538, win 65535, options [mss 1440,nop,wscale 4,nop,nop,TS val 292740150 ecr 0,sackOK,eol], length 0 10:53:32.868748 IP 177.54.114.32.52711 > 192.168.12.4.80: Flags [S], seq 329952672, win 65535, options [mss 1440,nop,wscale 4,nop,nop,TS val 292740150 ecr 0,sackOK,eol], length 0 10:53:33.171616 IP 177.54.114.32.52713 > 192.168.12.4.80: Flags [S], seq 3482892199, win 65535, options [mss 1440,nop,wscale 4,nop,nop,TS val 292740447 ecr 0,sackOK,eol], length 0 10:53:33.971436 IP 177.54.114.32.52711 > 192.168.12.4.80: Flags [S], seq 329952672, win 65535, options [mss 1440,nop,wscale 4,nop,nop,TS val 292741243 ecr 0,sackOK,eol], length 0 10:53:33.971570 IP 177.54.114.32.52712 > 192.168.12.4.80: Flags [S], seq 2821848538, win 65535, options [mss 1440,nop,wscale 4,nop,nop,TS val 292741243 ecr 0,sackOK,eol], length 0 10:53:34.274381 IP 177.54.114.32.52713 > 192.168.12.4.80: Flags [S], seq 3482892199, win 65535, options [mss 1440,nop,wscale 4,nop,nop,TS val 292741542 ecr 0,sackOK,eol], length 0 10:53:35.076124 IP 177.54.114.32.52711 > 192.168.12.4.80: Flags [S], seq 329952672, win 65535, options [mss 1440,nop,wscale 4,nop,nop,TS val 292742336 ecr 0,sackOK,eol], length 0 10:53:35.076265 IP 177.54.114.32.52712 > 192.168.12.4.80: Flags [S], seq 2821848538, win 65535, options [mss 1440,nop,wscale 4,nop,nop,TS val 292742336 ecr 0,sackOK,eol], length 0 10:53:35.378447 IP 177.54.114.32.52713 > 192.168.12.4.80: Flags [S], seq 3482892199, win 65535, options [mss 1440,nop,wscale 4,nop,nop,TS val 292742630 ecr 0,sackOK,eol], length 0 10:53:36.179796 IP 177.54.114.32.52711 > 192.168.12.4.80: Flags [S], seq 329952672, win 65535, options [mss 1440,nop,wscale 4,nop,nop,TS val 292743422 ecr 0,sackOK,eol], length 0 10:53:36.179936 IP 177.54.114.32.52712 > 192.168.12.4.80: Flags [S], seq 2821848538, win 65535, options [mss 1440,nop,wscale 4,nop,nop,TS val 292743422 ecr 0,sackOK,eol], length 0 10:53:36.482287 IP 177.54.114.32.52713 > 192.168.12.4.80: Flags [S], seq 3482892199, win 65535, options [mss 1440,nop,wscale 4,nop,nop,TS val 292743719 ecr 0,sackOK,eol], length 0 10:53:36.834363 ARP, Request who-has 192.168.12.4 tell 192.168.12.2, length 28 10:53:36.834383 ARP, Reply 192.168.12.4 is-at be:b2:76:d3:4f:ff, length 28 10:53:37.276263 IP 177.54.114.32.52711 > 192.168.12.4.80: Flags [S], seq 329952672, win 65535, options [mss 1440,nop,wscale 4,nop,nop,TS val 292744514 ecr 0,sackOK,eol], length 0 10:53:37.284936 IP 177.54.114.32.52712 > 192.168.12.4.80: Flags [S], seq 2821848538, win 65535, options [mss 1440,nop,wscale 4,nop,nop,TS val 292744514 ecr 0,sackOK,eol], length 0 10:53:37.587990 IP 177.54.114.32.52713 > 192.168.12.4.80: Flags [S], seq 3482892199, win 65535, options [mss 1440,nop,wscale 4,nop,nop,TS val 292744810 ecr 0,sackOK,eol], length 0 10:53:39.385848 IP 177.54.114.32.52711 > 192.168.12.4.80: Flags [S], seq 329952672, win 65535, options [mss 1440,nop,wscale 4,nop,nop,TS val 292746596 ecr 0,sackOK,eol], length 0 10:53:39.394545 IP 177.54.114.32.52712 > 192.168.12.4.80: Flags [S], seq 2821848538, win 65535, options [mss 1440,nop,wscale 4,nop,nop,TS val 292746596 ecr 0,sackOK,eol], length 0 10:53:39.697490 IP 177.54.114.32.52713 > 192.168.12.4.80: Flags [S], seq 3482892199, win 65535, options [mss 1440,nop,wscale 4,nop,nop,TS val 292746893 ecr 0,sackOK,eol], length 0 10:53:43.920488 IP 177.54.114.32.52711 > 192.168.12.4.80: Flags [S], seq 329952672, win 65535, options [mss 1440,sackOK,eol], length 0 10:53:43.928976 IP 177.54.114.32.52712 > 192.168.12.4.80: Flags [S], seq 2821848538, win 65535, options [mss 1440,sackOK,eol], length 0 10:53:43.930672 IP 177.54.114.32.52713 > 192.168.12.4.80: Flags [S], seq 3482892199, win 65535, options [mss 1440,sackOK,eol], length 0 And thats it, I can't see any response back from realserver to gateway and those syn packets are all I can see from/to those ips related to LVS. PS: I try to setup some MASQUERADE iptables nat rule, but it didn't change nothing. What am I missing? 2014-03-25 8:41 GMT-03:00 Tiago <syt...@gmail.com>: > Yes, I've said about RPF, but I think I should see the packets "trying" to > go out I guess. > > I'll give a try using NAT, because at my first try it didn't work either. > I get back to you this morning yet. > > Just a note, I'm using ubuntu 12.04-4 server. Is any problem related to it? > > > 2014-03-25 4:42 GMT-03:00 Ferenc Wagner <wf...@niif.hu>: > > Malcolm Turnbull <malc...@loadbalancer.org> writes: >> >> > But rp_filter just controls where the reply packet goes >> >> As far as I know, rp_filter is on the input side: it drops a packet if >> the hypothetical reply packet *would* go out on a different interface to >> the one the packet arrived on. Thus it must be switched off in >> asymmetric routing scenarios. >> -- >> Regards, >> Feri. >> >> _______________________________________________ >> Please read the documentation before posting - it's available at: >> http://www.linuxvirtualserver.org/ >> >> LinuxVirtualServer.org mailing list - lvs-users@LinuxVirtualServer.org >> Send requests to lvs-users-requ...@linuxvirtualserver.org >> or go to http://lists.graemef.net/mailman/listinfo/lvs-users >> > > _______________________________________________ Please read the documentation before posting - it's available at: http://www.linuxvirtualserver.org/ LinuxVirtualServer.org mailing list - lvs-users@LinuxVirtualServer.org Send requests to lvs-users-requ...@linuxvirtualserver.org or go to http://lists.graemef.net/mailman/listinfo/lvs-users