[Vserver] problem with netfilter and vservers
Hello Gang, Ive been having some serious problems with vservers and iptables! Sometimes i need to had SNAT rules for my vservers to route outside the root server and, someother times, i cant access my vservers from the outside :o( Take this script for example: http://lms.ispgaya.pt/goodies/iptables On the server where i use it everything worked like a charm! Until...i had to add support in the kernel for another NIC. [EMAIL PROTECTED] ~# lspci 00:00.0 Host bridge: Intel Corp.: Unknown device 2570 (rev 02) 00:01.0 PCI bridge: Intel Corp.: Unknown device 2571 (rev 02) 00:1d.0 USB Controller: Intel Corp.: Unknown device 24d2 (rev 02) 00:1d.1 USB Controller: Intel Corp.: Unknown device 24d4 (rev 02) 00:1d.2 USB Controller: Intel Corp.: Unknown device 24d7 (rev 02) 00:1d.3 USB Controller: Intel Corp.: Unknown device 24de (rev 02) 00:1d.7 USB Controller: Intel Corp.: Unknown device 24dd (rev 02) 00:1e.0 PCI bridge: Intel Corp. 82801BA/CA/DB PCI Bridge (rev c2) 00:1f.0 ISA bridge: Intel Corp.: Unknown device 24d0 (rev 02) 00:1f.1 IDE interface: Intel Corp.: Unknown device 24db (rev 02) 00:1f.3 SMBus: Intel Corp.: Unknown device 24d3 (rev 02) 00:1f.5 Multimedia audio controller: Intel Corp.: Unknown device 24d5 (rev 02) 01:00.0 VGA compatible controller: nVidia Corporation RIVA TNT2 Model 64 (rev 15) 02:05.0 Ethernet controller: 3Com Corporation: Unknown device 1700 (rev 12) 02:0a.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL-8139/8139C/8139C+ (rev 10) 02:0b.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL-8139/8139C/8139C+ (rev 10) 02:0d.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL-8139/8139C/8139C+ (rev 10) [EMAIL PROTECTED] ~# Since the 3com (gigabit builtin) ethernet device is unknown, i added support to it and recompiled the kernel. After rebooting the machine, i couldnt access any services on 192.168.3.81 (vserver called ciisp) from the outside). I disabled support for that NIC again, recompiled and rebootedand everything went back to normal again! Can anybody help me with this? Is this normal behaviour? I also dont understand why some vservers need for me to -j SNAT --to root-server and others dont! Thanks in advance, + | Luís Miguel Ferreira da Silva | Network Administrator @ISPGaya | Instituto Superior Politécnico Gaya | Rua António Rodrigues da Rocha, 291/341 | Sto. Ovídio 4400-025 V. N. de Gaia | Tel: +351 223745730/3/5 | GSM: +351 912671471 +351 936371253 + Este email foi enviado via o webmail do ISPGaya Instituto Superior Politécnico Gaya binUQlPZuJsAl.bin Description: Chave Pública PGP ___ Vserver mailing list Vserver@list.linux-vserver.org http://list.linux-vserver.org/mailman/listinfo/vserver
Re: [Vserver] problem with netfilter and vservers
On Sat, Sep 17, 2005 at 12:52:00PM +0100, Luís Miguel Silva wrote: Hello Gang, Ive been having some serious problems with vservers and iptables! Sometimes i need to had SNAT rules for my vservers to route outside the root server and, someother times, i cant access my vservers from the outside :o( Take this script for example: http://lms.ispgaya.pt/goodies/iptables *sigh* well, let's interpret parts of it ... $IPTABLES -A POSTROUTING -t nat -s 192.168.3.0/24 -d 192.168.2.0/24 -j ACCEPT $IPTABLES -A POSTROUTING -t nat -s 192.168.3.0/24 -d 192.168.4.0/24 -j ACCEPT accept network traffic from 192.168.3 to 192.168.2 and 192.168.4 (unmodified, unconditional) $IPTABLES -A POSTROUTING -t nat -s 192.168.3.0/24 -d 193.126.109.240/255.255.255.248 -j ACCEPT same for traffic from 192.168.3 to 193.126.109.240-247 hmm, why would KQPT Network Operations want packets from a private network? $IPTABLES -A POSTROUTING -t nat -s 192.168.3.0/24 -d 193.126.229.32/255.255.255.248 -j ACCEPT hmm, seems they definitely want private traffic :) $IPTABLES -A POSTROUTING -t nat -s 192.168.3.0/24 -d ! 192.168.0.0/16 -j SNAT --to 192.168.3.2 everything not destinated at 192.168 will appear as private IP 192.168.3.2 (strange, why would we want that?) $IPTABLES -A POSTROUTING -t nat -s 172.28.10.0/24 -d ! 172.28.10.0/24 -j SNAT --to-source 172.28.10.254 and similar for 172.28.10, which had no role yet, but seem to be valid IPs for output, and we SNAT them all to 172.28.10.254 ... so this setup assumes that both 192.168.3.2 and 172.28.10.254 can reach the outside (whatever that might mean) and that there are either two routes or the router can handle both IPs ... On the server where i use it everything worked like a charm! Until...i had to add support in the kernel for another NIC. [EMAIL PROTECTED] ~# lspci 00:00.0 Host bridge: Intel Corp.: Unknown device 2570 (rev 02) 00:01.0 PCI bridge: Intel Corp.: Unknown device 2571 (rev 02) 00:1d.0 USB Controller: Intel Corp.: Unknown device 24d2 (rev 02) 00:1d.1 USB Controller: Intel Corp.: Unknown device 24d4 (rev 02) 00:1d.2 USB Controller: Intel Corp.: Unknown device 24d7 (rev 02) 00:1d.3 USB Controller: Intel Corp.: Unknown device 24de (rev 02) 00:1d.7 USB Controller: Intel Corp.: Unknown device 24dd (rev 02) 00:1e.0 PCI bridge: Intel Corp. 82801BA/CA/DB PCI Bridge (rev c2) 00:1f.0 ISA bridge: Intel Corp.: Unknown device 24d0 (rev 02) 00:1f.1 IDE interface: Intel Corp.: Unknown device 24db (rev 02) 00:1f.3 SMBus: Intel Corp.: Unknown device 24d3 (rev 02) 00:1f.5 Multimedia audio controller: Intel Corp.: Unknown device 24d5 (rev 02) 01:00.0 VGA compatible controller: nVidia Corporation RIVA TNT2 Model 64 (rev 15) 02:05.0 Ethernet controller: 3Com Corporation: Unknown device 1700 (rev 12) 02:0a.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL-8139/8139C/8139C+ (rev 10) 02:0b.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL-8139/8139C/8139C+ (rev 10) 02:0d.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL-8139/8139C/8139C+ (rev 10) hmm, four different network controllers easily confuse the unpracticed eye and often also the admin attached to it :) [EMAIL PROTECTED] ~# Since the 3com (gigabit builtin) ethernet device is unknown, i added support to it and recompiled the kernel. After rebooting the machine, i couldnt access any services on 192.168.3.81 (vserver called ciisp) from the outside). hmm, let's see .. *turns on the seeing orb* ... ah, looks like you 3com card got detected _before_ the other three realtek ones, so it was named eth0, instead of eth4 ... hmmm, ... and that probably messed up all other NICs, as they are now eth1 instead of eth0, eth2 instead of eth1 ... now, most likely some of your guests have the interface coded (it's a little blurry now) and other just an ip I disabled support for that NIC again, recompiled and rebootedand everything went back to normal again! Can anybody help me with this? Is this normal behaviour? I guess yes, it is the typical linux networking behaviour so nothing critical ... I also dont understand why some vservers need for me to -j SNAT --to root-server and others dont! this also escapes my imagination (basically because of lack of information) but I 'assume' that some have real IPs and/or communicate on private IPs where others have to use the host IP for outgoing traffic ... best, Herbert Thanks in advance, + | Luís Miguel Ferreira da Silva | Network Administrator @ISPGaya | Instituto Superior Politécnico Gaya | Rua António Rodrigues da Rocha, 291/341 | Sto. Ovídio 4400-025 V. N. de Gaia | Tel: +351 223745730/3/5 | GSM: +351 912671471 +351 936371253 + Este email foi enviado via o webmail do ISPGaya Instituto Superior Politécnico Gaya
Re: [Vserver] problem with netfilter and vservers
Dear Herbert, Allthough i really thought your email was full of sarcasm (*really* sorry if i missinterpreted it :o) ), please read along the email to find some comments replying your comments... *sigh* well, lets interpret parts of it ... $IPTABLES -A POSTROUTING -t nat -s 192.168.3.0/24 -d 192.168.2.0/24 -j ACCEPT $IPTABLES -A POSTROUTING -t nat -s 192.168.3.0/24 -d 192.168.4.0/24 -j ACCEPT accept network traffic from 192.168.3 to 192.168.2 and 192.168.4 (unmodified, unconditional) $IPTABLES -A POSTROUTING -t nat -s 192.168.3.0/24 -d 193.126.109.240/255.255.255.248 -j ACCEPT same for traffic from 192.168.3 to 193.126.109.240-247 hmm, why would KQPT Network Operations want packets from a private network? $IPTABLES -A POSTROUTING -t nat -s 192.168.3.0/24 -d 193.126.229.32/255.255.255.248 -j ACCEPT hmm, seems they definitely want private traffic :) $IPTABLES -A POSTROUTING -t nat -s 192.168.3.0/24 -d ! 192.168.0.0/16 -j SNAT --to 192.168.3.2 everything not destinated at 192.168 will appear as private IP 192.168.3.2 (strange, why would we want that?) $IPTABLES -A POSTROUTING -t nat -s 172.28.10.0/24 -d ! 172.28.10.0/24 -j SNAT --to-source 172.28.10.254 and similar for 172.28.10, which had no role yet, but seem to be valid IPs for output, and we SNAT them all to 172.28.10.254 ... so this setup assumes that both 192.168.3.2 and 172.28.10.254 can reach the outside (whatever that might mean) and that there are either two routes or the router can handle both IPs ... On the server where i use it everything worked like a charm! Until...i had to add support in the kernel for another NIC. [EMAIL PROTECTED] ~# lspci 00:00.0 Host bridge: Intel Corp.: Unknown device 2570 (rev 02) 00:01.0 PCI bridge: Intel Corp.: Unknown device 2571 (rev 02) 00:1d.0 USB Controller: Intel Corp.: Unknown device 24d2 (rev 02) 00:1d.1 USB Controller: Intel Corp.: Unknown device 24d4 (rev 02) 00:1d.2 USB Controller: Intel Corp.: Unknown device 24d7 (rev 02) 00:1d.3 USB Controller: Intel Corp.: Unknown device 24de (rev 02) 00:1d.7 USB Controller: Intel Corp.: Unknown device 24dd (rev 02) 00:1e.0 PCI bridge: Intel Corp. 82801BA/CA/DB PCI Bridge (rev c2) 00:1f.0 ISA bridge: Intel Corp.: Unknown device 24d0 (rev 02) 00:1f.1 IDE interface: Intel Corp.: Unknown device 24db (rev 02) 00:1f.3 SMBus: Intel Corp.: Unknown device 24d3 (rev 02) 00:1f.5 Multimedia audio controller: Intel Corp.: Unknown device 24d5 (rev 02) 01:00.0 VGA compatible controller: nVidia Corporation RIVA TNT2 Model 64 (rev 15) 02:05.0 Ethernet controller: 3Com Corporation: Unknown device 1700 (rev 12) 02:0a.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL-8139/8139C/8139C+ (rev 10) 02:0b.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL-8139/8139C/8139C+ (rev 10) 02:0d.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL-8139/8139C/8139C+ (rev 10) hmm, four different network controllers easily confuse the unpracticed eye and often also the admin attached to it :) [EMAIL PROTECTED] ~# Since the 3com (gigabit builtin) ethernet device is unknown, i added support to it and recompiled the kernel. After rebooting the machine, i couldnt access any services on 192.168.3.81 (vserver called ciisp) from the outside). hmm, lets see .. *turns on the seeing orb* ... ah, looks like you 3com card got detected _before_ the other three realtek ones, so it was named eth0, instead of eth4 ... hmmm, ... and that probably messed up all other NICs, as they are now eth1 instead of eth0, eth2 instead of eth1 ... now, most likely some of your guests have the interface coded (its a little blurry now) and other just an ip I disabled support for that NIC again, recompiled and rebootedand everything went back to normal again! Can anybody help me with this? Is this normal behaviour? I guess yes, it is the typical linux networking behaviour so nothing critical ... I also dont understand why some vservers need for me to -j SNAT --to root-server and others dont! this also escapes my imagination (basically because of lack of information) but I assume that some have real IPs and/or communicate on private IPs where others have to use the host IP for outgoing traffic ... best, Herbert There are 4 NICs on the root server called leonardo-root. eth0 - 192.168.3.2 [connecting to our internal network / outside world] eth1 - 10.69.69.1 [connecting to the outside world (ADSL connection)] eth2 - 172.28.10.254 [connecting to some IP cameras] eth3 - not used...[until i added support for the 3com card off course :o)] Also, as you stated, after i added support to the 3com card, all the other NICs switched names... Well, a little comment on this :o) 1º of all, im no idiot and i obviously know that and changed all the cables 2º why did you assume that? 3º ever thought i could be using modules in my kernel and aliasing the NICs? ;o) Either way...by default, the packets to unknown networks go throw eth1 [gw: 10.69.69.254]. There
Re: [Vserver] problem with netfilter and vservers
On Saturday 17 September 2005 05:38 pm, Luís Miguel Silva wrote: I had the problem with a 3com switching the order too. i wanted that last, so I compiled it as a module rather than internal kernel support and loaded the module in the local startup file last thing. that insured it became eth2 on my system always. If you use the init script to start the vservers, since that wont be called until the ethernet devices are loaded, why not load the module in the init script start() code before anything else is done... i would think that would keep it as the last adapter number. I need a nap so I only skimmed through this but reading the discussion about ethx flipping around reminded me of this. Dear Herbert, Allthough i really thought your email was full of sarcasm (*really* sorry if i missinterpreted it :o) ), please read along the email to find some comments replying your comments... *sigh* well, lets interpret parts of it ... $IPTABLES -A POSTROUTING -t nat -s 192.168.3.0/24 -d 192.168.2.0/24 -j ACCEPT $IPTABLES -A POSTROUTING -t nat -s 192.168.3.0/24 -d 192.168.4.0/24 -j ACCEPT accept network traffic from 192.168.3 to 192.168.2 and 192.168.4 (unmodified, unconditional) $IPTABLES -A POSTROUTING -t nat -s 192.168.3.0/24 -d 193.126.109.240/255.255.255.248 -j ACCEPT same for traffic from 192.168.3 to 193.126.109.240-247 hmm, why would KQPT Network Operations want packets from a private network? $IPTABLES -A POSTROUTING -t nat -s 192.168.3.0/24 -d 193.126.229.32/255.255.255.248 -j ACCEPT hmm, seems they definitely want private traffic :) $IPTABLES -A POSTROUTING -t nat -s 192.168.3.0/24 -d ! 192.168.0.0/16 -j SNAT --to 192.168.3.2 everything not destinated at 192.168 will appear as private IP 192.168.3.2 (strange, why would we want that?) $IPTABLES -A POSTROUTING -t nat -s 172.28.10.0/24 -d ! 172.28.10.0/24 -j SNAT --to-source 172.28.10.254 and similar for 172.28.10, which had no role yet, but seem to be valid IPs for output, and we SNAT them all to 172.28.10.254 ... so this setup assumes that both 192.168.3.2 and 172.28.10.254 can reach the outside (whatever that might mean) and that there are either two routes or the router can handle both IPs ... On the server where i use it everything worked like a charm! Until...i had to add support in the kernel for another NIC. [EMAIL PROTECTED] ~# lspci 00:00.0 Host bridge: Intel Corp.: Unknown device 2570 (rev 02) 00:01.0 PCI bridge: Intel Corp.: Unknown device 2571 (rev 02) 00:1d.0 USB Controller: Intel Corp.: Unknown device 24d2 (rev 02) 00:1d.1 USB Controller: Intel Corp.: Unknown device 24d4 (rev 02) 00:1d.2 USB Controller: Intel Corp.: Unknown device 24d7 (rev 02) 00:1d.3 USB Controller: Intel Corp.: Unknown device 24de (rev 02) 00:1d.7 USB Controller: Intel Corp.: Unknown device 24dd (rev 02) 00:1e.0 PCI bridge: Intel Corp. 82801BA/CA/DB PCI Bridge (rev c2) 00:1f.0 ISA bridge: Intel Corp.: Unknown device 24d0 (rev 02) 00:1f.1 IDE interface: Intel Corp.: Unknown device 24db (rev 02) 00:1f.3 SMBus: Intel Corp.: Unknown device 24d3 (rev 02) 00:1f.5 Multimedia audio controller: Intel Corp.: Unknown device 24d5 (rev 02) 01:00.0 VGA compatible controller: nVidia Corporation RIVA TNT2 Model 64 (rev 15) 02:05.0 Ethernet controller: 3Com Corporation: Unknown device 1700 (rev 12) 02:0a.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL-8139/8139C/8139C+ (rev 10) 02:0b.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL-8139/8139C/8139C+ (rev 10) 02:0d.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL-8139/8139C/8139C+ (rev 10) hmm, four different network controllers easily confuse the unpracticed eye and often also the admin attached to it :) [EMAIL PROTECTED] ~# Since the 3com (gigabit builtin) ethernet device is unknown, i added support to it and recompiled the kernel. After rebooting the machine, i couldnt access any services on 192.168.3.81 (vserver called ciisp) from the outside). hmm, lets see .. *turns on the seeing orb* ... ah, looks like you 3com card got detected _before_ the other three realtek ones, so it was named eth0, instead of eth4 ... hmmm, ... and that probably messed up all other NICs, as they are now eth1 instead of eth0, eth2 instead of eth1 ... now, most likely some of your guests have the interface coded (its a little blurry now) and other just an ip I disabled support for that NIC again, recompiled and rebootedand everything went back to normal again! Can anybody help me with this? Is this normal behaviour? I guess yes, it is the typical linux networking behaviour so nothing critical ... I also dont understand why some vservers need for me to -j SNAT --to root-server and others dont! this also escapes my imagination (basically because of lack of information) but I assume that
Re: [Vserver] problem with netfilter and vservers
On Sat, Sep 17, 2005 at 10:38:58PM +0100, Luís Miguel Silva wrote: Dear Herbert, Allthough i really thought your email was full of sarcasm (*really* sorry if i missinterpreted it :o) ), please read along the email to find some comments replying your comments... just a little bit of sarcasm ... and just because I find it funny that most folks simply _assume_ that everybody knows their network setup, number of machines, routers, etc ... so the message (and it's not the first time, and for sure will not be the last time) is, please provide more information ... from the beginning :) networking isn't trivial, you know that, and doing a diagnosis based on _assumptions_ can only be funny and general at best ... *sigh* well, lets interpret parts of it ... $IPTABLES -A POSTROUTING -t nat -s 192.168.3.0/24 -d 192.168.2.0/24 -j ACCEPT $IPTABLES -A POSTROUTING -t nat -s 192.168.3.0/24 -d 192.168.4.0/24 -j ACCEPT accept network traffic from 192.168.3 to 192.168.2 and 192.168.4 (unmodified, unconditional) $IPTABLES -A POSTROUTING -t nat -s 192.168.3.0/24 -d 193.126.109.240/255.255.255.248 -j ACCEPT same for traffic from 192.168.3 to 193.126.109.240-247 hmm, why would KQPT Network Operations want packets from a private network? $IPTABLES -A POSTROUTING -t nat -s 192.168.3.0/24 -d 193.126.229.32/255.255.255.248 -j ACCEPT hmm, seems they definitely want private traffic :) $IPTABLES -A POSTROUTING -t nat -s 192.168.3.0/24 -d ! 192.168.0.0/16 -j SNAT --to 192.168.3.2 everything not destinated at 192.168 will appear as private IP 192.168.3.2 (strange, why would we want that?) $IPTABLES -A POSTROUTING -t nat -s 172.28.10.0/24 -d ! 172.28.10.0/24 -j SNAT --to-source 172.28.10.254 and similar for 172.28.10, which had no role yet, but seem to be valid IPs for output, and we SNAT them all to 172.28.10.254 ... so this setup assumes that both 192.168.3.2 and 172.28.10.254 can reach the outside (whatever that might mean) and that there are either two routes or the router can handle both IPs ... On the server where i use it everything worked like a charm! Until...i had to add support in the kernel for another NIC. [EMAIL PROTECTED] ~# lspci 00:00.0 Host bridge: Intel Corp.: Unknown device 2570 (rev 02) 00:01.0 PCI bridge: Intel Corp.: Unknown device 2571 (rev 02) 00:1d.0 USB Controller: Intel Corp.: Unknown device 24d2 (rev 02) 00:1d.1 USB Controller: Intel Corp.: Unknown device 24d4 (rev 02) 00:1d.2 USB Controller: Intel Corp.: Unknown device 24d7 (rev 02) 00:1d.3 USB Controller: Intel Corp.: Unknown device 24de (rev 02) 00:1d.7 USB Controller: Intel Corp.: Unknown device 24dd (rev 02) 00:1e.0 PCI bridge: Intel Corp. 82801BA/CA/DB PCI Bridge (rev c2) 00:1f.0 ISA bridge: Intel Corp.: Unknown device 24d0 (rev 02) 00:1f.1 IDE interface: Intel Corp.: Unknown device 24db (rev 02) 00:1f.3 SMBus: Intel Corp.: Unknown device 24d3 (rev 02) 00:1f.5 Multimedia audio controller: Intel Corp.: Unknown device 24d5 (rev 02) 01:00.0 VGA compatible controller: nVidia Corporation RIVA TNT2 Model 64 (rev 15) 02:05.0 Ethernet controller: 3Com Corporation: Unknown device 1700 (rev 12) 02:0a.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL-8139/8139C/8139C+ (rev 10) 02:0b.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL-8139/8139C/8139C+ (rev 10) 02:0d.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL-8139/8139C/8139C+ (rev 10) hmm, four different network controllers easily confuse the unpracticed eye and often also the admin attached to it :) [EMAIL PROTECTED] ~# Since the 3com (gigabit builtin) ethernet device is unknown, i added support to it and recompiled the kernel. After rebooting the machine, i couldnt access any services on 192.168.3.81 (vserver called ciisp) from the outside). hmm, lets see .. *turns on the seeing orb* ... ah, looks like you 3com card got detected _before_ the other three realtek ones, so it was named eth0, instead of eth4 ... hmmm, ... and that probably messed up all other NICs, as they are now eth1 instead of eth0, eth2 instead of eth1 ... now, most likely some of your guests have the interface coded (its a little blurry now) and other just an ip I disabled support for that NIC again, recompiled and rebootedand everything went back to normal again! Can anybody help me with this? Is this normal behaviour? I guess yes, it is the typical linux networking behaviour so nothing critical ... I also dont understand why some vservers need for me to -j SNAT --to root-server and others dont! this also escapes my imagination (basically because of lack of information) but I assume that some have real IPs and/or communicate on private IPs where others have to use the host IP for outgoing traffic ... best, Herbert There are 4 NICs on the root server called leonardo-root. eth0 - 192.168.3.2 [connecting to our internal network /
