Re: [LARTC] Re: Routing packets over multiple links (NICS) all on the same ISP all with same gateway.
On 8/17/06, Luciano Ruete [EMAIL PROTECTED] wrote: I will try all this tommorow, don't have acces to the box right now. About turning of arp. If you turn them all of, wouldnt the nics (ip's) be unfindable from the outside world? Or does the switch they connect to respond to such an arp request aswell?what you're turning off is that the device answers arp(whohas) requests forips that are not from that specific device itself. The ips assigned to the device will cotinue answer as normal. But what you are saying is that if i had a box with 2 nics each connected to a different ISP, so each nic with a different gateway and ip. That if nic1 would recieve a arp request for the ip from nic2, it would respond with mac-adres from nic1.exactly! So that could mean that all packets would come in over nic1, even if they have destination ip(nic2)?exactly, and will arrive destiny anyway, but they are incoming for the wrong iface. Seems to me that this is never realy desirable.It is a really cuestionable default, but for shure that there are reasons forthis(maybe a thread in lkml archives could answer this cuestion), reasons that i do not know. Especialy if nic2 would have an static ip and i would unplug nic2 on purpose.well thats the other scenario where i face the same problem, first guess isobvious MAC cache, and you tend to think that is fucked up, and no, it is this crossed arp answer.--Luciano Hi Luciano and Jarek, Thanks for all your help so far. Ive had some try on the box this weekend. Unfortunatly its not in the student complex anymore, but it will be very soon. So for now ive simulated the situation by connecting 2 nics to a switch and that switch to a router. This is the script i used so far, no NAT rules yet because i have no extra computer to connect to the box. Its Jareks script completed with some hints from Luciano ip route add default nexthop via x.x.x.x dev eth1 onlink\ nexthop via x.x.x.x dev eth2 onlink ip route add 192.168.0.0/24 dev eth0 table 101 ip route add default via x.x.x.x dev eth1 table 101 ip route add 192.168.0.0/24 dev eth0 table 102 ip route add default via x.x.x.x dev eth2 table 102 ip rule add fwmark 1 table 101 ip rule add fwmark 2 table 102 iptables -t mangle -A POSTROUTING -o eth1 -j MARK --set-mark 1 iptables -t mangle -A POSTROUTING -o eth2 -j MARK --set-mark 2 iptables -t mangle -A POSTROUTING -j CONNMARK --save-mark iptables -t mangle -A PREROUTING -j CONNMARK --restore-mark echo 0 /proc/sys/net/ipv4/conf/all/rp_filterecho1 /proc/sys/net/ipv4/conf/all/arp_ignore I am thinking if it wouldn't be better to include a src = "" to the route add default entries. ofcourse this would go wrong if i would recive a new ip from the isp on one of the nics. So if its not needed then i would rather leave it out. ive tested the script with ip route get from IPE1 to 204.152.189.113ip route get from IPE2 to 204.152.189.113 it does indeed answer wit eth1 for the first and eth2 for the seccond. It didnt do this with my other attempts. Im not sure if this is a real vallid tes though. Regards, Jacques ___ LARTC mailing list LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc
[LARTC] Re: Routing packets over multiple links (NICS) all on the same ISP all with same gateway.
On 16-08-2006 16:18, Jacques Rompen wrote: On 8/16/06, *Jarek Poplawski* [EMAIL PROTECTED] mailto:[EMAIL PROTECTED] wrote: As far as I know multipath cached should work for routing output of a current box but it's not working for router (if something has not changed lately). Multipath without cached should do simple randomization, but maybe for the begining it would be better to assign fixed paths by source ip and save experimenting with multipaths for later. Jarek P. Hi Jarek, I will try with only multipath. Ive tried the static routes. But still ive had trouble getting this code to work ip ro add default nexthop via x.x.x.x dev eth1 weight 1 nexthop via y.y.y.y dev eth2 ip route add default table provider1 via x.x.x.x dev eth1 ip route add default table provider2 via y.y.y.y dev eth2 because x.x.x.x = y.y.y.y in my case. Somehow it doesnt look at the eth device anymore. Anyway in my opinion it should work. Try something like this: #IP_ROUTE_MULTIPATH = y #(if 2.6.x IP_ROUTE_MULTIPATH_CACHED = n !) #Here with CONNMARK method (so without Anastasov's patch!). #Change 192.168.0.0/24 and eth0 to your local network #address/mask and interface. ip route add default nexthop via x.x.x.x dev eth1 \ nexthop via x.x.x.x dev eth2 ip route add 192.168.0.0/24 dev eth0 table 101 ip route add default via x.x.x.x dev eth1 table 101 ip route add 192.168.0.0/24 dev eth0 table 102 ip route add default via x.x.x.x dev eth2 table 102 ip rule add fwmark 1 table 101 ip rule add fwmark 2 table 102 iptables -t mangle -A POSTROUTING -o eth1 -j MARK --set-mark 1 iptables -t mangle -A POSTROUTING -o eth2 -j MARK --set-mark 2 iptables -t mangle -A POSTROUTING -j CONNMARK --save-mark iptables -t mangle -A PREROUTING -j CONNMARK --restore-mark echo 0 /proc/sys/net/ipv4/conf/all/rp_filter You should also add NAT (MASQERADE) rules. Nexthop devices will be changed randomly per flow (with some caching) so try with destinations from different networks. Jarek P. ___ LARTC mailing list LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc
Re: [LARTC] Re: Routing packets over multiple links (NICS) all on the same ISP all with same gateway.
On Thursday 17 August 2006 03:54, Jarek Poplawski wrote: On 16-08-2006 16:18, Jacques Rompen wrote: On 8/16/06, *Jarek Poplawski* [EMAIL PROTECTED] mailto:[EMAIL PROTECTED] wrote: As far as I know multipath cached should work for routing output of a current box but it's not working for router (if something has not changed lately). Multipath without cached should do simple randomization, but maybe for the begining it would be better to assign fixed paths by source ip and save experimenting with multipaths for later. Jarek P. Hi Jarek, I will try with only multipath. Ive tried the static routes. But still ive had trouble getting this code to work ip ro add default nexthop via x.x.x.x dev eth1 weight 1 nexthop via y.y.y.y dev eth2 ip route add default table provider1 via x.x.x.x dev eth1 ip route add default table provider2 via y.y.y.y dev eth2 because x.x.x.x = y.y.y.y in my case. Somehow it doesnt look at the eth device anymore. Anyway in my opinion it should work. Try something like this: #IP_ROUTE_MULTIPATH = y #(if 2.6.x IP_ROUTE_MULTIPATH_CACHED = n !) #Here with CONNMARK method (so without Anastasov's patch!). #Change 192.168.0.0/24 and eth0 to your local network #address/mask and interface. ip route add default nexthop via x.x.x.x dev eth1 \ nexthop via x.x.x.x dev eth2 ip route add 192.168.0.0/24 dev eth0 table 101 ip route add default via x.x.x.x dev eth1 table 101 ip route add 192.168.0.0/24 dev eth0 table 102 ip route add default via x.x.x.x dev eth2 table 102 ip rule add fwmark 1 table 101 ip rule add fwmark 2 table 102 iptables -t mangle -A POSTROUTING -o eth1 -j MARK --set-mark 1 iptables -t mangle -A POSTROUTING -o eth2 -j MARK --set-mark 2 iptables -t mangle -A POSTROUTING -j CONNMARK --save-mark iptables -t mangle -A PREROUTING -j CONNMARK --restore-mark echo 0 /proc/sys/net/ipv4/conf/all/rp_filter if all ethx are on the same switch, you will need: echo 1 /proc/sys/net/ipv4/conf/all/arp_ignore #you can also do it in a per iface basis by default, if you recive an arp request(whohas) on a iface(iface1), asking for an ip that another iface(iface2) has, then the arp request is ansered with the MAC address of the first iface(iface1). If all devices are in the same fisical segment this can lead to a single device answer for all IPs, or to a total caos. This one is very hard to catch and i forgot to mention, now solved :-) -- Luciano ___ LARTC mailing list LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc
Re: [LARTC] Re: Routing packets over multiple links (NICS) all on the same ISP all with same gateway.
On Thursday 17 August 2006 14:48, Jacques Rompen wrote: On 8/17/06, Luciano Ruete [EMAIL PROTECTED] wrote: if all ethx are on the same switch, you will need: echo 1 /proc/sys/net/ipv4/conf/all/arp_ignore #you can also do it in a per iface basis by default, if you recive an arp request(whohas) on a iface(iface1), asking for an ip that another iface(iface2) has, then the arp request is ansered with the MAC address of the first iface(iface1). If all devices are in the same fisical segment this can lead to a single device answer for all IPs, or to a total caos. This one is very hard to catch and i forgot to mention, now solved :-) -- Luciano I will try all this tommorow, don't have acces to the box right now. About turning of arp. If you turn them all of, wouldnt the nics (ip's) be unfindable from the outside world? Or does the switch they connect to respond to such an arp request aswell? what you're turning off is that the device answers arp(whohas) requests for ips that are not from that specific device itself. The ips assigned to the device will cotinue answer as normal. But what you are saying is that if i had a box with 2 nics each connected to a different ISP, so each nic with a different gateway and ip. That if nic1 would recieve a arp request for the ip from nic2, it would respond with mac-adres from nic1. exactly! So that could mean that all packets would come in over nic1, even if they have destination ip(nic2)? exactly, and will arrive destiny anyway, but they are incoming for the wrong iface. Seems to me that this is never realy desirable. It is a really cuestionable default, but for shure that there are reasons for this(maybe a thread in lkml archives could answer this cuestion), reasons that i do not know. Especialy if nic2 would have an static ip and i would unplug nic2 on purpose. well thats the other scenario where i face the same problem, first guess is obvious MAC cache, and you tend to think that is fucked up, and no, it is this crossed arp answer. -- Luciano ___ LARTC mailing list LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc
[LARTC] Re: Routing packets over multiple links (NICS) all on the same ISP all with same gateway.
On 14-08-2006 17:42, Jacques Rompen wrote: ... What we want to do is the following: I live in a student complex with 7 other people. Every room has its own internet connection from the same ISP. Ip, gateway, subnet are asigned through dhcp on mac-adres basis. Every internet connection is capped at 20mbit up/down. We want to get all computers on an internal network. So we need some sort of router that accepts all 8 internet connections and routes it out over a gbit nic - switch - internal network So we have a computer with 10 nics and hopefully enough internal bandwith. Maybe you have some other reasons to make one gateway but to have internal network you only need to set a second address from local range (not colliding with that used by ISP) on each box. But if you really need this together and with multipath, setting CONFIG_IP_ROUTE_MULTIPATH = y and CONFIG_IP_ROUTE_MULTIPATH_CACHED = n (!) should change multipath randomly. Jarek P. ___ LARTC mailing list LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc
Re: [LARTC] Re: Routing packets over multiple links (NICS) all on the same ISP all with same gateway.
On 8/16/06, Jarek Poplawski [EMAIL PROTECTED] wrote: On 14-08-2006 17:42, Jacques Rompen wrote:... What we want to do is the following: I live in a student complex with 7 other people. Every room has its own internet connection from the same ISP. Ip, gateway, subnet are asigned through dhcp on mac-adres basis. Every internet connection is capped at 20mbit up/down. We want to get all computers on an internal network. So we need some sort of router that accepts all 8 internet connections and routes it out over a gbit nic - switch - internal network So we have a computer with 10 nics and hopefully enough internal bandwith. Maybe you have some other reasons to make one gateway but to haveinternal network you only need to set a second address from localrange (not colliding with that used by ISP) on each box.But if you really need this together and with multipath, setting CONFIG_IP_ROUTE_MULTIPATH = y andCONFIG_IP_ROUTE_MULTIPATH_CACHED = n (!) should changemultipath randomly.Jarek P.___ LARTC mailing listLARTC@mailman.ds9a.nlhttp://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc The 2 ips on every box wouldnt work because: Most people here are complete network newbies, we want to connect more then 8 computers (most people have a laptop and pc), we would only have a 20mbit internal network. Im a bit confused. In the 2.4 kernel there issent even an option ip_route_multipath_cached, only ip_route_multipath. Or am i mistaking here? If there issent, then I couldnt have put it to 'Y' so why didnt the random function work? In the 2.6 kernel there is a option ip_route_multipath and ip_route_multipath_cached. But if i dontsay 'Y' to ip_route_multipatch_cached i cant say yes to round_robin, random_match etc. I thought i needed that option random_match set to 'y'. (im not sure if its exactly spelled like that, but i'm talking about the four or five sub options to ip_route_multipath_cached) ___ LARTC mailing list LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc
[LARTC] Re: Routing packets over multiple links (NICS) all on the same ISP all with same gateway.
On 16-08-2006 14:50, Jacques Rompen wrote: ... The 2 ips on every box wouldnt work because: Most people here are complete network newbies, we want to connect more then 8 computers (most people have a laptop and pc), we would only have a 20mbit internal network. Im a bit confused. In the 2.4 kernel there issent even an option ip_route_multipath_cached, only ip_route_multipath. Or am i mistaking here? If there issent, then I couldnt have put it to 'Y' so why didnt the random function work? In the 2.6 kernel there is a option ip_route_multipath and ip_route_multipath_cached. But if i dont say 'Y' to ip_route_multipatch_cached i cant say yes to round_robin, random_match etc. I thought i needed that option random_match set to 'y'. (im not sure if its exactly spelled like that, but i'm talking about the four or five sub options to ip_route_multipath_cached) As far as I know multipath cached should work for routing output of a current box but it's not working for router (if something has not changed lately). Multipath without cached should do simple randomization, but maybe for the begining it would be better to assign fixed paths by source ip and save experimenting with multipaths for later. Jarek P. ___ LARTC mailing list LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc
Re: [LARTC] Re: Routing packets over multiple links (NICS) all on the same ISP all with same gateway.
On 8/16/06, Jarek Poplawski [EMAIL PROTECTED] wrote: As far as I know multipath cached should work for routing outputof a current box but it's not working for router (if something has not changed lately). Multipath without cached should dosimple randomization, but maybe for the begining it would bebetter to assign fixed paths by source ip and save experimentingwith multipaths for later. Jarek P. Hi Jarek, I will try with only multipath. Ive tried the static routes. But still ive had trouble getting this code to work ip ro add default nexthop via x.x.x.x dev eth1 weight 1 nexthop viay.y.y.y dev eth2ip route add default table provider1 via x.x.x.x dev eth1ip route add default table provider2 via y.y.y.y dev eth2 because x.x.x.x = y.y.y.y in my case. Somehow it doesnt look at the eth device anymore. ___ LARTC mailing list LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc
