Re: [LARTC] Multipath routing
Piotr Chytla pisze: First of all equal cost multipathing is evil ;>, It simply doesn't work for packets in forwarding path besides support in kernel is not maintained Realy if you want load balance both uplinks disable CONFIG_IP_ROUTE_MULTIPATH_CACHED and you will have random traffic distribiution between both links. Unfortunately it still doesn't work as expected. When i ping some host it always go trough one nexthop. It does per-destination loadbalancing, I am afraid. -- Michał Margula, [EMAIL PROTECTED], http://alchemyx.uznam.net.pl/ "W życiu piękne są tylko chwile" [Ryszard Riedel] ___ LARTC mailing list LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc
Re: [LARTC] Multipath routing
Piotr Chytla pisze: First of all equal cost multipathing is evil ;>, It simply doesn't work for packets in forwarding path besides support in kernel is not maintained Realy if you want load balance both uplinks disable CONFIG_IP_ROUTE_MULTIPATH_CACHED and you will have random traffic distribiution between both links. More details : http://lists.openwall.net/netdev/2007/03/14/50 http://lists.openwall.net/netdev/2007/03/12/76 http://lists.quagga.net/pipermail/quagga-users/2007-May/008469.html Oh. I see. Thanks. BTW: google doesn't show that links when looking for multipath on linux :-) Random load sharing over multiple routes is not a good idea, or maybe is it? Am I guessing right that with enough amount of traffic and having two nexthops it will split 50%/50% ? BGP always have alternative paths in BGP RIB and mostly don't insert them as multipath route to FIB. Of course there is path : http://lebon.org.ua/quagga.html that force route to be inserted to kernel with multiple gateways - but realy this is some kind of dirty-hack. I know that site, but I thought that those patches were obsoleted, because of --multipath option when compiling quagga. Check thread 'Linux and BGP multipath' on quagga-dev, and especially this mail: http://lists.quagga.net/pipermail/quagga-dev/2007-April/004700.html I know that also :). BTW: until now I was quite pleased with linux networking which quality is amazing. But now, when I need loadbalancing I am disapointed, because it doesn't support things that with cisco hardware you take for granted. I miss mostly recursive routes. Something like that ip route add 80.245.177.4/32 via 80.245.176.11 ip route add 10.0.0.0/24 via 80.245.177.4 It would solve problems with multipath bgp and loadbalancing because I could add remove additional routes to 80.245.177.4 (or some other imaginary loopback) and it would work as expected. I hope it will be added some day :) Thank you for your help! -- Michał Margula, [EMAIL PROTECTED], http://alchemyx.uznam.net.pl/ "W życiu piękne są tylko chwile" [Ryszard Riedel] ___ LARTC mailing list LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc
Re: [LARTC] Multipath routing
On Tue, Jun 05, 2007 at 11:13:52AM +0200, Michał Margula wrote: > Hello! > Hi > I have trouble with multipath routing. Those options are enabled in > kernel: > > [*] IP: policy routing > [*] IP: equal cost multipath > [*] IP: equal cost multipath with caching support (EXPERIMENTAL) > <*> MULTIPATH: round robin algorithm > First of all equal cost multipathing is evil ;>, It simply doesn't work for packets in forwarding path besides support in kernel is not maintained Realy if you want load balance both uplinks disable CONFIG_IP_ROUTE_MULTIPATH_CACHED and you will have random traffic distribiution between both links. More details : http://lists.openwall.net/netdev/2007/03/14/50 http://lists.openwall.net/netdev/2007/03/12/76 http://lists.quagga.net/pipermail/quagga-users/2007-May/008469.html > Also I have trouble using multipath quagga, it simply doesn't put > multipath route in routing table. > > For example: > > faramir# sh ip bgp 10.100.0.1 > BGP routing table entry for 10.101.0.0/22 > Paths: (2 available, best #1, table Default-IP-Routing-Table) > Not advertised to any peer > Local > 80.245.176.13 (metric 1) from 80.245.176.13 (80.245.177.4) > Origin IGP, metric 0, localpref 100, weight 150, valid, internal, > best > Last update: Tue Jun 5 01:59:29 2007 > > Local > 80.245.176.10 (metric 1) from 80.245.176.10 (80.245.176.10) > Origin IGP, metric 0, localpref 100, weight 100, valid, internal > Last update: Tue Jun 5 01:28:02 2007 > BGP always have alternative paths in BGP RIB and mostly don't insert them as multipath route to FIB. Of course there is path : http://lebon.org.ua/quagga.html that force route to be inserted to kernel with multiple gateways - but realy this is some kind of dirty-hack. Check thread 'Linux and BGP multipath' on quagga-dev, and especially this mail: http://lists.quagga.net/pipermail/quagga-dev/2007-April/004700.html > # ip r s > [...] > 10.100.0.0/22 via 80.245.176.11 dev eth0 proto zebra > > But if I manually put something like that in quagga: > > faramir(config)# ip route 1.2.3.0/24 80.245.176.13 > faramir(config)# ip route 1.2.3.0/24 80.245.176.11 > yeah this is static route. /pch -- Dyslexia bug unpatched since 1977 ... exploit has been leaked to the underground. ___ LARTC mailing list LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc
Re: [LARTC] Multipath Routing Problems
On 26/06/06, Armin ranjbar <[EMAIL PROTECTED]> wrote: On Sun, 25 Jun 2006 21:46:06 +0300 "Vladimir Vitkov" <[EMAIL PROTECTED]> wrote: > Remove multipath caching and try again if i remove CONFIG_IP_ROUTE_MULTIPATH_CACHED i will be unable to use : CONFIG_IP_ROUTE_MULTIPATH_RR CONFIG_IP_ROUTE_MULTIPATH_RANDOM CONFIG_IP_ROUTE_MULTIPATH_WRANDOM CONFIG_IP_ROUTE_MULTIPATH_DRR right ? yes correct you will get only random multipath. It works perfectly well -- You will always get the greatest recognition for the job you least like. ___ LARTC mailing list LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc -- С уважение, Владимир Витков http://www.netsecad.com http://www.supportbg.com ___ LARTC mailing list LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc
Re: [LARTC] Multipath Routing Problems
On Monday 26 June 2006 08:28, Armin ranjbar wrote: > On Sun, 25 Jun 2006 21:46:06 +0300 > > "Vladimir Vitkov" <[EMAIL PROTECTED]> wrote: > > Remove multipath caching and try again > > if i remove > CONFIG_IP_ROUTE_MULTIPATH_CACHED > > i will be unable to use : > CONFIG_IP_ROUTE_MULTIPATH_RR > CONFIG_IP_ROUTE_MULTIPATH_RANDOM > CONFIG_IP_ROUTE_MULTIPATH_WRANDOM > CONFIG_IP_ROUTE_MULTIPATH_DRR > > right ? yes, this modules are also EXPERIMENTAL code, and multipath work perfectly without them. -- Luciano ___ LARTC mailing list LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc
Re: [LARTC] Multipath Routing Problems
On Sun, 25 Jun 2006 21:46:06 +0300 "Vladimir Vitkov" <[EMAIL PROTECTED]> wrote: > Remove multipath caching and try again if i remove CONFIG_IP_ROUTE_MULTIPATH_CACHED i will be unable to use : CONFIG_IP_ROUTE_MULTIPATH_RR CONFIG_IP_ROUTE_MULTIPATH_RANDOM CONFIG_IP_ROUTE_MULTIPATH_WRANDOM CONFIG_IP_ROUTE_MULTIPATH_DRR right ? -- You will always get the greatest recognition for the job you least like. ___ LARTC mailing list LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc
Re: [LARTC] Multipath Routing Problems
CONFIG_IP_ROUTE_MULTIPATH=y CONFIG_IP_ROUTE_MULTIPATH_CACHED=y CONFIG_IP_ROUTE_MULTIPATH_RR=y CONFIG_IP_ROUTE_MULTIPATH_RANDOM=y CONFIG_IP_ROUTE_MULTIPATH_WRANDOM=y CONFIG_IP_ROUTE_MULTIPATH_DRR=y and all of this algorithms works perfectly over icmp , but there is some (maybe) problems with tcp round robin algorithm : with : ip route add default mpath rr nexthop via 1.1.1.2 dev eth0 nexthop via 1.1.1.3 dev eth0 wget i attempt to wget a file over tcp port 80 , all packets goes to 1.1.1.2 , nothing on 1.1.1.3 . but with : ip route add default mpath random nexthop via 1.1.1.2 dev eth0 nexthop via 1.1.1.3 dev eth0 i've got something on 1.1.1.3 and everything goes as expected , any idea ? we have to report bug ? There are some problems with the balancing code in newer kernels. As stated few days earlier removing multipath caching does help prety good. Also a side remark with one client (load) you will see almost no effect. When you load the link wih lots of connections then the balancing works. Remove multipath caching and try again If your ISP's can offer you BGP (even read only/listen) resort to it. It will do much better. -- С уважение, Владимир Витков http://www.netsecad.com http://www.supportbg.com ___ LARTC mailing list LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc
Re: [LARTC] Multipath Routing Problems
On Fri, 23 Jun 2006 20:04:53 -0300 Luciano Ruete <[EMAIL PROTECTED]> wrote: > > Equalize was available as a patch in 2.4 kernels and AFAIK there are not 2.6 > patches, so to set that flag do nothing in most cases. > its quite strange , its somesort of working under 2.6.16 , > I think you have CONFIG_IP_ROUTE_MULTIPATH_CACHED set in your kernel(which is > EXPERIMENTAL and mostly undocumented) and you're not selecting any of the 4 > multipath cache algos. Try adding 'mpath' option to your 'tc' command: > > ip route add default mpath rr nexthop via 1.1.1.1 dev eth0 nexthop via > 1.1.1.2 > dev eth1 > > If that do not work, recompile your kernel without > CONFIG_IP_ROUTE_MULTIPATH_CACHED multipath will probably work at first try. > > Plz don't forget to post your results to the list. my kernel compiled with : CONFIG_IP_ROUTE_MULTIPATH=y CONFIG_IP_ROUTE_MULTIPATH_CACHED=y CONFIG_IP_ROUTE_MULTIPATH_RR=y CONFIG_IP_ROUTE_MULTIPATH_RANDOM=y CONFIG_IP_ROUTE_MULTIPATH_WRANDOM=y CONFIG_IP_ROUTE_MULTIPATH_DRR=y and all of this algorithms works perfectly over icmp , but there is some (maybe) problems with tcp round robin algorithm : with : ip route add default mpath rr nexthop via 1.1.1.2 dev eth0 nexthop via 1.1.1.3 dev eth0 wget i attempt to wget a file over tcp port 80 , all packets goes to 1.1.1.2 , nothing on 1.1.1.3 . but with : ip route add default mpath random nexthop via 1.1.1.2 dev eth0 nexthop via 1.1.1.3 dev eth0 i've got something on 1.1.1.3 and everything goes as expected , any idea ? we have to report bug ? > > -- > Luciano thank you -- You will be aided greatly by a person whom you thought to be unimportant. ___ LARTC mailing list LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc
Re: [LARTC] Multipath Routing Problems
El Friday 23 June 2006 06:59, Armin ranjbar escribió: > Hi all :) > > there is somekind of strange Routing problem that im getting with > Linux-2.6.16 and iproute 2.6.16 , when i use command like : > > ip route add default nexthop via 1.1.1.1 dev eth0 nexthop via 1.1.1.2 dev > eth1 > > all packets goes on 1.1.1.2 ( always last interface ) , whats is the > problem ? this situation also tested with equalize flag , on two physical > interface and etc ... Equalize was available as a patch in 2.4 kernels and AFAIK there are not 2.6 patches, so to set that flag do nothing in most cases. I think you have CONFIG_IP_ROUTE_MULTIPATH_CACHED set in your kernel(which is EXPERIMENTAL and mostly undocumented) and you're not selecting any of the 4 multipath cache algos. Try adding 'mpath' option to your 'tc' command: ip route add default mpath rr nexthop via 1.1.1.1 dev eth0 nexthop via 1.1.1.2 dev eth1 If that do not work, recompile your kernel without CONFIG_IP_ROUTE_MULTIPATH_CACHED multipath will probably work at first try. Plz don't forget to post your results to the list. -- Luciano ___ LARTC mailing list LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc
RE: [LARTC] multipath routing
Title: Re: [LARTC] multipath routing Hi, using the following: ip route add equalize 10.200.1.0/24 nexthop via 10.200.0.2 dev neta nexthop> via 10.200.0.2 dev neta2 while doing a -> while [ 1 ] do ip route flush cache done the transfer of packets almost seems equal? thx jason From: [EMAIL PROTECTED] on behalf of comp.techsSent: Thu 10/27/2005 10:02 AMTo: Edmundo Carmona; lartc@mailman.ds9a.nlSubject: RE: [LARTC] multipath routing Hi, I also used TEQL this worked very well, but I require the (weight) option. thx jason From: [EMAIL PROTECTED] on behalf of Edmundo CarmonaSent: Thu 10/27/2005 8:20 AMTo: lartcSubject: Re: [LARTC] multipath routing Multipath takes a little more that just setting the default route. Youhave to set separate routing tables for each interface involved in themultipath routing (though I haven't understood yet why they areneeded.. the fact is that if you don't set them, multipath won'troute).Also, even if you set it all right, it doesn't mean that if you sendtwo packets to a location X, one will go through one interface and thesecond will go through the other. Routes are cached, and after arouting decision has been made for the first packet, packets going tothat same host will go through the same interface till the cachingtime has gone by.On 10/26/05, comp.techs <[EMAIL PROTECTED]> wrote:> Hi, I am tring to us ip route to load balance between two interfaces.>>>> ip route add equalize 10.200.1.0/24 nexthop via 10.200.0.2 dev neta nexthop> via 10.200.0.2 dev neta2>> Where neta and neta2 are gre tunnels. Testing show that packets travel in> a single sided manner.>> Do I need to use the multipath (IP_ROUTE_MULTIPATH_CACHED) module?>> thx jason> ___> LARTC mailing list> LARTC@mailman.ds9a.nl> http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc>>>___LARTC mailing listLARTC@mailman.ds9a.nlhttp://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc ___ LARTC mailing list LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc
RE: [LARTC] multipath routing
Title: Re: [LARTC] multipath routing Hi, I also used TEQL this worked very well, but I require the (weight) option. thx jason From: [EMAIL PROTECTED] on behalf of Edmundo CarmonaSent: Thu 10/27/2005 8:20 AMTo: lartcSubject: Re: [LARTC] multipath routing Multipath takes a little more that just setting the default route. Youhave to set separate routing tables for each interface involved in themultipath routing (though I haven't understood yet why they areneeded.. the fact is that if you don't set them, multipath won'troute).Also, even if you set it all right, it doesn't mean that if you sendtwo packets to a location X, one will go through one interface and thesecond will go through the other. Routes are cached, and after arouting decision has been made for the first packet, packets going tothat same host will go through the same interface till the cachingtime has gone by.On 10/26/05, comp.techs <[EMAIL PROTECTED]> wrote:> Hi, I am tring to us ip route to load balance between two interfaces.>>>> ip route add equalize 10.200.1.0/24 nexthop via 10.200.0.2 dev neta nexthop> via 10.200.0.2 dev neta2>> Where neta and neta2 are gre tunnels. Testing show that packets travel in> a single sided manner.>> Do I need to use the multipath (IP_ROUTE_MULTIPATH_CACHED) module?>> thx jason> ___> LARTC mailing list> LARTC@mailman.ds9a.nl> http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc>>>___LARTC mailing listLARTC@mailman.ds9a.nlhttp://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc ___ LARTC mailing list LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc
Re: [LARTC] multipath routing
Multipath takes a little more that just setting the default route. You have to set separate routing tables for each interface involved in the multipath routing (though I haven't understood yet why they are needed.. the fact is that if you don't set them, multipath won't route). Also, even if you set it all right, it doesn't mean that if you send two packets to a location X, one will go through one interface and the second will go through the other. Routes are cached, and after a routing decision has been made for the first packet, packets going to that same host will go through the same interface till the caching time has gone by. On 10/26/05, comp.techs <[EMAIL PROTECTED]> wrote: > Hi, I am tring to us ip route to load balance between two interfaces. > > > > ip route add equalize 10.200.1.0/24 nexthop via 10.200.0.2 dev neta nexthop > via 10.200.0.2 dev neta2 > > Where neta and neta2 are gre tunnels. Testing show that packets travel in > a single sided manner. > > Do I need to use the multipath (IP_ROUTE_MULTIPATH_CACHED) module? > > thx jason > ___ > LARTC mailing list > LARTC@mailman.ds9a.nl > http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc > > > ___ LARTC mailing list LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc
Re: [LARTC] Multipath Routing..
On Tue, Aug 16, 2005 at 06:11:26AM +0200, Daniel Frederiksen wrote: > Ok folks, here goes.. > > I have been boggling with a problem for the past week, and still haven't > found a solution.. > > I'm trying to route traffic from two providers through a Linux machine. > But that is not the problem. The ISP's have provided me with a WAN IP > class for both of the lines, to be routed into a DMZ where the machines > a to respond to their respective designated WAN IP on both lines. > Every machine on the DMZ has two IP's one on each ISP WAN Class. > > I think I'll better draw a map: > > > WAN1(eth2), WAN2(eth3) >- (eth0) >| |-\ -- >| DMZ |---\ \/---| ISP1 |- >-\ \ /-- \ > \ \/ \ > - > | Linux GW/FW | WWW > - > / \ (eth1)/ >-/ \-- / >| LAN |---/ \---| ISP2 |- >- -- > NAT(eth4) > > > The DMZ has two WAN IP classes routed from the ISP. > > The thing I just can not figure out is how to make the respective WAN IP > from the DMZ route out the right ISP link, and the right request from > the ISP route into the DMZ. > > .. and finally how can I make the LAN able to access it all.. you need to use ip ru my ip ru looks like 0: from all lookup local 200:from 141.168.16.16 lookup cable 201:from 220.233.15.63 lookup adsl 32766: from all lookup main 32767: from all lookup default I created 200 and 201 which means that all traffic that came in on the cable 141.168.16.16 will go out the cable ip ro sh tab cable 192.168.11.0/24 dev br0 scope link 192.168.10.0/24 dev eth3 scope link 192.168.9.0/24 dev eth4 scope link default via 141.168.16.1 dev eth0 src 141.168.16.16 metric 30 and the routing tab for the adsl uses the adsl as its default gw. does that help ? > > Thanks for your time.. > > /Daniel Frederiksen > > ___ > LARTC mailing list > LARTC@mailman.ds9a.nl > http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc > signature.asc Description: Digital signature ___ LARTC mailing list LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc
Re: [LARTC] Multipath Routing in same subnet - Please take a look
Christian Schmid wrote: nexthop via 80.237.244.1 dev eth1 weight 100 nexthop via 80.237.244.33 dev eth1 weight 100 I have read postings on the net but all of them are using huge scripts because they are on different networks. My problem seems to be a much easier problem but I just cant get this to work. :( Saw this on netdev, solution disable IP_ROUTE_MULTIPATH_CACHED http://news.gmane.org/find-root.php?group=gmane.linux.network&article=25774 Andy. ___ LARTC mailing list LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc
Re: [LARTC] Multipath Routing in same subnet - Please take a look
On Fri, May 06, 2005 at 08:41:55PM -0700, gypsy wrote: > Christian Schmid wrote: > > > > Hello. > > > 80.237.244.0/26 dev eth1 proto kernel scope link src 80.237.244.52 > > default > > nexthop via 80.237.244.1 dev eth1 weight 100 > > nexthop via 80.237.244.33 dev eth1 weight 100 > > Do not use weight parameters exceeding a single digit! > > > I have read postings on the net but all of them are using huge scripts > > because they are on different > > networks. My problem seems to be a much easier problem but I just cant get > > this to work. :( > > > > Please help. > > > > Best regards, > > Chris > > I'm no expert, but my suggestion is to use 2 NICs and connect one to > each uplink or at least add 2 entries into /etc/iproute2/rt_tables. I > think you'll find a similar situation answered in the ML within the last > 10 days or so, but I can't recall the subject of the thread. Nor can I > find anything specifying exactly what rt_tables needs to contain :/ > > You can review what I've gleaned from this ML at > http://yesican.chsoft.biz/lartc > > The most urgent things for you to know: > 1) The LARTC HOWTO is wrong > 2) You must apply Julian's patch > 3) You _really_ need to read nano.txt > 4) All of the intelligible success stories WRT multipath are either on > yesican or are linked to from there. here is my setup (firewall with 2 default routes) eth0 = cable, ppp0 adsl. I changed the dhcpd client to add the default route for each in on a different metric, that way if the one of the lines is out the other default route will still work! from ip r (whith some stuff removed - but the essential stuff is here) default metric 5 nexthop via 141.168.16.1 dev eth0 weight 4 nexthop via 202.7.162.89 dev ppp0 weight 2 default via 141.168.16.1 dev eth0 metric 10 default via 202.7.162.89 dev ppp0 metric 20 from ip ru 0: from all lookup local 200:from 141.168.16.16 lookup cable 201:from 60.240.81.237 lookup adsl 32766: from all lookup main 32767: from all lookup default the important ones are 200, 201 these setup up the routing for each of the different legs - cause they will be different! so 141.168.16.16 is on eth0 and 60.240.81.237 is on ppp0 (they are the actual address on the interfaces) cat /etc/iproute2/rt_tables (these are where the names come from above!) # # reserved values # 255 local 254 main 253 default 0 unspec # # local # #1 inr.ruhep 200 cable 201 adsl These are the routing tables for 200 and 201, the 192.168 address are the local address. The routing engine workes from the lowest numbered ru (from ip ru ) and works to the larger numbered ones until it finds a rule that matches! # ip r sh tab 200 192.168.11.0/24 dev br0 scope link 192.168.10.0/24 dev eth3 scope link 192.168.9.0/24 dev eth4 scope link default via 141.168.16.1 dev eth0 default via 141.168.16.1 dev eth0 metric 10 # ip r sh tab 201 192.168.11.0/24 dev br0 scope link 192.168.10.0/24 dev eth3 scope link 192.168.9.0/24 dev eth4 scope link default via 202.7.162.89 dev ppp0 Hope that helps > > --gypsy > ___ > LARTC mailing list > LARTC@mailman.ds9a.nl > http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc > signature.asc Description: Digital signature ___ LARTC mailing list LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc
Re: [LARTC] Multipath Routing in same subnet - Please take a look
Christian Schmid wrote: > > Hello. > 80.237.244.0/26 dev eth1 proto kernel scope link src 80.237.244.52 > default > nexthop via 80.237.244.1 dev eth1 weight 100 > nexthop via 80.237.244.33 dev eth1 weight 100 Do not use weight parameters exceeding a single digit! > I have read postings on the net but all of them are using huge scripts > because they are on different > networks. My problem seems to be a much easier problem but I just cant get > this to work. :( > > Please help. > > Best regards, > Chris I'm no expert, but my suggestion is to use 2 NICs and connect one to each uplink or at least add 2 entries into /etc/iproute2/rt_tables. I think you'll find a similar situation answered in the ML within the last 10 days or so, but I can't recall the subject of the thread. Nor can I find anything specifying exactly what rt_tables needs to contain :/ You can review what I've gleaned from this ML at http://yesican.chsoft.biz/lartc The most urgent things for you to know: 1) The LARTC HOWTO is wrong 2) You must apply Julian's patch 3) You _really_ need to read nano.txt 4) All of the intelligible success stories WRT multipath are either on yesican or are linked to from there. --gypsy ___ LARTC mailing list LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc
RE : [LARTC] Multipath routing + traffic separation problem.
hello, first thx for your answer. i have also tried with others marks under 10 to avoid confusion between decimal and hexa => same result. i confirm that no default route are present in my main table, only routes for the LAN and DMZ networks. and the reason why i want the http(s) and ftp traffic not to be balanced is for "political reason", i install several firewall for differents client and each one have their proper wish ;) i really dont understand where the problem is. if i use the ip filter capabilities (from, to, iif), the traffic is correctly routed, but with the netfilter mark it dont works... i checked packets stats with iptables to see if traffic going through the mangle rules and it seems to be ok, and with the realms mark i check if the routing rule is read and it seems to be ok too... > -Message d'origine- > De : Nguyen Dinh Nam [mailto:[EMAIL PROTECTED] > Envoyà : jeudi 7 avril 2005 02:55 > à : Laurent LAVAUD > Cc : lartc@mailman.ds9a.nl > Objet : Re: [LARTC] Multipath routing + traffic separation problem. > > > Your settings seem to be correct, I just don't know why you don't want to > balance http, https and ftp > traffic between both connections? > > About the bug, I haven't used linux 2.4 for a long time, for 2.6, fwmark is > in hexa, so be careful with 10 vs. 0xa, you'd better use values less than 0xa > to avoid confusing. > > Also make sure that no default route is added to your main table. > > >> On Wed, 2005-04-06 at 12:09 +0200, Laurent LAVAUD wrote: >> Hello, >> >> I have set up a multipath gateway. >> System is a linux 2.4.29 kernel, iproute 20010824, iptables 1.2.11. >> >> here is the setup: >> >> >> firewall:/# ip rule >> 0: from all lookup local >> 100:from all lookup main >> 152:from all fwmark 10 lookup wan1 >> 153:from all fwmark 20 lookup wan2 >> 201:from 213.223.96.121 lookup wan1 >> 202:from 82.236.230.217 lookup wan2 >> 1000: from all lookup away >> >> Fw-cgarp:/etc/firegate# ip route ls table wan1 >> default via 213.223.96.122 dev eth0 src 213.223.96.121 >> prohibit default metric 1 >> >> Fw-cgarp:/etc/firegate# ip route ls table wan2 >> default via 82.236.230.254 dev eth3 src 82.236.230.217 >> prohibit default metric 1 >> >> Fw-cgarp:/etc/firegate# ip route ls table away >> default >>nexthop via 82.236.230.254 dev eth3 weight 1 >>nexthop via 213.223.96.122 dev eth0 weight 1 >> >> Fw-cgarp:/etc/firegate# iptables-save -t mangle >> # Generated by iptables-save v1.2.11 on Wed Apr 6 11:57:06 2005 >> *mangle >> :PREROUTING ACCEPT [3281:1066576] >> :INPUT ACCEPT [411:32992] >> :FORWARD ACCEPT [2870:1033584] >> :OUTPUT ACCEPT [339:63745] >> :POSTROUTING ACCEPT [3195:1096657] >> -A PREROUTING -p tcp -m tcp --dport 25 -j MARK --set-mark 0xa >> -A PREROUTING -p tcp -m mport --dports 80,443,21 -j MARK --set-mark 0x14 >> COMMIT >> # Completed on Wed Apr 6 11:57:06 2005 >> >> >> >> So with this configuration all the http,https and ftp traffic must be >> routed by the 'wan2' connection. >> I have done severals tests and it dont work, i have also had a realms mark >> to my routing rule and with > the "rtacct" command i saw that traffic going >> through the correct rule, but http traffic continues to > be balanced >> between the two connections... >> >> If someone see the problem ? >> Thx in advance. >> ___ >> LARTC mailing list >> LARTC@mailman.ds9a.nl >> http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc ___ LARTC mailing list LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc
Re: [LARTC] Multipath routing + traffic separation problem.
Your settings seem to be correct, I just don't know why you don't want to balance http, https and ftp traffic between both connections? About the bug, I haven't used linux 2.4 for a long time, for 2.6, fwmark is in hexa, so be careful with 10 vs. 0xa, you'd better use values less than 0xa to avoid confusing. Also make sure that no default route is added to your main table. On Wed, 2005-04-06 at 12:09 +0200, Laurent LAVAUD wrote: Hello, I have set up a multipath gateway. System is a linux 2.4.29 kernel, iproute 20010824, iptables 1.2.11. here is the setup: firewall:/# ip rule 0: from all lookup local 100:from all lookup main 152:from all fwmark 10 lookup wan1 153:from all fwmark 20 lookup wan2 201:from 213.223.96.121 lookup wan1 202:from 82.236.230.217 lookup wan2 1000: from all lookup away Fw-cgarp:/etc/firegate# ip route ls table wan1 default via 213.223.96.122 dev eth0 src 213.223.96.121 prohibit default metric 1 Fw-cgarp:/etc/firegate# ip route ls table wan2 default via 82.236.230.254 dev eth3 src 82.236.230.217 prohibit default metric 1 Fw-cgarp:/etc/firegate# ip route ls table away default nexthop via 82.236.230.254 dev eth3 weight 1 nexthop via 213.223.96.122 dev eth0 weight 1 Fw-cgarp:/etc/firegate# iptables-save -t mangle # Generated by iptables-save v1.2.11 on Wed Apr 6 11:57:06 2005 *mangle :PREROUTING ACCEPT [3281:1066576] :INPUT ACCEPT [411:32992] :FORWARD ACCEPT [2870:1033584] :OUTPUT ACCEPT [339:63745] :POSTROUTING ACCEPT [3195:1096657] -A PREROUTING -p tcp -m tcp --dport 25 -j MARK --set-mark 0xa -A PREROUTING -p tcp -m mport --dports 80,443,21 -j MARK --set-mark 0x14 COMMIT # Completed on Wed Apr 6 11:57:06 2005 So with this configuration all the http,https and ftp traffic must be routed by the 'wan2' connection. I have done severals tests and it dont work, i have also had a realms mark to my routing rule and with the "rtacct" command i saw that traffic going through the correct rule, but http traffic continues to be balanced between the two connections... If someone see the problem ? Thx in advance. ___ LARTC mailing list LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc ___ LARTC mailing list LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc
RE: [LARTC] multipath routing
I have a private lan that is connected to the world via 3 dsl lines. I putup a linux box that handles all the dsl lines, lan gateway and all isworking well...until...one of the dsl lines goes down. My routing table is:x.x.x.x dev ppp0 proto kernel scope link src x.x.x.xx.x.x.x dev ppp1 proto kernel scope link src x.x.x.xx.x.x.x dev ppp2 proto kernel scope link src x.x.x.x192.168.1.0/24 dev eth0 proto kernel scope link src 192.168.1.1255.255.255.0/24 dev eth0 scope link169.254.0.0/16 dev eth0 scope link127.0.0.0/8 dev lo scope linkdefault equalize nexthop via x.x.x.x dev ppp0 weight 1 nexthop via x.x.x.x dev ppp1 weight 1 nexthop via x.x.x.x dev ppp2 weight 1When one of the dsl lines has trouble (temporarily looses sync, etc), thekernel takes the entire default route out until the line comes back up.When the line comes up, it puts a single default route back in instead ofthe multipath route. If I can make it so the multipath route is maintainedeven when one of the dsl lines goes down, that would be great. Is there away to have just one of the hops removed from the default multipath whilethe line is down and then reinserted back in as a hop in the multipathroute? Thanks for any input.-Chris Do you have the stateful firewall settings in your script? According to nano.txt http://www.ssi.bg/~ja/nano.txt "At least for netfilter (not sure for ipfwadm/ipchains), the firewallmust be stateful. This can be done by: iptables -t filter -N keep_state iptables -t filter -A keep_state -m state --state RELATED,ESTABLISHED \ -j ACCEPT iptables -t filter -A keep_state -j RETURN iptables -t nat -N keep_state iptables -t nat -A keep_state -m state --state RELATED,ESTABLISHED \ -j ACCEPT iptables -t nat -A keep_state -j RETURNand calling this at the beginning of the script: iptables -t nat -A PREROUTING -j keep_state iptables -t nat -A POSTROUTING -j keep_state iptables -t nat -A OUTPUT -j keep_state iptables -t filter -A INPUT -j keep_state iptables -t filter -A FORWARD -j keep_state iptables -t filter -A OUTPUT -j keep_state" /sbin/iptables-save # Generated by iptables-save v1.2.7a on Wed Mar 24 15:54:00 2004*nat:PREROUTING ACCEPT [9983:812849]:POSTROUTING ACCEPT [0:0]:OUTPUT ACCEPT [3:174]:keep_state - [0:0]-A PREROUTING -j keep_state -A POSTROUTING -o ppp+ -j MASQUERADE -A POSTROUTING -j keep_state -A OUTPUT -j keep_state -A keep_state -m state --state RELATED,ESTABLISHED -j ACCEPT -A keep_state -j RETURN COMMIT# Completed on Wed Mar 24 15:54:00 2004# Generated by iptables-save v1.2.7a on Wed Mar 24 15:54:00 2004*filter:INPUT ACCEPT [1020:161876]:FORWARD DROP [0:0]:OUTPUT ACCEPT [425:33288]:keep_state - [0:0]-A INPUT -i lo -m state --state NEW,ESTABLISHED -j ACCEPT -A INPUT -i ppp+ -m state --state INVALID -j DROP -A INPUT -i ppp+ -m state --state ESTABLISHED -j ACCEPT -A INPUT -i ppp+ -p tcp -j DROP -A INPUT -i ppp+ -p udp -j DROP -A INPUT -i ppp+ -p icmp -j DROP -A INPUT -j keep_state -A FORWARD -i ppp+ -o eth+ -m state --state RELATED,ESTABLISHED -j ACCEPT -A FORWARD -i eth+ -o ppp+ -j ACCEPT -A FORWARD -j keep_state -A OUTPUT -o lo -m state --state NEW,ESTABLISHED -j ACCEPT -A OUTPUT -o ppp+ -m state --state NEW,ESTABLISHED -j ACCEPT -A OUTPUT -j keep_state -A keep_state -m state --state RELATED,ESTABLISHED -j ACCEPT -A keep_state -j RETURN COMMIT# Completed on Wed Mar 24 15:54:00 2004 The link you reference (http://www.ssi.bg/~ja/nano.txt) suggests several patches to be applied to the kernel for the routing described to be possible. I would like to do this, but it is a company box and they want a "standard" installation which basically means no patching for me. The box is running the most up to date kernel for a RedHat 9.0 install. Thanks for any input. -Chris
Re: [LARTC] multipath routing
I have a private lan that is connected to the world via 3 dsl lines. I put up a linux box that handles all the dsl lines, lan gateway and all is working well...until...one of the dsl lines goes down. My routing table is: x.x.x.x dev ppp0 proto kernel scope link src x.x.x.x x.x.x.x dev ppp1 proto kernel scope link src x.x.x.x x.x.x.x dev ppp2 proto kernel scope link src x.x.x.x 192.168.1.0/24 dev eth0 proto kernel scope link src 192.168.1.1 255.255.255.0/24 dev eth0 scope link 169.254.0.0/16 dev eth0 scope link 127.0.0.0/8 dev lo scope link default equalize nexthop via x.x.x.x dev ppp0 weight 1 nexthop via x.x.x.x dev ppp1 weight 1 nexthop via x.x.x.x dev ppp2 weight 1 When one of the dsl lines has trouble (temporarily looses sync, etc), the kernel takes the entire default route out until the line comes back up. When the line comes up, it puts a single default route back in instead of the multipath route. If I can make it so the multipath route is maintained even when one of the dsl lines goes down, that would be great. Is there a way to have just one of the hops removed from the default multipath while the line is down and then reinserted back in as a hop in the multipath route? Thanks for any input. -Chris Do you have the stateful firewall settings in your script? According to nano.txt http://www.ssi.bg/~ja/nano.txt "At least for netfilter (not sure for ipfwadm/ipchains), the firewall must be stateful. This can be done by: iptables -t filter -N keep_state iptables -t filter -A keep_state -m state --state RELATED,ESTABLISHED \ -j ACCEPT iptables -t filter -A keep_state -j RETURN iptables -t nat -N keep_state iptables -t nat -A keep_state -m state --state RELATED,ESTABLISHED \ -j ACCEPT iptables -t nat -A keep_state -j RETURN and calling this at the beginning of the script: iptables -t nat -A PREROUTING -j keep_state iptables -t nat -A POSTROUTING -j keep_state iptables -t nat -A OUTPUT -j keep_state iptables -t filter -A INPUT -j keep_state iptables -t filter -A FORWARD -j keep_state iptables -t filter -A OUTPUT -j keep_state "
Re: [LARTC] Multipath Routing Question with Public networks
On Mon, 2002-11-18 at 12:59, Julian Anastasov wrote: > > ensure traffic always go through the right interface. > > > - TCP connect() for unbound socket uses saddr=0.0.0.0 daddr=REMOTE_IP. > > > The routing then returns the best source IP to use for this connection > > > after creating a connected route in the routing cache. > > What do you mean by "unbound socket" ? > > socket(), connect(), i.e. when there is no bind() to local addr. > > Regards Thanks julian ! I'll let you know how it goes. Thanks again. Regards, Vincent. > > -- > Julian Anastasov <[EMAIL PROTECTED]> > > ___ > LARTC mailing list / [EMAIL PROTECTED] > http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/ -- Vincent Jaussaud <[EMAIL PROTECTED]> Kelkoo.com ___ LARTC mailing list / [EMAIL PROTECTED] http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
Re: [LARTC] Multipath Routing Question with Public networks
Hello, On 18 Nov 2002, Vincent Jaussaud wrote: > Right. And disabling rp_filter might open a security hole; so I'll For internal interfaces rp_filter is optional. > ensure traffic always go through the right interface. > > - TCP connect() for unbound socket uses saddr=0.0.0.0 daddr=REMOTE_IP. > > The routing then returns the best source IP to use for this connection > > after creating a connected route in the routing cache. > What do you mean by "unbound socket" ? socket(), connect(), i.e. when there is no bind() to local addr. Regards -- Julian Anastasov <[EMAIL PROTECTED]> ___ LARTC mailing list / [EMAIL PROTECTED] http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
Re: [LARTC] Multipath Routing Question with Public networks
On Sun, 2002-11-17 at 21:43, Julian Anastasov wrote: > Hello, > Hi ! > On Sun, 17 Nov 2002, Vincent Jaussaud Mailing Listes wrote: > > Yes, this is a problem, job for user space tools to change > the routing settings on failure. Ok, I think I can manage to write some scripts to manage the routing rules, depending on the state of the links. > Should not happen for TCP servers but sometimes the UDP servers > are not smart enough when used on multihomed servers. See below. Ok. If all TCP Servers behaves correctly, then it's all I need. > > Firewall with rp_filter set on internal interfaces > expects the traffic to come from the right internal interface (I > assume you have the two pubnets configured on different internal > interfaces). There is no such problem if the internal interfaces > do not use rp_filter. Right. And disabling rp_filter might open a security hole; so I'll ensure traffic always go through the right interface. > > > I mean, we don't really care what link is beeing used for a reply, as soon as > > the SRC IP & DST IP are correct. It's likely that ISP1 & ISP2 router won't do > > source address validation anyway. Am I wrong ? > > If the ISPs allow spoofing then while the links are alive > there is no problem, it comes when some ISP fails. We should stop > using its addresses in this case. > Right. > daddr is always used. > > Some examples (of course, there are other route keys used, > not shown here): > > - TCP connect() for unbound socket uses saddr=0.0.0.0 daddr=REMOTE_IP. > The routing then returns the best source IP to use for this connection > after creating a connected route in the routing cache. What do you mean by "unbound socket" ? > - TCP connect() after bind() uses saddr=LOCAL_IP daddr=REMOTE_IP > > - TCP listener uses saddr=LOCAL_IP daddr=REMOTE_IP when replying to > SYN > > - UDP can also avoid using 0.0.0.0 as saddr if the socket is bound > or when IP_PKTINFO contains local IP information. If the app does > not take steps to inform the kernel that this socket is bound > to some local IP when sending the packet then 0.0.0.0 is used > as src IP for the route lookup (ignoring the fact that this > UDP packet has known saddr in iphdr). So, it depends both on > transport and on app to feed the routing with the right keys. > Ok. Seems like I'll have to make some heavy testing. :) Thanks again. Vincent. > > Vincent. > > Regards > > -- > Julian Anastasov <[EMAIL PROTECTED]> > > ___ > LARTC mailing list / [EMAIL PROTECTED] > http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/ -- Vincent Jaussaud <[EMAIL PROTECTED]> Kelkoo.com ___ LARTC mailing list / [EMAIL PROTECTED] http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
Re: [LARTC] Multipath Routing Question with Public networks
Hello, On Sun, 17 Nov 2002, Vincent Jaussaud Mailing Listes wrote: > On top of this, defaults gateways for internal servers are unlikely to become > unreachable, since they are on the firewall itself. > Gateways which are likely to colapse, are the ones used by the firewall, eg > ISP1 & ISP2 routers, and this will never be noticed by the internal servers > since they are not directly connected to them. Am I wrong ? Yes, this is a problem, job for user space tools to change the routing settings on failure. > > The server uses the 10.0.0.1 as source when resolving > > route for the reply. Then it depends on the routing rules. > That's good news. So it means that if my routing rules are setup correctly, it > will work. I was afraid of having some packets sent back using a wrong SRC IP :) > > I assume however that this will work only for usual protocol such as http, > smtp, ssh... I'll have to find a workaround for protocols like FTP, if any. > Any idea ? Should not happen for TCP servers but sometimes the UDP servers are not smart enough when used on multihomed servers. See below. > But, just beeing curious, why would rp_filter drop such packets ? Firewall with rp_filter set on internal interfaces expects the traffic to come from the right internal interface (I assume you have the two pubnets configured on different internal interfaces). There is no such problem if the internal interfaces do not use rp_filter. > I mean, we don't really care what link is beeing used for a reply, as soon as > the SRC IP & DST IP are correct. It's likely that ISP1 & ISP2 router won't do > source address validation anyway. Am I wrong ? If the ISPs allow spoofing then while the links are alive there is no problem, it comes when some ISP fails. We should stop using its addresses in this case. > > The transport and the application decide what source IP > > to put in the reply. Then they decide how to call the routing. > > The right thing to do when addresses to both ends are known is > > to feed the routing with saddr and daddr. If callers use 0.0.0.0 > > as saddr when resolving routes, they will hit the multipath > > route which is bad. > > > I'm not sure to understand this point, especially "to feed the routing with > saddr and daddr." > I know we can instruct the routing to use a specific saddr, but what about daddr ? daddr is always used. Some examples (of course, there are other route keys used, not shown here): - TCP connect() for unbound socket uses saddr=0.0.0.0 daddr=REMOTE_IP. The routing then returns the best source IP to use for this connection after creating a connected route in the routing cache. - TCP connect() after bind() uses saddr=LOCAL_IP daddr=REMOTE_IP - TCP listener uses saddr=LOCAL_IP daddr=REMOTE_IP when replying to SYN - UDP can also avoid using 0.0.0.0 as saddr if the socket is bound or when IP_PKTINFO contains local IP information. If the app does not take steps to inform the kernel that this socket is bound to some local IP when sending the packet then 0.0.0.0 is used as src IP for the route lookup (ignoring the fact that this UDP packet has known saddr in iphdr). So, it depends both on transport and on app to feed the routing with the right keys. > Vincent. Regards -- Julian Anastasov <[EMAIL PROTECTED]> ___ LARTC mailing list / [EMAIL PROTECTED] http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
Re: [LARTC] Multipath Routing Question with Public networks
> Hello, > Hi julian ! > On Sat, 16 Nov 2002, Vincent Jaussaud wrote: > > > On All servers, we setup multipath default route, so that they can use both link as well. > > That means they know which link is alive or it does not > matter? :) > Well, at this time, it doesn't matter yet :-) Anyway, it would be best if they could, but I'm afraid that all internal servers will have to run your patch in order for this to work. And I'm talking about 20 production servers. So, I can't easily patch them & reboot :) On top of this, defaults gateways for internal servers are unlikely to become unreachable, since they are on the firewall itself. Gateways which are likely to colapse, are the ones used by the firewall, eg ISP1 & ISP2 routers, and this will never be noticed by the internal servers since they are not directly connected to them. Am I wrong ? > > Let's say we have Server A in our internal public network, with 2 network adapters, one > > using 10.0.0.1 (routed through ISP1), other using 172.16.0.1 (routed through ISP2) > > > > Assuming that rp_filter is configured correctly on our firewall, what happens when a > > client want to reach Server A using 10.0.0.1 ? What path will be used for the replies, > > with what source IP ? > > The server uses the 10.0.0.1 as source when resolving > route for the reply. Then it depends on the routing rules. That's good news. So it means that if my routing rules are setup correctly, it will work. I was afraid of having some packets sent back using a wrong SRC IP :) I assume however that this will work only for usual protocol such as http, smtp, ssh... I'll have to find a workaround for protocols like FTP, if any. Any idea ? > > > I assume that if rp_filter is configured correctly, return path do not matter (since we > > don't do NAT), but I'm worried about the SRC IP beeing choosed for the reply. Because, if > > the kernel choose the src IP according to the output default route beeing choosed, then > > half clients<->servers sessions will just break. > > I'm sure you need correct routes on firewall and all internal > hosts, for example: > > ip rule add prio ... from pubnet_X to 0/0 table table_for_ISP_X > ip rule add prio ... from pubnet_Y to 0/0 table table_for_ISP_Y > ip rule add prio ... from 0/0 to 0/0 table your_multipath_route > > Of course, the internal hosts use the proper firewall > IP as gateway. > Yes, of course. Firewall already have routing policy setup this way, and all internal servers will be reconfigured in a similar way. > That is all, traffic from specific pub IP should use only > its gateway. You can expect rp_filter drops in firewall if > the internal servers select wrong NIC by using multipath > route for all route resolutions. The multipath route should be > used only for source address autoselection: > Ok, this is not a problem. But, just beeing curious, why would rp_filter drop such packets ? I mean, we don't really care what link is beeing used for a reply, as soon as the SRC IP & DST IP are correct. It's likely that ISP1 & ISP2 router won't do source address validation anyway. Am I wrong ? > - originating connections without bind() > - selecting masquerade address (for NAT) > - etc > > I.e. it is a bad idea to use only multipath route. > > Internal servers creating outgoing connections after > using bind() to specific pubip should not reach the multipath route. > Ok. I'll ensure that multipath route is beeing used only if SRC IP isn't set, on all internal servers. > > So, basically my question is, what rules decide of the SRC IP to be used in a reply > > packet on a system with several default route through different network interfaces ? > > The transport and the application decide what source IP > to put in the reply. Then they decide how to call the routing. > The right thing to do when addresses to both ends are known is > to feed the routing with saddr and daddr. If callers use 0.0.0.0 > as saddr when resolving routes, they will hit the multipath > route which is bad. > I'm not sure to understand this point, especially "to feed the routing with saddr and daddr." I know we can instruct the routing to use a specific saddr, but what about daddr ? > > Has anyone already experiencing such setup ? > > Not exactly, but everything is in the details :) Once again, you're saving me :-) Thanks a lot. Regards, Vincent. > > > Vincent. > > Regards > > -- > Julian Anastasov <[EMAIL PROTECTED]> -- Kelkoo.com: http://www.kelkoo.com ___ LARTC mailing list / [EMAIL PROTECTED] http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
Re: [LARTC] Multipath Routing Question with Public networks
Hello, On Sat, 16 Nov 2002, Vincent Jaussaud wrote: > On All servers, we setup multipath default route, so that they can use both link as >well. That means they know which link is alive or it does not matter? :) > Let's say we have Server A in our internal public network, with 2 network adapters, >one > using 10.0.0.1 (routed through ISP1), other using 172.16.0.1 (routed through ISP2) > > Assuming that rp_filter is configured correctly on our firewall, what happens when a > client want to reach Server A using 10.0.0.1 ? What path will be used for the >replies, > with what source IP ? The server uses the 10.0.0.1 as source when resolving route for the reply. Then it depends on the routing rules. > I assume that if rp_filter is configured correctly, return path do not matter (since >we > don't do NAT), but I'm worried about the SRC IP beeing choosed for the reply. >Because, if > the kernel choose the src IP according to the output default route beeing choosed, >then > half clients<->servers sessions will just break. I'm sure you need correct routes on firewall and all internal hosts, for example: ip rule add prio ... from pubnet_X to 0/0 table table_for_ISP_X ip rule add prio ... from pubnet_Y to 0/0 table table_for_ISP_Y ip rule add prio ... from 0/0 to 0/0 table your_multipath_route Of course, the internal hosts use the proper firewall IP as gateway. That is all, traffic from specific pub IP should use only its gateway. You can expect rp_filter drops in firewall if the internal servers select wrong NIC by using multipath route for all route resolutions. The multipath route should be used only for source address autoselection: - originating connections without bind() - selecting masquerade address (for NAT) - etc I.e. it is a bad idea to use only multipath route. Internal servers creating outgoing connections after using bind() to specific pubip should not reach the multipath route. > So, basically my question is, what rules decide of the SRC IP to be used in a reply > packet on a system with several default route through different network interfaces ? The transport and the application decide what source IP to put in the reply. Then they decide how to call the routing. The right thing to do when addresses to both ends are known is to feed the routing with saddr and daddr. If callers use 0.0.0.0 as saddr when resolving routes, they will hit the multipath route which is bad. > Has anyone already experiencing such setup ? Not exactly, but everything is in the details :) > Vincent. Regards -- Julian Anastasov <[EMAIL PROTECTED]> ___ LARTC mailing list / [EMAIL PROTECTED] http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
Re: Re: [LARTC] multipath routing problem [Shorter version] - Helpstill needed :-)
Hello, On 29 Oct 2002, Vincent Jaussaud wrote: > I was thinking about using the metric value for this. > > Let's say: > > ip route add table dual-gw proto static 192.168.0.0/24 via GW1 dev eth1 > metric 1 > ip route add table dual-gw proto static 192.168.0.0/24 via GW2 dev eth1 > metric 2 No, the metrics work only if the routes disappear. This usually happens when device goes down (for example, ppp). For gateways reachable via ARP it can't work. You need to define alternative routes (ip route append), see my docs about alt routes that use same metric. > I assume the kernel will always use the best route, that is the one with > best metric. So that all packets will use the same route. > If GW1 breaks, patched kernel should mark first route as dead, and force > all further packets to use GW2 instead. No, dead gateway detection currently works for routes with same metric. But even then the detection is passive and needs help from user space. Without such checks you can expect almost random results. OTOH, you can run your own checks and to keep only the alive routes. Regards -- Julian Anastasov <[EMAIL PROTECTED]> ___ LARTC mailing list / [EMAIL PROTECTED] http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
Re: Re: [LARTC] multipath routing problem [Shorter version] - Helpstill needed :-)
On Mon, 2002-10-28 at 23:21, Julian Anastasov wrote: > > Hello, > > On 28 Oct 2002, Vincent Jaussaud wrote: > > > My question is, if we ensure that EVERY packets, whatever path they use > > to arrive, finally pass through a single peer doing NAT, is this suppose > > to work around my TOS problem ? > > Sounds correct. The requirement is each packet from one > connection to be NAT-ed only from one NAT router and to same > masquerade address and port. The routing cache can not guarantee > that. It can be done only from the patched masquerade. > Hmmm.. then that's why it doesn't work.. final gateway doing NAT isn't patched, only the first one is. I think I'll have to drop the idea of using both gateways simultaneously. Now, If I only want do to fail-over (eg; only one gateway used at the same time, other one used only in case the first one breaks.) I was thinking about using the metric value for this. Let's say: ip route add table dual-gw proto static 192.168.0.0/24 via GW1 dev eth1 metric 1 ip route add table dual-gw proto static 192.168.0.0/24 via GW2 dev eth1 metric 2 I assume the kernel will always use the best route, that is the one with best metric. So that all packets will use the same route. If GW1 breaks, patched kernel should mark first route as dead, and force all further packets to use GW2 instead. Is this suppose to work ? Or can we use different metric value inside a multipath route, like: ip route add table dual-gw proto static 192.168.0.0/24 nexthop via GW1 dev eth1 metric 1 nexthop via GW2 dev eth1 metric 2 ? Anyway, the more I think about this setup, the more I think I should use a clustering solution instead. Maybe a cluster of gateway with one VIP is much more appropriate for what I want to build. I'll use multipath routing for ISP redundency then :) Thanks to both of you, I've learn a lot during the last past few days, this was one of my main concern too. Thanks again. Cheers, Vincent. > > What about the rp_filter kernel value ? Could it be a problem in such > > setup ? > > The patches are designed to work with rp_filter enabled. > You can safely use it, it is changed to work only with the defined > paths. > > > Thanks again. > > Vincent. > > Regards > > -- > Julian Anastasov <[EMAIL PROTECTED]> > > ___ > LARTC mailing list / [EMAIL PROTECTED] > http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/ -- Vincent Jaussaud Kelkoo.com Security Manager email: [EMAIL PROTECTED] "The UNIX philosophy is to design small tools that do one thing, and do it well." ___ LARTC mailing list / [EMAIL PROTECTED] http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
Re: Re: [LARTC] multipath routing problem [Shorter version] - Helpstill needed :-)
Hello, On 28 Oct 2002, Vincent Jaussaud wrote: > My question is, if we ensure that EVERY packets, whatever path they use > to arrive, finally pass through a single peer doing NAT, is this suppose > to work around my TOS problem ? Sounds correct. The requirement is each packet from one connection to be NAT-ed only from one NAT router and to same masquerade address and port. The routing cache can not guarantee that. It can be done only from the patched masquerade. > What about the rp_filter kernel value ? Could it be a problem in such > setup ? The patches are designed to work with rp_filter enabled. You can safely use it, it is changed to work only with the defined paths. > Thanks again. > Vincent. Regards -- Julian Anastasov <[EMAIL PROTECTED]> ___ LARTC mailing list / [EMAIL PROTECTED] http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
Re: Re: [LARTC] multipath routing problem [Shorter version] - Helpstill needed :-)
> > It seems you can safely alter the TOS for all packets > entering your box/site. > Ok, I'll dig into this tip, and see how it goes. If I can't figure out this NAT problem, I'll do this. > May be you can hunt it with tcpdump. I assume your are > using the patches because the plain kernel has the same problem > for NAT. > Yes, I am running your patch. Kernel is 2.2.22 with routes-2.2.20-7.diff patch applied. (I'm sure of this, otherwise dead gateway detection will simply not work.) My question is, if we ensure that EVERY packets, whatever path they use to arrive, finally pass through a single peer doing NAT, is this suppose to work around my TOS problem ? Eg, end services will only see packets coming from the last NAT address, which is single whatever path packets used to arrive. Something like: LAN --> Multipath Firewall | | GW1GW2 | | --- | Gateway (NAT) | - Remote Network What about the rp_filter kernel value ? Could it be a problem in such setup ? Thanks again. Vincent. > > A big thanks to both of you. I've learned a lot today :) > > > > Thanks again. > > Regards, > > Vincent. > > Regards > > -- > Julian Anastasov <[EMAIL PROTECTED]> > > ___ > LARTC mailing list / [EMAIL PROTECTED] > http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/ -- Vincent Jaussaud Kelkoo.com Security Manager email: [EMAIL PROTECTED] "The UNIX philosophy is to design small tools that do one thing, and do it well." ___ LARTC mailing list / [EMAIL PROTECTED] http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
Re: Re: [LARTC] multipath routing problem [Shorter version] - Helpstill needed :-)
Hello, On 25 Oct 2002, Vincent Jaussaud wrote: > > 2.4 mask 0x1C, inverted 0xE3 > > 2.2 mask 0x1E, inverted 0xE1 > > > > So, for 2.2 may be: > > > > ipchains -I input -d 0.0.0.0/0 22 -t 0xE3 0x00 > Just tried. Now SSH connections don't break anymore !!! :) Thanks ! > Am I suppose to do this on both side, or doing this on the firewall > itself is enough ? I now see that my example ipchains command is wrong, use 0xE1 for 2.2 as the above table. > The only problem with this, is that I will need to do this trick for any > applications changing it's TOS during the session. It seems that FTP > behaves exactly the same way as SSH, regarding the TOS field. It seems you can safely alter the TOS for all packets entering your box/site. > Do you guys know if many applications do this ? Or is this just > particular to SSH & FTP ? The TOS is usually used for routing between routers in your site, then the border gateways can assign different priorities based on the TOS values, for traffic control purposes. > Anyway, I really would like to understand why it doesn't work when doing > NAT. May be you can hunt it with tcpdump. I assume your are using the patches because the plain kernel has the same problem for NAT. > A big thanks to both of you. I've learned a lot today :) > > Thanks again. > Regards, > Vincent. Regards -- Julian Anastasov <[EMAIL PROTECTED]> ___ LARTC mailing list / [EMAIL PROTECTED] http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
Re: Re: [LARTC] multipath routing problem [Shorter version] - Helpstill needed :-)
On Fri, 2002-10-25 at 20:45, Julian Anastasov wrote: > > Hello, > > On Fri, 25 Oct 2002, Arthur van Leeuwen wrote: > > > > Now I see, then the TOS is a big problem for you. May > > > be your problem will be solved if TOS is not a routing key but > > > it does not sound as a thing that is easy to fix in kernel. > > > > Actually, you can simply play whack-a-mole with the TOS value, using > > ipchains (or iptables), killing all TOS values present on the packets. > > Ofcourse, this is not very *nice*, but it'll work. > > This is a good idea. Vincent, may be you can play with > ipchains -t AND XOR in the input chain to see what happens. Just make sure > you don't touch bits 0, 1, 5, 6, 7. It seems the routing uses only bits > 2, 3 and 4 for routing key (if I'm not overlooking something). > This is for kernel 2.4. For kernel 2.2 it seems bit 1 is also > included in the routing key. > > 2.4 mask 0x1C, inverted 0xE3 > 2.2 mask 0x1E, inverted 0xE1 > > So, for 2.2 may be: > > ipchains -I input -d 0.0.0.0/0 22 -t 0xE3 0x00 Just tried. Now SSH connections don't break anymore !!! :) Thanks ! Am I suppose to do this on both side, or doing this on the firewall itself is enough ? > > What are the TOS values used during the SSH session? Right after authentication, TOS value is set to 0x10 20:53:46.515566 192.168.0.2.ssh > 172.1.1.3.2418: R 4008315859:4008315859(0) win 0 [tos 0x10] The only problem with this, is that I will need to do this trick for any applications changing it's TOS during the session. It seems that FTP behaves exactly the same way as SSH, regarding the TOS field. Do you guys know if many applications do this ? Or is this just particular to SSH & FTP ? Anyway, I really would like to understand why it doesn't work when doing NAT. A big thanks to both of you. I've learned a lot today :) Thanks again. Regards, Vincent. > > Regards > > -- > Julian Anastasov <[EMAIL PROTECTED]> -- Vincent Jaussaud Kelkoo.com Security Manager email: [EMAIL PROTECTED] "The UNIX philosophy is to design small tools that do one thing, and do it well." ___ LARTC mailing list / [EMAIL PROTECTED] http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
Re: Re: [LARTC] multipath routing problem [Shorter version] - Helpstill needed :-)
Hello, On Fri, 25 Oct 2002, Arthur van Leeuwen wrote: > > Now I see, then the TOS is a big problem for you. May > > be your problem will be solved if TOS is not a routing key but > > it does not sound as a thing that is easy to fix in kernel. > > Actually, you can simply play whack-a-mole with the TOS value, using > ipchains (or iptables), killing all TOS values present on the packets. > Ofcourse, this is not very *nice*, but it'll work. This is a good idea. Vincent, may be you can play with ipchains -t AND XOR in the input chain to see what happens. Just make sure you don't touch bits 0, 1, 5, 6, 7. It seems the routing uses only bits 2, 3 and 4 for routing key (if I'm not overlooking something). This is for kernel 2.4. For kernel 2.2 it seems bit 1 is also included in the routing key. 2.4 mask 0x1C, inverted 0xE3 2.2 mask 0x1E, inverted 0xE1 So, for 2.2 may be: ipchains -I input -d 0.0.0.0/0 22 -t 0xE3 0x00 What are the TOS values used during the SSH session? Regards -- Julian Anastasov <[EMAIL PROTECTED]> ___ LARTC mailing list / [EMAIL PROTECTED] http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
Re: Re: [LARTC] multipath routing problem [Shorter version] - Helpstill needed :-)
On Fri, 2002-10-25 at 20:21, Arthur van Leeuwen wrote: > On 25 Oct 2002, Vincent Jaussaud wrote: > > > However, I don't get why, in the same SSH session, TOS may differ from > > one packet to another. Using tcpdump, it seems that TOS value change > > right after the authentication has been successfully made. > > Shit... you figured that one out *quite* a bit faster than I did at the > time... took me two weeks. > :-) > What openssh does is first authenticate, then set the TOS value depending on > whether you're doing interactive communications (ssh) or bulk transfer > (scp). One could see this as a way of minimizing information leakage... > OK, now I know why openssh is changing it's TOS !. Thanks. :-) > Oh, and yes, it does what you deduced. I finally got that from reading the > sources... I could mangle the TOS field as you suggested, but I don't like this, since packets *should* be able to find their way out, whatever path they use to come back. The thing I don't understand, is that even by NAT'ing everything, everywhere, my connections still break. I've tried to NAT on the firewall everything coming from a test IP, just to see how it goes. No luck. I even tried NAT'ing on the firewall, then on the gateways, then on the final router, in the other network. Still no luck ! This is non sense ! There has to be something wrong, somewhere. Thanks for your reply. Regards, Vincent. > > Doei, Arthur. > > -- > /\/ | [EMAIL PROTECTED] | Work like you don't need the money > /__\ / | A friend is someone with whom | Love like you have never been hurt > /\/__ | you can dare to be yourself | Dance like there's nobody watching > > ___ > LARTC mailing list / [EMAIL PROTECTED] > http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/ -- Vincent Jaussaud Kelkoo.com Security Manager email: [EMAIL PROTECTED] "The UNIX philosophy is to design small tools that do one thing, and do it well." ___ LARTC mailing list / [EMAIL PROTECTED] http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
Re: Re: [LARTC] multipath routing problem [Shorter version] - Helpstill needed :-)
On 25 Oct 2002, Vincent Jaussaud wrote: > However, I don't get why, in the same SSH session, TOS may differ from > one packet to another. Using tcpdump, it seems that TOS value change > right after the authentication has been successfully made. Shit... you figured that one out *quite* a bit faster than I did at the time... took me two weeks. What openssh does is first authenticate, then set the TOS value depending on whether you're doing interactive communications (ssh) or bulk transfer (scp). One could see this as a way of minimizing information leakage... Oh, and yes, it does what you deduced. I finally got that from reading the sources... Doei, Arthur. -- /\/ | [EMAIL PROTECTED] | Work like you don't need the money /__\ / | A friend is someone with whom | Love like you have never been hurt /\/__ | you can dare to be yourself | Dance like there's nobody watching ___ LARTC mailing list / [EMAIL PROTECTED] http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
Re: Re: [LARTC] multipath routing problem [Shorter version] - Helpstill needed :-)
On Fri, 25 Oct 2002, Julian Anastasov wrote: > Hello, > On 25 Oct 2002, Vincent Jaussaud wrote: > > But traffic is NAT-ed after multipath routing occurs ! > > Eg, the box which do multipath routing do not NAT traffic; traffic get > > NAT-ed when leaving the gateways: > > > > LAN --> FW w/ multipath-routing > >|| > > Gateway1 Gateway2 > >| (NAT) | (NAT) > >|| > > Remote Network > > > > Packets reach the Remote Network using one of the Gateway NAT-ed IP, so > > that when packets come back they should use the proper return path. Am I > > wrong ? > > Now I see, then the TOS is a big problem for you. May > be your problem will be solved if TOS is not a routing key but > it does not sound as a thing that is easy to fix in kernel. Actually, you can simply play whack-a-mole with the TOS value, using ipchains (or iptables), killing all TOS values present on the packets. Ofcourse, this is not very *nice*, but it'll work. Doei, Arthur. -- /\/ | [EMAIL PROTECTED] | Work like you don't need the money /__\ / | A friend is someone with whom | Love like you have never been hurt /\/__ | you can dare to be yourself | Dance like there's nobody watching ___ LARTC mailing list / [EMAIL PROTECTED] http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
Re: Re: [LARTC] multipath routing problem [Shorter version] - Helpstill needed :-)
On Fri, 2002-10-25 at 18:12, Julian Anastasov wrote: > > Hello, > > On 25 Oct 2002, Vincent Jaussaud wrote: > > > But traffic is NAT-ed after multipath routing occurs ! > > Eg, the box which do multipath routing do not NAT traffic; traffic get > > NAT-ed when leaving the gateways: > > > > LAN --> FW w/ multipath-routing > >|| > > Gateway1 Gateway2 > >| (NAT) | (NAT) > >|| > > Remote Network > > > > Packets reach the Remote Network using one of the Gateway NAT-ed IP, so > > that when packets come back they should use the proper return path. Am I > > wrong ? > > Now I see, then the TOS is a big problem for you. May > be your problem will be solved if TOS is not a routing key but > it does not sound as a thing that is easy to fix in kernel. > > > Hmmm... Then this is where the problem is. So, if I understand > > correctly, packets coming from a single TCP connections will use any of > > the multipath route _if_ they are not NAT-ed ? Isn't the routing cache > > Yes, traffic A->B with TOS=XXX will use one path, traffic > A->B with TOS=YYY can use different path. > Ok, I understand a lot of things now, thanks to you. However, I don't get why, in the same SSH session, TOS may differ from one packet to another. Using tcpdump, it seems that TOS value change right after the authentication has been successfully made. > > suppose to ensure that every packets coming from a single connection use > > the same path ? > > The routing does not cache connections but route resolutions keyed > by saddr, daddr, tos, etc. Even there are no TCP/UDP ports (yet?). > Ok. > > Now, If I understand the whole topic, in order to fix my problem, I need > > to: > > > > - NAT everything on the FW itself > > This is a solution Yes, but unfortunately, and regarding my current networks architecture, this will require massive changes in the firewall ACLs / routing rules. Currently, this is not an option. > > > - or disable NAT on both gateways, and ensure that routing is done > > properly > > Not sure what you mean. But you can run multipath routes > on the both gateways. Then the internal hosts can select any of > the gateways as default (or to use alternative default gateways). Each gateway only have one route to the remote network. So I don't see where the multipath routing will send packets to, except maybe to the other gateway. But this may become a big problem, in case all tunnels to the remote networks are down; this will create a routing loop. > Another solution is your apps not to change the TOS, it can be > changed at the border gateways (if useful at all). > Actually, and thanks to you, now I know where the problem is. Considering that doing NAT on the firewall itself is not possible, and that disabling NAT on both gateways will quickly become a big headache, because of the current networks setup, I think of adding another layout of NAT on the final gateway. Eg, something like: LAN --> FW w/ multipath-routing || Gateway1 Gateway2 | (NAT) | (NAT) || Remote Gateway (other side) | (NAT) | Remote Network This'll ensure that all packets arriving to the remove network comes from the same IP address, whatever path they used. What do you think of such setup ? Is it likely to work ? Thanks again. Regards, Vincent. > > Am I right ? I'm becomming a bit lost, here :-\ > > > > Many Thanks for your time. > > Vincent. > > Regards > > -- > Julian Anastasov <[EMAIL PROTECTED]> > > ___ > LARTC mailing list / [EMAIL PROTECTED] > http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/ -- Vincent Jaussaud Kelkoo.com Security Manager email: [EMAIL PROTECTED] "The UNIX philosophy is to design small tools that do one thing, and do it well." ___ LARTC mailing list / [EMAIL PROTECTED] http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
Re: Re: [LARTC] multipath routing problem [Shorter version] - Helpstill needed :-)
Hello, On 25 Oct 2002, Vincent Jaussaud wrote: > But traffic is NAT-ed after multipath routing occurs ! > Eg, the box which do multipath routing do not NAT traffic; traffic get > NAT-ed when leaving the gateways: > > LAN --> FW w/ multipath-routing > || > Gateway1 Gateway2 > | (NAT) | (NAT) > || > Remote Network > > Packets reach the Remote Network using one of the Gateway NAT-ed IP, so > that when packets come back they should use the proper return path. Am I > wrong ? Now I see, then the TOS is a big problem for you. May be your problem will be solved if TOS is not a routing key but it does not sound as a thing that is easy to fix in kernel. > Hmmm... Then this is where the problem is. So, if I understand > correctly, packets coming from a single TCP connections will use any of > the multipath route _if_ they are not NAT-ed ? Isn't the routing cache Yes, traffic A->B with TOS=XXX will use one path, traffic A->B with TOS=YYY can use different path. > suppose to ensure that every packets coming from a single connection use > the same path ? The routing does not cache connections but route resolutions keyed by saddr, daddr, tos, etc. Even there are no TCP/UDP ports (yet?). > Now, If I understand the whole topic, in order to fix my problem, I need > to: > > - NAT everything on the FW itself This is a solution > - or disable NAT on both gateways, and ensure that routing is done > properly Not sure what you mean. But you can run multipath routes on the both gateways. Then the internal hosts can select any of the gateways as default (or to use alternative default gateways). Another solution is your apps not to change the TOS, it can be changed at the border gateways (if useful at all). > Am I right ? I'm becomming a bit lost, here :-\ > > Many Thanks for your time. > Vincent. Regards -- Julian Anastasov <[EMAIL PROTECTED]> ___ LARTC mailing list / [EMAIL PROTECTED] http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
Re: Re: [LARTC] multipath routing problem [Shorter version] - Helpstill needed :-)
On Fri, 2002-10-25 at 16:55, Julian Anastasov wrote: > > Hello, Hi julian ! > > On 25 Oct 2002, Vincent Jaussaud wrote: > > > > ssh tends to play with TOS fields (and rightly so). Routing is keyed to the > > > *triple* (src, dst, tos), something that most people (including me) normally > > > forget. However, in this particular case that may be the reason for your > > > ssh's breaking. > > > > > Hmm... that's really interesting. Thanks for the pointer. I remember now > > that I've read something regarding SSH & TOS field some days ago. If I'm > > right, it use the Minimum Delay TOS value. > > > > Now, how am I suppose to deal with this TOS issue ? What TOS value > > should do the trick ? > > In theory, you should not reach multipath route for > traffic that is already NAT-ed. May be you have to fix your > routes. The TOS field plays in the input routing performed > on forwarding and traffic between two public IP addresses can select > different nexthop if the TOS is different or if the routing > cache is somehow flushed (on route/address add/del, expiration). But traffic is NAT-ed after multipath routing occurs ! Eg, the box which do multipath routing do not NAT traffic; traffic get NAT-ed when leaving the gateways: LAN --> FW w/ multipath-routing || Gateway1 Gateway2 | (NAT) | (NAT) || Remote Network Packets reach the Remote Network using one of the Gateway NAT-ed IP, so that when packets come back they should use the proper return path. Am I wrong ? > > > I'm using a 2.2 kernel with ipchains. > > > > > The reason for FTP breaking possibly has to do with packets for > > > the control connection going out the one gateway and for the data going > > > out the other... but this is speculation on my part. > > > > That sounds wise. However, routes are suppose to be cached using the src > > IP field as well (If I'm not mistaken), so that every packets coming > > from a particular IP are likely to take the same route than the others. > > Am I wrong ? > > Yes, TOS is a routing key just like SADDR and DADDR. > By using multipath route between 2 IP addresses you agree that > the packets can _safely_ choose any of the paths. When using > two or more ISPs you simply can't do this if the ISPs have > source spoofing disabled. In such cases only the traffic that > is NAT-ed from your box has the right to use the multipath route. > This is a key requirement for the patches you are using. Once > the NAT connections are established they don't hit multipath > route. > Hmmm... Then this is where the problem is. So, if I understand correctly, packets coming from a single TCP connections will use any of the multipath route _if_ they are not NAT-ed ? Isn't the routing cache suppose to ensure that every packets coming from a single connection use the same path ? Now, If I understand the whole topic, in order to fix my problem, I need to: - NAT everything on the FW itself - or disable NAT on both gateways, and ensure that routing is done properly Am I right ? I'm becomming a bit lost, here :-\ Many Thanks for your time. Vincent. > > A BIG Thanks for your reply :-) > > Cheers, > > Vincent. > > Regards > > -- > Julian Anastasov <[EMAIL PROTECTED]> > > ___ > LARTC mailing list / [EMAIL PROTECTED] > http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/ -- Vincent Jaussaud Kelkoo.com Security Manager email: [EMAIL PROTECTED] "The UNIX philosophy is to design small tools that do one thing, and do it well." ___ LARTC mailing list / [EMAIL PROTECTED] http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
Re: Re: [LARTC] multipath routing problem [Shorter version] - Helpstill needed :-)
Hello, On 25 Oct 2002, Vincent Jaussaud wrote: > > ssh tends to play with TOS fields (and rightly so). Routing is keyed to the > > *triple* (src, dst, tos), something that most people (including me) normally > > forget. However, in this particular case that may be the reason for your > > ssh's breaking. > > > Hmm... that's really interesting. Thanks for the pointer. I remember now > that I've read something regarding SSH & TOS field some days ago. If I'm > right, it use the Minimum Delay TOS value. > > Now, how am I suppose to deal with this TOS issue ? What TOS value > should do the trick ? In theory, you should not reach multipath route for traffic that is already NAT-ed. May be you have to fix your routes. The TOS field plays in the input routing performed on forwarding and traffic between two public IP addresses can select different nexthop if the TOS is different or if the routing cache is somehow flushed (on route/address add/del, expiration). > I'm using a 2.2 kernel with ipchains. > > > The reason for FTP breaking possibly has to do with packets for > > the control connection going out the one gateway and for the data going > > out the other... but this is speculation on my part. > > That sounds wise. However, routes are suppose to be cached using the src > IP field as well (If I'm not mistaken), so that every packets coming > from a particular IP are likely to take the same route than the others. > Am I wrong ? Yes, TOS is a routing key just like SADDR and DADDR. By using multipath route between 2 IP addresses you agree that the packets can _safely_ choose any of the paths. When using two or more ISPs you simply can't do this if the ISPs have source spoofing disabled. In such cases only the traffic that is NAT-ed from your box has the right to use the multipath route. This is a key requirement for the patches you are using. Once the NAT connections are established they don't hit multipath route. > A BIG Thanks for your reply :-) > Cheers, > Vincent. Regards -- Julian Anastasov <[EMAIL PROTECTED]> ___ LARTC mailing list / [EMAIL PROTECTED] http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
Re: Re: [LARTC] multipath routing problem [Shorter version] - Helpstill needed :-)
On Fri, 2002-10-25 at 16:24, Arthur van Leeuwen wrote: > On 25 Oct 2002, Vincent Jaussaud wrote: > > > When only one gateway is used to reach remote networks, everything is > > working just fine. (Whatever gateway we choose to use) > > Whenever we attempt to activate multipath routing over both gateways, > > then SSH don't work anymore. We can ping, traceroute, telnet, ... but > > not SSH nor FTP (PASV). > > ssh tends to play with TOS fields (and rightly so). Routing is keyed to the > *triple* (src, dst, tos), something that most people (including me) normally > forget. However, in this particular case that may be the reason for your > ssh's breaking. > Hmm... that's really interesting. Thanks for the pointer. I remember now that I've read something regarding SSH & TOS field some days ago. If I'm right, it use the Minimum Delay TOS value. Now, how am I suppose to deal with this TOS issue ? What TOS value should do the trick ? I'm using a 2.2 kernel with ipchains. > The reason for FTP breaking possibly has to do with packets for > the control connection going out the one gateway and for the data going > out the other... but this is speculation on my part. That sounds wise. However, routes are suppose to be cached using the src IP field as well (If I'm not mistaken), so that every packets coming from a particular IP are likely to take the same route than the others. Am I wrong ? A BIG Thanks for your reply :-) Cheers, Vincent. > > Doei, Arthur. > > -- > /\/ | [EMAIL PROTECTED] | Work like you don't need the money > /__\ / | A friend is someone with whom | Love like you have never been hurt > /\/__ | you can dare to be yourself | Dance like there's nobody watching -- Vincent Jaussaud Kelkoo.com Security Manager email: [EMAIL PROTECTED] "The UNIX philosophy is to design small tools that do one thing, and do it well." ___ LARTC mailing list / [EMAIL PROTECTED] http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
Re: [LARTC] multipath routing problem [Shorter version] - Help stillneeded :-)
On 25 Oct 2002, Vincent Jaussaud wrote: > When only one gateway is used to reach remote networks, everything is > working just fine. (Whatever gateway we choose to use) > Whenever we attempt to activate multipath routing over both gateways, > then SSH don't work anymore. We can ping, traceroute, telnet, ... but > not SSH nor FTP (PASV). ssh tends to play with TOS fields (and rightly so). Routing is keyed to the *triple* (src, dst, tos), something that most people (including me) normally forget. However, in this particular case that may be the reason for your ssh's breaking. The reason for FTP breaking possibly has to do with packets for the control connection going out the one gateway and for the data going out the other... but this is speculation on my part. Doei, Arthur. -- /\/ | [EMAIL PROTECTED] | Work like you don't need the money /__\ / | A friend is someone with whom | Love like you have never been hurt /\/__ | you can dare to be yourself | Dance like there's nobody watching ___ LARTC mailing list / [EMAIL PROTECTED] http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
