Re: [LARTC] Problem with Load Balancing
The second connections will problably gets routed though the same link because of route cache I think. []s. William T Mullaney wrote: Well, if you had a download manager and the system at the other side allowed you to start your transfers in the middle of the file (which isn't out of the question) that could potentially work. The problem is that as far as I see, there's nothing to force the second connection onto the second line. It's been kind of a crap shoot of what line gets more information. In theory you could start the first download stream (and it's routed to ISP A), then perhaps your email client goes out to check your POP account, so that goes over ISP B. The next connection, the second stream, now goes out over ISP B again. Honestly I don't know exactly how the equalize command for ip route works, though I would think it says to always use the "less used" connection (perhaps on PPS, BPS, % use, whatever, on a per second, 30 second, minute average?), but in my experience that and the weight options don't ever get you exactly 50/50 (or whatever you specify) traffic. Things like bit torrent would probably perform better because there are (possibly) many streams for each file, as would having 50 people downloading files vs one. It seems to be just like rolling dice, if you only roll twice you might get two evens or two odds, but if you roll tons of times, you should tend to get a more even distribution. -Will -Original Message- From: Raj Mathur [mailto:[EMAIL PROTECTED] Sent: Sunday, September 24, 2006 2:49 PM To: lartc@mailman.ds9a.nl Subject: RE: [LARTC] Problem with Load Balancing >>>>> "William" == William T Mullaney writes: William> To my knowledge, there is no way to download one file William> from two different connections connected to two different William> ISPs at the same time. If you are running BGP then you William> might be able to load balance across the two links, but William> that would require your upstream providers to allow you William> to use it, and possibly the purchase of a public AS William> number an IP address space depending on the setup. If William> you are doing NAT past this link (IE both of your lines William> go two the same ISP and same address blocks, but they William> want to give you 2x 10mb links for 20mb total), then you William> can look at doing load balancing on layer 2 (Fast William> EtherChannel, bonding, Link Aggregate Groups, whatever), William> or creating 2 PPP style links between the computers and William> using a routing protocol like OSPF, EIGRP (but not on William> Linux) or something. I believe OSPF does equal cost load William> balancing, BGP and EIGRP can, I think, do unequal cost William> load balancing. But either way, I don't think that's the William> solution in your case. William> The only other option I can think of would be some sort William> of software that sends every other packet to a different William> IP or something, which would need to run at the end you William> are downloading at or maybe at your ISPs, but I can't William> think of anything like that. Wouldn't some download manager software that splits the file up into multiple simultaneous downloads do the trick? Agreed, not a single download across multiple ISPs, but definitely a single file across multiple ISPs. Regards, -- Raju -- Raj Mathur[EMAIL PROTECTED] http://kandalaya.org/ GPG: 78D4 FC67 367F 40E2 0DD5 0FEF C968 D0EF CC68 D17F It is the mind that moves ___ LARTC mailing list LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc __ This email has been scanned by the MessageLabs Email Security System. For more information please visit http://www.messagelabs.com/email __ ___ LARTC mailing list LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc ___ LARTC mailing list LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc
RE: [LARTC] Problem with Load Balancing
Title: RE: [LARTC] Problem with Load Balancing Well, if you had a download manager and the system at the other side allowed you to start your transfers in the middle of the file (which isn't out of the question) that could potentially work. The problem is that as far as I see, there's nothing to force the second connection onto the second line. It's been kind of a crap shoot of what line gets more information. In theory you could start the first download stream (and it's routed to ISP A), then perhaps your email client goes out to check your POP account, so that goes over ISP B. The next connection, the second stream, now goes out over ISP B again. Honestly I don't know exactly how the equalize command for ip route works, though I would think it says to always use the "less used" connection (perhaps on PPS, BPS, % use, whatever, on a per second, 30 second, minute average?), but in my experience that and the weight options don't ever get you exactly 50/50 (or whatever you specify) traffic. Things like bit torrent would probably perform better because there are (possibly) many streams for each file, as would having 50 people downloading files vs one. It seems to be just like rolling dice, if you only roll twice you might get two evens or two odds, but if you roll tons of times, you should tend to get a more even distribution. -Will -Original Message- From: Raj Mathur [mailto:[EMAIL PROTECTED]] Sent: Sunday, September 24, 2006 2:49 PM To: lartc@mailman.ds9a.nl Subject: RE: [LARTC] Problem with Load Balancing >>>>> "William" == William T Mullaney writes: William> To my knowledge, there is no way to download one file William> from two different connections connected to two different William> ISPs at the same time. If you are running BGP then you William> might be able to load balance across the two links, but William> that would require your upstream providers to allow you William> to use it, and possibly the purchase of a public AS William> number an IP address space depending on the setup. If William> you are doing NAT past this link (IE both of your lines William> go two the same ISP and same address blocks, but they William> want to give you 2x 10mb links for 20mb total), then you William> can look at doing load balancing on layer 2 (Fast William> EtherChannel, bonding, Link Aggregate Groups, whatever), William> or creating 2 PPP style links between the computers and William> using a routing protocol like OSPF, EIGRP (but not on William> Linux) or something. I believe OSPF does equal cost load William> balancing, BGP and EIGRP can, I think, do unequal cost William> load balancing. But either way, I don't think that's the William> solution in your case. William> The only other option I can think of would be some sort William> of software that sends every other packet to a different William> IP or something, which would need to run at the end you William> are downloading at or maybe at your ISPs, but I can't William> think of anything like that. Wouldn't some download manager software that splits the file up into multiple simultaneous downloads do the trick? Agreed, not a single download across multiple ISPs, but definitely a single file across multiple ISPs. Regards, -- Raju -- Raj Mathur [EMAIL PROTECTED] http://kandalaya.org/ GPG: 78D4 FC67 367F 40E2 0DD5 0FEF C968 D0EF CC68 D17F It is the mind that moves ___ LARTC mailing list LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc __ This email has been scanned by the MessageLabs Email Security System. For more information please visit http://www.messagelabs.com/email __ ___ LARTC mailing list LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc
RE: [LARTC] Problem with Load Balancing
> "William" == William T Mullaney writes: William> To my knowledge, there is no way to download one file William> from two different connections connected to two different William> ISPs at the same time. If you are running BGP then you William> might be able to load balance across the two links, but William> that would require your upstream providers to allow you William> to use it, and possibly the purchase of a public AS William> number an IP address space depending on the setup. If William> you are doing NAT past this link (IE both of your lines William> go two the same ISP and same address blocks, but they William> want to give you 2x 10mb links for 20mb total), then you William> can look at doing load balancing on layer 2 (Fast William> EtherChannel, bonding, Link Aggregate Groups, whatever), William> or creating 2 PPP style links between the computers and William> using a routing protocol like OSPF, EIGRP (but not on William> Linux) or something. I believe OSPF does equal cost load William> balancing, BGP and EIGRP can, I think, do unequal cost William> load balancing. But either way, I don't think that's the William> solution in your case. William> The only other option I can think of would be some sort William> of software that sends every other packet to a different William> IP or something, which would need to run at the end you William> are downloading at or maybe at your ISPs, but I can't William> think of anything like that. Wouldn't some download manager software that splits the file up into multiple simultaneous downloads do the trick? Agreed, not a single download across multiple ISPs, but definitely a single file across multiple ISPs. Regards, -- Raju -- Raj Mathur[EMAIL PROTECTED] http://kandalaya.org/ GPG: 78D4 FC67 367F 40E2 0DD5 0FEF C968 D0EF CC68 D17F It is the mind that moves ___ LARTC mailing list LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc
RE: [LARTC] Problem with Load Balancing
Title: RE: [LARTC] Problem with Load Balancing To my knowledge, there is no way to download one file from two different connections connected to two different ISPs at the same time. If you are running BGP then you might be able to load balance across the two links, but that would require your upstream providers to allow you to use it, and possibly the purchase of a public AS number an IP address space depending on the setup. If you are doing NAT past this link (IE both of your lines go two the same ISP and same address blocks, but they want to give you 2x 10mb links for 20mb total), then you can look at doing load balancing on layer 2 (Fast EtherChannel, bonding, Link Aggregate Groups, whatever), or creating 2 PPP style links between the computers and using a routing protocol like OSPF, EIGRP (but not on Linux) or something. I believe OSPF does equal cost load balancing, BGP and EIGRP can, I think, do unequal cost load balancing. But either way, I don't think that's the solution in your case. The only other option I can think of would be some sort of software that sends every other packet to a different IP or something, which would need to run at the end you are downloading at or maybe at your ISPs, but I can't think of anything like that. -Will -Original Message- From: Vladimir Burciaga Aguilar [mailto:[EMAIL PROTECTED]] Sent: Monday, September 18, 2006 12:09 PM To: lartc@mailman.ds9a.nl Subject: RE: [LARTC] Problem with Load Balancing >We have also set up a somewhat similar method of load balancing. Our >traffic is never a 50-50 split (well 3:2 is how we have it set, but it >doesn't always get close to that), but as the load picks up, it tends to be >closer to the actual amount. [snip] ___ LARTC mailing list LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc
RE: [LARTC] Problem with Load Balancing
We have also set up a somewhat similar method of load balancing. Our traffic is never a 50-50 split (well 3:2 is how we have it set, but it doesn't always get close to that), but as the load picks up, it tends to be closer to the actual amount. Well, then there is not much to do about this. Dead gateway detection has never worked for us, and one day I'll probably bother other members of the LARTC group to get some help, but the method that we use is to check the output of the ip neighbor command. Basically, if our two ISPs are 10.1.1.254 and 10.2.2.254, we run a bash script via cron every minute that does a call something like: ETH1 = ip neigh 10.1.1.254 | egrep "REACHABLE|DELAY|PROBE|STALE" -c ETH2 = ip neigh 10.2.2.254 | egrep "REACHABLE|DELAY|PROBE|STALE" -c The neighbor system basically monitors ARP and if it sees a message leave an interface without a reply after something like 3-5 seconds, it moves the interface to DELAY, after another few seconds it moves to PROBE and does an active arp request, and if that fails to work in a few seconds, it becomes INCOMPLETE or FAILED or just simply isn't listed. If no data is sent either way for a while, the entry can be marked STALE or removed. With the above lines, we get a 1 in the ETH1 or ETH2 variables if the next neighbor is up, and a 0 if not. From there you can use some if scripts to detect if both are up, or if only one is up, which one. In our case, if both are up we clear the default route and then make it something like ip route add default nexthop via 10.1.1.254 dev eth1 weight 1 \ nexthop via 10.2.2.254 dev eth2 weight 1 and if only one is up we clear it and make it : ip route add default nexthop via 10.1.1.254 dev eth1 or ip route add default nexthop via 10.2.2.254 dev eth2 Ok, William, this looks like what I'm looking for. I'm going to test it and tell you how it works for us. By the way, about the download of a single file between the two conections, do you know if there is a way to do it? Thanks for your help and time and sorry for the delay! _ Prodigy/MSN Spaces: Tu espacio en la red http://spaces.msn.com/ ___ LARTC mailing list LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc
RE: [LARTC] Problem with Load Balancing
Title: RE: [LARTC] Problem with Load Balancing Vlad, We have also set up a somewhat similar method of load balancing. Our traffic is never a 50-50 split (well 3:2 is how we have it set, but it doesn't always get close to that), but as the load picks up, it tends to be closer to the actual amount. Dead gateway detection has never worked for us, and one day I'll probably bother other members of the LARTC group to get some help, but the method that we use is to check the output of the ip neighbor command. Basically, if our two ISPs are 10.1.1.254 and 10.2.2.254, we run a bash script via cron every minute that does a call something like: ETH1 = ip neigh 10.1.1.254 | egrep "REACHABLE|DELAY|PROBE|STALE" -c ETH2 = ip neigh 10.2.2.254 | egrep "REACHABLE|DELAY|PROBE|STALE" -c The neighbor system basically monitors ARP and if it sees a message leave an interface without a reply after something like 3-5 seconds, it moves the interface to DELAY, after another few seconds it moves to PROBE and does an active arp request, and if that fails to work in a few seconds, it becomes INCOMPLETE or FAILED or just simply isn't listed. If no data is sent either way for a while, the entry can be marked STALE or removed. With the above lines, we get a 1 in the ETH1 or ETH2 variables if the next neighbor is up, and a 0 if not. From there you can use some if scripts to detect if both are up, or if only one is up, which one. In our case, if both are up we clear the default route and then make it something like ip route add default nexthop via 10.1.1.254 dev eth1 weight 1 \ nexthop via 10.2.2.254 dev eth2 weight 1 and if only one is up we clear it and make it : ip route add default nexthop via 10.1.1.254 dev eth1 or ip route add default nexthop via 10.2.2.254 dev eth2 With some additional scripting we can allow this to be overridden, we can set the link to prefer using only one line, but switch to the other if the preferred line fails, and to take input from programs like Nagios to auto-prefer one line or another if ping times get high, etc. In addition, the script remembers the state it was in (so that it only changes the routing table when needed), controls DNS, can flush the DNS cache, and reports status back to Nagios. Once I get all the bugs out and some documentation, I'd be happy to post it to the news group, though you or anyone else can send me an email if you would like to take a look at it before then. In practice, this method usually detects and adjusts outbound connections quickly without user intervention; DNS changes with short TTLS take care of inbound connections. Just be careful... if you don't have something sending traffic out to your upstream routers (and back) every few minutes, the entry in your ARP table can potentially be removed and thus cause your system to think an unused gateway has failed, or that a recovered gateway is still down. This could be checked with a quick "if ip neigh test fails, ping neighbor 5 times, then test again before making decisions". Running an uptime monitor that pings or does something else to/through the gateway (regardless of default route) also takes care of this. -Will -Original Message- From: Vladimir Burciaga Aguilar [mailto:[EMAIL PROTECTED]] Sent: Thursday, September 14, 2006 10:25 PM To: lartc@mailman.ds9a.nl Subject: [LARTC] Problem with Load Balancing Hi everybody! I'm trying to implement the load balancing for a LAN with two ISPs. I've installed a Suse Linux Enterpise Server 9 with iproute2 for that porpouse. The server have two NICs, one of them is for both the LAN and ISP 1. I've setup both NICs with YAST (if I use ip for this, then the whole thing doesn't work!) and execute the following commands to setup the routing tables: ip route flush cache ip route flush default ip route flush table 1 ip route flush table 2 [snip] ___ LARTC mailing list LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc
[LARTC] Problem with Load Balancing
Hi everybody! I'm trying to implement the load balancing for a LAN with two ISPs. I've installed a Suse Linux Enterpise Server 9 with iproute2 for that porpouse. The server have two NICs, one of them is for both the LAN and ISP 1. I've setup both NICs with YAST (if I use ip for this, then the whole thing doesn't work!) and execute the following commands to setup the routing tables: ip route flush cache ip route flush default ip route flush table 1 ip route flush table 2 ip route add 10.1.254.0/24 dev eth0 src 10.1.254.251 ip route add 10.1.1.0/24 dev eth1 src 10.1.1.200 ip route show table main | while read ROUTE ; do ip route add table 1 $ROUTE ; done ip route show table main | while read ROUTE ; do ip route add table 2 $ROUTE ; done ip route add table 1 default via 10.1.254.254 ip route add table 2 default via 10.1.1.254 ip rule add from 10.1.254.251 table 1 ip rule add from 10.1.1.200 table 2 ip route add default equalize nexthop via 10.1.254.254 dev eth0 weight 1 \ nexthop via 10.1.1.254 dev eth1 weight 1 (All of this came from http://linux.lcampino.cl/wiki/index.php/Balanceo_de_Carga#Configurando_la_red (in Spanish)). At this point, things seems to work fine. I browse in the server and watch with iptraf that both NICs have traffic, but not at 50%-50% rate. Now I need to implement the following: - Make load balancing the nearest to 50%-50% rate (Both Internet connections have equal bandwith). - When one of the links goes down, all the traffic redirects to the other inmediately and automatically. Also, when the link is up again, the load balancing restart too. - If posible, when downloading a single big file (i.e. *.exe, *.iso), the packets come from both connections. I've read a lot about this, but I'm still confuse and very very lost...! :-))) I'm not sure if I need to adjust some kernel paramters and rebuit it, execute another sequence of commands, apply a kernel patch, etc. I really apreciate if someone could bring me some light in all this, or tell me what documentation, web page or patch do I need to use. Thanks in advanced for all your time and recomendations! P.D. Sorry for my English! _ Platica con tus amigos en linea con MSN Messenger http://messenger.t1msn.com.mx/ ___ LARTC mailing list LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc
Re: [LARTC] Problem With Load Balancing the Links
Hello, On Mon, 23 Dec 2002, hare ram wrote: > as you have mentioned in the documents, the last time should be default, but > why iam getting 253 here Table "default" (253) is always there but we don't use it because we want table "main" before 201 and 202. > 0: from all lookup local > 50: from all lookup main > 201:from 202.63.96.0/24 lookup 201 > 202:from 202.63.111.150/24 lookup 202 > 222:from all lookup 222 > 32766: from all lookup main > 32767: from all lookup 253 > > and iam not able to Ping or go out from My gateway IP > i can only Ping my Gateway IP's Check the settings again. > thanks > hare Regards -- Julian Anastasov <[EMAIL PROTECTED]> ___ LARTC mailing list / [EMAIL PROTECTED] http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
Re: [LARTC] Problem With Load Balancing the Links
Hi thanks, for the suggestion let me look and install this latest one as you have mentioned in the documents, the last time should be default, but why iam getting 253 here 0: from all lookup local 50: from all lookup main 201:from 202.63.96.0/24 lookup 201 202:from 202.63.111.150/24 lookup 202 222:from all lookup 222 32766: from all lookup main 32767: from all lookup 253 and iam not able to Ping or go out from My gateway IP i can only Ping my Gateway IP's thanks hare - Original Message - From: "Julian Anastasov" <[EMAIL PROTECTED]> To: "hare ram" <[EMAIL PROTECTED]> Cc: <[EMAIL PROTECTED]> Sent: Monday, December 23, 2002 3:57 PM Subject: Re: [LARTC] Problem With Load Balancing the Links > > Hello, > > On Mon, 23 Dec 2002, hare ram wrote: > > > iam using redhat 8.0 with Latest Kernel > > 2.4.18-18.0 > > with iproute > > [root@pdn root]# rpm -q iproute > > iproute-2.4.7-5 > > I'm not sure if 2.4.7-5 covers the desired version but here it is: > > ftp://ftp.inr.ac.ru/ip-routing/iproute2-2.4.7-now-ss020116-try.tar.gz > > > is this Corect or i need to still upgrade > > if so, i did not find latest than this > > You can always list the above directory for latest tarballs > > > thanks > > hare > > Regards > > -- > Julian Anastasov <[EMAIL PROTECTED]> > > ___ > LARTC mailing list / [EMAIL PROTECTED] > http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/ > ___ LARTC mailing list / [EMAIL PROTECTED] http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
Re: [LARTC] Problem With Load Balancing the Links
Hello, On Mon, 23 Dec 2002, hare ram wrote: > iam using redhat 8.0 with Latest Kernel > 2.4.18-18.0 > with iproute > [root@pdn root]# rpm -q iproute > iproute-2.4.7-5 I'm not sure if 2.4.7-5 covers the desired version but here it is: ftp://ftp.inr.ac.ru/ip-routing/iproute2-2.4.7-now-ss020116-try.tar.gz > is this Corect or i need to still upgrade > if so, i did not find latest than this You can always list the above directory for latest tarballs > thanks > hare Regards -- Julian Anastasov <[EMAIL PROTECTED]> ___ LARTC mailing list / [EMAIL PROTECTED] http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
Re: [LARTC] Problem With Load Balancing the Links
Hi iam using redhat 8.0 with Latest Kernel 2.4.18-18.0 with iproute [root@pdn root]# rpm -q iproute iproute-2.4.7-5 is this Corect or i need to still upgrade if so, i did not find latest than this thanks hare - Original Message - From: "Julian Anastasov" <[EMAIL PROTECTED]> To: "hare ram" <[EMAIL PROTECTED]> Cc: <[EMAIL PROTECTED]> Sent: Sunday, December 22, 2002 10:12 PM Subject: Re: [LARTC] Problem With Load Balancing the Links > > Hello, > > On Sun, 22 Dec 2002, hare ram wrote: > > > iam following the Julian Nano.txt, iam able to setup all > > but when i check the following command > > > > [root@pdn root]# ip route list table 222 > > default proto static > > nexthop via 202.x.x.254 dev eth0 weight 256 dead onlink pervasive > > nexthop via 203.x.x.17 dev eth1 weight 1 > > This smells like one iproute utility bug, just upgrade to iproute2 > from Year 2002 which fixes similar problem. > > Regards > > -- > Julian Anastasov <[EMAIL PROTECTED]> > > ___ > LARTC mailing list / [EMAIL PROTECTED] > http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/ > ___ LARTC mailing list / [EMAIL PROTECTED] http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
Re: [LARTC] Problem With Load Balancing the Links
Hello, On Sun, 22 Dec 2002, hare ram wrote: > iam following the Julian Nano.txt, iam able to setup all > but when i check the following command > > [root@pdn root]# ip route list table 222 > default proto static > nexthop via 202.x.x.254 dev eth0 weight 256 dead onlink pervasive > nexthop via 203.x.x.17 dev eth1 weight 1 This smells like one iproute utility bug, just upgrade to iproute2 from Year 2002 which fixes similar problem. Regards -- Julian Anastasov <[EMAIL PROTECTED]> ___ LARTC mailing list / [EMAIL PROTECTED] http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
[LARTC] Problem With Load Balancing the Links
Hi iam following the Julian Nano.txt, iam able to setup all but when i check the following command [root@pdn root]# ip route list table 222 default proto static nexthop via 202.x.x.254 dev eth0 weight 256 dead onlink pervasive nexthop via 203.x.x.17 dev eth1 weight 1 why iam getting this dead onlink pervasive, the document says i should get weight 1 what ism doing wrong with this setup some one suggest me thanks hare ___ LARTC mailing list / [EMAIL PROTECTED] http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
