[LARTC] another multipath problem
Hi, I've been trying to setup 2 DSL links in the office, both on the same firewall/router/gateway. the firewall is running kernel 2.6.11 (custom built on debian). this is the network diagram: 150.101.124.y/29 (internode's gw) | | |150.101.124.x/29 (eth2) +-+ | |192.168.10.1/24 (eth1) | FW| | | 192.168.10.0/24 (internal net) +-+ |203.100.236.x (ppp0)/61.8.x.x/24 (eth0) | | 203.9.190.y (pacific.net's gw) what I want to achieve: I have a bunch of ips on the 61.8.x.net/25 net, and they are routed via 203.9.190.y on one of our ISPs (pacific internet). this setup works fine, and I have heaps of NATed rules on the firewall to nat traffic to internal servers on the 192.168.10.0/24 network. We just got another DSL from another ISP, to have a few services running out of it. so we got 5 ips on the 150.101.124.net/29 net, and I'm trying to also nat them to internal servers. I want to have a few of the servers reachable from both ISPs. But more than that, I want to choose which of the servers should primarily go through a chosen link. I could manage to setup outgoing traffic from specific servers through internode's DSL, as the default is pacific internet. as you can see on my routing tables: * main routing table: [EMAIL PROTECTED]:~# ip route sh tab main 203.9.190.y dev ppp0 proto kernel scope link src 203.100.236.x 203.100.236.x dev ppp0 scope link src 203.100.236.x 150.101.124.net/29 dev eth2 proto kernel scope link src 150.101.124.x 192.168.10.0/25 dev eth1 scope link src 192.168.10.1 61.8.x.net/25 dev eth0 scope link 192.168.10.0/24 dev eth1 proto kernel scope link src 192.168.10.1 127.0.0.0/8 dev lo scope link default via 203.9.190.y dev ppp0 * internode routing table: [EMAIL PROTECTED]:~# ip route sh tab internode 203.100.236.x via 203.100.236.x dev ppp0 150.101.124.net/29 dev eth2 scope link src 150.101.124.x 61.8.x.net/25 dev eth0 scope link 192.168.10.0/24 via 192.168.10.1 dev eth1 127.0.0.0/8 dev lo scope link default via 150.101.124.y dev eth2 * pacificnet routing table: [EMAIL PROTECTED]:~# ip route sh tab pacificnet 203.9.190.y dev ppp0 scope link src 203.100.236.x 203.100.236.x dev ppp0 scope link src 203.100.236.x 61.8.x.net/25 dev ppp0 scope link src 203.100.236.x 192.168.10.0/24 via 192.168.10.1 dev eth1 127.0.0.0/8 dev lo scope link default via 203.9.190.190 dev ppp0 and on my routing rules: [EMAIL PROTECTED]:~# ip rule sh 0: from all lookup local 32744: from all to 192.168.10.20 lookup internode 32745: from 192.168.10.20 lookup internode 32762: from 150.101.124.178 lookup internode 32763: from 203.100.236.222 lookup pacificnet 32766: from all lookup main 32767: from all lookup default as an example, I'll use server 192.168.10.20. I could manage to make all traffic from that server go through internode, but I couldn't figure out a way to create a nat rule to access 192.168.10.20 from outside. I have the following iptables NAT rules, that should nat traffic to 150.101.124.x or 61.8.x.x on port 143 to port 22 on 192.168.10.20: Chain PREROUTING (policy ACCEPT) target prot opt source destination DNAT tcp -- 0.0.0.0/0150.101.124.x tcp dpt:143 to:192.168.10.20:22 DNAT tcp -- 0.0.0.0/061.8.x.x tcp dpt:143 to:192.168.10.20:22 Chain POSTROUTING (policy ACCEPT) target prot opt source destination SNAT all -- 192.168.10.200.0.0.0/0 to:150.101.124.x telneting to port 143 from an outside server on 61.8.x.x 143 works fine: [EMAIL PROTECTED]:~# telnet 61.8.29.31 143 Trying 61.8.29.31... Connected to 61.8.29.31. Escape character is '^]'. SSH-2.0-OpenSSH_3.8.1p1 Debian-8.sarge.4 but telneting to the same port on 150.101.124.x doesn't, it actually does, but is incredibly slow. don't know if it's clear, if not, let me know and I'll clarify. thanks in advance! cheers, Fernando ___ LARTC mailing list LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc
RE: [LARTC] Diffserv Research Network
Sorry about the link in previous mail correct link is http://discovery.bits-pilani.ac.in/gridone/Piyush-Gupta-Revised-Final-Report-Dissertatyon-Oct-2005.pdf -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Amit Vyas Sent: Monday, November 14, 2005 12:39 AM To: lartc@mailman.ds9a.nl Subject: [LARTC] Diffserv Research Network Hi all, We are a group of research students and working on IPv6 QoS aware Grid. >From our set of experiments we have a particular case: Suppose a host A want to initiate traffic of some kind to host B, where A and B are connected by two intermediate routers wherein there can be other connections from the router. Our needs are as follows 1. We assume that routers are Diffserv routers and provide us QoS but we will also assume that routers are not in our network boundaries. 2. The network will be working as Best Effort network until our application requests required end to end QoS. 3. Our application negotiates the QoS requirements and then network sets it up for us dynamically. Since we do not have any restrictions and we can also assume that routers are in our domain in case some disagrees over end to end QoS in Internet scenario. We want to setup this research network environment I would like to know 1. How can we build such Diffserv routers which can understand and provide us QoS (Signaling?)(We have experience of statically setting up QoS network environment using with tc scripts) 2. How can we use negotiate dynamically setting up of QoS? 3. The concept of Active networks where store and forward networks are transformed to store-compute and forward networks proposes that packets carry executable code with their data payload and its executed at designated active nodes. So is it possible to have routers as active nodes for the traffic from an authenticated sender? But then is it possible in core Internet routers? If not then what can be possible solutions and if yes how does one go about it? In case you are interested we have a Thesis report for previous work which can be accessed here http://discovery/gridone/Piyush-Gupta-Revised-Final-Report-Dissertatyon-Oct- 2005.pdf Thanks in advance. Amit Vyas ME CS. BITS Pilani. ___ LARTC mailing list LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc ___ LARTC mailing list LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc
[LARTC] Diffserv Research Network
Hi all, We are a group of research students and working on IPv6 QoS aware Grid. >From our set of experiments we have a particular case: Suppose a host A want to initiate traffic of some kind to host B, where A and B are connected by two intermediate routers wherein there can be other connections from the router. Our needs are as follows 1.We assume that routers are Diffserv routers and provide us QoS but we will also assume that routers are not in our network boundaries. 2.The network will be working as Best Effort network until our application requests required end to end QoS. 3.Our application negotiates the QoS requirements and then network sets it up for us dynamically. Since we do not have any restrictions and we can also assume that routers are in our domain in case some disagrees over end to end QoS in Internet scenario. We want to setup this research network environment I would like to know 1. How can we build such Diffserv routers which can understand and provide us QoS (Signaling?)(We have experience of statically setting up QoS network environment using with tc scripts) 2. How can we use negotiate dynamically setting up of QoS? 3. The concept of Active networks where store and forward networks are transformed to store-compute and forward networks proposes that packets carry executable code with their data payload and its executed at designated active nodes. So is it possible to have routers as active nodes for the traffic from an authenticated sender? But then is it possible in core Internet routers? If not then what can be possible solutions and if yes how does one go about it? In case you are interested we have a Thesis report for previous work which can be accessed here http://discovery/gridone/Piyush-Gupta-Revised-Final-Report-Dissertatyon-Oct- 2005.pdf Thanks in advance. Amit Vyas ME CS. BITS Pilani. ___ LARTC mailing list LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc
Re: [LARTC] Borrowing between HTB classes not working as expectd.
On 11/13/05, [EMAIL PROTECTED] <[EMAIL PROTECTED]> wrote: > Quoting Ryan Castellucci <[EMAIL PROTECTED]>: > > I did not mix these up. I'm using the 1:2 class for TCP and ICMP > > control packets, such as TCP acks which need an amount of bandwidth > > proportinate to the maximum download rate. > > There seems to be a misunderstanding of some kind. You say you're using > the 1:2 > class for control packets; but in the output you've sent, the 1:2 class is the > root HTB class, so it should be (indirectly) used for everything. Erp, I ment 1:3. > The only > classes you can use directly (that means classify packets to) are the leaf > classes (HTB classes which don't have any more children), in your setup that > would be one of the 1:3,356-361,612-617,869-873 leaf classes. > > Class 1:2 has a rate/ceil of 217kbit. Children of this class are 1:3 > (124/149), > 1:4 (128/243), 1:5 (102/243), and 1:6 (25/204). As I said before, the problem > is that the rates of these classes don't add up. These child classes added > together for example use 124+128+102+25=379kbit, although the parent provides > only 217kbit. Classes 1:4 and 1:5 in particular can borrow up to 243kbit each, > although the parent class can provide only 217kbit in total. So how exactly do > you expect the borrowing to work? Unless you have an understanding of > the inner > workings of HTB in great detail, the results of this setup are pretty much > unpredictable. > > The same problem can be found further down the tree; for example, the > class 1:4 > has a rate of 128kbit. Children of this class are 1:356-361, with a rate of > 128kbit each. Added together, they require a rate of 768kbit, but the parent > class only provides 128kbit (or it would if the parent class of this parent > class could provide as much). > > Same story with 1:5 and 1:6. > > The first thing you have to do is calculate the class rates so they add up > properly. Otherwise you will never get anywhere near a predictable borrowing > behaviour. I'll go though and make sure everything adds up, and try it again. -- Ryan Castellucci http://ryanc.org/ ___ LARTC mailing list LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc
Re: [LARTC] MSN keeps disconnecting with load balancing
Is it possible to increase the cached route timeout? Yes, I am using SNAT, will MASQUERADE help? Ryan Castellucci wrote: This problem is caused by the cached route to MSN expiring, and the kernel trying to route the existing connection over the other internet connection. If you're doing SNAT, this will result in the source IP address changing, and MSN will reset the connection. On 11/12/05, Corey Hickey <[EMAIL PROTECTED]> wrote: ro0ot wrote: Hi, I have the my gateway with load balancing traffic going out over two providers. Web browsing is fine...working great. But, my clients (office staff) complains that MSN keeps disconnecting (in 5 mins). Why? Do you mean MSN instant messenger? I've never used it, but I can give you a few generic steps to take when you want to figure out what's going wrong with a connection. Are you familiar with tcpdump and/or ethereal? 1. Go to the computer of a client who is complaining about disconnection. 2. ssh into your gateway and run: # tcpdump -i eth0 host 123.123.123.123 and port not ssh Change "eth0" to the inside interface and "123.123.123.123" to the IP address of your client. 3. See if tcpdump is catching lots and lots of packets. If it is, either stop programs on your clients machine that access the Internet or use more filters (like "and port not imaps"). 4. Once you're not catching lots of extraneous packets, kill tcpdump and run: # tcpdump -s 1500 -w log -i eth0 host 123.123.123.123 and port not ssh Include any other filters you have to use. 5. Have your client start up their program, and then sit there and wait. Don't do anything else that would send packets through the gateway; you don't want to clutter up the log. 6. See if the problem manifests. Most likely it won't, because that's just the way things are :) , but if it does you'll have a log. Kill tcpdump and examine the file with: # tcpdump -r log If you want more verbosity, use "-v", "-vv", or "-vvv". Or, if you want to use a gui, copy the log file to some machine with X11 and use: # ethereal -r log -Corey ___ LARTC mailing list LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc -- Ryan Castellucci http://ryanc.org/ ___ LARTC mailing list LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc ___ LARTC mailing list LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc
Re: [LARTC] MSN keeps disconnecting with load balancing
This is not a problem with linux it is a simple fact of IP/TCP and applications. A workaround is put the IP into a ipt_recent list then SNAT all that traffic the one way for a given time. There was a ipt_helper for MSN but I dont know where it is in development. I use the route taget in iptables rather than ip (sorry guys but you can just do so much funky stuff with it) So just to recap on my suggestion. Find the MSN traffic e.g. What ever port it uses for session initiation then put that ip into an ipt_recent list. The check that list before you snat. This will SNAT all the traffic from that IP to one interface for a certain amount of time that you set e.g. if it has not seen the packet within 600 secs clear it from the list. I use the for VoIP to multiple SNAT targets. On Mon, 2005-11-14 at 00:51 +0800, ro0ot wrote: > I have this in /etc/iproute2/rt_tables as below: - > > 216 https.out > 219 msn.out > > And, I have the below in my custom script: - > > $IPTABLES -t nat -A POSTROUTING -o eth3 -j SNAT --to-source 1.2.3.4 > $IPTABLES -t nat -A POSTROUTING -o eth4 -j SNAT --to-source 5.6.7.8 > > $IPTABLES -t mangle -A PREROUTING -i eth1 -p tcp --dport 443 -j MARK > --set-mark 16 > $IPTABLES -t mangle -A PREROUTING -i eth1 -p tcp --dport 1863 -j MARK > --set-mark 19 > > ip rule add fwmark 16 table https.out > ip rule add fwmark 19 table msn.out > > ip route add default via 1.2.3.4 dev eth3 table https.out > ip route add default via 1.2.3.4 dev eth3 table msn.out > > But, still I am facing complaints... > > Edmundo Carmona wrote: > > >We have exaclty the same problem in our load-balancing proxy. > > > >Remember that if you are load-balancing, traffic eventually will come > >out through another network interface,,, and hence, another source IP. > > > >The problem must be that the MSN service gets "confused" when he sees > >that the one session has changed source IPs... or maybe it's a IP > >stack problem and not related to the MSN specifically. Anybody can > >provide some more feedback on this? The IP session layer is supposed > >to keep sessions across changing IPs? > > > >Our "solution" was to tell the MSN clients to use a proxy server that > >has a single internet connection (separete from our main proxy server, > >which is the one loadbalancing). > > > >On 11/13/05, ro0ot <[EMAIL PROTECTED]> wrote: > > > > > >>Hi, > >> > >>I have the my gateway with load balancing traffic going out over two > >>providers. Web browsing is fine...working great. > >> > >>But, my clients (office staff) complains that MSN keeps disconnecting > >>(in 5 mins). Why? > >> > >>Please help me... > >> > >>Regards, > >>ro0ot > >> > >> > >> > >>___ > >>LARTC mailing list > >>LARTC@mailman.ds9a.nl > >>http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc > >> > >> > >> > > > > > > > > > > > > ___ > LARTC mailing list > LARTC@mailman.ds9a.nl > http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc ___ LARTC mailing list LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc
Re: [LARTC] MSN keeps disconnecting with load balancing
I have this in /etc/iproute2/rt_tables as below: - 216 https.out 219 msn.out And, I have the below in my custom script: - $IPTABLES -t nat -A POSTROUTING -o eth3 -j SNAT --to-source 1.2.3.4 $IPTABLES -t nat -A POSTROUTING -o eth4 -j SNAT --to-source 5.6.7.8 $IPTABLES -t mangle -A PREROUTING -i eth1 -p tcp --dport 443 -j MARK --set-mark 16 $IPTABLES -t mangle -A PREROUTING -i eth1 -p tcp --dport 1863 -j MARK --set-mark 19 ip rule add fwmark 16 table https.out ip rule add fwmark 19 table msn.out ip route add default via 1.2.3.4 dev eth3 table https.out ip route add default via 1.2.3.4 dev eth3 table msn.out But, still I am facing complaints... Edmundo Carmona wrote: We have exaclty the same problem in our load-balancing proxy. Remember that if you are load-balancing, traffic eventually will come out through another network interface,,, and hence, another source IP. The problem must be that the MSN service gets "confused" when he sees that the one session has changed source IPs... or maybe it's a IP stack problem and not related to the MSN specifically. Anybody can provide some more feedback on this? The IP session layer is supposed to keep sessions across changing IPs? Our "solution" was to tell the MSN clients to use a proxy server that has a single internet connection (separete from our main proxy server, which is the one loadbalancing). On 11/13/05, ro0ot <[EMAIL PROTECTED]> wrote: Hi, I have the my gateway with load balancing traffic going out over two providers. Web browsing is fine...working great. But, my clients (office staff) complains that MSN keeps disconnecting (in 5 mins). Why? Please help me... Regards, ro0ot ___ LARTC mailing list LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc ___ LARTC mailing list LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc
Again: Re: [LARTC] MSN keeps disconnecting with load balancing (fwd)
This 'MSN' is a web site? Im guessing it 'refresh'es every 5 minutes or so. They are proably testing cookies against the ip address they appear to be comming from. This is horribly wreckless of them if they arnt offering IPv6. Are they? They only way i have to remedy this problem is to get their IP range and bind it to the most stable connexion you have, defeating the load-balancing, almost, sorta, oh well... You can TRY sending to MSN up either pipe but using the same src address in both cases. some ISPs are really mean/useless and wont let you send from you own addresses if you dont lease them from the ISP. boo. this will ofcourse still bring the content all down one pipe still... upstream load balancing restored, downstream still skewed OR break the clients into two groups, those using pipe1 by default, those using pipe2 by default when talking to MSN. As long as both connextions stay up no one complains, if one goes down, half of them complain 'i got logged off'. Im sure you can rig a proxy on each upstream feed that conducts the signin process for them everytime. so much work... oh well best of luck. On Sun, 13 Nov 2005, ro0ot wrote: Date: Sun, 13 Nov 2005 12:21:37 +0800 From: ro0ot <[EMAIL PROTECTED]> To: lartc@mailman.ds9a.nl Subject: [LARTC] MSN keeps disconnecting with load balancing Hi, I have the my gateway with load balancing traffic going out over two providers. Web browsing is fine...working great. But, my clients (office staff) complains that MSN keeps disconnecting (in 5 mins). Why? Please help me... Regards, ro0ot ___ LARTC mailing list LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc Robin-David Hammond KB3IEN www.aresnyc.org. --===0492895792==-- ___ LARTC mailing list LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc
Re: [LARTC] Borrowing between HTB classes not working as expectd.
Quoting Ryan Castellucci <[EMAIL PROTECTED]>: I did not mix these up. I'm using the 1:2 class for TCP and ICMP control packets, such as TCP acks which need an amount of bandwidth proportinate to the maximum download rate. There seems to be a misunderstanding of some kind. You say you're using the 1:2 class for control packets; but in the output you've sent, the 1:2 class is the root HTB class, so it should be (indirectly) used for everything. The only classes you can use directly (that means classify packets to) are the leaf classes (HTB classes which don't have any more children), in your setup that would be one of the 1:3,356-361,612-617,869-873 leaf classes. Class 1:2 has a rate/ceil of 217kbit. Children of this class are 1:3 (124/149), 1:4 (128/243), 1:5 (102/243), and 1:6 (25/204). As I said before, the problem is that the rates of these classes don't add up. These child classes added together for example use 124+128+102+25=379kbit, although the parent provides only 217kbit. Classes 1:4 and 1:5 in particular can borrow up to 243kbit each, although the parent class can provide only 217kbit in total. So how exactly do you expect the borrowing to work? Unless you have an understanding of the inner workings of HTB in great detail, the results of this setup are pretty much unpredictable. The same problem can be found further down the tree; for example, the class 1:4 has a rate of 128kbit. Children of this class are 1:356-361, with a rate of 128kbit each. Added together, they require a rate of 768kbit, but the parent class only provides 128kbit (or it would if the parent class of this parent class could provide as much). Same story with 1:5 and 1:6. The first thing you have to do is calculate the class rates so they add up properly. Otherwise you will never get anywhere near a predictable borrowing behaviour. HTH Andreas Klauer ___ LARTC mailing list LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc
