Re: [Fwd: Re: 3 connections as one]
- Original Message - Subject: Re: [Fwd: Re: 3 connections as one] : Martes Wigglesworth wrote: : Forwarded Message : From: Martes Wigglesworth [EMAIL PROTECTED] : Reply-To: [EMAIL PROTECTED] : To: Andres Chavez [EMAIL PROTECTED] : Subject: Re: 3 connections as one : Date: Tue, 24 Jun 2008 16:34:04 -0400 : : I have been researching this issue for almost a month now, and what I : have found is that you can bind the ports together for outbound traffic, : and the same can be done for inbound traffic, the problem comes when you : try to get the inbound packets, or sessions to dispurse across the : load-balanced ports. I.E.: Who is on the other side of the multiple : DSL/Cable links to filter the traffic across the associated pipes so as : to balance the load, so to speak? : : It can be done, however, without an upstream, or maybe a vps that is : being used as an external gateway, you will not be able to get the : different session traffic to load balance across the multiple links, : when downloading. : : At least that seems to be the situation, without some nifty DNS tricks. : I have not seen how the appliances get around this, however, it took : me this long for either list that I was on, to even admitt that the : theory was not stupid, and to engage me in productive inquiry. : : : the usual way is to NAT traffic out though each interface : so that the internet is not aware that sessions from apparently : different places are actually the same.. : : : you can do the same with multiple NAT instances and some way to divide : up the load between interfaces.. I had write a patch, a long time before (probably in 4.x days - before libnat get the kernel bits) wich you can set two (fixed by patch at that time) alias address on natd. Another option has been added to natd, a number wich can be set from 0 to 100 to determine the use of the second alias address. This is intended to be used as %, so 50 should be read as 50/50% balanced link. So when a connection has to be established for the first time, the patch use the value of balance option to determine what alias address should be used for this new connection. The natd will use the default alias address or the optional alias address based on the balance set. So natd is generating new connections in two diferent IPs (for two diferent connections) based on a % value, wich allow the use of unequal bandwidth links. At that time the patch work like a charm and is very usefull (as set 0/100 disable the use of one link and 100/0 disable the use of the other link without change any other configuration). Ipfw should be configured to deliver each IP/network to the proper gateway. Due to the number of changes in recent libalias/natd the patch need to be rewrite and the only thing i am not happy (and IMHO should be revised is the number of connections that should be from 1 to any and not limited to two). This should be a simple task for a natd/libalias developer (not enough time for me). -luiz ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: [Fwd: Re: 3 connections as one]
You can do it for outgoing connections fairly easily using the NAT trick (with PF), but you can't really load balance multiple links without support from some outside entity. If one of the tunnels goes down you can fail-over but any pre-existing connections will die and have to be re-established on the remaining link(s). That generally works ok for TCP but is total hell for UDP (because the source address will suddenly 'change' on an existing 'connection' and often trigger security blocks or simply break the program in question when it does). I've got a DSL connection and a Cable internet connection at home now, having replaced the T1 I had had for many years. I tried using the NAT trick using PF for outgoing but was never happy with the results under max load (and my links are typically running at 100% 24x7). I wasn't able to get fail-over to work properly with PF at all... the network was actually less reliable instead of more reliable and using NAT meant I had very little control over port selection or reverse-IP. I eventually gave up and now just use my DSL line for all my normal traffic, and my cable link for my off-site backup traffic. -- I'm planning out a new solution, one that a friend of mine implemented with a portable class C he owns at a colo with a single link which I want to extend to multiple links. The idea is to chop off a subnet from the colo-routed class C and run it to the home box over multiple tunnels (one over COMCAST, one over DSL). I am going to run all the tunnels through a single user program on my router box and backhaul it into a TUN interface (using PF on the TUN interface for QOS), and have the user program do all the load balancing and fail-over. Since the whole mess is routing a single subnet, no NAT tricks are needed and packets can be routed 100% dynamically. There would be no disconnections or UDP IP address changes. The only caveat is that the colo adds another 10ms to the round-trip time verses a direct connection. But on the plus side the home network can operate uninterrupted over however many discrete internet links I have access to, including modem dial backup or a directional WIFI link between friend's houses. -- I still gotta find time to write that program but there's nothing fancy about the concept. Maintain multiple links, route packets over the links that are up... simple stuff really. DragonFly has a number of utilities that make the job easy which FreeBSD folks might want to look into: http://www.dragonflybsd.org/cvsweb/src/usr.sbin/vknetd/ (vknetd is a packet switch, complete with a MAC cache forwarding). + SOCK_SEQPACKET support in the kernel for unix domain sockets. (it wouldn't be too hard for FreeBSD to implement SOCK_SEQPACKET and stream connection support via unix domain sockets, it took less then a day to get it into DFly). Having a packetized stream socket connection to a user program (vknetd) which implements a packet switch takes all the effort out of messing around with network routing, literally. -Matt ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: [Fwd: Re: 3 connections as one]
Matthew Dillon wrote: You can do it for outgoing connections fairly easily using the NAT trick (with PF), but you can't really load balance multiple links without support from some outside entity. If one of the tunnels goes down you can fail-over but any pre-existing connections will die and have to be re-established on the remaining link(s). That generally works ok for TCP but is total hell for UDP (because the source address will suddenly 'change' on an existing 'connection' and often trigger security blocks or simply break the program in question when it does). I've got a DSL connection and a Cable internet connection at home now, having replaced the T1 I had had for many years. I tried using the NAT trick using PF for outgoing but was never happy with the results under max load (and my links are typically running at 100% 24x7). I wasn't able to get fail-over to work properly with PF at all... the network was actually less reliable instead of more reliable and using NAT meant I had very little control over port selection or reverse-IP. I eventually gave up and now just use my DSL line for all my normal traffic, and my cable link for my off-site backup traffic. -- I'm planning out a new solution, one that a friend of mine implemented with a portable class C he owns at a colo with a single link which I want to extend to multiple links. The idea is to chop off a subnet from the colo-routed class C and run it to the home box over multiple tunnels (one over COMCAST, one over DSL). I am going to run all the tunnels through a single user program on my router box and backhaul it into a TUN interface (using PF on the TUN interface for QOS), and have the user program do all the load balancing and fail-over. Since the whole mess is routing a single subnet, no NAT tricks are needed and packets can be routed 100% dynamically. There would be no disconnections or UDP IP address changes. The only caveat is that the colo adds another 10ms to the round-trip time verses a direct connection. But on the plus side the home network can operate uninterrupted over however many discrete internet links I have access to, including modem dial backup or a directional WIFI link between friend's houses. I've done that running mpd to split the load over the tunnels from the colo. if either tunnel goes down mpd hickups nd hten everything keeps going.. -- I still gotta find time to write that program but there's nothing fancy about the concept. Maintain multiple links, route packets over the links that are up... simple stuff really. DragonFly has a number of utilities that make the job easy which FreeBSD folks might want to look into: http://www.dragonflybsd.org/cvsweb/src/usr.sbin/vknetd/ (vknetd is a packet switch, complete with a MAC cache forwarding). ng_bridge.. + SOCK_SEQPACKET support in the kernel for unix domain sockets. (it wouldn't be too hard for FreeBSD to implement SOCK_SEQPACKET and stream connection support via unix domain sockets, it took less then a day to get it into DFly). Having a packetized stream socket connection to a user program (vknetd) which implements a packet switch takes all the effort out of messing around with network routing, literally. mpd does this for me.. -Matt ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: [Fwd: Re: 3 connections as one]
:I've done that running mpd to split the load over the tunnels from the :colo. : :if either tunnel goes down mpd hickups nd hten everything keeps going.. :.. :mpd does this for me.. That looks almost perfect for the colo idea. I see how the links are set up and I see the bundle directive, but how do I configure a common subnet? Do I specify the same subnet for all the links and make them part of the same bundle (on both ends)? Is there a way to backhaul the bundle onto a single TUN interface? Or is that what ng_eiface is for? I need a tie-in for PF's QOS. There's no choice about that, my network is 100% saturated 24x7 and if I don't use PF's QOS with fair-share scheduling it becomes unusable. Looks like I might have to update netgraph on DFly to use mpd, but it doesn't look too difficult. But, gods, all those M_NOWAIT kernel mallocs... how can that possibly be reliable? -Matt Matthew Dillon [EMAIL PROTECTED] ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: [Fwd: Re: 3 connections as one]
Matthew Dillon wrote: :I've done that running mpd to split the load over the tunnels from the :colo. : :if either tunnel goes down mpd hickups nd hten everything keeps going.. :.. :mpd does this for me.. That looks almost perfect for the colo idea. I see how the links are set up and I see the bundle directive, but how do I configure a common subnet? Do I specify the same subnet for all the links and make them part of the same bundle (on both ends)? Is there a way to backhaul the bundle onto a single TUN interface? Or is that what ng_eiface is for? I need a tie-in for PF's QOS. There's no choice about that, my network is 100% saturated 24x7 and if I don't use PF's QOS with fair-share scheduling it becomes unusable. the ng_iface that is created (ng0 or whatever) is the virtual connection back to the colo. now that we have multiple routing tables we can make the tunnel and its contents use different routing tables which can simplify things. mpd allows you to backhaul through udp or tcp transport 'tunnels' to the remote poitn of your choice. Looks like I might have to update netgraph on DFly to use mpd, but it doesn't look too difficult. But, gods, all those M_NOWAIT kernel mallocs... how can that possibly be reliable? what can I say without degenerating into a dogfight? The code is designed to copy with failure to allocate.. the error will propogate up.. not much is allocated once you have it set up. -Matt Matthew Dillon [EMAIL PROTECTED] ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: [Fwd: Re: 3 connections as one]
:what can I say without degenerating into a dogfight? :The code is designed to copy with failure to allocate.. the error will :propogate up.. : :not much is allocated once you have it set up. Well, not really trying to start a fight but unless the initialization code that sets this stuff up retries on ENOMEM, don't you risk load on the machine creating a situation where M_NOWAIT can return NULL under normal operation and cause a one-time initialization to basically fail? That doesn't sound good to me. I've noticed the same thing with many driver's softc allocations. A lot of them use M_NOWAIT but clearly do not handle a temporary allocation failure. They may not crash, but they won't initialize either. The user expectation is for the driver to only fail to initialize if something serious has occured. DragonFly is a bit more sensitive then FreeBSD. Maybe M_NOWAIT allocations on FreeBSD have no chance of failing unless the system is actually out of memory. But on DFly anything that would cause a thread switch during the allocation will fail if M_NOWAIT is specified. -Matt Matthew Dillon [EMAIL PROTECTED] ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
[Fwd: Re: 3 connections as one]
Forwarded Message From: Martes Wigglesworth [EMAIL PROTECTED] Reply-To: [EMAIL PROTECTED] To: Andres Chavez [EMAIL PROTECTED] Subject: Re: 3 connections as one Date: Tue, 24 Jun 2008 16:34:04 -0400 I have been researching this issue for almost a month now, and what I have found is that you can bind the ports together for outbound traffic, and the same can be done for inbound traffic, the problem comes when you try to get the inbound packets, or sessions to dispurse across the load-balanced ports. I.E.: Who is on the other side of the multiple DSL/Cable links to filter the traffic across the associated pipes so as to balance the load, so to speak? It can be done, however, without an upstream, or maybe a vps that is being used as an external gateway, you will not be able to get the different session traffic to load balance across the multiple links, when downloading. At least that seems to be the situation, without some nifty DNS tricks. I have not seen how the appliances get around this, however, it took me this long for either list that I was on, to even admitt that the theory was not stupid, and to engage me in productive inquiry. If you find anything else out on this topic, please let me know. On Wed, 2008-06-25 at 00:07 +, Andres Chavez wrote: Hi, a friend is challenge me to make use of 3 different connections (one adsl, one cable, and one Evdo) as one single connection to internet, i believe for make faster downloads or something such, its that can be possible ?, if so, can anybody help me with this?, this sounds interesting for know tricks on the FreeBSD operating system, he need to use this box as the network manager and firewall as well, but the connection thing its killing me i dont know how. -- ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED] ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: [Fwd: Re: 3 connections as one]
Martes Wigglesworth wrote: Forwarded Message From: Martes Wigglesworth [EMAIL PROTECTED] Reply-To: [EMAIL PROTECTED] To: Andres Chavez [EMAIL PROTECTED] Subject: Re: 3 connections as one Date: Tue, 24 Jun 2008 16:34:04 -0400 I have been researching this issue for almost a month now, and what I have found is that you can bind the ports together for outbound traffic, and the same can be done for inbound traffic, the problem comes when you try to get the inbound packets, or sessions to dispurse across the load-balanced ports. I.E.: Who is on the other side of the multiple DSL/Cable links to filter the traffic across the associated pipes so as to balance the load, so to speak? It can be done, however, without an upstream, or maybe a vps that is being used as an external gateway, you will not be able to get the different session traffic to load balance across the multiple links, when downloading. At least that seems to be the situation, without some nifty DNS tricks. I have not seen how the appliances get around this, however, it took me this long for either list that I was on, to even admitt that the theory was not stupid, and to engage me in productive inquiry. the usual way is to NAT traffic out though each interface so that the internet is not aware that sessions from apparently different places are actually the same.. you can do the same with multiple NAT instances and some way to divide up the load between interfaces.. If you find anything else out on this topic, please let me know. On Wed, 2008-06-25 at 00:07 +, Andres Chavez wrote: Hi, a friend is challenge me to make use of 3 different connections (one adsl, one cable, and one Evdo) as one single connection to internet, i believe for make faster downloads or something such, its that can be possible ?, if so, can anybody help me with this?, this sounds interesting for know tricks on the FreeBSD operating system, he need to use this box as the network manager and firewall as well, but the connection thing its killing me i dont know how. -- ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED] ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED] ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
