Split flows across Domains
Hi, Are there any ISPs that do, or desire, splitting traffic across different ASes for prefixes learnt via an exterior gateway protocol (say BGP)? For example, an ISP can learn two different equal cost routes to a foo.com server via two different autonomous domains. It can thus split different flows (based on src-dest IP, src-dest Port, TOS, etc) across these two paths. Do operators currently do this? Folks can send me replies offline in case this constitues a trade secret! Thanks, Glen
Re: Split flows across Domains
Glen Kent [EMAIL PROTECTED] writes: For example, an ISP can learn two different equal cost routes to a foo.com server via two different autonomous domains. It can thus split different flows (based on src-dest IP, src-dest Port, TOS, etc) across these two paths. Do operators currently do this? Folks can send me replies offline in case this constitues a trade secret! Works great with a flow-based router (or layer-three-switch-pronounced- 'crippled-router'). The downside, of course, is that you now have a flow-based router in your network, which has been shown to Not Work Well under other specified conditions (worm outbreaks come immediately to mind). ---rob
urgent request for a contact at rcn.com/.net
Hi guys, this is rather urgent, we would appreciate any help. I will make sure and update as things progress, but right now I believe public attention would only hinder our (DA/MWP/etc. TISF) incident response attempts. Thanks, Gadi.
Re: Split flows across Domains
On Tue, 24 Jan 2006, Robert E.Seastrom wrote: Glen Kent [EMAIL PROTECTED] writes: For example, an ISP can learn two different equal cost routes to a foo.com server via two different autonomous domains. It can thus split different flows (based on src-dest IP, src-dest Port, TOS, etc) across these two paths. Do operators currently do this? Folks can send me replies offline in case this constitues a trade secret! Works great with a flow-based router (or layer-three-switch-pronounced- 'crippled-router'). The downside, of course, is that you now have a flow-based router in your network, which has been shown to Not Work Well under other specified conditions (worm outbreaks come immediately to mind). I could be mistaken, but this is also a feature in mbgp, effectively loadsharing across two external paths. I presume the paths would have to be completely equal all the way down to the router-id and (probably) age-of-route ...
Password Security and Distribution
All, Our company is starting to grow rather quickly and we are starting to have growing pains. We are in the need for a better mechanism for sharing passwords between our engineers. Most of these passwords are for our client's systems where some of them are controlling the password schemes (aka requiring shared user accounts). We have a process in which we change passwords every X days but, distributing these passwords to everyone who needs them is starting to become a challenge. Also, handing off passwords to someone who is stepping in to help out at 3am securely is not easy. I have tried to do google searches but I have not been able to find a good way or process to do this. I am wondering if anyone has any ideas on how to handle this? In other companies we have used a PGP keyring to secure a text file that contained all of these passwords and then put them onto a shared customer portal. The problem with this strategy is what happens if you are not on your computer where PGP is installed? Any suggestions will be welcomed. Thanks in advance, Jeremy
Urgent Alert: Possible BlackWorm DDay February 3rd (Snort signatures included)
Hello. This is an urgent alert released by the cooperative efforts of the MWP / DA groups that also worked on the hurricane Rita scams. This task force is now known as the TISF BlackWorm task force. This task force involves many in the security (anti spam, CERTs, anti virus, academia, ISP's, etc.) community and industry, working together to combat threats to the security of the Internet in cooperation with law enforcement globally. Anti Viruses companies each have a chosen name for this, but for operational reasons as well as simplicity we choose BlackWorm. This is what we submit for CME. A CME entry should hopefully be created shortly. Buttom line: 1. Update anti viruses urgently. 2. See Snort signatures below. A special SANS Diary page should be setup soon to process information for Snort signatures for this as we refine them: http://isc.sans.org/blackworm (Current Snort sigs are at the footer of this email message) General information and updates will be found also at: http://blogs.securiteam.com Actual information and background: This worm will destroy certain data files on an infected user's machine. So far about 700K users have been infected. We know this because of a counter which the malware author made use of. That machine is nothing but a counter and there is no reason at this time to blackhole it, as it would harm our attempts to respond to this incident. We are however coordinating a possible action of this sort with the right people if that becomes necessary. We believe the counter to be real and the number of infected users to be mostly accurate. We are working with law enforcement and the ISP to get a list of infected IP's so that we can inform the respected ISP's of the possibly infected users in their net-space. DDay is February 3rd (i.e. that is when the worm becomes destructive). However effective or ineffective this may be, we urge users to update their anti viruses as soon as possible and scan their computers and/or networks. This risk may turn out to be nothing and whatever happens, the Internet is NOT going to die. We would however rather attempt to prevent this DDay on February 3rd regardless. Further, Joe Stewart ([EMAIL PROTECTED]) has come up with the Snort signatures below to help detect infected users in your net-space. False positives should be reported to him. It should be noted that the worm connects to the counter only once on connection, however it keeps trying to DDoS Microsoft. Both these methods can be used to track down the infected users at risk. These signatures and this alert should soon also be on BleedingSnort and the SANS Diary, as well as come from different CERTs. Snort SIgnatures: 1. This sig alerts if someone visits any counter at webstats.web.rcn.net without a Referrer: header in their URL. Could be an infected user, could be one of us checking out the counter stats: alert tcp any any - any 80 (msg:webstats.web.rcn.net count.cgi request without referrer (possible BlackWorm infection); content:GET /cgi-bin/Count.cgi|3f|; depth:23; content:df|3d|; content:Host|3a 20|webstats.web.rcn.net; content:!Referer|3a|; classtype:misc-activity; sid:1000376; rev:1;) 2. This sig alerts on the specific pattern BlackWorm uses to test connectivity to www.microsoft.com. It's unique in that the request doesn't have a User-agent: header. So this will catch BlackWorm and possibly other automated requests to microsoft (which could happen if someone codes a sloppy app that uses the exact same pattern - but they should probably be flogged anyway) alert tcp any any - any 80 (msg:Agentless HTTP request to www.microsoft.com (possible BlackWorm infection); dsize:92; content:GET / HTTP/1.1|0d0a|Host|3a20|www.microsoft.com|0d0a| Connection|3a20|Keep-Alive|0d0a|Cache-Control|3a20|no-cache|0d0a0d0a|; classtype:misc-activity; sid:1000377; rev:1;) Thanks, we will update further as information becomes available, if necessary. Good luck, Gadi.
RE: Password Security and Distribution
Jeremy - I've not found a better solution than PGP. Perhaps more a formalized process for communicating password updates proactively is all you need. Ideally, distributing passwords at 3am is too late. In the past I've used small password database programs on a network share. You are then left with verbal or PGP encrypted communications to distribute a single new password to access the database versus distributing all of the changed passwords. If you're interested try http://www.anypassword.com There are others who read this list that prefer distributing passwords on paper. You can't hack into a piece of paper :) and if you have physical access to the paper then you most likely have physical access to the network equipment as well... McLean -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Jeremy Stinson Sent: Tuesday, January 24, 2006 10:49 AM To: nanog@merit.edu Subject: Password Security and Distribution All, Our company is starting to grow rather quickly and we are starting to have growing pains. We are in the need for a better mechanism for sharing passwords between our engineers. Most of these passwords are for our client's systems where some of them are controlling the password schemes (aka requiring shared user accounts). We have a process in which we change passwords every X days but, distributing these passwords to everyone who needs them is starting to become a challenge. Also, handing off passwords to someone who is stepping in to help out at 3am securely is not easy. I have tried to do google searches but I have not been able to find a good way or process to do this. I am wondering if anyone has any ideas on how to handle this? In other companies we have used a PGP keyring to secure a text file that contained all of these passwords and then put them onto a shared customer portal. The problem with this strategy is what happens if you are not on your computer where PGP is installed? Any suggestions will be welcomed. Thanks in advance, Jeremy
State of Spoofing [was: Re: BLS FastAccess internal tech needed]
On Thu, Jan 12, 2006 at 11:09:13PM -0500, Steven M. Bellovin wrote: RFC2827/BCP38? The problem is that an ISP can do all the source filtering it wants, but if it only blocks SYNs to port 25 all it takes is one unfiltered dial-up to spoof that ISP's addresses. On the subject of filtering and IP spoofing... In the past year, our spoofer project has collected nearly 1200 unique reports from across the Internet and we have an interesting, if not wholly representative, dataset. The latest version of our spoofer tester includes a number of new features that may be interesting to the community. One particular new feature is the ability to determine where along a tested path filtering is employed with what we're calling a reverse traceroute mechanism [1]. Knowing the filtering depth is of particular interest to us since there is an operational tension between the specificity of router-level filters and the ability to properly maintain them. We also test fun stuff such as how far into the adjacent neighbor address space the client can spoof, filtering inconsistencies, etc. We'd appreciate any runs of the spoofer tester to help us gather additional data. The client, details of the reverse traceroute as well as our State of IP spoofing summary results are all the web page: http://spoofer.csail.mit.edu/ Thanks, rob [1] The idea for the reverse traceroute arose from a fruitful discussion with John Curran.
Re: Split flows across Domains
Christopher L. Morrow [EMAIL PROTECTED] writes: On Tue, 24 Jan 2006, Robert E.Seastrom wrote: Glen Kent [EMAIL PROTECTED] writes: For example, an ISP can learn two different equal cost routes to a foo.com server via two different autonomous domains. It can thus split different flows (based on src-dest IP, src-dest Port, TOS, etc) across these two paths. Do operators currently do this? Folks can send me replies offline in case this constitues a trade secret! Works great with a flow-based router (or layer-three-switch-pronounced- 'crippled-router'). The downside, of course, is that you now have a flow-based router in your network, which has been shown to Not Work Well under other specified conditions (worm outbreaks come immediately to mind). I could be mistaken, but this is also a feature in mbgp, effectively loadsharing across two external paths. I presume the paths would have to be completely equal all the way down to the router-id and (probably) age-of-route ... He said via two different autonomous domains, which I took to mean two upstreams... and my understanding is that (on ciscos anyway) you're talking per-packet, not per-flow load balancing. What happens when you intentionally bugger stuff up so that you are per-packet load balancing your outbound traffic between two diverse (ie, non-congruent-to-the-same-upstream-and-pop) paths is left as an exercise to the reader... but I don't think TCP is gonna like it. :) ---rob
BlackWorm technical information
Technical information on the worm itself can be found here: http://www.f-secure.com/v-descs/nyxem_e.shtml and http://blogs.securiteam.com/index.php/archives/229 Gadi.
Re: Split flows across Domains
On 24-Jan-2006, at 12:07, Robert E.Seastrom wrote: He said via two different autonomous domains, which I took to mean two upstreams... and my understanding is that (on ciscos anyway) you're talking per-packet, not per-flow load balancing. If you can get two candidate routes for the same destination into the FIB, then you'll get per-flow load balancing as long as CEF is running, no? Joe
Re: Split flows across Domains
On 24-Jan-2006, at 13:05, Joe Abley wrote: On 24-Jan-2006, at 12:07, Robert E.Seastrom wrote: He said via two different autonomous domains, which I took to mean two upstreams... and my understanding is that (on ciscos anyway) you're talking per-packet, not per-flow load balancing. If you can get two candidate routes for the same destination into the FIB, then you'll get per-flow load balancing as long as CEF is running, no? (Substitute other, similar, hash-based ECMP route selection schemes for CEF, for routers not made by cisco. Apologies for my transient lapse into assumptions about vendor choice. And also for replying to my own mail. If anybody needs me, I'll be standing in the corner looking embarrassed.)
Re: Split flows across Domains
Joe Abley [EMAIL PROTECTED] writes:
On 24-Jan-2006, at 12:07, Robert E.Seastrom wrote:
He said via two different autonomous domains, which I took to mean
two upstreams... and my understanding is that (on ciscos anyway)
you're talking per-packet, not per-flow load balancing.
If you can get two candidate routes for the same destination into the
FIB, then you'll get per-flow load balancing as long as CEF is
running, no?
Yes and no. CEF is {src, dst} hash IIRC, and per-flow usually means
{src, srcport, dst, dstport, [proto, tos]} hash in my experience.
---Rob
Re: Split flows across Domains
On 24-Jan-2006, at 13:09, Robert E.Seastrom wrote:
Joe Abley [EMAIL PROTECTED] writes:
If you can get two candidate routes for the same destination into the
FIB, then you'll get per-flow load balancing as long as CEF is
running, no?
Yes and no. CEF is {src, dst} hash IIRC, and per-flow usually means
{src, srcport, dst, dstport, [proto, tos]} hash in my experience.
Even if the hash is only calculated over source and destination IP
addresses, the end effect is still that packets associated with a
single flow still follow the same route where there is more than one
candidate route available.
Joe
Re: Split flows across Domains
Joe Abley [EMAIL PROTECTED] writes:
On 24-Jan-2006, at 13:09, Robert E.Seastrom wrote:
Joe Abley [EMAIL PROTECTED] writes:
If you can get two candidate routes for the same destination into the
FIB, then you'll get per-flow load balancing as long as CEF is
running, no?
Yes and no. CEF is {src, dst} hash IIRC, and per-flow usually means
{src, srcport, dst, dstport, [proto, tos]} hash in my experience.
Even if the hash is only calculated over source and destination IP
addresses, the end effect is still that packets associated with a
single flow still follow the same route where there is more than one
candidate route available.
And conversely, that different flows that ought to be load-balanced
aren't. But we're splitting semantic hairs here... ;-)
---rob
Re: Password Security and Distribution
Hi, That sounds like it could be useful. The major problem I have with password safe is that it is hard to do things like copy a group of passwords to another .dat file. That makes it hard to do anything put either keep several .dat files floating around for different users, aka accountants, programmers, managers.. Which leads to some of them being way out of date and people going back to the sticky note db method.. I have some of those myself I am sorry to say.. I also found this: http://jason.diamond.name/weblog/2005/04/07/cracking-my-password-safe He goes into a lot of detail on how password safe works.. He also has a link to what he did in Python.. http://jason.diamond.name/weblog/2005/10/04/pypwsafe-release-1 Thanks, Eric At 10:03 AM 1/24/2006, John Kinsella wrote: One of my guys found a package called Password Gorilla, which is basically a GUI which sits on top of Password Safe that came out of Counterpane in 2002 or so. Either allows you to organize passwords by group and machine, and the whole database is encrypted by blowfish: http://www.fpx.de/fp/Software/Gorilla/ One thing I've been thinking of from my managed service/consulting background is to have a main database which has all users/passwords for all companies in a central database (LAMP architecture), then depending on what a user has access to, a custom Password Safe database is created for them. This would handle how to distribute password changes out to admins who have varying levels of access. Sounds like about a week's worth of work - if people voiced enough interest or if somebody cared to help me out, I'd finally get motivated to write it and put it up on Sourceforge... John On Tue, Jan 24, 2006 at 11:28:23AM -0500, McLean Pickett wrote: Jeremy - I've not found a better solution than PGP. Perhaps more a formalized process for communicating password updates proactively is all you need. Ideally, distributing passwords at 3am is too late. In the past I've used small password database programs on a network share. You are then left with verbal or PGP encrypted communications to distribute a single new password to access the database versus distributing all of the changed passwords. If you're interested try http://www.anypassword.com There are others who read this list that prefer distributing passwords on paper. You can't hack into a piece of paper :) and if you have physical access to the paper then you most likely have physical access to the network equipment as well... McLean -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Jeremy Stinson Sent: Tuesday, January 24, 2006 10:49 AM To: nanog@merit.edu Subject: Password Security and Distribution All, Our company is starting to grow rather quickly and we are starting to have growing pains. We are in the need for a better mechanism for sharing passwords between our engineers. Most of these passwords are for our client's systems where some of them are controlling the password schemes (aka requiring shared user accounts). We have a process in which we change passwords every X days but, distributing these passwords to everyone who needs them is starting to become a challenge. Also, handing off passwords to someone who is stepping in to help out at 3am securely is not easy. I have tried to do google searches but I have not been able to find a good way or process to do this. I am wondering if anyone has any ideas on how to handle this? In other companies we have used a PGP keyring to secure a text file that contained all of these passwords and then put them onto a shared customer portal. The problem with this strategy is what happens if you are not on your computer where PGP is installed? Any suggestions will be welcomed. Thanks in advance, Jeremy
BlackWorm naming confusing [CME entry now available]
The CME entry should appear on their site shortly: http://cme.mitre.org Gadi.
Re: Split flows across Domains
On Tue, 24 Jan 2006, Joe Abley wrote: On 24-Jan-2006, at 12:07, Robert E.Seastrom wrote: He said via two different autonomous domains, which I took to mean two upstreams... and my understanding is that (on ciscos anyway) you're talking per-packet, not per-flow load balancing. If you can get two candidate routes for the same destination into the FIB, then you'll get per-flow load balancing as long as CEF is running, no? that was my thought... and yes, it could get ugly for tcp services. Why would you knowningly induce this complication?
RE: Password Security and Distribution
Our company is starting to grow rather quickly and we are starting to have growing pains. We are in the need for a better mechanism for sharing passwords between our engineers. I wish there was a system that let you do the following: * Store and encrypt logins/passwords and access logs in a database * Assign permissions (add new logins/passwords, change password...) to those passwords on a per user/group basis, based on an existing authentication scheme (Windows AD, LDAP, Kerberos...) * SSL web frontend * Reporting. If a user leaves and you want to know which passwords he had access to or has ever accessed so you can change them, this would be really really nice. I've been playing around with Network Password Manager from www.sowsoft.com. It seems like the best product available in this area that I could find that makes sharing passwords kinda easy, but it's a service that runs on Windows, requires a Windows client software installation, and lacks any sort of reporting.
T1 bonding
Can someone shed some technical light on the details of how two T1's are bonded (typically). We've got two sets of T's at two different location with vendor 'X' (name starts w/ an 'A') and it appears that we're really only getting about 1 full T's worth of bandwidth and maybe 20% of the second. Seems like they're bonded perhaps using destination IP? It's a vendor managed solution and I need to get some answers faster than they're coming in. Thanks. Matt
BlackWorm infected IP's reporting
Hi. In the next day or so some of us will cooperate to bring to the attention of all effected AS's information about infected users in their net-space. This will be coordinated with several groups and organizations. Please expect these emails, thanks. Gadi.
Re: T1 bonding
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Matt Bazan wrote: Can someone shed some technical light on the details of how two T1's are bonded (typically). We've got two sets of T's at two different location with vendor 'X' (name starts w/ an 'A') and it appears that we're really only getting about 1 full T's worth of bandwidth and maybe 20% of the second. Seems like they're bonded perhaps using destination IP? It's a vendor managed solution and I need to get some answers faster than they're coming in. Thanks. Matt More than likely they are not bonded t1's they are just load balanced by the router which by default on Cisco is per session. Meaning pc1 to t1#1, pc2to t1#2, pc3 to t1#1. If they are truly bonded with some sort of MUX for a 3 meg port then you would not see the results you are seeing. - -- http://www.digitalrage.org/ The Information Technology News Center -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.2 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFD1sXyt06NWq3hlzkRAvi4AJ0R4RVii+Wrxzs5WI5es+FYhxHD0ACgioFW /UHUMapXnmuPFSpKrXzD3JU= =MqxV -END PGP SIGNATURE-
RE: T1 bonding
If you're treating them as two separate links (e.g. two POPs, etc.) then that's correct, it'll be done by the routers choice of load-balancing (L3). If you are going to the same POP (or box potentially) you can do MLPPP and have a more effective L2 load balancing. Otherwise, it's possible to get an iMux DSU (Digital Link is a vendor as I recall, but there may be others) that allow that magical bonding to occur prior to the router seeing the link. At that point, the router just sees a bigger line coming in (some do 6xT-1 and have a 10meg ethernet output to your router). If you're seeing the balancing the way that you are, most likely that vendor (I have no specific knowledge about the A-vendor) is doing usage-based aggregation which isn't exactly a balancing act. The ones at some of my sites are MLPPP which is a vendor-agnostic approach for the most part. Scott -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Elijah Savage Sent: Tuesday, January 24, 2006 7:28 PM To: Matt Bazan Cc: nanog@merit.edu Subject: Re: T1 bonding -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Matt Bazan wrote: Can someone shed some technical light on the details of how two T1's are bonded (typically). We've got two sets of T's at two different location with vendor 'X' (name starts w/ an 'A') and it appears that we're really only getting about 1 full T's worth of bandwidth and maybe 20% of the second. Seems like they're bonded perhaps using destination IP? It's a vendor managed solution and I need to get some answers faster than they're coming in. Thanks. Matt More than likely they are not bonded t1's they are just load balanced by the router which by default on Cisco is per session. Meaning pc1 to t1#1, pc2to t1#2, pc3 to t1#1. If they are truly bonded with some sort of MUX for a 3 meg port then you would not see the results you are seeing. - -- http://www.digitalrage.org/ The Information Technology News Center -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.2 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFD1sXyt06NWq3hlzkRAvi4AJ0R4RVii+Wrxzs5WI5es+FYhxHD0ACgioFW /UHUMapXnmuPFSpKrXzD3JU= =MqxV -END PGP SIGNATURE-
RE: Password Security and Distribution
On Tue, 24 Jan 2006, (nanog) Brian Battle wrote: I wish there was a system that let you do the following: * Store and encrypt logins/passwords and access logs in a database * Assign permissions (add new logins/passwords, change password...) to those passwords on a per user/group basis, based on an existing authentication scheme (Windows AD, LDAP, Kerberos...) * SSL web frontend * Reporting. If a user leaves and you want to know which passwords he had access to or has ever accessed so you can change them, this would be really really nice. BBN Planet had a nice application that did all of this. It was super nice. The only time you were in trouble was when you needed the password for the box that ran the application. (don't laugh! it happened to me once- but for some reason, i was precient enough to have it on a little piece of paper in my wallet) matto [EMAIL PROTECTED]darwin The only thing necessary for the triumph of evil is for good men to do nothing. - Edmund Burke
Re: T1 bonding
Is it ATT? If so, they only use Cisco Express Forwarding on the router, or so that's at least what I was told by the level 1 techs. If packet order reassembly is a an issue and the link is oversubscribed (IE: Heavy VoIP/gaming use), this method isn't the greatest over others like MLPPP, or per-flow CEF, but in 99% of circumstances it works great (and has other advantages). Can you max out the T-1 with two or three separate flows (IE: simultaneous transfers?) If so, it is possible that they are doing per flow and not per-packet load balancing. It should be per packet. Call them up. Once you get screened and transferred to a Cisco guy, fire away with your questions -- they know their stuff in my experience. Or if is your equipment, log into the router and see if ip load-sharing per-packet is set (assuming it is CEF), and confirm they did the same. Off topic, but in my experience MLPPP usually does a better job of getting 190% of a T-1's speed with two of them. CEF usually tops out at around 160-170% with a single flow, but will max out with as little as two flows. I don't know why though, and haven't cared since I've never really had a dual T-1 all to myself without any other users. 2.5 megabit seems to be the single flow norm on our ATT Circuits at 3 AM with no usage., 2.8-2.9 with two or three flows. As for the technical details, here is some reading material that explains it quite nicely. http://www.cisco.com/en/US/products/hw/modules/ps2033/products_white_paper09186a0080091d4b.shtml http://www.swcp.com/~jgentry/cisco/cisco-load.html http://www.cisco.com/univercd/cc/td/doc/product/software/ios120/120newft/120limit/120s/120s21/pplb.htm Test file here for speed tests: ftp://ftp1.optonline.net/test64 Matt Bazan wrote: Can someone shed some technical light on the details of how two T1's are bonded (typically). We've got two sets of T's at two different location with vendor 'X' (name starts w/ an 'A') and it appears that we're really only getting about 1 full T's worth of bandwidth and maybe 20% of the second. Seems like they're bonded perhaps using destination IP? It's a vendor managed solution and I need to get some answers faster than they're coming in. Thanks. Matt Matt Bazan wrote: Can someone shed some technical light on the details of how two T1's are bonded (typically). We've got two sets of T's at two different location with vendor 'X' (name starts w/ an 'A') and it appears that we're really only getting about 1 full T's worth of bandwidth and maybe 20% of the second. Seems like they're bonded perhaps using destination IP? It's a vendor managed solution and I need to get some answers faster than they're coming in. Thanks. Matt
Re: T1 bonding
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Scott Morris wrote: If you're treating them as two separate links (e.g. two POPs, etc.) then that's correct, it'll be done by the routers choice of load-balancing (L3). If you are going to the same POP (or box potentially) you can do MLPPP and have a more effective L2 load balancing. Otherwise, it's possible to get an iMux DSU (Digital Link is a vendor as I recall, but there may be others) that allow that magical bonding to occur prior to the router seeing the link. At that point, the router just sees a bigger line coming in (some do 6xT-1 and have a 10meg ethernet output to your router). If you're seeing the balancing the way that you are, most likely that vendor (I have no specific knowledge about the A-vendor) is doing usage-based aggregation which isn't exactly a balancing act. The ones at some of my sites are MLPPP which is a vendor-agnostic approach for the most part. Scott -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Elijah Savage Sent: Tuesday, January 24, 2006 7:28 PM To: Matt Bazan Cc: nanog@merit.edu Subject: Re: T1 bonding Matt Bazan wrote: Can someone shed some technical light on the details of how two T1's are bonded (typically). We've got two sets of T's at two different location with vendor 'X' (name starts w/ an 'A') and it appears that we're really only getting about 1 full T's worth of bandwidth and maybe 20% of the second. Seems like they're bonded perhaps using destination IP? It's a vendor managed solution and I need to get some answers faster than they're coming in. Thanks. Matt More than likely they are not bonded t1's they are just load balanced by the router which by default on Cisco is per session. Meaning pc1 to t1#1, pc2to t1#2, pc3 to t1#1. If they are truly bonded with some sort of MUX for a 3 meg port then you would not see the results you are seeing. -- http://www.digitalrage.org/ The Information Technology News Center Remember he said both t1's are coming from different vendors, which would only leave the Mux route which is why I said what I said :) - -- http://www.digitalrage.org/ The Information Technology News Center -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.2 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFD1tJWt06NWq3hlzkRApDsAJ9nq+J+26EKYy9cwlFRmN3zhT/EFQCfdf2v IX2wkyZvsGM1sPvcEMSyK+0= =WINE -END PGP SIGNATURE-
RE: T1 bonding
I'm re-reading it, and slowly, but I don't see mention of having two different vendors. Perhaps I need to put the beer a bit further away, but he talks about generic vendor 'x' and notes that it starts with letter 'A' as further definition, not as two separate vendors. *shrug* Scott -Original Message- From: Elijah Savage [mailto:[EMAIL PROTECTED] Sent: Tuesday, January 24, 2006 8:20 PM To: [EMAIL PROTECTED] Cc: 'Matt Bazan'; nanog@merit.edu Subject: Re: T1 bonding -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Scott Morris wrote: If you're treating them as two separate links (e.g. two POPs, etc.) then that's correct, it'll be done by the routers choice of load-balancing (L3). If you are going to the same POP (or box potentially) you can do MLPPP and have a more effective L2 load balancing. Otherwise, it's possible to get an iMux DSU (Digital Link is a vendor as I recall, but there may be others) that allow that magical bonding to occur prior to the router seeing the link. At that point, the router just sees a bigger line coming in (some do 6xT-1 and have a 10meg ethernet output to your router). If you're seeing the balancing the way that you are, most likely that vendor (I have no specific knowledge about the A-vendor) is doing usage-based aggregation which isn't exactly a balancing act. The ones at some of my sites are MLPPP which is a vendor-agnostic approach for the most part. Scott -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Elijah Savage Sent: Tuesday, January 24, 2006 7:28 PM To: Matt Bazan Cc: nanog@merit.edu Subject: Re: T1 bonding Matt Bazan wrote: Can someone shed some technical light on the details of how two T1's are bonded (typically). We've got two sets of T's at two different location with vendor 'X' (name starts w/ an 'A') and it appears that we're really only getting about 1 full T's worth of bandwidth and maybe 20% of the second. Seems like they're bonded perhaps using destination IP? It's a vendor managed solution and I need to get some answers faster than they're coming in. Thanks. Matt More than likely they are not bonded t1's they are just load balanced by the router which by default on Cisco is per session. Meaning pc1 to t1#1, pc2to t1#2, pc3 to t1#1. If they are truly bonded with some sort of MUX for a 3 meg port then you would not see the results you are seeing. -- http://www.digitalrage.org/ The Information Technology News Center Remember he said both t1's are coming from different vendors, which would only leave the Mux route which is why I said what I said :) - -- http://www.digitalrage.org/ The Information Technology News Center -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.2 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFD1tJWt06NWq3hlzkRApDsAJ9nq+J+26EKYy9cwlFRmN3zhT/EFQCfdf2v IX2wkyZvsGM1sPvcEMSyK+0= =WINE -END PGP SIGNATURE-
Re: T1 bonding
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Scott Morris wrote: I'm re-reading it, and slowly, but I don't see mention of having two different vendors. Perhaps I need to put the beer a bit further away, but he talks about generic vendor 'x' and notes that it starts with letter 'A' as further definition, not as two separate vendors. *shrug* Scott -Original Message- From: Elijah Savage [mailto:[EMAIL PROTECTED] Sent: Tuesday, January 24, 2006 8:20 PM To: [EMAIL PROTECTED] Cc: 'Matt Bazan'; nanog@merit.edu Subject: Re: T1 bonding Scott Morris wrote: If you're treating them as two separate links (e.g. two POPs, etc.) then that's correct, it'll be done by the routers choice of load-balancing (L3). If you are going to the same POP (or box potentially) you can do MLPPP and have a more effective L2 load balancing. Otherwise, it's possible to get an iMux DSU (Digital Link is a vendor as I recall, but there may be others) that allow that magical bonding to occur prior to the router seeing the link. At that point, the router just sees a bigger line coming in (some do 6xT-1 and have a 10meg ethernet output to your router). If you're seeing the balancing the way that you are, most likely that vendor (I have no specific knowledge about the A-vendor) is doing usage-based aggregation which isn't exactly a balancing act. The ones at some of my sites are MLPPP which is a vendor-agnostic approach for the most part. Scott -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Elijah Savage Sent: Tuesday, January 24, 2006 7:28 PM To: Matt Bazan Cc: nanog@merit.edu Subject: Re: T1 bonding Matt Bazan wrote: Can someone shed some technical light on the details of how two T1's are bonded (typically). We've got two sets of T's at two different location with vendor 'X' (name starts w/ an 'A') and it appears that we're really only getting about 1 full T's worth of bandwidth and maybe 20% of the second. Seems like they're bonded perhaps using destination IP? It's a vendor managed solution and I need to get some answers faster than they're coming in. Thanks. Matt More than likely they are not bonded t1's they are just load balanced by the router which by default on Cisco is per session. Meaning pc1 to t1#1, pc2to t1#2, pc3 to t1#1. If they are truly bonded with some sort of MUX for a 3 meg port then you would not see the results you are seeing. -- http://www.digitalrage.org/ The Information Technology News Center Remember he said both t1's are coming from different vendors, which would only leave the Mux route which is why I said what I said :) -- http://www.digitalrage.org/ The Information Technology News Center Uh Scott I think it is I whom by the way is getting up right now and going to put the rest of the beer back in the fridge. PS - -- http://www.digitalrage.org/ The Information Technology News Center -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.2 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFD1tPCt06NWq3hlzkRAqTUAJ44ss3rZxpxv20zXab94GbIbRoudgCaA1J9 3dTi8Msj+xp6qkJvfrSylsY= =CTM7 -END PGP SIGNATURE-
Re: Split flows across Domains
On Tue, 24 Jan 2006, Christopher L. Morrow wrote: that was my thought... and yes, it could get ugly for tcp services. Why would you knowningly induce this complication? When you want single flows to go faster than a single member link? (not that I am saying this is a good idea) Actually, TCP handles out of order packets rather well as long as the reordering isn't too severe. You see a bunch of SACKs flying around, but as long as it doesn't get too out of hand it doesn't affect throughput. It is the non-TCP protocols that often suffer. Many of them implement sequence numbers and simply drop out of order packets. From my own experience, RealPlayer UDP streams and PPTP are two examples that fail (or at least feel like 50% packetloss) under heavy reodering, where TCP continues to work reasonably well. Years ago, I had ISDN and IDSL between home and the ISP I worked at, and out of curiosity I experimented with per-packet load balancing across these links. Reordering was rather severe, as these links had slow uneven speeds, and uneven latencies. TCP transfers got about 192kbit (75% total link capacity, 1.5 times single link capacity), but things like RealPlayer and PPTP VPNs were downright unusable.
Re: Router upgrade for small colo provider
josh == josh harrington [EMAIL PROTECTED] writes: josh [option #3 - Cisco 6509 switch'router' w/MSFC2] [...] josh - 'not a router' as some would say [though this one is as good josh as it gets for a switch with router ability built in, so i read josh at least] It routes packets, therefore it is a router :-) Seriously, the people who call it not a router are talking through their hats. josh - bgp4 support appears limited in previous versions, but the josh MSFC2 processor supposedly can handle (2) bgp4 sessions josh properly [makes me nervous] I have some of these running with combinations ranging from 5 full-routes sessions + iBGP through to 2 full + iBGP + 70+ peers. You don't need to be nervous about the MSFC2's ability to do BGP (though for serious work you do want the maximum memory in both the MSFC2 _and_ the Sup2 (512M and 256M respectively) - the 256M on the Sup2 is _important_ if you're going to have full routes). josh - no support for anything but 100mbit, or gigE links, wont work josh with t3, or oc3 lines I understand there are modules for other interface types. No idea how easy they are to get hold of; we only use gigE. josh - 'all eggs in 1 basket' theory, if it breaks you loose all josh your ether switches! [at least with separate routers/switches i josh can swap in an old 7206 router spare and get back online fast josh in a worst case scenario. We solve this by having multiple routers... Other negative factors you didn't list: - PFC2 has a hardware forwarding table limit of about 256k prefixes (of which I think some are reserved). uRPF cuts that in half. Current routing table size is ~176k prefixes... so no uRPF possible with full routes, and the total routing table size may become an issue. - PFC2 doesn't support IPv6. At all. I don't know if any IOS versions available for the 65xx support IPv6 in software, but...: - MSFC2 has relatively limited capacity for forwarding traffic in software. This normally isn't a problem, but it means you have to be careful not to do things (like trying to log traffic in ACLs) that result in your main traffic flows being punted to the MSFC. There are lots of other advantages besides the ones you mentioned, though. -- Andrew, Supernews http://www.supernews.com
Re: Split flows across Domains
On 24-Jan-2006, at 14:17, Matt Buford wrote: Actually, TCP handles out of order packets rather well as long as the reordering isn't too severe. There's packet reordering, and there's oscillating RTT on segments that travel by different paths. I suspect the veracity of your statement depends (a) on whose implementation of TCP you're using, and (b) what exactly you mean by rather well. Joe
Re: T1 bonding
They can be bonded via MLPPP or IMA, as stated previously. Also they can be load-balanced via EIGRP. What are you using to test your bandwidth (IPerf is pretty handy)? I'm kinda assuming that the T1's are point to point, how far apart are the offices? -Wil Matt Bazan wrote: Can someone shed some technical light on the details of how two T1's are bonded (typically). We've got two sets of T's at two different location with vendor 'X' (name starts w/ an 'A') and it appears that we're really only getting about 1 full T's worth of bandwidth and maybe 20% of the second. Seems like they're bonded perhaps using destination IP? It's a vendor managed solution and I need to get some answers faster than they're coming in. Thanks. Matt
Re: Router upgrade for small colo provider
On Wed, 25 Jan 2006, Andrew - Supernews wrote: I have some of these running with combinations ranging from 5 full-routes sessions + iBGP through to 2 full + iBGP + 70+ peers. You don't need to be nervous about the MSFC2's ability to do BGP (though for serious work you do want the maximum memory in both the MSFC2 _and_ the Sup2 (512M and 256M respectively) - the 256M on the Sup2 is _important_ if you're going to have full routes). It's actually 512M on both. With later/bigger IOS versions, you might actually utilize 256M on the Sup2. Max both out at time of purchase so you don't have to take it down later for upgrades. #remote command switch show mem HeadTotal(b) Used(b) Free(b) Lowest(b) Largest(b) Processor 448543E0 393919520 108659576 285259944 273392008 211623448 I/O80067108880103533285672567256755512 That's from a WS-X6K-SUP2-2GE. -- Jon Lewis | I route Senior Network Engineer | therefore you are Atlantic Net| _ http://www.lewis.org/~jlewis/pgp for PGP public key_
