Re: Network inventory and configuration tracking tools
On Wed, 7 Aug 2002, Sean Donelan wrote: How about an operations oriented question. What is the current preferences amoung network operators for network inventory and configuration management tools? Not so much status monitoring (up, down) but other stuff network operator wants to know like circuit IDs (how many IDs can a circuit have?), network contacts, design layout reports (layer 1/2/3), what's supposed to be connected to that port? The stuff you can't get out of the box itself. Most ISPs seem to end up with a combination of homegrown systems, opensource, and commercial products. The commercial integrated systems have lots of stuff, and according to the vendors can do anything including splice fiber. We ended up in large part developing our own tools in-house. One is an SQL database to store and link network elements (routers, interfaces/ports, circuits, IP addresses, contacts, etc) with hooks into other internal databases and other outward-facing applications, such as our rwhois server. Another is a tool that polls our network devices once every few hours and backs up their configuration into an RCS filestore so we have journaling capabilities. We do use some commercial tools, but those are mainly for customer presentation (VitalSuite) and up/down reporting and event correlation (Netcool). jms
Re: PSINet/Cogent Latency
On Mon, 22 Jul 2002, Alex Rubenstein wrote: Yes, it's horrid. I've been peering with PSI for going on three years, and it's never been as bad as it is now. I took advantage of their free peering offer back in the day, and ended up peering with them for about 18 months (06/1999 - 01/2001). It took about 9 months for them to get the circuit installed. For the first few months, everything was great, but then we started getting massive spikes in latency (300-700ms) just getting across the pipe between my router and PSI's router. I liken it to owning an old Audi - they were great when they ran, but spent more time in the shop than on the road. The process of opening tickets and getting clued people in their NOC to talk to me was an adventure. PSI, much like some other providers, went to great pains to try keeping $CUSTOMER from having a direct path to $CLUEDPEOPLE. They could never adequately explain the latency, other than it would mysteriously go away and re-appear, more or less independent of the amount of traffic on the circuit. Eventually an upper-level engineer told me that the saturation was due to congestion on their end of the pipe, and getting some fatter pipe in there would take 60 days. Fine. 90 days later, the bigger pipe is installed on their end and the latency goes away for a few weeks, then comes back. Wash. Rinse. Repeat. A few more months of that, and I cancelled the peering. oddly enough, we see 30+ msec across a DS3 to them, which isn't that loaded (35 to 40 mb/s). Then, behind whatever we peer with, we see over 400 msec, with 50% loss, during business hours.
Re: AS number inconsistencies
On Mon, 8 Jul 2002, Marwan Fayed wrote: I am a CS PhD student trying to track ASes (for reasons I'm happy to discuss offline). There is a grave inconsistency I have come across and can't explain. Simply, there seems to be many AS numbers in the non-private range that come into use at some point in time and advertise a range of IPs, but these AS numbers are not allocated until much later. More specifically, archived BGP tables show many AS numbers which ARIN shows not to have allocated (in their allocation history tables) until many months, sometimes a year/two, later. The number of such ASes has shrunk over time (from about 100 in 1999/2000 to 20-30 in 2002) but still exists. I don't want to name ASes grin. Does any one have any explanations? Are network operators notified of their new AS number well in advance of the actual receipt of that number on paper, for example? Any help is appreciated (and hopefully this occurence is of interest to nanog). The most plausible explanations I can think of for people not using their ASNs in their production networks for a long time after receiving them from their RIR are: 1) There are technical challenges to be overcome before the AS can start to originate routes. For example, the AS migrations, or some other large network cutover or architecture change. 2) After the ASN is allocated, business/technical drivers shift as they often do in this industry, and the project that required the new ASN is now pushed back/scaled down/eliminated entorely. I've seen examples of both in the wild. jms
Re: What's wrong with provisioning tools?
On Wed, 12 Jun 2002, Stephen Griffin wrote: In the referenced message, David Daley said: snip 4) There isn't anything to track non sanctioned changes to the network (i.e.: hacker induced re-configurations) I would be really surprised if anything other than mom-and-pop shops didn't have _at least_ this. rtrmon or rancid can do great config archiving and provide difference output. I didn't find anything that really suited my needs at the time (late 2000/early 2001), so I ended up writing my own archiver. From time to time I've thought about adding it to the COSI-NMS project on Sourceforge, but never gotten around to it. I've also other similar tools outside of Sourceforce, such as Pancho (http://pancho.lunarmedia.net/). I wrote the code behind mine to be fairly modular, so that adding a module to back up a config from a new device is pretty easy. It currently backs up these devices using either SNMP or Expect scripts for devices that require it: Cisco IOS 12.0 Cisco IOS =12.0 Cisco CatOS Cisco 5000 VPN concentrators (the Compatible Systems ones, not Altiga) Cisco LocalDirectors Lucent TAOS (Max TNTs) Alteon WebOS (ACEdirectors) Redback AOS Nortel BayRS (Bay Networks nee Wellfleet) -config is binary other odds and ends as they come up, like Netopia routers, etc. I haven't written anything to back up Junipers yet because I don't have any to test against. Aside from the Nortel routers, I support versioning on everything else. Keep in mind this is only one piece of the puzzle - backing up what's already out there. I intentionally left out the functionality to allow a config to be uploaded to one of the devices above for reasons already specified in this thread - it's just too dangerous. You can melt down a whole network really quickly if you're not careful. jms
Re: Arbor Networks DoS defense product
On Tue, 14 May 2002, Pete Kruckenberg wrote: Have any large networks gathered statistics on how much traffic DDoS/DoS/DRDoS attacks consume on an average day? The attacks I have been able to detect represent around 10-15% of my traffic on an on-going basis. I'm curious about the business case for investing in DoS defense mechanisms. DoS traffic is boosting service provider revenues through increased customer bandwidth usage. I disagree. If many of your customers have flat-rate as opposed to burstable connectivity, such as a full point-to-point T1 or a dedicated 10 meg switch port to host a colo box, the revenue you derive from those customers doesn't change regardless of how much/how little traffic your network carries for them. If your customers have burstable connectivity, their bill only goes up if you have mechanisms in place to do those calculations - I'll hazard a guess that many providers don't. I would argue that in many cases a service provider loses revenue due to DoS traffic - network performance/availability can be impacted as your network absorbs a DoS attack and your NOC/network engineers/security people have to spend cycles analyzing (calling vendors, upstreams, etc) and dampening the attack. Both of these impact windows have costs associated with them. I haven't done any formal ROI calculations on Arbor or any of the other DoS defense products out there. However, from my viewpoint, I'd be willing to bet that if/once my NOC/network engineers/security people are properly trained on how to handle a DoS attack, anything that allows me to shrink those impact windows, e.g. reduce my costs related with dealing with an attack, is a good thing. So the investment in defense mechanisms like Arbor would have to replace or increase that revenue. Will these issues inhibit wide-spread implementation of DoS defenses? That depends on how those products are priced, how well they're marketed, and of course, how effective they are in helping to stop DoS attacks. jms
Re: CPE/OC12 Question
On Wed, 15 May 2002, Sonya Blake wrote: What kind of OC12 CPE devices (routers) are people using out there? Initially for Internet connectivity, but probably would need to do advance features, i.e. BGP, etc. Are you referring to an OC12c that you're using as a single 622 Mb/s pipe, or an OC12 that you're bringing into a SONET add/drop mux and breaking out STS-1 slots for DS3s or OC3/OC3c slots? If you're talking about an OC12c, your choices would probably be: Cisco 7600 Cisco 1 Cisco 12xxx Juniper M-series - I think even an M5 could do an OC12c, though I'm not sure. Other offerings by Riverstone, Avici and others that I'm not as familiar with. You can put an OC12c into a Cisco 7200/7500 *in theory* using an OC12c DPT card, but the router will likely crap out long before you come close to saturating the pipe. For a channelized OC12, assuming you want to do the breakouts yourself, you could use pretty much anything that supports channelized OC12 (Cisco 1/12000, Juniper, etc) as long as it can break the slots out the way you want. jms
Re: news-peering
On Wed, 1 May 2002, Lyndon Nerenberg wrote: } I'm trolling for newspeers, if there is anyone out there still using } NNTP.. http://www.usenet-se.net/peering/ A useless list based on my experience (zero responses to requests to twelve different sites). Some of the information on there is most likely stale. Several sites we used to peer with when we ran a news server have been borged into other companies or are no longer serving news, but are still on the list. jms
Re: UUNET instability?
On Thu, 25 Apr 2002, Sean Donelan wrote: That's unusual. A train derailment usually effects more than one provider, and normally does not cause network-wide BGP resets. I'd heard something about IS-IS instability, and it doesn't surprise me. On big networks, IGP stability is super-important, be it IS-IS, EIGRP, or OSPF. jms
Re: Internet Exchange Questions
On Tue, 19 Mar 2002, Andy Dills wrote: This industry is so far in the shitter...so many of the big names, including players from the early days, are in chapter 11 or about to be. I'm honestly surprised that I haven't had someone try to offer me a 'genuine steal' on 20-year IRUs in awhile ;-) jms
