Looking for Verizon-GNI network engineer
Sorry if this is off-topic frustration has set in. I've got what looks like a routing loop or a wedge in your network and I cant get past tier2 saying it is an internet problem. I asked to speak with an engineer directly was told Verizon engineers don't talk directly with customers. Issue going on for 4 days. $ traceroute www.tickerforum.org traceroute to www.tickerforum.org (70.169.168.7), 64 hops max, 40 byte packets 1 10.254.123.1 (10.254.123.1) 3.219 ms 1.085 ms 0.915 ms 2 L301.VFTTP-02.CLPPVA.verizon-gni.net (71.171.93.1) 6.329 ms 6.281 ms 5.036 ms 3 P2-3.LCR-02.CLPPVA.verizon-gni.net (130.81.37.194) 4.885 ms 4.091 ms 6.490 ms 4 so-7-0-0-0.PEER-RTR1.ASH.verizon-gni.net (130.81.10.94) 4.731 ms 8.248 ms 5.167 ms 5 130.81.15.238 (130.81.15.238) 5.926 ms 130.81.15.190 (130.81.15.190) 6.586 ms 9.158 ms 6 * * * 7 * * * 8 * $ traceroute www.google.com traceroute: Warning: www.google.com has multiple addresses; using 64.233.169.104 traceroute to www.l.google.com (64.233.169.104), 64 hops max, 40 byte packets 1 10.254.123.1 (10.254.123.1) 1.774 ms 1.117 ms 0.909 ms 2 L301.VFTTP-02.CLPPVA.verizon-gni.net (71.171.93.1) 5.820 ms 4.029 ms 4.861 ms 3 P2-3.LCR-01.CLPPVA.verizon-gni.net (130.81.37.192) 8.036 ms 6.346 ms 7.671 ms 4 so-6-3-1-0.BB-RTR2.RES.verizon-gni.net (130.81.29.82) 6.524 ms 8.161 ms 8.408 ms 5 * * * 6 * * * Ticket # is VAD01QVDW -Scott
RE: outage
We are also seeing issues with google in our Washington DC and Phoenix POP. LA and Dallas seem to be fine. -Scott From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Jeff Baart Sent: Friday, March 23, 2007 4:41 PM To: [EMAIL PROTECTED] Subject: outage Anyone know if there is just some sort of Google hosting outage . or if there is a larger Tier 1 issue going on this afternoon ? Thanks.
Re: IPv6 news
On Oct 12, 2005, at 8:00 PM, Randy Bush wrote: but, if you read my message, the point is that all the major hosted services will not be dual stack. half of them can't even provide well-deployed ipv4 service; try united.com. That is not entirely the fault of the hosting companies.. Note that verio, he.net, towardex, and many other progressive hosting companies have been dual stack for a long time. Perhaps the services that are not able to do dual stack will vote with their wallets and either move to a company who can help them with this or at least buy better engineers. Something has to sort of make them do it though, I can't see united.com just coming up with this idea on their own. -Scott
RE: Two questions [controlling broadcast storms netflow software]; seeking offlist responses
Hi Drew, -Original Message- SNIP One idea I had was to use the black diamond as a layer2 switch and then use the GSR to do the routing, but that seems kind of round-about. Why does this seem round-about? Depending on the line cards and IOS revision you are running in that GSR it could be a really good solution. Black Diamonds in general are not a favorite in the network community right now (re PAIX) but turning into a layer2 only device probably will get you the performance/stability that you need at least until you max-out that layer of your network. It seems like your design is very much a vanilla hosting setup, using small blocks of IP's and isolating each server on its own vlan are great practices. You will find it rather easy to migrate to the above setup because of the way you have done things. Again make sure the line cards and IOS that you use can support the # of vlans and ARP entries that you currently support in your BD setup. _Scott
RE: Major AboveNet problems?
I saw the Above.Net issue and noticed that Glbx is taking an emergency maintenance window for tomorrow morning to upgrade router software on ALL routers (nice). I wonder if this is related since both networks use Juniper. If anyone has info to share, it would be on-topic I think :) -Scott Bethke -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Chris A. Epler Sent: Friday, January 21, 2005 1:43 PM To: nanog@merit.edu Subject: Major AboveNet problems? -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Anyone have any details on what is going on with AboveNet? Evidently something major but our support contacts didn't have a lot of details, said there'd be something out later this afternoon about it. Wondering if others are experiencing problems with them. - -- ~ /\ ~ \ / ASCII RIBBON CAMPAIGN ~ XAGAINST HTML MAIL ~ / \ -BEGIN PGP SIGNATURE- Version: GnuPG v1.2.6 (GNU/Linux) iD8DBQFB8U0/25hr1at2zS8RApGYAJ9DosyIFlaCoR/vjWj4QYJyYhcVkQCgj6Db y16tFmLYkDM/jep4Ug9t1Vs= =i27H -END PGP SIGNATURE-
RE: I want my own IPs
-Original Message-
From: Owen DeLong
{SNIP}
It's not a catch-22 at all. You submit a form with proper documentation
and justification. They ask you some questions, you submit the answers
and any additional supporting documentation. If you have fully justified
explained your need, you receive the address space.
I have to second this, it really is a simple process. I continue to hear
horror stories from people who BELIEVE that it is hard to get PI space.
Read the policy, submit the documentation that they ask for and you will do
fine. In general I really like the fear factor. Honestly I think it helps
keep overall utilization of v4 space down :)
-Scott
RE: AboveNet major backbone issues
Actually I'm not sure if it is related or not but Above.Net did have what they called a Global Maintenance window last night in order to configure MPLS. And now that I see it, they did say These changes will be transparent and will not involve routing interruptions. So it's probably something completely different. I mean who would actually jinx themselves with such a statement. :) -Scott -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Jon Lewis Sent: Saturday, June 12, 2004 1:56 PM To: [EMAIL PROTECTED] Subject: Re: AboveNet major backbone issues snip Maybe they told him. :) They don't say exactly what's broken, but Above.net did send out a notice Date: Sat, 12 Jun 2004 10:11:27 -0700 (PDT) Subject: Network Issues US Europe ~12:03 EDT June 12, 2004 snip
Re: Pitfalls of annoucing /24s
Hi Andy, Verio says they accept old class-a space at the /22 orshorter level so that isn't it. I am fairly certain you can not successfully multihome with PA class-A space.. If you are not announcing that /22 to ATT then anyone that is single-homed to ATT (or preferring them) will probably not be able to reach your /22. I ran into this problem with some 4/8 space that Level3 assigned to me by mistake. So you are dealing with more of a Policy issue rather than general prefix filter. -Scott - Original Message - From: Andy Ellifson [EMAIL PROTECTED] To: [EMAIL PROTECTED] Sent: Wednesday, October 15, 2003 6:28 PM Subject: Re: Pitfalls of annoucing /24s I have a /24 allocated to my by XO Communications in Phoenix, AZ (67.X.X.0/24). I am currently announcing it to Verio in Europe. A friend of mine that is an XO customer in Phoenix with BGP to XO can get to that address block within XO's network. But on the flip side. I also have a /22 from ATT (12.X.X.0/22). When I announce that network block to Verio in Europe (and nowhere else), only certain places get to the Europe location. Networks that prefer ATT go to ATT's network and die since the route isn't there. I don't know if I am missing something but it think it may have to do with how the network's peering/filter schemes work. I may just be walking around the problem since I am a transit customer of Verio and they normally filter. -Andy --- Phil Rosenthal [EMAIL PROTECTED] wrote: On Oct 15, 2003, at 5:24 PM, H. Michael Smith, Jr. wrote: What about the /24's that many ISPs (especially tier 2-3) are assigning to multi-homed customers? What about an IX or critical infrastructure providers that may be issued a /24 from ARIN (Policy 2001-3)? As long as it's provider assigned, and your provider announces the supernet that the /24 is from, it will still work. If you announce PI space out of the old class A space in /24's, many networks wont be able to reach you.
Re: East Coast outage?
http://www.cnn.com/2003/US/08/14/power.outage/index.html Looks like we lost the Niagara-Mohawk power grid , says it is not related to Terrorism. _Scott - Original Message - From: Aaron D. Britt [EMAIL PROTECTED] To: [EMAIL PROTECTED] Sent: Thursday, August 14, 2003 4:31 PM Subject: East Coast outage? I just lost 80 circuits (Voice and Data), across multiple states on the East Coast in the last 10 minutes. Is there a Northeast power outage or fiber cut that anyone knows about? Any info would be appreciated... -Aaron
Re: it's 1918 in bologna
Looks like a Bologon to me :) -scotty - Original Message - From: Randy Bush [EMAIL PROTECTED] To: [EMAIL PROTECTED] Sent: Thursday, July 10, 2003 6:19 AM Subject: it's 1918 in bologna laptop plugged into an internet shop's ether in bologna. i decided to trace to an address i had roam.psg.com:/etc# traceroute 139.7.30.125 traceroute to 139.7.30.125 (139.7.30.125), 64 hops max, 44 byte packets 1 192.168.10.1 (192.168.10.1) 0.256 ms 0.199 ms 0.137 ms 2 192.168.20.1 (192.168.20.1) 3.208 ms 0.929 ms 0.826 ms 3 37.255.104.1 (37.255.104.1) 4.618 ms 3.974 ms 3.799 ms 4 10.3.6.91 (10.3.6.91) 4.371 ms 3.890 ms 3.863 ms 5 10.3.7.9 (10.3.7.9) 3.904 ms 3.745 ms 3.790 ms 6 10.254.1.181 (10.254.1.181) 3.859 ms 3.925 ms 3.850 ms 7 213.140.31.133 (213.140.31.133) 8.148 ms 8.039 ms 8.150 ms 8 81.208.50.6 (81.208.50.6) 8.231 ms 8.073 ms 8.042 ms 9 mno-vcn-i1-geth3-0.telia.net (213.248.103.229) 8.155 ms 8.357 ms 8.234 ms ... note the 37. address. cute, eh? and i thought omphaloskepsis was greek! randy
Re: Router too busy???
Wow thought I was alone in the world on that one. I dont run a web server on my VXR but telnet and ssh did indeed go away. this was after about 250 days of uptime. I had been very happy with this version of IOS. I was able to access the router OOB on the console port so it wasnt too urgent, and much like you guys a reboot fixed everything. I can swear in a court of law that everything else seemed to work fine (Save the normal cef bugs and general other IOS Roulette thingys) c7200-ik2s-mz.121-5.T10.bin -Scotty - Original Message - From: Dan Armstrong [EMAIL PROTECTED] To: Mark J. Scheller [EMAIL PROTECTED]; [EMAIL PROTECTED] Sent: Tuesday, April 01, 2003 2:05 PM Subject: Re: Router too busy??? We had what I would say is exactly the same problem last Thursday around 3:00am. The traffic lights on the router were pegged solid as usual, so it appeard to be up and running, but not really passing any useful traffic. Telnetting to it was pretty much useless, although it did glimmer to work for a minute but not enough to get in and see what was going on. It did not reload itself. We power cycled it, and it was fine. Running c7200-jk9o3s-mz.122-8.T5.bin Dan. Mark J. Scheller wrote: This last Saturday (29 Mar 2003), about 4pm Eastern time my router -- for lack of a better term -- wigged out. I was able to ping to through it, however any attempt to get a TCP connection (specifically ssh and http) was almost immediately terminated. I think DNS was working fine, which would hint that UDP was getting through as well, but I won't swear to that in court. After convincing someone to drive to its location and do a power cycle, it rebooted happily and has run fine since. My mrtg graphs show that the CPU was pegged at 100% during the time it was acting up; memory was fine; traffic was (not surprisingly) very low -- and no spike prior to the CPU getting pegged. I've been running this version of IOS since it was released as a response to the flaw found in SNMP and the router has been rock solid! CPU is normally 15-20% with occasional spikes, but never for long. Memory erodes slowly, but never dropping below 20MB. Has anyone seen anything like this before? Basically, I'm wondering whether this may be an IOS bug or whether I may have hardware on its way out or whether this was some kind of new crafty DoS attack. TIA! Mark J. Scheller ([EMAIL PROTECTED])
suggestion for IBX in Washington DC
I need a recommendation for an IBX/colo environment located in the city of Washington DC itself.. I know Tyson/McLean is Paradise but looking for a good solution actually IN the DC portion of lata 236 Our main goals are Transit availability from good providers (Worldcom is a must) and good peering options -Scotty
Re: IP QoS case-studies
Pete, There was an article in the last network world about Worldcom using DiffServ in its VPN offering, and I found this article as well: http://www.netcentrex.net/news_and_events/2002_3_22_CommNews_VPN-Outsourcing Options.shtml -Scotty - Original Message - From: Pete Kruckenberg [EMAIL PROTECTED] To: [EMAIL PROTECTED] Sent: Monday, February 03, 2003 4:04 PM Subject: IP QoS case-studies I've found there's no shortage of advice and theory about the viability of IP QoS (DiffServ) in a large wide-area (converged) network. I have not had much luck with finding documentation about experiences implementing and operating such a beast. Presumably that's yet another (silent) confirmation that It Doesn't Work or There's a Better/Easier Way. Nevertheless, I'd still like to find anyone who has tried (successfully or not) to converge (ie VoIP/H.323/data) a high-speed (~ 1Gb/s) IP network and use IP QoS for what it is sold to do. White paper/presentation references or off-line conversation would be appreciated. Pete.
Re: Level3 routing issues?
BIll, - Original Message - From: Bill Woodcock [EMAIL PROTECTED] I'd agree with it. Except the herds of losers who still buy exploding crap from Vendor M don't seem to be thinning themselves out quickly dude, the Exploding Cars are so much easier to drive than the ones from Vendor L. (tic) enough. Maybe they're sexually attractive to each other, and reproduce before their stupidity kills them. That would be unfortunate. Or maybe it's just that none of this computer stuff actually matters, so exploding crap isn't actually fatal. Maybe that's it. I think it sucks that they are exploding on MY highway. With that in mind is it time yet to talk about solutions to problems like this from the network point of view? Sure its easy to put up access list's when needed but I have 100megs available to me on egress and I was trying to push 450megs. Is there anything protocol, vendor specific or otherwise that will not allow rogue machines to at will take up 100% of available resources? I know extreme networks has the concept of Max Port utilization on thier switches, will this help? Suggestions? -Scotty
Re: Worm / UDP1434
David, - Original Message - From: Freedman David [EMAIL PROTECTED] Anybody here on list using Extreme products (Summit/Alpine/Blackdiamond)? They sure don't like this traffic one bit. It causes them to not only drop traffic, but spew out every available error message under the sun... We use extremes in our core and it did not log much other than CPU issues: 01/25/2003 02:20.23 INFO:SYST task tNetTask cpu utilization is 88% PC: 80266eb4 01/25/2003 02:20.23 CRIT:SYST task tNetTask cpu utilization is 88% PC: 80266eb4 and... 01/25/2003 02:24.43 INFO:SYST task tNetTask cpu utilization is 93% PC: 80266eb4 01/25/2003 02:24.42 CRIT:SYST task tNetTask cpu utilization is 93% PC: 80266eb4 I did notice console messages while investigating the sources of the traffic, but of course have no log of them now. The switches stayed up the whole time though (yay) Also picked up some strange messages from one of the offenders: 01/25/2003 02:23.48 WARN:IPRT IGMP: snooping.c 376: updateGroupSenderListPortMask: PTAGalloc 237.189.185.65/64.237.99.79 01/25/2003 02:23.48 WARN:IPRT IGMP: snooping.c 376: updateGroupSenderListPortMask: PTAGalloc 237.137.210.243/64.237.99.79 01/25/2003 02:23.48 WARN:IPRT IGMP: snooping.c 376: updateGroupSenderListPortMask: PTAGalloc 225.134.14.67/64.237.99.79 No idea yet what that is, though I assume it is coming from the monitor port. -Scotty
Re: Level3 routing issues?
On 1/25/03 2:53 PM, Christopher L. Morrow [EMAIL PROTECTED] wrote: Keep in mind that these problems aren't from 'well behaved' hosts, and 'well behaved' hosts normally listen to ECN/tcp-window/Red/WRED classic DoS attack scenario. :( Well not everyone plays fair out there. I imagine this is built into SLA's too right? My network will be up as long as everyone is well behaved I understand the evils, but are we really at the mercy of situations like this? Of course we can firewall the common sense things ahead of time, and we can jump right in and block evil traffic when it happens, after it takes down our network but what sorts of things can we design into our networks today to help with these situations? -Scotty
Re: W32.SqlSlammer
Drew, There *IS* a difference between windows SP3 and Microsoft SQL2000 SP3.. you do know that right? -Scotty By the way, I know you guys probably don't care but McAfee is saying that if you have SP3 on your windows2000 server you will not be infected with SQLSlammer, this is absolutely NOT true, I have a box with sp3 and it IS infected. -Drew
Re: fast ethernet limits
I used to see these exact same results when I would setup Wireless pop's on towers taller than 400Ft. I was able to push the envelope a bit, however when I saw the issues that you speak of, it was when I had bad crimps, or sometimes a bad cable all together. Cat5 should be fine for this... if you figure 12ft risers you are probably cutting it close on the distance but not going over it. -Scotty - Original Message - From: Steve Rude [EMAIL PROTECTED] To: [EMAIL PROTECTED] Sent: Friday, January 10, 2003 3:01 PM Subject: fast ethernet limits Hi NANOG, Could someone please help me with a fast ethernet problem I am having. We have a POP in a 27 floor building, and have a rj45 run from the the bottom of the building (in the telco room) to the top of the building. We have cisco switches on either end and we are seeing about 5-20% packet loss on the trunk. Are we running into a distance limitation of fast ethernet, or are we suffering from another problem? I read that 328 feet is the limitation of fast ethernet. Is there anything short of getting a repeater for the cable run that I can do to boost the signal? TIA for your help. Ciao. Steve Rude
Re: Alternative to NetFlow for Measuring Traffic flows
Hi Bill, Impressive numbers but of course, slackers aside, if it was your connection and resources wouldnt you want more accurate information than just a guess? This may be effective for an IX decision if you created some sort of a map based on ALL the ASN's of the people on the peering switch.. but in most cases anyone pushing any real traffic will probably not have fine grained samples enough to determine a peering relationship based on a single AS with this method. Maybe Im wrong but hey if you are taking 200megs from any one ASN I would hope you knew about it. Interesting idea. Comments? Again it seems to iffy. What if you get a short DOS when you shift an ASN.. how much of a chump will you look like when you need that peer to be 1gbps and you hook up and its only pulling 2mpbs ? The other approach some ISPs use is to set up a trial peering session, usually using a private cross connect to measure the traffic volume and relative traffic ratios. Then both side can get an idea of the traffic before engaging in a contractual Settlement-Free Peering relationship. I like this one the best if I didnt have Netflow stat's... however I doubt everyone will allow this because of time, money, resources, security, etc. I tend to look at peering as something you need to know when to do because the data tells you so. In this industry as it stands now why would you NOT run netflow stats to give you this information? all you are doing is wasting more money paying for transit that could be offloaded to peering. And the flipside is also true.. why even worry about peering if you cant get more than a meg or two max to each AS?
Re: Spam. Again.. -- and blocking net blocks?
Ok on a serious note can we not try to solve the spam problem here? its a never ending loop (tech problem or social problem who cares.. its a problem and we all know it, be a good operator and kill anyone who wants to spam on your network). On a not-so-serious note maybe if we just assigned spammers 69.0.0.0/8 ip space the problem would take care of itself. -Scotty - Original Message - From: hostmaster [EMAIL PROTECTED] To: [EMAIL PROTECTED] Sent: Tuesday, December 10, 2002 1:00 PM Subject: Re: Spam. Again.. -- and blocking net blocks? The only solution for eliminating spam is a radical change in social behavior of those whom are causing, allowing and facilitating it. All reasonable attempts to do so have failed, mainly due to commercial interests. Thus only a primitive and for some painful interference helps. Though few want to admit it, as long as all the backbones - unanimously - are not seriously addressing this problem, and factually accept the financial consequences of cut off's, and forcefully propagate those policies to whomever is connected to them, only the hard way remains. I advocate that spews and others are tough, but apparently necessary means. The more spam, the harder the action-pack to combat it. The problem is not necessarily only Korea, Nigeria, Costa Rica, etc. We, in the US are a significant source of this activity ourselves, probably the biggest. Painfully enough we lack the initiative to set a standard for the rest for the World. best, Bert [EMAIL PROTECTED]
