L3 Issues
Hello and Good Morning, Are there reports of L3 having issues this morning ? Starting at about 10:10 A Pacific, I started seeing huge drops in traffic at various sites, including San Diego, Houston, San Antonio, Charlotte, NC, Philadelphia, etc. Anyone seeing a similar behavior ?
Re: L3 Issues
On Mon, 01 Aug 2011 12:39:43 -0500, Khurram Khan wrote: Hello and Good Morning, Are there reports of L3 having issues this morning ? Starting at about 10:10 A Pacific, I started seeing huge drops in traffic at various sites, including San Diego, Houston, San Antonio, Charlotte, NC, Philadelphia, etc. Anyone seeing a similar behavior ? Yes we are seeing Loss from Houston To LA (not NYC to LA) dropping out in Dallas -=Tom -- Using Opera's revolutionary email client: http://www.opera.com/mail/
RE: L3 Issues
Seeing a big drop in outbound traffic on our L3 link starting about that time (we're a web host). Getting calls about sites down too; all the traces I've walked customers through so far have died on L3 networks. David > -Original Message- > From: Khurram Khan [mailto:brokenf...@gmail.com] > Sent: Monday, August 01, 2011 1:40 PM > To: nanog@nanog.org > Subject: L3 Issues > > Hello and Good Morning, > > Are there reports of L3 having issues this morning ? Starting at about > 10:10 A Pacific, I started seeing huge drops in traffic at various > sites, including San Diego, Houston, San Antonio, Charlotte, NC, > Philadelphia, etc. > Anyone seeing a similar behavior ? > > >
Re: L3 Issues
On Mon, Aug 1, 2011 at 10:39 AM, Khurram Khan wrote: > Hello and Good Morning, > > Are there reports of L3 having issues this morning ? Starting at about > 10:10 A Pacific, I started seeing huge drops in traffic at various > sites, including San Diego, Houston, San Antonio, Charlotte, NC, > Philadelphia, etc. > Anyone seeing a similar behavior ? > > Seeing similar issues here. Traffic going over Level3 seems to be dying. -- Brent Jones br...@servuhome.net
Re: L3 Issues
Yes. I am getting customer calls from people not able to reach us via Level3. --Original Message-- From: Khurram Khan To: nanog@nanog.org Subject: L3 Issues Sent: Aug 1, 2011 10:39 AM Hello and Good Morning, Are there reports of L3 having issues this morning ? Starting at about 10:10 A Pacific, I started seeing huge drops in traffic at various sites, including San Diego, Houston, San Antonio, Charlotte, NC, Philadelphia, etc. Anyone seeing a similar behavior ? Sent from my Verizon Wireless BlackBerry
RE: L3 Issues
We were seeing issues here as well, we have BGP to Level 3 down until they stabilize. We were seeing a number of sites as unreachable, but ping tests from the Level3 IP address on that interface were working. Looks like perhaps they stopped advertising our addresses or were advertising them through an incorrect path. -Original Message- From: David Hubbard [mailto:dhubb...@dino.hostasaurus.com] Sent: Monday, August 01, 2011 1:43 PM To: nanog@nanog.org Subject: RE: L3 Issues Seeing a big drop in outbound traffic on our L3 link starting about that time (we're a web host). Getting calls about sites down too; all the traces I've walked customers through so far have died on L3 networks. David > -Original Message- > From: Khurram Khan [mailto:brokenf...@gmail.com] > Sent: Monday, August 01, 2011 1:40 PM > To: nanog@nanog.org > Subject: L3 Issues > > Hello and Good Morning, > > Are there reports of L3 having issues this morning ? Starting at about > 10:10 A Pacific, I started seeing huge drops in traffic at various > sites, including San Diego, Houston, San Antonio, Charlotte, NC, > Philadelphia, etc. > Anyone seeing a similar behavior ? > > >
Re: L3 Issues
В 11:39 -0600 на 01.08.2011 (пн), Khurram Khan написа: > Hello and Good Morning, > > Are there reports of L3 having issues this morning ? Starting at about > 10:10 A Pacific, I started seeing huge drops in traffic at various > sites, including San Diego, Houston, San Antonio, Charlotte, NC, > Philadelphia, etc. > Anyone seeing a similar behavior ? > Yes, l3 in Dallas is dropping about 40% of the traffic we're sending them. -- Regards, Vasil Kolev signature.asc Description: This is a digitally signed message part
Re: L3 Issues
On Mon, 1 Aug 2011, Khurram Khan wrote: Hello and Good Morning, Are there reports of L3 having issues this morning ? Starting at about 10:10 A Pacific, I started seeing huge drops in traffic at various sites, including San Diego, Houston, San Antonio, Charlotte, NC, Philadelphia, etc. Anyone seeing a similar behavior ? Yes...about the same time here...maybe a few minutes later. I was seeing traces stopping at ae-63-63.ebr3.Atlanta2.Level3.net. Things seem to be moving again. I just unshut our L3 BGP session. Interestingly, while Level3 was having problems, our local Akamai cluster was apparently not working properly. images.apple.com resolves to our local Akamai cluster, but requests there were timing out. -- Jon Lewis, MCP :) | I route Senior Network Engineer | therefore you are Atlantic Net| _ http://www.lewis.org/~jlewis/pgp for PGP public key_
L3 Issues this Morning?
Hello All, This is my first time writing to this list and wanted to check if anyone experienced issues with L3 circuits between 12:50 ET and 13:05 ET. All our core backbone circuits re-converged and we saw a significant drop in traffic. Regards, Khurram
Re: L3 Issues this Morning?
Learn something new everyday, that's awesome. We've got several data centers between San Diego, Denver, Tulsa, Chicago, Washington DC. All of the circuit's between those POP's , and all are L3, just dropped traffic. On Thu, Sep 30, 2010 at 11:35 AM, James Smith wrote: > None Down here in Canada > > Sent from my iPhone > > On Sep 30, 2010, at 2:32 PM, Khurram Khan wrote: > >> Hello All, >> >> This is my first time writing to this list and wanted to check if >> anyone experienced issues with L3 circuits between 12:50 ET and 13:05 >> ET. All our core backbone circuits re-converged and we saw a >> significant drop in traffic. >> >> Regards, >> >> Khurram >> > -- Please use the following public key for encrypted transport. —–BEGIN PGP SIGNATURE—– Version: GnuPG v1.4.7 (GNU/Linux) iD8DBQFGZO+K6ECuipL6LVERAjSvAJsFslrQU61la+yCqD2bfDSW/OGKigCg0QOK LQTWZQDmFma6eh7onZfi99A= =LlU0 —–END PGP SIGNATURE—–
Re: L3 Issues this Morning?
Not sure if this is related but my Level 3 BGP peer went down at 3:33:57 GMT for just over 6 hours. This was in the San Jose/Santa Clara area. Their reason was an OSPF problem. Zaid On 9/30/10 10:39 AM, "Khurram Khan" wrote: > Learn something new everyday, that's awesome. We've got several data > centers between San Diego, Denver, Tulsa, Chicago, Washington DC. All > of the circuit's between those POP's , and all are L3, just dropped > traffic. > > On Thu, Sep 30, 2010 at 11:35 AM, James Smith > wrote: >> None Down here in Canada >> >> Sent from my iPhone >> >> On Sep 30, 2010, at 2:32 PM, Khurram Khan wrote: >> >>> Hello All, >>> >>> This is my first time writing to this list and wanted to check if >>> anyone experienced issues with L3 circuits between 12:50 ET and 13:05 >>> ET. All our core backbone circuits re-converged and we saw a >>> significant drop in traffic. >>> >>> Regards, >>> >>> Khurram >>> >> > >
Community troubleshooting étiquette/BCP (was: L3 Issues)
On 2011-08-01, at 1:48 PM, Jon Lewis wrote: > Things seem to be moving again. I happen to have an L3 link out of NYC, but unfortunately I don't have a list of on-net L3 prefixes in any of the reportedly affected regions, so I'm unable to provide any data from my vantage point up here. I'm sure others are in my position as well. Is there any sort of etiquette/BCP for reporting issues like this to the community? Something that might specify a method of providing information a little more specific than just specifying the affected region(s)? Maybe a list of a few affected hosts/prefixes/URLS/etc? (incidentally, images.apple.com also resolves to our local Akamai cluster)
RE: Community troubleshooting étiquette/BCP (was: L3 Issues)
I did finally see a Level 3 network event posted about this in their portal. Actually they list two separate ones: A routing issue failure between Dallas, TX and Los Angeles, CA is impacting IP services. Impacted for: 1 hour 29 minutes A loss of connectivity to servers in Dallas, TX, Tustin, CA, and Tokyo, Japan caused an impact to CDN services. The second one probably explains the Akamai issues one poster mentioned. -Original Message- From: Jason Lixfeld [mailto:ja...@lixfeld.ca] Sent: Monday, August 01, 2011 2:03 PM To: nanog@nanog.org Subject: Community troubleshooting étiquette/BCP (was: L3 Issues) On 2011-08-01, at 1:48 PM, Jon Lewis wrote: > Things seem to be moving again. I happen to have an L3 link out of NYC, but unfortunately I don't have a list of on-net L3 prefixes in any of the reportedly affected regions, so I'm unable to provide any data from my vantage point up here. I'm sure others are in my position as well. Is there any sort of etiquette/BCP for reporting issues like this to the community? Something that might specify a method of providing information a little more specific than just specifying the affected region(s)? Maybe a list of a few affected hosts/prefixes/URLS/etc? (incidentally, images.apple.com also resolves to our local Akamai cluster)
