just got off the phone with our Level3 Service Manager - he indicated all should be well. Watching and Waiting...
On Thu, Aug 27, 2015 at 2:33 PM, Jameel Akari via Outages < outages@outages.org> wrote: > On Thu, 27 Aug 2015, Matthew Topper via Outages wrote: > > Seeing on and off in upstate NY >> >> >> Matthew Topper >> > > Confirmed upstate NY (Albany) as well. We've rerouted in the meanwhile; > were seeing lost routes and split routing back to us. > > -- > Jameel Akari > > > > >> -----Original Message----- >> From: Outages [mailto:outages-boun...@outages.org] On Behalf Of Jason >> Hellenthal via Outages >> Sent: Thursday, August 27, 2015 2:23 PM >> To: outages@outages.org >> Subject: Re: [outages] Level3/TW Telecom issues? >> >> Retracting my statement from yesterday this is still a solid issue here. >> Yes absolutely still continuing. Root Cause would be a great explanation at >> this point. >> >> On Aug 27, 2015, at 13:10, Ryan Brooks via Outages <outages@outages.org> >>> wrote: >>> >>> 24 hours of on and off packet loss, more than just Chicago. More >>> communication would be helpful. NOC can't seem to decide if it's >>> transport or routing. Any insights? >>> >>> On Aug 27, 2015, at 1:08 PM, Bobin Joseph via Outages < >>>> outages@outages.org> wrote: >>>> >>>> We just got another update saying they still have issues. >>>> >>>> >>>> IP services are affected across multiple markets. The IP NOC and IP and >>>> Transport Tier III are actively troubleshooting the issue. Operations >>>> Engineering is engaged and Field Services is on site in Chicago, IL >>>> investigating the issue. >>>> tw telecom has identified a network failure affecting your services. tw >>>> telecom tier-2 group is investigating the issue at this time. >>>> >>>> Thanks, >>>> Bobin >>>> >>>> From: Outages [mailto:outages-boun...@outages.org] On Behalf Of Aaron >>>> Gross via Outages >>>> Sent: Thursday, August 27, 2015 10:55 AM >>>> To: Bob Hancock >>>> Cc: outages@outages.org >>>> Subject: Re: [outages] Level3/TW Telecom issues? >>>> >>>> We are seeing the same things that happened yesterday on our circuits >>>> in Milwaukee and Chicago areas. Lots of packet loss and large latency >>>> through Chicago exchanges. >>>> >>>> On Thu, Aug 27, 2015 at 12:33 PM, Bob Hancock via Outages < >>>> outages@outages.org> wrote: >>>> We were seeing substantial loss through Chicago this morning but it >>>> seems to have subsided at this time. >>>> >>>> >>>> Bob Hancock >>>> >>>> >>>> On 08/27/2015 12:19 PM, Steven Fischer via Outages wrote: >>>> >>>>> we are still seeing significant packet loss to 3 of our sites in >>>>> the Midwest. >>>>> >>>>> On Thu, Aug 27, 2015 at 12:47 PM, Bobin Joseph via Outages >>>>> <outages@outages.org <mailto:outages@outages.org>> wrote: >>>>> >>>>> We got this from Level3____ >>>>> >>>>> __ __ >>>>> >>>>> Hello, >>>>> >>>>> The network outage has been resolved. >>>>> >>>>> *Event Conclusion Summary* >>>>> >>>>> Start: August 26, 2015 19:15 GMT >>>>> Stop: August 27, 2015 00:14 GMT >>>>> >>>>> Root Cause: An unknown issue in Chicago, IL was impacting IP >>>>> services. >>>>> Fix Action: Adjustments were made to resolve the issue and restore >>>>> services. >>>>> >>>>> Summary: >>>>> The IP NOC responded to an unknown issue in Chicago, IL that was >>>>> impacting IP services. Tier III Technical Support were engaged to >>>>> assist with investigations, and adjustments were made to resolve >>>>> the >>>>> issue and stabilize traffic. Monitoring continues to ensure >>>>> stability. Should further issues be experienced, please contact the >>>>> Level 3 Technical Service Center. >>>>> >>>>> >>>>> ____ >>>>> >>>>> __ __ >>>>> >>>>> Thanks,____ >>>>> >>>>> Bobin____ >>>>> >>>>> __ __ >>>>> >>>>> *From:*Outages [mailto:outages-boun...@outages.org >>>>> <mailto:outages-boun...@outages.org>] *On Behalf Of *Ryan Gelobter >>>>> via Outages >>>>> *Sent:* Thursday, August 27, 2015 9:44 AM >>>>> *To:* Rafael Possamai >>>>> *Cc:* outages@outages.org <mailto:outages@outages.org> >>>>> *Subject:* Re: [outages] Level3/TW Telecom issues?____ >>>>> >>>>> __ __ >>>>> >>>>> Was anyone able to get some useful information out of Level3 >>>>> regarding this yesterday? Also is anyone else seeing similar >>>>> issues?____ >>>>> >>>>> On Aug 26, 2015 2:40 PM, "Rafael Possamai via Outages" >>>>> <outages@outages.org <mailto:outages@outages.org>> wrote:____ >>>>> >>>>> Here's a graph showing some packet loss: http://imgur.com/KVAzetA >>>>> >>>>> <http://cp.mcafee.com/d/FZsSd38w83hJ5xyXxEVsKCMrKrhKCy-qen4TXCQrFEL >>>>> CzBNBBUSztd5YQsK9LECQrFKnuv7fICO1fomHoG2BqG_2rqI_fZvBPqJlvxdJmvD-LO >>>>> VJV1NxR1Z_HYYMM-YeoWZOWarxEVV5MsMqemrKth5dqWqJTT-l3PWApmU6CQjr1J7HF >>>>> kr8kJ6h3PtPo0eiF5KfbU1gp41jVy1KHsZ8_gKgGT2TQ1lmVWh-D00jqb2pEVsdwLQz >>>>> h0qmWJPQzZ3iWq80HVDPSs182Cy04-vWVEwBizvQ96SjhOrehG_98MC>____ >>>>> >>>>> __ __ >>>>> >>>>> __ __ >>>>> >>>>> On Wed, Aug 26, 2015 at 3:16 PM, Matt Elliott via Outages >>>>> <outages@outages.org <mailto:outages@outages.org>> wrote:____ >>>>> >>>>> Anyone else having issues on Level3? We’re seeing packet loss with >>>>> anything that traverses Chicago.____ >>>>> >>>>> __ __ >>>>> >>>>> Thanks,____ >>>>> >>>>> Matt____ >>>>> >>>>> This message and attachment(s), if any, is intended for the sole >>>>> use >>>>> of the individual and/or entity for which it is addressed, and may >>>>> contain information that is privileged, confidential and prohibited >>>>> from disclosure under applicable law. If you are not the addressee, >>>>> or authorized to receive this on behalf of the addressee, you are >>>>> hereby notified that you may not use, copy, disclose or distribute >>>>> to anyone this message or any part thereof. If you have received >>>>> this in error, please immediately advise the sender by e-mail and >>>>> delete this information and all attachments from your computer and >>>>> network. Thank you. ____ >>>>> >>>>> >>>>> _______________________________________________ >>>>> Outages mailing list >>>>> Outages@outages.org <mailto:Outages@outages.org> >>>>> https://puck.nether.net/mailman/listinfo/outages >>>>> >>>>> <http://cp.mcafee.com/d/avndy0QrhooKUqenbFI6XCQrFELCzBNd-VJ6WqbVEVs >>>>> ppudETjhvd7byrW9J6WrBTDNPX9IwjS5GSawFmGLMCSHfP_nVsSHlnUjrlDV_HYKrug >>>>> sotgvvW_fccfL3CeLsKyCUqeuhs7c6zBCXDkhjmKCHtZ_BgY-F6lK1FJcSMrhWWl6O5 >>>>> bhAgYTsSjDdqymoIToHMd9_7wrwCHIcfBisEeROHsZ8_gTlKuAvEn8lrxrW0GHsZ8_j >>>>> w09J5xcQsK6MnWhEwdbtmVWh-xFtd40lYPVXe0A1jh02vfZsQgiFhLW4zr9EVdBPdH> >>>>> ____ >>>>> >>>>> __ __ >>>>> >>>>> >>>>> _______________________________________________ >>>>> Outages mailing list >>>>> Outages@outages.org <mailto:Outages@outages.org> >>>>> https://puck.nether.net/mailman/listinfo/outages >>>>> >>>>> <http://cp.mcafee.com/d/2DRPoQ938ArhooKUqenbFI6XCQrFELCzBNd-VJ6WqbV >>>>> EVsppudETjhvd7byrW9J6WrBTDNPX9IwjS5GSawFmGLMCSHfP_nVsSHlnUjrlDV_HYK >>>>> rugsotgvvW_fccfL3CeLsKyCUqeuhs7c6zBCXDkhjmKCHtZ_BgY-F6lK1FJASMrhWWl >>>>> 6O5bhAgYTsSjDdqymoIToHMd9_7wrwCHIcfBisEeROHsZ8_gTlKuAvEn8lrxrW0GHsZ >>>>> 8_jw09J5xcQsK6MnWhEwdbtmVWh-xFtd40lYPVXe0A1jh02vfZsQgiFhLW4zr9EVdWo >>>>> Uq>____ >>>>> >>>>> >>>>> _______________________________________________ >>>>> Outages mailing list >>>>> Outages@outages.org <mailto:Outages@outages.org> >>>>> https://puck.nether.net/mailman/listinfo/outages >>>>> >>>>> <http://cp.mcafee.com/d/k-Kr41ESyMNtMQsKnjodTdETjhvd7byrZPqdQQnPhOU >>>>> OOYrhKCy-qen4TQjqdQTbLfzDSjp0DIblIl1iJlvxdJmvD-LOVJmGLMCSHfP_nVsSYw >>>>> UMWw-_R-uoovu7ctuVt5dMQsYyUeod7bdTeEyCJtdmXX_axVZicHs3jr1JwSzRQGdAa >>>>> mz8xVKVIDeqR4INpKNnwqj-f0T1dnoovaAVgtHBmVWh-xKHsZ8_gKgGT2TQ1lmVWh-D >>>>> 00jqb2pEVsdwLQzh0qmWJPQzZ3iWq80HVDPSs182Cy04-vWVEwBizvQ96SjhOrVbh4> >>>>> >>>>> >>>>> >>>>> >>>>> -- >>>>> To him who is able to keep you from falling and to present you >>>>> before his glorious presence without fault and with great joy >>>>> >>>>> >>>>> _______________________________________________ >>>>> Outages mailing list >>>>> Outages@outages.org >>>>> http://cp.mcafee.com/d/avndzhJ5xyXxEVsKC-rKrhKCy-qen4TXCQrFELCzBNBB >>>>> USztd5YQsK9LECQrFKnuv7fICO1fomHoG2BqG_2rqI_fZvBPqJlvxdJmvD-LOVJV1Nx >>>>> R1Z_HYYMM-YeoWZOWarxEVV5MsMqemrKth5dqWqJTT-l3PWApmU6CQjq9J7HFkr8kJ6 >>>>> h3PtPpesRG9pyPtyL0QDYu1K2qKMM-l9OwXnaJPQzZ3tmVWh-xsxlK5LE2GJPQzZe00 >>>>> CQm4PhOUr1vF6y0QJRrDF7W6BQQg1nPfDIU2g5d409Y_RPh1aB6_EidICzASH16Ohmp >>>>> >>>>> The information contained in this E-mail message is privileged, >>>> confidential, and may be protected from disclosure; please be aware that >>>> any unauthorized use, printing, copying, disclosure or dissemination of >>>> this communication may be subject to legal restriction or sanction. If you >>>> think that you have received this E-mail message in error, please reply to >>>> the sender. This E-mail message and any attachments have been scanned for >>>> viruses and are believed to be free of any virus or other defect that might >>>> affect any computer system into which it is received and opened. It is the >>>> responsibility, however, of the recipient to ensure that it is virus free. >>>> _______________________________________________ >>>> Outages mailing list >>>> Outages@outages.org >>>> https://puck.nether.net/mailman/listinfo/outages >>>> >>>> _______________________________________________ >>>> Outages mailing list >>>> Outages@outages.org >>>> https://puck.nether.net/mailman/listinfo/outages >>>> >>> >>> _______________________________________________ >>> Outages mailing list >>> Outages@outages.org >>> https://puck.nether.net/mailman/listinfo/outages >>> >> >> >> -- >> Jason Hellenthal >> JJH48-ARIN >> >> >> >> >> _______________________________________________ >> Outages mailing list >> Outages@outages.org >> https://puck.nether.net/mailman/listinfo/outages >> >> > -- > Jameel Akari > _______________________________________________ > Outages mailing list > Outages@outages.org > https://puck.nether.net/mailman/listinfo/outages > > -- To him who is able to keep you from falling and to present you before his glorious presence without fault and with great joy
_______________________________________________ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages