Re: [WISPA] A Ridiculous Failure of Critical Infrastructure
And in August of 2003, a tree limb in Ohio caused the biggest blackout in US history. If the operators of 100+ year technology (the Power Grid) are incapable of routing around major outages, I suspect the same lack of thinking and planning is at fault in this issue as well. It's always the It will never happen scenario that happens. Infamous last words She's unsinkable! -Original Message- From: wireless-boun...@wispa.org [mailto:wireless-boun...@wispa.org] On Behalf Of Matt Larsen - Lists Sent: Tuesday, December 01, 2009 2:54 AM To: Telecom Regulation the Internet; WISPA General List; motorola-us...@wispa.org Subject: [WISPA] A Ridiculous Failure of Critical Infrastructure Some kind of combination of failure between Charter and Qwest has left tens of thousands of people in Nebraska without Internet and has disrupted the Internet and phone services for thousands more.Right now, the outage is going on 12 hours and there is no ETA for repair in sight. The word coming down is that the outage is on a Qwest fiber, but it looks to me like both parties should be on the hot seat for not having the ability to route around the problem.There was a four hour outage on Charter a week ago that was caused by a fiber cut in Gothenburg, Nebraska. That one killed everything west of the cut, but it was small potatoes compared to this one. Is this truly the level of performance that we can expect from our major Internet backbone providers? It took me about 10 seconds to re-route my traffic to a backup provider - you would think that a couple of multimillion dollar companies would be able to sort out a problem of this nature in a reasonable amount of time. The small CLEC that I use for my backup connection had enough capacity to route around the problem and was even able to lend me a little bit after 5pm when the traffic on their network (mostly businesses) dropped off. It isn't rocket science to figure out how to route around an outage. Almost as frustrating is that there was NO news about the outages anywhere except on the social networking sites (Facebook, Twitter). One TV station in Hastings, NE put up a short story on their website, but I got more news from the tweets and FB posts that people where posting from their cell phones than I did from anywhere else. None of the network outage sites have any news about this. Could this be a harbinger of things to come? I am feeling pretty thankful right now that I have a choice in backbone providers and that I kept a second one. Diversity is a good thing, and this is a great example of why we need competition and multiple options for Internet. Matt Larsen vistabeam.com WISPA Wants You! Join today! http://signup.wispa.org/ WISPA Wireless List: wireless@wispa.org Subscribe/Unsubscribe: http://lists.wispa.org/mailman/listinfo/wireless Archives: http://lists.wispa.org/pipermail/wireless/ WISPA Wants You! Join today! http://signup.wispa.org/ WISPA Wireless List: wireless@wispa.org Subscribe/Unsubscribe: http://lists.wispa.org/mailman/listinfo/wireless Archives: http://lists.wispa.org/pipermail/wireless/
Re: [WISPA] A Ridiculous Failure of Critical Infrastructure
Mediacomm was having some bad equipment problems in Marshalltown or Des Moines a couple months ago. It was on a hand off to Sprint on some fiber shelf. It bit us twice, each on a Sunday, for most of the day. After the second episode, which apparently happened after scheduled maintenance, I am convinced the big boys decided to mess with a couple little guys. Sunday outages usually affect residential users, which are my bread and butter. Unfortunately, I didn't have enough sense to have my upstream provider supply me with a map of exactly how the traffic was routed, and the outages affected my primary connection AND my backup. Seems both Dynamic Broadband and Mediacomm hand off to Sprint here in this part of Iowa. So, even my back-up as currently configured can be at risk. Needless to say, I have a winter project to engineer something going to the east from here. Matt, I hope they get it figured out for you guys soon. If you're anything like me, all hairs are already gray. Mike At 01:54 AM 12/1/2009, you wrote: Some kind of combination of failure between Charter and Qwest has left tens of thousands of people in Nebraska without Internet and has disrupted the Internet and phone services for thousands more.Right now, the outage is going on 12 hours and there is no ETA for repair in sight. The word coming down is that the outage is on a Qwest fiber, but it looks to me like both parties should be on the hot seat for not having the ability to route around the problem.There was a four hour outage on Charter a week ago that was caused by a fiber cut in Gothenburg, Nebraska. That one killed everything west of the cut, but it was small potatoes compared to this one. Is this truly the level of performance that we can expect from our major Internet backbone providers? It took me about 10 seconds to re-route my traffic to a backup provider - you would think that a couple of multimillion dollar companies would be able to sort out a problem of this nature in a reasonable amount of time. The small CLEC that I use for my backup connection had enough capacity to route around the problem and was even able to lend me a little bit after 5pm when the traffic on their network (mostly businesses) dropped off. It isn't rocket science to figure out how to route around an outage. Almost as frustrating is that there was NO news about the outages anywhere except on the social networking sites (Facebook, Twitter). One TV station in Hastings, NE put up a short story on their website, but I got more news from the tweets and FB posts that people where posting from their cell phones than I did from anywhere else. None of the network outage sites have any news about this. Could this be a harbinger of things to come? I am feeling pretty thankful right now that I have a choice in backbone providers and that I kept a second one. Diversity is a good thing, and this is a great example of why we need competition and multiple options for Internet. Matt Larsen vistabeam.com WISPA Wants You! Join today! http://signup.wispa.org/ WISPA Wireless List: wireless@wispa.org Subscribe/Unsubscribe: http://lists.wispa.org/mailman/listinfo/wireless Archives: http://lists.wispa.org/pipermail/wireless/ WISPA Wants You! Join today! http://signup.wispa.org/ WISPA Wireless List: wireless@wispa.org Subscribe/Unsubscribe: http://lists.wispa.org/mailman/listinfo/wireless Archives: http://lists.wispa.org/pipermail/wireless/
Re: [WISPA] A Ridiculous Failure of Critical Infrastructure
MAKE news. Matt Larsen - Lists wrote: Some kind of combination of failure between Charter and Qwest has left tens of thousands of people in Nebraska without Internet and has disrupted the Internet and phone services for thousands more.Right now, the outage is going on 12 hours and there is no ETA for repair in sight. The word coming down is that the outage is on a Qwest fiber, but it looks to me like both parties should be on the hot seat for not having the ability to route around the problem.There was a four hour outage on Charter a week ago that was caused by a fiber cut in Gothenburg, Nebraska. That one killed everything west of the cut, but it was small potatoes compared to this one. Is this truly the level of performance that we can expect from our major Internet backbone providers? It took me about 10 seconds to re-route my traffic to a backup provider - you would think that a couple of multimillion dollar companies would be able to sort out a problem of this nature in a reasonable amount of time. The small CLEC that I use for my backup connection had enough capacity to route around the problem and was even able to lend me a little bit after 5pm when the traffic on their network (mostly businesses) dropped off. It isn't rocket science to figure out how to route around an outage. Almost as frustrating is that there was NO news about the outages anywhere except on the social networking sites (Facebook, Twitter). One TV station in Hastings, NE put up a short story on their website, but I got more news from the tweets and FB posts that people where posting from their cell phones than I did from anywhere else. None of the network outage sites have any news about this. Could this be a harbinger of things to come? I am feeling pretty thankful right now that I have a choice in backbone providers and that I kept a second one. Diversity is a good thing, and this is a great example of why we need competition and multiple options for Internet. Matt Larsen vistabeam.com WISPA Wants You! Join today! http://signup.wispa.org/ WISPA Wireless List: wireless@wispa.org Subscribe/Unsubscribe: http://lists.wispa.org/mailman/listinfo/wireless Archives: http://lists.wispa.org/pipermail/wireless/ -- Randy Cosby Vice President InfoWest, Inc 435-674-0165 x 2010 http://www.infowest.com/ WISPA Wants You! Join today! http://signup.wispa.org/ WISPA Wireless List: wireless@wispa.org Subscribe/Unsubscribe: http://lists.wispa.org/mailman/listinfo/wireless Archives: http://lists.wispa.org/pipermail/wireless/
Re: [WISPA] A Ridiculous Failure of Critical Infrastructure
How far away from Illinois are you? - Scott Piehn - Original Message - From: Mike m...@aweiowa.com To: WISPA General List wireless@wispa.org Sent: Tuesday, December 01, 2009 7:52 AM Subject: Re: [WISPA] A Ridiculous Failure of Critical Infrastructure Mediacomm was having some bad equipment problems in Marshalltown or Des Moines a couple months ago. It was on a hand off to Sprint on some fiber shelf. It bit us twice, each on a Sunday, for most of the day. After the second episode, which apparently happened after scheduled maintenance, I am convinced the big boys decided to mess with a couple little guys. Sunday outages usually affect residential users, which are my bread and butter. Unfortunately, I didn't have enough sense to have my upstream provider supply me with a map of exactly how the traffic was routed, and the outages affected my primary connection AND my backup. Seems both Dynamic Broadband and Mediacomm hand off to Sprint here in this part of Iowa. So, even my back-up as currently configured can be at risk. Needless to say, I have a winter project to engineer something going to the east from here. Matt, I hope they get it figured out for you guys soon. If you're anything like me, all hairs are already gray. Mike At 01:54 AM 12/1/2009, you wrote: Some kind of combination of failure between Charter and Qwest has left tens of thousands of people in Nebraska without Internet and has disrupted the Internet and phone services for thousands more.Right now, the outage is going on 12 hours and there is no ETA for repair in sight. The word coming down is that the outage is on a Qwest fiber, but it looks to me like both parties should be on the hot seat for not having the ability to route around the problem.There was a four hour outage on Charter a week ago that was caused by a fiber cut in Gothenburg, Nebraska. That one killed everything west of the cut, but it was small potatoes compared to this one. Is this truly the level of performance that we can expect from our major Internet backbone providers? It took me about 10 seconds to re-route my traffic to a backup provider - you would think that a couple of multimillion dollar companies would be able to sort out a problem of this nature in a reasonable amount of time. The small CLEC that I use for my backup connection had enough capacity to route around the problem and was even able to lend me a little bit after 5pm when the traffic on their network (mostly businesses) dropped off. It isn't rocket science to figure out how to route around an outage. Almost as frustrating is that there was NO news about the outages anywhere except on the social networking sites (Facebook, Twitter). One TV station in Hastings, NE put up a short story on their website, but I got more news from the tweets and FB posts that people where posting from their cell phones than I did from anywhere else. None of the network outage sites have any news about this. Could this be a harbinger of things to come? I am feeling pretty thankful right now that I have a choice in backbone providers and that I kept a second one. Diversity is a good thing, and this is a great example of why we need competition and multiple options for Internet. Matt Larsen vistabeam.com WISPA Wants You! Join today! http://signup.wispa.org/ WISPA Wireless List: wireless@wispa.org Subscribe/Unsubscribe: http://lists.wispa.org/mailman/listinfo/wireless Archives: http://lists.wispa.org/pipermail/wireless/ WISPA Wants You! Join today! http://signup.wispa.org/ WISPA Wireless List: wireless@wispa.org Subscribe/Unsubscribe: http://lists.wispa.org/mailman/listinfo/wireless Archives: http://lists.wispa.org/pipermail/wireless/ WISPA Wants You! Join today! http://signup.wispa.org/ WISPA Wireless List: wireless@wispa.org Subscribe/Unsubscribe: http://lists.wispa.org/mailman/listinfo/wireless Archives: http://lists.wispa.org/pipermail/wireless/
Re: [WISPA] A Ridiculous Failure of Critical Infrastructure
Scott, We are a little over 100 miles west of the Quad Cities, close to hwy 30, just east of central Iowa. Your thoughts? Mike At 08:00 AM 12/1/2009, you wrote: How far away from Illinois are you? - Scott Piehn - Original Message - From: Mike m...@aweiowa.com To: WISPA General List wireless@wispa.org Sent: Tuesday, December 01, 2009 7:52 AM Subject: Re: [WISPA] A Ridiculous Failure of Critical Infrastructure Mediacomm was having some bad equipment problems in Marshalltown or Des Moines a couple months ago. It was on a hand off to Sprint on some fiber shelf. It bit us twice, each on a Sunday, for most of the day. After the second episode, which apparently happened after scheduled maintenance, I am convinced the big boys decided to mess with a couple little guys. Sunday outages usually affect residential users, which are my bread and butter. Unfortunately, I didn't have enough sense to have my upstream provider supply me with a map of exactly how the traffic was routed, and the outages affected my primary connection AND my backup. Seems both Dynamic Broadband and Mediacomm hand off to Sprint here in this part of Iowa. So, even my back-up as currently configured can be at risk. Needless to say, I have a winter project to engineer something going to the east from here. Matt, I hope they get it figured out for you guys soon. If you're anything like me, all hairs are already gray. Mike At 01:54 AM 12/1/2009, you wrote: Some kind of combination of failure between Charter and Qwest has left tens of thousands of people in Nebraska without Internet and has disrupted the Internet and phone services for thousands more.Right now, the outage is going on 12 hours and there is no ETA for repair in sight. The word coming down is that the outage is on a Qwest fiber, but it looks to me like both parties should be on the hot seat for not having the ability to route around the problem.There was a four hour outage on Charter a week ago that was caused by a fiber cut in Gothenburg, Nebraska. That one killed everything west of the cut, but it was small potatoes compared to this one. Is this truly the level of performance that we can expect from our major Internet backbone providers? It took me about 10 seconds to re-route my traffic to a backup provider - you would think that a couple of multimillion dollar companies would be able to sort out a problem of this nature in a reasonable amount of time. The small CLEC that I use for my backup connection had enough capacity to route around the problem and was even able to lend me a little bit after 5pm when the traffic on their network (mostly businesses) dropped off. It isn't rocket science to figure out how to route around an outage. Almost as frustrating is that there was NO news about the outages anywhere except on the social networking sites (Facebook, Twitter). One TV station in Hastings, NE put up a short story on their website, but I got more news from the tweets and FB posts that people where posting from their cell phones than I did from anywhere else. None of the network outage sites have any news about this. Could this be a harbinger of things to come? I am feeling pretty thankful right now that I have a choice in backbone providers and that I kept a second one. Diversity is a good thing, and this is a great example of why we need competition and multiple options for Internet. Matt Larsen vistabeam.com -- -- WISPA Wants You! Join today! http://signup.wispa.org/ -- -- WISPA Wireless List: wireless@wispa.org Subscribe/Unsubscribe: http://lists.wispa.org/mailman/listinfo/wireless Archives: http://lists.wispa.org/pipermail/wireless/ WISPA Wants You! Join today! http://signup.wispa.org/ WISPA Wireless List: wireless@wispa.org Subscribe/Unsubscribe: http://lists.wispa.org/mailman/listinfo/wireless Archives: http://lists.wispa.org/pipermail/wireless/ WISPA Wants You! Join today! http://signup.wispa.org/ WISPA Wireless List: wireless@wispa.org Subscribe/Unsubscribe: http://lists.wispa.org/mailman/listinfo/wireless Archives: http://lists.wispa.org/pipermail/wireless/ WISPA Wants You! Join today! http://signup.wispa.org
Re: [WISPA] A Ridiculous Failure of Critical Infrastructure
Timing of this Failure of Critical Infrastructure seems suspect to Charter's bankruptcy. All existing outstanding shares have been cancelled. I wonder if Paul Allen somehow left Qwest holding the bag like he did the rest of his shareholders...just a thought. http://finance.yahoo.com/news/Charter-Communications-bw-3756327554.html?x=0; .v=1 Charter hasn't made a profit since 1999...this was inevitable. Best, Brad -Original Message- From: wireless-boun...@wispa.org [mailto:wireless-boun...@wispa.org] On Behalf Of Matt Larsen - Lists Sent: Tuesday, December 01, 2009 1:54 AM To: Telecom Regulation the Internet; WISPA General List; motorola-us...@wispa.org Subject: [WISPA] A Ridiculous Failure of Critical Infrastructure Some kind of combination of failure between Charter and Qwest has left tens of thousands of people in Nebraska without Internet and has disrupted the Internet and phone services for thousands more.Right now, the outage is going on 12 hours and there is no ETA for repair in sight. The word coming down is that the outage is on a Qwest fiber, but it looks to me like both parties should be on the hot seat for not having the ability to route around the problem.There was a four hour outage on Charter a week ago that was caused by a fiber cut in Gothenburg, Nebraska. That one killed everything west of the cut, but it was small potatoes compared to this one. Is this truly the level of performance that we can expect from our major Internet backbone providers? It took me about 10 seconds to re-route my traffic to a backup provider - you would think that a couple of multimillion dollar companies would be able to sort out a problem of this nature in a reasonable amount of time. The small CLEC that I use for my backup connection had enough capacity to route around the problem and was even able to lend me a little bit after 5pm when the traffic on their network (mostly businesses) dropped off. It isn't rocket science to figure out how to route around an outage. Almost as frustrating is that there was NO news about the outages anywhere except on the social networking sites (Facebook, Twitter). One TV station in Hastings, NE put up a short story on their website, but I got more news from the tweets and FB posts that people where posting from their cell phones than I did from anywhere else. None of the network outage sites have any news about this. Could this be a harbinger of things to come? I am feeling pretty thankful right now that I have a choice in backbone providers and that I kept a second one. Diversity is a good thing, and this is a great example of why we need competition and multiple options for Internet. Matt Larsen vistabeam.com WISPA Wants You! Join today! http://signup.wispa.org/ WISPA Wireless List: wireless@wispa.org Subscribe/Unsubscribe: http://lists.wispa.org/mailman/listinfo/wireless Archives: http://lists.wispa.org/pipermail/wireless/ WISPA Wants You! Join today! http://signup.wispa.org/ WISPA Wireless List: wireless@wispa.org Subscribe/Unsubscribe: http://lists.wispa.org/mailman/listinfo/wireless Archives: http://lists.wispa.org/pipermail/wireless/
Re: [WISPA] A Ridiculous Failure of Critical Infrastructure
We are tapped into fiber about 10 miles from Iowa on Illinois. Traffic runs to Chicago, not across sprint. Last I knew, Iowa had lots of options, but if they don't pan out, the neighbor between us youSQ might have an option was hoping to offer an option, but 100 miles is probably to far - Scott Piehn - Original Message - From: Mike m...@aweiowa.com To: WISPA General List wireless@wispa.org Sent: Tuesday, December 01, 2009 8:20 AM Subject: Re: [WISPA] A Ridiculous Failure of Critical Infrastructure Scott, We are a little over 100 miles west of the Quad Cities, close to hwy 30, just east of central Iowa. Your thoughts? Mike At 08:00 AM 12/1/2009, you wrote: How far away from Illinois are you? - Scott Piehn - Original Message - From: Mike m...@aweiowa.com To: WISPA General List wireless@wispa.org Sent: Tuesday, December 01, 2009 7:52 AM Subject: Re: [WISPA] A Ridiculous Failure of Critical Infrastructure Mediacomm was having some bad equipment problems in Marshalltown or Des Moines a couple months ago. It was on a hand off to Sprint on some fiber shelf. It bit us twice, each on a Sunday, for most of the day. After the second episode, which apparently happened after scheduled maintenance, I am convinced the big boys decided to mess with a couple little guys. Sunday outages usually affect residential users, which are my bread and butter. Unfortunately, I didn't have enough sense to have my upstream provider supply me with a map of exactly how the traffic was routed, and the outages affected my primary connection AND my backup. Seems both Dynamic Broadband and Mediacomm hand off to Sprint here in this part of Iowa. So, even my back-up as currently configured can be at risk. Needless to say, I have a winter project to engineer something going to the east from here. Matt, I hope they get it figured out for you guys soon. If you're anything like me, all hairs are already gray. Mike At 01:54 AM 12/1/2009, you wrote: Some kind of combination of failure between Charter and Qwest has left tens of thousands of people in Nebraska without Internet and has disrupted the Internet and phone services for thousands more.Right now, the outage is going on 12 hours and there is no ETA for repair in sight. The word coming down is that the outage is on a Qwest fiber, but it looks to me like both parties should be on the hot seat for not having the ability to route around the problem.There was a four hour outage on Charter a week ago that was caused by a fiber cut in Gothenburg, Nebraska. That one killed everything west of the cut, but it was small potatoes compared to this one. Is this truly the level of performance that we can expect from our major Internet backbone providers? It took me about 10 seconds to re-route my traffic to a backup provider - you would think that a couple of multimillion dollar companies would be able to sort out a problem of this nature in a reasonable amount of time. The small CLEC that I use for my backup connection had enough capacity to route around the problem and was even able to lend me a little bit after 5pm when the traffic on their network (mostly businesses) dropped off. It isn't rocket science to figure out how to route around an outage. Almost as frustrating is that there was NO news about the outages anywhere except on the social networking sites (Facebook, Twitter). One TV station in Hastings, NE put up a short story on their website, but I got more news from the tweets and FB posts that people where posting from their cell phones than I did from anywhere else. None of the network outage sites have any news about this. Could this be a harbinger of things to come? I am feeling pretty thankful right now that I have a choice in backbone providers and that I kept a second one. Diversity is a good thing, and this is a great example of why we need competition and multiple options for Internet. Matt Larsen vistabeam.com -- -- WISPA Wants You! Join today! http://signup.wispa.org/ -- -- WISPA Wireless List: wireless@wispa.org Subscribe/Unsubscribe: http://lists.wispa.org/mailman/listinfo/wireless Archives: http://lists.wispa.org/pipermail/wireless/ WISPA Wants You! Join today! http://signup.wispa.org/ WISPA Wireless List: wireless@wispa.org Subscribe/Unsubscribe: http://lists.wispa.org/mailman/listinfo/wireless Archives: http://lists.wispa.org/pipermail/wireless
Re: [WISPA] A Ridiculous Failure of Critical Infrastructure
I appreciate the gesture. Iowa does have a lot of options; just not in my area which is very rural. Qwest has fiber to the home northwest of me. I am trying to engineer another backhaul from a point there back to my tower. The wheels of progress sometimes turn slowly. Mike At 08:27 AM 12/1/2009, you wrote: We are tapped into fiber about 10 miles from Iowa on Illinois. Traffic runs to Chicago, not across sprint. Last I knew, Iowa had lots of options, but if they don't pan out, the neighbor between us youSQ might have an option was hoping to offer an option, but 100 miles is probably to far - Scott Piehn - Original Message - From: Mike m...@aweiowa.com To: WISPA General List wireless@wispa.org Sent: Tuesday, December 01, 2009 8:20 AM Subject: Re: [WISPA] A Ridiculous Failure of Critical Infrastructure Scott, We are a little over 100 miles west of the Quad Cities, close to hwy 30, just east of central Iowa. Your thoughts? Mike At 08:00 AM 12/1/2009, you wrote: How far away from Illinois are you? - Scott Piehn - Original Message - From: Mike m...@aweiowa.com To: WISPA General List wireless@wispa.org Sent: Tuesday, December 01, 2009 7:52 AM Subject: Re: [WISPA] A Ridiculous Failure of Critical Infrastructure Mediacomm was having some bad equipment problems in Marshalltown or Des Moines a couple months ago. It was on a hand off to Sprint on some fiber shelf. It bit us twice, each on a Sunday, for most of the day. After the second episode, which apparently happened after scheduled maintenance, I am convinced the big boys decided to mess with a couple little guys. Sunday outages usually affect residential users, which are my bread and butter. Unfortunately, I didn't have enough sense to have my upstream provider supply me with a map of exactly how the traffic was routed, and the outages affected my primary connection AND my backup. Seems both Dynamic Broadband and Mediacomm hand off to Sprint here in this part of Iowa. So, even my back-up as currently configured can be at risk. Needless to say, I have a winter project to engineer something going to the east from here. Matt, I hope they get it figured out for you guys soon. If you're anything like me, all hairs are already gray. Mike At 01:54 AM 12/1/2009, you wrote: Some kind of combination of failure between Charter and Qwest has left tens of thousands of people in Nebraska without Internet and has disrupted the Internet and phone services for thousands more.Right now, the outage is going on 12 hours and there is no ETA for repair in sight. The word coming down is that the outage is on a Qwest fiber, but it looks to me like both parties should be on the hot seat for not having the ability to route around the problem.There was a four hour outage on Charter a week ago that was caused by a fiber cut in Gothenburg, Nebraska. That one killed everything west of the cut, but it was small potatoes compared to this one. Is this truly the level of performance that we can expect from our major Internet backbone providers? It took me about 10 seconds to re-route my traffic to a backup provider - you would think that a couple of multimillion dollar companies would be able to sort out a problem of this nature in a reasonable amount of time. The small CLEC that I use for my backup connection had enough capacity to route around the problem and was even able to lend me a little bit after 5pm when the traffic on their network (mostly businesses) dropped off. It isn't rocket science to figure out how to route around an outage. Almost as frustrating is that there was NO news about the outages anywhere except on the social networking sites (Facebook, Twitter). One TV station in Hastings, NE put up a short story on their website, but I got more news from the tweets and FB posts that people where posting from their cell phones than I did from anywhere else. None of the network outage sites have any news about this. Could this be a harbinger of things to come? I am feeling pretty thankful right now that I have a choice in backbone providers and that I kept a second one. Diversity is a good thing, and this is a great example of why we need competition and multiple options for Internet. Matt Larsen vistabeam.com -- -- WISPA Wants You! Join today! http://signup.wispa.org/ -- -- WISPA Wireless List: wireless@wispa.org Subscribe/Unsubscribe: http://lists.wispa.org/mailman/listinfo/wireless Archives: http://lists.wispa.org/pipermail/wireless
Re: [WISPA] A Ridiculous Failure of Critical Infrastructure
This is why we have 3 different providers, with different paths out of our NOC and on different fiber pairs leaving town. Qwest had an outage here about 9 months ago that took two of my competitors completely down for 5 hours... yet we were completely unaffected. :) Travis Microserv Matt Larsen - Lists wrote: Some kind of combination of failure between Charter and Qwest has left tens of thousands of people in Nebraska without Internet and has disrupted the Internet and phone services for thousands more.Right now, the outage is going on 12 hours and there is no ETA for repair in sight. The word coming down is that the outage is on a Qwest fiber, but it looks to me like both parties should be on the hot seat for not having the ability to route around the problem.There was a four hour outage on Charter a week ago that was caused by a fiber cut in Gothenburg, Nebraska. That one killed everything west of the cut, but it was small potatoes compared to this one. Is this truly the level of performance that we can expect from our major Internet backbone providers? It took me about 10 seconds to re-route my traffic to a backup provider - you would think that a couple of multimillion dollar companies would be able to sort out a problem of this nature in a reasonable amount of time. The small CLEC that I use for my backup connection had enough capacity to route around the problem and was even able to lend me a little bit after 5pm when the traffic on their network (mostly businesses) dropped off. It isn't rocket science to figure out how to route around an outage. Almost as frustrating is that there was NO news about the outages anywhere except on the social networking sites (Facebook, Twitter). One TV station in Hastings, NE put up a short story on their website, but I got more news from the tweets and FB posts that people where posting from their cell phones than I did from anywhere else. None of the network outage sites have any news about this. Could this be a harbinger of things to come? I am feeling pretty thankful right now that I have a choice in backbone providers and that I kept a second one. Diversity is a good thing, and this is a great example of why we need competition and multiple options for Internet. Matt Larsen vistabeam.com WISPA Wants You! Join today! http://signup.wispa.org/ WISPA Wireless List: wireless@wispa.org Subscribe/Unsubscribe: http://lists.wispa.org/mailman/listinfo/wireless Archives: http://lists.wispa.org/pipermail/wireless/ WISPA Wants You! Join today! http://signup.wispa.org/ WISPA Wireless List: wireless@wispa.org Subscribe/Unsubscribe: http://lists.wispa.org/mailman/listinfo/wireless Archives: http://lists.wispa.org/pipermail/wireless/
Re: [WISPA] A Ridiculous Failure of Critical Infrastructure
This is where it would be nice if WISPs were friendly enough with each other in their area to interconnect their networks. - Mike Hammett Intelligent Computing Solutions http://www.ics-il.com -- From: Matt Larsen - Lists li...@manageisp.com Sent: Tuesday, December 01, 2009 1:54 AM To: Telecom Regulation the Internet cyberteleco...@listserv.aol.com; WISPA General List wireless@wispa.org; motorola-us...@wispa.org Subject: [WISPA] A Ridiculous Failure of Critical Infrastructure Some kind of combination of failure between Charter and Qwest has left tens of thousands of people in Nebraska without Internet and has disrupted the Internet and phone services for thousands more.Right now, the outage is going on 12 hours and there is no ETA for repair in sight. The word coming down is that the outage is on a Qwest fiber, but it looks to me like both parties should be on the hot seat for not having the ability to route around the problem.There was a four hour outage on Charter a week ago that was caused by a fiber cut in Gothenburg, Nebraska. That one killed everything west of the cut, but it was small potatoes compared to this one. Is this truly the level of performance that we can expect from our major Internet backbone providers? It took me about 10 seconds to re-route my traffic to a backup provider - you would think that a couple of multimillion dollar companies would be able to sort out a problem of this nature in a reasonable amount of time. The small CLEC that I use for my backup connection had enough capacity to route around the problem and was even able to lend me a little bit after 5pm when the traffic on their network (mostly businesses) dropped off. It isn't rocket science to figure out how to route around an outage. Almost as frustrating is that there was NO news about the outages anywhere except on the social networking sites (Facebook, Twitter). One TV station in Hastings, NE put up a short story on their website, but I got more news from the tweets and FB posts that people where posting from their cell phones than I did from anywhere else. None of the network outage sites have any news about this. Could this be a harbinger of things to come? I am feeling pretty thankful right now that I have a choice in backbone providers and that I kept a second one. Diversity is a good thing, and this is a great example of why we need competition and multiple options for Internet. Matt Larsen vistabeam.com WISPA Wants You! Join today! http://signup.wispa.org/ WISPA Wireless List: wireless@wispa.org Subscribe/Unsubscribe: http://lists.wispa.org/mailman/listinfo/wireless Archives: http://lists.wispa.org/pipermail/wireless/ WISPA Wants You! Join today! http://signup.wispa.org/ WISPA Wireless List: wireless@wispa.org Subscribe/Unsubscribe: http://lists.wispa.org/mailman/listinfo/wireless Archives: http://lists.wispa.org/pipermail/wireless/
Re: [WISPA] A Ridiculous Failure of Critical Infrastructure
Outages mailing list had one member claim it was resolved at 2:30am. Is this not so? On 12/1/09, Travis Johnson t...@ida.net wrote: This is why we have 3 different providers, with different paths out of our NOC and on different fiber pairs leaving town. Qwest had an outage here about 9 months ago that took two of my competitors completely down for 5 hours... yet we were completely unaffected. :) Travis Microserv Matt Larsen - Lists wrote: Some kind of combination of failure between Charter and Qwest has left tens of thousands of people in Nebraska without Internet and has disrupted the Internet and phone services for thousands more.Right now, the outage is going on 12 hours and there is no ETA for repair in sight. The word coming down is that the outage is on a Qwest fiber, but it looks to me like both parties should be on the hot seat for not having the ability to route around the problem.There was a four hour outage on Charter a week ago that was caused by a fiber cut in Gothenburg, Nebraska. That one killed everything west of the cut, but it was small potatoes compared to this one. Is this truly the level of performance that we can expect from our major Internet backbone providers? It took me about 10 seconds to re-route my traffic to a backup provider - you would think that a couple of multimillion dollar companies would be able to sort out a problem of this nature in a reasonable amount of time. The small CLEC that I use for my backup connection had enough capacity to route around the problem and was even able to lend me a little bit after 5pm when the traffic on their network (mostly businesses) dropped off. It isn't rocket science to figure out how to route around an outage. Almost as frustrating is that there was NO news about the outages anywhere except on the social networking sites (Facebook, Twitter). One TV station in Hastings, NE put up a short story on their website, but I got more news from the tweets and FB posts that people where posting from their cell phones than I did from anywhere else. None of the network outage sites have any news about this. Could this be a harbinger of things to come? I am feeling pretty thankful right now that I have a choice in backbone providers and that I kept a second one. Diversity is a good thing, and this is a great example of why we need competition and multiple options for Internet. Matt Larsen vistabeam.com WISPA Wants You! Join today! http://signup.wispa.org/ WISPA Wireless List: wireless@wispa.org Subscribe/Unsubscribe: http://lists.wispa.org/mailman/listinfo/wireless Archives: http://lists.wispa.org/pipermail/wireless/ WISPA Wants You! Join today! http://signup.wispa.org/ WISPA Wireless List: wireless@wispa.org Subscribe/Unsubscribe: http://lists.wispa.org/mailman/listinfo/wireless Archives: http://lists.wispa.org/pipermail/wireless/ -- Josh Luthman Office: 937-552-2340 Direct: 937-552-2343 1100 Wayne St Suite 1337 Troy, OH 45373 The secret to creativity is knowing how to hide your sources. --- Albert Einstein WISPA Wants You! Join today! http://signup.wispa.org/ WISPA Wireless List: wireless@wispa.org Subscribe/Unsubscribe: http://lists.wispa.org/mailman/listinfo/wireless Archives: http://lists.wispa.org/pipermail/wireless/
Re: [WISPA] A Ridiculous Failure of Critical Infrastructure
It was resolved about 1:30am MST. I watched the first pings start passing from my edge router and switched back over within about 10 seconds. Charter didn't call anyone until 5am, so that is the time we are using to figure our credits. I get a $40 credit on next months bill. Whoopideee d!! Matt Larsen vistabeam.com Josh Luthman wrote: Outages mailing list had one member claim it was resolved at 2:30am. Is this not so? On 12/1/09, Travis Johnson t...@ida.net wrote: This is why we have 3 different providers, with different paths out of our NOC and on different fiber pairs leaving town. Qwest had an outage here about 9 months ago that took two of my competitors completely down for 5 hours... yet we were completely unaffected. :) Travis Microserv Matt Larsen - Lists wrote: Some kind of combination of failure between Charter and Qwest has left tens of thousands of people in Nebraska without Internet and has disrupted the Internet and phone services for thousands more.Right now, the outage is going on 12 hours and there is no ETA for repair in sight. The word coming down is that the outage is on a Qwest fiber, but it looks to me like both parties should be on the hot seat for not having the ability to route around the problem.There was a four hour outage on Charter a week ago that was caused by a fiber cut in Gothenburg, Nebraska. That one killed everything west of the cut, but it was small potatoes compared to this one. Is this truly the level of performance that we can expect from our major Internet backbone providers? It took me about 10 seconds to re-route my traffic to a backup provider - you would think that a couple of multimillion dollar companies would be able to sort out a problem of this nature in a reasonable amount of time. The small CLEC that I use for my backup connection had enough capacity to route around the problem and was even able to lend me a little bit after 5pm when the traffic on their network (mostly businesses) dropped off. It isn't rocket science to figure out how to route around an outage. Almost as frustrating is that there was NO news about the outages anywhere except on the social networking sites (Facebook, Twitter). One TV station in Hastings, NE put up a short story on their website, but I got more news from the tweets and FB posts that people where posting from their cell phones than I did from anywhere else. None of the network outage sites have any news about this. Could this be a harbinger of things to come? I am feeling pretty thankful right now that I have a choice in backbone providers and that I kept a second one. Diversity is a good thing, and this is a great example of why we need competition and multiple options for Internet. Matt Larsen vistabeam.com WISPA Wants You! Join today! http://signup.wispa.org/ WISPA Wireless List: wireless@wispa.org Subscribe/Unsubscribe: http://lists.wispa.org/mailman/listinfo/wireless Archives: http://lists.wispa.org/pipermail/wireless/ WISPA Wants You! Join today! http://signup.wispa.org/ WISPA Wireless List: wireless@wispa.org Subscribe/Unsubscribe: http://lists.wispa.org/mailman/listinfo/wireless Archives: http://lists.wispa.org/pipermail/wireless/ WISPA Wants You! Join today! http://signup.wispa.org/ WISPA Wireless List: wireless@wispa.org Subscribe/Unsubscribe: http://lists.wispa.org/mailman/listinfo/wireless Archives: http://lists.wispa.org/pipermail/wireless/
Re: [WISPA] A Ridiculous Failure of Critical Infrastructure
My neighbor who is also on Charter was able to route through us, and did so during their last outage. He still has a couple of T1s through ATT that are paid through the end of December, so he ended up routing through those during this one. OSPF and properly setup costs/NAT rules is wonderful. Two months ago, power outages struck a line that cut the power to the three towers that provide redundant connections to the far eastern side of my network. The five towers on the other side re-routed out through my neighbor's network for a couple of hours until the connection was restored. I need to find someone to connect with in Casper, WY and Rawlins, WY and then I'll have full survivability out to the edges of my system. Matt Larsen vistabeam.com Mike Hammett wrote: This is where it would be nice if WISPs were friendly enough with each other in their area to interconnect their networks. - Mike Hammett Intelligent Computing Solutions http://www.ics-il.com -- From: Matt Larsen - Lists li...@manageisp.com Sent: Tuesday, December 01, 2009 1:54 AM To: Telecom Regulation the Internet cyberteleco...@listserv.aol.com; WISPA General List wireless@wispa.org; motorola-us...@wispa.org Subject: [WISPA] A Ridiculous Failure of Critical Infrastructure Some kind of combination of failure between Charter and Qwest has left tens of thousands of people in Nebraska without Internet and has disrupted the Internet and phone services for thousands more.Right now, the outage is going on 12 hours and there is no ETA for repair in sight. The word coming down is that the outage is on a Qwest fiber, but it looks to me like both parties should be on the hot seat for not having the ability to route around the problem.There was a four hour outage on Charter a week ago that was caused by a fiber cut in Gothenburg, Nebraska. That one killed everything west of the cut, but it was small potatoes compared to this one. Is this truly the level of performance that we can expect from our major Internet backbone providers? It took me about 10 seconds to re-route my traffic to a backup provider - you would think that a couple of multimillion dollar companies would be able to sort out a problem of this nature in a reasonable amount of time. The small CLEC that I use for my backup connection had enough capacity to route around the problem and was even able to lend me a little bit after 5pm when the traffic on their network (mostly businesses) dropped off. It isn't rocket science to figure out how to route around an outage. Almost as frustrating is that there was NO news about the outages anywhere except on the social networking sites (Facebook, Twitter). One TV station in Hastings, NE put up a short story on their website, but I got more news from the tweets and FB posts that people where posting from their cell phones than I did from anywhere else. None of the network outage sites have any news about this. Could this be a harbinger of things to come? I am feeling pretty thankful right now that I have a choice in backbone providers and that I kept a second one. Diversity is a good thing, and this is a great example of why we need competition and multiple options for Internet. Matt Larsen vistabeam.com WISPA Wants You! Join today! http://signup.wispa.org/ WISPA Wireless List: wireless@wispa.org Subscribe/Unsubscribe: http://lists.wispa.org/mailman/listinfo/wireless Archives: http://lists.wispa.org/pipermail/wireless/ WISPA Wants You! Join today! http://signup.wispa.org/ WISPA Wireless List: wireless@wispa.org Subscribe/Unsubscribe: http://lists.wispa.org/mailman/listinfo/wireless Archives: http://lists.wispa.org/pipermail/wireless/ WISPA Wants You! Join today! http://signup.wispa.org/ WISPA Wireless List: wireless@wispa.org Subscribe/Unsubscribe: http://lists.wispa.org/mailman/listinfo/wireless Archives: http://lists.wispa.org/pipermail/wireless/
Re: [WISPA] A Ridiculous Failure of Critical Infrastructure
We do that regularly in the Hoosier State! :) -Original Message- From: wireless-boun...@wispa.org [mailto:wireless-boun...@wispa.org] On Behalf Of Mike Hammett Sent: Tuesday, December 01, 2009 11:33 AM To: WISPA General List Subject: Re: [WISPA] A Ridiculous Failure of Critical Infrastructure This is where it would be nice if WISPs were friendly enough with each other in their area to interconnect their networks. - Mike Hammett Intelligent Computing Solutions http://www.ics-il.com -- From: Matt Larsen - Lists li...@manageisp.com Sent: Tuesday, December 01, 2009 1:54 AM To: Telecom Regulation the Internet CYBERTELECOM- l...@listserv.aol.com; WISPA General List wireless@wispa.org; motorola-us...@wispa.org Subject: [WISPA] A Ridiculous Failure of Critical Infrastructure Some kind of combination of failure between Charter and Qwest has left tens of thousands of people in Nebraska without Internet and has disrupted the Internet and phone services for thousands more. Right now, the outage is going on 12 hours and there is no ETA for repair in sight. The word coming down is that the outage is on a Qwest fiber, but it looks to me like both parties should be on the hot seat for not having the ability to route around the problem.There was a four hour outage on Charter a week ago that was caused by a fiber cut in Gothenburg, Nebraska. That one killed everything west of the cut, but it was small potatoes compared to this one. Is this truly the level of performance that we can expect from our major Internet backbone providers? It took me about 10 seconds to re-route my traffic to a backup provider - you would think that a couple of multimillion dollar companies would be able to sort out a problem of this nature in a reasonable amount of time. The small CLEC that I use for my backup connection had enough capacity to route around the problem and was even able to lend me a little bit after 5pm when the traffic on their network (mostly businesses) dropped off. It isn't rocket science to figure out how to route around an outage. Almost as frustrating is that there was NO news about the outages anywhere except on the social networking sites (Facebook, Twitter). One TV station in Hastings, NE put up a short story on their website, but I got more news from the tweets and FB posts that people where posting from their cell phones than I did from anywhere else. None of the network outage sites have any news about this. Could this be a harbinger of things to come? I am feeling pretty thankful right now that I have a choice in backbone providers and that I kept a second one. Diversity is a good thing, and this is a great example of why we need competition and multiple options for Internet. Matt Larsen vistabeam.com - --- WISPA Wants You! Join today! http://signup.wispa.org/ - --- WISPA Wireless List: wireless@wispa.org Subscribe/Unsubscribe: http://lists.wispa.org/mailman/listinfo/wireless Archives: http://lists.wispa.org/pipermail/wireless/ --- - WISPA Wants You! Join today! http://signup.wispa.org/ --- - WISPA Wireless List: wireless@wispa.org Subscribe/Unsubscribe: http://lists.wispa.org/mailman/listinfo/wireless Archives: http://lists.wispa.org/pipermail/wireless/ No virus found in this incoming message. Checked by AVG - www.avg.com Version: 8.5.426 / Virus Database: 270.14.88/2538 - Release Date: 12/01/09 07:59:00 WISPA Wants You! Join today! http://signup.wispa.org/ WISPA Wireless List: wireless@wispa.org Subscribe/Unsubscribe: http://lists.wispa.org/mailman/listinfo/wireless Archives: http://lists.wispa.org/pipermail/wireless/
Re: [WISPA] A Ridiculous Failure of Critical Infrastructure
Well... it's always easier to cooperate if you know who's 'er? Rick Harnish wrote: We do that regularly in the Hoosier State! :) -Original Message- From: wireless-boun...@wispa.org [mailto:wireless-boun...@wispa.org] On Behalf Of Mike Hammett Sent: Tuesday, December 01, 2009 11:33 AM To: WISPA General List Subject: Re: [WISPA] A Ridiculous Failure of Critical Infrastructure This is where it would be nice if WISPs were friendly enough with each other in their area to interconnect their networks. - Mike Hammett Intelligent Computing Solutions http://www.ics-il.com -- Jack Unger - President, Ask-Wi.Com, Inc. Author - Deploying License-Free Wireless WANs Serving the Broadband Wireless Industry Since 1993 www.ask-wi.com 818-227-4220 jun...@ask-wi.com Sent from my Pizzicato PluckString... WISPA Wants You! Join today! http://signup.wispa.org/ WISPA Wireless List: wireless@wispa.org Subscribe/Unsubscribe: http://lists.wispa.org/mailman/listinfo/wireless Archives: http://lists.wispa.org/pipermail/wireless/
Re: [WISPA] A Ridiculous Failure of Critical Infrastructure
I have a backup with another WISP as well, but I don't quite have enough to offer much in return. - Mike Hammett Intelligent Computing Solutions http://www.ics-il.com -- From: Rick Harnish rharn...@wispa.org Sent: Tuesday, December 01, 2009 12:01 PM To: 'WISPA General List' wireless@wispa.org Subject: Re: [WISPA] A Ridiculous Failure of Critical Infrastructure We do that regularly in the Hoosier State! :) -Original Message- From: wireless-boun...@wispa.org [mailto:wireless-boun...@wispa.org] On Behalf Of Mike Hammett Sent: Tuesday, December 01, 2009 11:33 AM To: WISPA General List Subject: Re: [WISPA] A Ridiculous Failure of Critical Infrastructure This is where it would be nice if WISPs were friendly enough with each other in their area to interconnect their networks. - Mike Hammett Intelligent Computing Solutions http://www.ics-il.com -- From: Matt Larsen - Lists li...@manageisp.com Sent: Tuesday, December 01, 2009 1:54 AM To: Telecom Regulation the Internet CYBERTELECOM- l...@listserv.aol.com; WISPA General List wireless@wispa.org; motorola-us...@wispa.org Subject: [WISPA] A Ridiculous Failure of Critical Infrastructure Some kind of combination of failure between Charter and Qwest has left tens of thousands of people in Nebraska without Internet and has disrupted the Internet and phone services for thousands more. Right now, the outage is going on 12 hours and there is no ETA for repair in sight. The word coming down is that the outage is on a Qwest fiber, but it looks to me like both parties should be on the hot seat for not having the ability to route around the problem.There was a four hour outage on Charter a week ago that was caused by a fiber cut in Gothenburg, Nebraska. That one killed everything west of the cut, but it was small potatoes compared to this one. Is this truly the level of performance that we can expect from our major Internet backbone providers? It took me about 10 seconds to re-route my traffic to a backup provider - you would think that a couple of multimillion dollar companies would be able to sort out a problem of this nature in a reasonable amount of time. The small CLEC that I use for my backup connection had enough capacity to route around the problem and was even able to lend me a little bit after 5pm when the traffic on their network (mostly businesses) dropped off. It isn't rocket science to figure out how to route around an outage. Almost as frustrating is that there was NO news about the outages anywhere except on the social networking sites (Facebook, Twitter). One TV station in Hastings, NE put up a short story on their website, but I got more news from the tweets and FB posts that people where posting from their cell phones than I did from anywhere else. None of the network outage sites have any news about this. Could this be a harbinger of things to come? I am feeling pretty thankful right now that I have a choice in backbone providers and that I kept a second one. Diversity is a good thing, and this is a great example of why we need competition and multiple options for Internet. Matt Larsen vistabeam.com - --- WISPA Wants You! Join today! http://signup.wispa.org/ - --- WISPA Wireless List: wireless@wispa.org Subscribe/Unsubscribe: http://lists.wispa.org/mailman/listinfo/wireless Archives: http://lists.wispa.org/pipermail/wireless/ --- - WISPA Wants You! Join today! http://signup.wispa.org/ --- - WISPA Wireless List: wireless@wispa.org Subscribe/Unsubscribe: http://lists.wispa.org/mailman/listinfo/wireless Archives: http://lists.wispa.org/pipermail/wireless/ No virus found in this incoming message. Checked by AVG - www.avg.com Version: 8.5.426 / Virus Database: 270.14.88/2538 - Release Date: 12/01/09 07:59:00 WISPA Wants You! Join today! http://signup.wispa.org/ WISPA Wireless List: wireless@wispa.org Subscribe/Unsubscribe: http://lists.wispa.org/mailman/listinfo/wireless Archives: http://lists.wispa.org/pipermail/wireless/ WISPA Wants You! Join today! http://signup.wispa.org/ WISPA Wireless List: wireless@wispa.org Subscribe
[WISPA] A Ridiculous Failure of Critical Infrastructure
Some kind of combination of failure between Charter and Qwest has left tens of thousands of people in Nebraska without Internet and has disrupted the Internet and phone services for thousands more.Right now, the outage is going on 12 hours and there is no ETA for repair in sight. The word coming down is that the outage is on a Qwest fiber, but it looks to me like both parties should be on the hot seat for not having the ability to route around the problem.There was a four hour outage on Charter a week ago that was caused by a fiber cut in Gothenburg, Nebraska. That one killed everything west of the cut, but it was small potatoes compared to this one. Is this truly the level of performance that we can expect from our major Internet backbone providers? It took me about 10 seconds to re-route my traffic to a backup provider - you would think that a couple of multimillion dollar companies would be able to sort out a problem of this nature in a reasonable amount of time. The small CLEC that I use for my backup connection had enough capacity to route around the problem and was even able to lend me a little bit after 5pm when the traffic on their network (mostly businesses) dropped off. It isn't rocket science to figure out how to route around an outage. Almost as frustrating is that there was NO news about the outages anywhere except on the social networking sites (Facebook, Twitter). One TV station in Hastings, NE put up a short story on their website, but I got more news from the tweets and FB posts that people where posting from their cell phones than I did from anywhere else. None of the network outage sites have any news about this. Could this be a harbinger of things to come? I am feeling pretty thankful right now that I have a choice in backbone providers and that I kept a second one. Diversity is a good thing, and this is a great example of why we need competition and multiple options for Internet. Matt Larsen vistabeam.com WISPA Wants You! Join today! http://signup.wispa.org/ WISPA Wireless List: wireless@wispa.org Subscribe/Unsubscribe: http://lists.wispa.org/mailman/listinfo/wireless Archives: http://lists.wispa.org/pipermail/wireless/
