Re: UUNET is not the Internet (and neither is AOL)
Hi there, What really confuses the heck out of me is that a company this size can't control/monitor their change management??. Then again not having all the facts has had everyone perplexed. It really should not confuse you. At least one year ago, there had been a Very Large company that used a single shared inbox for customer communication regarding BGP filter updates. That company had no concept of a ticketing system. Alex
Re: UUNET is not the Internet (and neither is AOL)
On Sun, 6 Oct 2002, Vicky O. Mair wrote: What really confuses the heck out of me is that a company this size can't control/monitor their change management??. Then again not having all the facts has had everyone perplexed. Actually I believe they have very good change management. Change management is about tracking, documentating, implementing consistent changes. It is possible to consistently implement something that is broken. By definition change is doing something different. But Innovation Control doesn't have the same ring to it. I like change. Change has been very good to me. But I also like informed judgement, not only by my vendor but also by me.
RE: UUNET is not the Internet (and neither is AOL)
...so what exactly did we(AOL) do to get referenced in this email thread? Cleve... Cleve Mickles Network Architecture America Online, Network Operations
Re: UUNET is not the Internet (and neither is AOL)
On 10/5/2002 at 12:30:36 +, Tim Thorne said: After reading all the stories about what supposedly happened does anyone know what really happened? Did UUNet US really do an IOS upgrade on a sizable proportion of their border routers in one go? This seems like suicide to me. What possible reason could there be for a network-wide roll out of an untested IOS apart from being in the mire already? The assumption that it was untested is probably an unfair one. Once a network reaches a certain size, it is very difficult to simulate it in a lab. Number of routes/updates, variety of packet destinations, different card revisions and layouts... heck, even statistically, you have problems. An issue that appears 5% of the time will only show up in a a 10-router test lab half the time, but in a 400-router network it'll pop up on about 20 routers and wreck your whole day. And when you're out of cash, you can't really afford to devote lots of hardware to a lab. I'm not saying that their testing procedures were correct, or that they tested the image as well as they could have... but the assumption that, if it blew up in the field it must not have been tested at all probably isn't accurate. -Dave
Re: UUNET is not the Internet (and neither is AOL)
The assumption that it was untested is probably an unfair one. Once a network reaches a certain size, it is very difficult to simulate it in a lab. Number of routes/updates, variety of packet destinations, different card revisions and layouts... heck, even statistically, you have problems. An issue that appears 5% of the time will only show up in a a 10-router test lab half the time, but in a 400-router network it'll pop up on about 20 routers and wreck your whole day. And when you're out of cash, you can't really afford to devote lots of hardware to a lab. Having a lab does help you but usually (this might be different if you are WorldCom) vendors are not too interested in fixing problems you unearth in a lab but instead only agree to raise priority of issues if their boxes fail in production. I´ve been hearing that the change in economic situation has been improving the response, but haven´t tried it personally. Not too many years back, a P2 case could take a year to get a fix where P3 rested in never-never land longer. P1 worked. Pete
Re: UUNET is not the Internet (and neither is AOL)
Hi there, What really confuses the heck out of me is that a company this size can't control/monitor their change management??. Then again not having all the facts has had everyone perplexed. later, vicky At 07:38 PM 10/5/2002 -0400, you wrote: On Sat, 5 Oct 2002, Tim Thorne wrote: After reading all the stories about what supposedly happened does anyone know what really happened? Did UUNet US really do an IOS upgrade on a sizable proportion of their border routers in one go? This seems like suicide to me. What possible reason could there be for a network-wide roll out of an untested IOS apart from being in the mire already? Corporate culture is the hardest thing to change in a company. You'll need to talk with your Worldcom account rep about what happened, and what Worldcom intends to do about it. In the past, Worldcom has not been very open or transparent when it has had network problems.
Re: UUNET is not the Internet (and neither is AOL)
Sean Donelan [EMAIL PROTECTED] wrote: But cyberspace hasn't gone senile. Those massive e-mail delays, slow Internet connections and downed e-businesses were all caused by a software upgrade that went horribly wrong at WorldCom's UUNet division, a large provider network communications. After reading all the stories about what supposedly happened does anyone know what really happened? Did UUNet US really do an IOS upgrade on a sizable proportion of their border routers in one go? This seems like suicide to me. What possible reason could there be for a network-wide roll out of an untested IOS apart from being in the mire already? Tim
Re: UUNET is not the Internet (and neither is AOL)
On Sat, 5 Oct 2002, Tim Thorne wrote: After reading all the stories about what supposedly happened does anyone know what really happened? Did UUNet US really do an IOS upgrade on a sizable proportion of their border routers in one go? This seems like suicide to me. What possible reason could there be for a network-wide roll out of an untested IOS apart from being in the mire already? Corporate culture is the hardest thing to change in a company. You'll need to talk with your Worldcom account rep about what happened, and what Worldcom intends to do about it. In the past, Worldcom has not been very open or transparent when it has had network problems.
UUNET is not the Internet (and neither is AOL)
http://www.wired.com/news/technology/0,1282,55580,00.html How and Why the Internet Broke By Michelle Delio 9:35 a.m. Oct. 4, 2002 PDT The Internet was very confused on Thursday. But cyberspace hasn't gone senile. Those massive e-mail delays, slow Internet connections and downed e-businesses were all caused by a software upgrade that went horribly wrong at WorldCom's UUNet division, a large provider network communications. [...]
