MSRFCs versus RFCs?

[joej]

Happy Thanksgiving all!

While I don't think I'll get a response to this
question over the holidays, I thought I'd at least present
it for response post Thanksgiving.
I have a site that (along with others) has decided
to use MSExchange as their SMTP hub. One of the problems I am
seeing with this is that the current configuration allows for
any inbound domain traffic. In otherwords, the exchange server
seems to allow emails destin for any domain, then sends a None
Delivery Report to the "Mail From" party. My argument is that 
there lies an exploit with this senario. In otherwords (and those
of you that probably know where I am going with this just skip 
ahead) If I send an email to [EMAIL PROTECTED] and spoof the
Mail From as [EMAIL PROTECTED] to an Exchange Server
setup in this manor, the Exchange server will bounce an email
to the [EMAIL PROTECTED] While this is all fine and
dandy, if a person(s) decides to use this as a mailbomb method
and exploit this, its rather simple to do. So, in short I am 
aguing that
1> Mail destine for a domain not handled should be 550 Denied.
2> None Delivery Reports should only be sent for Domains Handled.
3> That a Firewall should not be doing Domain checking for SMTP

What I am at a loss for is RFCs that explicitly state this, that
is NDR for other domains, and accepting for other domains.
Perhaps I missed something or one of them. 
Anyone have to deal with this situation? 
Any suggestions on how to argue this? 
Am I perhaps missing something? 
Does Bill Gates feel that "Monopoly is just a game, I want the world!"
Just kidding.

Thanks in advance, and again Happy Thanksgiving!

-Joe

Re: Spanning tree melt down ?

[Sean Donelan]

On Wed, 27 Nov 2002, Bohdan Tashchuk wrote:
>   Dr. John Halamka, the former emergency-room physician
>   who runs Beth Israel Deaconess Medical Center's gigantic
>   computer network.
>
> Is a physician, after years of medical school, internship, residency,
> etc. the right person to be in charge of a "gigantic" computer network?
> Are arteries and veins the equivalent of fiber and CAT-5?

Do a Google search on John Halamka.

http://www.hms.harvard.edu/office/halamka.html

I suspect he knows more about networks than several posters on
this topic.

Nevertheless, it does show that "stuff happens."  I am a bit
surprised it took three days to fix things, but it wouldn't
be unprecedented.  Learning how to diagnose problems is hard
for both doctors and engineers.  Even more difficult is teaching
people how to design networks for failures.  Unfortunately,
many high availability designs make it more difficult to diagnose
and fix problems.  Sometimes you are better off with a simplier
design which fails in simple ways.

FYI: South Asia Network Operators Group (SANOG)

[Philip Smith]

This is an FYI only...

The first meeting of the South Asia Network Operators Group (SANOG) will 
take place alongside ITConference2003 (http://www.itconference.org.np/) in 
Kathmandu, Nepal on 23rd-27th January 2003.

Its aimed at ISPs and Network Operators in the South Asia region, basically 
encompassing the Indian subcontinent and neighbouring countries. SANOG 
doesn't quite yet have a website, but hopefully this isn't too far off.

This is the first attempt to create such an operators group in this part of 
Asia; if anyone is interested in supporting this new NOG, please feel free 
to contact Gaurab Raj Upadhaya <[EMAIL PROTECTED]> who is organising the event.

philip
--

Re: Risk of Internet collapse grows

[Stephen J. Wilcox]

On Wed, 27 Nov 2002, Sean Donelan wrote:

> On Wed, 27 Nov 2002 [EMAIL PROTECTED] wrote:
> > The full paper is available at:
> >
> > http://whopper.sbs.ohio-state.edu/grads/tgrubesi/survive.pdf
> >
> > password: grubesic
> >
> > It was posted on the www.cybergeography.org website with the password,
> > plus I'm sure Tony would like the feedback.
> 
> Was this paper peer reviewed ?
> 
> I'm interested in the problem, but this is not the paper.

Not -the- answer but a part of perhaps. I think the paper helps in appreciation
of the maths and processes behind the concept

> AT&T's network is the most vulnerable? While Onyx is among the least
> vulnerable?  Onyx is bankrupt, and their network is no longer in
> operation. I guess you could argue Onyx not vulnerable any more.  This
> paper starts out with some bad assumptions, such as there is one NAP in a
> city, one path between cities or the marketing maps in Boardwatch are
> meaningful.

It does mention there being more than one NAP...

Its also highlighting a point about increased resiliency through mesh redundancy
and it does acknowledge differences of scale.

> Until we figure out how to collect some meaningful starting data, we
> can't draw these types of conclusions.

And therein lies the problem! Plenty of room for theorising tho!

Steve

Re: Spanning tree melt down ?

[Scott Granados]

Just a data point here, most hospital networks and it departments are headed
by Doctor's.  They have to sign off on everything from equipment selection,
platform changes everything.  Some have a clew but admittedly its no more
than a self taught clew of the very basics from having 3 servers at home
that their kids use to game.  Others have even less.  I've personally had to
explain to heads of medical networks that their servic plan is invalid if
they continue to install quake servers on the image archive system.  

Hospital networks are really frightening and there is no best practices in
place for the most part.  There are a few exceptions but in general its
unbelievable.

- Original Message -
From: "Bohdan Tashchuk" <[EMAIL PROTECTED]>
To: <[EMAIL PROTECTED]>
Sent: Wednesday, November 27, 2002 4:39 PM
Subject: Re: Spanning tree melt down ?


>
> > Minimal social engineering plus a weak network security infrastructure
> > is a disaster waiting to happen for any major medical facility.
>
> You forgot to mention probable political infighting. And maybe
> inexperienced leadership. My favorite snippet from the article is:
>
> Dr. John Halamka, the former emergency-room physician
> who runs Beth Israel Deaconess Medical Center's gigantic
> computer network.
>
> Is a physician, after years of medical school, internship, residency,
> etc. the right person to be in charge of a "gigantic" computer network?
> Are arteries and veins the equivalent of fiber and CAT-5?
>
> I'd love to be the Cisco rep selling $3 million of new network equipment
> to this guy. What is the probability that he as ANY idea what "spanning
> tree protocol" means?
>

Re: Spanning tree melt down ?

[Bohdan Tashchuk]

> Minimal social engineering plus a weak network security infrastructure
> is a disaster waiting to happen for any major medical facility.

You forgot to mention probable political infighting. And maybe
inexperienced leadership. My favorite snippet from the article is:

Dr. John Halamka, the former emergency-room physician
who runs Beth Israel Deaconess Medical Center's gigantic
computer network.

Is a physician, after years of medical school, internship, residency,
etc. the right person to be in charge of a "gigantic" computer network?
Are arteries and veins the equivalent of fiber and CAT-5?

I'd love to be the Cisco rep selling $3 million of new network equipment
to this guy. What is the probability that he as ANY idea what "spanning
tree protocol" means?

Re: Risk of Internet collapse grows

[Mike (meuon) Harrison]

Sean: 
> the marketing maps in Boardwatch are meaningful.

ROFLMAO!!  - I needed a good laugh (I'm the NOC babysitter for this round 
 of Holidays.. looking at a  boring 4 days. )

On the semi-productive almost on-topic side: 

Are there any GOOD maps out there? I don't have a cool research
project, just lots of curiosity. 



meuon---
Music playing:   Stevie Wonder - Boogie On Reggae Woman.mp3

RE:

[Deepak Jain]

I guess that means that L3 is giving up its pure-play protestations... 

Black(3) Rocket anyone?? ;)

DJ

> -Original Message-
> From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]]On Behalf Of
> Steve Bellovin
> Sent: Wednesday, November 27, 2002 6:25 PM
> To: [EMAIL PROTECTED]
> Subject: 
> 
> 
> 
> Genuity filed for bankruptcy today as part of a deal to sell nearly all 
> of its assets to Level 3.
> 
> http://biz.yahoo.com/djus/021127/1744000711_1.html
> 
>   --Steve Bellovin, http://www.research.att.com/~smb (me)
>   http://www.wilyhacker.com ("Firewalls" book)
> 
> 
> 
> 

Re: Risk of Internet collapse grows

[Sean Donelan]

On Wed, 27 Nov 2002 [EMAIL PROTECTED] wrote:
> The full paper is available at:
>
> http://whopper.sbs.ohio-state.edu/grads/tgrubesi/survive.pdf
>
> password: grubesic
>
> It was posted on the www.cybergeography.org website with the password,
> plus I'm sure Tony would like the feedback.

Was this paper peer reviewed ?

I'm interested in the problem, but this is not the paper.

AT&T's network is the most vulnerable? While Onyx is among the least
vulnerable?  Onyx is bankrupt, and their network is no longer in
operation. I guess you could argue Onyx not vulnerable any more.  This
paper starts out with some bad assumptions, such as there is one NAP in a
city, one path between cities or the marketing maps in Boardwatch are
meaningful.

Until we figure out how to collect some meaningful starting data, we
can't draw these types of conclusions.

[no subject]

[Steve Bellovin]

Genuity filed for bankruptcy today as part of a deal to sell nearly all 
of its assets to Level 3.

http://biz.yahoo.com/djus/021127/1744000711_1.html

--Steve Bellovin, http://www.research.att.com/~smb (me)
http://www.wilyhacker.com ("Firewalls" book)

Re: Risk of Internet collapse grows

[Richard Irving]

I thought we agreed, no politics
or, =functional= public disruption strategies!

  :D

.Richard.

==
A historic moment, the very first head of "homeland security",
makes a patriotic speech at a GOP convention:

 http://www.webcorp.com/video/mcarth2a.avi

(Click Twice, QID ;)
Vadim Antonov wrote:
> 
> On Wed, 27 Nov 2002 [EMAIL PROTECTED] wrote:
> 
> > It depends which exchange point is hit.  There are a couple of buildings
> > in London which if hit would have a disasterous affect on UK and European
> > peering.
> 
> Why hit buildings when removing relatively small number of people will
> render Internet pretty much defunct.  It does not fly itself (courtesy to
> the acute case of featuritis developed by top vendors).
> 
> Feeling safer?
> 
> --vadim

Re: Broken Netmask?

[Brian]

If they werent there before, it means someone added 
them, likely the person in charge of bgp rtr config for that as #
 
    Brian
 

  - Original Message - 
  From: 
  Palmer, 
  John 
  To: [EMAIL PROTECTED] 
  Sent: Wednesday, November 27, 2002 1:06 
  PM
  Subject: Broken Netmask?
  
  
  We have a /23 
  (199.5.156/23) and seem to be having a problem with our ISP (Network Access 
  Solutions - NAS) .
   
  We do not do our own route 
  advertising - they do it for us and route the block to our 
  connection.
   
  The problem is that the 
  second part of the block (199.5.157.0 - 199.5.157.255) seems to be mis-routed 
  within the ISP's network. I think its a netmask 
  problem.
   
  The symptoms are that 
  packets get through from some destinations and not from others. Also, packets 
  sometimes fail based on port numbers (ie: if I come from x.y.z.w to 
  199.5.157.x on port 80, it works but not from x.y.z.w to 199.5.157.x on port 
  25) Furthermore, the port and source addresses that have problems changes over 
  time (ie: x.y.z.w to port 25 will work tommorrow). We dont 
  block these ports nor do they.
   
  NAS seems to be light on 
  technical talent and can't seem to solve this 
  problem.
   
  Interesting note, if you 
  check 199.5.157.1 bgp from any of the looking glass websites you get multiple 
  occurrences of NAS (and other AS numbers): ie: (from 
  AADS)
   
  BGP routing table entry 
  for 199.5.156.0/23, version 6041537
  Paths: (6 available, best 
  #1)
    Advertised to 
  peer-groups:
   AS4544-INTERNAL 
  AS4544-CLIENT AS4544-HOT-ROUTE AS4544-DATA-CENTER
   
  AS4544-CORE-CUSTOMER-FULL
    16631 16631 16631 13953 13953 13953 
  13953 13953 13953
      206.220.243.177 from 
  206.220.243.177 (66.28.1.8)
    Origin IGP, 
  metric 100, localpref 80, valid, external, best
    Community: 
  4544:300
    16631 16631 16631 13953 13953 13953 
  13953 13953 13953, (received-only)
      206.220.243.177 from 
  206.220.243.177 (66.28.1.8)
    Origin IGP, 
  metric 30802, localpref 100, valid, external
    16631 16631 16631 13953 13953 13953 
  13953 13953 13953, (received & used)
      206.204.251.196 (metric 
  113851) from 206.204.251.196 (206.204.251.196)
    Origin IGP, 
  metric 100, localpref 80, valid, internal
    Community: 
  4544:300 4544:5005
    16631 16631 16631 13953 13953 13953 
  13953 13953 13953, (received & used)
      206.204.251.206 (metric 
  43484) from 206.204.251.206 (206.204.251.206)
    Origin IGP, 
  metric 100, localpref 80, valid, internal
    Community: 
  4544:300 4544:5001
    6461 16631 16631 16631 13953 13953 
  13953 13953 13953 13953
      206.220.243.71 from 
  206.220.243.71 (207.126.96.35)
    Origin IGP, 
  metric 110, localpref 80, valid, external
    Community: 
  4544:300
    6461 16631 16631 16631 13953 13953 
  13953 13953 13953 13953, (received-only)
      206.220.243.71 from 
  206.220.243.71 (207.126.96.35)
    Origin IGP, 
  metric 1295, localpref 100, valid, external
   
  Notice the multiple 
  occurrences of 16631 and 13953. I know there are valid reasons to add multiple 
  occurrences, but can anyone tell if this is broken? These duplicate numbers 
  are new - they were'nt there before (when things 
  worked).
   
  The intermittant failure 
  problem reminds me of a time when someone's IGRP had a bad netmask somwhere. 
  I'm wondering if they may have a /24 netmask internally somewhere and not a 
  /23 like they should.
   
  Any help would be 
  appreciated. NAS doesn't seem to have a clue.
   
   

Re: Risk of Internet collapse grows

[Vadim Antonov]

On Wed, 27 Nov 2002 [EMAIL PROTECTED] wrote:

> It depends which exchange point is hit.  There are a couple of buildings 
> in London which if hit would have a disasterous affect on UK and European 
> peering.

Why hit buildings when removing relatively small number of people will 
render Internet pretty much defunct.  It does not fly itself (courtesy to 
the acute case of featuritis developed by top vendors).

Feeling safer?

--vadim

Broken Netmask?

[Palmer, John]

We
have a /23 (199.5.156/23) and seem to be having a problem with our ISP (Network
Access Solutions - NAS) .

 

We
do not do our own route advertising - they do it for us and route the block to
our connection.

 

The
problem is that the second part of the block (199.5.157.0 - 199.5.157.255)
seems to be mis-routed within the ISP's network. I think its
a netmask problem.

 

The
symptoms are that packets get through from some destinations and not from
others. Also, packets sometimes fail based on port numbers (ie: if I come from
x.y.z.w to 199.5.157.x on port 80, it works but not from x.y.z.w to 199.5.157.x
on port 25) Furthermore, the port and source addresses that have problems
changes over time (ie: x.y.z.w to port 25 will work tommorrow). We dont block these ports nor do they.

 

NAS
seems to be light on technical talent and can't seem to solve this problem.

 

Interesting
note, if you check 199.5.157.1 bgp from any of the looking glass websites you
get multiple occurrences of NAS (and other AS numbers): ie: (from AADS)

 

BGP
routing table entry for 199.5.156.0/23, version 6041537

Paths:
(6 available, best #1)

  Advertised to peer-groups:

 AS4544-INTERNAL AS4544-CLIENT
AS4544-HOT-ROUTE AS4544-DATA-CENTER

 AS4544-CORE-CUSTOMER-FULL

  16631 16631 16631 13953 13953 13953 13953
13953 13953

    206.220.243.177 from 206.220.243.177
(66.28.1.8)

  Origin IGP, metric 100, localpref 80,
valid, external, best

  Community: 4544:300

  16631 16631 16631 13953 13953 13953 13953
13953 13953, (received-only)

    206.220.243.177 from 206.220.243.177
(66.28.1.8)

  Origin IGP, metric 30802, localpref 100,
valid, external

  16631 16631 16631 13953 13953 13953 13953
13953 13953, (received & used)

    206.204.251.196 (metric 113851) from
206.204.251.196 (206.204.251.196)

  Origin IGP, metric 100, localpref 80,
valid, internal

  Community: 4544:300 4544:5005

  16631 16631 16631 13953 13953 13953 13953
13953 13953, (received & used)

    206.204.251.206 (metric 43484) from 206.204.251.206
(206.204.251.206)

  Origin IGP, metric 100, localpref 80,
valid, internal

  Community: 4544:300 4544:5001

  6461 16631 16631 16631 13953 13953 13953
13953 13953 13953

    206.220.243.71 from 206.220.243.71
(207.126.96.35)

  Origin IGP, metric 110, localpref 80,
valid, external

  Community: 4544:300

  6461 16631 16631 16631 13953 13953 13953
13953 13953 13953, (received-only)

    206.220.243.71 from 206.220.243.71
(207.126.96.35)

  Origin IGP, metric 1295, localpref 100,
valid, external

 

Notice
the multiple occurrences of 16631 and 13953. I know there are valid reasons to
add multiple occurrences, but can anyone tell if this is broken? These
duplicate numbers are new - they were'nt there before (when things worked).

 

The
intermittant failure problem reminds me of a time when someone's IGRP had a bad
netmask somwhere. I'm wondering if they may have a /24 netmask internally
somewhere and not a /23 like they should.

 

Any
help would be appreciated. NAS doesn't seem to have a clue.

 

 

updates to complexity pages

[David Meyer]

Many people have asked to to update my complexity pages
with a bit of theoretical background to to support some
of the material there (in particular, percolation
theory). So, as promised, I've updated

  http://www.maoz.com/~dmm/complexity_and_the_internet 

with a little (very little) bit of material on
percolation theory. 

Questions and comments on all of this very much welcomed,


Dave

Re: Spanning tree melt down ?

[Scott Granados]

Oh wow I worked for a company who integrated some fairly large network based
imaging systems in there and things were broken then too.

Their techs kept cutting fibers and disconnecting nodes and it took days for
them to figure out why.

- Original Message -
From: "Huff, Mark" <[EMAIL PROTECTED]>
To: "Nanog (E-mail)" <[EMAIL PROTECTED]>
Sent: Wednesday, November 27, 2002 8:19 AM
Subject: Spanning tree melt down ?


> Cisco wins...
>
> As a result of the crash, Beth Israel Deaconess plans to spend $3 million
to
> replace its entire network - creating an entire parallel set of wires and
> switches, double the capacity the medical center thought it needed.
>
>
>
>
>
>

Re: Spanning tree melt down ?

[Chris Kilbourn]

At 11:10 AM -0500 11/27/02, Eric Gauthier wrote:

I don't know which scares me more: that the hospital messed up spanning-tree
so badly (which means they likely had it turned off) that it imploded
their entire network.  Or that it took them 4 days to figure it out.


If it's anything like a former employer I used to work for, it's 
possible the physical wiring plant is owned/managed by the telco 
group which jealously guards its infrastructure from the networking 
group.

A subnet I used to work on was dropped dead for a day when a 
telco-type punched a digital phone down into the computer network 
causing a broadcast storm. It took half a day just to get the wiring 
map, then another half day to track down the offending port because 
the tech in the network group dispatched to solve the problem did not 
have a current network map.

The subnet in question contained a unix cluster with cross-mounted 
file systems that processed CAT scans for brain trauma research. The 
sysadmin of that system told me that they lost a week's worth of 
research because of that cock-up.

Hospitals are very soft targets network-wise, with hundreds, if not 
thousands of nodes of edge equipment unmanned for hours long 
stretches. On a regular basis, I saw wiring closets propped open and 
used as storage space for other equipment.

Track down a pair of scrubs, and you can walk just about anywhere in 
a hospital without being challenged as long as you look like you know 
where you are going and what you are doing.

Ten years later, there are still routers there that I can log into as 
the passwords have never been changed because the administrators of 
them were reorganized out or laid off and the equipment was orphaned.

Minimal social engineering plus a weak network security 
infrastructure is a disaster waiting to happen for any major medical 
facility.
--


Regards,

Chris Kilbourn
Founder
_
digital.forest Int'l: +1-425-483-0483
where Internet solutions grow   http://www.forest.net

Re: Spanning tree melt down ?

[alex]

> Cisco wins...
> 
> As a result of the crash, Beth Israel Deaconess plans to spend $3 million to
> replace its entire network - creating an entire parallel set of wires and
> switches, double the capacity the medical center thought it needed. 

The question is for how long that parallel network would be around before it
falls due the same problem ( dclue/dt < 0 on the part of those who run it )
manifesting itself in a different way.


Alex

Spanning tree melt down ?

[Huff, Mark]

Cisco wins...

As a result of the crash, Beth Israel Deaconess plans to spend $3 million to
replace its entire network - creating an entire parallel set of wires and
switches, double the capacity the medical center thought it needed. 








Huff, Mark.vcf
Description: Binary data

NANOG27 Call for Presentations

[Susan Harris]

 * * * * * * * * * * * * * * * * *
   
   CALL FOR PRESENTATIONS
  NANOG 27  

  GENERAL SESSION
 TUTORIALS   
   CASE STUDIES
   OPERATIONS RESEARCH POSTER SESSION

February 9 - 11, 2003

 * * * * * * * * * * * * * * * * *


The North American Network Operators' Group (NANOG) will hold its 27th
meeting February 9-11, 2003, in Phoenix, Arizona. The meeting will be
hosted by Rodney Joffe and UltraDNS. Registration opens January 2.

NANOG conferences provide a forum for the coordination and dissemination
of technical information related to large-scale (i.e.,
national/international) Internet backbone networking technologies and
operational practices.  Meetings are held three times each year, and
include two days of short presentations, plus afternoon/evening tutorial
sessions. The meetings are informal, with an emphasis
on relevance to current backbone engineering practices. NANOG
conferences draw over 500 participants, mainly consisting of engineering
staff from national service providers, and members of the research and
education community.

For more information about NANOG meetings, schedules, and logistics,
see:

 http://www.nanog.org
--

CALL FOR PRESENTATIONS

NANOG invites presentations on backbone engineering, coordination, and
research topics. Presentations should highlight issues relating to
technology already deployed or soon to be deployed in core Internet
backbones and exchange points.

Network operators are invited to present case studies detailing their
experiences with network planning and design, protocol implementation,
provisioning, automation, useful tools, traffic engineering,
troubleshooting, problems solved, and DoS.

Researchers are invited to present short (10-minute) summaries of their
work for operator feedback. Topics include routing, network performance,
statistical measurement and analysis, and protocol development and
implementation. Studies presented may be works in progress. Researchers
from academia, government, and industry are encouraged to present.

Previous NANOG meetings have included presentations on:

- Backbone traffic engineering 
- Freely available configuration, management, and measurement tools
- Inter-provider security and routing protocol authentication 
- Routing scalability in backbone infrastructures 
- Security issues for the Internet core 
- Routing policy specification and backbone router configuration 
- Building large-scale measurement infrastructure 
- Cooperative inter-provider caching 
- Alternatives to hot-potato routing 
- Recommendations on queue management and congestion avoidance 
- Experience with differentiated services 
- Inter-domain multicast deployment 
- Backbone network failure analysis 

Tutorials have covered topics such as:

- BGP troubleshooting techniques
- ISP security: real world techniques 
- IPv6 basics
- IP multicast technologies
--
HOW TO PRESENT

Submit a detailed abstract or outline describing the presentation in email
to [EMAIL PROTECTED]  The deadline for proposals is January 6,
2003.  While the majority of speaking slots will be filled by January 6, a
limited number of slots will be available after that date for topics that
are exceptionally timely and important. Submissions will be reviewed by
the NANOG Program Committee, and presenters will be notified of acceptance
by January 20, 2003. Final drafts of presentation slides are due by
January 29, and final versions February 5.

NANOG also welcomes suggestions/recommendations for tutorials, panels,
and other presentation topics.
---

Re: Risk of Internet collapse grows

[sgorman1]

The full paper is available at:

http://whopper.sbs.ohio-state.edu/grads/tgrubesi/survive.pdf

password: grubesic

It was posted on the www.cybergeography.org website with the password,
plus I'm sure Tony would like the feedback.  

Re: Spanning tree melt down ?

[Eric Gauthier]

> Anyone have any idea what really happened :
> http://www.boston.com/dailyglobe2/330/science/Got_paper_+.shtml

I know someone who worked on it, but I've avoided asking what 
really happened so I don't freak out the day the ambulence drives
me up to their emergency room :)  The other day, I did forward the article 
over to our medical school in the hopes that they might "check" their 
network for similar "issues" before something happens :)

I don't know which scares me more: that the hospital messed up spanning-tree
so badly (which means they likely had it turned off) that it imploded
their entire network.  Or that it took them 4 days to figure it out.

Eric :)

Re: Spanning tree melt down ?

[Stephen J. Wilcox]

Sure, which is why

"Within a few hours, Cisco Systems, the hospital's network provider, was loading
thousands of pounds of network equipment onto an airplane in California, bound "

seems somewhat excessive! :)

and 

"The crisis began on a Wednesday afternoon, Nov. 13, and lasted nearly four
days"

sounds like an opportunity for any consultants on nanog who have half a clue
about how to setup a LAN!

Steve

On Wed, 27 Nov 2002, Joe Abley wrote:

> 
> On Wednesday, Nov 27, 2002, at 10:25 Canada/Eastern, Stephen J. Wilcox 
> wrote:
> 
> > Hmm, well until the comment about STP it sounded like the guy did 
> > something
> > stupid on a program/database on a mainframe..
> >
> > I cant see how STP could do this or require that level of DR. Perhaps 
> > its just
> > the scapegoat for the Doc's mistake which he didnt want to admit!
> 
> If it's anything like any other layer-2 IT network meltdown I've seen, 
> it'll be some combination of:
> 
>   + no documentation on what the network looks like, apart from a large
> yellow autocad diagram which was stapled to the wall in the basement
> wiring closet in 1988
> 
>   + a scarcity of diagnostic tools, and no knowledge of how to use the
> ones that do exist
> 
>   + complete ignorance of what traffic flows when the network is not
> broken
> 
>   + a cable management standard that was first broken in 1988 and has
> only been used since to pad out RFPs
> 
>   + consideration to network design which does not extend beyond the
> reassuring knowledge that the sales guy who sold you the hardware
> is a good guy, and will look after you
> 
>   + random unauthorised insertion of hubs and switches into the fabric
> by users who got fed up of waiting eight months to get another
> ethernet port installed in their lab
> 
>   + customers who have been trained by its vendors to believe that
> certification is more important than experience
> 
>   + customers who believe in the cost benefit of a large distributed
> layer-2 network over a large distributed (largely self-documenting)
> layer-3 network.
> 
> Just another day at the office.
> 
> 
> Joe
> 
> 

Re: Spanning tree melt down ?

[Joe Abley]

On Wednesday, Nov 27, 2002, at 10:25 Canada/Eastern, Stephen J. Wilcox 
wrote:

Hmm, well until the comment about STP it sounded like the guy did 
something
stupid on a program/database on a mainframe..

I cant see how STP could do this or require that level of DR. Perhaps 
its just
the scapegoat for the Doc's mistake which he didnt want to admit!

If it's anything like any other layer-2 IT network meltdown I've seen, 
it'll be some combination of:

 + no documentation on what the network looks like, apart from a large
   yellow autocad diagram which was stapled to the wall in the basement
   wiring closet in 1988

 + a scarcity of diagnostic tools, and no knowledge of how to use the
   ones that do exist

 + complete ignorance of what traffic flows when the network is not
   broken

 + a cable management standard that was first broken in 1988 and has
   only been used since to pad out RFPs

 + consideration to network design which does not extend beyond the
   reassuring knowledge that the sales guy who sold you the hardware
   is a good guy, and will look after you

 + random unauthorised insertion of hubs and switches into the fabric
   by users who got fed up of waiting eight months to get another
   ethernet port installed in their lab

 + customers who have been trained by its vendors to believe that
   certification is more important than experience

 + customers who believe in the cost benefit of a large distributed
   layer-2 network over a large distributed (largely self-documenting)
   layer-3 network.

Just another day at the office.


Joe

Re: Spanning tree melt down ?

[Stephen J. Wilcox]

Hmm, well until the comment about STP it sounded like the guy did something
stupid on a program/database on a mainframe..

I cant see how STP could do this or require that level of DR. Perhaps its just
the scapegoat for the Doc's mistake which he didnt want to admit!

STeve

On Wed, 27 Nov 2002, Marshall Eubanks wrote:

> 
> Anyone have any idea what really happened :
> 
> http://www.boston.com/dailyglobe2/330/science/Got_paper_+.shtml
> 
> 
> It was too late. Somewhere in the web of copper wires and glass fibers that
> connects the hospital's two campuses and satellite offices, the data was stuck
> in an endless loop. Halamka's technicians shut down part of the network to
> contain it, but that created a cascade of new problems.
> 
> The entire system crashed, freezing the massive stream of information -
> prescriptions, lab tests, patient histories, Medicare bills - that shoots
> through the hospital's electronic arteries every day, touching every aspect of
> care for hundreds of patients. 
> ...
> The crisis had nothing to do with the particular software the researcher was
> using. The problem had to do with a system called ''spanning tree protocol,''
> which finds the most efficient way to move information through the network and
> blocks alternate routes to prevent data from getting stuck in a loop. The large
> volume of data the researcher was uploading happened to be the last drop that
> made the network overflow.
> 
> 
> Regards 
> Marshall Eubanks
> 

Spanning tree melt down ?

[Marshall Eubanks]

Anyone have any idea what really happened :

http://www.boston.com/dailyglobe2/330/science/Got_paper_+.shtml


It was too late. Somewhere in the web of copper wires and glass fibers that
connects the hospital's two campuses and satellite offices, the data was stuck
in an endless loop. Halamka's technicians shut down part of the network to
contain it, but that created a cascade of new problems.

The entire system crashed, freezing the massive stream of information -
prescriptions, lab tests, patient histories, Medicare bills - that shoots
through the hospital's electronic arteries every day, touching every aspect of
care for hundreds of patients. 
...
The crisis had nothing to do with the particular software the researcher was
using. The problem had to do with a system called ''spanning tree protocol,''
which finds the most efficient way to move information through the network and
blocks alternate routes to prevent data from getting stuck in a loop. The large
volume of data the researcher was uploading happened to be the last drop that
made the network overflow.


Regards 
Marshall Eubanks

Re: Risk of Internet collapse grows

[David Diaz]

Actually, I think we should all be more concerned that in most metro 
cities, there is always 1 major mega CO.  In the CO, not only do the 
RBOCs have tremendous critical technology aggregated there, but 
almost every telcom provider also locates key technology and network 
there.  Knocking out that facility would critically damage both voice 
and data in that region.  It is also a critical interconnect point 
btw operators since they all happen to be there.

Does anyone still remember that event in Atlanta where worldcom lost 
power in one of their facilities after a storm knocked out power, and 
the generator had contaminated fuel (circa 1997).  That had a major 
affect on communications and we are not even talking RBOC CO.



At 5:46 -0800 11/27/02, Eliot Lear wrote:
Yah, the abstract indicates what most of us already know.  Good 
coverage and redundancy options in urban areas; less so for rural 
areas.  Why should this shock anyone?  Imminent death of the 'net is 
*not predicted ;-)

Eliot

--

David Diaz
[EMAIL PROTECTED] [Email]
[EMAIL PROTECTED] [Pager]
www.smoton.net [Peering Site under development]
Smotons (Smart Photons) trump dumb photons

Re: Risk of Internet collapse grows

[David Diaz]

Exactly my thought.  I didnt mention it for fear of rambling.  But 
there are areas of limited redundancy, and those are larger targets. 
I used to receive "interesting" messages from rebels in S. America 
because at the time we were working with some of the larger companies 
down there by hosting their sites, and running IP connections.  An 
attack at key sites like landing centers etc could cut off a lot of 
S. America.

It was also a selling point pushed by people like PanAmSat that would 
claim it was hard to knock out a bird, and they were going direct to 
each customer.

It does seem that most hostile groups out there are more interested 
in something more gory then saying "ha we have denied the infidels 
their spam this week..."


At 13:28 + 11/27/02, [EMAIL PROTECTED] wrote:
On Wed, 27 Nov 2002, David Diaz wrote:


 I think this is old news.  There was a cover story back in 1996 time
 frame on  Mae_east.  We have to ask how likely is this with many of
 the top backbones doing private peering over local loops, how much
 damage would occur if an exchange point where hit?


It depends which exchange point is hit.  There are a couple of buildings
in London which if hit would have a disasterous affect on UK and European
peering.

What about fibre landing stations?  Are these diverse enough?  Again, most
of the transatlantic fibre (for the UK) appears to come in near Lands End.

Rich


--

David Diaz
[EMAIL PROTECTED] [Email]
[EMAIL PROTECTED] [Pager]
www.smoton.net [Peering Site under development]
Smotons (Smart Photons) trump dumb photons

Re: Risk of Internet collapse grows

[E.B. Dreger]

DRD> Date: Wed, 27 Nov 2002 07:46:26 -0500 (EST)
DRD> From: David R. Dick


DRD> Doesn't that argue for a more mesh-like architecture?

>From a resiliency standpoint, yes.  Economics, and to a certain
indirect sense capacity concerns, favor aggregation over route
diversity.  N! is not your friend.


Eddy
--
Brotsman & Dreger, Inc. - EverQuick Internet Division
Bandwidth, consulting, e-commerce, hosting, and network building
Phone: +1 (785) 865-5885 Lawrence and [inter]national
Phone: +1 (316) 794-8922 Wichita

~
Date: Mon, 21 May 2001 11:23:58 + (GMT)
From: A Trap <[EMAIL PROTECTED]>
To: [EMAIL PROTECTED]
Subject: Please ignore this portion of my mail signature.

These last few lines are a trap for address-harvesting spambots.
Do NOT send mail to <[EMAIL PROTECTED]>, or you are likely to
be blocked.

Re: Risk of Internet collapse grows

[Eliot Lear]

Yah, the abstract indicates what most of us already know.  Good coverage 
and redundancy options in urban areas; less so for rural areas.  Why 
should this shock anyone?  Imminent death of the 'net is *not predicted ;-)

Eliot

Re: Risk of Internet collapse grows

[Andrew Odlyzko]

  > On Wed, 27 Nov 2002, [EMAIL PROTECTED] wrote:

  On Wed, 27 Nov 2002, David Diaz wrote:

  > I think this is old news.  There was a cover story back in 1996 time 
  > frame on  Mae_east.  We have to ask how likely is this with many of 
  > the top backbones doing private peering over local loops, how much 
  > damage would occur if an exchange point where hit?

  It depends which exchange point is hit.  There are a couple of buildings 
  in London which if hit would have a disasterous affect on UK and European 
  peering.
   
  What about fibre landing stations?  Are these diverse enough?  Again, most
  of the transatlantic fibre (for the UK) appears to come in near Lands End.

  Rich



There is not all that much diversity in many aspects of the
telecommunications infrastructure.  There are some interesting
pages prepared by John Young at Cryptome .
It is a nice combination of public source maps and aerial photographs.

Eyeballing Telephone Switching Hubs in Downtown Manhattan (10th July 2002)
http://cryptome.org/nytel-eyeball.htm

Eyeballing US Transpacific Cable Landings (July 2002)
http://cryptome.org/cablew-eyeball.htm

Eyeballing US Transatlantic Cable Landings (7th July 2002)

Full list of Eyeballing projects
http://cryptome.org/eyeball.htm

Andrew 

Re: Risk of Internet collapse grows

[Stephen J. Wilcox]

On Wed, 27 Nov 2002 [EMAIL PROTECTED] wrote:

> 
> On Wed, 27 Nov 2002, David Diaz wrote:
> 
> > I think this is old news.  There was a cover story back in 1996 time 
> > frame on  Mae_east.  We have to ask how likely is this with many of 
> > the top backbones doing private peering over local loops, how much 
> > damage would occur if an exchange point where hit?

well recent issues have suggested an exchange can cause short term issues at
least, for a longer outage i dont think we have an example.. in the short term
flap dampening causes unreachability and circuits hitting capacity prior to a
reroute by the noc are big problems but these may be solvable (or worsened) if
an outage were to persist..

> It depends which exchange point is hit.  There are a couple of buildings 
> in London which if hit would have a disasterous affect on UK and European 
> peering.

Europe would reroute, UK would suffer.. but this comes back to the regional
effect

> What about fibre landing stations?  Are these diverse enough?  Again, most
> of the transatlantic fibre (for the UK) appears to come in near Lands End.

Hmm, I know of multiple landings including lands end... so it is diverse, but
the sheer bandwidth down one cable is very large, an outage would be noticable.

Steve

Re: Risk of Internet collapse grows

[variable]

On Wed, 27 Nov 2002, David Diaz wrote:

> I think this is old news.  There was a cover story back in 1996 time 
> frame on  Mae_east.  We have to ask how likely is this with many of 
> the top backbones doing private peering over local loops, how much 
> damage would occur if an exchange point where hit?

It depends which exchange point is hit.  There are a couple of buildings 
in London which if hit would have a disasterous affect on UK and European 
peering.
 
What about fibre landing stations?  Are these diverse enough?  Again, most
of the transatlantic fibre (for the UK) appears to come in near Lands End.

Rich

Re: Risk of Internet collapse grows

[David Diaz]

Title: Re: Risk of Internet collapse
grows


I think this is old news.  There was a cover story back in
1996 time frame on  Mae_east.  We have to ask how likely is
this with many of the top backbones doing private peering over local
loops, how much damage would occur if an exchange point where
hit?

I have 2 different questions.  1) In the current
environment, are peering circuits running fuller then in previous
years.  I ask after there has been questions on UUNET/L3 Capacity
in europe etc.  If the case is so, then an attack in one peering
location/region might cause major problems as other peering sessions
become overloaded.

2)  Wouldnt an attach on particular servers that are NOT
redundant have a more significant affect?  Are microsoft's
servers mirrored?

Just posing a scenario.

Thought this might be worth passing
on:
http://news.bbc.co.uk/2/hi/technology/2514651.stm
 
There is a recent book out called
"Linked: The New Science of Networks" which details the
potential for causing widespread Internet damage by targeting a few
hubs instead of random or widespread attacks against large numbers of
hosts.  This simulation seems to backup the author's
concerns.
 
Irwin


-- 


David Diaz
[EMAIL PROTECTED] [Email]
[EMAIL PROTECTED] [Pager]
www.smoton.net [Peering Site under development]
Smotons (Smart Photons) trump dumb photons

Re: Risk of Internet collapse grows

[David R. Dick]

Doesn't that argue for a more mesh-like architecture?

> 
> Thought this might be worth passing on:
> http://news.bbc.co.uk/2/hi/technology/2514651.stm
>  
>  
> There is a recent book out called "Linked: The New Science of Networks"
> which details the potential for causing widespread Internet damage by
> targeting a few hubs instead of random or widespread attacks against large
> numbers of hosts.  This simulation seems to backup the author's concerns.
>  
> Irwin
> 

New IPv4 blcok allocated to RIPE NCC

[leo vegoda]

Dear Colleagues,

The RIPE NCC received the IPv4 address range 82.0.0.0/8 from the
IANA in November 2002. We will begin allocating from this range 
in the near future.

You may wish to adjust any filters you have in place accordingly.

More information on the IP space administered by the RIPE NCC
can be found on our web site at:

   

Kind regards,

-- 
leo vegoda
RIPE NCC
Registration Services

Re: Risk of Internet collapse grows

[Sharif Torpis]

http://www.sciencedirect.com/science?_ob=ArticleURL&_udi=B6V1H-461XHCP
-1&_user=10&_coverDate=02%2F28%2F2003&_rdoc=4&_fmt=summary&_orig=brows
e&_srch=%23toc%235675%232003%2399978%23346577!&_cdi=5675&_sort=d&_
docanchor=&wchp=dGLbVzb-lSzBA&_acct=C50221&_version=1&_urlVersion=
0&_userid=10&md5=07d46c9a1f4d02e61db9a1aaff89514e

---
"Whenever I'm caught between two evils, I take the one I've never
tried." - Mae West

On Wed, 27 Nov 2002 03:06:30 -0500 (EST), Sean Donelan wrote:
>
>On Tue, 26 Nov 2002, Irwin Lazar wrote:
>>Thought this might be worth passing on:
>>http://news.bbc.co.uk/2/hi/technology/2514651.stm
>>
>
>Its difficult to tell what the authors have discovered since the
>paper
>won't be published for four months.  From the press release I notice
>some language which would indicate it may have the same issues other
>Internet models have predicting the impact of physical disruptions.
>
>Q: What's the difference between airline traffic and highway traffic
>during a snow storm in Chicago?
>
>A: A snowstorm in Chicago doesn't have much of an impact on highway
>traffic through Dallas.  But a snowstorm in Chicago does impact air
>traffic in Dallas.
>
>Air traffic in the US is a tightly coupled system. Air traffic is
>coordinated nationally, and passengers must make connections at
fixed
>points which are difficult to change.  Its difficult to get on a
>different
>plane heading in the general direction of your destination.
>Automotive
>traffic is loosly coupled.  Auto traffic is locally controlled and
>cars
>may be individually re-routed towards its destination at many
>different
>points.
>
>Which analogy is closer to what happens to the Internet?  Air
>traffic or
>highway traffic?  Or maybe Internet traffic is like Internet
traffic.

Re: Risk of Internet collapse grows

[Sean Donelan]

On Tue, 26 Nov 2002, Irwin Lazar wrote:
> Thought this might be worth passing on:
> http://news.bbc.co.uk/2/hi/technology/2514651.stm
> 

Its difficult to tell what the authors have discovered since the paper
won't be published for four months.  From the press release I notice
some language which would indicate it may have the same issues other
Internet models have predicting the impact of physical disruptions.

Q: What's the difference between airline traffic and highway traffic
during a snow storm in Chicago?

A: A snowstorm in Chicago doesn't have much of an impact on highway
traffic through Dallas.  But a snowstorm in Chicago does impact air
traffic in Dallas.

Air traffic in the US is a tightly coupled system. Air traffic is
coordinated nationally, and passengers must make connections at fixed
points which are difficult to change.  Its difficult to get on a different
plane heading in the general direction of your destination. Automotive
traffic is loosly coupled.  Auto traffic is locally controlled and cars
may be individually re-routed towards its destination at many different
points.

Which analogy is closer to what happens to the Internet?  Air traffic or
highway traffic?  Or maybe Internet traffic is like Internet traffic.