Re: cooling door

[vijay gill]

On Wed, Apr 2, 2008 at 3:06 AM, <[EMAIL PROTECTED]> wrote:

>
>
> > I doubt we'll ever see the day when running gigabit across
> > town becomes cost effective when compared to running gigabit
> > to the other end of your server room/cage/whatever.
>
> You show me the ISP with the majority of their userbase located
> at the other end of their server room, and I'll concede the argument.
>
> Last time I looked the eyeballs were across town so I already have
> to deliver my gigabit feed across town. My theory is that you can
> achieve some scaling advantages by delivering it from multiple locations
> instead of concentrating one end of that gigabit feed in a big blob
> data center where the cooling systems will fail within an hour or two
> of a major power systems failure.


It might be worth the effort to actually operate a business with real
datacenters and customers before going off with these homilies. Experience
says that for every transaction sent to the user, there are a multiplicity
of transactions on the backend that need to occur. This is why the bandwidth
into a datacenter is often 100x smaller than the bandwidth inside the
datacenter.

Communication within a rack, communication within a cluster, communication
within a colo and then communication within a campus are different than
communication with a user.

/vijay



>
>
> --Michael Dillon
>

Re: cooling door

[vijay gill]

On Mon, Mar 31, 2008 at 8:24 AM, <[EMAIL PROTECTED]> wrote:

>
> > Here is a little hint - most distributed applications in
> > traditional jobsets, tend to work best when they are close
> > together. Unless you can map those jobsets onto truly
> > partitioned algorithms that work on local copy, this is a
> > _non starter_.
>
> Let's make it simple and say it in plain English. The users
> of services have made the decision that it is "good enough"
> to be a user of a service hosted in a data center that is
> remote from the client. Remote means in another building in
> the same city, or in another city.


Try reading for comprehension. The users of services have made the decision
that it is good enough to be a user of a service hosted in a datacenter, and
thanks to the wonders of AJAX and pipelining, you can even get snappy
performance. What the users haven't signed up for is the massive amounts of
scatter gathers that happen _behind_ the front end. Eg, I click on a web
page to log in. The login process then kicks off a few authentication
sessions with servers located halfway around the world. Then you do the data
gathering, 2 phase locks, distributed file systems with the masters and lock
servers all over the place. Your hellish user experience, let me SHOW YOU
IT.


>
>
> Now, given that context, many of these "good enough" applications
> will run just fine if the "data center" is no longer in one
> physical location, but distributed across many. Of course,
> as you point out, one should not be stupid when designing such
> distributed data centers or when setting up the applications
> in them.



Other than that minor handwaving, we are all good. Turns out that desining
such distributed datacenters and setting up applications that you just
handwaved away is a bit harder than it looks. I eagerly await papers on
distributed database transactions with cost estimates for a distributed
datacenter model vs. a traditional model.


>
>
> I would assume that every data center has local storage available
> using some protocol like iSCSI and probably over a separate network
> from the external client access. That right there solves most of
> your problems of traditional jobsets. And secondly, I am not suggesting
> that everybody should shut down big data centers or that every
> application
> should be hosted across several of these distributed data centers.


See above. That right there doesn't quite solve most of the problems of
traditional jobsets but its kind of hard to hear with the wind in my ears.


>
> There will always be some apps that need centralised scaling. But
> there are many others that can scale in a distributed manner, or
> at least use distributed mirrors in a failover scenario.


Many many others indeed.

>
>
> > No matter how much optical technology you have, it will tend
> > to be more expensive to run, have higher failure rates, and
> > use more power, than simply running fiber or copper inside
> > your datacenter. There is a reason most people, who are
> > backed up by sober accountants, tend to cluster stuff under one roof.
>
> Frankly I don't understand this kind of statement. It seems
> obvious to me that high-speed metro fibre exists and corporate
> IT people already have routers and switches and servers in the
> building, connected to the metro fiber. Also, the sober accountants
> do tend to agree with spending money on backup facilities to
> avoid the risk of single points of failure. Why should company A
> operate two data centers, and company B operate two data centers,
> when they could outsource it all to ISP X running one data center
> in each of the two locations (Company A and Company B).


I guess I can try to make it clearer by example: look at the cross-sectional
bandwidth availability of a datacenter, now compare and contrast what it
would take to pull it apart by a few tens of miles and conduct the cost
comparison.

/vijay


>
> In addition, there is a trend to commoditize the whole data center.
> Amazon EC2 and S3 is not the only example of a company who does
> not offer any kind of colocation, but you can host your apps out
> of their data centers. I believe that this trend will pick up
> steam and that as the corporate market begins to accept running
> virtual servers on top of a commodity infrastructure, there is
> an opportunity for network providers to branch out and not only
> be specialists in the big consolidated data centers, but also
> in running many smaller data centers that are linked by fast metro
> fiber.
>
> --Michael Dillon
>

Re: cooling door

[vijay gill]

On Sat, Mar 29, 2008 at 3:04 PM, Frank Coluccio <[EMAIL PROTECTED]>
wrote:

>
> Michael Dillon is spot on when he states the following (quotation below),
> although he could have gone another step in suggesting how the distance
> insensitivity of fiber could be further leveraged:


Dillon is not only not spot on, dillon is quite a bit away from being spot
on. Read on.

>
>
> >The high speed fibre in Metro Area Networks will tie it all together
> >with the result that for many applications, it won't matter where
> >the servers are.
>
> In fact, those same servers, and a host of other storage and network
> elements,
> can be returned to the LAN rooms and closets of most commercial buildings
> from
> whence they originally came prior to the large-scale data center
> consolidations
> of the current millennium, once organizations decide to free themselves of
> the
> 100-meter constraint imposed by UTP-based LAN hardware and replace those
> LANs
> with collapsed fiber backbone designs that attach to remote switches
> (which could
> be either in-building or remote), instead of the minimum two switches on
> every
> floor that has become customary today.


Here is a little hint - most distributed applications in traditional
jobsets, tend to work best when they are close together. Unless you can map
those jobsets onto truly partitioned algorithms that work on local copy,
this is a _non starter_.

>
>
> We often discuss the empowerment afforded by optical technology, but we've
> barely
> scratched the surface of its ability to effect meaningful architectural
> changes.


No matter how much optical technology you have, it will tend to be more
expensive to run, have higher failure rates, and use more power, than simply
running fiber or copper inside your datacenter. There is a reason most
people, who are backed up by sober accountants, tend to cluster stuff under
one roof.

>
> The earlier prospects of creating consolidated data centers were once
> near-universally considered timely and efficient, and they still are in
> many
> respects. However, now that the problems associated with a/c and power
> have
> entered into the calculus, some data center design strategies are
> beginning to
> look more like anachronisms that have been caught in a whip-lash of
> rapidly
> shifting conditions, and in a league with the constraints that are imposed
> by the
> now-seemingly-obligatory 100-meter UTP design.



Frank, lets assume we have abundant dark fiber, and a 800 strand ribbon
fiber cable costs the same as a utp run. Can you get me some quotes from a
few folks about terminating and patching 800 strands x2?

/vijay


>
>
> Frank A. Coluccio
> DTI Consulting Inc.
> 212-587-8150 Office
> 347-526-6788 Mobile
>
> On Sat Mar 29 13:57 ,  sent:
>
> >
> >> Can someone please, pretty please with sugar on top, explain
> >> the point behind high power density?
> >
> >It allows you to market your operation as a "data center". If
> >you spread it out to reduce power density, then the logical
> >conclusion is to use multiple physical locations. At that point
> >you are no longer centralized.
> >
> >In any case, a lot of people are now questioning the traditional
> >data center model from various angles. The time is ripe for a
> >paradigm change. My theory is that the new paradigm will be centrally
> >managed, because there is only so much expertise to go around. But
> >the racks will be physically distributed, in virtually every office
> >building, because some things need to be close to local users. The
> >high speed fibre in Metro Area Networks will tie it all together
> >with the result that for many applications, it won't matter where
> >the servers are. Note that the Google MapReduce, Amazon EC2, Haddoop
> >trend will make it much easier to place an application without
> >worrying about the exact locations of the physical servers.
> >
> >Back in the old days, small ISPs set up PoPs by finding a closet
> >in the back room of a local store to set up modem banks. In the 21st
> >century folks will be looking for corporate data centers with room
> >for a rack or two of multicore CPUs running XEN, and Opensolaris
> >SANs running ZFS/raidz providing iSCSI targets to the XEN VMs.
> >
> >--Michael Dillon
>
>
>
>
>

Re: rack power question

[vijay gill]

On Sun, Mar 23, 2008 at 2:15 PM, <[EMAIL PROTECTED]> wrote:

>
> Given that power and HVAC are such key issues in building
> big datacenters, and that fiber to the office is now a reality
> virtually everywhere, one wonders why someone doesn't start
> building out distributed data centers. Essentially, you put
> mini data centers in every office building, possibly by
> outsourcing the enterprise data centers. Then, you have a
> more tractable power and HVAC problem. You still need to
> scale things but it since each data center is roughly comparable
> in size it is a lot easier than trying to build out one
> big data center.
>

Latency matters. Also, multiple small data centers will be more expensive
than a few big ones, especially if you are planning on average load vs peak
load heat rejection models.


>
> If you move all the entreprise services onto virtual servers
> then you can free up space for colo/hosting services.


There is no such thing in my experience. You free up a few thousand cores,
they get consumed by the next lower priority project that was sitting around
waiting on cpu.

>
>
> You can even still sell to bulk customers because few will
> complain that they have to deliver equipment to three
> dara centers, one two blocks west, and another three blocks
> north. X racks spread over 3 locations will work for everyone
> except people who need the physical proximity for clustering
> type applications.
>

Racks spread over n locations that aren't within a campus will be more
expensive to connect.

/vijay


>
> --Michael Dillon
>

Re: Curious question on hop identity...

[vijay gill]

Joseph Jackson wrote:

I'm pretty new to the networking world.  While I don't run a huge and
complex network in a service provider market. We're just an enterprise
network.  I have read a lot of useful info about networking from the
nanog list. But I do have to say that when I speak to the designers and
such at larger companies and I mention NANOG most of them brush it off
and say "The NANOG people are the past and what they have to say doesn't
matter anymore".  
That's the general feel I get from others when it concerns NANOG. 


Designers and such at larger companies indeed. Any specifics on which ones?

/vijay

Re: Boeing's Connexion announcement

[vijay gill]

Owen DeLong wrote:
This may be a nit, but, you will _NEVER_ see AC power at any, let alone 
all of

the seats.  Seat power that works with the iGo system is DC and is not
conventional 110 AC.



Is this your final answer? I've used AC power in lufthansa business 
class. Makes the 8 or 9 hour trip back to the states much more 
interesting if you can have internet via connexion the whole way.


/vijay

Re: 2006.06.05 NANOG-NOTES Peering BOF notes

[vijay gill]

Matthew Petach wrote:

Thank you Matt, these notes are almost like being there. Excellent work.

Also Ted Seely at the peering bof? Shocked there wasn't a riot.


They're getting into the peering fray, and only a
year old.
This is gigs and gigs, has potential to dwarf
current peering traffic.  Current issues could be
tiny compared to the flood of potential issues when
hundreds of Gbs comes flooding towards customers.


Problem is extrapolating far into the future from rates seen at the very 
start. I remember some time ago numbers being thrown around of how quick 
imode was being adopted, which,if those rates had continued, would have 
meant most of the world being on imode today.




Swedish police, 100Gb of peer-to-peer traffic at
peak, AMSIX lost 10gig, LINX lost 5gigs, probably
lost about 40Gb weds/thurs last week when the swedish
police shut it down.  Which site?  Pirate Bay torrent
tracker.



Do we have weekly and monthly stats? This looks interesting.



Comment from audience is that live events are
still going to be the challenge; HDTV is getting
gb/sec from cameras, needs to feed it out, no
chance to cache, so multicast ends up being the
only viable option for it.


Multicast is caching with zero retention time -avg.


Robert Seastrom--should you do v6 at all?
Should you be a pioneer, and make the v6 people
happy, sure, do it; if you want to make money,
no.


I think Alain from comcast had a different take on it.


DanGolding notes that the tier1's are stuck on
a treadmill; they can't peer with you even if
they desire it, for fear of messing with their
own ratios.


I don't think sprint or 701 care too much about their own ratios any more.



Dan Golding, network neutrality on the peering
front.

One year old company, video content, already doing
20gig/sec.  This is a buttload of traffic, and they're
already getting into the peering mode; will this traffic
ultimately dwarf the rest of our traffic?


Depends on if this is a sustainable business model. During the dotcom 
days, lots of companies were going to dwarf the rest of our traffic, but 
 things tend to return to mean. It gets harder to sustain 20% growth 
rates month over month, the further up and to the right you go.





Others are tempted to jump on.
MSOs see non-neutrality as a way to keep other's
VoIP off their networks
On the other hand, the Bells will squeeze them; the cable
companies lack peering.


I am not convinced lack of peering is such a huge impediment, esp. at 
transit rates going on now a days (see matt's earlier notes).




Patrick Gilmore: Tier 0, how does that work when
VZ and ATT don't make up the bulk of the internet
anymore.  What about the rest of the world?


Would this statement be true for bulk of the internet in the US? How is 
this determination made?




Patrick Gilmore asks for non-US peering coordinators;
who would care if ATT/VZ depeered you?
Not many respond


It is possible we could have learned more if the question were posed in 
the following fashion


1) How many non-US peering coordinators (have I mentioned how much I 
dislike this term, I prefer SFI Secretaries) have current, active 
peering with VZ/ATT?


2) Of those that answered yes to #1, how many would care if they 
depeered you?


Easy not to care when you don't have SFI in the first place.



Dan Golding notes that if we had many different
ways of getting local loop to your house, it
would be less of an issue.

Incent development of alternate methods; wifi,
3G/4G/5G networks, etc.


Hah, wireless is never going to compete on a purely bandwidth 
perspective with fiber/copper (regardless of the chorus of people 
sounding off of how wifi is used to get the majority of bits on 
cable/dsl - true, but thats a very limited scope deployment, we are 
talking about replacement of cable/dsl here, not how to get from your 
couch to the wall-jack). The way to get wireless working is to emphasize 
the mobile aspect of wireless, but with youtube et al pushing huge bits, 
wireless as a replacement cable/dsl, not so likely.



Mikhail Abramson, with high speed cellular,
mobile is making network neutrality less of an
issue, since you do have more options.  The GSM
providers are happy with the internet bandwidth
usage on mobile data, it's making them really good
money.


Details and breakdown of revenue? Is it ringtone and SMS bandwidth, or 
is it gprs/hspda type bandwidth.




If we all went to a common $10 provider, we could
create a new tier0 and bypass the bellheads.


The last mile _consumers_ aren't likely to be able to go to this $10 
carrier.


/vijay

Re: the iab simplifies internet architecture!

[vijay gill]

Randy Bush wrote:

but it will be a classic.  if you can get and edit it, send
it to boing boing or /.

Pearls before swine.


that's what a number of i* members have publicly stated is their
opinion of talking to us operators.  i saved in my mementos the
following quote from an ipv6 architect and current iab member,
"operators won't accept the h ratio because they don't know what
a logarithm is."


I'd like to see that. Should prove good fodder for quotes on slides.

/vijay <-- winner of several math prizes

Re: cogent+ Level(3) are ok now

[vijay gill]

Pete Templin wrote:



John Curran wrote:


Cold-potato only addresses the long-haul; there's still cost on the
receiving network even if its handed off at the closest interconnect
to the final destination(s).


And there's still revenue, as the traffic is going to customers (we all 
filter our prefixes carefully, right?).  What's the problem with 
cold-potato again, or should we all just try to double-dip?


pt


ah yes, double dipping. On-net traffic should be charged a lot less, 
because after all, it is double dipping.


/vijay

Re: Scalability issues in the Internet routing system

[vijay gill]

Andre Oppermann wrote:

vijay gill wrote:


Moore's law for CPUs is kaput. Really, Moore's Law is more of an 
observation, than a law. We need to stop fixating on Moore's law for 
the love of god.  It doesn't exist in a vacuum, Components don't get 
on the curve for free.  Each generation requires enormously more 
capital to engineer the improved Si process, innovation, process, 
which only get paid for by increasing demand.   If the demand slows 
down then the investment won't be recovered and the cycle will stop, 
possibly before the physics limits, depending on the amount of demand, 
amount of investment required for the next turn etc.


Predicting the future was a tricky business ten years ago and still is
today.  What makes you think the wheel stops turning today?  Customer
access speed will no increase?  No more improvements in DSL, Cable and
Wireless technologies?  Come on, you're kidding.  Right?


Missing the point. We can deal with increased speeds by going wider, the 
network topology data/control plane isn't going wider, THAT is where the 
moore's observation was targeted at.




Also, no network I know is on the upgrade path at a velocity that they 
are swapping out components in a 18 month window. Ideally, for an 
economically viable network, you want to be on an upgrade cycle that 
lags Moore's observation. Getting routers off your books is not an 18 
month cycle, it is closer to 48 months or even in some cases 60 months.


When you are buying a router today do you specify it to cope with 200k
routes or more?  Planning ahead is essential.



And we're paying for it. But again, assuming that the prefix/memory 
bandwidth churn can be accommodated by the next generation of cpus. I am 
not going to throw out my router in 18 months. Its still on the books.



Then we have the issue of an memory bandwidth to keep the ever 
changing prefixes updated and synced.


Compared to link speed this is nothing.  And nowhere near to memory 
bandwidth.


Each update to and fro from memory takes cycles, and as the routing 
tables become bigger, the frequency of access to the memory for keeping 
the system in sync impose a larger burden. This is orthogonal to link speed.


/vijay

Re: Scalability issues in the Internet routing system

[vijay gill]

Andre Oppermann wrote:


I guess it's time to have a look at the actual scalability issues we
face in the Internet routing system.  Maybe the area of action becomes
a bit more clear with such an assessment.

In the current Internet routing system we face two distinctive scalability
issues:

1. The number of prefixes*paths in the routing table and interdomain
   routing system (BGP)

This problem scales with the number of prefixes and available paths
to a particlar router/network in addition to constant churn in the
reachablility state.  The required capacity for a routers control
plane is:

 capacity = prefix * path * churnfactor / second

I think it is safe, even with projected AS and IP uptake, to assume
Moore's law can cope with this.


Moore's law for CPUs is kaput. Really, Moore's Law is more of an 
observation, than a law. We need to stop fixating on Moore's law for the 
love of god.  It doesn't exist in a vacuum, Components don't get on the 
curve for free.  Each generation requires enormously more capital to 
engineer the improved Si process, innovation, process, which only get 
paid for by increasing demand.   If the demand slows down then the 
investment won't be recovered and the cycle will stop, possibly before 
the physics limits, depending on the amount of demand, amount of 
investment required for the next turn etc.


Also, no network I know is on the upgrade path at a velocity that they 
are swapping out components in a 18 month window. Ideally, for an 
economically viable network, you want to be on an upgrade cycle that 
lags Moore's observation. Getting routers off your books is not an 18 
month cycle, it is closer to 48 months or even in some cases 60 months.


Then we have the issue of an memory bandwidth to keep the ever changing 
prefixes updated and synced.



/vijay

Re: Peering vs SFI (was Re: Cogent/Level 3 depeering)

[vijay gill]

Richard Irving wrote:

vijay gill wrote:


 "There can only be *one* !"  - WorldCom chant,  Circa 1995.

WorldCom didn't know what IP SFI was in 95. Perhaps you mean UUNET/MFS?



 Or, perhaps I mean Alternet, eh  ?



Perhaps this is a rolex on my wrist, but they seemed to have made a 
typo. They misspelled Rolex as timex?


Service vs company might be useful distinctions.

/vijay

Re: Peering vs SFI (was Re: Cogent/Level 3 depeering)

[vijay gill]

Richard Irving wrote:

Richard Irving wrote:

  Maybe not, the depeering L3 is involved in is sort of like 
blackmail,



we can all thank the indicted ex-CEO of WorldCom, Bernie Ebbers,
for the modern peering "There can only be one" rule set.



Because you were there at the time Ebbers was going around? Do you 
have any idea of how this works? I am going to go ahead and say no.



 Brzzzt!  lost both points.

  My prior email was [EMAIL PROTECTED], Charter Nanog member.

8-)




Then perhaps you'd know better than to think that Bernie knew what 
peering even was? Apparently not.



/vijay

Peering vs SFI (was Re: Cogent/Level 3 depeering)

[vijay gill]

Richard Irving wrote:





 Maybe not, the depeering L3 is involved in is sort of like blackmail,
we can all thank the indicted ex-CEO of WorldCom, Bernie Ebbers,
for the modern peering "There can only be one" rule set.


Because you were there at the time Ebbers was going around? Do you have 
any idea of how this works? I am going to go ahead and say no.




  Big guys double dip, and little guys are paying half the big
guys double dip... great deal if you can con someone into
accepting it, or are big enough to -force- them into accepting it.



Time to quote Geoff Huston one more time.

"A true peer relationship is based on the supposition that either party 
can terminate the interconnection relationship and that the other party 
does not consider such an action a competitively hostile act. If one 
party has a high reliance on the interconnection arrangement and the 
other does not, then the most stable business outcome is that this 
reliance is expressed in terms of a service contract with the other 
party, and a provider/client relationship is established"



There is no big guy double dipping conspiracy theory here, it's really 
very simply laid out and explained in the above paragraph. Really a work 
of art that should be printed out and nailed to the forehead of the 
Peering Coordinators everywhere.



So, part of this entire debacle is terminology. We need to kill the use 
of the heavily overloaded word Peer and replace it with what it tries to 
mean - Settlement Free Interconnect or SFI.


Most of the so-called Peering Coordinators need to realize that what 
they really are doing is better described by SFI Secretary instead.




Case in point.

L3 wants CoGent to kneel, and kiss the ring,
nothing more, nothing less.

 "They must smell blood in the water".



Lets apply Geoffs Razor to the above. I think L3 thinks that the value 
they get from connecting with Cogent is not justifying their cost, so 
they are pulling the plug. "Smelling blood in the water" indeed. Welcome 
to the inet-access of the new millennium.




   Some providers, a legacy of course, are "transit free", and rely on 
direct routes.. Soon,

there won't be many of these left... and it will be a non-issue.

 "There can only be *one* !"  - WorldCom chant,  Circa 1995.



WorldCom didn't know what IP SFI was in 95. Perhaps you mean UUNET/MFS?



 Like I said, light a fire, and lets burn Bernie at the stake!





 "I saw him fly up into the sky with the Devil himself !" *

  :-P








Best to stay in lurk mode for a while longer.

/vijay

Re: OSPF -vs- ISIS

[vijay gill]

Dan Evans wrote:

All,

Can anyone point me to information on what the top N service providers
are using for their IGP? I'm trying to build a case for switching from
OSPF to IS-IS. Those on this list who are currently running IS-IS, do
you find better scalability and stability running IS-IS than OSPF? I
understand that this question is a lot more complex than a simple yes
or no since factors like design and routing policy will certainly
affect the protocols behavior.

Any insights or experiences that you can share would be most helpful.

Thanks,

Daniel Evans
Alltel Communications


Daniel, in short, we've found ISIS to be slightly easier to maintain and 
run, with slightly more peace of mind in terms of securitiy than OSPF. 
Performance and stability wise, no major difference that was noticable.


For more information, see the talk by Dave Katz at 
http://www.nanog.org/mtg-0006/katz.html


Also, AOL's experience in switching from OSPF to ISIS is covered at 
http://www.nanog.org/mtg-0310/gill.html
the PDF on that page is actually an older version. The full version I 
used at NANOG is available at http://www.vijaygill.com/oi.pdf


/vijay

Re: [NON-OPERATIONAL] Re: NANOG Evolution

[vijay gill]

Hannigan, Martin wrote:



It shouldn't be complicated. I think "members" are looking
for Operator experience. I don't think it's too hard to make that
easily discernable as long as it's fair. 


Members aren't looking for Operator experience (sic). Members are 
looking for talks that do not suck.


/vijay

soBGP deployment

[vijay gill]

If you are an operator, would you deploy soBGP or something like it? If 
not, why not.

http://www.cisco.com/en/US/about/ac123/ac147/ac174/ac236/about_cisco_ipj_archive_article09186a00801c5a9b.html
/vijay

Re: MD5 for TCP/BGP Sessions

[vijay gill]

Christopher L. Morrow wrote:
provided your gear supports it an acl (this is one reason layered acls
would be nice on routers) per peer with:
permit /30 eq 179 /30
permit /30 /30 eq 179
deny all-network-gear-ip-space (some folks call it backbone ip space, Paul
Quinn at cisco says: "Infrastructure ip space")
no more traffic to the peer except BGP from the peer /30. No more ping, no
more traceroute of interface... (downsides perhaps?) and the 'customer'
can still DoS himself :( (or his compromised machine can DoS him)
or forge the source ip on the neighbors /30 or /31 (why aren't you using 
/31s anyway) and call it done.

/vijay

Re: MD5 for TCP/BGP Sessions

[vijay gill]

Stephen J. Wilcox wrote:
without wishing to repeat what can be googled for.. putting acls on your edge to 
protect your ebgp sessions wont work for obvious reasons -- to spoof data and 
disrupt a session you have to spoof the srcip which of course the acl will allow 
in

This is why you either have a stateful firewall in your router that 
pushes a dynamic acl down to the linecard (or equivalent forwarding 
place where the for_us vs through_us decision is made), and filter it 
there. That makes guessing the correct 5 tuple a bit harder. Obviously 
GTSM is the closest we have yet to hardening (note I did not use 
securing) the session.

On average, the stateful filter will cause the attacker to to try 32000 
times to find correct 4-tuple. Conversely, attacker resources will need 
to be on average 32000 times greater to adversely affect a router. The 
cost of attacking infrastructure has risen, but the cost to defender is 
minor.

Each configured BGP session already has all the state needed above to 
populate the filter.

/vijay

Re: Please verify RFC1918 filters

[vijay gill]

On Tue, Mar 22, 2005 at 03:13:07PM -0800, Randy Bush wrote:
> y'all might give us something pingable in that space so we can
> do a primitive and incomplete test in a simple fashion.
> 
> randy
> 

try 172.128.1.1


/vijay

Please verify RFC1918 filters

[vijay gill]

We here at AOL have noticed that there are still some people filtering
172.0.0.0/8, which is causing AOL subscribers to get blocked from some
sites.  As a matter of general IP route filtering hygene I thought it
worth mentioning (again) to see if we can get this tamped down (or, better
still, stamped out).

For reference, RFC1918 20 bit block space is
172.16.0.0  -   172.31.255.255  (172.16/12 prefix)

ARIN-assigned AOL block ranges that have 172 in the first octet are:

172.128.0.0/10
172.192.0.0/12
172.208.0.0/14

Please double check your filters to make sure you are not accidently
blocking AOL in the non-RFC1918 space.  It would be useful to pass this
along to your downstreams as well.  AOL is also working directly with
the companies who have misconfigured firewalls where we notice problems
with filters.

/vijay

Re: public accessible snmp devices?

[vijay gill]

Petri Helenius wrote:
And lately, for reasons undetermined so far there has been instances of 
both vendor C and J where counters suddenly go to zero either 
temporarily (like 1,2,3,4,0,6,7,8,0,10,etc.) or reset altogether without 
any reason.

Pete
I am unclear as to what Vendors "C" and "J" are. Could you clarify please?
thanks
/vijay

Re: Anycast 101

[vijay gill]

On Fri, Dec 17, 2004 at 02:31:06PM -0500, Hannigan, Martin wrote:
> 
> 
> 
> Link outages are higher than router failures when you 
> subtract "human error" RFO's.
> 
> 
> Overall, "fat fingers" account for the larger percentage
> of all outages.
> 

See my preso at the eugene nanog

/vijay

Re: Important IPv6 Policy Issue -- Your Input Requested

[vijay gill]

On Wed, Nov 10, 2004 at 02:17:41AM -0500, Jerry Eyers wrote:
> 
> Ok, let me throw some cold reality water on this discussion...

...

> in the UK, the largest 'chemist' in the UK, built the largest 
> website in the world (2.4 million cc transactions/month with over 460
> servers) and coordinated an IP renumber for over 3,300 stores

cold reality is about right. 2.4 million cc transactions/mo with over four
hundred and sixty (460)! servers makes it the largest web site in the
world?

I better tell that to our datacenter folks or the guys over at amazon or
msn or latency.net - Adam Rothschild does more than that on his laptop
with a wifi card and a hacked-up wrt58g.

> network infrastructure maintainers out there.  You must remember
> to take us into account when talking about global addressing 
> policies.  In the end, we pay most the bills.  :-)

And we thank you for it.

/vijay

Re: Spammers Skirt IP Authentication Attempts

[vijay gill]

On Wed, Sep 08, 2004 at 12:14:54PM +0100, Paul Jakma wrote:
> On Wed, 8 Sep 2004, vijay gill wrote:
> 
> >But if instead of foobar.com, it is vix.com or citibank.com, then 
> >their SPF records will not point at randomgibberish.comcast.net as 
> >an authorized sender. That means that if I do get a mail purporting 
> >to be from citi from randomgibberish, I can junk it without 
> >hesitation.
> 
> Yes, all we need for SPF to work is for spammers to play along and 
> cooperate, and we'll be able to filter out the spam they send.
> 
> Earth calling... ;)

I'm probably going into an argument with a net.kook but just to be sure,
let me clarify this: How do you think spammers will be able to subvert
citibank.com to have random.cablemodem.net as a permitted sender?

I've never believed spf was the ultimate solution, just that it allows me to
better filter some of the joe-bobs.

/vijay - falling yet again into another argument which is probably more
annoying than a thorned thong.

Re: Spammers Skirt IP Authentication Attempts

[vijay gill]

On Wed, Sep 08, 2004 at 11:54:32AM +0100, Paul Jakma wrote:
> 
> Except that, SPF records are as easy to setup for a spammer, as for 
> you and I. If the above is a spammer, then SPF for foobar.com will 
> list randomgibberish.comcast.net as an authorised sender.
> 
> SPF will absolutely not have any effect on spam.

But if instead of foobar.com, it is vix.com or citibank.com, then their
SPF records will not point at randomgibberish.comcast.net as an
authorized sender. That means that if I do get a mail purporting to be
from citi from randomgibberish, I can junk it without hesitation.

/vijay

Re: OT- need a new GSM provider

[vijay gill]

On Thu, Sep 02, 2004 at 07:48:00PM -0700, Joe Rhett wrote:
> vijay gill wrote:
>  
> Sorry, again YMMV but I had no trouble with this in either Taiwan or
> Singapore, when I was responsible for support in those countries, Japan and
> Korea combined.  I never saw a problem calling between any of those.
> 
> Not saying it isn't so, just saying I never had this trouble me-self ;-)

While this thread is rapidly become more annoying than a rectal itch to
edward scissorhands, at the risk of continuing this some more, here is
the roaming from ATT wireless for india, 

http://www.attwireless.com/international/travelguide/coverage_details.jsp?CIDL=356

Calls to international designations other than the USA are not allowed.

Lets see Malayasia

  You may not be able to place calls to international destinations other
  than United States while roaming in this country. Calls can be
  completed within the visited country and back to the United States.

Ditto Pakistan

Bangladesh

Indonesia

etc.

I am sure there are many counterexamples where it worked for you, but
I've recently as of May this year, have international calls other than
the US fail for me while on ATT while in SE Asia.

YMMV, HTH, HAND, etc etc

/vijay

Re: OT- need a new GSM provider

[vijay gill]

On Thu, Sep 02, 2004 at 06:23:31PM -0700, Fred Baker wrote:
> 
> At 06:04 PM 09/02/04 -0700, Joe Rhett wrote:
> >> Also note due to fraud mitigation, most phones only allow you to call
> >> within the country you are in or back to the home country, all the while
> >> charging you an exhorbitant price.
> >
> >Um, sorry but I've never seen this.  I used to world-roam on AT&T, and now
> >I do it with T-Mobile and never had any such drama.
> 
> ditto. color me clueless, but AT&T worked once upon a time, and T-Mobile 
> works quite well for me now. 

This is more of an issue in SE asia in my experience than in europe.

/vijay

Re: OT- need a new GSM provider

[vijay gill]

On Fri, Sep 03, 2004 at 10:47:43AM +1200, Randy Bush wrote:

> strongly recommended.  or, as here in fiji, one can get a phone
> unlocked for a few bucks (couple of guys on a bench in a street
> stall).


Triband phones mostly operate on 900/1800/1900 frequencies. There is a
major US deployment of GSM on the "cellular" GSM 850 band. So if you are
with a triband phone on anyone other than Tmobile (which uses only
1900gsm in the US), you will not get adequately covered. You want either
a US centric triband for use in the US with ATT/cingular that operates
on GSM 850/1800/1900 and then get a world triband on GSM 900/1800/1900
and swap sims in and out (trivially easy to get most gsm phones
unlocked), OR you want a quadband like the moto v600 or treo 600 GSM
which operate on 850/900/1800/1900.



> voicestream is t-mobile.  telephant stupidity and error rate are
> proportional to size.  hence, coverage and intl roaming with clue
> and good billing are not likely.

Verizon now has a worldphone that will roam onto vodafone GSM
internationally. Their rates don't appear to be too prohibitive. Though
if you are going to be calling a lot while abroad, I suggest picking up
an unlocked nokia 6310i and prepaid sims as you fly into airports.

Put up a web page with your current phone number of choice.

Also note due to fraud mitigation, most phones only allow you to call
within the country you are in or back to the home country, all the while
charging you an exhorbitant price.

If you _really_ need to be connected at all times, get a sat phone. Some
mobile gsm roaming charges are more expensive than a globalstar.

/vijay

Re: Summary with further Question: Domain Name System protection

[vijay gill]

On Tue, Aug 17, 2004 at 03:57:17AM +, [EMAIL PROTECTED] wrote:

> > 5. 'bogon'in BIND configuration could be used to
> > filter requests from RFC1918 address;
> 
>   this should be pushed to
>   the router.  don't waste CPU cycles 
>   on the Nameserver.

Hosts tend to be a faster writeoff cycle than routers in companies I've
worked at, therefore getting the benefit of moores law about 25% faster
than the routers.  Turn on firewalling in the host. That said, I do
filter 1918 at my edge.


/vijay

Re: 2511 line break

[vijay gill]

On Mon, Jul 26, 2004 at 04:32:53PM -0400, [EMAIL PROTECTED] wrote:
> I don't know how you run your lab nets, but if I have something on a lab net,
> it still gets secured the same way as a world-visible machine would.
> 
> 1) That protects it if ever I add a gateway machine that talks to the world.
> 
> 2) It keeps you in the habit of securing *everything*.
> 
> Apparently, the knee-jerk 'ewww' at using telnet, even on a lab network, wasn't
> ingrained enough to configure ssh instead... Thus there's indeed a high likelyhood
> that there's still telnet being used in some corner of the production net


as we all fall over ourselves to prove to the world how cool we are by
busting on robs lab demo and by the way, also mentioning in passing how
secure our labs are, and how rigorously we engineer them, perhaps one
might take the time to consider the fact that perhaps rob enabled this
in his lab to illustrate his answer.

/vijay

Re: concern over public peering points [WAS: Peering point speed publicly available?]

[vijay gill]

--On Tuesday, July 06, 2004 08:46 -0400 Leo Bicknell <[EMAIL PROTECTED]> 
wrote:


Everyone running their cable wherever they want with no controls,
and abandoning it all in place makes a huge mess, and is one way
to think about it.
[snipped]
I believe the problem Vijay is referencing isn't "throw it over the
wall", but rather where people have to hide the fact that they are
throwing it over the wall.  When some colo providers want to do
things like charge a 0-mile local loop for a fiber across the room
people think it's too much, and run their own "over the wall" fiber.
However since it's technically not allowed it's hidden, unlabeled,
abandoned when unused, and creates a huge mess.
Thanks. Precisely the issue. Being humans involved in this, there is a
tendency to sometimes hack around a problem and then leave it in
place. I know I am susceptible to this and have to be on guard against
this mentality at all times. And I've seen plenty of this in various orgs.
The key here is to maintain an engineering discipline and be on constant
guard against 'just this once' kind of thought. There should be no 
negotiations
with yourself.

Even the best of intentions lead to massive entropy when doing hacks around 
issues.

Temporary fixes aren't.
/vijay

Re: concern over public peering points [WAS: Peering point speed publicly available?]

[vijay gill]

On Tue, Jul 06, 2004 at 01:43:14AM +, Paul Vixie wrote:
> 
> [EMAIL PROTECTED] (vijay gill) writes:
> > Throwing ethernet cables over the ceiling does not scale.
> 
> i think it's important to distinguish between "things aol and uunet don't
> think are good for aol and uunet" and "things that aren't good for anybody."
> 
> what i found through my PAIX experience is that the second tier is really
> quite deep and broad, and that the first tier doesn't ignore them like their
> spokesmodels claim they do.

Paul, I think you took a left at the pass and went down the wrong road
here.   I am not saying ethernet doesn't scale or even vni/pni doesn't
scale, but the mentality embodied in the approach "throw it over the
wall" doesn't bode well if you are to scale.

> not agree is free to behave that way.  but it's not useful to try to dissuade
> cooperating adults from peering any way they want to.
> 
> i've been told that if i ran a tier-1 i would lose my love for the vni/pni
> approach, which i think scales quite nicely even when it involves an ethernet
> cable through the occasional ceiling.  perhaps i'll eat these words when and
> if that promotion comes through.  meanwhile, disintermediation is still my
> favorite word in the internet dictionary.  i like it when one's competitors

As we have seen before in previous lives, and I'm pretty sure stephen
stuart will step in, normalizing "throw the ethernet over the wall"
school of design just leads to an incredible amount of pain when trying
to operate, run and actually document what you've got.

I thought it was illustrative to take a look at some of the other
messages in this thread. People rushing in to argue the scale comment
when the actual heart of the matter was something else entirely,
which apparently only Tony managed to get.

In any case, I am going to pull a randy here and strongly encourage my
competitors to deploy this "ethernet over ceiling tile" engineering
methodology.

/vijay

Re: concern over public peering points [WAS: Peering point speed publicly available?]

[vijay gill]

On Mon, Jul 05, 2004 at 10:55:42AM -0700, joe mcguckin wrote:
> 
> $5000 for an ethernet switch port? It makes me long for the days of throwing
> ethernet cables over the ceiling to informally peer with other networks in a


Throwing ethernet cables over the ceiling does not scale.

/vijay

Re: TCP RST attack (the cause of all that MD5-o-rama)

[vijay gill]

On Tue, Apr 20, 2004 at 09:45:01PM +, vijay gill wrote:

> infrastructure today - a large amount of PPS at the _router_ (with or
> without md5 or tcpsecure) will blow it out of the water. A 10mbits/s
> of packets at the juniper without md5 will also destroy it.

To be clear, I was just using jnx as an example. There are very few
currently shipping boxes that will survive a large PPS attack.

(also to be fair, been a while since I verified the above numbers)

/vijay

Re: TCP RST attack (the cause of all that MD5-o-rama)

[vijay gill]

On Tue, Apr 20, 2004 at 02:42:07PM -0700, Rodney Joffe wrote:
> 
> 
> vijay gill wrote:
> > 
> > 
> > Yes it does. About 5 mbit of md5 should peg a juniper at 100% according
> > to my friend alex.  I have not verified this in the lab.  I suggest
> > you try it out.
> > 
> > Also, this is why the GTSM (ttl hack) was written up ;)
> 
> So then you're suggesting that the GTSM is the correct work-around?
> 

No, the correct workaround is the
http://www.ietf.org/internet-drafts/draft-ietf-tcpm-tcpsecure-00.txt
draft. MD5 is also the correct workaround. However, neither of the
two protect against what is the most vulnerable thing in the internet
infrastructure today - a large amount of PPS at the _router_ (with or
without md5 or tcpsecure) will blow it out of the water. A 10mbits/s
of packets at the juniper without md5 will also destroy it.

GTSM protects against that, the fact that it also works against this
is just an unexpected side benefit.

/vijay

Re: TCP RST attack (the cause of all that MD5-o-rama)

[vijay gill]

On Tue, Apr 20, 2004 at 02:11:02PM -0700, Dan Hollis wrote:
> 
> On Tue, 20 Apr 2004, Crist Clark wrote:
> > But it has limited effectiveness for multi-hop sessions. There is the
> > appeal of a solution that does not depend of the physical layout of the
> > BGP peers.
> 
> Does MD5 open the door to cpu DOS attacks on routers though? Eg can 
> someone craft a DOS attack to take out the CPU on a router by forcing it 
> to MD5 authenticate torrents of junk packets, using less bandwidth than 
> it would take to DOS the links themselves?

Yes it does. About 5 mbit of md5 should peg a juniper at 100% according
to my friend alex.  I have not verified this in the lab.  I suggest
you try it out.

Also, this is why the GTSM (ttl hack) was written up ;)

/vijay

Re: Backbone IP network Economics - peering and transit

[vijay gill]

On Tue, Apr 20, 2004 at 05:15:48AM +, Paul Vixie wrote:
> 
> > > Peering?  Who needs peering if transit can be
> > > had for $20 per megabit per second?
> 
> anyone whose applications are too important to risk dependency on OPNs
> (other people's networks).


OPNs also carry some of the consumers of your bits and you consume
some of theirs.  Unless you're peering with every laptop directly,
somewhere, somehow, you'll be traveling on OPNs, wether it is dark
fiber, a circuit, or a wavelength.

Time to bust out the cardinal vs ordinal optimization argument again.


  option a) getting the best decision for certain (cost $1 million)
  option b) Getting a decision within the top 5% With probability
 = 0.99 (cost $1 million/x), In real life, we often settle
 for such a tradeoff with x=100 to 10,000

Under independent sampling, variance decreases as 1/sqrt(n). Each order
of magnitude increase in certainty requires 2 orders of magnitude
increase in sampling cost. To go from p=0.99 to certainty (p=0.9)
implies a 1,000,000 fold increase in sampling cost.  

So, instead of creating very nice soundbites like OPNs (which I will
be shamelessly appropriating for my own use thank you very much),
I suggest we spend a bit more time actually _analysing_ using
techniques from operations research as to _what_ gives us the
most bang for the buck.

Dan golding has it right re: peering not being a philosophy, but
rather an _engineering_ decision.  I touched upon this at the 
Great Peering Debate at the NANOG Miami, which was hosted by
Bill Norton.

/vijay

Re: BGP TTL check in 12.3(7)T

[vijay gill]

On Thu, Apr 08, 2004 at 11:30:38AM +0200, Hank Nussbacher wrote:
> 
> 
> 
> From Dave Meyer's NANOG 27 presentation:
> http://www.nanog.org/mtg-0302/hack.html
> 
> Not bad - Feb 2003 till April 2004 to code, test and implement a change 
> driven by NANOG :-)
> 
> Interesting that it is listed under the Routing enhancements and not under 
> the Security enhancements of 12.3(7)T.

The TTL mechanism is just a way to distinguish at low cost between
good for_us traffic and junk. So more of a classifer than a security
layer, though it can be argued both ways.  And even though it
does have security in the title, it is _not_ a panacea for "securing"
bgp or any routing information.

http://www.faqs.org/rfcs/rfc3682.html

/vijay


/vijay

Re: Publish or (gulp) Perish

[vijay gill]

On Tue, Mar 23, 2004 at 03:01:56PM -0500, Daniel Golding wrote:


[ various journals ]


> Any thoughts? Have NANOG powerpoint presentations made these sorts of
> journals obsolete? :)


Powerpoints have a hard time matching the depth of a refereed journal
submission, because with the powerpoint, soundbites tend to take
precedence over content.

/vijay

Re: Source address validation (was Re: UUNet Offer New Protection Against DDoS)

[vijay gill]

On Sun, Mar 07, 2004 at 08:35:54PM +, Christopher L. Morrow wrote:
> 
> 
> Here is a sticky point... There are reasons to allow 10.x.x.x sources to
> transit a network. Mostly the reasons come back to 'broken' configurations
> or 'broken' hardware. The reasons still equate to customer calls and
> 'broken' networking fromm their perspective. I think the thing you are
> actually driving at is the 'intent' of the packet, which is quite tough
> for the router to determine.


Putting rubber to the road eventually, we actually went ahead and
packetfiltered rfc1918 space on our edge. I know paul and stephen
will be crowing with joy here, as we had several arguments about
it in previous lives, but having gone ahead and filtered it,
nothing appears to have broken, or at least nothing got called
in. We've been doing it for several months now.

/vijay

Re: Converged Networks Threat (Was: Level3 Outage)

[vijay gill]

On Thu, Feb 26, 2004 at 11:28:09AM +, [EMAIL PROTECTED] wrote:
> 
> > Wouldn't it be great 
> >if routers had the equivalent of 'User mode Linux' each process 
> >handling a service, isolated and protected from each other.  The 
> >physical router would be nothing more than a generic kernel handling 
> >resource allocation.  Each virtual router would have access to x amount 
> >of resources and will either halt, sleep, crash when it exhausts those 
> >resources for a given time slice. 
> 
> This is possible today. Build your own routers using
> the right microkernel, OSKIT and the Click Modular Router
> software and you can have this. When we restrict ourselves
> only to router packages from major vendors then we are 
> doomed to using outdated technology at inflated prices.

Tell you what Michael, build me some of those, have it pass my labs
and I'll give you millions in business. Deal? 

Let me draw it out here: 

Step 1: Buy box
Step 2: Install Click Modular Router Software
Step 3: Profit

/vijay

Re: How relable does the Internet need to be? (Was: Re: Converged Network Threat)

[vijay gill]

On Thu, Feb 26, 2004 at 11:48:17AM +, [EMAIL PROTECTED] wrote:


> Similarly, the Internet has always had N+1 or better vendor resiliency 
> so IOS can have problems while the non-IOS vendor (or vendors) keep on
> running. In fact, I would argue that N+1 vendor resiliency is a good 
> thing for you to implement in your network and N+2 vendor resiliency is 
> a good thing for the Internet as a whole. Let's hope that vendor P manages 
> 
> to get some critical mass in the market along with J and C.

Unfortunately, while this sounds excellent in theory, what really
happens is that you have a large chunk of equipment in the network
belonging to vendor X, and then you introduce vendor Y. Most people
I know don't suddenly throw out vendor X (assuming that this was
a somewhat competent choice in the first place, jumped up l2 boxes
with slow-first-path-to-setup-tcams-for-subsequent-flows don't
count as somewhat competent). People don't do that because it costs
a lot of capital and opex.  So now we have a partial X and partial
Y network, X goes down, and chances are your network got hammered
like an icecube in a blender set to Frappe.

You could theroetically have a multiplane network with each plane
comprising of a different vendor (and we do that on some of our DWDM
rings), but that is a luxury ill-afforded to most people.

/vijay

Re: Converged Networks Threat (Was: Level3 Outage)

[vijay gill]

On Thu, Feb 26, 2004 at 02:48:55PM +, [EMAIL PROTECTED] wrote:
> 
> >> This is possible today. Build your own routers using
> >> the right microkernel, OSKIT and the Click Modular Router
> >> software and you can have this. When we restrict ourselves
> >> only to router packages from major vendors then we are 
> >> doomed to using outdated technology at inflated prices.
> 
> >Tell you what Michael, build me some of those, have it pass my labs
> >and I'll give you millions in business. Deal? 
> 
> The problem with your lab is that you have too many millions
> to give. In order to win those millions people would have to prove
> that their box is at least as good as C and J in the core of the
> largest Internet backbones in the world. That is an awfully big

Let me try this one more time. From the top.

You said:
begin quote
  software and you can have this. When we restrict ourselves
  only to router packages from major vendors then we are
  doomed to using outdated technology at inflated prices.
end quote

So now we have
> to give. In order to win those millions people would have to prove
> that their box is at least as good as C and J in the core of the

So the outdated technology at inflated prices is too high of a hurdle
to pass for the magic Click Modular Software router, the ones that are
allegedly NOT antiquated and are not using outdated technology?
But somehow still cannot function in a core? 


> History shows that if you can build a mousetrap that is technically
> better than anything on the market, your best route for success is

Thought it went build a better mousetrap and the world will beat a 
path to your door, etc etc etc.  


> to sell it into niche markets where the customer appreciates the
> technical advances that you can provide and is willing to pay for
> those technical advances. I don't think that describes the larger
> Internet provider networks.

How would you know this?  Historically, the cutting edge technology
has always gone into the large cores first because they are the
ones pushing the bleeding edge in terms of capacity, power, and
routing.

/vijay

Re: Converged Networks Threat (Was: Level3 Outage)

[vijay gill]

On Thu, Feb 26, 2004 at 10:05:03AM -0800, David Barak wrote:
> 
> --- vijay gill <[EMAIL PROTECTED]> wrote:
> > How would you know this?  Historically, the cutting
> > edge technology
> > has always gone into the large cores first because
> > they are the
> > ones pushing the bleeding edge in terms of capacity,
> > power, and
> > routing.
> > 
> > /vijay
> 
> I'm not sure that I'd agree with that statement: most
> of the large providers with whom I'm familiar tend to
> be relatively conservative with regard to new
> technology deployments, for a couple of reasons:
> 
> 1) their backbones currently "work" - changing them
> into something which may or may not "work better" is a
> non-trivial operation, and risks the network.

This is perhaps current. Check back to see large deployments
GSR - sprint/UUNEt
GRF - uunet
Juniper - UUNET/CWUSA

In all of the above cases, those were the large isps that forced
development of the boxes. Most of the smaller "cutting edge"
networks are still running 7513s.

GSR was invented because the 7513s were running out of PPS.
CEF was designed to support offloading the RP.

> 2) they have an installed base of customers who are
> living with existing functionality - this goes back to
> reason 1 - unless there is money to be made, nobody
> wants to deploy anything.
> 
> 3) It makes more sense to deploy a new box at the
> edge, and eventually permit it to migrate to the core
> after it's been thoroughly proven - the IP model has
> features living on the edges of the network, while
> capacity lives in the core.  If you have 3 high-cap
> boxes in the core, it's probably easier to add a
> fourth than it is to rip the three out and replace
> them with two higher-cap boxes.

The core has expanded to the edge, not the other way around.
The aggregate backplane bandwidth requirements tend to
drive core box evolution first while the edge box normally
has to deal with high touch features and port multiplexing.
These of course are becoming more and more specialized over
time.

> 4) existing management infrastructure permits the
> management of existing boxes - it's easier to deploy
> an all-new network than it is to upgrade from one
> technology/platform to another.

Only if you are willing to write off your entire capital
investment. No one is willing to do that today.


> 
> -David Barak
> -Fully RFC 1925 Compliant
> 


/vijay
> __
> Do you Yahoo!?
> Get better spam protection with Yahoo! Mail.
> http://antispam.yahoo.com/tools

Re: Converged Networks Threat (Was: Level3 Outage)

[vijay gill]

On Thu, Feb 26, 2004 at 09:32:07PM +0200, Petri Helenius wrote:

> along. It might still exist. CEF was developed to address the issue of 
> route cache insertion and purging. The unneccessarily painful 60 second 
> interval new destination stall was widely documented before CEF got 
> widespread use. The "fast switching" approach was also particularly 
> painful when DDOS attacks occurred.


Thanks for the correction. I clearly was not paying enough attention
when composing.

/vijay

Re: Unbelievable Spam.

[vijay gill]

On Tue, Feb 03, 2004 at 10:31:00AM +, [EMAIL PROTECTED] wrote:
> 
> inject large volumes of email into the system? The existing
> non-hierarchical email exchange network is not scalable.
> I hope that everyone on this list can understand what the
> email exchange overlay network is and recognize that it
> is subject to similar scaling rules as the underlying IP 
> network.

So you are saying that the existing email exchange network
is not scalable because it is non-hierarchical?

It looks like its scaled pretty well so far. Maybe I'm 
not understanding something.

The entire paragraph quoted appears to be entirely content
free except for some various assertions. Could you elaborate
further on

1) how the existing non-hierarchical email exchange network is
not scalable. What are your definitions of "scale" and what
are the current choking points that you see?

2) the email exchange overlay network and the similarities
to the underlying ip network with elaborations on how the
hierarchy of IP maps to the hierarchy in email overlays.

thanks

/vijay

Re: Outbound Route Optimization

[vijay gill]

On Mon, Jan 26, 2004 at 08:47:54AM -0700, Wayne E. Bouchard wrote:
> 
> Although in principle I agree with what you say here, I will point out
> that the number and frequency of "significant" network outages
> (excluding things like the recent power failure in LAX) has become
> rare as compared to what they were 5 or 6 years ago. Part of this is
> due to attitudes about the 'net maturing, part due to increased
> experience of the average engineer, and part due to things such as
> MPLS fast reroute.

I am going to have to call bullshit on the MPLS fast reroute thing
there Wayne.  The canonical counterexample is Sprint.

Excellent engineering and ops folks top the list, followed closely
by sufficient capacity, not pushing the envelope any more (basically
we are now on the scale of growth where things like running out of pps
don't happen any more), and the fact that now we are growing
in an organic fashion, so the people at the bleeding edge are
sufficiently clued up that the vendors products are together for the
people following.  Major protocol implementations  have been 
beaten into shape, and now it is (mostly) a matter of bigger bandwidth
and routers, not any fundamental architectural change.



/vijay





> 
> I would also point out that, although there remain single points of
> interconnect, MPLS has meant that the path packets take intra-network
> doesn't have to be a single route between two boxes. BGP picks the
> exit point and engineers have configured MPLS to spread the traffic
> over 3 or 4 tunnels to get there thereby reducing the impact of a
> single failure.
> 
> But as you say, this really gets into the realm of overbuilt backbones
> which, of course, not everyone has. BGP isn't the best. I think many
> people have recognized that for some years now. However, when
> propperly managed, it suits current needs.
> 
> Perhaps it's time for the next generation of BGP to come into being;
> something that can use up to 4 paths through a network for any single
> destination rather than simply leaving alternate paths innactive until
> something changes. Heavens knows there are many instances where there
> are two or more "good" (and even equal) paths through a network that
> are not chosen simply because we're only allowed one.
> 
> On Mon, Jan 26, 2004 at 10:35:38AM +, [EMAIL PROTECTED] wrote:
> > 
> > >BGP is relatively good at determining the best path when you a major
> > >carrier with connectivity to "everyone" (i.e. when traffic flows
> > >"naturally"), in many locations, and you engineer your network so that 
> > you
> > >have sufficient capacity to support the traffic flows.
> > 
> > In other words, BGP really only works well when most networks
> > are overbuilt so that there is a single uncongested best
> > path through each network from every ingress to every egress
> > and the paths within any given network's core are roughly
> > similar in capacity.
> > 
> > Nowadays there is a lot more variability both within networks
> > and between different networks. How can a simple protocol
> > provide optimal behavior between an MPLS network, an IP over
> > ATM network, a network that is half GRE tunnels, and a network
> > that has core links ranging from DS3 to OC48? I think BGP is 
> > another example where something that is "good enough" has risen
> > to prominence in spite of the fact that it is not optimal.
> > 
> > And another thing. How do we know this problem can ever be
> > solved when we continue to use routing protocols which choose
> > the *BEST* path. The best path is always a single path and,
> > by definition, this is a single point of failure. How can we
> > ever have a diverse and reliable network when its core routing
> > paradigm is a single point of failure?
> > 
> > Note that people have built IP networks that provide two
> > diverse paths at all times using multicast
> > http://www.simc-inc.org/archive9899/Jun01-1999/bach2/Default.htm
> > and such things may also be possible with MPLS. But are any of
> > the researchers seriously looking at how to provide a network
> > in which all packets flow through two diverse paths to provide
> > better reliability?
> > 
> > --Michael Dillon
> > 
> > 
> 
> ---
> Wayne Bouchard
> [EMAIL PROTECTED]
> Network Dude
> http://www.typo.org/~web/

Re: Outbound Route Optimization

[vijay gill]

On Wed, Jan 21, 2004 at 09:05:46PM +, Paul Vixie wrote:
> 
> > My questions are these:
> > 
> > "Is sub-optimal routing caused by BGP so pervasive it needs to be
> > addressed?" 
> 
> that depends on your isp, and whether their routing policies (openness
> or closedness of peering, shortest vs. longest exit, respect for MEDs)
> are a good match for their technology/tools, skills/experience, and
> resources/headroom.

In practice, all of the above just turn out to be marketing sauce
or in some cases, outright lies.

There is no substitute for dollar spend (opex and capex) to make
a network perform.  There is no magic sauce, there is no silver
bullet. You have adequate resources, you will have adequate
performance.

> metrics, which is usually not very good.  but there's another limit, which
> is bgp path symmetry.  most tcp implementations are still stone-aged

AKA optimizing for outbound doesn't do you any good on optimizing
for inbound.

> (experience says they're not going to trust your MEDs even if they're close
> enough to hear them.)

Most people don't trust MEDs for a reason paul, and it is not because
they want to mess with your customers.

/vijay

Re: mSQL Attack/Peering/OBGP/Optical exchange

[Vijay Gill]

Stephen Stuart <[EMAIL PROTECTED]> writes:

> Optical switch technology, and the control systems that cause the
> technology to implement the business rules of an exchange point, have
> a ways to go before they're ready for prime-time.

We don't know anything we could do with 50ms provisioning without
making a disaster (c) smd 2001.

/vijay

Re: mSQL Attack/Peering/OBGP/Optical exchange

[Vijay Gill]

David Diaz <[EMAIL PROTECTED]> writes:

> was to "pay" for what you used when you used it.  The biggest
> technical factor was "how the heck do you bill it."

Actually I'd think the biggest technical factor would be the trained
monkey that would sit at the switch and do OIR of line cards on the
router as appropriate and reroute patches.

> If a customer goes from their normal OC3 ---> OC12 for 4hrs three
> times in a month... what do you bill them for?  Do you take it down to
> the DS0/min level and just multiple or do you do a flat rate or a per
> upgrade???

Does this include the monkey cost as the monkey switches the ports
around? (well, technically you can get software switchable oc3/oc12
ports, but substitute for 48/192 and go from there)


> The point was you could bump up on the fly as needed, capacity
> willing, then down.  The obvious factor is having enough spare
> capacity in the bucket.  This should not be an issue within the 4

And the monkey. I really don't have enough capital sitting around to
leave a spare port idle for the 4 hours a day I need it.

> This sort of addressed the "how do i design my backbone"
> argument. Where engineers ahve to decide whether to built for peak
> load and provide max QoS but also the highest cost backbone; or
> whether to built for avg sustained utilization.  This way you can
> theoretically get the best of both worlds.  As long as the billing
> goes along with that.

I don't plan to be buying service from anyone who is building to
average sustained utilization (sic).  My traffic tends to be bursty.

/vijay

Re: mSQL Attack/Peering/OBGP/Optical exchange

[Vijay Gill]

David Diaz <[EMAIL PROTECTED]> writes:

> With the rapid onset of an attack such as the one sat morning. Models
> I have show that not only would the spare capacity been utilized
> quickly but that in a tiered (colored) customer system. That the lower
> service level customers (lead colored, silver etc) would have had

Does your model(s) also take into account that people's capital
structure may not allow them the luxury of leaving multiple OC-X
ports wired up and sitting idle waiting for a surge?

One thing I found somewhat interesting among the "dymanic" allocation
of resources type infrastructure was the fact that my capacity
planning is on the order of weeks, while the exchanges assume
something on the order of minutes. I don't have enough capital sitting
around that I can afford to deploy and hook up a bunch of OC-x ports
to an exchange and then sit there waiting for them to be used maybe
sometimes in the future, for sure, etc etc.

So perhaps the thought of an optical exchange running out of resources
might be a bit of an overkill at this stage?

/vijay

Re: FW: Re: Is there a line of defense against Distributed Reflective attacks?

[Vijay Gill]

"Al Rowland" <[EMAIL PROTECTED]> writes:

> mention the effect everyone on AOL going to broadband and downloading
> Disney clips all the time would have on their settlement plans with
> backbone providers.

Of course, because you are definitely being kept in the loop regarding
the AOL settlement plans?

/vijay

Re: Scaled Back Cybersecuruty

[Vijay Gill]

Avi Freedman <[EMAIL PROTECTED]> writes:


> - Routers must be configured by end of 2003 so that all packets 
>   to the control plane must be logically separated from user
>   packets (or demonstrate the ability to take 200mb of attack
>   traffic to the router CPU without having an effect) 

This at least, we're working on actively.

http://www.ietf.org/internet-drafts/draft-gill-btsh-01.txt

Also, there will be a talk about this, as well as hardware features
we'd like to see on RPs specifically addressing the router cpu
attack.

/vijay

Re: Scaled Back Cybersecuruty

[Vijay Gill]

Avi Freedman <[EMAIL PROTECTED]> writes:


> Many networks of sizable import have no capex budget, though - or
> sometimes very little if no engineering resources.  They all pay
> attention to sales - and especially to RFIs and RFQs from the Feds,
> though.

I suspect this will be a self correcting problem once a few whacks
at the infrastructure take place.

/vijay

Re: Scaled Back Cybersecuruty

[Vijay Gill]

Avi Freedman <[EMAIL PROTECTED]> writes:

> Perhaps the Feds (and maybe states) could use their purchasing power
> to effect change.  Short of that, or regulation, the I don't see how
> the serious issues we have with the 'net will get resolved.
> 
> I suppose that the "problem" is likely that people don't understand 
> what a nice actually well-written worm could/would do if it were 
> targeted at the infrastructure.

People do. I've been beating this particular horse for a while now,
and we are starting to deploy the capex hammer.  I suggest others start
to do the same. See my presentation at the eugene nanog.

/vijay