Re: Vyatta as a BRAS

[Tony Varriale]

- Original Message - 
From: "Joe Greco" 

To: "Dobbins, Roland" 
Cc: "NANOG list" 
Sent: Wednesday, July 14, 2010 7:03 PM
Subject: Re: Vyatta as a BRAS



On Jul 14, 2010, at 10:17 PM, Joe Greco wrote:

> The truth is that you can keep throwing CPU at a problem as well.  I 
> can =

size a software based router such that it can remain available.

Not against mpps, or even high kpps, you can't, unfortunately.


Really?  I'm positive that I can, because I *have*, and other people
*have*.  The sweet spot for protecting a 100Mbps circuit, in particular,
moved from hardware to software about five years ago.  That simply means
it's more cost-effective for a competent admin to spend some time to set
up the box than it is to spend money on dedicated silicon that'll be
obsolete in a few years, a fact that's conveniently ignored by a lot of
the advocates of such solutions.  To drive the point home, FreeBSD based
routers that we built in 2004 are able to cope with full routing tables
and IPv6 *today*, at the same traffic levels they were designed for, and
those particular qualities don't seem to be present in many of the
hardware-based offerings of the era.  If and when they cease to be useful
in that capacity, they can be trivially repurposed as firewalls or web
servers or other similar tasks, because unlike the pricey purpose-built
router hardware, there are advantages to general purpose hardware.

Quite frankly, this is starting to be a little annoying.  Perhaps you
could do some research, or find some competent admins and test a few well
built setups yourself before you make any more disprovable claims.  My
claims are not ridiculous and are not a figment of my imagination; I can
point to many-years-old documented examples, such as

http://lists.freebsd.org/pipermail/freebsd-net/2004-September/004840.html

http://info.iet.unipi.it/~luigi/polling/

These are tests of forwarding capabilities, true, but the reality is that
the same sorts of things that make this possible make it relatively easy
to support large numbers of packets directed "at the control plane", since
the concept of the control plane isn't as separated in the FreeBSD 
software

model as it is in the hardware model.  As a result, a FreeBSD box can take
and sink quite a bit of traffic.  Doing so does not cripple it.

For giggles, I took two out-of-the-box FreeBSD 8.0 servers, twiddled
*only* device polling to on, and started them running traffic at each
other.  Both were sending north of 100Mbps (>>100Kpps) of traffic at
the other, both when listening and when not, no problems, no crashes,
no issues.  That doesn't sound too great until I reveal that I was
lazy and it's only some excess capacity on a VMware box that's
available to these two virtual servers.

> Software based platforms have an incredible edge in areas that hardware 
> b=
ased platforms don't, including capex and the ability to find replacement 
p=

arts after a disaster.

I agree 100% with this, and with much of what you say.  My point is that 
at=
 the *edge* - like a BRAS, which is how this thread started - one must 
have=
 platforms which can be adequately protected against attack/abuse, and 
hard=

ware-based platforms are the only practical way to do that.


In some cases, for some purposes, yes.  Otherwise, no.

... JG
--
Joe Greco - sol.net Network Services - Milwaukee, WI - http://www.sol.net
"We call it the 'one bite at the apple' rule. Give me one chance [and] 
then I
won't contact you again." - Direct Marketing Ass'n position on e-mail 
spam(CNN)
With 24 million small businesses in the US alone, that's way too many 
apples.




I briefly browsed the links and I didn't see any traffic profiles included.

If you are talking about pushing x mbps with no specifics and/or general 
traffic, I think most of us agree you can do that easily and probably 
consistently without any issues.  And for some icing, you may even do it at 
<90% average CPU util.  Does that mean it should be an edge device at any 
service provider?  No.  Some?  Sure.


Can you point to any specific tests of attack vectors and/or traffic 
profiles with: CPU utilization, packet loss levels and pps/mbps/etc data? 
The reason I ask is that Roland is in a specific business and has a specific 
point.


As a side, were those 2 VMs on the same box?  That traffic out on the wire? 
What's the traffic profile?


tv 

Re: Vyatta as a BRAS

[Joe Greco]

> On Jul 14, 2010, at 10:17 PM, Joe Greco wrote:
> 
> > The truth is that you can keep throwing CPU at a problem as well.  I can =
> size a software based router such that it can remain available.
> 
> Not against mpps, or even high kpps, you can't, unfortunately.

Really?  I'm positive that I can, because I *have*, and other people
*have*.  The sweet spot for protecting a 100Mbps circuit, in particular,
moved from hardware to software about five years ago.  That simply means
it's more cost-effective for a competent admin to spend some time to set
up the box than it is to spend money on dedicated silicon that'll be
obsolete in a few years, a fact that's conveniently ignored by a lot of
the advocates of such solutions.  To drive the point home, FreeBSD based
routers that we built in 2004 are able to cope with full routing tables
and IPv6 *today*, at the same traffic levels they were designed for, and
those particular qualities don't seem to be present in many of the 
hardware-based offerings of the era.  If and when they cease to be useful
in that capacity, they can be trivially repurposed as firewalls or web
servers or other similar tasks, because unlike the pricey purpose-built
router hardware, there are advantages to general purpose hardware.

Quite frankly, this is starting to be a little annoying.  Perhaps you 
could do some research, or find some competent admins and test a few well
built setups yourself before you make any more disprovable claims.  My
claims are not ridiculous and are not a figment of my imagination; I can
point to many-years-old documented examples, such as

http://lists.freebsd.org/pipermail/freebsd-net/2004-September/004840.html

http://info.iet.unipi.it/~luigi/polling/

These are tests of forwarding capabilities, true, but the reality is that
the same sorts of things that make this possible make it relatively easy
to support large numbers of packets directed "at the control plane", since
the concept of the control plane isn't as separated in the FreeBSD software
model as it is in the hardware model.  As a result, a FreeBSD box can take
and sink quite a bit of traffic.  Doing so does not cripple it.

For giggles, I took two out-of-the-box FreeBSD 8.0 servers, twiddled 
*only* device polling to on, and started them running traffic at each 
other.  Both were sending north of 100Mbps (>>100Kpps) of traffic at
the other, both when listening and when not, no problems, no crashes, 
no issues.  That doesn't sound too great until I reveal that I was 
lazy and it's only some excess capacity on a VMware box that's 
available to these two virtual servers.

> > Software based platforms have an incredible edge in areas that hardware b=
> ased platforms don't, including capex and the ability to find replacement p=
> arts after a disaster.
> 
> I agree 100% with this, and with much of what you say.  My point is that at=
>  the *edge* - like a BRAS, which is how this thread started - one must have=
>  platforms which can be adequately protected against attack/abuse, and hard=
> ware-based platforms are the only practical way to do that.

In some cases, for some purposes, yes.  Otherwise, no.

... JG
-- 
Joe Greco - sol.net Network Services - Milwaukee, WI - http://www.sol.net
"We call it the 'one bite at the apple' rule. Give me one chance [and] then I
won't contact you again." - Direct Marketing Ass'n position on e-mail spam(CNN)
With 24 million small businesses in the US alone, that's way too many apples.

Re: Vyatta as a BRAS

[Joel Jaeggli]

On 7/13/10 11:11 AM, Dobbins, Roland wrote:


On Jul 14, 2010, at 1:02 AM, Matthew Kaufman wrote:


Dangerous in places where forwarding table exceeds hardware cache
limits. (See Code Red worm stories)



During the Code Red/Nimda period (2001), and on into the
Slammer/Blaster/Nachi period (2003), all the routers I personally
know of which were adversely affected were software-based, didn't
make use of ASICs for forwarding.


Having msdp turned on was a great way to get nuked by slammer regardless 
of your choice of forwarding technology.


Which reminds me control plane protection is about more than just acls 
and rate limiting.



---



Roland Dobbins  //


Injustice is relatively easy to bear; what stings is justice.

-- H.L. Mencken

Root Zone DNSSEC Deployment Technical Status Update

[Joe Abley]

Root Zone DNSSEC Deployment
Technical Status Update 2010-07-14

This is the eleventh of a series of technical status updates intended
to inform a technical audience on progress in signing the root zone
of the DNS.


RESOURCES

Details of the project, including documentation published to date,
can be found at .

We'd like to hear from you. If you have feedback for us, please
send it to roots...@icann.org.


KSK CEREMONY 2 COMPLETE

The second KSK ceremony for the root zone was completed this week
in El Segundo, CA, USA. The Ceremony Administrator was Mehmet Akcin.

The second production Key Signing Request (KSR) generated by VeriSign
has now been processed by ICANN using the root zone KSK generated
in KSK Ceremony 1, and the resulting Signed Key Response (SKR) has
been accepted by VeriSign. This SKR contains signatures for Q4 2010,
for use between 2010-10-01 and 2010-12-31.

Audit materials relating to both the first and second ceremonies
will be published today at .


FULL PRODUCTION SIGNED ROOT ZONE

The transition from Deliberately-Unvalidatable Root Zone (DURZ) to
production signed root zone is scheduled take place on 2010-07-15
within a maintenance window which begins at 1930 UTC and ends at
2330 UTC. This is the usual window for the generation and distribution
of root zones with SOA serials ending in 01.


PLANNED DEPLOYMENT SCHEDULE

Already completed:

  2010-01-27: L starts to serve DURZ

  2010-02-10: A starts to serve DURZ

  2010-03-03: M, I start to serve DURZ

  2010-03-24: D, K, E start to serve DURZ

  2010-04-14: B, H, C, G, F start to serve DURZ

  2010-05-05: J starts to serve DURZ

  2010-06-16: First Key Signing Key (KSK) Ceremony

  2010-07-12: Second Key Signing Key (KSK) Ceremony

To come:

  2010-07-15: Distribution of validatable, production, signed root
zone; publication of root zone trust anchor

  (Please note that this schedule is tentative and subject to change
  based on testing results or other unforeseen factors.)

Re: Vyatta as a BRAS

[Per Carlson]

> Is the CRS-1 hardware or software?
> Lots of custom hardware in there - but lots of processing cores that look
> suspiciously like software engines too.

It might well be software engines in there, but that's not the point
here. The linecards (MSC/PLIM etc.) in a CRS is designed to handle
wirerate traffic *of any packet length* and simultaneously do ACLs,
QoS or whatever else you throw at them. If that is done using FPGAs,
CPUs or trained monkeys isn't really that interesting, as long as it
works. And it does. But it won't come for free.

A software-based router, i.e something equipped with a central CPU
doing all heavy lifting, of *any kind* isn't designed to do that.

The old corollary 7a in RFC1925 still make sense: "Good, Fast, Cheap:
Pick any two (you can't have all three)."

Some of the arguments expressed also vaguely resembles truth 11 in the
same RFC:
Every old idea will be proposed again with a different name and a
different presentation, regardless of whether it works.

There is a reason internet isn't built on Dell hardware...

-- 
Pelle

A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?
A: Top-posting.
Q: What is the most annoying thing in e-mail?

Re: Vyatta as a BRAS

[Dobbins, Roland]

On Jul 15, 2010, at 1:49 AM, Lamar Owen wrote:

> CEF is CEF is CEF, whether done on a 2600 or a 7200 or a GSR.  Now, don't get 
> me wrong; the engineers who make massively parallel forwarding engines are 
> creative and smart folks, and have come up with very elegant methods of 
> moving the bits ever faster, but the fundamental forwarding architectures, 
> even of these accelerated boxes, can be implemented in pure software, as 
> evidenced by the Cisco Nexus 1000V.

This is a *vast* oversimplification of the 'life of a packet' in a box like a 
CRS-1 vs. a 7200, for example.  You know this, of course.

> Marketingspeak doesn't necessarily reflect reality.  The original draft of 
> one of my replies in this thread  said this 'Let's run this rabbit, and 
> dispel some marketing hype while we're at it.'

I wasn't a marketing person during my time at Cisco, and I'm not a marketing 
person now.  Technical people within Cisco and outside Cisco routinely make use 
of the terms 'hardware-based' and 'software-based' to describe these 
fundamental classes of routers, and have for years.

>  The reality is that 'hardware-based' routers really are AMP (asymmetrical 
> multiprocessing) software-based routers, with specialized processors running 
> specialized software.

Right - the 'hardware-based routers' idiom comes from the 'specialized 
processors', known as NPUs, ASICs, TCAMs, and so forth.

> And when implemented properly they are very good at what they do; I have 
> GSR's, they work great, and regardless of what engine is on the linecard some 
> software at some level running on some processor is making the forwarding 
> decisions at the data plane,

'Some processor' = ASIC or NPU = 'hardware-baed'.

> and they can even for certain things require a punt to the MIPS processor on 
> the linecard (IPv6 on Engine 1, anyone?).

Yes, this is true - or even to the system RP.

>  Knowing the technology and its architecture, without blindly buying into 
> marketingspeak, helps operators make better procurement decisions.

Nobody in this conversation so far is 'blindly buying into marketingspeak', to 
my knowledge - certainly not me.

>  And Cisco's website has most of the information you need to make that 
> decision, if you use their hardware, which is very good.

The content on Cisco's Web site is confusing, redundant, full of *actual* 
marketing-speak, and highly confusing in many aspects.  This isn't a problem 
unique to Cisco, mind, but afflicts almost all technology companies.

>  Dig deeply enough, and past the hardware versus software pseudodichotomy, 
> and you can make very informed decisions indeed.

Yes, but you aren't going to do that by looking at product marketing materials 
on a Web site.

>  As a tongue in cheek example, remember the 'switching router' versus 
> 'routing switch' distinction?

Yes, which was nonsense.  OTOH, 'hardware-based' vs. 'software-based' is a 
useful distinction commonly employed by technical, non-marketing people within 
both the vendor and operational communities alike.

> If a specialized network processing engine in an AMP router can protect the 
> control plane CPU, then code running on different processors in an SMP system 
> could do the same.

Not on a general-purpose SMP system running commodity processors such as those 
found in PCs.

>  Perhaps not as efficiently, but the end result can be the same.  I mean, I 
> wonder if Blue Gene or Jaguar could give a CRS series a run for its money in 
> terms of routing power (especially on the control plane),

Possibly - certainly not on the forwarding plane.

> and what the price/performance ratio would be to throwing something like 
> Jaguar (224K Opteron processors, running Linux) at the relatively mundane 
> task of throwing precisely metered bits around the wires. :-)

Fruitless speculation, IMHO.

> Regardless of recommendations, people are using commodity server-grade SMP 
> hardware to run commodity OS's to get the job done, and given the people who 
> have chimed in here, apparently are doing it without lots of problems.

My experience is that folks doing this on the edges of their networks 
eventually end up regretting it, after they get zorched a time or two.

>  The increase on this and other lists of questions about Mikrotik, Vyatta, 
> and other nontraditional routing platforms is an interesting throwback to the 
> days of Proteon routers, the original SUN, and Cisco's multibus roots, and it 
> shows that people are deploying these newer and much faster boxen in the real 
> world, bugs and all.

It shows me that a lot of folks, because they haven't been in the industry very 
long and/or don't learn from the mistakes of others in the past, seem 
determined to repeat those same mistakes, ad nauseam, ad infinitum.

> It's not a 'software-based = bad; hardware-based = good' world, even at the 
> edge anymore;

I very strongly disagree with this, based upon my hands-on operational 
experience in production networks.  

Re: Vyatta as a BRAS

[sthaug]

> > I wasn't aware that the 7206 and M20 classified as software-based.
> 
> I don't see why you could call it anything but a software router.

The 7206 yes. The M20, no.

Steinar Haug, Nethelp consulting, sth...@nethelp.no

Re: Vyatta as a BRAS

[sthaug]

> Regardless of recommendations, people are using commodity server-grade SMP 
> hardware to run commodity OS's to get the job done, and given the people who 
> have chimed in here, apparently are doing it without lots of problems.  The 
> increase on this and other lists of questions about Mikrotik, Vyatta, and 
> other nontraditional routing platforms is an interesting throwback to the 
> days of Proteon routers, the original SUN, and Cisco's multibus roots, and it 
> shows that people are deploying these newer and much faster boxen in the real 
> world, bugs and all.

How many of them are deploying server-grade SMP hardware / commodity OS
to handle 10 Gig links and expect to handle DoS attacks using minimum
sized packets? That's 14.88 Mpps with Ethernet encapsulation, for each
10 Gig link.

Anybody?

Steinar Haug, Nethelp consulting, sth...@nethelp.no

Re: Vyatta as a BRAS

[Jon Lewis]

On Tue, 13 Jul 2010 valdis.kletni...@vt.edu wrote:


I wasn't aware that the 7206 and M20 classified as software-based.


I don't see why you could call it anything but a software router.  That's 
sort of why things like it and the 7500 before it lasted so long.  As the 
thing ages, cisco comes out with new NPE (or RSP/VIP) processors with 
faster CPUs / more memory capacity that are able to move more packets. 
i.e. NPE-100->NPE-150->NPE-200->NPE-225->NPE-300->NPE-400->NPE-G1->NPE-G2


You could start with a VXR with NPE-225 and keep upgrading the CPU and 
keep the thing in service with the same interface cards 15 years or more.


--
 Jon Lewis   |  I route
 Senior Network Engineer |  therefore you are
 Atlantic Net|
_ http://www.lewis.org/~jlewis/pgp for PGP public key_

Re: Vyatta as a BRAS

[Lamar Owen]

On Wednesday, July 14, 2010 08:39:50 am Dobbins, Roland wrote:
> And it's not *my* definition - 'hardware-based' vs. 'software-based' are the 
> terms to describe these two fundamental architectural classes of router 
> *within Cisco itself*.

[snip]

> There's a world of difference in packet-handling mechanisms and sheer 
> performance between a 7200 and a CRS-1, or a GSR, or a CRS-3, or Juniper 
> T-series - and not just one of 'more, faster processors', but of fundamental 
> architecture. 

CEF is CEF is CEF, whether done on a 2600 or a 7200 or a GSR.  Now, don't get 
me wrong; the engineers who make massively parallel forwarding engines are 
creative and smart folks, and have come up with very elegant methods of moving 
the bits ever faster, but the fundamental forwarding architectures, even of 
these accelerated boxes, can be implemented in pure software, as evidenced by 
the Cisco Nexus 1000V.

> This is why 'hardware-based' vs. 'software-based' is a useful distinction; 
> again, note that within Cisco, these are the common terms used to describe 
> these general classes of device, with 7200s and ISRs being termed 
> 'software-based', and the other models mentioned above described as 
> 'hardware-based'.

Marketingspeak doesn't necessarily reflect reality.  The original draft of one 
of my replies in this thread  said this 'Let's run this rabbit, and dispel some 
marketing hype while we're at it.'  

The reality is that 'hardware-based' routers really are AMP (asymmetrical 
multiprocessing) software-based routers, with specialized processors running 
specialized software. And when implemented properly they are very good at what 
they do; I have GSR's, they work great, and regardless of what engine is on the 
linecard some software at some level running on some processor is making the 
forwarding decisions at the data plane, and they can even for certain things 
require a punt to the MIPS processor on the linecard (IPv6 on Engine 1, 
anyone?).  

Knowing the technology and its architecture, without blindly buying into 
marketingspeak, helps operators make better procurement decisions.  And Cisco's 
website has most of the information you need to make that decision, if you use 
their hardware, which is very good.  Dig deeply enough, and past the hardware 
versus software pseudodichotomy, and you can make very informed decisions 
indeed.  As a tongue in cheek example, remember the 'switching router' versus 
'routing switch' distinction?

If a specialized network processing engine in an AMP router can protect the 
control plane CPU, then code running on different processors in an SMP system 
could do the same.  Perhaps not as efficiently, but the end result can be the 
same.  I mean, I wonder if Blue Gene or Jaguar could give a CRS series a run 
for its money in terms of routing power (especially on the control plane), and 
what the price/performance ratio would be to throwing something like Jaguar 
(224K Opteron processors, running Linux) at the relatively mundane task of 
throwing precisely metered bits around the wires. :-)

Regardless of recommendations, people are using commodity server-grade SMP 
hardware to run commodity OS's to get the job done, and given the people who 
have chimed in here, apparently are doing it without lots of problems.  The 
increase on this and other lists of questions about Mikrotik, Vyatta, and other 
nontraditional routing platforms is an interesting throwback to the days of 
Proteon routers, the original SUN, and Cisco's multibus roots, and it shows 
that people are deploying these newer and much faster boxen in the real world, 
bugs and all.

It's not a 'software-based = bad; hardware-based = good' world, even at the 
edge anymore; a few years ago, sure, I wouldn't dream of doing such a thing.  I 
for one love what a good parallel state machine in an FPGA can do to your 
software's performance (we're doing that here, doing interferometric 
correlation at 96Gb/s on a relatively inexpensive FPGA platform based on IBOB); 
or what GPU acceleration can do to graphics performance, but it is a help to 
realize that those activities, accelerated though they may be, are still 
software-based.

And while it's not a BRAS, one of the most exciting products I've seen in a 
long while from Cisco is the above-mentioned Nexus 1000V.  Pure software 
virtual switching for VMware.

NOC Best Practices

[Kasper Adel]

Hello Everyone,

I am currently working on building a NOC so i'm looking for
materials/pointers to Best Practices documented out there.

On the top of my head are things like:

1) Documenting Incidents and handling them
2) Documenting Syslog messages
3) Documenting Vendor Software Bugs
4) Shift to Shift Hand over procedures
5) Commonly used scripts for monitoring
6) Frequently testing High Availability
7) Capturing config changes.
etc

I can see that this is years of experience but i am wondering if any of this
was captured some where.

Thanks,
Kim

Re: Vyatta as a BRAS

[Dobbins, Roland]

On Jul 14, 2010, at 10:17 PM, Joe Greco wrote:

> The truth is that you can keep throwing CPU at a problem as well.  I can size 
> a software based router such that it can remain available.

Not against mpps, or even high kpps, you can't, unfortunately.

> Software based platforms have an incredible edge in areas that hardware based 
> platforms don't, including capex and the ability to find replacement parts 
> after a disaster.


I agree 100% with this, and with much of what you say.  My point is that at the 
*edge* - like a BRAS, which is how this thread started - one must have 
platforms which can be adequately protected against attack/abuse, and 
hardware-based platforms are the only practical way to do that.

---
Roland Dobbins  // 

Injustice is relatively easy to bear; what stings is justice.

-- H.L. Mencken

Re: Vyatta as a BRAS

[Joe Greco]

> On Jul 14, 2010, at 5:45 AM, Joe Greco wrote:
> > That's just a completely ignorant statement to make.
> 
> It's based on a great deal of real-world experience; I'm sorry you consider=
>  that to be 'ignorant'.

You're speaking to someone who has extensive experience with "software"
based routers, and you're failing to acknowledge the upsides of such an
architecture, when I've already conceded the upsides of a hardware
architecture.

> >  I notice in particular how carefully you qualify that with "[w]hen BCPs =
> are=20
> > followed"; the fact that hardware router manufacturers have declared
> > everything and anything that derails their bullet trains as "not a
> > BCP" is a perfect example of this deceptive sort of misinformation.
> 
> Anti-spoofing, iACLs, CoPP (or its equivalent on non-Cisco platforms), et. =
> al. aren't 'misinformation'.  They're useful, proven techniques/features wh=
> ich any operator ought to implement.

The things that any given use scenario ought to implement are highly
dependent on the actual application.

> > There are plenty of FreeBSD based devices out there that are passing
> > tons of traffic; almost any of them are more competent than any Cisco
> > router I'm aware of when hitting them directly with traffic
> 
> Then your experience of Cisco routers (and/or those from other vendors) mus=
> t be limited to the lower-end platforms; I can assure you that faster Cisco=
>  boxes such as ASRs, GSRs, CRSes, and so forth are in another league entire=
> ly, and can handle mpps of to-us traffic, when properly configured.  Softwa=
> re-based routers simply can't do that; it's not an indictment of them, it's=
>  just that they aren't suited to purpose, just as station wagons generally =
> aren't to be found in the Indy 500.

So your solution is to keep throwing heavier hardware at the problem until
it works.  Okay, I see that.  Now, let me quote from a different message:

> If maintaining availability is important, then hardware-based (semantic
> hairsplitting aside) devices are a requirement.

The truth is that you can keep throwing CPU at a problem as well.  I can
size a software based router such that it can remain available.

This is neither new nor exciting technology.  Luigi Rizzo was doing
extensive work on this about a decade ago: he took an Athlon 750 platform
with 4 100Mbit ethernet interfaces in it (Athlon 750 = 1999 tech) and was
able to exceed 100Mbps levels without a problem.  The UNIX based platforms
have extensive capabilities to defend against attack, even without a
firewall.  As with a hardware based platform, there are both good things
and bad things you can do that will impact availability.

Software based platforms have an incredible edge in areas that hardware
based platforms don't, including capex and the ability to find replacement
parts after a disaster.  I spent some time after the Haiti quake getting
FreeBSD-based routers up and running, a task made easier because it's a
lot easier to find a working PC and scavenge some network cards than it is
to find a working Cisco router in a city where all inbound and outbound
transportation is paralyzed.

You can continue to defend your position, of course, but it's just looking
a bit silly.  A wise engineer knows that there are several ways to tackle
any task, and "one tool for every job" is not a sound policy.

If you'd like to revise your position to "Cisco and Juniper software based
solutions are underpowered PoS", that's probably a defensible position,
and you won't get any argument from me.  Please don't generalize such a
position into all software based devices, though.  Overall, there are a
lot more software based routers out there than hardware based devices.
Your cablemodem, your ADSL modem, your wifi access point, all these are
probably software based devices.  Some of them will melt under a too-great
load.  Some won't.  This is a function of many different factors.  There
is nothing inherent in a software-based device that's going to make it
fail under load - just as there's nothing inherent in a hardware-based
device that's going to make it succeed (which is why you have to qualify
your defense of these with "must follow BCP").  It's the related
engineering that ultimately determines whether or not it all works out.

... JG
-- 
Joe Greco - sol.net Network Services - Milwaukee, WI - http://www.sol.net
"We call it the 'one bite at the apple' rule. Give me one chance [and] then I
won't contact you again." - Direct Marketing Ass'n position on e-mail spam(CNN)
With 24 million small businesses in the US alone, that's way too many apples.

Re: OER/PfR with BGP for inbound load sharing

[Dobbins, Roland]

On Jul 14, 2010, at 9:55 PM, Dylan Ebner wrote:

>  I should look for other options to balance my inbound traffic. 


Beyond the binary choice to advertise or not to advertise a given prefix via a 
given peer/upstream and/or any TE policies your peers/upstreams may support via 
community/attribute tagging, you've really no control over inbound traffic path 
selection.  You can prepend, but whether other networks honor it or ignore it - 
or if they do honor it, *how* they honor it - is entirely beyond your control.

So, vendor marketing claims aside, the concept of 'load-balancing' inbound 
traffic isn't really a valid one.

The only actual path-selection control you have is over your outbound traffic, 
and that only for a single hop beyond your network, into each of your 
peers/upstreams.  What happens after that is beyond your control, as well.

---
Roland Dobbins  // 

Injustice is relatively easy to bear; what stings is justice.

-- H.L. Mencken

OER/PfR with BGP for inbound load sharing

[Dylan Ebner]

Does anyone have any experience with using OER for inbound load sharing? I am 
looking to see if people are generaly satisfied with it's abilities or if I 
should look for other options to balance my inbound traffic. I have two 
connections (one 50Mb and one 25Mb) with partial BGP routes across two routers 
that I would like to move from an active/standby model to a more active/active 
model. Our organization moves very little information out of our forward facing 
precessence servers so outbound balancing is not as important. Almost 80% of 
our inbound traffic is VPN as well so using application discovery isn't that 
important either.

Some specific questions I have are:
Is OER aware of traffic-shape or bandwidth contstraints that are applied to an 
interface?
Does OER simply add prepending for my advertised routes to my upstreams or does 
it actually prepend AS from inbound providers to move specific traffic from an 
AS to one of my underutilized connections?
Does OER have to ability to control inbound traffic on a subnet level within 
the same AS? We receive a ton of data from a few AS numbers, so simply moving a 
single AS to a different connection may not be extremely effective.

Thanks.

Dylan Ebner

Re: Vyatta as a BRAS

[Dobbins, Roland]

On Jul 14, 2010, at 8:59 PM, Florian Weimer wrote:

> There might be contractual reasons not to enable that feature. 8-/

Ignoring is generally pretty harmless; dropping can break traceroute, RSVP, et. 
al.

Conversely, there are also generally pretty strong contractual reasons not to 
have one's edge routers go down due to excessive punts.

;>

> Some vendors can process options in hardware, though.

True.

> It's probably not a high-priority issue for vendors until there are
> network issues (as opposed to potential problems seen in labs),

This is always true when it comes to security, and especially to availability.  
That being said, I know that at least one major vendor is cognizant of the 
header-extenstion issue, and is taking steps to mitigate the associated risk.

> so it's going to take quite a bit of time.

Yes, this is always the case, unfortunately.

>  Demand for devices with some IP-layer inspection capability that can handle 
> (Fast or Gigabit)
> Ethernet at line rate, no matter what type of frames come in, is also
> a pretty recent thing, and I would be surprised if vendors can provide
> such capabilities across their entire relevant product line (where
> they advertise line-based forwarding).


With large vendors, these things are generally accomplished piecemeal, on a 
BU-by-BY, product-by-product basis.  Unfortunate, but true, nonetheless.

---
Roland Dobbins  // 

Injustice is relatively easy to bear; what stings is justice.

-- H.L. Mencken

Re: Vyatta as a BRAS

[Florian Weimer]

* Roland Dobbins:

> On Jul 14, 2010, at 8:38 PM, Florian Weimer wrote:
>
>> There's also the question of IP options (or extension headers). 8-)
>
> I know that some modern hardware-based routers have the ability to
> either ignore options, or to drop option packets altogether.

There might be contractual reasons not to enable that feature. 8-/
Some vendors can process options in hardware, though.

> I believe the same is now true of IPv6 extension-headere, or soon
> will be.  You're absolutely correct that this is a significant
> possible attack vector, causing the packets in question to be
> punted, if there isn't a mechanism available to ignore them or to
> drop said packets.

It's probably not a high-priority issue for vendors until there are
network issues (as opposed to potential problems seen in labs), so
it's going to take quite a bit of time.  Demand for devices with some
IP-layer inspection capability that can handle (Fast or Gigabit)
Ethernet at line rate, no matter what type of frames come in, is also
a pretty recent thing, and I would be surprised if vendors can provide
such capabilities across their entire relevant product line (where
they advertise line-based forwarding).

-- 
Florian Weimer
BFK edv-consulting GmbH   http://www.bfk.de/
Kriegsstraße 100  tel: +49-721-96201-1
D-76133 Karlsruhe fax: +49-721-96201-99

Re: Vyatta as a BRAS

[Dobbins, Roland]

On Jul 14, 2010, at 8:48 PM, Florian Weimer wrote:

> From or to your customers?

Both.

> Stopping customer-sourced attacks is probably a good thing for the Internet 
> at learge.

Concur 100%.

>  And you can't combat attacks targeted at customers within your own network 
> unless you've got very large WAN
> pipes, moving you into the realm of special-purpose hardware for other 
> reasons.

Sure, you can, via S/RTBH, IDMS, et. al.  While DNS reflection/amplification 
attacks are used to create crushing volumes of attack traffic, and even 
smallish botnets can create high-volume attacks, most packet-flooding attacks 
are predicated on throughput - i.e., pps - rather than bandwidth, and tend to 
use small packets.  Of course, they can use *lots and lots* of small packets, 
and often do, but one can drop these packets via the various mechanisms one has 
available, then reach out to the global opsec community for filtering closer to 
the sources.

The thing is, with many DDoS attacks, the pps/bps/cps/tps required to disrupt 
the targets can be quite small, due to the unpreparedness of the defenders.  
Many high-profile attacks discussed in the press such as the Mafiaboy attacks, 
the Estonian attacks, the Russian/Georgian/Azerbaijan attacks, the China DNS 
meltdown, and the RoK/USA DDoS attacks were all a) low-volume, b) 
low-throughput, c) exceedingly unsophisticated, and d) eminently avoidable via 
sound architecture, deployment of BCPs, and sound operational practices.

In fact, many DDoS attacks are quite simplistic in nature and many are low in 
bandwidth/throughput; the miscreants only use the resources necessary to 
achieve their goals, and due to the unpreparedness of defenders, they don't 
have a need to make use of overwhelming and/or complex attack methodologies.

This doesn't mean that high-bandwidth, high-throughput, and/or complex DDoS 
attacks don't occur, or that folks shouldn't be prepared to handle them; quite 
the opposite, we see a steady increase in attack volume, thoughput and 
sophistication at the high end.  But the fact of the matter is that many DDoS 
targets - and associated network infrastructure, and services such as DNS - are 
surprisingly fragile, and thus are vulnerable to surprisingly simple/small 
attacks, or even inadvertent/accidental attacks.

> Previously, this was really a no-brainer because you couldn't get PCI
> cards with the required interfaces, but with Ethernet everywhere, the
> bandwidths you can handle on commodity hardware will keep increasing.

Concur 100%.

> Eventually, you'll need special-purpose hardware only for a smallish
> portion at the top of the router market, or if you can't get the
> software with the required protocol support on other devices.

I believe that the days of software-based routers are numbered, period, due to 
the factors you describe.  Of course, the 'top of the router market' seems to 
keep moving upwards, despite many predictions to the contrary.

---
Roland Dobbins  // 

Injustice is relatively easy to bear; what stings is justice.

-- H.L. Mencken

Re: Vyatta as a BRAS

[Florian Weimer]

* Roland Dobbins:

> That's what I meant - even a very small botnet can easily overwhelm
> software-based edge routers.

>From or to your customers?

Stopping customer-sourced attacks is probably a good thing for the
Internet at learge.  And you can't combat attacks targeted at
customers within your own network unless you've got very large WAN
pipes, moving you into the realm of special-purpose hardware for other
reasons.

Previously, this was really a no-brainer because you couldn't get PCI
cards with the required interfaces, but with Ethernet everywhere, the
bandwidths you can handle on commodity hardware will keep increasing.
Eventually, you'll need special-purpose hardware only for a smallish
portion at the top of the router market, or if you can't get the
software with the required protocol support on other devices.

-- 
Florian Weimer
BFK edv-consulting GmbH   http://www.bfk.de/
Kriegsstraße 100  tel: +49-721-96201-1
D-76133 Karlsruhe fax: +49-721-96201-99

Re: Vyatta as a BRAS

[Dobbins, Roland]

On Jul 14, 2010, at 8:38 PM, Florian Weimer wrote:

> There's also the question of IP options (or extension headers). 8-)

I know that some modern hardware-based routers have the ability to either 
ignore options, or to drop option packets altogether.

I believe the same is now true of IPv6 extension-headere, or soon will be.  
You're absolutely correct that this is a significant possible attack vector, 
causing the packets in question to be punted, if there isn't a mechanism 
available to ignore them or to drop said packets.

---
Roland Dobbins  // 

Injustice is relatively easy to bear; what stings is justice.

-- H.L. Mencken

Re: Vyatta as a BRAS

[Florian Weimer]

* Valdis Kletnieks:

> (cue weasel-words about those routers using ASICs for most forwarding, but
> doing multicast forwarding in software in 5.. 4.. 3..)

There's also the question of IP options (or extension headers). 8-)

-- 
Florian Weimer
BFK edv-consulting GmbH   http://www.bfk.de/
Kriegsstraße 100  tel: +49-721-96201-1
D-76133 Karlsruhe fax: +49-721-96201-99

Re: Vyatta as a BRAS

[Dobbins, Roland]

On Jul 14, 2010, at 7:01 PM,  
 wrote:

> But as others have stated, the 7206 has at least some hardware acceleration,

Unfortunately, said statements are factually incorrect.  7200s have no hardware 
acceleration of any type whatsoever.

from 
:

'Processor

1.67-GHz Motorola Freescale 7448 processor'

> so it's *not* a router that uses *only* centralized general-purpose CPUs.

Actually, it is.  Same with ISRs.

from 


Note the 'Multicore Processor' line-item - singular.

The SREs for the ISR2s do each contain their own Intel x86 processor - so, the 
ISR2 models which can take SREs are distributed platforms, but aren't 
hardware-based in the sense that they contain high-performance forwarding 
chips.  The processors in the SREs are used to run applications on-board the 
router itself - so, they're kind of like special-purpose servers on a card, 
rather than high-performance linecards as one finds in higher-end platforms.

> So basically, your definition of "hardware based" router is "one that has 
> enough
> FPGAs to not tank under some arbitrary workload". Not very useful,that.

It's extremely useful to differentiate routers which have special-purpose 
forwarding hardware from those which don't, as the latter crumble quite quickly 
when packeted.  If you don't believe me, run some tests of your own with purely 
software-based routers, such as 7200s, and then with a hardware-based router 
such as an ASR1K, ASR9K, GSR, CRS-1, N7K, what-have-you.

I've seen this divergent behavior between software-based and hardware-based 
platforms time and time again in real, live production networks, during real, 
live attacks.  It isn't something which can simply be dismissed by semantic 
hairsplitting.

And it's not *my* definition - 'hardware-based' vs. 'software-based' are the 
terms to describe these two fundamental architectural classes of router *within 
Cisco itself*.

> Let's face it Roland - it's a continuum from hardware to software, and in many
> places it's downright murky which it is. Is the CRS-1 hardware or software?

Hardware, obviously - it has special-purpose NPUs on the linecards, along with 
special-purpose ASICs, and TCAMs.  

> Lots of custom hardware in there - but lots of processing cores that look 
> suspiciously like software engines too.


There's a world of difference in packet-handling mechanisms and sheer 
performance between a 7200 and a CRS-1, or a GSR, or a CRS-3, or Juniper 
T-series - and not just one of 'more, faster processors', but of fundamental 
architecture.  This is why 'hardware-based' vs. 'software-based' is a useful 
distinction; again, note that within Cisco, these are the common terms used to 
describe these general classes of device, with 7200s and ISRs being termed 
'software-based', and the other models mentioned above described as 
'hardware-based'.

Anyway, enough on this topic.  If folks wish to continue to deploy 
software-based routers at the edges of their networks, then they oughtn't to be 
surprised or dismayed when said software-based routers fall over under 
relatively small amounts of packeting, either deliberate attacks or as the 
result of misconfiguration, et. al.  If, on the other hand, they prize 
availability, then investing in hardware-based platforms and then configuring 
said hardware-based routers with the appropriate BCPs greatly reduces the risk 
of such an occurrence.

---
Roland Dobbins  // 

Injustice is relatively easy to bear; what stings is justice.

-- H.L. Mencken

RE: Receive Digest

[Yasir Munir Abbasi]

I got it.

Thanks you to all...

Yasir Munir Abbasi
Senior Network Engineer

Ciklum Pakistan
2nd floor, Software Technology Park II, Evacuee Trust Plaza
F-5/1, Islamabad
Tel  + 92 51 2826114
Fax +92 51 2870756
Mob +92 333 5605512
EMail: y...@ciklum.net

-Original Message-
From: Marc Powell [mailto:li...@xodus.org] 
Sent: Wednesday, July 14, 2010 4:53 PM
To: nanog@nanog.org
Subject: Re: Receive Digest


On Jul 14, 2010, at 1:13 AM, Yasir Munir Abbasi wrote:

> Dear,
> 
> I always receive digest with volume number, where number of email 
> correspondence shown. That is very grim for reading. Is there any possibility 
> I can receive individual email correspondence. Thanks

Standard mailman. Go to https://mailman.nanog.org/mailman/listinfo/nanog. Log 
in at the 'NANOG Subscribers' section and change your delivery option.

--
Marc

Re: Vyatta as a BRAS

[Valdis . Kletnieks]

On Wed, 14 Jul 2010 02:18:18 -, "Dobbins, Roland" said:
> Right.  And to date, such routers make use of ASICs - i.e., 'hardware-based'
> routers, in the vernacular.
>
> Routers which use only centralized, general-purpose processors can't handle
> even a fraction of 'line-rate' without tanking

But as others have stated, the 7206 has at least some hardware acceleration,
so it's *not* a router that uses *only* centralized general-purpose CPUs. So
at least some hardware-assisted routers tank under loads too.

And even the most heavily hardware-assisted systems have to do call outs from
the FPGA's for *some* stuff, and can be tanked by suitably creative abuse of
their weaknesses.  Of course, in general the more hardware assist, the harder
it is to tank (but it's never impossible).

So basically, your definition of "hardware based" router is "one that has enough
FPGAs to not tank under some arbitrary workload". Not very useful,that.

Let's face it Roland - it's a continuum from hardware to software, and in many
places it's downright murky which it is. Is the CRS-1 hardware or software?
Lots of custom hardware in there - but lots of processing cores that look
suspiciously like software engines too.



pgpvXXVQOMD51.pgp
Description: PGP signature

Re: Receive Digest

[Marc Powell]

On Jul 14, 2010, at 1:13 AM, Yasir Munir Abbasi wrote:

> Dear,
> 
> I always receive digest with volume number, where number of email 
> correspondence shown. That is very grim for reading. Is there any possibility 
> I can receive individual email correspondence. Thanks

Standard mailman. Go to https://mailman.nanog.org/mailman/listinfo/nanog. Log 
in at the 'NANOG Subscribers' section and change your delivery option.

--
Marc

Re: Vyatta as a BRAS

[Dobbins, Roland]

On Jul 14, 2010, at 1:34 PM, Mikael Abrahamsson wrote:

>  CRS-1 uses multicore processors (hundreds of cores) for forwarding on their 
> linecards, and they achieve 40+ Mpps per linecard.


The CRS-1 makes use of the Metro subsystem for forwarding, with multiple Metros 
per Modular Service Card (MSC).  Each Metro complex (there are two per MSC) 
consists of the Metro chip itself, an NPU which contains 188 embedded RISC 
cores; two TCAM banks; SRAM; and FCRAM.

There's also a gatekeeper ASIC of some sort on the MSC which handles traffic 
incoming from the fabric, as well as another interface module ASIC on the 
Physical Layer Interface Module (PLIM).

I believe the CRS-3-specific MSCs each contain two QFAP complexes, which allow 
for 140gb/sec per linecard, and that there are various additional supporting 
ASICs on the MSCs and the PLIMs, as well.

---
Roland Dobbins  // 

Injustice is relatively easy to bear; what stings is justice.

-- H.L. Mencken