SV: BGP peering strategies for smaller routers

[Gustav Ulander]

Hello.

Yea its a bit strange that we would need to add memory to the platform just 
because we upgraded the image. 
But since we were in kind of a tight spot we opted to upgrade it and then move 
away from the platform in that role. 

//Gustav

-Ursprungligt meddelande-
Från: William Herrin [mailto:b...@herrin.us] 
Skickat: den 3 maj 2016 22:32
Till: Gustav Ulander 
Kopia: Eric Sabotta ; NANOG list 
Ämne: Re: BGP peering strategies for smaller routers

On Tue, May 3, 2016 at 3:50 PM, Gustav Ulander 
 wrote:
> Yes I can confirm that we also had the issue with the asr1001s.
> For us the router was fine until we upgraded it. When we rebooted it 
> after the upgrade it ran out of memory when populating 2 full feeds.
> When we contacted TAC they confirmed that indeed it was a memory 
> problem and that we would need to add more memory to the box.

Hi Gustav,

IMO, you should not accept that answer from the TAC. An IOS release that 
crashes with two 600k BGP feeds in 4 gigs of RAM is badly defective.

Regards,
Bill Herrin



--
William Herrin  her...@dirtside.com  b...@herrin.us Owner, 
Dirtside Systems . Web: 

Re: BGP peering strategies for smaller routers

[Blake Hudson]

I turned up a full ip4 feed on an RP1 today. Took approximately 5 minutes to 
fill the rib and probably another 5 minutes to push to the fib. The CLI 
responsiveness was noticeably slowed during this process, but the router didn't 
drop traffic. I'm guessing a second feed would involve fewer rib and fib 
changes and would converge faster. 

> On May 3, 2016, at 6:02 PM, Carlos Alcantar  wrote:
> 
> I know this thread has been primarily about memory to hold the routing 
> tables, but how well does it do with the BGP convergence time??  which could 
> be the other killer with multiple full route tables.
> 

Re: NOC AS1836 green.ch AG

[Arnold Nipper]

On 03.05.2016 20:05, Marco Paesani wrote:

> I need direct contact with NOC of AS1836.
> Can you help me ?

Have a look at PeeringDB [0], Marco!


Ciao, Arnold
[0] https://peeringdb.com/net/232
-- 
Arnold Nipper
email: arn...@nipper.de  phone: +49 6224 5593407 2
mobile: +49 172 2650958  fax:   +49 6224 5593407 9



signature.asc
Description: OpenPGP digital signature

Re: BGP peering strategies for smaller routers

[William Herrin]

On Tue, May 3, 2016 at 6:18 PM, Nick Hilliard  wrote:
> William Herrin wrote:
>> And it is poor code quality. Even slicing and dicing the ram in odd
>> ways, there's just no excuse for an order-of-magnitude increase in ram
>> required to run the same algorithms on the same data.
>
> If RAM were expensive, your argument would make sense, but RAM is not
> expensive.

Hi Nick,

You missed the point. Sloppy memory management is a "canary in a coal
mine." It's a user-visible symptom that reflects poor code quality
underneath. Programmers who don't care how much ram they're consuming
are the same fools who catch and then ignore exceptions, don't bother
evaluating the big-oh running time of their algorithms (often have no
idea what that is) and engage in a variety of other bad practices that
you as the customer suffer for but never directly see.

It's not the cost of the ram, it's the attitude that ram is cheap so I
won't care. It's a bad attitude, a dangerous attitude when found in a
computer programmer. One which consistently leads to failure.

If you challenge poor code quality when you spot it, your vendor might
correct course. If you let it slide then by the time the code base is
damaged enough for the pointy-hairs to understand there's a problem on
their own, your only real choice will be to switch to a different
product or vendor.

> Can we move on now?

Sure, why not. I've proselytized enough for one day.

Regards,
Bill Herrin




-- 
William Herrin  her...@dirtside.com  b...@herrin.us
Owner, Dirtside Systems . Web: 

Re: BGP peering strategies for smaller routers

[Łukasz Bromirski]

Blake,

> On 04 May 2016, at 00:23, Blake Hudson  wrote:
> 
> Łukasz Bromirski wrote on 5/3/2016 4:13 PM:
>>> On 03 May 2016, at 22:31, William Herrin  wrote:
>>> 
>>> On Tue, May 3, 2016 at 3:50 PM, Gustav Ulander
>>>  wrote:
 Yes I can confirm that we also had the issue with the asr1001s.

[...]

> I feel like you're trying to fit some other (possible, but far fetched) 
> scenario from where we started.

Yeah, sorry for that - saw 1001 in quote and kept that as original platform.

For 1002 with SSO off you may be fine, sure. BTW, the versions you're quoting 
as working were also quoted by me as the ones that could have been OK even on 
the 1001 (I know, I know).

-- 
Łukasz Bromirski
CCIE R/SP #15929, CCDE #2012::17, PGP Key ID: 0xFD077F6A

Re: BGP peering strategies for smaller routers

[Carlos Alcantar]

I know this thread has been primarily about memory to hold the routing tables, 
but how well does it do with the BGP convergence time??  which could be the 
other killer with multiple full route tables.


​
Carlos Alcantar
Race Communications / Race Team Member
1325 Howard Ave. #604, Burlingame, CA. 94010
Phone: +1 415 376 3314 / car...@race.com / http://www.race.com



From: NANOG  on behalf of Blake Hudson 
Sent: Tuesday, May 3, 2016 3:23:42 PM
To: nanog@nanog.org
Subject: Re: BGP peering strategies for smaller routers

Łukasz Bromirski wrote on 5/3/2016 4:13 PM:
>> On 03 May 2016, at 22:31, William Herrin  wrote:
>>
>> On Tue, May 3, 2016 at 3:50 PM, Gustav Ulander
>>  wrote:
>>> Yes I can confirm that we also had the issue with the asr1001s.
>>> For us the router was fine until we upgraded it. When
>>> we rebooted it after the upgrade it ran out of memory
>>> when populating 2 full feeds.
>>> When we contacted TAC they confirmed that indeed
>>> it was a memory problem and that we would need to
>>> add more memory to the box.
>> Hi Gustav,
>>
>> IMO, you should not accept that answer from the TAC. An IOS release
>> that crashes with two 600k BGP feeds in 4 gigs of RAM is badly
>> defective.
> Not necessarily.
>
> In essence, your physical memory gets halved in two after
> router boots up, then it may be further halved if you’re
> using features like SSO. So, with 4GB RAM config and with
> SSO running, you may be left with around 600-650MB free after
> boot and with IOS-XE loaded, and then all the features kick
> in. Including your BGP feeds that need around 300MB of memory
> just to store the tables, then there’s CEF RAM representation,
> and so on.
>
> Here’s a good WP w/r to memory usage & architecture on ASR 1k:
> http://www.cisco.com/c/en/us/support/docs/routers/asr-1000-series-aggregation-services-routers/116777-technote-product-00.html
>
> It actually contains the same recommendation given by TAC -
> with recent/current code if you want to run full tables with
> BGP, get 8GB of RAM on ASR 1k. In the 3.10-3.12S era I believe
> it was still possible to fit (without the SSO) full tables
> in RAM and be fine.
>
> As Nick just responded, it’s faster to source the RAM or modify
> the config to cut down on number of BGP prefixes rather than
> ping back and forth here discussing all the possibilities.
>

I feel like you're trying to fit some other (possible, but far fetched)
scenario from where we started. Mike, the op, said he has a 1002 which
has an RP1 configured with 4GB of RAM. First, this is not a 1001.
Second, SSO is off by default and is unlikely to be configured on an
ASR1002 (the op certainly never stated he had enabled it). I just
checked a few ASR 1k RP1s I have access to and the 3.10 IOS image has
similar memory usage to the 3.13 (the latest MD release for this
platform). On the RP1 platform, with a default only BGP feed, the
systems have ~ 1.3GB of proc mem available. With a single full IP4 BGP
feed, the free proc mem goes down to ~850MB. With two full feeds, the
proc mem goes down to ~ 800MB. RP memory goes from ~1.8GB used (default
only), ~2.7GB used (1x full), ~2.8GB used (2x full). This still leaves
over 1GB of free RP memory with two full BGP feeds. The amount of FIB is
dependent on the ESP installed by the OP; Mike hasn't yet stated what
ESP he has installed.

So yes, the 1002 can, and will continue to, work with two full BGP feeds
when fitted with an ESP10.

Re: BGP peering strategies for smaller routers

[Blake Hudson]

Łukasz Bromirski wrote on 5/3/2016 4:13 PM:

On 03 May 2016, at 22:31, William Herrin  wrote:

On Tue, May 3, 2016 at 3:50 PM, Gustav Ulander
 wrote:

Yes I can confirm that we also had the issue with the asr1001s.
For us the router was fine until we upgraded it. When
we rebooted it after the upgrade it ran out of memory
when populating 2 full feeds.
When we contacted TAC they confirmed that indeed
it was a memory problem and that we would need to
add more memory to the box.

Hi Gustav,

IMO, you should not accept that answer from the TAC. An IOS release
that crashes with two 600k BGP feeds in 4 gigs of RAM is badly
defective.

Not necessarily.

In essence, your physical memory gets halved in two after
router boots up, then it may be further halved if you’re
using features like SSO. So, with 4GB RAM config and with
SSO running, you may be left with around 600-650MB free after
boot and with IOS-XE loaded, and then all the features kick
in. Including your BGP feeds that need around 300MB of memory
just to store the tables, then there’s CEF RAM representation,
and so on.

Here’s a good WP w/r to memory usage & architecture on ASR 1k:
http://www.cisco.com/c/en/us/support/docs/routers/asr-1000-series-aggregation-services-routers/116777-technote-product-00.html

It actually contains the same recommendation given by TAC -
with recent/current code if you want to run full tables with
BGP, get 8GB of RAM on ASR 1k. In the 3.10-3.12S era I believe
it was still possible to fit (without the SSO) full tables
in RAM and be fine.

As Nick just responded, it’s faster to source the RAM or modify
the config to cut down on number of BGP prefixes rather than
ping back and forth here discussing all the possibilities.



I feel like you're trying to fit some other (possible, but far fetched) 
scenario from where we started. Mike, the op, said he has a 1002 which 
has an RP1 configured with 4GB of RAM. First, this is not a 1001. 
Second, SSO is off by default and is unlikely to be configured on an 
ASR1002 (the op certainly never stated he had enabled it). I just 
checked a few ASR 1k RP1s I have access to and the 3.10 IOS image has 
similar memory usage to the 3.13 (the latest MD release for this 
platform). On the RP1 platform, with a default only BGP feed, the 
systems have ~ 1.3GB of proc mem available. With a single full IP4 BGP 
feed, the free proc mem goes down to ~850MB. With two full feeds, the 
proc mem goes down to ~ 800MB. RP memory goes from ~1.8GB used (default 
only), ~2.7GB used (1x full), ~2.8GB used (2x full). This still leaves 
over 1GB of free RP memory with two full BGP feeds. The amount of FIB is 
dependent on the ESP installed by the OP; Mike hasn't yet stated what 
ESP he has installed.


So yes, the 1002 can, and will continue to, work with two full BGP feeds 
when fitted with an ESP10.

Re: BGP peering strategies for smaller routers

[Nick Hilliard]

William Herrin wrote:
> I respectfully disagree. Sourcing more ram won't fix the next bit of
> sloppiness with the software. Or the one after that. Once the manager
> of that team starts to accept poor code quality, the only thing with a
> chance of fixing it is strong customer push-back.

You could feasibly argue that the box was shipped with too little RAM
when it was released in 2008/2009.  Alternatively, you could argue that
4G was fine way back then and that it's hardly surprising that it's too
little now because the DFZ has grown from 220k prefixes to 620k.

Anyway, this is all completely academic because the box is end-of-life
and the chances of getting this problem fixed are zero.

So there is simply no point in berating some poor TAC representative
about this problem, end of story.

The replacement box (ASR1001-X) has 8G ram.  That's fine for today.  In
8 years, it probably won't be fine.

> And it is poor code quality. Even slicing and dicing the ram in odd
> ways, there's just no excuse for an order-of-magnitude increase in ram
> required to run the same algorithms on the same data.

If RAM were expensive, your argument would make sense, but RAM is not
expensive.

I don't really think it's any more viable to complain about an 8yo
end-of-life box being ill equipped to handle today's Internet than it is
to complain about the fact that you can't run a desktop operating system
on a computer with 8 megs of ram, like you used to be able to.  Truth
is, 8 megs of RAM then was more expensive than 8 gigs of RAM these days.

Can we move on now?

Nick

Re: BGP peering strategies for smaller routers

[William Herrin]

On Tue, May 3, 2016 at 5:13 PM, Łukasz Bromirski  wrote:
> On 03 May 2016, at 22:31, William Herrin  wrote:
>> IMO, you should not accept that answer from the TAC. An IOS release
>> that crashes with two 600k BGP feeds in 4 gigs of RAM is badly
>> defective.
>
> Not necessarily.
>
> In essence, your physical memory gets halved in two after
> router boots up, then it may be further halved if you’re
> using features like SSO. So, with 4GB RAM config and with
> SSO running, you may be left with around 600-650MB free after
> boot and with IOS-XE loaded, and then all the features kick
> in. Including your BGP feeds that need around 300MB of memory
> just to store the tables, then there’s CEF RAM representation,
> and so on.
>
> Here’s a good WP w/r to memory usage & architecture on ASR 1k:
> http://www.cisco.com/c/en/us/support/docs/routers/asr-1000-series-aggregation-services-routers/116777-technote-product-00.html

Hi Łukasz,

You make some great points and that's an excellent document.


> As Nick just responded, it’s faster to source the RAM or modify
> the config to cut down on number of BGP prefixes rather than
> ping back and forth here discussing all the possibilities.

I respectfully disagree. Sourcing more ram won't fix the next bit of
sloppiness with the software. Or the one after that. Once the manager
of that team starts to accept poor code quality, the only thing with a
chance of fixing it is strong customer push-back.

And it is poor code quality. Even slicing and dicing the ram in odd
ways, there's just no excuse for an order-of-magnitude increase in ram
required to run the same algorithms on the same data.

Regards,
Bill Herrin



-- 
William Herrin  her...@dirtside.com  b...@herrin.us
Owner, Dirtside Systems . Web: 

Re: BGP peering strategies for smaller routers

[Łukasz Bromirski]

> On 03 May 2016, at 22:31, William Herrin  wrote:
> 
> On Tue, May 3, 2016 at 3:50 PM, Gustav Ulander
>  wrote:
>> Yes I can confirm that we also had the issue with the asr1001s.
>> For us the router was fine until we upgraded it. When
>> we rebooted it after the upgrade it ran out of memory
>> when populating 2 full feeds.
>> When we contacted TAC they confirmed that indeed
>> it was a memory problem and that we would need to
>> add more memory to the box.
> 
> Hi Gustav,
> 
> IMO, you should not accept that answer from the TAC. An IOS release
> that crashes with two 600k BGP feeds in 4 gigs of RAM is badly
> defective.

Not necessarily.

In essence, your physical memory gets halved in two after
router boots up, then it may be further halved if you’re
using features like SSO. So, with 4GB RAM config and with
SSO running, you may be left with around 600-650MB free after
boot and with IOS-XE loaded, and then all the features kick
in. Including your BGP feeds that need around 300MB of memory
just to store the tables, then there’s CEF RAM representation,
and so on.

Here’s a good WP w/r to memory usage & architecture on ASR 1k:
http://www.cisco.com/c/en/us/support/docs/routers/asr-1000-series-aggregation-services-routers/116777-technote-product-00.html

It actually contains the same recommendation given by TAC -
with recent/current code if you want to run full tables with
BGP, get 8GB of RAM on ASR 1k. In the 3.10-3.12S era I believe
it was still possible to fit (without the SSO) full tables
in RAM and be fine. 

As Nick just responded, it’s faster to source the RAM or modify
the config to cut down on number of BGP prefixes rather than
ping back and forth here discussing all the possibilities.

-- 
Łukasz Bromirski
CCIE R/SP #15929, CCDE #2012::17, PGP Key ID: 0xFD077F6A

Re: BGP peering strategies for smaller routers

[Nick Hilliard]

William Herrin wrote:
> IMO, you should not accept that answer from the TAC. An IOS release
> that crashes with two 600k BGP feeds in 4 gigs of RAM is badly
> defective.

I suspect the time the OP would spend raging down the phone would be
better spent sourcing a third party memory upgrade to 8G or 16G.  The
upgrade would certainly be the cheaper option of the two, in addition to
being the only option with a useful outcome.

Nick

Re: BGP peering strategies for smaller routers

[William Herrin]

On Tue, May 3, 2016 at 3:50 PM, Gustav Ulander
 wrote:
> Yes I can confirm that we also had the issue with the asr1001s.
> For us the router was fine until we upgraded it. When
> we rebooted it after the upgrade it ran out of memory
> when populating 2 full feeds.
> When we contacted TAC they confirmed that indeed
> it was a memory problem and that we would need to
> add more memory to the box.

Hi Gustav,

IMO, you should not accept that answer from the TAC. An IOS release
that crashes with two 600k BGP feeds in 4 gigs of RAM is badly
defective.

Regards,
Bill Herrin



-- 
William Herrin  her...@dirtside.com  b...@herrin.us
Owner, Dirtside Systems . Web: 

Re: NOC AS1836 green.ch AG

[Nick Hilliard]

Fredrik Korsbäck wrote:
> As I've told you when you hunted me down the last time through backside 
> channels.. 

I can only hope you meant "back channels".

Nick

Re: NOC AS1836 green.ch AG

[Fredrik Korsbäck]

On 03/05/16 21:23, Marco Paesani wrote:
> Hi Arnold,
> nobody answer at 'peer...@green.ch' for this reason I write here on NANOG.
> Ciao,
> 
> 
> Marco Paesani
> 
> 
> Skype: mpaesani
> Mobile: +39 348 6019349
> Success depends on the right choice !
> Email: ma...@paesani.it
> 

Marco.

As I've told you when you hunted me down the last time through backside 
channels.. :)

When you don't get an answer on peering@ addresses its not because they don't 
read the emails. Instead of declining a peering-request formally people tend to 
just use the silent treatment and have that represent a "No". If you don't hear 
back - you probably don't fulfill the requirements to be eligible to peer with 
a network and for a big network with selective or restrictive policy, answering
"no" to emails all day long isn't productive for anyone.

This is one of MANY unwritten rules in the peering-world which can be hard to 
grasp at first. To understand the nomenclature and the "rules", going on a 
field trip to conferences such as NANOG, RIPE, EPF, GPF and the such is a great 
way to understand the game, so one can act accordingly to how the playfield 
looks like.

-- 
hugge @ AS2603

Re: Friday's Random Comment - About: Arista and FIB/RIB's

[Baldur Norddahl]

On 29 April 2016 at 22:25, Nick Hilliard  wrote:

> Baldur Norddahl wrote:
> > With two uplinks that is highly unlikely to the point of being
> impossible.
> > There is no topology change upstream that can cause a situation where it
> is
> > not possible to do a high degree of aggregation of the full default free
> > routing table before loading it in the FIB.
>
> which is why I qualified this in a previous posting:
>
> > The more paths you receive from different sources, the more likely it
> > is that this list of 120k "superfluous" prefixes will converge
> > towards zero.
>
> Agreed that small numbers of paths are most unlikely to create the
> conditions for this problem to occur.
>


I agree that a larger number of peers makes the situation more complicated.
It might warrant more studies.

Your thesis is that there might be a problem, but mine is there likely is
not. Let me argue why.

We can consider networks of various sizes:

1) the dual homed network with full tables
2) the lightly peered ISP with more than two full tables
3) the well peered ISP
4) tier 1 backbone provider

Each of those might experience different gain from the proposal and indeed
it is likely that the backbone provider would not be interested in the
solution no matter what. Even so the proposal could help deliver
considerable cheaper hardware solutions to say #1 and #2 class providers.

We already agree that the #1 class provider will not see an external event
that can explode the number of needed FIB entries after compression.

The #2 class provider is not much different. The number of routes he takes
in as peering routes as opposed to transit are few. If he runs his network
with proper max routes on every BGP session, there is nothing a free peer
can do to wreck havok. Any entity with say max routes 50 can only break up
a max of 50 of your optimized FIB entries and while that can cascade such a
/16 breaks into a series of /17, /18, /19, ..., /24 that will never add up
to anything that is a problem. In any case the real problem here will be a
rogue peer injecting fake routes into your network.

Can the more than two transit providers with full tables become a problem?
No not really. These guys are all sending mostly the same routes to you and
anything large happening will be reflected on all your transits.

There is also the point about the weekly routing report:

BGP routing table entries examined:  593320
Prefixes after maximum aggregation (per Origin AS):  217357
Deaggregation factor:  2.73
Unique aggregates announced (without unneeded subnets):  290159

Now can you really say any one entity has the power to magically make all
that aggregation disappear just so he can crash your network? I will put
that in the "impossible" and "the net already crashed long before that"
categories.

There is a trend that some network are deaggregating their prefixes. Why
not use software to aggregate that right back to what it ought to be before
loading the routes into FIB? According to the above stat, that would save
at least half the FIB memory and make some routers able to handle full
tables for very much longer (possible forever).

Regards,

Baldur

SV: BGP peering strategies for smaller routers

[Gustav Ulander]

Hello all.

Yes I can confirm that we also had the issue with the asr1001s. 
For us the router was fine until we upgraded it. When we rebooted it after the 
upgrade it ran out of memory when populating 2 full feeds. 
When we contacted TAC they confirmed that indeed it was a memory problem and 
that we would need to add more memory to the box. 
Perhaps 1002 isnt as thirsty? 

//Gustav

-Ursprungligt meddelande-
Från: NANOG [mailto:nanog-boun...@nanog.org] För William Herrin
Skickat: den 3 maj 2016 18:13
Till: Eric Sabotta 
Kopia: NANOG list 
Ämne: Re: BGP peering strategies for smaller routers

On Mon, May 2, 2016 at 10:35 PM, Eric Sabotta  wrote:
> I just did this with a ASR1001.  I had to upgrade it to 8gb of ram (I 
> got the real Cisco stuff for ~ $500).  Before the router would crash 
> when loading the tables.

Hi Eric,

Something very fishy there because:

> router1#show ip bgp summary
> BGP using 296,835,018 total bytes of memory

Commas added for clarity.

Regards,
Bill Herrin




--
William Herrin  her...@dirtside.com  b...@herrin.us Owner, 
Dirtside Systems . Web: 

Re: NOC AS1836 green.ch AG

[Marco Paesani]

Hi Arnold,
nobody answer at 'peer...@green.ch' for this reason I write here on NANOG.
Ciao,


Marco Paesani


Skype: mpaesani
Mobile: +39 348 6019349
Success depends on the right choice !
Email: ma...@paesani.it



2016-05-03 20:41 GMT+02:00 Arnold Nipper :

> On 03.05.2016 20:05, Marco Paesani wrote:
>
> > I need direct contact with NOC of AS1836.
> > Can you help me ?
>
> Have a look at PeeringDB [0], Marco!
>
>
> Ciao, Arnold
> [0] https://peeringdb.com/net/232
> --
> Arnold Nipper
> email: arn...@nipper.de  phone: +49 6224 5593407 2
> mobile: +49 172 2650958  fax:   +49 6224 5593407 9
>
>

NOC AS1836 green.ch AG

[Marco Paesani]

Hi,
I need direct contact with NOC of AS1836.
Can you help me ?
Kind regards,

Marco Paesani


Skype: mpaesani
Mobile: +39 348 6019349
Success depends on the right choice !
Email: ma...@paesani.it

[NANOG-announce] NANOG 67 Announcements

[Betty Burke <be...@nanog.org>]

NANOGers,

We are beginning our final preparations in support of NANOG 67
 taking place June 13-15, 2016 in
Chicago, IL.   The meeting will be held at the Fairmont Chicago Millennium
Park.

In addition, prior to the start of NANOG 67, The NANOG Program Committee is
pleased to announce the first NANOG one-day Hackathon, Sunday, June 12, 2016
,  at the Fairmont
Chicago Millennium Park.  The event will feature new ideas and hacks for
automating production Internet networks. The NANOG event is planned in
partnership with FaceBook and will require a separate registration
process.  For those considering the NANOG Hackation, attendance is free,
however you must register here

.

The NANOG Program Committee continues in its dedication to present NANOG 67
attendees with quality presentations, track discussions, and tutorials.
The agenda topics are confirmed and posted on the NANOG Highlights
 page.  The draft
agenda is expected on or near May 16, 2016.  For those considering the
NANOG 67 Conference, please be aware the registration fee
 will increase soon!
Also, take a moment to join NANOG or renew your existing Membership
.  As a member, you receive a discounted
registration rate, a dedicated line at the registration desk, and the
ability to participate in annual elections and discussions about NANOG.

• Standard Registration now thru May 22, 2016
(member $500, non-member $525, student $100)

• Late Registration starting May 23, 2016
(member $575, non-member $600, student $100)

• On-Site Registration starting June 12, 2016
(member $650, non-member $675, student $100)


The conference hotel is sold out on Sunday night.  However, an additional
NANOG Room block remains available at the Hard Rock Hotel.  Both room
blocks are set to expire very soon.  Be sure to get your reservation made
ASAP.  We continue to seek and secure additional room nights, and will post
the information on the NANOG 67 hotel
 meeting page.

Should you have any questions regarding the Hackaton or NANOG 67, please
direct them to nanog-supp...@nanog.org.  If you are interested in
sponsorship opportunities, please email sponsor-supp...@naong.org.

I look forward to an exciting set of events in Chicago, and hope to see you
there!


Sincerely,

Betty


Betty J. Burke
NANOG Executive Director
2864 Carpenter Rd., Ste 100
Ann Arbor, MI 48108
+1 866-902-1336
___
NANOG-announce mailing list
nanog-annou...@mailman.nanog.org
http://mailman.nanog.org/mailman/listinfo/nanog-announce

Re: Fwd: hotel

[Chris Grundemann]

On Tue, May 3, 2016 at 1:22 AM, Randy Bush  wrote:

> > I tried booking earlier today, had the same issue and called in.  I was
> > told they were now full, and only non-block rooms were available (@ >
> > $500/night).
>
> find a non-exhorbitant fall-back?


 The Sheraton Grand Chicago is close and appears to have not-quite-exorbitant
rates (~$400).

Re: BGP peering strategies for smaller routers

[William Herrin]

On Mon, May 2, 2016 at 10:35 PM, Eric Sabotta  wrote:
> I just did this with a ASR1001.  I had to upgrade it to 8gb of ram
> (I got the real Cisco stuff for ~ $500).  Before the router would
> crash when loading the tables.

Hi Eric,

Something very fishy there because:

> router1#show ip bgp summary
> BGP using 296,835,018 total bytes of memory

Commas added for clarity.

Regards,
Bill Herrin




-- 
William Herrin  her...@dirtside.com  b...@herrin.us
Owner, Dirtside Systems . Web: 

RE: BGP peering strategies for smaller routers

[Eric Sabotta]

Mike,

I just did this with a ASR1001.  I had to upgrade it to 8gb of ram (I got the 
real Cisco stuff for ~ $500).  Before the router would crash when loading the 
tables.

Right now, I have full tables from two providers:

router1#show ip bgp summary
BGP router identifier 192.55.82.2, local AS number 4505
BGP table version is 11150622, main routing table version 11150622
582461 network entries using 144450328 bytes of memory
911730 path entries using 109407600 bytes of memory
148924/93298 BGP path/bestpath attribute entries using 36933152 bytes of memory
132977 BGP AS-PATH entries using 6043938 bytes of memory
0 BGP route-map cache entries using 0 bytes of memory
0 BGP filter-list cache entries using 0 bytes of memory
BGP using 296835018 total bytes of memory
BGP activity 962568/380103 prefixes, 5155645/4243915 paths, scan interval 60 
secs

NeighborV   AS MsgRcvd MsgSent   TblVer  InQ OutQ Up/Down  
State/PfxRcd
192.55.82.3 4 4505 2532914 1634867 1115062200 3w0d   
330377
192.55.82.4 4 4505  672950 1634865 1115062200 3w0d  
  1
209.117.103.33  4 2828 1837130   48052 1115055700 2w1d   
581351

router1#show ip cef summary
IPv4 CEF is enabled for distributed and running
VRF Default
 582527 prefixes (582527/0 fwd/non-fwd)
 Table id 0x0
 Database epoch:2 (582527 entries at this epoch)

-Eric



-Original Message-
From: NANOG [mailto:nanog-boun...@nanog.org] On Behalf Of Mike
Sent: Monday, May 2, 2016 3:07 PM
To: NANOG list 
Subject: BGP peering strategies for smaller routers

Hello,

 I have an ASR1000 router with 4gb of ram. The specs say I can get
'1 million routes' on it, but as far as I have been advised, a full table of 
internet routes numbers more than 530k by itself, so taking 2 full tables seems 
to be out of the question (?).

  I am looking to connect to a second ip transit provider and I'm looking 
for any advice or strategies that would allow me to take advantage and make 
good forwarding decisions while not breaking the bank on bgp memory 
consumption. I simply don't understand how this would likely play out and what 
memory consumption mitigation steps may be necessary here. Im open to ideas... 
a pair of route reflectors? 
selective bgp download? static route filter maps?

Thank you.

Mike-

Re: hotel

[Jérôme Fleury]

Same here - only availability is from Monday 13th to Thursday 16th

On Mon, May 2, 2016 at 7:26 PM, Randy Bush  wrote:

> excuse puking on list but the path to nanog admin action seems dead
>
> Date: Sun, 01 May 2016 13:48:10 +0900
> From: Randy Bush 
> To: action 
> Subject: hotel
>
> hi,
>
> sorry to bother, but
>
> fairmont chicago block supposedly good to 22 may.  tried to book just
> now, arriving 11th leaving 16th.  got told "No lodging matches your
> search criteria. Please change your search options at right and try
> again."
>
> thanks
>
> randy
>

Re: BGP peering strategies for smaller routers

[William Herrin]

On Mon, May 2, 2016 at 3:07 PM, Mike  wrote:
> I have an ASR1000 router with 4gb of ram. The specs say I can get '1
> million routes' on it, but as far as I have been advised, a full table of
> internet routes numbers more than 530k by itself, so taking 2 full tables
> seems to be out of the question (?).

Hi Mike,

There is no "routing table." Instead:

Routing Information Bases (RIB)
Forwarding Information Base (FIB)

RIB = the routing data from your neighbors. Each protocol (e.g. BGP,
OSPF, static, connected) has a RIB. If you have virtual routers (VRF)
then each VRF has its own RIBs. For the BGP RIB, each prefix (route)
from each neighbor will need its own slot in the RIB.

FIB = the next hop table. Only the best next hop for each prefix is
stored in the FIB. One FIB per protocol (IPv4, IPv6), unless you run
VRFs, then one FIB each per VRF.

All routers have both a FIB and RIBs. All routers keep the RIBs in
DRAM. Big Iron like yours typically store the FIB in special hardware
called Ternary Content Addressable Memory (TCAM). You're looking at
two physically different kinds of memory in your router to store the
RIB and the FIB. No matter how much DRAM you have, you only have so
much TCAM in that routing engine. The TCAM is not upgradable. In fact,
it's buried under a big heat sink.

All the RIBs are processed down to a single FIB for each protocol/VRF.
The best next hop is selected from all the possibilities in the RIBs
and only that one gets stored in the FIB. The FIB in the TCAM is
consulted every single time a packet is routed. The RIBs are only
consulted when new information is received from a neighbor of when the
FIB table needs to be rebuilt. The RIB is not consulted to route
individual packets.

Your "million routes" is a reflection of the TCAM on your routing
engine. The FIB table size. 4G DRAM is enough for 10-15 million routes
in the various RIBs. If you're running a single VRF, you have enough
FIB headroom for the next two or three years, until the v4 and v6 BGP
tables add up to around 900M prefixes. You already have too little FIB
headroom to run two BGP-speaking VRFs.


My piddly little 2811 carries a full IPv4 table, 580k routes. Its BGP
RIB consumes all of 154 megabytes of DRAM. Unlike your router, the
2811 does not have a "hardware fast path." No TCAM. It stores the FIB
in a radix tree in DRAM instead. As a result, it can only handle low
data rates, in the 10s of megabits per second.

I'm leaving out lots of details, but these are the most important with
respect to your question.


Regards,
Bill Herrin



-- 
William Herrin  her...@dirtside.com  b...@herrin.us
Owner, Dirtside Systems . Web: 

RE: Arista Routing Solutions

[Timothy Creswick]

> We're going to be getting some Arista gear soon and this issue came
> up. They made the same noises and vague overtures of "well, you
> *might* have problems with TAC if you go with 3rd party optics"...
> until I said "Oh really- well, that's a deal breaker, we can't really
> even consider that". And then they backpedaled at light speed and
> reassured me that 3rd party optics would be fine, they just "had to
> have the conversation".

Similar experience here, but the conversation went on far too long and 
ultimately lost Arista the deal. There was a ridiculous amount of insistence 
that we would have to carry a "stock of Arista optics", but every attempt to 
clarify exactly what that meant (how many, what they would cost etc) failed to 
get a straight answer.

It's 2016 and stupid conversations about vendor optics waste time and destroy 
deals.

The slight difference here is that pretty much the first thing we said to 
Arista was that transceivers were out of the question unless they could price 
them reasonably** (they chose not to). On this particular deal we were probably 
only talking about 500 SR 10G transceivers.

We've had similar conversations with Extreme, Brocade, Solarflare and Juniper, 
all of whom are quite happy with us running our own parts. Solarflare even 
certified our parts and put them on their website 
(http://solarflare.com/transceivers-and-cables).

> Also talked to a local Arista customer, much bigger than us and using
> a lot more of their gear. They have 0 Arista optics and 0 problems
> with 3rd party for a few years now. IMHO the whole thing was just
> sales guy FUD to try to squeeze a few extra bucks out.

Doesn't surprise me, and I'm sure if we'd pushed for another week we could have 
got to this position. Unfortunately for Arista, there was another vendor quite 
happy to get the deal done faster and without all the BS so we voted with our 
feet.

Clearly, mileage will vary on this one.

T



** in this context, "reasonably" means no more than _double_ what I currently 
buy at.

Re: BGP peering strategies for smaller routers

[Ken Chase]

got a quagga router in my life where bgpd+zebra takes up 1gig for 4.5 full
tables. Rest of the OS easily lives in 1 gig (could probably be much less.)
big-vendor solutions always seem much bloatier - same deal on power usage.

just a data point.

/kc
--
Ken Chase - m...@sizone.org Toronto

Re: BGP peering strategies for smaller routers

[Blake Hudson]

Mike wrote on 5/2/2016 9:43 PM:

On 05/02/2016 07:35 PM, Eric Sabotta wrote:

Mike,

I just did this with a ASR1001.  I had to upgrade it to 8gb of ram (I 
got the real Cisco stuff for ~ $500).  Before the router would crash 
when loading the tables.


Right now, I have full tables from two providers:

router1#show ip bgp summary
BGP router identifier 192.55.82.2, local AS number 4505
BGP table version is 11150622, main routing table version 11150622
582461 network entries using 144450328 bytes of memory
911730 path entries using 109407600 bytes of memory
148924/93298 BGP path/bestpath attribute entries using 36933152 bytes 
of memory

132977 BGP AS-PATH entries using 6043938 bytes of memory
0 BGP route-map cache entries using 0 bytes of memory
0 BGP filter-list cache entries using 0 bytes of memory
BGP using 296835018 total bytes of memory
BGP activity 962568/380103 prefixes, 5155645/4243915 paths, scan 
interval 60 secs


NeighborV   AS MsgRcvd MsgSent   TblVer  InQ OutQ 
Up/Down  State/PfxRcd
192.55.82.3 4 4505 2532914 1634867 111506220 0 
3w0d   330377
192.55.82.4 4 4505  672950 1634865 111506220 0 
3w0d1
209.117.103.33  4 2828 1837130   48052 111505570 0 
2w1d   581351


router1#show ip cef summary
IPv4 CEF is enabled for distributed and running
VRF Default
  582527 prefixes (582527/0 fwd/non-fwd)
  Table id 0x0
  Database epoch:2 (582527 entries at this epoch)

-Eric



But if I'm reading the above right, it looks like bgp is eating ~300mb 
on your box.


BGP using 296835018 total bytes of memory

You would seem to have plenty of free ram. In my case, the ASR1002 
doesn't have upgradable memory anyways so I'm stuck.


Mike-


Mike, I have a customer that has not had any operational issues taking 
two full IP4 feeds on an ASR1002 with RP1 @ 4GB of RAM. Again, I'd 
recommend an ESP10 to ensure you have enough TCAM to hold the FIB and to 
track netflow or other data that relies on the ESP. I can't comment on 
the RP2's memory usage or stability as I don't think I've ran into them.


Here's the output to show you what to expect:

#sh processes memory | inc BGP|Total:|PID
Processor Pool Total: 1725514176 Used:  918478524 Free:  807035652
 lsmpi_io Pool Total:6295088 Used:6294116 Free:972
 PID TTY  Allocated  FreedHoldingGetbufsRetbufs Process
 114   0252  0  23264  4  4 BGP 
Scheduler

 282   0 165948 327892 189212  0  0 BGP Task
 294   0  0  0  17264  0  0 BGP HA SSO
 317   0  610260780  0 22008033877153387715 BGP I/O
 355   0  0   17841472  23264  0  0 BGP 
Scanner
 405   0  0  0  23316  0  0 BGP 
Consistency

 422   0  0  0  23264  0  0 BGP Event
 528   0  560439808  646907016  498109200 51 51 BGP Router
 533   0  0  0  17264  0  0 BGP VA


#sh bgp su
BGP router identifier 1.2.3.4, local AS number c
BGP table version is 66249122, main routing table version 66249122
598594 network entries using 86197536 bytes of memory
1177526 path entries using 94202080 bytes of memory
208207/100401 BGP path/bestpath attribute entries using 28316152 bytes 
of memory

179429 BGP AS-PATH entries using 7090834 bytes of memory
4342 BGP community entries using 230370 bytes of memory
1 BGP extended community entries using 24 bytes of memory
0 BGP route-map cache entries using 0 bytes of memory
0 BGP filter-list cache entries using 0 bytes of memory
BGP using 216036996 total bytes of memory
BGP activity 2943907/2345309 prefixes, 10963622/9786096 paths, scan 
interval 60 secs


NeighborV   AS MsgRcvd MsgSent   TblVer  InQ OutQ 
Up/Down  State/PfxRcd

a.a.a.a 4  aaa 5523968   51273 6624907000 4w4d 578933
b.b.b.b 4b 3731440  101400 6624907000 4w4d 598586

#sh cef fib
599851 allocated IPv4 entries, 0 failed allocations
1 allocated IPv6 entry, 0 failed allocations

#sh ip route summary
IP routing table name is default (0x0)
IP routing table maximum-paths is 32
Route SourceNetworksSubnets Replicates  Overhead Memory (bytes)
connected   0   60  0   3600 10800
static  20  79  0   6060 17820
bgp c   180614  417599  0   35892780 107678340
  External: 598213 Internal: 0 Local: 0
internal6552 23993840
Total   187186  417738  0   35902440 131700800

CFP: DNS Track at NANOG 67

[Wessels, Duane]

Greetings,

For our upcoming meeting in Chicago I'm looking for folks willing to give brief 
presentations during a proposed DNS Track.  Possible topics include:

- Operational experiences
- New & interesting software features
- Protocol advancements
- Research results
- Performance & compliance testing
- Insights into DNS-related attacks

I'm expecting to have about 90 minutes to fill with presentations of about 15 
minutes or less.  If you're interested in presenting please respond to me 
directly.

Duane W.


signature.asc
Description: Message signed with OpenPGP using GPGMail