Re: NetFlow - path from Routers to Collector

On 1 September 2015 at 16:33, Serge Vautour wrote: > Hello, > > For those than run Internet connected routers, how do you get your NetFlow > data from the routers to your collectors? Do you let the flow export traffic > use the same links as your customer traffic to route

RE: NetFlow - path from Routers to Collector

you end up losing data or visibility while routes reconverge. -Original Message- From: NANOG [mailto:nanog-bounces+esundberg=nitelusa@nanog.org] On Behalf Of James Bensley Sent: Friday, September 11, 2015 3:35 AM To: se...@nbnet.nb.ca; nanog@nanog.org Subject: Re: NetFlow - path from

RE: NetFlow - path from Routers to Collector

6:23 PM To: nanog@nanog.org Subject: Re: NetFlow - path from Routers to Collector Absolutely. All kinds of creative lashups to get console access in difficult situations (and, as you noted previously, an increasing number of devices don't support serial console at all, which is highly

RE: NetFlow - path from Routers to Collector

e St. Louis, MO 63131 -Original Message- From: NANOG [mailto:nanog-boun...@nanog.org] On Behalf Of Frank Bulk Sent: Thursday, September 10, 2015 9:37 AM To: 'Roland Dobbins'; nanog@nanog.org Subject: RE: NetFlow - path from Routers to Collector Does anyone else have a serial to IP dongle

Re: NetFlow - path from Routers to Collector

e St. Louis, MO 63131 -Original Message- From: NANOG [mailto:nanog-boun...@nanog.org] On Behalf Of Frank Bulk Sent: Thursday, September 10, 2015 9:37 AM To: 'Roland Dobbins'; nanog@nanog.org Subject: RE: NetFlow - path from Routers to Collector Does anyone else have a serial to IP dongle

RE: NetFlow - path from Routers to Collector

How many IPv6 addresses do you get? Frank -Original Message- From: NANOG [mailto:nanog-boun...@nanog.org] On Behalf Of Avi Freedman Sent: Tuesday, September 01, 2015 7:31 PM To: Jared Mauch <ja...@puck.nether.net> Cc: NANOG <nanog@nanog.org> Subject: Re: NetFlow - path

Re: NetFlow - path from Routers to Collector

On 1/Sep/15 17:43, Roland Dobbins wrote: > Until there is. As with everything in life... Mark.

Re: NetFlow - path from Routers to Collector

On 1/Sep/15 17:59, Rod Beck wrote: > Roland is correct. With the caveat that your Internet customer traffic may > flow over the fibers as your separate management circuits. You should aim for > end to end physical diversity. This is all common sense, but laziness some > times takes

Re: NetFlow - path from Routers to Collector

On 1/Sep/15 19:12, Roland Dobbins wrote: > > > > Running flow telemetry in-band is penny-wise and pound-foolish, for > networks of any size, in any circumstances. All management-plane > traffic (and that's what flow telemetry is) should be segregated from > the production network data plane.

Re: NetFlow - path from Routers to Collector

> "Roland" == Roland Dobbins writes: Roland> On 2 Sep 2015, at 0:08, Steve Meuse wrote: >> Your advice is not "one size fits all". Roland> Actually, it is. Roland> Large backbone networks have DCNs/OOBs, and that's where they export Roland> their

Re: NetFlow - path from Routers to Collector

twork is require to secure > the flow data? > > Thanks again, your comments are very helpful. > > Serge > > ------------ > On Tue, 9/1/15, Serge Vautour <sergevaut...@yahoo.ca> wrote: > > Subject: NetFlow - path from Routers t

Re: NetFlow - path from Routers to Collector

Adding VRFs/VLAN's/anything else to separate the traffic to reduce fate sharing is only adding complexity that will likely result in operator errors. While many of us have clue, even when we don't agree on the solutions, there are many more out there typing at routers at 2am, when even the

Re: NetFlow - path from Routers to Collector

On 2/Sep/15 12:11, Roland Dobbins wrote: > > > Sure. But it's better than mixing it in with customer traffic. Does not make much of a difference - it's the same data plane infrastructure. > > Ideally, it should be - that's what I was trying to get across. I > understand that this isn't

Re: NetFlow - path from Routers to Collector

On 2 Sep 2015, at 22:22, Mark Tinka wrote: As you, Jared, say, and as I said in a previous post, both MPLS and IP traffic follows the same data plane. The routing table separation construct does not survive chassis-wide failures. Everyone here understands that. We also understand that pps

Re: NetFlow - path from Routers to Collector

On 2 Sep 2015, at 23:29, Serge Vautour wrote: > I assume if someone has the ability to do so, you've got bigger problems. This is the key, IMHO. --- Roland Dobbins

Re: NetFlow - path from Routers to Collector

On 2 Sep 2015, at 22:26, Mark Tinka wrote: When the line card congests, it doesn't care that one bit was part of a VRF and the other doesn't. It all goes kaboom (even with the best of QoS intentions). You don't necessarily have to put everything on the same fiber, interface, the same ASIC

Re: NetFlow - path from Routers to Collector

On 2/Sep/15 16:08, Jared Mauch wrote: > It’s really because some people who drink the MPLS/VPN/VRF/VLAN kook-aid > think it’s some magic that undoes fate sharing and proper engineering and > planning. That a few bytes for a label of VLAN tag make your data more > secure. > > It’s possible

Re: NetFlow - path from Routers to Collector

On 2/Sep/15 16:13, Roland Dobbins wrote: > > > But keeping stuff separate at the IP logical network level is better > than mixing it together, even on the same hardware. But how, Roland. When the line card congests, it doesn't care that one bit was part of a VRF and the other doesn't. It all

Re: NetFlow - path from Routers to Collector

On 2 Sep 2015, at 23:20, jim deleskie wrote: The stats prove out these types of errors are more likely to cause an outage then DDoS or anything else. Completely concur that there are always complexity tradeoffs. And of course, the goal is not to have to type on routers at all, or at least

Re: NetFlow - path from Routers to Collector

-- On Tue, 9/1/15, Serge Vautour <sergevaut...@yahoo.ca> wrote: Subject: NetFlow - path from Routers to Collector To: nanog@nanog.org Received: Tuesday, September 1, 2015, 12:33 PM Hello, For those than run Internet connected routers, how do you get your NetFlow data from the router

Re: NetFlow - path from Routers to Collector

On 2 Sep 2015, at 20:25, Niels Bakker wrote: Why? Do your customer packets have cooties? Because you don't want things which disrupt customer traffic to disrupt your ability to see what's happening. Just as you don't want it to disrupt your ability to configure/manage your

Re: NetFlow - path from Routers to Collector

> On Sep 2, 2015, at 10:02 AM, Roland Dobbins wrote: > > On 2 Sep 2015, at 20:25, Niels Bakker wrote: > >> Why? Do your customer packets have cooties? > > Because you don't want things which disrupt customer traffic to disrupt your > ability to see what's happening.

Re: NetFlow - path from Routers to Collector

On 2 Sep 2015, at 13:25, Pierfrancesco Caci wrote: 2 out of 2 large backbone networks I've experience with use inband for flow export. There was supposed to be a 'should' in there, apologies. I've already clarified that VLAN/VRF is Good Enough for flow telemetry, and that most who separate

Re: NetFlow - path from Routers to Collector

On 2 Sep 2015, at 13:02, Mark Tinka wrote: Not very straight forward when you have a network spanning several continents. Again, VLANs/VRFs are generally Good Enough, and more than a few globe-spanning networks do just that. --- Roland Dobbins

Re: NetFlow - path from Routers to Collector

On 2 Sep 2015, at 12:50, Avi Freedman wrote: + optimal tweaks include local flow proxy that can also rate limit / re-sample, and send topk talkers over 'true' OOB. Yes, a lot of distributed flow collection/analysis architectures are deployed this way, doing more than top-talkers, even. The

Re: NetFlow - path from Routers to Collector

On 2/Sep/15 11:38, Roland Dobbins wrote: > > > Again, VLANs/VRFs are generally Good Enough, and more than a few > globe-spanning networks do just that. Those VLAN's and VRF's are following the same path as the global table, just in a different routing table. That is easy, and we do that

Re: NetFlow - path from Routers to Collector

On 2 Sep 2015, at 16:48, Mark Tinka wrote: Those VLAN's and VRF's are following the same path as the global table, just in a different routing table. That is easy, and we do that already. Sure. But it's better than mixing it in with customer traffic. Your assertion, before, was that the

Re: NetFlow - path from Routers to Collector

* rdobb...@arbor.net (Roland Dobbins) [Wed 02 Sep 2015, 12:12 CEST]: On 2 Sep 2015, at 16:48, Mark Tinka wrote: Those VLAN's and VRF's are following the same path as the global table, just in a different routing table. That is easy, and we do that already. Sure. But it's better than mixing

Re: NetFlow - path from Routers to Collector

On 2 Sep 2015, at 21:08, Jared Mauch wrote: I see where Roland is trying to go and he’s in the “magic byte” realm of the extra label makes it “OOB” where as the rest of us just see 1’s and 0’s on the wire and know a bit is a bit regardless of tag-switching (the original name for MPLS) or

Re: NetFlow - path from Routers to Collector

* rdobb...@arbor.net (Roland Dobbins) [Wed 02 Sep 2015, 01:06 CEST]: Sure, or a VRF, or whatever. You just moved the goalposts. :> -- Niels.

Re: NetFlow - path from Routers to Collector

On 2 Sep 2015, at 0:18, Niels Bakker wrote: You're just wrong here. Sorry, I'm not. I've seen what happens when flow telemetry is 'squeezed out' by pipe-filling DDoS attacks, interrupted by fat-fingers, etc. It'll happen to you, one day. And then you'll understand.

Re: NetFlow - path from Routers to Collector

On 9/1/15, 1:36 PM, "NANOG on behalf of Roland Dobbins" wrote: >It should've already been spent for an OOB/DCN network, which should've >been provisioned with flow telemetry in mind. I'm going to interpret that "should" in the same way

Re: NetFlow - path from Routers to Collector

Roland, While your way may be best practice, sometimes real life gets in the way of best practice. Shane On 9/1/15 1:12 PM, Roland Dobbins wrote: On 2 Sep 2015, at 0:08, Steve Meuse wrote: Your advice is not "one size fits all". Actually, it is. Large backbone networks have DCNs/OOBs,

Re: NetFlow - path from Routers to Collector

Looking at probably 100 networks' flow paths over the last year, I'd say 1 or 2 have OOB for flow. Maybe another 10-20 have interest in taking simpler time series data of top talkers over their OOB networks, but not the flow itself. Agree w Roland that it can cause problems with telemetry if

Re: NetFlow - path from Routers to Collector

On Tue, Sep 1, 2015 at 12:14 PM, Roland Dobbins wrote: > On 1 Sep 2015, at 23:10, Job Snijders wrote: > > > To answer your first question: i see no issue in transporting flow > > export traffic over the same backbone used to serve customer traffic. > > This is not good

Re: NetFlow - path from Routers to Collector

* rdobb...@arbor.net (Roland Dobbins) [Tue 01 Sep 2015, 19:13 CEST]: Running flow telemetry in-band is penny-wise and pound-foolish, for networks of any size, in any circumstances. You're just wrong here. -- Niels.

Re: NetFlow - path from Routers to Collector

I think the key here is that Roland isn't often constrained by these financial considerations. I would respectfully disagree with Roland here and agree with Job, Niels, etc. A few networks have robust out of band networks, but most I've seen have an interesting mixture of

Re: NetFlow - path from Routers to Collector

* rdobb...@arbor.net (Roland Dobbins) [Tue 01 Sep 2015, 19:30 CEST]: On 2 Sep 2015, at 0:18, Niels Bakker wrote: You're just wrong here. Sorry, I'm not. I've seen what happens when flow telemetry is 'squeezed out' by pipe-filling DDoS attacks, interrupted by fat-fingers, etc. This is

Re: NetFlow - path from Routers to Collector

hey, It should've already been spent for an OOB/DCN network, which should've been provisioned with flow telemetry in mind. Bad advice. No amount of money will fix major platforms that are not happy to export flow telemetry via router management ports. Sometimes it can be done via nasty vrf

Re: NetFlow - path from Routers to Collector

On 2 Sep 2015, at 0:08, Steve Meuse wrote: Your advice is not "one size fits all". Actually, it is. Large backbone networks have DCNs/OOBs, and that's where they export their NDE. I've done netflow over production links for two very large backbone networks. Did you manage your routers

Re: NetFlow - path from Routers to Collector

So in your world, the money always exists for a separate flow telemetry network? On 9/1/15 1:29 PM, Roland Dobbins wrote: On 2 Sep 2015, at 0:18, Niels Bakker wrote: You're just wrong here. Sorry, I'm not. I've seen what happens when flow telemetry is 'squeezed out' by pipe-filling DDoS

Re: NetFlow - path from Routers to Collector

On 2 Sep 2015, at 0:32, Shane Ronan wrote: So in your world, the money always exists for a separate flow telemetry network? It should've already been spent for an OOB/DCN network, which should've been provisioned with flow telemetry in mind. --- Roland

Re: NetFlow - path from Routers to Collector

(Jared wrote): > Most people I've seen have little data or insight into their > networks, or don't have the level that they would desire as > tools are expensive or impossible to justify due to capital costs. > Tossing in a recurring opex cost of DC XC fee + transport + XC fee + >

Re: NetFlow - path from Routers to Collector

On 2 Sep 2015, at 3:45, Chuck Church wrote: Most OOB is lacking redundancy too, so a single failure can really take the shine off an OOB deployment. Even if you're using old, grandfathered equipment for it, there's no reason why your OOB/DCN can't have a reasonable degree of redundancy.

Re: NetFlow - path from Routers to Collector

> On Sep 1, 2015, at 6:51 PM, Roland Dobbins wrote: > > > On 2 Sep 2015, at 5:38, Jared Mauch wrote: > >> Please stop digging, > > Since I'm not digging, I've no reason to stop. I see and deal with the > various quirks of more different platforms exporting flow

Re: NetFlow - path from Routers to Collector

(Said Roland:) > Again, to clarify - I count VLANs/VRFs as being sufficiently out-of-band > to handle flow telemetry on a reasonable basis without mixing it in with > customer traffic. > > That changes the ratio. > I agree with you, Avi, and others that a dedicated OOB network *just for >

Re: NetFlow - path from Routers to Collector

On 2 Sep 2015, at 7:27, Avi Freedman wrote: We see well under 20% doing logical separation but definitely folks doing it... ~20% matches our subjective observations, as well. We're doing our best to increase that number. --- Roland Dobbins

Re: NetFlow - path from Routers to Collector

--- rdobb...@arbor.net wrote: Again, to clarify - I count VLANs/VRFs as being sufficiently out-of-band to handle flow telemetry on a reasonable basis without mixing it in with customer traffic. --- Glad you clarified that. Now, I understand your stance.

Re: NetFlow - path from Routers to Collector

I've not read every reply, but let me add my voice as some who has worked on and ran SEVERAL large networks, in no case in the last long number of years have I had access to an OOB network that was sized to carry anything in large volume, and in fact like many others replied on a robust number of

Re: NetFlow - path from Routers to Collector

On 2 Sep 2015, at 0:44, Jared Mauch wrote: I think the key here is that Roland isn't often constrained by these financial considerations. That's entirely true. I deal every day with customers who are, though. I would respectfully disagree with Roland here and agree with Job, Niels, etc.

Re: NetFlow - path from Routers to Collector

On 2 Sep 2015, at 5:49, Jared Mauch wrote: Other platforms (e.g.: IOS-XR based) have issues with the MgmtEther interfaces which make them inoperable for many use-cases. I'm agreeing with you. Dedicated management ports on many boxes don't actually support important management-plane

Re: NetFlow - path from Routers to Collector

On 2 Sep 2015, at 0:46, Niels Bakker wrote: This is the dumbest thing I've read on this mailing list in a while. It happens. You can deny it all you like, but I've seen it happen, and the resultant confusion and additional time to resolve problems it causes.

Re: NetFlow - path from Routers to Collector

On 2 Sep 2015, at 6:06, Jared Mauch wrote: You are, Avi has said that the number of people with a network is outnumbered about 50:1 using his most favorable numbers. Again, to clarify - I count VLANs/VRFs as being sufficiently out-of-band to handle flow telemetry on a reasonable basis

Re: NetFlow - path from Routers to Collector

On 2 Sep 2015, at 7:38, jim deleskie wrote: These networks survived many "large" DDoS attacks and far more fat finger incidents then I like to think about. What I'm saying is that keeping flow telemetry and other management-plane traffic from mixing with customer data-plane traffic is

Re: NetFlow - path from Routers to Collector

On 2 Sep 2015, at 5:38, Jared Mauch wrote: Please stop digging, Since I'm not digging, I've no reason to stop. I see and deal with the various quirks of more different platforms exporting flow telemetry than most folks, all day, every day, so I know just a little bit about this topic.

RE: NetFlow - path from Routers to Collector

Message- From: NANOG [mailto:nanog-boun...@nanog.org] On Behalf Of Tarko Tikan Sent: Tuesday, September 01, 2015 3:47 PM To: nanog@nanog.org Subject: Re: NetFlow - path from Routers to Collector hey, > It should've already been spent for an OOB/DCN network, which > should've been provi

Re: NetFlow - path from Routers to Collector

In a message written on Wed, Sep 02, 2015 at 12:29:25AM +0700, Roland Dobbins wrote: > On 2 Sep 2015, at 0:18, Niels Bakker wrote: > > You're just wrong here. > > Sorry, I'm not. I've seen what happens when flow telemetry is 'squeezed > out' by pipe-filling DDoS attacks, interrupted by

Re: NetFlow - path from Routers to Collector

On 2 Sep 2015, at 3:34, Nick Hilliard wrote: > If you want to handle netflow data export for large amounts of traffic, it > would be pretty dumb to push it through the management plane of the router. Concur 100%. You must use a port capable of doing so. ---

Re: NetFlow - path from Routers to Collector

> On Sep 1, 2015, at 6:36 PM, Roland Dobbins wrote: > > On 2 Sep 2015, at 3:24, Niels Bakker wrote: > >> Because variety of flow telemetry delivery options isn't the #1 ranked >> purchasing decider. > > Actually, it is more often than you think. No use routing packets if

Re: NetFlow - path from Routers to Collector

On 2 Sep 2015, at 0:55, Avi Freedman wrote: Looking at probably 100 networks' flow paths over the last year, I'd say 1 or 2 have OOB for flow. Far fewer have it than should, agreed. A reasonable compromise is VLANs, VRFs, and so on to at least keep it out of the data-plane of the

Re: NetFlow - path from Routers to Collector

On 01/09/2015 21:13, valdis.kletni...@vt.edu wrote: > On Tue, 01 Sep 2015 22:47:09 +0300, Tarko Tikan said: >> Bad advice. No amount of money will fix major platforms that are not >> happy to export flow telemetry via router management ports. > > And that box ended up in your rack, why exactly? >

Re: NetFlow - path from Routers to Collector

On 2 Sep 2015, at 3:24, Niels Bakker wrote: Because variety of flow telemetry delivery options isn't the #1 ranked purchasing decider. Actually, it is more often than you think. No use routing packets if you can't see what they do. Otherwise no Cisco would ever have been sold. Which is

Re: NetFlow - path from Routers to Collector

On 2 Sep 2015, at 2:38, George, Wes wrote: Often there is a separate management network that can deal with ethernet speeds, but it's separate for security reasons and not always as rigidly independent from the in band network for connectivity, i.e. It might be a VPN riding over the regular

Re: NetFlow - path from Routers to Collector

On 2 Sep 2015, at 2:47, Tarko Tikan wrote: In-band netflow works on all platforms without such issues. There's no law that says that you must only plug designated management ports into OOB/DCN management networks. --- Roland Dobbins

Re: NetFlow - path from Routers to Collector

On Tue, 01 Sep 2015 22:47:09 +0300, Tarko Tikan said: > hey, > > > It should've already been spent for an OOB/DCN network, which should've > > been provisioned with flow telemetry in mind. > > Bad advice. No amount of money will fix major platforms that are not > happy to export flow telemetry via

Re: NetFlow - path from Routers to Collector

On Tue, 01 Sep 2015 22:47:09 +0300, Tarko Tikan said: Bad advice. No amount of money will fix major platforms that are not happy to export flow telemetry via router management ports. Correct. And for a proper network you may not wish to have those connections from in-band ports to your

Re: NetFlow - path from Routers to Collector

> On Sep 1, 2015, at 6:37 PM, Roland Dobbins wrote: > > On 2 Sep 2015, at 3:34, Nick Hilliard wrote: > >> If you want to handle netflow data export for large amounts of traffic, it >> would be pretty dumb to push it through the management plane of the router. > > Concur

Re: NetFlow - path from Routers to Collector

> On Sep 1, 2015, at 6:39 PM, Roland Dobbins wrote: > > On 2 Sep 2015, at 3:45, Chuck Church wrote: > >> Most OOB is lacking redundancy too, so a single failure can really take the >> shine off an OOB deployment. > > Even if you're using old, grandfathered equipment for

Re: NetFlow - path from Routers to Collector

Agreed, we are as well :) VLAN, VRF, whatever. + optimal tweaks include local flow proxy that can also rate limit / re-sample, and send topk talkers over 'true' OOB. Avi Freedman CEO, Kentik avi at kentik dot com > On 2 Sep 2015, at 7:27, Avi Freedman wrote: > > > We see well under 20%

Re: NetFlow - path from Routers to Collector

On 1/Sep/15 17:33, Serge Vautour wrote: > Hello, > > For those than run Internet connected routers, how do you get your NetFlow > data from the routers to your collectors? Do you let the flow export traffic > use the same links as your customer traffic to route back to central > collectors?

Re: NetFlow - path from Routers to Collector

On 1 Sep 2015, at 22:39, Mark Tinka wrote: > Have been doing so for years, no major drama. Until there is. ;> --- Roland Dobbins

Re: NetFlow - path from Routers to Collector

ork > 36-30-859-5144 > rod.b...@hibernianetworks.com > > > From: NANOG <nanog-boun...@nanog.org> on behalf of Roland Dobbins < > rdobb...@arbor.net> > Sent: Tuesday, September 1, 2015 5:43 PM > To: nanog@nanog.org > Subject: Re: NetFlow

Re: NetFlow - path from Routers to Collector

On 1 Sep 2015, at 23:10, Job Snijders wrote: > To answer your first question: i see no issue in transporting flow > export traffic over the same backbone used to serve customer traffic. This is not good advice, for the reasons I stated previously in this thread.

NetFlow - path from Routers to Collector

Hello, For those than run Internet connected routers, how do you get your NetFlow data from the routers to your collectors? Do you let the flow export traffic use the same links as your customer traffic to route back to central collectors? Or do you send this traffic over private network

Re: NetFlow - path from Routers to Collector

2015 5:43 PM To: nanog@nanog.org Subject: Re: NetFlow - path from Routers to Collector On 1 Sep 2015, at 22:39, Mark Tinka wrote: > Have been doing so for years, no major drama. Until there is. ;> --- Roland Dobbins <rdobb...@arbor.net> This e-mail and

Re: NetFlow - path from Routers to Collector

On Tue, Sep 01, 2015 at 08:33:42AM -0700, Serge Vautour wrote: > For those than run Internet connected routers, how do you get your > NetFlow data from the routers to your collectors? Do you let the flow > export traffic use the same links as your customer traffic to route > back to central

Re: NetFlow - path from Routers to Collector

On 1 Sep 2015, at 22:33, Serge Vautour wrote: Or do you send this traffic over private network management type path? This is how to do it. So in case of a network partition event, you don't end up losing visibility into your network traffic when you need it the most. You should already