Re: Firewalls - Ease of Use and Maintenance?

[Jonathan Lassoff]

It really depends on what constraints you have. Do you care about:
cost? performance? support?

Personally, for cost-constrained applications of 1 Gbit/s or less
(assuming modestly-sized packets, not all-DNS for example), I like
OpenBSD/pf or Linux/netfilter and generic x86 64-bit servers.
It's cheap, deeply customizable and since everything touches a CPU, it
allows for deep traffic inspection.

The tradeoff is that there's no support from major vendors, but there
are many smaller but very experienced consulting shops that can
integrate any patches and fix and issues that may arise.


What kinds of things are you looking for?

Cheers,
jof

On Tue, Nov 8, 2011 at 3:06 PM, Jones, Barry
 wrote:
> Hello all.
> I am potentially looking at firewall products and wanted suggestions as to 
> the easiest firewalls to install, configure and maintain? I have a few small 
> networks ( 50 nodes at one site, 50 odd at another, and maybe 20 at another. 
> I have worked with Cisco Pix, ASA, Netscreen, and Checkpoint (Nokia), and 
> each have strong and not as strong features for ease of use. Like everyone, 
> I'm resource challenged and need an easy solution to stand up and operate.
>
> Feel free to ping me offline - and thank you for the assistance.
>
> 
> Barry Jones - CISSP GSNA
> Project Manager II
> Sempra Energy Utilities
> (760) 271-6822
>
> P please don't print this e-mail unless you really need to.
> 
>
>

Re: where was my white knight....

[Owen DeLong]

On Nov 8, 2011, at 7:28 PM, Randy Bush wrote:

> fwiw, we have not tested the scaling of rpki-rtr performance as much as
> we might have.  we synthesized an rpki cache with roas for all the
> prefixes in a current table, 370k of them or whatever, and let routers
> load that cache from zip to full.  for low-end routers and a mediocre
> cache server, either local or across noam, it took less than five
> seconds.  this was small enough that we moved on to other stuff.
> 
> randy

Did you do this on routers that already had fully converged tables, or,
did you bootstrap the table load into the routers at the same time
as would be the case in a power failure, post-crash reboot, software
upgrade, etc.?

If only the former, may I suggest that at least doing some level of the
latter might prove a useful exercise?

I apologize for this mildly operational question. Y'all can go back to
Randy's fud-laiden black helicopters now.

Owen

Re: where was my white knight....

[Randy Bush]

> Indeed, we can expect new and exciting ways to blow up networks with
> SIDR.

the black helicopters spraying fud are especially vicious

Re: where was my white knight....

[Randy Bush]

fwiw, we have not tested the scaling of rpki-rtr performance as much as
we might have.  we synthesized an rpki cache with roas for all the
prefixes in a current table, 370k of them or whatever, and let routers
load that cache from zip to full.  for low-end routers and a mediocre
cache server, either local or across noam, it took less than five
seconds.  this was small enough that we moved on to other stuff.

randy

Re: where was my white knight....

[Randy Bush]

> I understand what the manual says (actually, i read it).

cheating

> I'm just curious as to how this is going to work in real life.  Let's
> say you have a router cold boot with a bunch of ibgp peers, a transit
> or two and an rpki cache which is located on a non-connected network -
> e.g. small transit pop / AS boundary scenario.  The cache is not
> necessarily going to be reachable until it sees an update for its
> connected network.

once again, 
  o when you have no connection to a cache or no covering roa for a
a prefix, the result is specified as NotFound
  o we recommend you route on NotFound

so the result is the same as today.

> Until this happens, there will be no connectivity from the router to
> the cache

false

> Look, i understand that you're designing rpki <-> interactivity such that
> things will at least work in some fashion when your routers lose sight of
> their rpki caches.  The problem is that this approach weakens rpki's
> strengths - e.g. the ability to help stop youtube-like incidents from
> recurring by ignoring invalid prefix injection.

you can't have you cake and eat it to.  you can not detect invalid
originations until you have the data to do so.

randy

RE: Firewalls - Ease of Use and Maintenance?

[R. Benjamin Kessler]

We work with many vendor's firewalls and our current "favorites" are Palo Alto 
Networks - they're very full-featured and easy to manage.

www.paloaltonetworks.com

I don't want to get all "sales-weasel" on you but we can help if you want more 
info as we are one of their premier partners.

P.S. - we're also Juniper and Cisco partners too but we prefer the Palo Altos 
for firewalls these days.

Let me know how I can help

-Ben


R. Benjamin Kessler
CCIE #8762, CISSP, CCSE
President / Chief Network Geek
Zenetra Corporation
Email: ben.kess...@zenetra.com
http://www.zenetra.com
Office: 260-271-4330 << Note: New Number
Cell: 260-437-5774
Fax: 866-388-6652



-Original Message-
From: Jones, Barry [mailto:bejo...@semprautilities.com] 
Sent: Tuesday, November 08, 2011 6:07 PM
To: nanog@nanog.org
Subject: Firewalls - Ease of Use and Maintenance?

Hello all.
I am potentially looking at firewall products and wanted suggestions as to the 
easiest firewalls to install, configure and maintain? I have a few small 
networks ( 50 nodes at one site, 50 odd at another, and maybe 20 at another. I 
have worked with Cisco Pix, ASA, Netscreen, and Checkpoint (Nokia), and each 
have strong and not as strong features for ease of use. Like everyone, I'm 
resource challenged and need an easy solution to stand up and operate.

Feel free to ping me offline - and thank you for the assistance.


Barry Jones - CISSP GSNA
Project Manager II
Sempra Energy Utilities
(760) 271-6822

P please don't print this e-mail unless you really need to.

RE: Firewalls - Ease of Use and Maintenance?

[Blake T. Pfankuch]

As Hammer stated, you hit all the big ones.

ASA's are a classic fallback because of the stability implied by the cisco 
name.  Complaints about them tend to be cost on getting all the shiny bits 
attached to them (IDS, IPS, Content filtering).  This coming from a Cisco 
partner.  I am not a Netscreen fan myself due to past experiences and sour 
tastes.  Checkpoint's are OK, but I don't like the application need for 
configuring on SMB appliances.  

Add to the list Sonicwall.  We use them primarily for our customers at work and 
are partners with them as well.  They have appliances that go from 10 office 
size to Active/Active HA pairing that can do multi gbit of throughput.  They 
support all the standard features you look for IPSEC VPN, SSLVPN, L2TP, VLAN 
Interfaces, Dynamic routing support (OSPF and RIP in small models, BGP in the 
larger) LDAP auth for all of the above, content filtering, IPS, IDS, Anti 
Spyware stateful blah blah and centralized management.  Some of the newer 
things that are gaining popularity that you can license is the App 
Visualization (think netflow in a web UI with good filters), WAN Acceleration 
modules via a VMware Appliance, RBL Filtering (which can be applied to just 
about anything), DPI-SSL inspection for https traffic, Active/Active HA, 
Physical port redundancy per appliance, list goes on.  Configuration logic is 
similar to a ASA, however takes a little to get used to.  The nice thing is 
everything in the config is name based and searchable within the WebUI and you 
can talk non technical people through making changes in the config if you have 
to.  

The feature list is growing every day, and I almost prefer them anymore just 
because of the simplicity as well as the scalability.

Ping me if you have more questions or want a few example setups.

Blake

-Original Message-
From: Jones, Barry [mailto:bejo...@semprautilities.com] 
Sent: Tuesday, November 08, 2011 4:07 PM
To: nanog@nanog.org
Subject: Firewalls - Ease of Use and Maintenance?

Hello all.
I am potentially looking at firewall products and wanted suggestions as to the 
easiest firewalls to install, configure and maintain? I have a few small 
networks ( 50 nodes at one site, 50 odd at another, and maybe 20 at another. I 
have worked with Cisco Pix, ASA, Netscreen, and Checkpoint (Nokia), and each 
have strong and not as strong features for ease of use. Like everyone, I'm 
resource challenged and need an easy solution to stand up and operate.

Feel free to ping me offline - and thank you for the assistance.


Barry Jones - CISSP GSNA
Project Manager II
Sempra Energy Utilities
(760) 271-6822

P please don't print this e-mail unless you really need to.

comments due 2dec2011

[bmanning]

http://www.nist.gov/itl/cloud/index.cfm

/bill

Re: Firewalls - Ease of Use and Maintenance?

[-Hammer-]

You've worked with all the big dogs. What are you looking for? 
Alternative options?


-Hammer-

"I was a normal American nerd"
-Jack Herer



On 11/08/2011 05:06 PM, Jones, Barry wrote:

Hello all.
I am potentially looking at firewall products and wanted suggestions as to the 
easiest firewalls to install, configure and maintain? I have a few small 
networks ( 50 nodes at one site, 50 odd at another, and maybe 20 at another. I 
have worked with Cisco Pix, ASA, Netscreen, and Checkpoint (Nokia), and each 
have strong and not as strong features for ease of use. Like everyone, I'm 
resource challenged and need an easy solution to stand up and operate.

Feel free to ping me offline - and thank you for the assistance.


Barry Jones - CISSP GSNA
Project Manager II
Sempra Energy Utilities
(760) 271-6822

P please don't print this e-mail unless you really need to.


   

Firewalls - Ease of Use and Maintenance?

[Jones, Barry]

Hello all.
I am potentially looking at firewall products and wanted suggestions as to the 
easiest firewalls to install, configure and maintain? I have a few small 
networks ( 50 nodes at one site, 50 odd at another, and maybe 20 at another. I 
have worked with Cisco Pix, ASA, Netscreen, and Checkpoint (Nokia), and each 
have strong and not as strong features for ease of use. Like everyone, I'm 
resource challenged and need an easy solution to stand up and operate.

Feel free to ping me offline - and thank you for the assistance.


Barry Jones - CISSP GSNA
Project Manager II
Sempra Energy Utilities
(760) 271-6822

P please don't print this e-mail unless you really need to.

Re: where was my white knight....

[Christopher Morrow]

On Tue, Nov 8, 2011 at 5:26 PM, Dobbins, Roland  wrote:
>
> On Nov 9, 2011, at 4:22 AM, Christopher Morrow wrote:
>
>>  the routers have (in some form of the plan) a cache
>
> A cache that's persistent across reboots?
>

not across reboots, but in this case routers didn't necessarily reboot
(parts of them did though).
in the case of a reboot, sure, pull from your local cache, no 'walk up
the chian' is required here.

Re: where was my white knight....

[Matthias Waehlisch]

On Tue, 8 Nov 2011, bmann...@vacation.karoshi.com wrote:

> On Tue, Nov 08, 2011 at 08:16:10PM +0100, Randy Bush wrote:
> > > the answer seems to be NO, it would not have helped and would have
> > > actually contributed to network instability with large numbers of
> > > validation requests sent to the sidr/ca nodes...
> > 
> > utter bullshit.  maybe you would benefit by actually reading the doccos
> > and understanding the protocols.
> 
>   are they actually coherent enough to be read & understood?
>   
  I think so: at least a Bachelor student of my got along with them for 
his thesis.

  Btw: There is also a very nice overview by Geoff published in Cisco 
IPJ:

  * 
http://www.cisco.com/web/about/ac123/ac147/archived_issues/ipj_14-2/142_bgp.html



Cheers
  matthias

-- 
Matthias Waehlisch
.  Freie Universitaet Berlin, Inst. fuer Informatik, AG CST
.  Takustr. 9, D-14195 Berlin, Germany
.. mailto:waehli...@ieee.org .. http://www.inf.fu-berlin.de/~waehl
:. Also: http://inet.cpt.haw-hamburg.de .. http://www.link-lab.net

Re: where was my white knight....

[bmanning]

On Tue, Nov 08, 2011 at 08:16:10PM +0100, Randy Bush wrote:
> > the answer seems to be NO, it would not have helped and would have
> > actually contributed to network instability with large numbers of
> > validation requests sent to the sidr/ca nodes...
> 
> utter bullshit.  maybe you would benefit by actually reading the doccos
> and understanding the protocols.

are they actually coherent enough to be read & understood?

/bill

Re: where was my white knight....

[Leo Bicknell]

In a message written on Tue, Nov 08, 2011 at 10:19:24PM +, Nick Hilliard 
wrote:
> One solution is to have directly-connected rpki caches available to all 
> your bgp edge routers throughout your entire network.  This may turn out to 
> be expensive capex-wise, and will turn out to be yet another critical 
> infrastructure item to maintain, increasing opex.

Couldn't you just have a couple of these boxes on your network and
route them in your IGP, removing any BGP dependancy?  KISS.

-- 
   Leo Bicknell - bickn...@ufp.org - CCIE 3440
PGP keys at http://www.ufp.org/~bicknell/


pgpobc43FvOSA.pgp
Description: PGP signature

Re: where was my white knight....

[Dobbins, Roland]

On Nov 9, 2011, at 5:19 AM, Nick Hilliard wrote:

> One solution is to have directly-connected rpki caches available to all your 
> bgp edge routers throughout your entire network. 

They don't have to be directly-connected - they could be on the DCN, which 
ought to have at least some static 'hints' to critical resources.

---
Roland Dobbins  // 

The basis of optimism is sheer terror.

  -- Oscar Wilde

Re: where was my white knight....

[Dobbins, Roland]

On Nov 9, 2011, at 4:22 AM, Christopher Morrow wrote:

>  the routers have (in some form of the plan) a cache

A cache that's persistent across reboots?

---
Roland Dobbins  // 

The basis of optimism is sheer terror.

  -- Oscar Wilde

Re: where was my white knight....

[Nick Hilliard]

On 08/11/2011 21:32, valdis.kletni...@vt.edu wrote:

Anybody who puts their rpki cache someplace that isn't accessible until they
get the rpki initialized gets what they deserve.


One solution is to have directly-connected rpki caches available to all 
your bgp edge routers throughout your entire network.  This may turn out to 
be expensive capex-wise, and will turn out to be yet another critical 
infrastructure item to maintain, increasing opex.


Alternatively, you host rpki caches on all your AS-edge routers => upgrades 
- and lots of currently-sold kit will simply not handle this sort of thing 
properly.



Once you realize this, the rest of the "what do we do for routing until
it comes up" concern trolling in the rest of that paragraph becomes
pretty easy to sort out...


I humbly apologise for expressing concern about the wisdom of imposing a 
hierarchical, higher-layer validation structure for forwarding-info 
management on a pre-existing lower layer fully distributed system which is 
already pretty damned complex...


What's that principle called again?  Was it "Keep It Complex, Stupid"?  I 
can't seem to remember :-)



Caching just enough to validate the routes you need to get to a more capable
rpki server shouldn't have a high write life-cycle.


Lots of older flash isn't going to like this => higher implementation cost 
due to upgrades.



Heck, you could just manually
configure a host route pointing to the rpki server...


Yep, hard coding things - good idea, that.


And it would hardly be the first time that people have been unable to deploy
feature XYZ because it wouldn't fit in the flash on older boxes still in
production.


This is one of several points I'm making: there is a cost factor here, and 
it's not clear how large it is.


Nick

Re: where was my white knight....

[Leigh Porter]

On 8 Nov 2011, at 21:37, "Leo Bicknell"  wrote:

> In a message written on Tue, Nov 08, 2011 at 04:22:48PM -0500, Christopher 
> Morrow wrote:
>> I think actually it wouldn't have caused more validation requests, the
>> routers have (in some form of the plan) a cache from their local
>> cache, they use this for origin validation... there's not a
>> requirement to refresh up the entire chain. (I think).
> 
> I kinda think everyone is wrong here, but Chris is closer to accurate.
> :P
> 
> When a router goes boom, the rest of the routers recalculate around
> it.  Generally speaking all of the routers will have already had a
> route with the same origin, and thus have hopefully cached a lookup
> of the origin.  However, that lookup might have been done
> days/weeks/months ago, in a stable network.
> 
> While I'm not familar with the nitty gritty details here, caches
> expire for various reasons.  The mere act of the route changing
> paths, if it moved to a device with a stale cache, would trigger a
> new lookup, right?
> 
> Basically I would expect any routing change to generate a set of
> new lookups proportial to the cache expiration rules.

Which may very well fail because all the routing is hosed. I'm not all that 
familiar with the potential implementation issues, but I would think that 
network-local caches would be in order. 

Even with local caches, I would expect a high incidence of change to trigger 
something sensible to mitigate this kind of craziness from happening. I am sure 
enough people have had incorrectly scaled RADIUS farms blow up when a load of 
DSLAMS vanish and come back again not to repeat such storms.


--
Leigh Porter


__
This email has been scanned by the MessageLabs Email Security System.
For more information please visit http://www.messagelabs.com/email 
__

OT: Ars: Cicso v Adekeye

[Jay Ashworth]

http://arstechnica.com/tech-policy/news/2011/07/a-pound-of-flesh-how-ciscos-unmitigated-gall-derailed-one-mans-life.ars/1

-- 
Jay R. Ashworth  Baylink   j...@baylink.com
Designer The Things I Think   RFC 2100
Ashworth & Associates http://baylink.pitas.com 2000 Land Rover DII
St Petersburg FL USA  http://photo.imageinc.us +1 727 647 1274

Re: where was my white knight....

[Leo Bicknell]

In a message written on Tue, Nov 08, 2011 at 04:22:48PM -0500, Christopher 
Morrow wrote:
> I think actually it wouldn't have caused more validation requests, the
> routers have (in some form of the plan) a cache from their local
> cache, they use this for origin validation... there's not a
> requirement to refresh up the entire chain. (I think).

I kinda think everyone is wrong here, but Chris is closer to accurate.
:P

When a router goes boom, the rest of the routers recalculate around
it.  Generally speaking all of the routers will have already had a
route with the same origin, and thus have hopefully cached a lookup
of the origin.  However, that lookup might have been done
days/weeks/months ago, in a stable network.

While I'm not familar with the nitty gritty details here, caches
expire for various reasons.  The mere act of the route changing
paths, if it moved to a device with a stale cache, would trigger a
new lookup, right?

Basically I would expect any routing change to generate a set of
new lookups proportial to the cache expiration rules.

What am I missing?

-- 
   Leo Bicknell - bickn...@ufp.org - CCIE 3440
PGP keys at http://www.ufp.org/~bicknell/


pgpxc1AApJgzd.pgp
Description: PGP signature

Re: where was my white knight....

[Valdis . Kletnieks]

On Tue, 08 Nov 2011 20:51:00 GMT, Nick Hilliard said:

> I understand what the manual says (actually, i read it).  I'm just curious
> as to how this is going to work in real life.  Let's say you have a router
> cold boot with a bunch of ibgp peers, a transit or two and an rpki cache
> which is located on a non-connected network 

Anybody who puts their rpki cache someplace that isn't accessible until they
get the rpki initialized gets what they deserve. Once you realize this, the
rest of the "what do we do for routing until it comes up" concern trolling in
the rest of that paragraph becomes pretty easy to sort out...

> You could argue to have a local cache in every pop but may not be feasible
> either - a cache will require storage with a high write life-cycle (i.e.
> forget about using lots of types of flash), and you cannot be guaranteed
> that this is going to be available on a router.

Caching just enough to validate the routes you need to get to a more capable
rpki server shouldn't have a high write life-cycle.  Heck, you could just 
manually
configure a host route pointing to the rpki server...

And it would hardly be the first time that people have been unable to deploy
feature XYZ because it wouldn't fit in the flash on older boxes still in
production.



pgpybPmy4dIHo.pgp
Description: PGP signature

Re: where was my white knight....

[Christopher Morrow]

On Tue, Nov 8, 2011 at 4:08 PM, Leigh Porter
 wrote:
>
> On 8 Nov 2011, at 18:24, "Dobbins, Roland"  wrote:
>
>> Validation storm-control is something which must be accounted for in 
>> SIDR/DANE architecture, implementation, and deployment.  But at the end of 
>> the day, vendors are still responsible for their own code.
>>
>>
>
> Indeed, we can expect new and exciting ways to blow up networks with SIDR.

or, the same old ways... only with crypto!
really, there was some care taken in the process to create this and
NOT stomp all over how networks currently work.

comments welcome though.

Re: where was my white knight....

[Christopher Morrow]

On Tue, Nov 8, 2011 at 1:48 PM, Nick Hilliard  wrote:
> On 08/11/2011 18:14, bmann...@vacation.karoshi.com wrote:
>>  the answer seems to be NO, it would not have helped and would have actually
>> contributed to network instability with large numbers of validation requests
>> sent to the sidr/ca nodes...
>
> i'm curious about sidr cold bootup, specifically when you are attempting to
> validate prefixes from an rpki CA or cache to which you do not necessarily
> have network connectivity because your igp is not yet fully up.  The
> phrases "layering violation" and "chicken and egg" come to mind.

'lazy validation' - prefer to get at least somewhat converged, then validate.

>

Re: where was my white knight....

[Christopher Morrow]

On Tue, Nov 8, 2011 at 1:14 PM,   wrote:
>
>  that was/is kindof orthoginal to the question... would the sidr plan
> for routing security have been a help in this event?  nice to know
> unsecured IPv6 took some of the load when the unsecured IPv4 path
> failed.
>

if all routing goes boom, would secure routing have saved you?
no... all routing went boom.

>  the answer seems to be NO, it would not have helped and would have actually
> contributed to network instability with large numbers of validation requests
> sent to the sidr/ca nodes.

I think actually it wouldn't have caused more validation requests, the
routers have (in some form of the plan) a cache from their local
cache, they use this for origin validation... there's not a
requirement to refresh up the entire chain. (I think).

-chris

>
> /bill
>
> On Tue, Nov 08, 2011 at 10:01:04AM -0800, Mike Leber wrote:
>>
>> We saw an increase in IPv6 traffic which correlated time wise with the
>> onset of this IPv4 incident.
>>
>> Happy eyeballs in action, automatically shifting what it could.
>>
>> Mike.
>>
>> On 11/8/11 2:56 AM, bmann...@vacation.karoshi.com wrote:
>> >how would a sidr-enabled routing infrastructure have fared in yesterdays
>> >routing circus?
>> >
>> >/bill
>> >
>
>

Re: where was my white knight....

[Leigh Porter]

On 8 Nov 2011, at 18:24, "Dobbins, Roland"  wrote:

> 
> On Nov 9, 2011, at 1:14 AM,  wrote:
> 
>> that was/is kindof orthoginal to the question... would the sidr plan for 
>> routing security have been a help in this event? 
> 
> SIDR is intended to provide route-origination validation - it isn't intended 
> to be nor can it possibly be a remedy for vendor-specific implementation 
> problems.
> 
> Validation storm-control is something which must be accounted for in 
> SIDR/DANE architecture, implementation, and deployment.  But at the end of 
> the day, vendors are still responsible for their own code.
> 
> 

Indeed, we can expect new and exciting ways to blow up networks with SIDR. 


--
Leigh


__
This email has been scanned by the MessageLabs Email Security System.
For more information please visit http://www.messagelabs.com/email 
__

Re: where was my white knight....

[Nick Hilliard]

On 08/11/2011 19:19, Randy Bush wrote:
> what comes to my mind is that NotFound is the default and it is
> recommended to route on it.

I understand what the manual says (actually, i read it).  I'm just curious
as to how this is going to work in real life.  Let's say you have a router
cold boot with a bunch of ibgp peers, a transit or two and an rpki cache
which is located on a non-connected network - e.g. small transit pop / AS
boundary scenario.  The cache is not necessarily going to be reachable
until it sees an update for its connected network.  Until this happens,
there will be no connectivity from the router to the cache, and
consequently prefixes received in from the transit may be subject to an
incorrect and potentially inconsistent routing policy with respect to the
rest of the network.  Ok, they'll be revalidated once the cache comes on
line, but what do you do with them in the interim?  Route traffic to them,
knowing that they might or might not be correct?  Drop until the cache
comes online from the point of view of the router?  Forward potentially
incorrect UPDATEs to your other ibgp peers, and forward validated updates
when the cache comes on-line again?  If so, then what if your incorrect new
policy takes precedence over an existing path in your ibgp mesh?  And what
if your RP is low on memory from storing an unvalidated adj-rib-in?

You could argue to have a local cache in every pop but may not be feasible
either - a cache will require storage with a high write life-cycle (i.e.
forget about using lots of types of flash), and you cannot be guaranteed
that this is going to be available on a router.

Look, i understand that you're designing rpki <-> interactivity such that
things will at least work in some fashion when your routers lose sight of
their rpki caches.  The problem is that this approach weakens rpki's
strengths - e.g. the ability to help stop youtube-like incidents from
recurring by ignoring invalid prefix injection.

Nick

Re: Logs Bank

[Andrew Mulholland]

To answer your question.

"yes"

However, with almost everything I can think of, there will be an element of
development required in order to achieve the results you're after. - at a
previous work place a few years ago we fed all event logs into hadoop, from
where we produced reports, initially just into excel files,  and then later
created a webapp which produced near realtime stats/reports/graphs.

I've not looked recently at LogStash, or 8pussy, but primary concern would
be how well they deal with huge log volumes, how they scale when one server
is not big enough to hold all the logs any more, how they deal with many
users searching at the same time etc.

If you want to actually just get on with crunching logs, and drawing graphs
in a timely fashion, Splunk is proven, and works well up to big scale (we
were feeding almost 1TB/day of logs into it at my last company)...


Splunk is not cheap, but when considering the cost of development +
suppport if you went down the route of task of rolling something equivalent
in capabilities, its not bad value.

thanks

Andrew


On Tue, Nov 8, 2011 at 7:59 PM,  wrote:

> Hi,
>
> If I may ask, is there any OSS that can serve as a log bank or log server,
> where it aggregate logs from  different sources , and the logs can be
> accessed using the web from any location on the network and can do
> graphical presentations based on.the frequency or content os the logs.
>
> Thank you
>
> Joshua
>
> --
> Sent from my Nokia N9
>

[NANOG-announce] Call for Communications Committee (CC) Volunteers

[Sylvie LaPerriere]

NANOGers:



The NANOG Communications
Committee,
as defined by the NANOG Bylaws, is a committee that is responsible for the
NANOG mailing list, and other forms of electronic communication among the
NANOG community as agreed with the Board of Directors.



The Communications Committee is responsible for the administration and
minimal moderation of the NANOG list, nanog@nanog.org,  Also, the committee
is responsible for the administration of other mailing lists and other
forms of electronic communications among the NANOG community.



The NANOG Board is seeking volunteers who are, or willing to become,
members and are willing to serve on the Communications Committee.



Please submit your name and interest statement to bo...@nanog.org


Thanks!


Sylvie LaPerriere

NANOG Chair   -  www.nanog.org
___
NANOG-announce mailing list
nanog-annou...@nanog.org
https://mailman.nanog.org/mailman/listinfo/nanog-announce

Re: Logs Bank

[David]

Oh! And http://graylog2.org/ -- free, open source.
That's the last of the ones I can muster up.

On Tue, 2011-11-08 at 14:06 -0600, David wrote:
> http://www.8pussy.org/dokuwiki/doku.php -- free. open source.
> http://logstash.net/ -- free. open source.
> http://splunk.com (already mentioned, of course) -- pay to play. And
> expensive, too. 
> 
> There are far more out there.
> 
> 

Re: Logs Bank

[David]

http://www.8pussy.org/dokuwiki/doku.php -- free. open source.
http://logstash.net/ -- free. open source.
http://splunk.com (already mentioned, of course) -- pay to play. And
expensive, too. 

There are far more out there.

Re: Logs Bank

[Peter Kristolaitis]

Octopussy (8pussy.org) is another option as well.   Natively ties into 
various network monitoring packages (Nagios, Zabbix) for alerting 
capabilities.


- Peter


On 11/8/2011 3:00 PM, Charles N Wyble wrote:

Yes. Check out rsyslog and logstash.

joshua.kl...@gmail.com wrote:


Hi,

If I may ask, is there any OSS that can serve as a log bank or log
server, where it aggregate logs from  different sources , and the logs
can be accessed using the web from any location on the network and can
do graphical presentations based on.the frequency or content os the
logs.

Thank you

Joshua

--
Sent from my Nokia N9

Re: Logs Bank

[Derek Bodner]

On 11/08/2011 03:00 PM, John Adams wrote:

You probably want spunk, but if you want to do aggregation in an OSS fashion, 
scribe or flume is the way to go.




Agree with Splunk, while not open source, is the most functional of 
these products.  Be warned, while they offer a free license, once you 
start using it you'll be hooked, and their pricing beyond the free 
license is borderline extortionist.

Re: Juniper MX960 Mqchip packet lenght error

[Daniel Roesen]

On Tue, Nov 08, 2011 at 03:34:36PM +0100, Ferretti Antonio wrote:
> After upgrading Juniper MX960 from 4x10Gbe DPCE to 16x10Gbe MPC, I see 
> strange log messages like this:
> 
> Fpc10 MQCHIP(0) LI Packet length error, pt entry 22
> 
> Seems to be only a "cosmetic" message,

PR/593386 "and others". Wenth thru that half a year ago. :)

> but today my team found some traffic dropped on interface on this card
> (before we only request check about log message).
> Has anyone the same problem?

Yes. We were told: cosmetic only, and we saw no impact.

Trigger was a "then log" in the lo0.0 firewall filter.

> My software release is Junos 10.3R3.7.

Would match. Above PR is fixed in 10.3R4, 10.4R4, 11.1R2 and newer

Best regards,
Daniel

-- 
CLUE-RIPE -- Jabber: d...@cluenet.de -- dr@IRCnet -- PGP: 0xA85C8AA0

Re: Logs Bank

[Charles N Wyble]

Yes. Check out rsyslog and logstash.

joshua.kl...@gmail.com wrote:

>Hi,
>
>If I may ask, is there any OSS that can serve as a log bank or log
>server, where it aggregate logs from  different sources , and the logs
>can be accessed using the web from any location on the network and can
>do graphical presentations based on.the frequency or content os the
>logs.
>
>Thank you
>
>Joshua
>
>--
>Sent from my Nokia N9

--
Charles N Wyble @charlesnw char...@knownelement.com

Building a cost effective, open, secure bit moving platform for tomorrows 
default free zone.

Re: Logs Bank

[Landon Stewart]

On 8 November 2011 11:59,  wrote:

> Hi,
>
> If I may ask, is there any OSS that can serve as a log bank or log server,
> where it aggregate logs from  different sources , and the logs can be
> accessed using the web from any location on the network and can do
> graphical presentations based on.the frequency or content os the logs.
>

Do you mean like Splunk?  http://www.splunk.com

-- 
Landon Stewart 
SuperbHosting.Net by Superb Internet Corp.
Toll Free (US/Canada): 888-354-6128 x 4199
Direct: 206-438-5879
Web hosting and more "Ahead of the Rest": http://www.superbhosting.net

Re: Logs Bank

[John Adams]

You probably want spunk, but if you want to do aggregation in an OSS fashion, 
scribe or flume is the way to go.

-John

Sent from my iPhone

On Nov 8, 2011, at 11:59, joshua.kl...@gmail.com wrote:

> Hi, 
> 
> If I may ask, is there any OSS that can serve as a log bank or log server, 
> where it aggregate logs from  different sources , and the logs can be 
> accessed using the web from any location on the network and can do graphical 
> presentations based on.the frequency or content os the logs.
> 
> Thank you
> 
> Joshua
> 
> -- 
> Sent from my Nokia N9

Logs Bank

[joshua . klubi]

Hi,

If I may ask, is there any OSS that can serve as a log bank or log server, 
where it aggregate logs from  different sources , and the logs can be accessed 
using the web from any location on the network and can do graphical 
presentations based on.the frequency or content os the logs.

Thank you

Joshua

--
Sent from my Nokia N9

Re: where was my white knight....

[Randy Bush]

> i'm curious about sidr cold bootup, specifically when you are
> attempting to validate prefixes from an rpki CA or cache to which you
> do not necessarily have network connectivity because your igp is not
> yet fully up.  The phrases "layering violation" and "chicken and egg"
> come to mind.

what comes to my mind is that NotFound is the default and it is
recommended to route on it.

i know boys are not allowed to read the manual, but this is starting
to get boring.

randy

Re: where was my white knight....

[Randy Bush]

> the answer seems to be NO, it would not have helped and would have
> actually contributed to network instability with large numbers of
> validation requests sent to the sidr/ca nodes...

utter bullshit.  maybe you would benefit by actually reading the doccos
and understanding the protocols.

Re: [outages] More notes

[Jack Bates]

On 11/8/2011 12:05 PM, valdis.kletni...@vt.edu wrote:


And if JunOS is anything like CIsco IOS, a lot of shops didn't upgrade because
the newer release has *other* issues in their environments.  Nobody wants to
upgrade to fix a once-ever-few-months bug if it also buys them a daily crash in
something else.




Juniper runs a quarterly (roughly major) 10.1, 10.2, 10.3, 10.4, 

R is patch revisions for the major release. They are usually good at 
fixing and not breaking things on the R release. My last upgrade a bit 
ago was R7.5 of 10.4 (which has more revisions than older 10 releases, 
probably due to the fact that it will be the long term support release 
and gets non-critical patches as well).



Jack

Re: where was my white knight....

[bmanning]

On Tue, Nov 08, 2011 at 06:48:12PM +, Nick Hilliard wrote:
> On 08/11/2011 18:14, bmann...@vacation.karoshi.com wrote:
> >  the answer seems to be NO, it would not have helped and would have actually
> > contributed to network instability with large numbers of validation requests
> > sent to the sidr/ca nodes...
> 
> i'm curious about sidr cold bootup, specifically when you are attempting to
> validate prefixes from an rpki CA or cache to which you do not necessarily
> have network connectivity because your igp is not yet fully up.  The
> phrases "layering violation" and "chicken and egg" come to mind.
> 
> Nick

yeah...there is that.

/bill

Re: where was my white knight....

[bmanning]

On Tue, Nov 08, 2011 at 06:25:36PM +, Dobbins, Roland wrote:
> On Nov 9, 2011, at 1:22 AM, Dobbins, Roland wrote:
> 
> > Validation storm-control is something which must be accounted for in 
> > SIDR/DANE architecture, implementation, and deployment.  But at the end of 
> > the day, vendors are still responsible for their own code.
> 
> To be clear, I was alluding to some discussion centering around DANE or a 
> DANE-like mechanism to handle SIDR-type route validation.  Recursive 
> dependencies make this a non-starter, IMHO.
> 

well... your still stuck w/ knowing where your CA is...

/bill

Re: where was my white knight....

[Nick Hilliard]

On 08/11/2011 18:14, bmann...@vacation.karoshi.com wrote:
>  the answer seems to be NO, it would not have helped and would have actually
> contributed to network instability with large numbers of validation requests
> sent to the sidr/ca nodes...

i'm curious about sidr cold bootup, specifically when you are attempting to
validate prefixes from an rpki CA or cache to which you do not necessarily
have network connectivity because your igp is not yet fully up.  The
phrases "layering violation" and "chicken and egg" come to mind.

Nick

Re: where was my white knight....

[Dobbins, Roland]

On Nov 9, 2011, at 1:22 AM, Dobbins, Roland wrote:

> Validation storm-control is something which must be accounted for in 
> SIDR/DANE architecture, implementation, and deployment.  But at the end of 
> the day, vendors are still responsible for their own code.

To be clear, I was alluding to some discussion centering around DANE or a 
DANE-like mechanism to handle SIDR-type route validation.  Recursive 
dependencies make this a non-starter, IMHO.

---
Roland Dobbins  // 

The basis of optimism is sheer terror.

  -- Oscar Wilde

Re: where was my white knight....

[Dobbins, Roland]

On Nov 9, 2011, at 1:14 AM,  wrote:

> that was/is kindof orthoginal to the question... would the sidr plan for 
> routing security have been a help in this event? 

SIDR is intended to provide route-origination validation - it isn't intended to 
be nor can it possibly be a remedy for vendor-specific implementation problems.

Validation storm-control is something which must be accounted for in SIDR/DANE 
architecture, implementation, and deployment.  But at the end of the day, 
vendors are still responsible for their own code.

---
Roland Dobbins  // 

The basis of optimism is sheer terror.

  -- Oscar Wilde

Re: where was my white knight....

[bmanning]

 that was/is kindof orthoginal to the question... would the sidr plan
for routing security have been a help in this event?  nice to know 
unsecured IPv6 took some of the load when the unsecured IPv4 path
failed. 

 the answer seems to be NO, it would not have helped and would have actually
contributed to network instability with large numbers of validation requests
sent to the sidr/ca nodes...

/bill

On Tue, Nov 08, 2011 at 10:01:04AM -0800, Mike Leber wrote:
> 
> We saw an increase in IPv6 traffic which correlated time wise with the 
> onset of this IPv4 incident.
> 
> Happy eyeballs in action, automatically shifting what it could.
> 
> Mike.
> 
> On 11/8/11 2:56 AM, bmann...@vacation.karoshi.com wrote:
> >how would a sidr-enabled routing infrastructure have fared in yesterdays 
> >routing circus?
> >
> >/bill
> >

Re: [outages] More notes

[Valdis . Kletnieks]

On Tue, 08 Nov 2011 09:21:37 +0100, Stephane Bortzmeyer said:
> I disagree. The official bug statement from Juniper in August was
> trying very hard to downplay the importance of the bug ("Given the
> complexity of conditions required to trigger this issue, the
> probability of exploiting this defect is extremely low"). No wonder so
> few people (and not only at Level-3) did not upgrade.

August (and if that's when the *fix* came out, the bug is even older).

September.

October.

November.  So maybe the probability *is* low.

And if JunOS is anything like CIsco IOS, a lot of shops didn't upgrade because
the newer release has *other* issues in their environments.  Nobody wants to
upgrade to fix a once-ever-few-months bug if it also buys them a daily crash in
something else.




pgpLqqJ7QsVdF.pgp
Description: PGP signature

Re: where was my white knight....

[Mike Leber]

We saw an increase in IPv6 traffic which correlated time wise with the 
onset of this IPv4 incident.


Happy eyeballs in action, automatically shifting what it could.

Mike.

On 11/8/11 2:56 AM, bmann...@vacation.karoshi.com wrote:

how would a sidr-enabled routing infrastructure have fared in yesterdays 
routing circus?

/bill

Re: NANOG Digest, Vol 46, Issue 15

[Suess13]

Juniper core dump issue, patch is on the way.



On Nov 7, 2011, at 11:41 AM, "Steve Dispensa"  wrote:

> Level 3 was down in KC, Chi, and San Jose (at least) for us between
> about 8:10 and 8:40, plus or minus. Brought down SureWest in KC too.
> 
> -Steve
> 
>> -Original Message-
>> From: nanog-requ...@nanog.org [mailto:nanog-requ...@nanog.org]
>> Sent: Monday, November 07, 2011 10:05 AM
>> To: nanog@nanog.org
>> Subject: NANOG Digest, Vol 46, Issue 15
>> 
>> Send NANOG mailing list submissions to
>>nanog@nanog.org
>> 
>> To subscribe or unsubscribe via the World Wide Web, visit
>>https://mailman.nanog.org/mailman/listinfo/nanog
>> or, via email, send a message with subject or body 'help' to
>>nanog-requ...@nanog.org
>> 
>> You can reach the person managing the list at
>>nanog-ow...@nanog.org
>> 
>> When replying, please edit your Subject line so it is more specific
>> than "Re: Contents of NANOG digest..."
>> 
>> 
>> Today's Topics:
>> 
>>   1. Re: Time Warner Telecom problems (Jon Lewis)
>>   2. Re: Performance Issues - PTR Records (Leigh Porter)
>>   3. Re: Time Warner Telecom problems (Ray Van Dolson)
>>   4. General Internet Instability (Jared Mauch)
>>   5. Re: TATA problems? (Pierre-Yves Maunier)
>>   6. Re: TATA problems? (Leigh Porter)
>>   7. Re: Time Warner Telecom problems (Joe Greco)
>>   8. Re: TATA problems? (Kelly Kane)
>>   9. Re: Time Warner Telecom problems (Blake Hudson)
>>  10. RE: Time Warner Telecom problems (Thomas York)
>> 
>> 
>> --
>> 
>> Message: 1
>> Date: Mon, 7 Nov 2011 10:12:30 -0500 (EST)
>> From: Jon Lewis 
>> To: Peter Pauly 
>> Cc: nanog@nanog.org
>> Subject: Re: Time Warner Telecom problems
>> Message-ID: 
>> Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed
>> 
>> On Mon, 7 Nov 2011, Peter Pauly wrote:
>> 
>>> Gizmodo is reporting problems at Time Warner Telecom  we're
>> suffering
>>> from it too and calls to the NOC have not been answered so far...
> does
>>> anyone have any further information?
>>> 
>>> http://gizmodo.com/5857010/massive-time-warner-outage-hits-the-us
>> 
>> I noticed just a little while ago that we're having a lot of DNS fail.
>> Initial findings were that several of the root-servers we were trying
> to
>> reach via our TWTelecom link were unreachable after 2 hops into TWT.
>> 
>>  4  64-128-130-233.static.twtelecom.NET (64.128.130.233)  2.399 ms
> 2.298
>> ms  2.338 ms
>>  5  mia2-pr1-xe-1-3-0-0.us.twtelecom.net (66.192.253.18)  11.571 ms
>> 11.552 ms  9.467 ms
>>  6  * * *
>>  7  * * *
>>  8  * * *
>> 
>> For instance, a.root-servers.net is pingable from a rackspace server,
> but
>> not from our network (unless I shut off TWT, at which point it is, but
>> it's apparently not the same a.root-servers.net instance rackspace
> sees).
>> I assume this is one of the root-servers being anycast.
>> 
>> Shutting off our BGP with TWT didn't appear to help (though the
>> root-servers became reachable)...so I assume there's more going on
> than
>> just TWT routing fail.
>> 
>> --
>>  Jon Lewis, MCP :)   |  I route
>>  Senior Network Engineer |  therefore you are
>>  Atlantic Net|
>> _ http://www.lewis.org/~jlewis/pgp for PGP public key_
>> 
>> 
>> 
>> --
>> 
>> Message: 2
>> Date: Mon, 7 Nov 2011 15:29:30 +
>> From: Leigh Porter 
>> To: Bj?rn Mork 
>> Cc: "nanog@nanog.org" 
>> Subject: Re: Performance Issues - PTR Records
>> Message-ID: <53a4963f-4969-4a60-bf06-e690c7324...@ukbroadband.com>
>> Content-Type: text/plain; charset="iso-8859-1"
>> 
>> 
>> 
>> On 7 Nov 2011, at 14:03, "Bj?rn Mork"  wrote:
>> 
>>> Leigh Porter  writes:
>>> 
 Indeed, there is no way I would allow that either. But really,
 providing a reverse zone and forward zone to match is a case of
> five
 minutes and a shell script or a DNS that as Steinar said, will
 synthesise results.
 
 It's really not all that difficult..
>>> 
>>> No, not at all.  It's just totally pointless.  Any IPv6 address is
> just
>>> as pretty as a synthesized name.  Maybe even prettier. Do you prefer
>>> "2001:db8:1::2" or
> "20010db800010002.rev.example.com"?
>>> 
>>> If we're going to provide any reverse DNS for end users, then it is
>>> because we can create names which actually improves something.
>>> 
>>> 
>>> Bj?rn
>>> 
>>> 
>> 
>> Yup it is pointless.. Mine are all ipadrress.domain which is of
> course,
>> pointless.. I suppose at least somebody would glean that perhaps its a
>> home user rather than a business or server on that address but that's
> all.
>> 
>> With IPv6 arguably even more pointless as you say.
>> 
>> __
>> This email has been scanned by the MessageLabs Email Security System.
>> For more information please visit http://www.messagelabs.com/ema

R: Juniper MX960 Mqchip packet lenght error

[Ferretti Antonio]

Thx David,

PR587266 seems to be related to "maximum-packet-lenght" command that is not 
present in my config.

Biggest problem in this case are dropped packet on interface:

Physical interface: xe-10/0/1, Enabled, Physical link is Up
  Interface index: 363, SNMP ifIndex: 645, Generation: 483
  Description: No Description
  Link-level type: Ethernet, MTU: 1514, LAN-PHY mode, Speed: 10Gbps, Loopback: 
None, Source filtering: Disabled,
  Flow control: Enabled
  Device flags   : Present Running
  Interface flags: SNMP-Traps Internal: 0x4000
  Link flags : None
  CoS queues : 8 supported, 8 maximum usable queues
  Hold-times : Up 0 ms, Down 0 ms
  Current address: 00:24:dc:49:be:73, Hardware address: 00:24:dc:49:be:73
  Last flapped   : 2011-10-27 03:11:07 UTC (1w5d 12:18 ago)
  Statistics last cleared: 2011-11-08 13:15:03 UTC (02:14:38 ago)
  Traffic statistics:
   Input  bytes  :4674308593808   5182909688 bps
   Output bytes  :6092283580895   6656998280 bps
   Input  packets:   8393093736  1149495 pps
   Output packets:   6971170539   941600 pps
   IPv6 transit statistics:
Input  bytes  :23569859
Output bytes  :   0
Input  packets:  197870
Output packets:   0
  Dropped traffic statistics due to STP State:
   Input  bytes  :0
   Output bytes  :0
   Input  packets:0
   Output packets:0
  Input errors:
Errors: 0, Drops: 0, Framing errors: 0, Runts: 0, Policed discards: 0, L3 
incompletes: 0, L2 channel errors: 0,
L2 mismatch timeouts: 0, FIFO errors: 0, Resource errors: 0
  Output errors:
Carrier transitions: 0, Errors: 0, Drops: 0, Collisions: 0, Aged packets: 
0, FIFO errors: 0, HS link CRC errors: 0,
MTU errors: 0, Resource errors: 0
  Egress queues: 8 supported, 8 in use
  Queue counters:   Queued packets  Transmitted packets  Dropped packets
0 be6973872756   6887958719  85914037
1 ef  25695717 25695717  0
2 ef1  2062122  2062122  0
3 ef2 21430966 21430966  0
4 af   2605818  2605818  0
5 af1  1577987  1577987  0
6 nc   5928647  5928647  0
7 nc1 24705296 24705296  0

I'm wondering if is something related to CoS that is managed in different way 
from the two type of linecard.

Regards

Antonio Ferretti
Network Engineer
Telecom Italia Sparkle
TIS.AD.NDO.O Backbone IP&Data Operations
Seabone 2nd Level Support
Via di Macchia Palocco, 223
00125 Roma, Italia
-Messaggio originale-
Da: david@orange.com [mailto:david@orange.com]
Inviato: martedì 8 novembre 2011 16.13
A: Ferretti Antonio; nanog@nanog.org
Oggetto: RE: Juniper MX960 Mqchip packet lenght error

Hi,

See the PR587266. Maybe you have Syslog/log statement in a FW filter term.

Regards
David



David Roy
Orange - IP Domestic Backbone - TAC
Tel.   +33(0)299876472
Mob. +33(0)685522213
Email. david@orange.com
Skype : davidroy.35
JNCIE-SP #703

Guest Haiku :
IS-IS screams,
BGP peers are flapping:
I want my mommy!



-Message d'origine-
De : Ferretti Antonio [mailto:antonio.ferre...@telecomitalia.it]
Envoyé : mardi 8 novembre 2011 15:35
À : nanog@nanog.org
Objet : Juniper MX960 Mqchip packet lenght error

Hi all,

After upgrading Juniper MX960 from 4x10Gbe DPCE to 16x10Gbe MPC, I see strange 
log messages like this:

Fpc10 MQCHIP(0) LI Packet length error, pt entry 22

Seems to be only a "cosmetic" message, but today my team found some traffic 
dropped on interface on this card (before we only request check about log 
message).
Has anyone the same problem?

My software release is Junos 10.3R3.7.

Thanks in advance.

Antonio Ferretti
Network Engineer
Telecom Italia Sparkle
Backbone IP&Data Operations
Seabone 2nd Level Support
Questo messaggio e i suoi allegati sono indirizzati esclusivamente alle persone 
indicate. La diffusione, copia o qualsiasi altra azione derivante dalla 
conoscenza di queste informazioni sono rigorosamente vietate. Qualora abbiate 
ricevuto questo documento per errore siete cortesemente pregati di darne 
immediata comunicazione al mittente e di provvedere alla sua distruzione, 
Grazie.

This e-mail and any attachments is confidential and may contain privileged 
information intended for the addressee(s) only. Dissemination, copying, 
printing or use by anybody else is unauthorised. If you are not the intended 
recipient, please delete this message and any attachments and advise the sender 
by return e-mail, Thanks.

[cid:0001@TI.Disclaimer]Rispetta l'ambiente. Non 
stampare questa mail se non ? necessario.


**

Juniper MX960 Mqchip packet lenght error

[Ferretti Antonio]

Hi all,

After upgrading Juniper MX960 from 4x10Gbe DPCE to 16x10Gbe MPC, I see strange 
log messages like this:

Fpc10 MQCHIP(0) LI Packet length error, pt entry 22

Seems to be only a "cosmetic" message, but today my team found some traffic 
dropped on interface on this card (before we only request check about log 
message).
Has anyone the same problem?

My software release is Junos 10.3R3.7.

Thanks in advance.

Antonio Ferretti
Network Engineer
Telecom Italia Sparkle
Backbone IP&Data Operations
Seabone 2nd Level Support
Questo messaggio e i suoi allegati sono indirizzati esclusivamente alle persone 
indicate. La diffusione, copia o qualsiasi altra azione derivante dalla 
conoscenza di queste informazioni sono rigorosamente vietate. Qualora abbiate 
ricevuto questo documento per errore siete cortesemente pregati di darne 
immediata comunicazione al mittente e di provvedere alla sua distruzione, 
Grazie.

This e-mail and any attachments is confidential and may contain privileged 
information intended for the addressee(s) only. Dissemination, copying, 
printing or use by anybody else is unauthorised. If you are not the intended 
recipient, please delete this message and any attachments and advise the sender 
by return e-mail, Thanks.

[cid:0001@TI.Disclaimer]Rispetta l'ambiente. Non 
stampare questa mail se non ? necessario.

<>

Re: XO blocking individual IP's

[Jay Ashworth]

- Original Message -
> From: clay...@haydel.org

> > "transit provider". Is XO the end-access provider for either you or the
> > destination site? Or are both of those on some other connection, and XO
> > is a bystander along the way?
> 
> We're a direct customer. The IP's that I've seen them block have been
> both on our network and on remote networks, so I suspect their
> filtering would affect any traffic that happened to pass over XO.

Ah, ok.  Well, that certainly gives them standing to be filtering the traffic;
whether you think their reasoning is justified becomes a different level of
question at that point.

I concur with you that their filtering probably isn't justified, but I suspect
you'd find your contract permits it.

Cheers,
-- jra
-- 
Jay R. Ashworth  Baylink   j...@baylink.com
Designer The Things I Think   RFC 2100
Ashworth & Associates http://baylink.pitas.com 2000 Land Rover DII
St Petersburg FL USA  http://photo.imageinc.us +1 727 647 1274

Re: Performance Issues - PTR Records

[Jeroen Massar]

On 2011-11-08 13:27 , Mark Andrews wrote:
> In message <4eb90ef2.3030...@unfix.org>, Jeroen Massar writes:
>> On 2011-11-08 12:05 , Mark Andrews wrote:
>>> In message <4eb8f028.8040...@dds.nl>, Seth Mos writes:
>> [..]
>>> Sounds like FUD.  Who has trusted the contents of a PTR record in the
>>> last 2 decades?
>>
>> Lots of tools (read: SSH, Spam-checks, oh and IRCd's ;) trust PTR, but
>> only if the reverse => forward => reverse. And you don't want to know
>> how many silly people enable the "if user comes from .in they must be
>> from Indonesia^WIndia thus block them" Apache option as recently
>> mentioned on this very thread.
> 
> They arn't trusting the reverse record.  They are trusting the forward
> record to verify the reverse record. They know that the reverse record
> is untrustworthy as the owner of the reverse zone can put whatever they
> want there without spoofing anything.

Of course that is the case. The PTR itself is useless, but in combo with
checking it with the forward it is a very valuable resource.

(Add DNSSEC to the mix and you are even sure that nobody spoofed it on
the wire for you ;)

>> Also, note that your precious operating system will likely store the
>> PTR, sometimes even without doing the reverse->forward->reverse check.
> 
>> As such, you set up a PTR + Forward properly for a host, try to 'hack' a
>> box by password guessing, the log entries will only have the PTR
>> recorded, and you just drop the PTR+Forward from DNS (as they are under
>> your control) the admin comes in, sees all those nice hosts in their
>> logs but as it is gone from DNS will never ever find you. This
>> especially goes for 'who' (utmp) which makes that mistake. Fortunately
>> SSH at least logs both IP + hostname, the more info the better.
> 
> Who trusts logs of names without actual addresses?   No one sane
> does.

Well, only one decade back some people from this very list mentioned
that to a certain OS that is used quite a lot by a lot of people:

http://www.freebsd.org/cgi/query-pr.cgi?pr=22595

And today that is still the case:
http://www.freebsd.org/cgi/man.cgi?query=utmp&sektion=5

Note there is just ut_host there is no address being stored, I hope you
yourself btw don't use any FreeBSD based devices as otherwise that
little attempt at an insult goes for you too ;)

>> That said though the PTR->forward->PTR check is a proper check and a
>> really great way to figure out if the source SMTP host was actually set
>> up with at least some admin doing it the right way. If they can't be
>> bothered to set that up, why should you bother to accept that mail, or a
>> better choice, just score it a bit negatively at least.
> 
> Which only works as a filter because ISP's decided to prevent home
> users from putting valid PTR records in the DNS for their own
> machines.  It has nothing to do with clue or knowlege.  

I don't think ISPs 'decide' to not let users set up reverse DNS, it is
generally a 'feature' for which they can ask more moneyz.

If ISPs would allow it (which I am for btw) then they only pass the test
anyway if they can properly setup reverse->forward->reverse.
Which is likely the case anyway for quite some ISPs who populate
reverses with a matching forward&reverse based on the IP.

Greets,
 Jeroen

Re: where was my white knight....

[Dobbins, Roland]

On Nov 8, 2011, at 5:56 PM,  wrote:

> how would a sidr-enabled routing infrastructure have fared in yesterdays 
> routing circus?

The effects of large amounts of route-churn on the auth chain - perhaps DANE? - 
might've been interesting . . .

---
Roland Dobbins  // 

The basis of optimism is sheer terror.

  -- Oscar Wilde

Re: Performance Issues - PTR Records

[Mark Andrews]

In message <4eb90ef2.3030...@unfix.org>, Jeroen Massar writes:
> On 2011-11-08 12:05 , Mark Andrews wrote:
> > In message <4eb8f028.8040...@dds.nl>, Seth Mos writes:
> [..]
> > Sounds like FUD.  Who has trusted the contents of a PTR record in the
> > last 2 decades?
> 
> Lots of tools (read: SSH, Spam-checks, oh and IRCd's ;) trust PTR, but
> only if the reverse => forward => reverse. And you don't want to know
> how many silly people enable the "if user comes from .in they must be
> from Indonesia^WIndia thus block them" Apache option as recently
> mentioned on this very thread.

They arn't trusting the reverse record.  They are trusting the forward
record to verify the reverse record.  They know that the reverse record
is untrustworthy as the owner of the reverse zone can put whatever they
want there without spoofing anything.
 
> Also, note that your precious operating system will likely store the
> PTR, sometimes even without doing the reverse->forward->reverse check.

> As such, you set up a PTR + Forward properly for a host, try to 'hack' a
> box by password guessing, the log entries will only have the PTR
> recorded, and you just drop the PTR+Forward from DNS (as they are under
> your control) the admin comes in, sees all those nice hosts in their
> logs but as it is gone from DNS will never ever find you. This
> especially goes for 'who' (utmp) which makes that mistake. Fortunately
> SSH at least logs both IP + hostname, the more info the better.

Who trusts logs of names without actual addresses?   No one sane
does.

> That said though the PTR->forward->PTR check is a proper check and a
> really great way to figure out if the source SMTP host was actually set
> up with at least some admin doing it the right way. If they can't be
> bothered to set that up, why should you bother to accept that mail, or a
> better choice, just score it a bit negatively at least.

Which only works as a filter because ISP's decided to prevent home
users from putting valid PTR records in the DNS for their own
machines.  It has nothing to do with clue or knowlege.  

> Greets,
>  Jeroen
-- 
Mark Andrews, ISC
1 Seymour St., Dundas Valley, NSW 2117, Australia
PHONE: +61 2 9871 4742 INTERNET: ma...@isc.org

Re: XO blocking individual IP's

[Ryan Rawdon]

On Nov 7, 2011, at 10:06 PM, clay...@haydel.org wrote:

> 
>> "transit provider".  Is XO the end-access provider for either you or the
>> destination site?  Or are both of those on some other connection, and XO
>> is a bystander along the way?
> 
> We're a direct customer.  The IP's that I've seen them block have been
> both on our network and on remote networks, so I suspect their filtering
> would affect any traffic that happened to pass over XO.
> 

While troubleshooting another issue last week, someone in the NOC at one of our 
ISPs mentioned that they had encountered something similar recently.

"This 
looks suspiciously like another XO issue we ran across in the last few 
months where they used a network security device that blocked 'suspicious' 
traffic on particular ports (although it was tcp based from what I could 
remember)."

In our case the symptoms looked like GBLX was eating traffic which hashed to a 
certain theoretical link (certain src-dst-srcport-dstport combinations) in a 
LAG or similar, but it was happening right near the XO-GBLX edge in the forward 
path so it's possible it was a security device at XO's edge.

Re: Performance Issues - PTR Records

[Jeroen Massar]

On 2011-11-08 12:05 , Mark Andrews wrote:
> In message <4eb8f028.8040...@dds.nl>, Seth Mos writes:
[..]
> Sounds like FUD.  Who has trusted the contents of a PTR record in the
> last 2 decades?

Lots of tools (read: SSH, Spam-checks, oh and IRCd's ;) trust PTR, but
only if the reverse => forward => reverse. And you don't want to know
how many silly people enable the "if user comes from .in they must be
from Indonesia^WIndia thus block them" Apache option as recently
mentioned on this very thread.

Also, note that your precious operating system will likely store the
PTR, sometimes even without doing the reverse->forward->reverse check.

As such, you set up a PTR + Forward properly for a host, try to 'hack' a
box by password guessing, the log entries will only have the PTR
recorded, and you just drop the PTR+Forward from DNS (as they are under
your control) the admin comes in, sees all those nice hosts in their
logs but as it is gone from DNS will never ever find you. This
especially goes for 'who' (utmp) which makes that mistake. Fortunately
SSH at least logs both IP + hostname, the more info the better.


That said though the PTR->forward->PTR check is a proper check and a
really great way to figure out if the source SMTP host was actually set
up with at least some admin doing it the right way. If they can't be
bothered to set that up, why should you bother to accept that mail, or a
better choice, just score it a bit negatively at least.

Greets,
 Jeroen

Re: Performance Issues - PTR Records

[bmanning]

On Tue, Nov 08, 2011 at 10:05:12PM +1100, Mark Andrews wrote:
> 
> In message <4eb8f028.8040...@dds.nl>, Seth Mos writes:
> > On 7-11-2011 14:46, sth...@nethelp.no wrote:
> > >>> The practice of filling out the reverse zone with fake PTR record
> > >>> started before there was wide spread support for UPDATE/DNS.  There
> > >>> isn't any need for this to be done anymore.  Machines are capable
> > >>> of adding records for themselves.
> > >>
> > >> How do I setup this for DHCPv6-PD?  Say, I delegate 2001:db8:42::/48 to
> > >> the end user.  Should I delegate reverse DNS as well?  If so, to whom?
> > >>
> > >> Or is it the CPEs responibility to dynamically add records for whatever
> > >> addresses it sees on the internal LAN(s)?  Are there CPEs capable of
> > >> doing this?
> > >>
> > >> Or will the end systems themselves do the update against my DNS server?
> > >> If so, how do I authenticate that?
> > > 
> > > With my ISP hat on, I find the idea of customer CPEs updating their
> > > own PTR records to be completely unacceptable. So I guess I'll either
> > > live without the reverse DNS, or use a name server that can synthesize
> > > answers on the fly.
> > 
> > That seems like a really nice feature, create a reverse record to spoof
> > a mail server and the reverse DNS will match up.
> > 
> > If the domain does not employ SPF it will look legit, forward and
> > reverse won't match up ofcourse. Not sure how many mailservers have
> > issues with that if the reverse matches up.
> > 
> > Sounds like a fine way to employ a spam botnet.
> 
> Sounds like FUD.  Who has trusted the contents of a PTR record in the
> last 2 decades?
> 
> > Regards,
> > 
> > Seth
> -- 
> Mark Andrews, ISC
> 1 Seymour St., Dundas Valley, NSW 2117, Australia
> PHONE: +61 2 9871 4742 INTERNET: ma...@isc.org


the same people who trust the contents of an A record in the
last 2 decades.

/bill

Re: Performance Issues - PTR Records

[Mark Andrews]

In message <4eb8f028.8040...@dds.nl>, Seth Mos writes:
> On 7-11-2011 14:46, sth...@nethelp.no wrote:
> >>> The practice of filling out the reverse zone with fake PTR record
> >>> started before there was wide spread support for UPDATE/DNS.  There
> >>> isn't any need for this to be done anymore.  Machines are capable
> >>> of adding records for themselves.
> >>
> >> How do I setup this for DHCPv6-PD?  Say, I delegate 2001:db8:42::/48 to
> >> the end user.  Should I delegate reverse DNS as well?  If so, to whom?
> >>
> >> Or is it the CPEs responibility to dynamically add records for whatever
> >> addresses it sees on the internal LAN(s)?  Are there CPEs capable of
> >> doing this?
> >>
> >> Or will the end systems themselves do the update against my DNS server?
> >> If so, how do I authenticate that?
> > 
> > With my ISP hat on, I find the idea of customer CPEs updating their
> > own PTR records to be completely unacceptable. So I guess I'll either
> > live without the reverse DNS, or use a name server that can synthesize
> > answers on the fly.
> 
> That seems like a really nice feature, create a reverse record to spoof
> a mail server and the reverse DNS will match up.
> 
> If the domain does not employ SPF it will look legit, forward and
> reverse won't match up ofcourse. Not sure how many mailservers have
> issues with that if the reverse matches up.
> 
> Sounds like a fine way to employ a spam botnet.

Sounds like FUD.  Who has trusted the contents of a PTR record in the
last 2 decades?

> Regards,
> 
> Seth
-- 
Mark Andrews, ISC
1 Seymour St., Dundas Valley, NSW 2117, Australia
PHONE: +61 2 9871 4742 INTERNET: ma...@isc.org

where was my white knight....

[bmanning]

how would a sidr-enabled routing infrastructure have fared in yesterdays 
routing circus?

/bill

Re: Performance Issues - PTR Records

[Seth Mos]

On 7-11-2011 14:46, sth...@nethelp.no wrote:
>>> The practice of filling out the reverse zone with fake PTR record
>>> started before there was wide spread support for UPDATE/DNS.  There
>>> isn't any need for this to be done anymore.  Machines are capable
>>> of adding records for themselves.
>>
>> How do I setup this for DHCPv6-PD?  Say, I delegate 2001:db8:42::/48 to
>> the end user.  Should I delegate reverse DNS as well?  If so, to whom?
>>
>> Or is it the CPEs responibility to dynamically add records for whatever
>> addresses it sees on the internal LAN(s)?  Are there CPEs capable of
>> doing this?
>>
>> Or will the end systems themselves do the update against my DNS server?
>> If so, how do I authenticate that?
> 
> With my ISP hat on, I find the idea of customer CPEs updating their
> own PTR records to be completely unacceptable. So I guess I'll either
> live without the reverse DNS, or use a name server that can synthesize
> answers on the fly.

That seems like a really nice feature, create a reverse record to spoof
a mail server and the reverse DNS will match up.

If the domain does not employ SPF it will look legit, forward and
reverse won't match up ofcourse. Not sure how many mailservers have
issues with that if the reverse matches up.

Sounds like a fine way to employ a spam botnet.

Regards,

Seth

Re: [outages] More notes

[Bjørn Mork]

Stephane Bortzmeyer  writes:

>  ("Given the
> complexity of conditions required to trigger this issue, the
> probability of exploiting this defect is extremely low"). 

Which translates to

"This bug has such catastrophic consequenses that we do not want to
disclose how to trigger it."

Do you think any such bug would be discovered and/or disclosed *at all*
unless it already was triggered in the wild? And if it was triggered
once, what are the chances it will happen again?



Bjørn

Re: XO blocking individual IP's

[Leigh Porter]

So if you want to launch a DoS attack against a specific IP address you spoof 
TCP3389 SYNs to networks single homed to XO and they will null it for you.

-- 
Leigh


On 8 Nov 2011, at 04:36, "Blake T. Pfankuch"  wrote:

> Oh yes!  Good lord I about went insane with this.  I was working with a 
> customer single homed to cBeyond.  I spent 3 hours on the phone with cBeyond 
> to figure out what was going on, it looks like a broken route.  Come to find 
> out it was an XO "security null".  The engineer on the phone from cBeyond 
> said to me "Well, I have learned 2 things today.  1, XO nulls for 'security 
> purposes' at random.  2, I am no longer shocked by any ridiculous policy I 
> will ever come across again."
> 
> In this case majority traffic was going from cBeyond to anywhere (via XO) and 
> being eaten, however it was VERY tough to diagnose as all parties involved 
> assumed this would not be occurring between source and destination without 
> good public documentation or at least any record of this happening to someone 
> else.  Also I guess we all assumed that major bandwidth players don't filter 
> anything.
> 
> I personally think its good on paper, but very bad real life until there is a 
> way to notify the end customer of the violation quickly.  This issue 
> literally took 3 full weeks to figure out what was going on.  Yes this works 
> great in a colo datacenter as you have the customer contact info (hopefully). 
>  But in the case where my customers provider was having the IP filtered by 
> their transit it was hell to diagnose.  In my case the customer had a single 
> infected machine that was making outbound connections on TCP3389 in the range 
> of about 100 connections every 5 minutes and because of this was entirely 
> being "security nulled".
> 
> Blake
> 
> -Original Message-
> From: clay...@haydel.org [mailto:clay...@haydel.org] 
> Sent: Monday, November 07, 2011 7:43 PM
> To: nanog@nanog.org
> Subject: XO blocking individual IP's
> 
> 
> I'm hoping someone has had the same experiences, and is further toward a 
> resolution on this than I am. About 6 months ago, we noticed that XO was 
> blackholing one specific IP out of a /24.  Traces to that IP stopped on XO's 
> network, traces to anything else out of the block went through fine.
> XO finally admitted that they had a new security system that identifies 
> suspicious traffic and automatically blocks the IP for 30 minutes.  We had to 
> get the IP in question "whitelisted" by their security guys.  The traffic was 
> all legit, it was just on a high port # that they considered suspicious.
> 
> There have several more cases like this, and XO has not been forthcoming with 
> information. We're either looking to be exempted from this filtering or at 
> least get a detailed description of how the system works.  I'm not sure how 
> they think this is acceptable from a major transit provider.
> Anybody else had similar problems?
> 
> 
> Clayton Haydel
> 
> 
> 
> 
> __
> This email has been scanned by the MessageLabs Email Security System.
> For more information please visit http://www.messagelabs.com/email 
> __

__
This email has been scanned by the MessageLabs Email Security System.
For more information please visit http://www.messagelabs.com/email 
__

Re: [outages] More notes

[Stephane Bortzmeyer]

On Mon, Nov 07, 2011 at 08:37:55PM -0700,
 brian nikell  wrote 
 a message of 38 lines which said:

> Actually, Juniper does disclose code bugs. Though not always to the
> public at first, importantly to Juniper customers. Juniper had
> advised all of their customers last August of this bug, however
> Level3 chose to continue running it on their peer routers. Thus if
> Level3 and its clue(full) management might have listened to their
> operators & network engineers

I disagree. The official bug statement from Juniper in August was
trying very hard to downplay the importance of the bug ("Given the
complexity of conditions required to trigger this issue, the
probability of exploiting this defect is extremely low"). No wonder so
few people (and not only at Level-3) did not upgrade.