Re: Reaching out to ARIN members about their RPKI INVALID prefixes

[Jakob Heitz (jheitz) via NANOG]

Owen,

You are correct in that RPKI leaves many problems unsolved.

One that it does solve is prefix splitting.
If I issue a ROA for prefix 10.1.2.0/23, any announcement of 10.1.2.0/24 
(including mine) will be declared INVALID, because that announcement is covered 
by the ROA and the mask length is longer than maxlen.

Of course, as you rightly point out, if I do NOT announce that prefix myself, 
then anyone is free to announce it anywhere and have it declared VALID just by 
prepending my ASN.

Regards,
Jakob.

-Original Message-
Date: Tue, 18 Sep 2018 14:18:55 -0700
From: Owen DeLong 

What does RPKI offer other than a way to know what to spoof in a prepend for 
your forged announcement?

SAFNOG-4/EANOG/tzNOG: 4 Days To Go!

[Mark Tinka]

Hi all.

With just about 4 days to go until the 4th edition of the SAFNOG
meeting, in collaboration with EANOG and tzNOG, we are geared for an
exciting week in warm & sunny Dar Es Salaam.

The agenda will cover key topics for the region, such as:

  * Is Africa's continued telecommunications investment into Europe wise
for the continent?
  * Data centre infrastructure development in the region.
  * What the recent IRR changes at RIPE mean for Africa.
  * The advances of IPv4 brokers into Africa, using AFRINIC very own
resources in doing so.
  * e.t.c.

More details about the meeting, registration, venue and agenda at:

    http://www.safnog.org/

If you haven't yet, please register your attendance to join us next week.

We look forward to seeing you all there.

Cheers,

Mark Tinka
On Behalf of the SAFNOG-4/EANOG/tzNOG Organizing Committee

Re: Massive Price Increase for X-conns at Telehouse Chelsea, NYC

[Owen DeLong]

> On Sep 19, 2018, at 00:44 , Christopher Morrow  
> wrote:
> 
> 
> 
> On Tue, Sep 18, 2018 at 12:28 PM Owen DeLong  > wrote:
> 
> I’d argue that the difference between reasonable (≤$500 one-time and ≤$50 
> MRC) and $300 MRC is within range of argument, but I cannot see any way in 
> which an argument can be made that $5840 MRC is not a distortion in that same 
> circumstance.
> 
> 
> captive audiences (or those which view themselves as captive) are so much 
> fun... :(
> I imagine that 'away from Telaviv' in the 20ms arena is  actually pretty 
> hard to find, right?
> If you wanted to offer 'local' content locally (and offer it quickly/fastly 
> (ha!)) it's probably pretty hard in that part of the world, right? Whether it 
> actually 'costs' that much to pull a x-connect and maintain that x-connect is 
> probably not as important as 'gosh it's really hard to be 'close' to  
> ' right? which is what they are capitalizing on here.
> 
> Hank, how far away is the next closest large network metro ? Riyad? Rome? 
> Sofia?... I mean, it's all 'far' from 'isreal' (or really any part of the 
> world)  to the next decent network POP :(
>  

At those prices, it doesn’t take a lot of XCs to justify the cost of building 
an additional datacenter.

Owen

Re: the prefixes that wont be able to reach Cloudflare by the end of the year (unless RPKI ROAs are fixed)

[Owen DeLong]

Yep… It’s also better not to do SSL or IRR entries than to do it badly. Agreed.

Owen


> On Sep 19, 2018, at 18:00 , Michel Py  wrote:
> 
>> Owen DeLong wrote :
>> Note to self… It’s better not to do RPKI than to do it badly.
> 
> Not worse than IRR entries or SSL certificates. If you mess it up, resource 
> will go down.
> 
> Michel.
> 
> TSI Disclaimer:  This message and any files or text attached to it are 
> intended only for the recipients named above and contain information that may 
> be confidential or privileged. If you are not the intended recipient, you 
> must not forward, copy, use or otherwise disclose this communication or the 
> information contained herein. In the event you have received this message in 
> error, please notify the sender immediately by replying to this message, and 
> then delete all copies of it from your system. Thank you!...

Re: Reaching out to ARIN members about their RPKI INVALID prefixes

[Owen DeLong]

Looks like a certain CDN has volunteered to do that for you.

Owen


> On Sep 19, 2018, at 01:19 , Job Snijders  wrote:
> 
> On Wed, Sep 19, 2018 at 01:07:42AM -0700, Christopher Morrow wrote:
>>> it is about whether it is acceptable that RIRs (and more
>>> specifically ARIN in this mailing list's context) notify affected
>>> parties of their prefixes that suffer from stale ROAs.
>> 
>> This I still think is a bad plan.. mostly because I don't think it'll
>> help :( I think what helps is: "Oh, I cant get to  and  and
>> "  I think folk that CARE will do the right
>> thing, folk that 'think they care' won't and will soon get
>> disconnected from the tubez.
>> 
>> I apologize a tad if my view that: "breaking people will force them to
>> fix themselves" is  rough :(
> 
> What about an one-off outreach effort?
> 
> We need to somehow kickstart the feedback loop, especially if we expect
> effects to become forceful. I'm hoping that if the invalid count is low
> enough it'll become more attractive for more people flip the switch and
> deploy OV.
> 
> Kind regards,
> 
> Job

Re: Reaching out to ARIN members about their RPKI INVALID prefixes

[Owen DeLong]

> On Sep 19, 2018, at 00:46 , nusenu  wrote:
> 
> Owen DeLong:
>> Personally, since all RPKI accomplishes is providing a
>> cryptographically signed notation of origin ASNs that hijackers
>> should prepend to their announcements in order to create an aura of
>> credibility, I think we should stop throwing resources down this
>> rathole.
> 
> regardless of how one might think about RPKI, there are ROAs out 
> there that reduce the visibility/reachability of certain prefixes and the 
> general assumption is that announced prefixes would like to be reachable
> even if the operator doesn't care about RPKI and ROAs from the past anymore, 
> he most likely cares
> about reachability from a pure operational point of view.

Yep… And the easy recipe for one which doesn’t care about RPKI to restore 
reachability is “delete the ROAs”.

> my email was not about: "How much does one like RPKI?”

I have no impression that it was.

I thought it was about “Should we consume more RIR resources dealing with this 
additional pain likely to be caused by RPKI?”

> it is about whether it is acceptable that RIRs (and more specifically ARIN in 
> this mailing list's context) 
> notify affected parties of their prefixes that suffer from stale ROAs.

I agree with Mr. Morrow that this would end in pain.

> Even if one dislikes RPKI entirely the opinion could still be "yes notifying 
> those parties makes sense
> to restore reachability”.

Agreed. However, whether I liked RPKI or not, I’d still say that notification 
by the RIRs is prone
to sadness. My initial intent was merely to state that I prefer the RIRs not 
waste additional
resources on this, including notification.

Owen

Re: Reaching out to ARIN members about their RPKI INVALID prefixes

[Jared Mauch]

> On Sep 19, 2018, at 8:55 PM, Owen DeLong  wrote:
> 
> Actually, from my perspective, neither one is practical/useful due to the 
> lack of supporting data to achieve it.

I suggest you look at some of the cool research that was done with various 
prefixes from different regions.

You can see the problem with ARIN prefixes fairly easily and how they’re harder 
to secure as a result.  This seems to be broken by design on the part of ARIN 
based on my limited experiences interacting with the community folk.

https://nlnog.net/static/nlnogday2018/8_Measuring_RPKI_ben_NLNOG_2018.pdf

And the video here:

https://www.youtube.com/watch?v=uDIQDpGObdc

It’s super interesting to see which RIR prefixes perform better when it comes 
to the same security technology.

- Jared

RE: the prefixes that wont be able to reach Cloudflare by the end of the year (unless RPKI ROAs are fixed)

[Michel Py]

> Owen DeLong wrote :
> Note to self… It’s better not to do RPKI than to do it badly.

Not worse than IRR entries or SSL certificates. If you mess it up, resource 
will go down.

Michel.

TSI Disclaimer:  This message and any files or text attached to it are intended 
only for the recipients named above and contain information that may be 
confidential or privileged. If you are not the intended recipient, you must not 
forward, copy, use or otherwise disclose this communication or the information 
contained herein. In the event you have received this message in error, please 
notify the sender immediately by replying to this message, and then delete all 
copies of it from your system. Thank you!...

Re: Reaching out to ARIN members about their RPKI INVALID prefixes

[Owen DeLong]

> On Sep 19, 2018, at 00:44 , Job Snijders  wrote:
> 
> On Tue, Sep 18, 2018 at 06:18:00PM -0700, Owen DeLong wrote:
>> That depends. If you ONLY allow the maintainer of NET-192.159.10.0/24
>> to update the route objects for it, then the word ONLY is effectively
>> present by the lack of any other route objects.
> 
> Ah, so you are now applying the RPKI Origin Validation procedure to a
> non-existent flavor of IRR? :-)
> 
> Today I can create route-objects covering 192.159.10.0/24 in a number of
> IRR databases and there is nothing you can do to prevent that, this
> simply is not the case with RPKI. I prefer an existing system (RPKI)
> over hypotheticals (Owen's IRR). 
> 
> Secondly, I've also noticed you only emphasize an adversarial angle
> (origin spoofing), but there are other angles too.
> 
> The majority of today's BGP problems are attributable to operator
> mistakes (misconfigurations). Analysis has shown that most BGP incidents
> happen on weekdays rather than in the weekend. The number keys on our
> keyboards are quite close to each other and Origin Validation is very
> effective against typos. Another angle is bugs in BGP implementations:
> your neighbors doing origin validation reduces the impact and
> propagation of incorrect announcements from your network should you run
> into a software defect.

Sure, but given the email thread that started this all, if people start 
enforcing
RPKI based origin validation, do you think it will create fewer or more mistakes
in this regard?

It appears to me that the complexity of RPKI and other issues will lead to
RPKI causing more human-factors based routing accidents than it will
likely prevent.

>> So RPKI is great if we can just reduce the internet diameter to 1
> 
> Agreed, in other words: RPKI is offers tangible benefits to those that
> peer directly with each other, or use peerlock.

Agreed, noting the assumptions built into the theory that everyone can
use peerlock.

>> in which case MD5 passwords on your BGP sessions pretty much
>> accomplishes the same thing with a lot less kerfuffle.
> 
> Uhh... you may want to refresh your memory on what BGP is and how
> TCP-MD5 works.

Admittedly, it doesn’t cover the fat finger cases above, but, it does
cover the “know who you’re accepting the route from” issue for one
hop out and in a way that doesn’t allow you to create a time-bomb
that becomes visible on a delayed basis when you fat-finger it.

Owen

Re: Reaching out to ARIN members about their RPKI INVALID prefixes

[Owen DeLong]

> On Sep 18, 2018, at 21:29 , Christopher Morrow  
> wrote:
> 
> 
> 
> On Tue, Sep 18, 2018 at 6:22 PM Owen DeLong  > wrote:
> 
> 
> > On Sep 18, 2018, at 15:07 , Job Snijders  > > wrote:
> > 
> > On Tue, Sep 18, 2018 at 02:44:30PM -0700, Owen DeLong wrote:
> >> ROAs are useful for one hop level validation. At the second AS hop
> >> they are 100% useless.
> > 
> > This conversation cannot be had without acknowledging there are multiple
> > layers of defense in securing BGP. We should also acknowledge that the
> > majority of Internet traffic passes over AS_PATHs that are only one hop.
> > Networks that exchange significant amounts of traffic, tend to peer
> > directly with each other.
> 
> Actually, I don’t buy that at all.
> 
> Without going into too much detail, I know of at least one former employer 
> who is quite
> well peered, distributes a great deal of traffic and sends a tremendous 
> amount of it
> via multiple ASNs.
> 
> 
> an IP resource holder can sign multiple ROA for a single prefix, it's 
> perfectly valid to have:
>   157.130.0.0/16  AS112
>   157.130.0.0/16  AS113
>   157.130.0.0/16  AS701

I did not mean that they originate the traffic from multiple ASNs, I mean that 
a tremendous amount of their traffic goes origin->ASHOP1->ASHOP2->…->END USER.

> So 'peered well via multiple ASN isn't really a problem here'. Are you making 
> an argument of the form: "1% of the problems are not solved so this solution 
> can't possibly work” ?

Except you misunderstood my meaning. Perhaps my fault, but hopefully the above 
clarification helps you see my point.

> Using ROA from the RPKI system tied through/to the IRR data and applied as 
> filters on the bgp sessions of a network should provide that ASN with more 
> assurance that they will not accept and propagate a route hijack or mistake. 
> Adding in validation is nice as well, and may allow you to be a little less 
> diligent about route filtering... I think more than 1 protection is a good 
> plan though (OV + filters seems sane to me, especially in a world where you 
> can automate that whole lot)

Again, unless you can trust the data in the IRR to build a complete list of 
valid AS Paths from the ORIGIN, you can’t safely reject a fake route that has 
the correct prepend.

The ability to do that strikes me as questionable at best in the vast majority 
of cases.

> >>> In other words, RPKI and the Prefix-to-AS validation procedure give
> >>> us much more definitive inputs for routing policies compared to what
> >>> can be derived from the IRR.
> >> 
> >> Please explain to me how you distinguish good from bad in the
> >> following scenario… You peer with AS6939. You receive routes for
> >> 2001:db8:f300::/48 with the following AS Paths
> >> 
> >>  1. 6939 1239 54049 2312 1734
> >>  2. 6939 44046 12049 174 1734
> >> 
> >> Which one is valid? Which one is not? How did the ROA tell you this?
> > 
> > Both path 1 and 2 are invalid, because of peerlock we'd never accept
> > 1239 behind 6939, or 174 behind 6939. AS_PATH filtering combined with
> > Origin Validation is where the magic is.
> 
> OK, poor examples crafted at random. Point is there are plenty of valid AS 
> Paths
> out there that you can’t actually validate.
> 
> I think Job's point is that there really are note that many... perhaps that's 
> an NTT particular view? I know that my production network's view is similar 
> in 'shorter as  paths'. I expect that in large-isp-land it's more prevalent 
> than not.

Presuming s/note/not/

I suspect there might be some validity to that viewpoint for supposed “Tier 1” 
networks.
Once you hit Tier 2 and consider their subscribers in the mix, it gets a lot 
fuzzier and less likely that it is a valid perspective.
For the average eyeball network, I bet it’s a complete non-starter.

> 
> >>> RPKI is useful in context of a defense in depth strategy. If one
> >>> combines "peerlock" AS_PATH filters with origin validation the end
> >>> result is bullet proof. Even if NTT is the only one to deploy this
> >>> combination, the benefits are notable.
> >> 
> >> Indeed, if peerlock gets wider deployment, it might prove useful. But
> >> unless I’m really misunderstanding peerlock, I don’t see that RPKI
> >> brings much else to the table in addition.
> > 
> > Wide deployment is not relevant, this is a unilateral defense mechanism,
> > so I fear there may indeed be a degree of misunderstanding.
> 
> Point being that there are very very few ASNs using peer lock. Peer lock
> alone AIUI pretty well covers the issue. RPKI provides very little (if any)
> verification.
> 
> 
> I suppose if you are happy just doing peer lock on/for your network and 
> customers then cool!
> The RPKI system isn't being forced on you. It seems like a helpful addition 
> to me, but that may not be your outlook.

Actually, from my perspective, neither one is 

Re: the prefixes that wont be able to reach Cloudflare by the end of the year (unless RPKI ROAs are fixed)

[Owen DeLong]

Note to self… It’s better not to do RPKI than to do it badly.

Owen


> On Sep 19, 2018, at 09:32 , nusenu  wrote:
> 
> Hi,
> 
> apparently Cloudflare will be enforcing RPKI route origin validation
> "by the end of the year" [1].
> 
> https://blog.cloudflare.com/rpki-details/
> 
> If this is actually the case then some prefixes run at risk of loosing
> the ability to reach Cloudflare.
> 
> This is a heads-up so you can check if you would be affected,
> you can ctrl-f the list of 75 prefixes (ARIN region only)
> bellow for your prefix or ASN.
> 
> (data as of 2018-09-19 14:06 UTC)
> 
> 
> https://bgp.he.net/net/168.245.235.0/24  (AS13428)
> https://bgp.he.net/net/69.28.67.0/24  (AS13768)
> https://bgp.he.net/net/69.28.82.0/23  (AS13768)
> https://bgp.he.net/net/68.64.234.0/24  (AS14237)
> https://bgp.he.net/net/209.24.0.0/24  (AS15562)
> https://bgp.he.net/net/172.98.214.0/24  (AS17139)
> https://bgp.he.net/net/66.85.44.0/24  (AS19437)
> https://bgp.he.net/net/23.139.0.0/24  (AS20860)
> https://bgp.he.net/net/68.64.227.0/24  (AS262913)
> https://bgp.he.net/net/192.136.187.0/24  (AS26827)
> https://bgp.he.net/net/208.76.33.0/24  (AS26938)
> https://bgp.he.net/net/208.76.39.0/24  (AS26938)
> https://bgp.he.net/net/68.64.231.0/24  (AS30167)
> https://bgp.he.net/net/192.133.106.0/24  (AS33060)
> https://bgp.he.net/net/192.133.107.0/24  (AS33060)
> https://bgp.he.net/net/192.209.63.0/24  (AS393398)
> https://bgp.he.net/net/198.28.13.0/24  (AS393451)
> https://bgp.he.net/net/2606:8e80:3000::/48  (AS394308)
> https://bgp.he.net/net/2606:8e80:1000::/48  (AS394497)
> https://bgp.he.net/net/2606:8e80:4000::/48  (AS394497)
> https://bgp.he.net/net/2606:8e80::/48  (AS394497)
> https://bgp.he.net/net/2606:8e80:5000::/48  (AS394497)
> https://bgp.he.net/net/2606:8e80:2000::/48  (AS394644)
> https://bgp.he.net/net/2606:8e80:4000::/48  (AS394644)
> https://bgp.he.net/net/104.245.239.0/24  (AS395970)
> https://bgp.he.net/net/172.98.209.0/24  (AS395970)
> https://bgp.he.net/net/172.98.210.0/24  (AS395970)
> https://bgp.he.net/net/172.98.212.0/24  (AS395970)
> https://bgp.he.net/net/172.98.214.0/24  (AS395970)
> https://bgp.he.net/net/172.98.215.0/24  (AS395970)
> https://bgp.he.net/net/172.98.208.0/24  (AS41139)
> https://bgp.he.net/net/172.98.211.0/24  (AS41139)
> https://bgp.he.net/net/172.98.213.0/24  (AS41139)
> https://bgp.he.net/net/168.245.223.0/24  (AS43181)
> https://bgp.he.net/net/208.84.64.0/24  (AS52129)
> https://bgp.he.net/net/208.86.200.0/24  (AS52129)
> https://bgp.he.net/net/199.68.168.0/22  (AS53429)
> https://bgp.he.net/net/199.68.175.0/24  (AS53429)
> https://bgp.he.net/net/162.208.108.0/24  (AS55079)
> https://bgp.he.net/net/162.208.109.0/24  (AS55079)
> https://bgp.he.net/net/162.208.110.0/24  (AS55079)
> https://bgp.he.net/net/162.208.111.0/24  (AS55079)
> https://bgp.he.net/net/198.176.44.0/24  (AS55079)
> https://bgp.he.net/net/198.176.46.0/24  (AS55079)
> https://bgp.he.net/net/198.176.47.0/24  (AS55079)
> https://bgp.he.net/net/2604:a680:2::/48  (AS55079)
> https://bgp.he.net/net/2604:ab80:2::/48  (AS55079)
> https://bgp.he.net/net/2604:ab80:5::/48  (AS55079)
> https://bgp.he.net/net/198.176.45.0/24  (AS55097)
> https://bgp.he.net/net/2604:a680:4::/48  (AS55097)
> https://bgp.he.net/net/208.66.204.0/24  (AS6165)
> https://bgp.he.net/net/208.66.205.0/24  (AS6165)
> https://bgp.he.net/net/208.66.206.0/24  (AS6165)
> https://bgp.he.net/net/208.66.207.0/24  (AS6165)
> https://bgp.he.net/net/74.116.232.0/24  (AS6165)
> https://bgp.he.net/net/74.116.233.0/24  (AS6165)
> https://bgp.he.net/net/74.116.234.0/24  (AS6165)
> https://bgp.he.net/net/74.116.235.0/24  (AS6165)
> https://bgp.he.net/net/74.116.236.0/24  (AS6165)
> https://bgp.he.net/net/74.116.237.0/24  (AS6165)
> https://bgp.he.net/net/74.116.238.0/24  (AS6165)
> https://bgp.he.net/net/198.24.10.0/24  (AS62541)
> https://bgp.he.net/net/198.24.11.0/24  (AS62541)
> https://bgp.he.net/net/104.171.208.0/20  (AS63267)
> https://bgp.he.net/net/104.171.208.0/24  (AS63267)
> https://bgp.he.net/net/69.28.64.0/20  (AS6364)
> https://bgp.he.net/net/69.28.80.0/23  (AS6364)
> https://bgp.he.net/net/69.28.84.0/23  (AS6364)
> https://bgp.he.net/net/69.28.86.0/24  (AS6364)
> https://bgp.he.net/net/69.28.87.0/24  (AS6364)
> https://bgp.he.net/net/69.28.88.0/23  (AS6364)
> https://bgp.he.net/net/69.28.88.0/24  (AS6364)
> https://bgp.he.net/net/69.28.90.0/23  (AS6364)
> https://bgp.he.net/net/69.28.92.0/22  (AS6364)
> https://bgp.he.net/net/172.93.121.0/24  (AS8100)
> https://bgp.he.net/net/66.85.45.0/24  (AS8100)
> https://bgp.he.net/net/206.53.202.0/24  (AS11492) that is probably supposed 
> to be invalid (DE-CIX Dallas peering LAN? :)
> 
> 
> [1] https://twitter.com/Jerome_UZ/status/1042433414371205120
> 
> -- 
> https://twitter.com/nusenu_
> https://mastodon.social/@nusenu
> 

Re: Console Servers

[Mike Hammett]

There's always the WOOBM! 

https://mikrotik.com/product/woobm 




- 
Mike Hammett 
Intelligent Computing Solutions 

Midwest Internet Exchange 

The Brothers WISP 

- Original Message -

From: "Owen DeLong"  
To: "Mike Hammett"  
Cc: "Saku Ytti" , nanog@nanog.org 
Sent: Wednesday, September 19, 2018 7:05:36 PM 
Subject: Re: Console Servers 

Why am I picturing you rigging up a Particle Electron as a dongle to each 
device you want remote access to? 


Owen 






On Sep 19, 2018, at 02:21 , Mike Hammett < na...@ics-il.net > wrote: 


Except for AT, most incumbents here aren't also mobile wireless providers, so 
that is an option in most cases for truly OOB. 




- 
Mike Hammett 
Intelligent Computing Solutions 

Midwest Internet Exchange 

The Brothers WISP 

- Original Message -

From: "Saku Ytti" < s...@ytti.fi > 
To: "James Bensley" < jwbens...@gmail.com > 
Cc: nanog@nanog.org 
Sent: Wednesday, September 19, 2018 4:04:58 AM 
Subject: Re: Console Servers 

On Wed, 19 Sep 2018 at 11:54, James Bensley  wrote: 

> I forgot to mention, it also depends how "out" of band your OOB needs 
> to be. We use Ciena 6500s for our DWDM infrastructure and they have a 
> wayside channel (like various DWDM vendors), so it's a separate 
> channel over the same physical fibre. For anything except a fibre cut 
> it seems to work. 

This is gold standard for incumbents, as they don't have anything true 
out-of-band they can consistently buy, everything travels in their 
network at some point anyhow. 

-- 
++ytti 

Re: Console Servers

[Owen DeLong]

Why am I picturing you rigging up a Particle Electron as a dongle to each 
device you want remote access to?

Owen


> On Sep 19, 2018, at 02:21 , Mike Hammett  wrote:
> 
> Except for AT, most incumbents here aren't also mobile wireless providers, 
> so that is an option in most cases for truly OOB.
> 
> 
> 
> -
> Mike Hammett
> Intelligent Computing Solutions 
>   
>  
>  
> 
> Midwest Internet Exchange 
>   
>  
> 
> The Brothers WISP 
>   
> 
> From: "Saku Ytti" 
> To: "James Bensley" 
> Cc: nanog@nanog.org
> Sent: Wednesday, September 19, 2018 4:04:58 AM
> Subject: Re: Console Servers
> 
> On Wed, 19 Sep 2018 at 11:54, James Bensley  wrote:
> 
> > I forgot to mention, it also depends how "out" of band your OOB needs
> > to be. We use Ciena 6500s for our DWDM infrastructure and they have a
> > wayside channel (like various DWDM vendors), so it's a separate
> > channel over the same physical fibre. For anything except a fibre cut
> > it seems to work.
> 
> This is gold standard for incumbents, as they don't have anything true
> out-of-band they can consistently buy, everything travels in their
> network at some point anyhow.
> 
> -- 
>   ++ytti

Re: Console Servers

[Owen DeLong]

> On Sep 19, 2018, at 01:50 , Saku Ytti  wrote:
> 
> Hey,
> 
>> In some DCs I've done mutual OOB swaps with other telcos in the same
>> suite, this is usually cheap or free (excluding the one time xconnect
> 
> We consciously decided to not ask or accept OOB swaps, because of fear
> that they might be provisioned outside processes which might make it
> impossible to repair them through normal commercial processes, which
> would potentially cost lot of downtime and NOC's resources.
> 
>> Sometimes the DC provider has an OOB connectivity service that uses
>> separate transit providers than we use and this often cheap too. Again
>> this is often bespoke per DC/colo provider though.
> 
> MRC quotes I have 400USD Equinix, 288USD Terramark, 300USD Coresite.
> Compared to PSTN which we see at 90-150USD. This makes me less
> inclined to focus on HW CAPEX and optimise for HW/SW that tooling and
> people already support.

Your PSTN figure doesn’t include the cost of the XC to bring that POTS line
into your suite/cage/cabinet.

Once you add that in, It looks to me like you probably exceeded the OOB service
price in each of the cases quoted above.

>> The most scalable solution I've been involved in so far is VDSL. Here
>> in the UK lots of DCs are on-net for the national incumbent VDSL
> 
> I think WAN indeed is very market situational, and if you need to
> support world, it is beneficial to have solution which supports many
> WAN options, without needing external boxes and external power bricks.
> We try to do just ethernet, but even that is already being provided as
> copper, fibre and in one market with PPPoE, all which are non-issues
> by going with Cisco. I do wish I had second option, I do wish JNPR SRX
> would support async serial ports.

https://opengear.com/products/cm7100-console-server 


Has SFP network ports.

Owen

Re: Console Servers

[David Kotlerewsky]

+++ for Opengear. Manages PDUs and UPS, some models have GPS and 4G LTE 
options. If additional intelligence is needed for a lights out facility, 
Uplogix has an interesting solution as well.



Sincerely,

David K.

Sent from my mobile device, please excuse any typos or brevity.

From: NANOG  on behalf of Erik Sundberg 

Sent: Tuesday, September 18, 2018 7:27:49 PM
To: Jun Tanaka; nanog@nanog.org; Alan Hannan; NANOG
Subject: RE: Console Servers

Perle IOLAN SCS series is great. We have them all over the United States.



From: NANOG  On Behalf Of Jun Tanaka
Sent: Tuesday, September 18, 2018 10:52 AM
To: nanog@nanog.org; Alan Hannan ; NANOG 
Subject: Re: Console Servers

How about SMART CS series by Seiko solutions?
https://www.seiko-sol.co.jp/en/products/console-server/
--
Jun Tanaka - NetComBB/S.N.I



CONFIDENTIALITY NOTICE: This e-mail transmission, and any documents, files or 
previous e-mail messages attached to it may contain confidential information 
that is legally privileged. If you are not the intended recipient, or a person 
responsible for delivering it to the intended recipient, you are hereby 
notified that any disclosure, copying, distribution or use of any of the 
information contained in or attached to this transmission is STRICTLY 
PROHIBITED. If you have received this transmission in error please notify the 
sender immediately by replying to this e-mail. You must destroy the original 
transmission and its attachments without reading or saving in any manner. Thank 
you.

RE: the prefixes that wont be able to reach Cloudflare by the end of the year (unless RPKI ROAs are fixed)

[Michel Py]

> nusenu wrote :
> apparently Cloudflare will be enforcing RPKI route origin validation "by the 
> end of the year" [1].
> https://blog.cloudflare.com/rpki-details/
> If this is actually the case then some prefixes run at risk of loosing the 
> ability to reach Cloudflare.

This is the way we are going to get people to clean up their invalid prefixes. 
When people start to actually discard or block them and something breaks.

I still think that ARIN should be contacting them, if they are willing to do it.


> Phil Lavin wrote :
> That said, having recently done this with ARIN... they've got a long way to 
> go before it's a simple process (like RIPE). Submitting numerous tickets over 
> a 3 day period doesn't strike me as particularly efficient.

I was wondering if this is the reason ARIN is so far behind RIPE in terms of 
RPKI adoption. I did not find it bad personally, but I could understand that it 
may discourage people with a large number of prefixes.
There must be something else than the process not being as simple as RIPE's, 
IMHO.

Michel.

TSI Disclaimer:  This message and any files or text attached to it are intended 
only for the recipients named above and contain information that may be 
confidential or privileged. If you are not the intended recipient, you must not 
forward, copy, use or otherwise disclose this communication or the information 
contained herein. In the event you have received this message in error, please 
notify the sender immediately by replying to this message, and then delete all 
copies of it from your system. Thank you!...

Re: Console Servers

[Andrew Latham]

Note: newer Lantronix don't require Java for the config interface at all.

Also note that you can organize OOBM and in band management with
https://guacamole.apache.org/ if needed.

On Wed, Sep 19, 2018 at 12:47 PM Jeremy Bresley  wrote:

> On 9/19/18 04:40, James Bensley wrote:
> > On Tue, 18 Sep 2018 at 14:38, Alan Hannan  wrote:
> >> I'd like your input on suggestions for an alternate serial port manager.
> >>
> >> Long ago I used Cisco 2511/2611 and was fairly happy.  A little later I
> used portmaster and was less so.  Recently I've been using Opengear and
> they work fairly well but the price is fairly high.   I use the CM7100 and
> IM7100.
> >>
> >> General specs I'm looking for are:
> >>
> >>   * 8 to 48 or more rs232 serial ports on rj45
> >>   * nice-to-have software selectable pinouts (cisco v. straight)
> >>   * gig-e ethernet port (100mbps ok)
> >>   * 1U form factor
> >>   * redundant AC power
> >>   * access physical serial connections via local port #
> >>   * access physical serial connections via local IP alias (nice to have)
> >>
> > Hi Alan,
> >
> > I'd be reluctant to deploy Cisco 2800s (or similar) today unless there
> > is a newer variant, is there an ISGv2 variant with serial connectivity
> > that Cisco will be supporting for a few more years? I know OpenGrear
> > are expensive but in my current outfit, they do "just work" and the
> > few we had at my old place, again they did "just work".
> The ISR G2s do have several options for async available as do the
> current generation ISR4Ks.
>
> The ISR G2s (1900/2900/3900s) can take the HWIC-8A, HWIC-16A, or SM-32A
> for 8/16/32 ports (SM-32A only in 2911 and higher due to being a Service
> Module form factor)
>
> Data sheet:
>
> https://www.cisco.com/c/en/us/products/collateral/interfaces-modules/1800-2800-3800-series-16-port-async-high-speed-wan-interface-card/product_data_sheet0900aecd80274416.html
>
> The ISR G2 routers were all announced for End-of-Sale a while back, the
> modules for them were also announced recently, but are still available
> for sale until Feb 2019.  They'll still be supported until Feb 2024.
>
> EOL Announcement:
>
> https://www.cisco.com/c/en/us/products/collateral/interfaces-modules/network-modules/eos-eol-notice-c51-741231.html
>
> The ISR 4Ks have the NIM-16A, NIM-24A, and the SM-X-64A (16/24/64
> ports).  The SM-X is only supported in 4331 and higher due to the SM-X
> form factor, the 16/24 port ones support at least 2 modules in all
> ISR4Ks even the low-end 4221.  The NIM-16A and the SM-X-64A can use the
> same cables as the older async modules, the NIM-24A requires the newer
> low profile cable for 1 of the ports (can use it for all ports).
>
> Data sheet:
>
> https://www.cisco.com/c/en/us/products/collateral/routers/4000-series-integrated-services-routers-isr/datasheet-c78-739968.html
>
> Talk to your favorite SE or partner for more info and pricing.
>
> Jeremy
>
> Disclaimer, I do work for Cisco, this info is provided to the list as it
> was requested and hoping to clarify what's available.
>
> My personal $0.02: I've also used some of the older Opengear boxes in
> the past, they're solid, and Opengear are very good with customer
> suggestions/feedback.  Lantronix SLCs work once you get them configured,
> but their configuration web interface was intolerably slow (page
> refreshes would eat whatever you input into a second option box you
> clicked to change) and their built-in terminal required Java.  Benefit
> of Opengear is the other "things" you can do with them since they're
> Linux based (TFTP/syslog/etc). Benefit of a Cisco ISR is they're
> straight IOS (G2s)/IOS-XE (4Ks) so any configuration tool that can
> handle a Cisco box can work with them.
>
>

-- 
- Andrew "lathama" Latham -

Re: Console Servers

[Jeremy Bresley]

On 9/19/18 04:40, James Bensley wrote:

On Tue, 18 Sep 2018 at 14:38, Alan Hannan  wrote:

I'd like your input on suggestions for an alternate serial port manager.

Long ago I used Cisco 2511/2611 and was fairly happy.  A little later I used 
portmaster and was less so.  Recently I've been using Opengear and they work 
fairly well but the price is fairly high.   I use the CM7100 and IM7100.

General specs I'm looking for are:

  * 8 to 48 or more rs232 serial ports on rj45
  * nice-to-have software selectable pinouts (cisco v. straight)
  * gig-e ethernet port (100mbps ok)
  * 1U form factor
  * redundant AC power
  * access physical serial connections via local port #
  * access physical serial connections via local IP alias (nice to have)


Hi Alan,

I'd be reluctant to deploy Cisco 2800s (or similar) today unless there
is a newer variant, is there an ISGv2 variant with serial connectivity
that Cisco will be supporting for a few more years? I know OpenGrear
are expensive but in my current outfit, they do "just work" and the
few we had at my old place, again they did "just work".
The ISR G2s do have several options for async available as do the 
current generation ISR4Ks.


The ISR G2s (1900/2900/3900s) can take the HWIC-8A, HWIC-16A, or SM-32A 
for 8/16/32 ports (SM-32A only in 2911 and higher due to being a Service 
Module form factor)


Data sheet: 
https://www.cisco.com/c/en/us/products/collateral/interfaces-modules/1800-2800-3800-series-16-port-async-high-speed-wan-interface-card/product_data_sheet0900aecd80274416.html


The ISR G2 routers were all announced for End-of-Sale a while back, the 
modules for them were also announced recently, but are still available 
for sale until Feb 2019.  They'll still be supported until Feb 2024.


EOL Announcement: 
https://www.cisco.com/c/en/us/products/collateral/interfaces-modules/network-modules/eos-eol-notice-c51-741231.html


The ISR 4Ks have the NIM-16A, NIM-24A, and the SM-X-64A (16/24/64 
ports).  The SM-X is only supported in 4331 and higher due to the SM-X 
form factor, the 16/24 port ones support at least 2 modules in all 
ISR4Ks even the low-end 4221.  The NIM-16A and the SM-X-64A can use the 
same cables as the older async modules, the NIM-24A requires the newer 
low profile cable for 1 of the ports (can use it for all ports).


Data sheet: 
https://www.cisco.com/c/en/us/products/collateral/routers/4000-series-integrated-services-routers-isr/datasheet-c78-739968.html


Talk to your favorite SE or partner for more info and pricing.

Jeremy

Disclaimer, I do work for Cisco, this info is provided to the list as it 
was requested and hoping to clarify what's available.


My personal $0.02: I've also used some of the older Opengear boxes in 
the past, they're solid, and Opengear are very good with customer 
suggestions/feedback.  Lantronix SLCs work once you get them configured, 
but their configuration web interface was intolerably slow (page 
refreshes would eat whatever you input into a second option box you 
clicked to change) and their built-in terminal required Java.  Benefit 
of Opengear is the other "things" you can do with them since they're 
Linux based (TFTP/syslog/etc). Benefit of a Cisco ISR is they're 
straight IOS (G2s)/IOS-XE (4Ks) so any configuration tool that can 
handle a Cisco box can work with them.

the prefixes that wont be able to reach Cloudflare by the end of the year (unless RPKI ROAs are fixed)

[nusenu]

Hi,

apparently Cloudflare will be enforcing RPKI route origin validation
"by the end of the year" [1].

https://blog.cloudflare.com/rpki-details/

If this is actually the case then some prefixes run at risk of loosing
the ability to reach Cloudflare.

This is a heads-up so you can check if you would be affected,
you can ctrl-f the list of 75 prefixes (ARIN region only)
bellow for your prefix or ASN.

(data as of 2018-09-19 14:06 UTC)


https://bgp.he.net/net/168.245.235.0/24  (AS13428)
https://bgp.he.net/net/69.28.67.0/24  (AS13768)
https://bgp.he.net/net/69.28.82.0/23  (AS13768)
https://bgp.he.net/net/68.64.234.0/24  (AS14237)
https://bgp.he.net/net/209.24.0.0/24  (AS15562)
https://bgp.he.net/net/172.98.214.0/24  (AS17139)
https://bgp.he.net/net/66.85.44.0/24  (AS19437)
https://bgp.he.net/net/23.139.0.0/24  (AS20860)
https://bgp.he.net/net/68.64.227.0/24  (AS262913)
https://bgp.he.net/net/192.136.187.0/24  (AS26827)
https://bgp.he.net/net/208.76.33.0/24  (AS26938)
https://bgp.he.net/net/208.76.39.0/24  (AS26938)
https://bgp.he.net/net/68.64.231.0/24  (AS30167)
https://bgp.he.net/net/192.133.106.0/24  (AS33060)
https://bgp.he.net/net/192.133.107.0/24  (AS33060)
https://bgp.he.net/net/192.209.63.0/24  (AS393398)
https://bgp.he.net/net/198.28.13.0/24  (AS393451)
https://bgp.he.net/net/2606:8e80:3000::/48  (AS394308)
https://bgp.he.net/net/2606:8e80:1000::/48  (AS394497)
https://bgp.he.net/net/2606:8e80:4000::/48  (AS394497)
https://bgp.he.net/net/2606:8e80::/48  (AS394497)
https://bgp.he.net/net/2606:8e80:5000::/48  (AS394497)
https://bgp.he.net/net/2606:8e80:2000::/48  (AS394644)
https://bgp.he.net/net/2606:8e80:4000::/48  (AS394644)
https://bgp.he.net/net/104.245.239.0/24  (AS395970)
https://bgp.he.net/net/172.98.209.0/24  (AS395970)
https://bgp.he.net/net/172.98.210.0/24  (AS395970)
https://bgp.he.net/net/172.98.212.0/24  (AS395970)
https://bgp.he.net/net/172.98.214.0/24  (AS395970)
https://bgp.he.net/net/172.98.215.0/24  (AS395970)
https://bgp.he.net/net/172.98.208.0/24  (AS41139)
https://bgp.he.net/net/172.98.211.0/24  (AS41139)
https://bgp.he.net/net/172.98.213.0/24  (AS41139)
https://bgp.he.net/net/168.245.223.0/24  (AS43181)
https://bgp.he.net/net/208.84.64.0/24  (AS52129)
https://bgp.he.net/net/208.86.200.0/24  (AS52129)
https://bgp.he.net/net/199.68.168.0/22  (AS53429)
https://bgp.he.net/net/199.68.175.0/24  (AS53429)
https://bgp.he.net/net/162.208.108.0/24  (AS55079)
https://bgp.he.net/net/162.208.109.0/24  (AS55079)
https://bgp.he.net/net/162.208.110.0/24  (AS55079)
https://bgp.he.net/net/162.208.111.0/24  (AS55079)
https://bgp.he.net/net/198.176.44.0/24  (AS55079)
https://bgp.he.net/net/198.176.46.0/24  (AS55079)
https://bgp.he.net/net/198.176.47.0/24  (AS55079)
https://bgp.he.net/net/2604:a680:2::/48  (AS55079)
https://bgp.he.net/net/2604:ab80:2::/48  (AS55079)
https://bgp.he.net/net/2604:ab80:5::/48  (AS55079)
https://bgp.he.net/net/198.176.45.0/24  (AS55097)
https://bgp.he.net/net/2604:a680:4::/48  (AS55097)
https://bgp.he.net/net/208.66.204.0/24  (AS6165)
https://bgp.he.net/net/208.66.205.0/24  (AS6165)
https://bgp.he.net/net/208.66.206.0/24  (AS6165)
https://bgp.he.net/net/208.66.207.0/24  (AS6165)
https://bgp.he.net/net/74.116.232.0/24  (AS6165)
https://bgp.he.net/net/74.116.233.0/24  (AS6165)
https://bgp.he.net/net/74.116.234.0/24  (AS6165)
https://bgp.he.net/net/74.116.235.0/24  (AS6165)
https://bgp.he.net/net/74.116.236.0/24  (AS6165)
https://bgp.he.net/net/74.116.237.0/24  (AS6165)
https://bgp.he.net/net/74.116.238.0/24  (AS6165)
https://bgp.he.net/net/198.24.10.0/24  (AS62541)
https://bgp.he.net/net/198.24.11.0/24  (AS62541)
https://bgp.he.net/net/104.171.208.0/20  (AS63267)
https://bgp.he.net/net/104.171.208.0/24  (AS63267)
https://bgp.he.net/net/69.28.64.0/20  (AS6364)
https://bgp.he.net/net/69.28.80.0/23  (AS6364)
https://bgp.he.net/net/69.28.84.0/23  (AS6364)
https://bgp.he.net/net/69.28.86.0/24  (AS6364)
https://bgp.he.net/net/69.28.87.0/24  (AS6364)
https://bgp.he.net/net/69.28.88.0/23  (AS6364)
https://bgp.he.net/net/69.28.88.0/24  (AS6364)
https://bgp.he.net/net/69.28.90.0/23  (AS6364)
https://bgp.he.net/net/69.28.92.0/22  (AS6364)
https://bgp.he.net/net/172.93.121.0/24  (AS8100)
https://bgp.he.net/net/66.85.45.0/24  (AS8100)
https://bgp.he.net/net/206.53.202.0/24  (AS11492) that is probably supposed to 
be invalid (DE-CIX Dallas peering LAN? :)


[1] https://twitter.com/Jerome_UZ/status/1042433414371205120

-- 
https://twitter.com/nusenu_
https://mastodon.social/@nusenu



signature.asc
Description: OpenPGP digital signature

Re: Reaching out to ARIN members about their RPKI INVALID prefixes

[John Curran]

On 18 Sep 2018, at 1:23 PM, Owen DeLong  wrote:
> 
> Personally, since all RPKI accomplishes is providing a cryptographically 
> signed notation of origin ASNs that hijackers should prepend to their 
> announcements in order to create an aura of credibility, I think we should 
> stop throwing resources down this rathole.

Owen - 
 
Other than use of resources towards an initiative that some operators value 
(and others do not), is there any reason that notifying RPKI users about 
invalid prefixes is not a wise idea; i.e. are you aware of any technical 
downside to doing so?

/John

John Curran
President and CEO
ARIN

Re: Reaching out to ARIN members about their RPKI INVALID prefixes

[Joe Provo]

There's a lot to sift through in this thread (most of all
assertions lacking evidence), but this needs to be called 
out:

On Tue, Sep 18, 2018 at 06:21:56PM -0700, Owen DeLong wrote:
[snip]
> Point being that there are very very few ASNs using peer lock. Peer lock

Despite the cutesy neologism, filtering against the 
acceptance of big/important/private peers's ASNs from 
unplanned vectors is very common and was a standard part 
of the belt and suspenders toolkit long, long ago.*  When 
I drove 6079, I distinctly recall it coming up in 
conversations with representatives from 2828 (it might 
have been the concentric days) and others in the hallways 
at NANOG. 

Cheers,

Joe

* Barring those who never cared about forwarding quality or 
  path integrity and would say "LOL someone gives me free 
  transit to you".

-- 
Posted from my personal account - see X-Disclaimer header.
Joe Provo / Gweep / Earthling 

Re: Reaching out to ARIN members about their RPKI INVALID prefixes

[nusenu]

Phil Lavin:
> That said, having recently done this with ARIN... they've got a long
> way to go before it's a simple process (like RIPE). Submitting
> numerous tickets over a 3 day period doesn't strike me as
> particularly efficient. 

> If outreach was done and widely taken up,

I just want to reiterate that this is not about an outreach program
"Create ROAs for all your prefixes NOW", this is just about
notifying those ~30 orgs that have likely misconfigured ROAs that
result in unreachable prefixes in an ROV environment.

> I'd
> think ARIN's help desk will struggle to meet the demand. If this is
> the case and it's a multi-week process to get RPKI set up, it would
> be expected that people will give up part way through the process.
That is a great input, we certainly would not want to cause a bottleneck
at ARIN's helpdesk.

To avoid overwhelming the help desk, these ~30 organizations could
be notified one at a time (i.e. notify 5 organizations per week and be done in 
a ~month),
I assume that is a manageable amount of tickets per day.



-- 
https://twitter.com/nusenu_
https://mastodon.social/@nusenu



signature.asc
Description: OpenPGP digital signature

Re: Reaching out to ARIN members about their RPKI INVALID prefixes

[Alex Band]

> On 19 Sep 2018, at 10:37, Christopher Morrow  wrote:
> 
> 
> 
> On Wed, Sep 19, 2018 at 1:33 AM Phil Lavin  wrote:
> > What about an one-off outreach effort?
> 
>> Makes sense to me. As someone who (at least pretends to) care, I was very 
>> much unaware of RPKI before seeing discussion about it on NANOG and #ix.
>> 
>> That said, having recently done this with ARIN... they've got a long way to 
>> go before it's a simple process (like RIPE). Submitting numerous tickets 
>> over a 3 day period doesn't strike me as particularly efficient. If outreach 
>> was done and widely taken up, I'd think ARIN's help desk will struggle to 
>> meet the demand. If this is the case and it's a multi-week process to get 
>> RPKI set up, it would be expected that people will give up part way through 
>> the process.
>> 
> Phil. Thanks, this is interesting input.. I expected that the system arin 
> setup was on-par with that which ripe/apnic have setup... huh, I'm surprised 
> that it required any tickets at all to accomplish :(

ARIN offers all of the features that the other RIRs do, but usability remains a 
(big) barrier. I did a talk at NANOG several years ago demonstrating how 
usability of the hosted RPKI system greatly impacted adoption and data quality 
in the RIPE region:

https://youtu.be/R2VV_APOFL8

At the time, a lot of effort went into providing a hosted RPKI system that 
suggested ROAs based on best practices, showed what the impact on BGP 
announcements was going to be and sent alerts when misconfigurations or hijacks 
occurred. This gives operators the confidence to use and maintain the system. 
As a result, the data set is now big and high quality enough for operators to 
start dropping invalids.

I’d be interested to hear how many operators in the ARIN region would be 
willing to set up ROAs (and maintain them!) if it weren’t so hard to do. This 
might entice ARIN to address the usability issue. Because non-repudiation or 
not, this process shouldn’t have to take several tickets and several days.

Be that as it may, we fully intend to build a Delegated CA that is on par with 
RIPE’s user experience so that operators can run RPKI themselves in a usable 
way.

Alex Band
NLnet Labs

Re: Console Servers

[Mike Hammett]

Except for AT, most incumbents here aren't also mobile wireless providers, so 
that is an option in most cases for truly OOB. 




- 
Mike Hammett 
Intelligent Computing Solutions 

Midwest Internet Exchange 

The Brothers WISP 

- Original Message -

From: "Saku Ytti"  
To: "James Bensley"  
Cc: nanog@nanog.org 
Sent: Wednesday, September 19, 2018 4:04:58 AM 
Subject: Re: Console Servers 

On Wed, 19 Sep 2018 at 11:54, James Bensley  wrote: 

> I forgot to mention, it also depends how "out" of band your OOB needs 
> to be. We use Ciena 6500s for our DWDM infrastructure and they have a 
> wayside channel (like various DWDM vendors), so it's a separate 
> channel over the same physical fibre. For anything except a fibre cut 
> it seems to work. 

This is gold standard for incumbents, as they don't have anything true 
out-of-band they can consistently buy, everything travels in their 
network at some point anyhow. 

-- 
++ytti 

Re: Console Servers

[Saku Ytti]

On Wed, 19 Sep 2018 at 11:54, James Bensley  wrote:

> I forgot to mention, it also depends how "out" of band your OOB needs
> to be. We use Ciena 6500s for our DWDM infrastructure and they have a
> wayside channel (like various DWDM vendors), so it's a separate
> channel over the same physical fibre. For anything except a fibre cut
> it seems to work.

This is gold standard for incumbents, as they don't have anything true
out-of-band they can consistently buy, everything travels in their
network at some point anyhow.

-- 
  ++ytti

Re: Console Servers

[James Bensley]

On Wed, 19 Sep 2018 at 09:50, Saku Ytti  wrote:

> I think WAN indeed is very market situational, and if you need to
> support world, it is beneficial to have solution which supports many
> WAN options, without needing external boxes and external power bricks.
> We try to do just ethernet, but even that is already being provided as
> copper, fibre and in one market with PPPoE, all which are non-issues
> by going with Cisco. I do wish I had second option, I do wish JNPR SRX
> would support async serial ports.

Agreed, this is where the Cisco's shine. We can insert a mixture of
ADSL/VDSL, Ethernet and serial cards into the same box. It's a nice
all in one solution that supports all our various OOB connection types
and the console connectivity, and we could connect a IP management
switch.

I mentioned earlier that OOB inside a DC is different to in a
telephone exchange, and the OP didn't mention which was required. I
forgot to mention that a third kind of OOB is for kit that is outside.
If you need temperature hardened and DC powered OOB kit your options
dramatically shrink and it's worth considering if you're in that boat.

Cheers,
James.

Re: Console Servers

[James Bensley]

On Tue, 18 Sep 2018 at 15:26, Saku Ytti  wrote:
>
> On Tue, 18 Sep 2018 at 16:39, Alan Hannan  wrote:
>
> > Long ago I used Cisco 2511/2611 and was fairly happy.  A little later I 
> > used portmaster and was less so.  Recently I've been using Opengear and 
> > they work fairly well but the price is fairly high.   I use the CM7100 and 
> > IM7100.
>
> Out of curiosity, how do you connect them? I see quotes around
> 200USD/MRC for ethernet in US, implying 12kUSD 5 year cost on just
> connectivity, add rack rental, and power and Opengear price is maybe
> 10% of TCO?
>
> Personally I still prefer Cisco, as not to have new operating system
> to automate. Add conserver to connect persistently to each console
> port, so that you get persistent logs from console to your NMS, and so
> that you can multiplex your console sessions.
> It's hard to recover the CAPEX benefit if you need OOB platform
> specific OPEX costs.
>
> --
>   ++ytti

Hi Saku,

I forgot to mention, it also depends how "out" of band your OOB needs
to be. We use Ciena 6500s for our DWDM infrastructure and they have a
wayside channel (like various DWDM vendors), so it's a separate
channel over the same physical fibre. For anything except a fibre cut
it seems to work.

Cheers,
James.

Re: Massive Price Increase for X-conns at Telehouse Chelsea, NYC

[Scott Christopher]

Christopher Morrow wrote:

> Whether it actually 'costs' that much to pull a x-connect and maintain
> that x-connect is probably not as important as 'gosh it's really hard
> to be 'close' to  ' right? which is what they are
> capitalizing on here.> 
> Hank, how far away is the next closest large network metro ? Riyad?
> Rome? Sofia?... I mean, it's all 'far' from 'isreal' (or really any
> part of the world)  to the next decent network POP :(>  

I'm not sure if Israelis can buy anything from Riyadh, though. It's
usually the case that Israelis and their neighbors can't do business
with each other, either because of their neighbor's laws or Israel's
laws, or both.
So your Tel Aviv data center has a much smaller market and can't benefit
from economies of scale like more developed markets such as United
States and western Europe which sell globally.
I agree that capitalism lets you charge whatever you can get but healthy
capitalism gives you competition. The *big* question: If prices are so
high in Israel, why don't competitors enter this market when it's 1)
pretty much commodity and 2) booming globally?
--S.C.

Re: Console Servers

[Saku Ytti]

Hey,

> In some DCs I've done mutual OOB swaps with other telcos in the same
> suite, this is usually cheap or free (excluding the one time xconnect

We consciously decided to not ask or accept OOB swaps, because of fear
that they might be provisioned outside processes which might make it
impossible to repair them through normal commercial processes, which
would potentially cost lot of downtime and NOC's resources.

> Sometimes the DC provider has an OOB connectivity service that uses
> separate transit providers than we use and this often cheap too. Again
> this is often bespoke per DC/colo provider though.

MRC quotes I have 400USD Equinix, 288USD Terramark, 300USD Coresite.
Compared to PSTN which we see at 90-150USD. This makes me less
inclined to focus on HW CAPEX and optimise for HW/SW that tooling and
people already support.

> The most scalable solution I've been involved in so far is VDSL. Here
> in the UK lots of DCs are on-net for the national incumbent VDSL

I think WAN indeed is very market situational, and if you need to
support world, it is beneficial to have solution which supports many
WAN options, without needing external boxes and external power bricks.
We try to do just ethernet, but even that is already being provided as
copper, fibre and in one market with PPPoE, all which are non-issues
by going with Cisco. I do wish I had second option, I do wish JNPR SRX
would support async serial ports.

--
  ++ytti

Re: Reaching out to ARIN members about their RPKI INVALID prefixes

[nusenu]

Christopher Morrow wrote:
> This seems bad, at first blush, but you will not always be here to offer
> these recalcitrant folk a pointer to how to fix themselves

that is correct but I don't expect that (to be around forever) to be necessary, 
once the amount of
invalids are low, big operators could deploy ROV, and once that is the case
operators will get an immediate effect should they create incorrect ROAs,
which will cause a learning effect. 
At that point the amount of misconfigured ROAs would automatically remain low
because ROV somewhat forces proper ROAs.

>> it is about whether it is acceptable that RIRs (and more specifically ARIN
>> in this mailing list's context)
>> notify affected parties of their prefixes that suffer from stale ROAs.
>>
> 
> This I still think is a bad plan.. mostly because I don't think it'll help
> :(

If such an attempt to make people aware about their broken ROAs has no effect 
at all but I did no harm, 
than I'm fine with it because we at least tried.
I'm not sure I can follow the "lets not send these 31 emails because it is such 
a big effort and they will just
end up in the spam folder with no effect." line of reasoning.
Do you think we would be doing more harm than good by sending out these 31 
emails?


> I think what helps is: "Oh, I cant get to  and  and  internet>"  I think folk that CARE will do the right thing, folk that
> 'think they care' won't and will soon get disconnected from the tubez.
> 
> I apologize a tad if my view that: "breaking people will force them to fix
> themselves" is  rough :(

I believe it would be more polite to tell them first before you force anything 
on
them by enabling ROV, but your way of doing it would certainly be more 
efficient ;)







-- 
https://twitter.com/nusenu_
https://mastodon.social/@nusenu



signature.asc
Description: OpenPGP digital signature

Re: Console Servers

[James Bensley]

On Tue, 18 Sep 2018 at 14:38, Alan Hannan  wrote:
>
> I'd like your input on suggestions for an alternate serial port manager.
>
> Long ago I used Cisco 2511/2611 and was fairly happy.  A little later I used 
> portmaster and was less so.  Recently I've been using Opengear and they work 
> fairly well but the price is fairly high.   I use the CM7100 and IM7100.
>
> General specs I'm looking for are:
>
>  * 8 to 48 or more rs232 serial ports on rj45
>  * nice-to-have software selectable pinouts (cisco v. straight)
>  * gig-e ethernet port (100mbps ok)
>  * 1U form factor
>  * redundant AC power
>  * access physical serial connections via local port #
>  * access physical serial connections via local IP alias (nice to have)
>
> Can you recommend a serial port server/concentrator that I could use in place 
> of opengear for a better value and/or lower cost?
>
> I'm just ignorant about the current market for serial port concentrators and 
> so far web searches have not revealed ideas, so your input is appreciated!
>
> Thanks!
>
> -alan

Hi Alan,

Ah the trusty Cisco solution - yep, used the 2800 series quite a bit
for exactly this, just last year even I was deploying them. 16-32
serial connections for OOB console, an Ethernet port for an OOB IP
MGMT switch and VDSL for the WAN connection. You can also use low end
Juniper SRX devices for this (SRX200 series) and a cheap console
server, I've used SRX + OpenGear (not so cheap) console server just
fine. You just need a couple of central firewalls to terminate some
IPSEC tunnels (again, cheap SRXs have served fine as the crypto
throughput is typically low).

Some companies don't deploy anything into production which is not
vendor supported, so the 2800s wouldn't fly in that case. We used to
buy 2800s off ebay and 2nd hand tin sellers. However, for lab work
some companies are more relaxed, this is an example 2800 config that I
use for console access in the lab if you want:
https://null.53bits.co.uk/index.php?page=hwic-16a-terminal-server
I'd be reluctant to deploy Cisco 2800s (or similar) today unless there
is a newer variant, is there an ISGv2 variant with serial connectivity
that Cisco will be supporting for a few more years? I know OpenGrear
are expensive but in my current outfit, they do "just work" and the
few we had at my old place, again they did "just work".

Cheers,
James.

Re: Reaching out to ARIN members about their RPKI INVALID prefixes

[Christopher Morrow]

On Wed, Sep 19, 2018 at 1:33 AM Phil Lavin  wrote:

> > What about an one-off outreach effort?
>
> Makes sense to me. As someone who (at least pretends to) care, I was very
> much unaware of RPKI before seeing discussion about it on NANOG and #ix.
>
> That said, having recently done this with ARIN... they've got a long way
> to go before it's a simple process (like RIPE). Submitting numerous tickets
> over a 3 day period doesn't strike me as particularly efficient. If
> outreach was done and widely taken up, I'd think ARIN's help desk will
> struggle to meet the demand. If this is the case and it's a multi-week
> process to get RPKI set up, it would be expected that people will give up
> part way through the process.
>

Phil. Thanks, this is interesting input.. I expected that the system arin
setup was on-par with that which ripe/apnic have setup... huh, I'm
surprised that it required any tickets at all to accomplish :(

RE: Reaching out to ARIN members about their RPKI INVALID prefixes

[Phil Lavin]

> What about an one-off outreach effort?

Makes sense to me. As someone who (at least pretends to) care, I was very much 
unaware of RPKI before seeing discussion about it on NANOG and #ix.

That said, having recently done this with ARIN... they've got a long way to go 
before it's a simple process (like RIPE). Submitting numerous tickets over a 3 
day period doesn't strike me as particularly efficient. If outreach was done 
and widely taken up, I'd think ARIN's help desk will struggle to meet the 
demand. If this is the case and it's a multi-week process to get RPKI set up, 
it would be expected that people will give up part way through the process.

Re: Reaching out to ARIN members about their RPKI INVALID prefixes

[Christopher Morrow]

On Wed, Sep 19, 2018 at 1:19 AM Job Snijders  wrote:

> On Wed, Sep 19, 2018 at 01:07:42AM -0700, Christopher Morrow wrote:
> > > it is about whether it is acceptable that RIRs (and more
> > > specifically ARIN in this mailing list's context) notify affected
> > > parties of their prefixes that suffer from stale ROAs.
> >
> > This I still think is a bad plan.. mostly because I don't think it'll
> > help :( I think what helps is: "Oh, I cant get to  and  and
> > "  I think folk that CARE will do the right
> > thing, folk that 'think they care' won't and will soon get
> > disconnected from the tubez.
> >
> > I apologize a tad if my view that: "breaking people will force them to
> > fix themselves" is  rough :(
>
> What about an one-off outreach effort?
>
>
first I'm certainly happy about any progress on the 'RPKI DONE RIGHT'
direction, and specifically Job you as a person have made some awesome
progress here getting IXP/ISP folk to move to OV and RPKI deployments, and
adding RPKI/ROA data into the NTT IRR.

but.. I'm skeptical of distinct efforts like this.
I think something like (I think these folk still offer this service: "bgp
monitoring") BgpMon's monitoring service is what we should aim for: "A
service that RPKI users have signed up for"

Else: "ends up in spam folder" :(


> We need to somehow kickstart the feedback loop, especially if we expect
> effects to become forceful. I'm hoping that if the invalid count is low
> enough it'll become more attractive for more people flip the switch and
> deploy OV.
>
>
Sure but: "can not access a majority of the internet?" seems like a
good signal to the affected folks.



> Kind regards,
>
> Job
>

Re: Console Servers

[James Bensley]

On Tue, 18 Sep 2018 at 15:26, Saku Ytti  wrote:
>
> On Tue, 18 Sep 2018 at 16:39, Alan Hannan  wrote:
>
> > Long ago I used Cisco 2511/2611 and was fairly happy.  A little later I 
> > used portmaster and was less so.  Recently I've been using Opengear and 
> > they work fairly well but the price is fairly high.   I use the CM7100 and 
> > IM7100.
>
> Out of curiosity, how do you connect them? I see quotes around
> 200USD/MRC for ethernet in US, implying 12kUSD 5 year cost on just
> connectivity, add rack rental, and power and Opengear price is maybe
> 10% of TCO?
>
> Personally I still prefer Cisco, as not to have new operating system
> to automate. Add conserver to connect persistently to each console
> port, so that you get persistent logs from console to your NMS, and so
> that you can multiplex your console sessions.
> It's hard to recover the CAPEX benefit if you need OOB platform
> specific OPEX costs.

For cheap OOB connectivity that scales, I've had some success with
VDSL for OOB console server connections. Note that I didn't say
"great"...

In some DCs I've done mutual OOB swaps with other telcos in the same
suite, this is usually cheap or free (excluding the one time xconnect
cost, in suite xconnects often have no recurring charge) but you need
to track them all, often every swap is bespoke, providers come and go
so you need to replace them, if it's free you sometimes don't get
maintenance alerts ;)

Sometimes the DC provider has an OOB connectivity service that uses
separate transit providers than we use and this often cheap too. Again
this is often bespoke per DC/colo provider though.

The most scalable solution I've been involved in so far is VDSL. Here
in the UK lots of DCs are on-net for the national incumbent VDSL
provider (BT). It means we can have the same style of connection to
most DCs, same physical presentation, same cost, it eases the contract
management for renewing them as we have one supplier etc. The biggest
problem I've experienced with this approach is getting the copper line
to the rack, some DCs charge a small fortune as copper pairs to a rack
is a bespoke service for them, some do it regularly.

I've just moved on from an LLU provider in the UK, a CLEC in US
terminology, we had about 1200 PoPs around the UK most of which were
telephone exchanges. If you want OOB in a DC it's different to a
telephone exchange (well it is here), seeing as the OP hasn't
mentioned if OOB will be in DCs/telephone exchanges/sailing boats/etc.
I think it's worth pointing out tjat VDSL is often not available
within an exchange here and maybe it's the same in the US.

Cheers,
James.

Re: Reaching out to ARIN members about their RPKI INVALID prefixes

[Job Snijders]

On Wed, Sep 19, 2018 at 01:07:42AM -0700, Christopher Morrow wrote:
> > it is about whether it is acceptable that RIRs (and more
> > specifically ARIN in this mailing list's context) notify affected
> > parties of their prefixes that suffer from stale ROAs.
> 
> This I still think is a bad plan.. mostly because I don't think it'll
> help :( I think what helps is: "Oh, I cant get to  and  and
> "  I think folk that CARE will do the right
> thing, folk that 'think they care' won't and will soon get
> disconnected from the tubez.
> 
> I apologize a tad if my view that: "breaking people will force them to
> fix themselves" is  rough :(

What about an one-off outreach effort?

We need to somehow kickstart the feedback loop, especially if we expect
effects to become forceful. I'm hoping that if the invalid count is low
enough it'll become more attractive for more people flip the switch and
deploy OV.

Kind regards,

Job

Re: Reaching out to ARIN members about their RPKI INVALID prefixes

[Christopher Morrow]

On Wed, Sep 19, 2018 at 12:51 AM nusenu  wrote:

> Owen DeLong:
> > Personally, since all RPKI accomplishes is providing a
> > cryptographically signed notation of origin ASNs that hijackers
> > should prepend to their announcements in order to create an aura of
> > credibility, I think we should stop throwing resources down this
> > rathole.
>
> regardless of how one might think about RPKI, there are ROAs out
> there that reduce the visibility/reachability of certain prefixes and the
> general assumption is that announced prefixes would like to be reachable
> even if the operator doesn't care about RPKI and ROAs from the past
> anymore, he most likely cares
> about reachability from a pure operational point of view.
>
>
So, a lot like dnssec ... if you enable the RPKI functions (publish roas) I
think it's very much a responsibility of the publisher to provide the
correct information in an on-going and stable manner.

This seems bad, at first blush, but you will not always be here to offer
these recalcitrant folk a pointer to how to fix themselves, and TODAY
there's: "little" penalty when it comes to getting this RPKI thing
wrongly... So, ideally the folk who are 'doin it wrong' can learn, get
operational proceses/procedures/personnel in place and take action for the
long term... right? :)


> my email was not about: "How much does one like RPKI?"
>

sorry, 'most' emails that mention RPKI are: "how much do you like the
flavor of rpki?" :)


> it is about whether it is acceptable that RIRs (and more specifically ARIN
> in this mailing list's context)
> notify affected parties of their prefixes that suffer from stale ROAs.
>

This I still think is a bad plan.. mostly because I don't think it'll help
:(
I think what helps is: "Oh, I cant get to  and  and "  I think folk that CARE will do the right thing, folk that
'think they care' won't and will soon get disconnected from the tubez.

I apologize a tad if my view that: "breaking people will force them to fix
themselves" is  rough :(

Even if one dislikes RPKI entirely the opinion could still be "yes
> notifying those parties makes sense
> to restore reachability".
>
>
> --
> https://twitter.com/nusenu_
> https://mastodon.social/@nusenu
>
>

Re: Reaching out to ARIN members about their RPKI INVALID prefixes

[Christopher Morrow]

> > in which case MD5 passwords on your BGP sessions pretty much
> > accomplishes the same thing with a lot less kerfuffle.
>
>
>
oh gosh, sorry I missed this in the previous conversation... for folk
following along at home:
  TCP-MD5  is really REALLY just: "better CRC(checksum)" on your  BGP
session, and is in no way related to which routes your bgp-peer
should/could/will be sending you over that peering.

Please do not confuse/conflate BGP / TCP-MD5 with routing-security :( Steve
Bellovin would be shocked and appalled at such conflation.

Re: netflix OCA in a CG-NAT world

[Christopher Morrow]

On Tue, Sep 18, 2018 at 1:21 AM Radu-Adrian Feurdean <
na...@radu-adrian.feurdean.net> wrote:

> On Mon, Sep 17, 2018, at 17:48, Jared Mauch wrote:
>
> > I also strongly suggest you look at how to get native IPv6 from your
> > clients behind the CG-NAT rolled out.  I know many folks have had issues
>
> Getting IPv6 to your customers is good, but they still have to use it.
>
> If I look at my stats, I can see that the IPv4:IPv6 ratio for Netflix is
> 5.5:1, while for Google it's 1.1:1 and for Facebook 1.33:1

(peak-time ratios, when traffic is mostly from residential users) . The
> best explanation I could get is people may use Netflix from devices that do
> not support IPv6, such as some/most (not-so-old) Smart TVs. There's also
> the issue of some brain-dead wifi APs that filter or severely limit traffic
> required for proper IPv6 operation (multicast comes to my mind).
>
>
so, first: "Thanks for getting v6 to your customers!!"
because srsly, some folks (verizon residential dsl/fios) can't seem to make
that happen, there's some form of serious magic obviously involved...

That said, it's funny that tv's (bluray/etc) are not v6 capable?? ugh :(

-chris

Re: Reaching out to ARIN members about their RPKI INVALID prefixes

[nusenu]

Owen DeLong:
> Personally, since all RPKI accomplishes is providing a
> cryptographically signed notation of origin ASNs that hijackers
> should prepend to their announcements in order to create an aura of
> credibility, I think we should stop throwing resources down this
> rathole.

regardless of how one might think about RPKI, there are ROAs out 
there that reduce the visibility/reachability of certain prefixes and the 
general assumption is that announced prefixes would like to be reachable
even if the operator doesn't care about RPKI and ROAs from the past anymore, he 
most likely cares
about reachability from a pure operational point of view.

my email was not about: "How much does one like RPKI?"
it is about whether it is acceptable that RIRs (and more specifically ARIN in 
this mailing list's context) 
notify affected parties of their prefixes that suffer from stale ROAs.
Even if one dislikes RPKI entirely the opinion could still be "yes notifying 
those parties makes sense
to restore reachability".


-- 
https://twitter.com/nusenu_
https://mastodon.social/@nusenu



signature.asc
Description: OpenPGP digital signature

Re: Massive Price Increase for X-conns at Telehouse Chelsea, NYC

[Christopher Morrow]

On Tue, Sep 18, 2018 at 12:28 PM Owen DeLong  wrote:

>
> I’d argue that the difference between reasonable (≤$500 one-time and ≤$50
> MRC) and $300 MRC is within range of argument, but I cannot see any way in
> which an argument can be made that $5840 MRC is not a distortion in that
> same circumstance.
>
>
captive audiences (or those which view themselves as captive) are so much
fun... :(
I imagine that 'away from Telaviv' in the 20ms arena is  actually
pretty hard to find, right?
If you wanted to offer 'local' content locally (and offer it quickly/fastly
(ha!)) it's probably pretty hard in that part of the world, right? Whether
it actually 'costs' that much to pull a x-connect and maintain that
x-connect is probably not as important as 'gosh it's really hard to be
'close' to  ' right? which is what they are capitalizing on
here.

Hank, how far away is the next closest large network metro ? Riyad? Rome?
Sofia?... I mean, it's all 'far' from 'isreal' (or really any part of the
world)  to the next decent network POP :(

Re: Reaching out to ARIN members about their RPKI INVALID prefixes

[Job Snijders]

On Tue, Sep 18, 2018 at 06:18:00PM -0700, Owen DeLong wrote:
> That depends. If you ONLY allow the maintainer of NET-192.159.10.0/24
> to update the route objects for it, then the word ONLY is effectively
> present by the lack of any other route objects.

Ah, so you are now applying the RPKI Origin Validation procedure to a
non-existent flavor of IRR? :-)

Today I can create route-objects covering 192.159.10.0/24 in a number of
IRR databases and there is nothing you can do to prevent that, this
simply is not the case with RPKI. I prefer an existing system (RPKI)
over hypotheticals (Owen's IRR). 

Secondly, I've also noticed you only emphasize an adversarial angle
(origin spoofing), but there are other angles too.

The majority of today's BGP problems are attributable to operator
mistakes (misconfigurations). Analysis has shown that most BGP incidents
happen on weekdays rather than in the weekend. The number keys on our
keyboards are quite close to each other and Origin Validation is very
effective against typos. Another angle is bugs in BGP implementations:
your neighbors doing origin validation reduces the impact and
propagation of incorrect announcements from your network should you run
into a software defect.

> So RPKI is great if we can just reduce the internet diameter to 1

Agreed, in other words: RPKI is offers tangible benefits to those that
peer directly with each other, or use peerlock.

> in which case MD5 passwords on your BGP sessions pretty much
> accomplishes the same thing with a lot less kerfuffle.

Uhh... you may want to refresh your memory on what BGP is and how
TCP-MD5 works.

Kind regards,

Job

Re: Piter-IX and GOOGLE (AS15169)

[Christopher Morrow]

On Tue, Sep 18, 2018 at 11:31 PM A.T  wrote:

> Hello!
>
> Thanks for reply!
> Announcement from route server contains only 15169 in as-path.
>
>
ok, cool... Ideally the folk with peering-db access are already along
fixing the records :)
(it's totally possible they are still sleeping... but they've got a request
in their inbox to assist)

thanks for pointing out the data problem AND for asking if the peering was
real :)
-chris


> Best regards,
> A.T
>
> > On Tue, Sep 18, 2018 at 3:34 PM A.T  wrote:
> >
> >> Hello,
> >>
> >> I see AS15169 announcements from Piter-IX
> >> (https://www.peeringdb.com/ix/2149), but Google PeeringDB entry don't
> >> seem
> >> include Piter-IX.
> >> Any idea, is PeeringDB out of date here or should I be worried?
> >>
> >>
> > send me an as-path you see please?
> >
> >
> >
> >> Best regards
> >> A.T
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >
>
>
>

Re: Piter-IX and GOOGLE (AS15169)

[A.T]

Hello!

Thanks for reply!
Announcement from route server contains only 15169 in as-path.

Best regards,
A.T

> On Tue, Sep 18, 2018 at 3:34 PM A.T  wrote:
>
>> Hello,
>>
>> I see AS15169 announcements from Piter-IX
>> (https://www.peeringdb.com/ix/2149), but Google PeeringDB entry don't
>> seem
>> include Piter-IX.
>> Any idea, is PeeringDB out of date here or should I be worried?
>>
>>
> send me an as-path you see please?
>
>
>
>> Best regards
>> A.T
>>
>>
>>
>>
>>
>>
>>
>