Re: Alien Waves

[Dave Cohen]

There are some single-market/regional providers that I'm aware of currently
offering spectrum, but I believe you'll be hard pressed to find others with
national footprints in the US that will. Zayo and Lumen both did a bit of a
will they/won't they with it for a long time, and my understanding is that
neither of them currently offer it, or at least will tell you that publicly.

On Sun, May 12, 2024 at 9:48 PM Mike Hammett  wrote:

> "a limited set of providers willing to sell it, if at all."
>
> I know of one (Windstream) that offers it on a portion of their footprint.
> I swore others did, but I couldn't find them. Does anyone know who else in
> the NANOG area who does this?
>
>
>
> -
> Mike Hammett
> Intelligent Computing Solutions
> http://www.ics-il.com
>
> Midwest-IX
> http://www.midwest-ix.com
>
> - Original Message -
> From: Mark Tinka 
> To: Dave Cohen 
> Cc: nanog@nanog.org, Mike Hammett 
> Sent: Sun, 12 May 2024 17:34:19 -0500 (CDT)
> Subject: Re: Alien Waves
>
>
>
> On 5/13/24 00:11, Dave Cohen wrote:
>
> > Mark,
> >
> > Many/all of these points are fair. My experience is purely terrestrial
> and obviously both the capacity and economic calculations are vastly
> different in those situations, which I should have called out.
>
> Actually, terrestrial economics are easier to consider because you have
> the one thing the subsea applications don't have in abundance... power.
>
> Fair point, terrestrial revenues are significantly lower than subsea
> revenues on a per-bit basis, but so are the deployment costs. That evens
> out, somewhat.
>
> > However, I don’t think that the optical vendor is really the challenge -
> I would agree that, generally, spectrum is going to be available through
> larger providers that are using “traditional carrier grade” platforms - but
> rather at the service provider level. When something invariably breaks at 3
> AM and the third shift Tier I NOC tech who hasn’t read the service playbook
> says “I don’t see any errors on your transponder, sorry, it’s not on our
> end” because they’re not aware that they actually don’t have access to the
> transponder and need to start looking elsewhere, that’s the sort of thing
> that creates systemic challenges for users regardless of whether the light
> is being shot across a Ciena 6500 or a Dave’s Box-o’-Lasers 1000.
>
> I think you are contradicting yourself a bit, unless I misunderstand
> your point.
>
> Legacy vendors who have spectrum controllers have made this concern less
> of an issue. But then again, to be fair, adopting spectrum controllers
> along with bandwidth expansions via things like gridless line systems
> and C+L backbone architectures that make spectrum sales a lot more
> viable at scale do come at a hefty $$ premium. So I can understand that
> offering spectrum independent of spectrum controllers is going to be
> more trouble than it is worth.
>
> Ultimately, what I'm saying is that technologically, this is now a
> solved problem, for the most part. That said, I don't think it will be
> the majority of DWDM operators offering spectrum services en masse, for
> at least a few more years. So even if you want to procure managed
> spectrum or spectrum sharing, you are likely to come up against a
> limited set of providers willing to sell it, if at all.
>
> Mark.
>


-- 
- Dave Cohen
craetd...@gmail.com

Re: Alien Waves

[Dave Cohen]

My point was that the technology has little to do with the operational
success of the service. Spectrum controllers better enabling providers to
get out of their own way in selling spectrum did not actually enable the
providers* to get out of their own way in selling spectrum. It *should* be
easier than it used to be, but it isn't, and the problem is not really
technical, but a question of 1) not having full-throated commitment to
wanting to sell spectrum and 2) not having the talent to support it, which
is really just a function of #1.

*Speaking specifically about the very largest CLEC wavelength providers in
North America

On Sun, May 12, 2024 at 6:34 PM Mark Tinka  wrote:

>
>
> On 5/13/24 00:11, Dave Cohen wrote:
>
> Mark,
>
> Many/all of these points are fair. My experience is purely terrestrial and 
> obviously both the capacity and economic calculations are vastly different in 
> those situations, which I should have called out.
>
>
> Actually, terrestrial economics are easier to consider because you have
> the one thing the subsea applications don't have in abundance... power.
>
> Fair point, terrestrial revenues are significantly lower than subsea
> revenues on a per-bit basis, but so are the deployment costs. That evens
> out, somewhat.
>
> However, I don’t think that the optical vendor is really the challenge - I 
> would agree that, generally, spectrum is going to be available through larger 
> providers that are using “traditional carrier grade” platforms - but rather 
> at the service provider level. When something invariably breaks at 3 AM and 
> the third shift Tier I NOC tech who hasn’t read the service playbook says “I 
> don’t see any errors on your transponder, sorry, it’s not on our end” because 
> they’re not aware that they actually don’t have access to the transponder and 
> need to start looking elsewhere, that’s the sort of thing that creates 
> systemic challenges for users regardless of whether the light is being shot 
> across a Ciena 6500 or a Dave’s Box-o’-Lasers 1000.
>
>
> I think you are contradicting yourself a bit, unless I misunderstand your
> point.
>
> Legacy vendors who have spectrum controllers have made this concern less
> of an issue. But then again, to be fair, adopting spectrum controllers
> along with bandwidth expansions via things like gridless line systems and
> C+L backbone architectures that make spectrum sales a lot more viable at
> scale do come at a hefty $$ premium. So I can understand that offering
> spectrum independent of spectrum controllers is going to be more trouble
> than it is worth.
>
> Ultimately, what I'm saying is that technologically, this is now a solved
> problem, for the most part. That said, I don't think it will be the
> majority of DWDM operators offering spectrum services en masse, for at
> least a few more years. So even if you want to procure managed spectrum or
> spectrum sharing, you are likely to come up against a limited set of
> providers willing to sell it, if at all.
>
> Mark.
>


-- 
- Dave Cohen
craetd...@gmail.com

Re: Alien Waves

[Dave Cohen]

Mark,

Many/all of these points are fair. My experience is purely terrestrial and 
obviously both the capacity and economic calculations are vastly different in 
those situations, which I should have called out. 

However, I don’t think that the optical vendor is really the challenge - I 
would agree that, generally, spectrum is going to be available through larger 
providers that are using “traditional carrier grade” platforms - but rather at 
the service provider level. When something invariably breaks at 3 AM and the 
third shift Tier I NOC tech who hasn’t read the service playbook says “I don’t 
see any errors on your transponder, sorry, it’s not on our end” because they’re 
not aware that they actually don’t have access to the transponder and need to 
start looking elsewhere, that’s the sort of thing that creates systemic 
challenges for users regardless of whether the light is being shot across a 
Ciena 6500 or a Dave’s Box-o’-Lasers 1000.

Dave Cohen
craetd...@gmail.com

> On May 12, 2024, at 5:34 PM, Mark Tinka  wrote:
> 
> 
> 
>> On 5/12/24 20:35, Dave Cohen wrote:
>> It’s one of those things that makes a lot more sense on paper than in 
>> practice.
> 
> Not anymore.
> 
> The majority of SDM subsea cables built with uncompensated fibre are using 
> managed spectrum and spectrum sharing as viable business models for a 
> not-so-insignificant population of their customer base.
> 
> 
>>  I’ve found it to be operationally difficult from the perspective the 
>> provider and the user, primarily but not solely because any “co-managed” 
>> system is going to lend itself to finger pointing when issues arise.
> 
> And there is a case to be made for that concern. However, if you are in a 
> position to be able to sell spectrum, it is very likely you are going to be 
> implementing a vendor of note. Since that is most likely to be the case, 
> those vendors have spectrum controllers that make this a viable business 
> model, albeit at a $$ premium.
> 
> This is not the type of service small DWDM operators using new-age DWDM 
> vendors would typically be looking to sell. Such operators have to deal with 
> keeping the lights on, never mind esoteric services like spectrum.
> 
> 
>>  Even if it makes more commercial sense at first blush to prefer a spectrum 
>> solution over dark or traditional waves, I suspect that factoring in “labor 
>> cost wasted over unproductive troubleshooting” changes the equation a bit.
> 
> Alien wave and spectrum services attract a very high income, mainly through a 
> capex-based upfront cost (IRU) that can be attractive to the host network. At 
> those levels, providing their vendor has decent support for spectrum 
> services, the revenue gain more-than makes up for all the logistical admin.
> 
> 
>>  I also suspect that continued price compression on optical hardware will 
>> lead to fewer and fewer situations where it might make commercial sense at 
>> first blush too.
> 
> Well, legacy DWDM vendors will continue to charge a premium even for what is 
> now standard electrical bandwidth services. Why? Because they still have all 
> that legacy stuff to support, all their R to recoup, and because the bulk 
> of their customers are no longer the telco, but content folk.
> 
> New-age DWDM vendors are focused on coherent optical networks, which are 
> primarily 100G and 400G. Why? Because that is where MSA and OpenROADM are 
> currently at re: commercial availability. The legacy vendors will develop 
> proprietary coherent pluggables that will support funky things such as 800G, 
> 1.2T and 1.6T, but those won't be industry standard for some time (800G is 
> getting there, though).
> 
> What all this means is that if you are a legacy operator that is careful 
> about spending money on newer DWDM technologies, a spectrum service from a 
> larger carrier is going to be more attractive than ripping out your entire 
> line system just so you can get from 10G or 40G to 400G. Of course, if you 
> are a monopoly and have no alternatives to lean on, this doesn't count.
> 
> 
> 
>> YMMV of course and there may be reasons beyond simple commercial models 
>> where spectrum might make sense for you, but I’d urge you to only consider 
>> it doing with a provider where you’ve had a track record of operational 
>> success working with them. 
> 
> New-age DWDM vendors are not the workhorse of most of the large DWDM operator 
> networks out there. That means that any operator of note you are likely to 
> run into is going to be a Ciena, Infinera, Nokia, Adva, e.t.c., house, or 
> something along those lines. Those vendors have reasonable spectrum-based 
> solutions that smaller DWDM operators or ISP's would be willing to spend 
>

Re: Alien Waves

[Dave Cohen]

It’s one of those things that makes a lot more sense on paper than in practice. 
I’ve found it to be operationally difficult from the perspective the provider 
and the user, primarily but not solely because any “co-managed” system is going 
to lend itself to finger pointing when issues arise. Even if it makes more 
commercial sense at first blush to prefer a spectrum solution over dark or 
traditional waves, I suspect that factoring in “labor cost wasted over 
unproductive troubleshooting” changes the equation a bit. I also suspect that 
continued price compression on optical hardware will lead to fewer and fewer 
situations where it might make commercial sense at first blush too. 

YMMV of course and there may be reasons beyond simple commercial models where 
spectrum might make sense for you, but I’d urge you to only consider it doing 
with a provider where you’ve had a track record of operational success working 
with them. 

Dave Cohen
craetd...@gmail.com

> On May 12, 2024, at 2:17 PM, Mike Hammett  wrote:
> 
> What are your experiences with alien waves, managed spectrum, spectrum as a 
> service, etc?
> 
> 
> 
> -
> Mike Hammett
> Intelligent Computing Solutions
> http://www.ics-il.com
> 
> Midwest-IX
> http://www.midwest-ix.com

Re: constant FEC errors juniper mpc10e 400g

[Dave Cohen]

LAN PHY dominates in the US too. Requests for WAN PHY were almost exclusively 
for terrestrial backhaul extending off of legacy subsea systems that still 
commonly had TDM-framed services. It’s been a couple of years since I’ve been 
in optical transport directly but these requests were essentially non-existent 
after 2018 or so. OTN became somewhat more common from 2014 onward as optical 
system interop improved, but actually was more common in the enterprise space 
as providers would generally go straight to fiber in most use cases, and with 
dark fiber opex costs coming down in many markets, I see OTN requests as 
winnowing here as well. 

Dave Cohen
craetd...@gmail.com

> On Apr 20, 2024, at 7:57 AM, Mark Tinka  wrote:
> 
> 
> 
>> On 4/20/24 13:39, Saku Ytti wrote:
>> 
>> 
>>  Oh I don't think OTN or WAN-PHY have any large deployment future, the
>> cheapest option is 'good enough'...
> 
> And what we find with EU providers is that Ethernet and OTN services are 
> priced similarly. It's a software toggle on a transponder, but even then, 
> Ethernet still continues to be preferred over OTN.
> 
> Mark.

Re: "Lit" Buildings

[Dave Cohen]

I’ve had experience with a few (wireline) organizations that did this and I don’t think there is a consist answer to your question, so this is definitely a YMMV situation.The best I can summarize:- There was always some degree of obfuscation of the fiber plant even relative to what we shared with customers, particularly but not just in relation to building entrances.- Publishing a building list was always desirable but not always practical. For example, at a previous employer, our agreement with a specific key customer prohibited us from publicly publishing any building in which we served them, even if the building was multi-tenant and we served other customers, so we found it more practical to avoid publishing a building list entirely.- Some of the third party data providers, which appears to be what you’re really asking about, make it onerous to provide granular metadata. Usually the decisions as to what we provided them had more to do with the path of least resistance than specific marketing or business decisions. - No real experience with residential, but at a previous employer we had built to a not insignificant number of MDUs on behalf of other providers and generally treated those locations like any other on-net building, although I’m not clear that was the correct approach contractually. - Ask 5 people what constitutes “near net” and get 10 different answers. The best I can generalize is that “near net” constitutes buildings where the provider thinks there’s value in broadcasting that it can reach but can’t in good faith say is on-net. My current employer has a fairly precise threshold as to what we’ll include and it’s the first network I have experience with where there was actually a deliberate decision made about this. Happy to provide some more specifics off-list if appropriate. Dave Cohencraetd...@gmail.comOn Dec 7, 2023, at 5:48 PM, Mike Hammett  wrote:For those of you who list your network (usually wireline, but sometimes wireless) with third parties, are you supplying just the KMZ or lit buildings as well? If lit buildings, are you including residential? How are you defining near-net?-Mike HammettIntelligent Computing SolutionsMidwest Internet ExchangeThe Brothers WISP

Re: [NNagain] transit and peering costs projections

[Dave Cohen]

I’m a couple years removed from dealing with this on the provider side but the 
focus has shifted rapidly to adding core capacity and large capacity ports to 
the extent that smaller capacity ports like 1 Gbps aren’t going to see much 
more price compression. Cost per bit will come down at higher tiers but there 
simply isn’t enough focus at lower levels at the hardware providers to afford 
carriers more price compression at 1 Gbps, even 10 Gbps. I would expect further 
price compression in access costs but not really in transit costs below 10 
Gbps. 

In general I agree that IXs continue to proliferate relative to quantity, 
throughput and geographic reach, almost to the degree that mainland Europe has 
been covered for years. In my home market of Atlanta, I’m aware of at least 
four IXs that have been established here or entered the market in the last 
three years - there were only two major ones prior to that. This is a net 
positive for a wide variety of reasons but I don’t think it’s created much of 
an impact in terms of pulling down transit prices. There are a few reasons for 
this, but primarily because that growth hasn’t really displaced transit demand 
(at least in my view) and has really been more about a relatively stable set of 
IX participants creating more resiliency and driving other performance 
improvements in that leg of the peering ecosystem. 

Dave Cohen
craetd...@gmail.com

> On Oct 14, 2023, at 7:02 PM, Dave Taht via Nnagain 
>  wrote:
> 
> This set of trendlines was very interesting. Unfortunately the data
> stops in 2015. Does anyone have more recent data?
> 
> https://drpeering.net/white-papers/Internet-Transit-Pricing-Historical-And-Projected.php
> 
> I believe a gbit circuit that an ISP can resell still runs at about
> $900 - $1.4k (?) in the usa? How about elsewhere?
> 
> ...
> 
> I am under the impression that many IXPs remain very successful,
> states without them suffer, and I also find the concept of doing micro
> IXPs at the city level, appealing, and now achievable with cheap gear.
> Finer grained cross connects between telco and ISP and IXP would lower
> latencies across town quite hugely...
> 
> PS I hear ARIN is planning on dropping the price for, and bundling 3
> BGP AS numbers at a time, as of the end of this year, also.
> 
> 
> 
> -- 
> Oct 30: https://netdevconf.info/0x17/news/the-maestro-and-the-music-bof.html
> Dave Täht CSO, LibreQos
> ___
> Nnagain mailing list
> nnag...@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/nnagain

Re: Low to Mid Range DWDM Platforms

[Dave Cohen]

My experience is primarily with the traditional carrier-grade folks like
Ciena, Infinera, etc. but over the last decade all of those vendors have
focused on improving how they scale down without sacrificing (most of the)
quality and functionality - to varying degrees of success. There are also
some more recent entrants that built their products to target the DCI
market but rather than focusing on bandwidth density have focused on cost
per bit for a mid-range solution. There are almost definitely multiple
quality options out there without having to buy the full 88 channel
n-degree ROADM Ciena 6500 that takes up a full rack - although given the
stated requirements, there may not be a one-size-fits-all solution that's
ideal for all of the OP's projects.

On Fri, Oct 6, 2023 at 9:43 AM Mark Tinka  wrote:

>
>
> On 10/6/23 15:07, Mike Hammett wrote:
>
> > I've been using various forms of passive WDM for years. I have a couple
> different projects on my plate that require me to look at the next level of
> platform.
> >
> > In some projects, I'll be looking for needing to have someone long
> distances of glass without any electronics. Some spans could be over 60
> miles.
> >
> > In some projects, I'll need to transport multiple 100-gig waves.
> >
> > What is the landscape like between basic passive and something like a 30
> terabit Ciena? I know of multiple vendors in that space, but I like to
> learn more about what features I need and what features I don't need from
> somewhere other than the vendor's mouth. Obviously, the most reliability at
> the least cost as well.
>
> 400G-ZR pluggables will get you 400Gbps on a p2p dark fibre over 80km -
> 100km. So your main cost there will be routers that will support.
>
> The smallest DCI solution from the leading DWDM vendors is likely to be
> your cheapest option. Alternatively, if you are willing to look at the
> open market, you can find gear based on older CMOS (40nm, for example),
> which will now be EoL for any large scale optical network, but cost next
> to nothing for a start-up with considerable capacity value.
>
> There is a DWDM vendor that showed up on the scene back in 2008 or
> thereabouts. They were selling a very cheap, 1U box that had a different
> approach to DWDM from other vendors at the time. I, for the life of me,
> cannot remember their name - but I do know that Randy introduced them to
> me back then. Maybe he can remember :-). Not sure if they are still in
> business.
>
> Mark.
>
>
>

-- 
- Dave Cohen
craetd...@gmail.com
@dCoSays
www.venicesunlight.com

Re: Zayo woes

[Dave Cohen]

To that point, the new management team seems really interested in the managed services game (the QoS acquisition also falls into this vain) and I wonder the extent to which that whipsawing of attention is already piling on to what was already a lackluster customer service organization before they took over. For what it’s worth, in my experience, Zayo historically integrated extremely quickly. With the smaller acquisitions the challenges with this tended to be data integrity related and not technical. With a couple of the bigger acquisitions it was all of the above. There were absolutely times where they bit off something different than what they expected to be chewing, but in the scheme of telco integrations we’ve lived through, they were far from the worst offenders, in my experience. My belief is that the challenges customers have experienced there have less to do with integration and acquisition pains and more to do with stewardship of the customer-interfacing aspects of the business, even moreso now that non-fiber folks are at the helmYMMV of course. Dave Cohencraetd...@gmail.comOn Sep 19, 2023, at 9:12 PM, Mike Hammett  wrote:The other way around. Zayo acquired ENA.That acquisition seemed odd. ENA did a lot of value add and non-IP services. Zayo seems to shed those on every acquisition.-Mike HammettIntelligent Computing SolutionsMidwest Internet ExchangeThe Brothers WISPFrom: "Bill Murphy" To: "Mike Hammett" , "Randy Carpenter" Cc: "nanog" Sent: Tuesday, September 19, 2023 8:53:13 AMSubject: Re: Zayo woes






Recently reached out to Zayo and found out we have a new account manager, and also discovered they were acquired by a company called ENA...




Bill



From: NANOG  on behalf of Mike Hammett 
Sent: Tuesday, September 19, 2023 7:19 AM
To: Randy Carpenter 
Cc: nanog 
Subject: Re: Zayo woes
 







External:
Increase caution when handling links and attachments.







I've never understood companies that acquire and don't completely integrate as quickly as they can.



-
Mike
 Hammett
Intelligent
 Computing Solutions

Midwest
 Internet Exchange

The
 Brothers WISP




From: "Randy Carpenter" 
To: "JASON BOTHE" 
Cc: "nanog" 
Sent: Monday, September 18, 2023 7:01:03 PM
Subject: Re: Zayo woes


The problem we have run into is that there does not appear to be a "Zayo." There are dozens of acquisitions of regional providers with completely different infrastructure and teams and they have done a very poor job at gluing it all together. I have seen service
 orders that have gone *years* without being complete. There are also currently some breaking-the-entire-regional-network sorts of outages going on currently. I am guessing what clued employees they still have are quite tied up.


-Randy


- On Sep 18, 2023, at 7:06 PM, JASON BOTHE via NANOG nanog@nanog.org wrote:

> Does anyone know what’s happening at Zayo? Tickets are taking weeks and months
> to get resolved, much less get a tech assigned to them.
> 
> The folks answering the noc phone are mere order takers and are only reading
> from a script, manager on duty/escalation lines go to voicemail.
> 
> If anyone can help get to a human in the transport group, that would be great.
> I’ve given up all hope at this point.
> 
> Appreciated.
> 
> Jason

Re: Lossy cogent p2p experiences?

[Dave Cohen]

At a previous $dayjob at a Tier 1, we would only support LAG for a customer
L2/3 service if the ports were on the same card. The response we gave if
customers pushed back was "we don't consider LAG a form of circuit
protection, so we're not going to consider physical resiliency in the
design", which was true, because we didn't, but it was beside the point.
The real reason was that getting our switching/routing platform to actually
run traffic symmetrically across a LAG, which most end users considered
expected behavior in a LAG, required a reconfiguration of the default hash,
which effectively meant that [switching/routing vendor]'s TAC wouldn't help
when something invariably went wrong. So it wasn't that it wouldn't work
(my recollection at least is that everything ran fine in lab environments)
but we didn't trust the hardware vendor support.

On Sat, Sep 9, 2023 at 3:36 PM Mark Tinka  wrote:

>
>
> On 9/9/23 20:44, Randy Bush wrote:
>
> > i am going to be foolish and comment, as i have not seen this raised
> >
> > if i am running a lag, i can not resist adding a bit of resilience by
> > having it spread across line cards.
> >
> > surprise!  line cards from vendor  do not have uniform hashing
> > or rotating algorithms.
>
> We spread all our LAG's across multiple line cards wherever possible
> (wherever possible = chassis-based hardware).
>
> I am not intimately aware of any hashing concerns for LAG's that
> traverse multiple line cards in the same chassis.
>
> Mark.
>


-- 
- Dave Cohen
craetd...@gmail.com
@dCoSays
www.venicesunlight.com

Re: MX204 Virtual Chassis Setup

[Dave Cohen]

Similarly, the carrier that employed me when 40G debuted did in fact offer
40G, but did its best to steer customers clear of it. By the time 40G
client optics were available to us from our optical vendors, those same
vendors were already making it clear that we were going to see a lot more
efficiency, both spectrally and economically, with 100G. We took that to
mean that 40G was going to be a stop gap and not much more than that. So
for those ~9 months from when 40G was made available to us until 100G was
ready for market, we were happy to sell 40G to anyone who asked; after
that, not so much (but we would, and did, until the demand essentially
completely disappeared). I imagine most/all of the large carriers were
getting the same messaging. By the time the L2/3 vendors were market ready
with 40G and 100G shortly thereafter, the die had already been cast, even
if none of those vendors saw 40G the same way the optical vendors did.

On Mon, Aug 28, 2023 at 10:52 AM Tom Beecher  wrote:

> I would agree with that. We've had gear with 40-gig ports for many years
>> (>6)? Never found a CDN or transport network that would do 40.
>
>
> Many 40G hardware options never made a ton of economic sense in CDN land
> with shared ASIC lanes for 40G and 100G ports. Using anything 40G blocked
> the associated 100G port, which were more valuable overall. You also didn't
> want to create a massive shuffle later, so it made much more sense to just
> use the 100Gs. You gained flexibility in initial deployment at the cost of
> inflexibility down the road.
>
> Newer stuff that has a dedicated 100G per port, but can run at either
> speed, might actually help 40G deployment since it's just an optic swap.
> But 100G optic costs have come down enough I think most people are just
> going to go there.
>
>
>
> On Mon, Aug 28, 2023 at 8:20 AM Mike Hammett  wrote:
>
>> I would agree with that. We've had gear with 40-gig ports for many years
>> (>6)? Never found a CDN or transport network that would do 40.
>>
>>
>>
>> -
>> Mike Hammett
>> Intelligent Computing Solutions <http://www.ics-il.com/>
>> <https://www.facebook.com/ICSIL>
>> <https://plus.google.com/+IntelligentComputingSolutionsDeKalb>
>> <https://www.linkedin.com/company/intelligent-computing-solutions>
>> <https://twitter.com/ICSIL>
>> Midwest Internet Exchange <http://www.midwest-ix.com/>
>> <https://www.facebook.com/mdwestix>
>> <https://www.linkedin.com/company/midwest-internet-exchange>
>> <https://twitter.com/mdwestix>
>> The Brothers WISP <http://www.thebrotherswisp.com/>
>> <https://www.facebook.com/thebrotherswisp>
>> <https://www.youtube.com/channel/UCXSdfxQv7SpoRQYNyLwntZg>
>> --
>> *From: *"Mark Tinka" 
>> *To: *"Mike Hammett" 
>> *Cc: *nanog@nanog.org
>> *Sent: *Sunday, August 27, 2023 10:33:07 PM
>> *Subject: *Re: MX204 Virtual Chassis Setup
>>
>>
>>
>> On 8/28/23 03:05, Mike Hammett wrote:
>>
>> Well, or they simply found a potential deal on hardware that came with 40
>> gig ports. 40 gigs is still a lot of bits to a lot of people.
>>
>>
>> For internal use, sure.
>>
>> But when connecting to another AS, the chances of them supporting 40Gbps
>> in one or more places is inconsistent to slim.
>>
>> Exchange points may be an exception.
>>
>> Mark.
>>
>>

-- 
- Dave Cohen
craetd...@gmail.com

Re: Flapping Transport

[Dave Cohen]

At a previous $dayjob, we employed a guy that was a bona fide optical guru.
He had effectively memorized the 400+ page Nortel 6500 operating guide, and
some of the hardware vendors would call him for advice when their TACs
couldn't figure a problem out. Allegedly, he was the person who discovered
that the early generations of OTU-4 line deployments were susceptible to
problems across cable in OPGW space because of the Faraday Effect. On the
rare occasion when he couldn't diagnose a problem he'd respond with
something like "voodoo doesn't always work".

To your question, it isn't acceptable but it is likely pretty normal.
Flapping isn't often a particularly straightforward issue to diagnose
and/or resolve in optical networks (especially ones where there's regen or
in-line amplification), and most transport providers don't employ guys like
that that can figure it out. And even then, voodoo doesn't always work.

Your hope is that whatever the "card issue" was was a localized event
rather than something that's now systemic, and while I don't really
understand why they wouldn't take a maintenance window to replace the cards
anyway (aside from being cheap, which is almost definitely the reason), if
they're not seeing continued issues (and of course you'd have to trust them
that they're not), it's equally likely as not that the problem has in fact
resolved.

On Tue, Aug 1, 2023 at 2:21 PM Mike Hammett  wrote:

> I have a wave transport vendor that suffered issues twice about ten days
> apart, causing my link to flap a bunch. I put in a ticket on the second set
> of occurrences. I was told that there was a card issue identified and would
> be notified when the replacement happened. Ticket closed.
>
> Three weeks later, I opened a new ticket asking for the status. The new
> card arrived the next day, but since no more flaps were happening, the card
> would not be replaced. Ticket closed.
>
>
> A) It doesn't seem like they actually did anything to fix the circuit.
> B) They admitted a problem and sent a new card.
> C) They later decided to not do anything.
>
>
> Is that normal?
> Is that acceptable?
>
>
> To avoid issues flapping causes, I disabled that circuit until repaired,
> but it seems like they're not going to do anything and I only know that
> because I asked.
>
>
>
> -
> Mike Hammett
> Intelligent Computing Solutions <http://www.ics-il.com/>
> <https://www.facebook.com/ICSIL>
> <https://plus.google.com/+IntelligentComputingSolutionsDeKalb>
> <https://www.linkedin.com/company/intelligent-computing-solutions>
> <https://twitter.com/ICSIL>
> Midwest Internet Exchange <http://www.midwest-ix.com/>
> <https://www.facebook.com/mdwestix>
> <https://www.linkedin.com/company/midwest-internet-exchange>
> <https://twitter.com/mdwestix>
> The Brothers WISP <http://www.thebrotherswisp.com/>
> <https://www.facebook.com/thebrotherswisp>
> <https://www.youtube.com/channel/UCXSdfxQv7SpoRQYNyLwntZg>
>


-- 
- Dave Cohen
craetd...@gmail.com

Re: Long hops on international paths

[Dave Cohen]

I guess it depends what you’re considering a “very few” number of routers but 
this seems to be an expected outcome. While there are a large number of wet 
cable landing stations, they are highly concentrated near a small number of 
metro areas, and with the exception of capacity owned by the ILECs, the 
supermajority of routers terminating that capacity in the US are going to live 
in fewer than ten discrete carrier hotel locations. (It’s worth noting that 
terrestrial capacity coming in from Mexico and Canada also terminates in a 
small number of locations, although the overlap between the two lists is fairly 
small). In addition, while the links likely terminate in multiple devices at a 
given location, carriers more likely to undersubscribe transoceanic core 
capacity than other areas of the core, which means that for many carriers it’s 
unlikely you’d see multiple paths show up in a trace unless you catch it during 
an outage situation. That said, seeing transoceanic links terminate in Chicago 
is likely an artifact of hops missing in a trace; although I am familiar with a 
couple of more niche providers that extend transoceanic capacity into 
non-coastal markets on optical gear in order to meet specific performance 
needs, this is unlikely to be seen in the network of a Tier 1 or similarly 
scaled network. 

Dave Cohen
craetd...@gmail.com

> On Jan 17, 2022, at 6:15 PM, Christopher Morrow  
> wrote:
> 
> 
> 
> 
>> On Mon, Jan 17, 2022 at 5:31 PM PAUL R BARFORD  wrote:
>> Dear Pengxiong,
>> 
>> Thanks for your questions:
>> 
>> We are using CAIDA’s Internet Topology Data Kit (ITDK) that uses the MIDAR 
>> alias resolution method to infer IP addresses assigned to the same router.
>> We understand the concerns about IP geolocation.  Interfaces of the router 
>> in question are assigned similar domain names e.g., 
>> “chi-b2-link.ip.twelve99.net” (62.115.50.61). We also used CAIDA’s ITDK, 
>> which provides geolocation information, and indicates that this router is 
>> located in Chicago.  We cross-reference with Maxmind where possible.  In 
>> this particular case, there is the telltale in the use of "chi" in the 
>> domain name. 
>> 
> 
> I think nick's point about ttl expiry and missing some context on topology 
> still stands.
> I'd be that the paths between 2 continents do not actually land in chicago... 
> that you're seeing (or not seeing) missing hops between the coast(s) and 
> chicago inside 1299's network in the US.
>  
>> Hope that helps.
>> 
>> Regards, PB
>> From: Pengxiong Zhu 
>> Sent: Monday, January 17, 2022 3:23 PM
>> To: PAUL R BARFORD 
>> Cc: nanog@nanog.org 
>> Subject: Re: Long hops on international paths
>>  
>> Hi Paul,
>> 
>> Just curious. How do you determine they are the same routers? Is it based on 
>> IP address or MAC addresses? Or using CAIDA’s router alias database?
>> 
>> Also how do you draw the conclusion that the AS1299 router is indeed in 
>> Chicago? IP-geolocation based on rDNS is not always accurate though. 
>> 
>> 
>> Pengxiong 
>> 
>> On Mon, Jan 17, 2022 at 10:03 AM PAUL R BARFORD  wrote:
>> Hello,
>> 
>> I am a researcher at the University of Wisconsin.  My colleagues at 
>> Northwestern University and I are studying international Internet 
>> connectivity and would appreciate your perspective on a recent finding.
>> 
>> We're using traceroute data from CAIDA's Ark project for our work.  We've 
>> observed that many international links (i.e., a single hop on an end-to-end 
>> path that connects two countries where end points on the hop are identified 
>> via rDNS) tend to originate/terminate at the same routers.  Said another 
>> way, we are observing a relatively small set of routers in different 
>> countries tend to have a majority of the international connections - this is 
>> especially the case for hops that terminate in the US.  For example, there 
>> is a router operated by Telia (AS1299) in Chicago that has a high 
>> concentration of such links.  We were a bit surprised by this finding since 
>> even though it makes sense that the set of providers is relatively small 
>> (i.e., those that offer global connectivity), we assumed that the set of 
>> routers that used for international connectivity within any one country 
>> would tend to be more widely distributed (at least with respect to how they 
>> appear in traceroute data - MPLS notwithstanding).
>> 
>> We're interested in whether or not this is indeed standard practice and if 
>> so, the cost/benefit for configuring international connectivity in this way?
>> 
>> Any thoughts or insights you might have would be greatly appreciated - 
>> off-list responses are welcome.
>> 
>> Thank you.
>> 
>> Regards, PB
>> 
>> Paul Barford
>> University of Wisconsin - Madison
>> 
>> -- 
>> 
>> Regards,
>> Pengxiong Zhu
>> Department of Computer Science and Engineering
>> University of California, Riverside

Re: Identifying submarine links via traceroute

[Dave Cohen]

As Mark says YMMV as different providers will have markedly different
conventions, however one additional challenge that will be widespread is
that most carriers are not placing their L2/3 hardware in the cable landing
stations, preferring instead to extend from the CLS to more centralized POP
locations via Layer 1. So what you will see between a city pair like
Tokyo-Seattle, which very obviously will require some wet capacity, will
actually be some combination of wet and terrestrial. Between the
terrestrial extensions and L2/3 overhead it would be difficult to determine
exactly what the underlying cable(s) are even if you had a good idea of
what the CLS to CLS latency was.

At a previous $dayjob, for example, we had both 100% terrestrial and
partially wet links in use to connect our core POPs in Seattle and
Vancouver directly. While at the Layer 1 level, the wet link had about a
20% longer optical distance, the distance was short enough that a trace
would generally return 3 or 4 ms between core nodes pretty much
irrespective of the situation (and the trunks terminated into the same
routers in the core anyway, which is a whole other story), so it would have
been impossible to tell which path was used even though I knew exactly what
the backbone architecture looked like.

Again, YMMV as different providers will have different standards and
different city pairs will be easier to determine than others, but there is
no "use this one weird trick" rule here.

On Wed, Sep 29, 2021 at 8:50 AM Mark Tinka  wrote:

>
>
> On 9/29/21 04:23, PAUL R BARFORD wrote:
>
> Hello,
>
> I am a researcher at the University of Wisconsin.  My colleagues at
> Northwestern University and I are studying submarine cable infrastructure.
>
> Our interest is in identifying submarine links in traceroute
> measurements.  Specifically, for a given end-to-end traceroute measurement,
> we would like to be able to identify when two hops are separated by a
> submarine cable.  Our initial focus has been on inter-hop latency, which
> can expose long links.  The challenge is that terrestrial long-haul links
> may have the same or longer link latencies as short submarine links. So,
> we're interested in whether there may be other features (e.g., persistent
> congestion, naming conventions in router interfaces, peering details, etc.)
> or techniques that would indicate submarine links.
>
> Any thoughts or insights you might have would be greatly appreciated -
> off-list responses are welcome.
>
>
> Back in the day, when submarine cables were not as rife, it was not
> uncommon to see things like "FLAG" or "APCN-2" or "SMW-3" in traceroutes. I
> haven't seen such in a very long time, but likely some operators may still
> do this.
>
> For traceroutes that cross oceans visibly, e.g., lhr-jfk, mrs-mba,
> hnd-lax, mru - cdg, e.t.c., you could glean from there. But many operators
> do not follow any "common norm" to annotate things like this, so YMMV.
>
> You also find some countries that will use a submarine festoon either as a
> primary or backup route for a terrestrial link. In such cases, the
> distances may be the same, or even shorter across the festoon, e.g.,
> consider a festoon cable between Cape Town - Durban, vs. a land-based run
> for the same two points.
>
> Considering how wide-spread submarine links are for both short and long
> spans, I think folk are simply treating them as any other link, from an
> operational perspective. You may be able to come up with a semi-automatic
> mechanism to measure this, but I fear without deliberate and consistent
> human intervention, the data could get stale very quickly.
>
> Mark.
>


-- 
- Dave Cohen
craetd...@gmail.com
@dCoSays
www.venicesunlight.com

Re: Are the days of the showpiece NOC office display gone forever?

[Dave Cohen]

Frankly, I think the days of the "showpiece NOC" being relevant ended a
while before COVID. I worked at a Tier 1 for >10 years in a customer-facing
capacity, largely dealing with "serious" enterprise customers. The number
of customers who toured the NOC in that time was less than 5 (i.e much less
than 1%), despite generally offering it up to new and quickly-scaling
customers. I actually spent more time visiting customer NOCs over that time
than I did my own with customers (and some of theirs were more impressive
anyway).

Now, I did spend a lot of time coordinating calls between the NOC and
customers, some of which was in the break-fix, mea culpa sort of vain, but
also a lot of "spend time getting comfortable" types of conversations too,
especially with customers considering their first service. So it wasn't
that folks didn't care about what was going on there inasmuch as they
recognized (quite rightfully IMO) that they weren't going to get any value
being there that they couldn't get meeting the folks and learning about
operational procedures, etc. over a conference call, so why waste the
time/money to travel for that sort of thing?

I don't know if I have a biased sample or if this is reflective of the norm
these days, but the "NOC Tour" was something that didn't get executed on
very often.

On Wed, Dec 16, 2020 at 3:51 PM Eric Kuhnke  wrote:

> With the covid19 situation, obviously lots of ISPs have their NOC
> personnel working from home, with VPN (or remote desktop) access to all the
> internal tools, VoIP at home, etc.
>
> In the traditional sense, by "showpiece NOC" I mean a room designed for
> the purpose of having large situational awareness displays on a wall,
> network weathermaps and charts, alerting systems, composed of four or more
> big flat panel displays. Ideally configured to be actually useful for NOC
> purposes and also something impressive looking for customer tours.
>
> To what extent potential customers find that sort of thing to be a
> signifier of seriousness on the part of an ISP, I suppose depends on what
> sort of customers they are, and their relative degree of technical
> sophistication.
>
> Are the days of such an environment gone forever?
>
>
>
>

-- 
- Dave Cohen
craetd...@gmail.com
@dCoSays
www.venicesunlight.com

Re: Passive Wave Primer

[Dave Cohen]

From the perspective of a large carrier, spectrum is an operational nightmare. 
At a former $dayjob it was an “offering” in the sense that we had deployed it, 
told customers we offered it but wouldn’t actually deploy it anymore. 

Logistically there are a lot of potential points of failure once you got beyond 
the distance threshold where mid-line amps would be required, and as alluded to 
here, no big carrier would want to take on risk to the network that they’re not 
in control of. There’s not much sense in doing it in shorter-distance scenarios 
when most folks needing enough bandwidth to even have the conversation are 
going to be able to run their own optical systems across dark fiber at that 
distance anyway. The customers that we talked to about it were almost 
exclusively other carriers that wanted to use the muxes they had in inventory 
(aka not the same platform as the photonic layer) without the burden of 
deploying/managing a lot of amps but had no issue taking OTN or even LAN PHY in 
bulk when push came to shove. 

FWIW, and I understand these terms have become fungible over time, the scenario 
above is spectrum from the perspective of the photonic layer owner and alien 
wave from the perspective of the customer. As the photonic layer owner, alien 
wave would generally be thought of as 1) accepting a handoff as a WDM-specific 
channel of light, whether on fixed or tunable optics, not a standard 1310/1550, 
and, typically but not necessarily, 2) accepting a signal that is framed as 
OTN, not LAN PHY or WAN PHY.  

For the record, the OP’s query about passive wave would suggest PON/GPON or 
similar low-power CWDM for short-haul use, and not spectrum or alien wave, both 
of which are decidedly non-passive. 

Dave Cohen
craetd...@gmail.com

> On Oct 13, 2020, at 4:24 PM, Brandon Martin  wrote:
> 
> On 10/13/20 4:01 PM, Mike Hammett wrote:
>> It seems incredibly simple to do, depending on the capabilities of your 
>> platform.
>> What am I missing?
> 
> If the span between the mux/demux pair is entirely passive, it's fairly 
> straightforward.  That's going to limit distances to around 80km or so with 
> conventional systems or maybe 120km with systems designed entirely around 
> modern coherent optics.
> 
> If there are photonic devices in the span, you now have customer-supplied 
> light being part of the rainbow that those photonics have to handle.  
> Balancing things at amplifiers requires careful coordination with the 
> customer (or adding a separate managed/monitored VOA for each alien wave 
> which somewhat defeats the point).  You end up with a scenario where a 
> customer can do something screwy and potentially affect other waves on 
> potentially multiple spans which your big-name carriers are obviously 
> completely freaked out by.
> 
> It's obviously possible, but the operational headache seems large enough that 
> the major mid-haul and long-haul carriers I've talked to (all North America 
> and all midwest, for that matter), don't seem to want to sell it despite all 
> the major optical transport platform vendors not just supporting it by 
> heavily pushing it.
> 
> I really do hope it becomes a real product that I (as a smaller, local island 
> operator) can buy, but it just doesn't seem to be there yet at least in my 
> region.
> -- 
> Brandon Martin

Re: 60 ms cross-continent

[Dave Cohen]

Doing some rough back of the napkin math, an ultra low-latency path from, say, 
the Westin to 1275 K in Seattle will be in the 59 ms range. This is 
considerably longer than the I-90 driving distance would suggest because:
- Best case optical distance is more like 5500 km, in part because the path 
actually will go Chicago-NJ-WDC and in part because a distance of 5000 km by 
right-of-way will be more like 5500 km when you account for things like 
maintenance coils, in-building wiring, etc.
- You’ll need (at least) three OEO regens on that distance, since there’s no 
value in spending 5x to deploy an optical system that wouldn’t need to (like 
the ones that would manage that distance subsea). This is in addition to ~60 
in-line amplification nodes, although that adds significantly less latency even 
in aggregate

Some of that is simply due to cost savings. In theory, you could probably spend 
a boatload of money to build a route that cuts off some of the distance 
inefficiency and gets you closer to 4500 km optical distance with minimal slack 
coil, and maybe no regens, so you get a real-world performance of 46 ms. But 
there are no algo trading sites of importance in DC, and for everybody else 
there’s not enough money in the difference between 46 and 59 ms for someone to 
go invest in that type of deployment. 

Dave Cohen
craetd...@gmail.com

> On Jun 20, 2020, at 12:44 PM, Tim Durack  wrote:
> 
> 
> And of course in your more realistic example:
> 
> 2742 miles = 4412 km ~ 44 ms optical rtt with no OEO in the path
> 
>> On Sat, Jun 20, 2020 at 12:36 PM Tim Durack  wrote:
>> Speed of light in glass ~200 km/s
>> 
>> 100 km rtt = 1ms
>> 
>> Coast-to-coast ~6000 km ~60ms
>> 
>> Tim:>
>> 
>>> On Sat, Jun 20, 2020 at 12:27 PM William Herrin  wrote:
>>> Howdy,
>>> 
>>> Why is latency between the east and west coasts so bad? Speed of light
>>> accounts for about 15ms each direction for a 30ms round trip. Where
>>> does the other 30ms come from and why haven't we gotten rid of it?
>>> 
>>> c = 186,282 miles/second
>>> 2742 miles from Seattle to Washington DC mainly driving I-90
>>> 
>>> 2742/186282 ~= 0.015 seconds
>>> 
>>> Thanks,
>>> Bill Herrin
>>> 
>>> -- 
>>> William Herrin
>>> b...@herrin.us
>>> https://bill.herrin.us/
>> 
>> 
>> -- 
>> Tim:>
> 
> 
> -- 
> Tim:>

Re: Outsourced NOC Solutions

[Dave Cohen]

There are two different approaches to deploying the fiber monitoring hardware 
that is effectively in line with the “get your gear off of my fiber” argument. 
If the monitoring service is intended for a specific customer, the signal will 
traverse a customer-specific pair. This is pretty rare though, for a wide 
variety of reasons that have been mostly mentioned here. 

The way this generally works then is that the provider reserves a strand or 
pair on the cable for monitoring purposes and uses the characterization data to 
make assumptions about the whole cable. This is pretty effective for “track 
down the precise location of a cut” or “why did this span just go from -18 to 
-30 for no apparent reason” and not necessarily the full gamut of 
characterization issues that can come up on other pairs of glass, which is 
still enough to meaningfully impact the part of MTTR that is under a provider’s 
control. For many of you consuming a dark fiber service today, this is the 
approach being used, so there’s no provider hardware touching your glass and 
certainly no lambda for your gear to contend with avoiding. 

Dave Cohen
craetd...@gmail.com

> On Jun 8, 2020, at 10:55 PM, Tom Beecher  wrote:
> 
> 
> United Cable Company is primarily a broker. 
> 
> To Rod's questions :
> 
> Sure, you can light a pair and monitor it many different ways. However, as 
> James has said already, most people who want dark fiber are going to want one 
> pair of glass from A to Z with nothing in the middle at all that they don't 
> know about. For me, I would want to know exactly what you had in place ( full 
> specifications , not hand waved 'monitoring device' ) , what wavelengths it 
> used, how it functioned (fully passive, etc), along with some extensive tests 
> to make sure I could do what I expected to without any interference or 
> surprises, before I would come near a contract with you. From my point of 
> view, any device on the glass I am leasing is essentially now part of my 
> network, so I need to know everything about it. Others may have different 
> standards of course, but that's perfectly fine. 
> 
> I would say personally though that if during due diligence, your NOC was 
> nothing more than an answering service to someone else, which it kinda sounds 
> like you want, I would personally not do business with that. Again others may 
> have different standards, and that's ok. 
> 
> 
> 
>> On Mon, Jun 8, 2020 at 7:54 PM Miles Fidelman  
>> wrote:
>> Rod Beck rod.beck at unitedcablecompany.com wrote
>> 
>>> I would calm down, Miles.  Dark fiber networks are built and usually 
>>> maintained by the same construction company that installed them. And a dark 
>>> fiber network does not even need a single full time optical engineer. If 
>>> the cable is damaged, then the guys who installed it will repair it. All 
>>> the expertise is there.
>>> 
>>> And no, I am not an executive at a undersea cable system. i was one of 
>>> Hibernia Atlantic's top salesmen during the early years from 2004-2011 
>>> after which I retired.
>>> 
>> Funny thing then, given that you signed your original query as:
>>> Roderick Beck
>>> VP of Business Development
>>> United Cable Company
>>> www.unitedcablecompany.com<http://www.unitedcablecompany.com>
>> And following the link to United Cable Company's web site reveals:
>> 
>> "Your source for the world's most distinctive submarine cable assets."  And 
>> the about page says "Its mission, as a leading telecom consulting company, 
>> is to represent the world’s most distinctive submarine and terrestrial cable 
>> assets."
>> 
>> Your original query asked:
>> 
>>> Am I wrong in believing that there should be a way of lighting a single 
>>> pair in the cable and then monitoring it for signal disruption? It is not a 
>>> perfect solution, but arguably better than learning that the cable has been 
>>> damaged from an irate customer.
>> In a followup message you say:
>> 
>>> Just to clarify, this is a dark fiber network already built and will be 
>>> repaired by the construction company that built it. I just a system to 
>>> inform them as soon as the fibers are damaged.
>> So... color me confused about who you are, who you represent, what you're 
>> trying to accomplish, what you're asking, and, perhaps, why you don't 
>> already know the answer to your question, or have someone internal to your 
>> organization who already knows.
>> 
>> Miles Fidelman
>> 
>> -- 
>> In theory, there is no difference between theory and practice.
>> In practice, there is.   Yogi Berra
>> 
>> Theory is when you know everything but nothing works. 
>> Practice is when everything works but no one knows why. 
>> In our lab, theory and practice are combined: 
>> nothing works and no one knows why.  ... unknown

Re: Outsourced NOC Solutions

[Dave Cohen]

There is a middle ground between “not managing customer light” and “not 
managing anything” though. The Adva ALM solution that a few folks that have 
mentioned, along with others like Lucent SmartLGX, effectively bridge this gap 
by helping trace the precise location of cuts and even smaller scale incidents 
like microbends to expedite diagnosis and repair to the extent possible. It’s 
not a panacea and definitely not a substitute for managing the hardware at the 
endpoints, but it does improve operational responsiveness in a measurable way. 
And yes, there are dark fiber providers in the Northeast that leverage this 
technology, at least on some portion of their fiber plants. 

Dave Cohen
craetd...@gmail.com

> On Jun 8, 2020, at 5:40 PM, James Jun  wrote:
> 
> On Mon, Jun 08, 2020 at 08:10:44PM +, Mel Beckman wrote:
>> 
>> I???m not talking about a full-time engineer for the life of the network, 
>> just for designing the infrastructure management before first customer light.
>> 
>> -mel via cell
>> 
> 
> Dude, it's dark fiber.
> 
> I for one, do _NOT_ in any shape or form, want my DF provider to put any 
> equipment (monitoring, or otherwise) on strands I lease, period.  I just want
> tubes in the ground, end of story.  This is certainly not an airplane and 
> does not need a pilot.  It's passive tubes sitting on right of way and 
> customer
> is licensed to pass light thru that passive tube.  Everything else is extra, 
> and I want no active service whatsoever (besides for power capacity at
> regen plant colo).
> 
> If there is a disturbance event that creates LOS alarm on customer equipment, 
> they will call in and open a ticket to begin troubleshooting.
> 
> Name me one dark fiber provider in northeast that (unless you buy their 
> managed dark fiber solution) will monitor your fiber strands and the customer
> light for you.  I can tell you, major fiber providers in northeast are all 
> the same:  the customer is the monitoring system.  If fiber is down, customers
> call in.  In fact, I can't recount how many times I've had dealing with a 
> large fiber provider here (unnamed to protect the guilty) who also requests
> and asks customers to shoot OTDR for them.
> 
> Generally speaking, dark fiber providers who also compete with their 
> customers (e.g. fiber provider that sells lit services) have tendency to react
> faster to certain fiber cuts on certain routes, if their backbone links are 
> sitting in them.  But for specialty dark fiber providers who only sell dark,
> it's not a bad idea to light one of the strands for internal continuity 
> checks; but at worst case scenario, when a customer calls in to report an LOS
> alarm and suspects fiber disturbance, that's usually enough information to 
> start sending your crews out and begin taking traces.
> 
> James

Re: US/Canada International border concerns for routing

[Dave Cohen]

Sorta, kinda. The various ASs operated by Zayo are more interconnected than 
that description would imply. The traditional mode of operation on an "acquired 
AS" has been to turn down any upstream transit as quickly as contractually 
possible and upgrade NNI capacity between that AS and 6461 to compensate. Over 
time, legacy devices are overbuilt or replaced with ones directly on 6461. The 
net-net of it is that most traffic will end up egressing to other providers via 
6461's peering after a fairly short period, although this isn't universally 
true, especially for "local" traffic (e.g. traffic originating on the Neo AS 
staying in France, etc.). 

Dave Cohen
craetd...@gmail.com

> On Aug 8, 2017, at 10:13 PM, Eric Kuhnke <eric.kuh...@gmail.com> wrote:
> 
> It is worth noting, however, that the former AllStream ASN (formerly AT
> Canada) AS15290 is a completely different thing, and has distinct
> infrastructure and routing from the AboveNet ASN which is operated by Zayo.
> Although they are probably using "Free" Zayo transport by now.
> 
> If I am grossly wrong and anybody from layer 3 network operations at Zayo
> wants to chime in and tell us about the 40,000 ft view of their plans to
> combine AS15290 and AS6461, I am sure the community would be very
> interested.
> 
>> On Tue, Aug 8, 2017 at 5:31 PM, Stephen Fulton <s...@lists.esoteric.ca> 
>> wrote:
>> 
>> TR,
>> 
>> MTS Allstream is no longer a combined entity.  MTS was purchased by Bell
>> Canada and Allstream was purchased by Zayo.
>> 
>> -- Stephen
>> 
>> 
>>> On 2017-08-08 8:19 PM, TR Shaw wrote:
>>> 
>>> Bill,
>>> 
>>> What does Bell buying MTS do? Does it change your statement or will the
>>> MTS portion of Bell still peer locally?
>>> 
>>> Tom
>>> 
>>>> On Aug 8, 2017, at 8:10 PM, Bill Woodcock <wo...@pch.net> wrote:
>>>> 
>>>> 
>>>>> On Jul 20, 2017, at 7:01 AM, Hiers, David <david.hi...@cdk.com> wrote:
>>>>> For traffic routing, is anyone constraining cross-border routing
>>>>> between Canada and the US?  IOW, if you are routing from Toronto to
>>>>> Montreal, do you have to guarantee that the path cannot go through, say,
>>>>> Syracuse, New York?
>>>>> 
>>>> 
>>>> No.  In fact, Bell Canada / Bell Aliant and Telus guarantee that you
>>>> _will_ go through Chicago, Seattle, New York, or Ashburn, since none of
>>>> them peer anywhere in Canada at all.
>>>> 
>>>> Last I checked (November of last year) the best-connected commercial
>>>> networks (i.e. not CANARIE) in Canada were Hurricane Electric, MTS
>>>> Allstream, Primus, and Zip Telecom, all of which peer at three or more
>>>> Canadian IXes.  So, they’re capable of keeping traffic in Canada so long as
>>>> the other end isn’t on Bell or Telus, which only sell U.S. bandwidth to
>>>> Canadians.
>>>> 
>>>> In November, only 27% of intra-Canadian routes stayed within Canada; 64%
>>>> went through the U.S.  That’s way worse than five years ago, when 60%
>>>> stayed within Canada, and 38% went through the U.S.
>>>> 
>>>> As has been pointed out, Canada has been building IXPs…  Just not as
>>>> fast as the rest of the world has.  They’re behind the global average
>>>> growth rate, and behind the U.S. growth rate, which is why the problem is
>>>> getting worse.  Bandwidth costs are falling faster elsewhere, so they’re
>>>> importing more foreign bandwidth.
>>>> 
>>>>-Bill
>>>> 
>>>> 
>>>> 
>>>> 
>>>> 
>>> 

Re: US/Canada International border concerns for routing

[Dave Cohen]

It seems to me the original question was asking about it more from a legal 
perspective, in other words does Canadian traffic have to stay in Canada. IANAL 
(or a Canadian), but the answer is "mostly, no, especially as related to 
publicly routed traffic" as should be evidenced based on what's already been 
discussed here. In other words, there is restricted traffic but unless you're 
making a play for MAN/WAN type service on owned infrastructure, those 
requirements are unlikely to arise. 

To support the macro point, there is some big-boy level peering in Toronto but 
not really much else outside that, but there are plenty of routes that don't 
cross the border if you don't have to jump networks to your destination, for 
example going to an AWS on ramp in Canada using a native partner network, 
especially in the Toronto-Ottawa-Montreal. 

Dave Cohen
craetd...@gmail.com

> On Aug 8, 2017, at 8:41 PM, Bill Woodcock <wo...@pch.net> wrote:
> 
> 
> --Apple-Mail=_8DA28412-F6D0-43D8-A90F-5E151E54468E
> Content-Transfer-Encoding: quoted-printable
> Content-Type: text/plain;
>charset=us-ascii
> 
> 
>> On Aug 8, 2017, at 5:33 PM, Clayton Zekelman <clay...@mnsi.net> wrote:
>> =20
>> =20
>> =20
>> With the peering policies of the major Canadian ISPs, you're virtually =
> guaranteed to hairpin through the US on most paths.
>> =20
>> Robellus (Rogers, Bell & Telus) will peer with you at any of their =
> major Canadian peering points, such as NYC, Chicago or LA.
> 
> To be fair, Rogers does peer in Toronto.  Along with New York, Chicago, =
> Seattle, and Ashburn.
> 
>-Bill
> 
> 
> 
> 
> 
> --Apple-Mail=_8DA28412-F6D0-43D8-A90F-5E151E54468E
> Content-Transfer-Encoding: 7bit
> Content-Disposition: attachment;
>filename=signature.asc
> Content-Type: application/pgp-signature;
>name=signature.asc
> Content-Description: Message signed with OpenPGP
> 
> -BEGIN PGP SIGNATURE-
> 
> iQIcBAEBCAAGBQJZilooAAoJEG+kcEsoi3+HgNsQAIPkgL/lVL/j1sdPyiyQsepE
> TCyHm4bsAq6m085kXoRj/IWn+KsVwmAq8ZGKnKEAiozmrSeyxAa2vmw5Kfs57l1/
> crBima+EOOlPT4VcD7tv9e8yEiVdjDuMp5tnLI238qCfIlHeHRtuU7CClzWPv6uD
> 3jCNIBEcScrLWz37Ofm/D2AkYRAhhK5H8I417Y/39TH4MIoIKFsGbvWwpl30Fv8r
> 5phO0MrTP6mB8niHne6HTxyMED5TGQpVEL2Qgh6qgaI9vzAs5/47KwwY57tZpxaL
> v9GjkPJ4Ql7QVWbsSkXnFmHxXzqaHXAfg8SR+gsCN42Jyn99AIyAAwdALhqc4RuZ
> ydi+lOlEutAMndA01CnrI81Eu/RpWrN+q/vi37W2rb6EPTPcCz2196JDlpC6VVW6
> tJOMNuP6Pa/ee52Cxu6RWwA4QZ6QVIT9fbDcRFXTGNuohwP8XVpujcsPLChzsFXA
> Y2nt+TliL697lTZNbTZEzQ0f9w2rpCDpcLjTMCR8MNWZ4MjQHL3eDgO5ZIWHPTQf
> ggR1Dz2EhPSXXZdvN7KPh1q9rhRb2VUPSn3EeEDo2TjgUVeUlunsDg/ILpf8lxUY
> RTsXe5Nky7YqXKDG4HSlLF3R/RtfaVqKJfjljYg351cs40rzivzjD2TJ8r35RQeW
> btKUtEvrcU28g15nOhLG
> =MTUG
> -END PGP SIGNATURE-
> 
> --Apple-Mail=_8DA28412-F6D0-43D8-A90F-5E151E54468E--
> 

Re: What's the meaning of virtual POP ?

[Dave Cohen]

The key is really that it could mean different things for different providers, 
although I would agree that the gist is that the location is enabled to look 
and feel like a POP without the provider installing the full complement of 
requisite hardware. A provider I worked at in the past, for example, defined a 
virtual POP as a non-POP location at which POP pricing was offered - the actual 
method of delivery there being both irrelevant to it being defined that way and 
unimportant to the concept as a whole. It let the company be price-competitive 
with others that may have made more extensive investments in hardware at 
higher-demand locations, and it was purely based on a business justification. 
There was no specific technical definition (although in reality we were 
transparent with our customers about methodology anyway) - this contrasts with 
other providers that are clearly using it in a way that does define a technical 
approach. It's just an approach specific to that provider.

> On Aug 23, 2016, at 6:51 PM, Rod Beck  wrote:
> 
> Yes, except it is done via Switched Ethernet and VLANs. The idea behind 
> virtual peering. Your gear is in Amsterdam and someone gives you VLANs to 
> LINX.
> 
> 
> - R.
> 
> 
> 
> From: NANOG  on behalf of William Herrin 
> 
> Sent: Wednesday, August 24, 2016 12:46 AM
> To: Yucong Sun
> Cc: NANOG
> Subject: Re: What's the meaning of virtual POP ?
> 
>> On Tue, Aug 23, 2016 at 6:31 PM, Yucong Sun  wrote:
>> I came across the idea of the virtual POP  , but the website for them have
>> way too much jargon to me[1][2][3], can someone explain it like i'm five
>> (:-D)?
> 
> A virtual Point Of Presence means that you provide services at a
> location via someone else's facilities.
> 
> The classic example was extending a PRI for dialup modems inside a
> particular local calling area via a point-to-point T1 back to your
> modem bank somewhere else that would have been a long distance call
> for those customers. If you put a modem bank in their local calling
> area, it's a POP. If you extend the circuit from their local calling
> area back to your modem bank elsewhere, it's a virtual POP.
> 
> Modern examples of virtual POPs are much fancier but it's the same basic idea.
> 
> 
>> 1. Is virtual POP basically a L2VPN?
> 
> It can be. Depends on what service you're extending from the "virtual" 
> location.
> 
> 
>> 2. Do such vPOP have guaranteed latency/bandwidth?
> 
> Depends on what you're extending and how.
> 
> 
>> 3. Is that really useful?
> 
> It can be. It can let you dip your toes in a market without a large
> up-front investment in equipment and backhaul.
> 
> Regards,
> Bill Herrin
> 
> 
> --
> William Herrin  her...@dirtside.com  b...@herrin.us
> Owner, Dirtside Systems . Web: 
> Dirtside Systems
> www.dirtside.com
> Welcome! You are our 370,765 th guest. Dirtside builds ground systems and 
> ground system software for the satellite and mobile communications industries.
> 
> 

Re: AWS Direct Connect - Peering VPCs to Tier 1's and MPLS

[Dave Cohen]

I can confirm that AWS (and Equinix, by extension, from a facility operator
perspective) permit carriers to have multiple end users share a physical
interface into the AWS gateway. The key is whether the providers that are
permitted into the DX environment (I believe AWS has limited the list to
only 7 or 8 in total - anyone else is reselling capacity off of those
carriers) are willing to deal with the constraints of that configuration -
essentially that the carrier needs to take responsibility of engaging
directly with AWS to associate the EVC on the provider interface with the
VPC on the AWS interface. I can confirm that at least one provider other
than Equinix will do this. Point being, it's not an AWS restriction as much
as whether the provider is willing to get its hands a bit dirtier. My $.02
at least.

- Dave

On Tue, Mar 1, 2016 at 7:59 PM, Mike Hammett <na...@ics-il.net> wrote:

> I haven't heard it from the horse's mouth, but I heard that the only way
> to have customers share an AWS DX (apparently) cross connect is through
> Equinix's cloud exchange service. Can anyone confirm that? It doesn't seem
> right that I could transport people to AWS all day long if they buy their
> own cross connect, but once we share, I have to go through someone offering
> a competitive service.
>
>
>
>
> -
> Mike Hammett
> Intelligent Computing Solutions
> http://www.ics-il.com
>
> Midwest-IX
> http://www.midwest-ix.com
>
> - Original Message -
>
> From: "Michael O'Connor" <m...@es.net>
> To: "Jay R. Ashworth" <j...@baylink.com>
> Cc: "North American Network Operators' Group" <nanog@nanog.org>
> Sent: Tuesday, March 1, 2016 2:41:35 PM
> Subject: Re: AWS Direct Connect - Peering VPCs to Tier 1's and MPLS
>
> Jay,
>
> VPC is supported over IPsec if your public path is sufficient into the AWS
> cloud.
>
> AWS shortens DirectConnect to DX not DC for some reason.
>
> The AWS DirectConnect service is built on 10G infrastructure so using
> potentially larger interconnects over public peerings with IPsec could be
> advantageous.
>
> DX requires fiber cross connects in addition to any other AWS peerings that
> you may have at a particular location.
>
> -Mike O'Connor
>
>
> On Tue, Mar 1, 2016 at 12:16 PM, Jay R. Ashworth <j...@baylink.com> wrote:
>
> > Just got this dropped on my desk an hour ago, and I'm not finding as much
> > material online as I might have hoped for...
> >
> > It looks like the easiest solution is to just hang a router/firewall at
> > Equinix Ashburn and AWS-DC to that, and then peer it to carriers both IP
> > and
> > MPLS; is there a "native" way to do that from an AWS VPC instead?
> >
> > Any public or private replies cheerfully accepted; will summarize what I
> > can to the list.
> >
> > Cheers,
> > -- jra
> >
> > --
> > Jay R. Ashworth Baylink
> > j...@baylink.com
> > Designer The Things I Think RFC
> > 2100
> > Ashworth & Associates http://www.bcp38.info 2000 Land
> > Rover DII
> > St Petersburg FL USA BCP38: Ask For It By Name! +1 727 647
> > 1274
> >
>
>
>
> --
> Michael O'Connor
> ESnet Network Engineering
> m...@es.net
> 631 344-7410
>
>


-- 
- Dave Cohen
eM: craetd...@gmail.com
AIM: dCo says

Re: Standard terminology for a dark fiber path?

[Dave Cohen]

FWIW, at my $dayjob (a fiber-based service provider), the accepted term is
"span", which accounts for any continuous segment between add/drop and/or
regen locations (i.e. no provider or end user electronics in the middle,
only at the endpoints). The most common alternate I come across is
"segment".

Re a couple of earlier suggestions - A patch between cables to provide
continuity, as compared to a fusion splice, doesn't inherently change this
view, as it has no bearing on the logical use of the span. Similarly,
"strand" isn't favored as it assumes a single fiber only, where the vast
majority of applications require a pair (or multiple pairs), so doesn't
accurately reflect the logical use of the span. I think "1F Span" is the
favored reference for a single-fiber deployment, for the sake of both
consistency and clarity.

On Thu, Feb 25, 2016 at 6:27 PM, Michael Loftis <mlof...@wgops.com> wrote:

> IDK what elsewhere uses but strand or (less common) span is the common
> term I've seen specifically for a passive piece of glass between two
> points.
>
> On Wed, Feb 24, 2016 at 12:55 PM, Fletcher Kittredge <fkitt...@gwi.net>
> wrote:
> > What is the standard terminology for strands of dark fiber spliced
> together
> > to form a continuous path between points A and Z?
> >
> > I have seen:
> >
> >- *fiber circuit* [but also seen used to denote a connection at the
> >network layer over a physical fiber connection. This definition of
> circuit
> >would include the dark fiber path, the transmitters and receivers and
> logic
> >making up the data and network layers.]
> >- *fiber loop *[ Does a loop define an electrical circuit with two
> >physically separate positive and negative strands? In that case, is
> this a
> >Bellhead remnant? ]
> >
> > I am particularly interested in last mile systems, but I don't see any
> > reason that the term wouldn't be the same in the middle mile.
> >
> > thanks,
> > Fletcher
> >
> > --
> > Fletcher Kittredge
> > GWI
> > 8 Pomerleau Street
> > Biddeford, ME 04005-9457
> > 207-602-1134
>
>
>
> --
>
> "Genius might be described as a supreme capacity for getting its possessors
> into trouble of all kinds."
> -- Samuel Butler
>



-- 
- Dave Cohen
eM: craetd...@gmail.com
AIM: dCo says