Re: Has virtualization become obsolete in 5G?

[Ahmed elBorno]

Maybe I am off topic a little bit here and i'd like to be educated if i am
wrong but I think those 5G applications will move from VMs into
containers/microservices when their vendors see a business case to
rearchitect them, maybe its already happening as we speak.

On the other side of that coin is that product managers of these 5G apps
seeing the margins on their apps diminish when they slice them to a form
that allows other "orchestrators" to deploy them.

Another side is that the software engineers working on these Apps have a
lot more prioritized items/things to develop (real core functions) so they
will delay this transformation.

However, some CSPs are doing well putting a wrapper/UX around Mobility
(e.g: Twilio)

Cheers

On Sat, Aug 1, 2020 at 6:36 PM John Levine  wrote:

> In article <20200801143522.e25a8...@m0117164.ppops.net> you write:
> >--- ed...@ieee.org wrote:
> >From: Etienne-Victor Depasquale 
> >
> >See, for example, Azhar Sayeed's (Red Hat) contribution here
> >@15:33.
> >
> >
> >
> >Don't send links to this list that require one to register
> >to read the article and then say, "By registering for our
> >site, your email will be added to our promotions list" and
> >"Occasionally our trusted partners may want to send you
> >information about exciting new products and services"
> >
> >No one's going to click on that!
>
> Sure we are.  That's what mailinator is for.
>
> R's,
> John
>

Re: Issue with Noction IRP default setting (Was: BGP route hijack by AS10990)

[Matt Erculiani]

Ryan,

To continue with your analogy, this would be more similar to someone who
has never driven before walking into a dealership and buying a new car to
drive off the lot. Ultimately the responsibility is on the driver, but the
dealership should have never sold them the car in the first place. Thus,
it's reasonable to place some of the responsibility on the vendors who
introduce this equipment into the wild without truly understanding (or
worse, not caring) how easy they've made it for their users to cause havoc
when most of the defaults are left untouched.

Again, it's not like anyone consciously sets these optimizers to evil,
they're just spontaneously dangerous when left misconfigured (which is
apparently easy to do). These devices throw caution to the wind in the name
of being easy to use.

Think of all the companies that set up AD servers with "corp.com" and how
big of a security risk that is. Microsoft ended up taking action because of
the potentially catastrophic implications,  but at least it only affected
the companies that made poor choices. Now imagine those same individuals
who threw security to the wind are sold one of these devices because it's a
turnkey way to make their network faster; that's exactly what we see here
and it's terrifying to think how many of those little route hijack
timebombs are out there, just waiting to ruin your day, night, or vacation
in the Bahamas.

-Matt

On Sat, Aug 1, 2020 at 6:12 PM Ca By  wrote:

>
>
> On Sat, Aug 1, 2020 at 4:47 PM Ryan Hamel  wrote:
>
>> Matt,
>>
>> Why are you blaming the ease of use on the vendor, for the operators lack
>> of knowledge regarding BGP? That is like blaming a vehicle manufacturer for
>> a person pressing the gas pedal in a car and not giving a toss about the
>> rules of the road. The base foundation regarding the rules of the road
>> mostly apply the same for driving a car, truck, bus, and semi/lorry truck.
>> There is no excuse for ignorance just because the user interface is
>> different (web browser vs. SSH client).
>>
>
> Vendors are responsible.  The FTC slammed D-Link for being insecure and
> they can slam Noction too
>
>
> https://www.ftc.gov/news-events/press-releases/2019/07/d-link-agrees-make-security-enhancements-settle-ftc-litigation
>
>
> Asking people in Pintos to not get in accidents is not an option.
>
> https://www.tortmuseum.org/ford-pinto/
>
>
>
>
>> Adding a take on this, there are kids born after 9/11, with IP
>> allocations and ASNs experimenting in the DFZ right now. If they can make
>> it work, and not cause harm to other members in this community, it clearly
>> demonstrates a lack of knowledge, or honest human error (which will never
>> go away).
>>
>> Anything that can be used, can be misused. With that said, why shouldn't
>> ALL BGP software implementations encourage best practice? They decided RPKI
>> validation was a good thing.
>>
>> Ryan
>> On Aug 1 2020, at 4:12 pm, Matt Erculiani  wrote:
>>
>> Ryan,
>>
>> The reason Noction is being singled out here as opposed to other BGP
>> speakers is that it inherently breaks several BGP protection mechanisms as
>> a means to achieve its purpose. BGP was never intended to be "optimized",
>> it was intended to be stable and scalable. While i'm sure there are
>> hundreds of operators that use these optimizers without incident, they are
>> a significant paint point for the rest of the internet.
>>
>> They have created a platform that has the ease of use of a residential
>> CPE, but with the consequences of misuse of any DFZ platform. This allows
>> users who have little experience speaking BGP with the world to make these
>> mistakes because they don't know any better, whereas the other platforms
>> you mention require some knowledge to configure. It's not a perfect filter,
>> but it does create a barrier for the inept.
>>
>> Since Noction has made it easy enough to configure their software so that
>> anyone can do it, with or without experience on the DFZ, they have SOME
>> responsibility to keep their software from accidentally breaking the
>> internet.
>>
>> -Matt
>>
>>
>> On Sat, Aug 1, 2020 at 2:30 PM Ryan Hamel  wrote:
>>
>> Job,
>>
>> I disagree on the fact that it is not fair to the BGP implementation
>> ecosystem, to enforce a single piece of software to activate the no-export
>> community by default, due to ignorance from the engineer(s) implementing
>> the solution. It should be common sense that certain routes that should be
>> advertised beyond the local AS, just like RFC1918 routes, and more. Also,
>> wasn't it you that said Cisco routers had a bug in ignoring NO_EXPORT?
>> Would you go on a rant with Cisco, even if Noction add that enabled
>> checkbox by default?
>>
>> Why are you not on your soap box about BIRD, FRrouting, OpenBGPd, Cisco,
>> Juniper, etc... about how they can possibly allow every day screw ups to
>> happen, but the same options like the NO_EXPORT community are available for
>> the engineer to use? One solution would be to 

Re: Has virtualization become obsolete in 5G?

[John Levine]

In article <20200801143522.e25a8...@m0117164.ppops.net> you write:
>--- ed...@ieee.org wrote:
>From: Etienne-Victor Depasquale 
>
>See, for example, Azhar Sayeed's (Red Hat) contribution here
>@15:33.
>
>
>
>Don't send links to this list that require one to register 
>to read the article and then say, "By registering for our 
>site, your email will be added to our promotions list" and 
>"Occasionally our trusted partners may want to send you 
>information about exciting new products and services"
>
>No one's going to click on that!

Sure we are.  That's what mailinator is for.

R's,
John

Re: Issue with Noction IRP default setting (Was: BGP route hijack by AS10990)

[Ca By]

On Sat, Aug 1, 2020 at 4:47 PM Ryan Hamel  wrote:

> Matt,
>
> Why are you blaming the ease of use on the vendor, for the operators lack
> of knowledge regarding BGP? That is like blaming a vehicle manufacturer for
> a person pressing the gas pedal in a car and not giving a toss about the
> rules of the road. The base foundation regarding the rules of the road
> mostly apply the same for driving a car, truck, bus, and semi/lorry truck.
> There is no excuse for ignorance just because the user interface is
> different (web browser vs. SSH client).
>

Vendors are responsible.  The FTC slammed D-Link for being insecure and
they can slam Noction too

https://www.ftc.gov/news-events/press-releases/2019/07/d-link-agrees-make-security-enhancements-settle-ftc-litigation


Asking people in Pintos to not get in accidents is not an option.

https://www.tortmuseum.org/ford-pinto/




> Adding a take on this, there are kids born after 9/11, with IP allocations
> and ASNs experimenting in the DFZ right now. If they can make it work, and
> not cause harm to other members in this community, it clearly demonstrates
> a lack of knowledge, or honest human error (which will never go away).
>
> Anything that can be used, can be misused. With that said, why shouldn't
> ALL BGP software implementations encourage best practice? They decided RPKI
> validation was a good thing.
>
> Ryan
> On Aug 1 2020, at 4:12 pm, Matt Erculiani  wrote:
>
> Ryan,
>
> The reason Noction is being singled out here as opposed to other BGP
> speakers is that it inherently breaks several BGP protection mechanisms as
> a means to achieve its purpose. BGP was never intended to be "optimized",
> it was intended to be stable and scalable. While i'm sure there are
> hundreds of operators that use these optimizers without incident, they are
> a significant paint point for the rest of the internet.
>
> They have created a platform that has the ease of use of a residential
> CPE, but with the consequences of misuse of any DFZ platform. This allows
> users who have little experience speaking BGP with the world to make these
> mistakes because they don't know any better, whereas the other platforms
> you mention require some knowledge to configure. It's not a perfect filter,
> but it does create a barrier for the inept.
>
> Since Noction has made it easy enough to configure their software so that
> anyone can do it, with or without experience on the DFZ, they have SOME
> responsibility to keep their software from accidentally breaking the
> internet.
>
> -Matt
>
>
> On Sat, Aug 1, 2020 at 2:30 PM Ryan Hamel  wrote:
>
> Job,
>
> I disagree on the fact that it is not fair to the BGP implementation
> ecosystem, to enforce a single piece of software to activate the no-export
> community by default, due to ignorance from the engineer(s) implementing
> the solution. It should be common sense that certain routes that should be
> advertised beyond the local AS, just like RFC1918 routes, and more. Also,
> wasn't it you that said Cisco routers had a bug in ignoring NO_EXPORT?
> Would you go on a rant with Cisco, even if Noction add that enabled
> checkbox by default?
>
> Why are you not on your soap box about BIRD, FRrouting, OpenBGPd, Cisco,
> Juniper, etc... about how they can possibly allow every day screw ups to
> happen, but the same options like the NO_EXPORT community are available for
> the engineer to use? One solution would be to implement "BGP Group/Session
> Profiles" (ISP/RTBH/DDOS Filtering/Route Optimizers/etc) or a "BGP Session
> Wizard" (ask the operator questions about their intentions), then
> automatically generate import and export policies based on known accepted
> practices.
>
> Another solution could be having the BGP daemon disclose the make, model
> family, and exact model of hardware it is running on, to BGP peers, and add
> more knobs into policy creation to match said values, and take action
> appropriately. That would be useful in getting around vendor specific
> issues, as well as belt & suspenders protection.
>
> Ryan
> On Aug 1 2020, at 9:58 am, Job Snijders  wrote:
>
> On Sat, Aug 01, 2020 at 06:50:55AM -0700, Ca By wrote:
> > I am not normally supporting a heavy hand in regulation, but i think it
> is
> > fair to say Noction and similar BGP optimizers are unsafe at any speed
> and
> > the FTC or similar should ban them in the USA. They harm consumers and
> are
> > a risk to national security / critical infrastructure
> >
> > Noction and similar could have set basic defaults (no-export, only create
> > /25 bogus routes to limit scope), but they have been clear that their
> greed
> > to suck up traffic does not benefit from these defaults and they wont do
> > it.
>
> Following a large scale BGP incident in March 2015, noction made it
> possible to optionally set the well-known NO_EXPORT community on route
> advertisements originated by IRP instances.
>
> "In order to further reduce the likelihood of these problems
> occurring in the future, 

Re: Has virtualization become obsolete in 5G?

[Mike Hammett]

Buzzwords have a limited life before the vendors need to make up something else 
to invoice you for. 




- 
Mike Hammett 
Intelligent Computing Solutions 
http://www.ics-il.com 

Midwest-IX 
http://www.midwest-ix.com 

- Original Message -

From: "Etienne-Victor Depasquale"  
To: "NANOG"  
Sent: Saturday, August 1, 2020 4:23:00 AM 
Subject: Has virtualization become obsolete in 5G? 


Hi folks, 


Over the past few weeks, I've attended webinars and watched videos organized by 
Intel. 
These activities have centred on 5G and examined applications (like "visual 
cloud" and "gaming"), 
as well as segment-oriented aspects (like edge networks, 5G RAN and 5G Core). 


I am stunned (no hyperbole) by the emphasis on Kubernetes in particular, 
and cloud-native computing in general. 
Equally stunning (for me), public telecommunications networks have been 
portrayed 
as having a history that moved from integrated software and hardware, 
to virtualization and now to cloud-native computing. 
See, for example Alex Quach, here @10:30). I reason that Intel's implication is 
that virtualization is becoming obsolete. 


Would anyone care to let me know his thoughts on this prediction? 




Cheers all, 


Etienne 

-- 


Ing. Etienne-Victor Depasquale 
Assistant Lecturer 
Department of Communications & Computer Engineering 
Faculty of Information & Communication Technology 
University of Malta 
Web. https://www.um.edu.mt/profile/etiennedepasquale I 

Re: Issue with Noction IRP default setting (Was: BGP route hijack by AS10990)

[Ryan Hamel]

Matt,

Why are you blaming the ease of use on the vendor, for the operators lack of 
knowledge regarding BGP? That is like blaming a vehicle manufacturer for a 
person pressing the gas pedal in a car and not giving a toss about the rules of 
the road. The base foundation regarding the rules of the road mostly apply the 
same for driving a car, truck, bus, and semi/lorry truck. There is no excuse 
for ignorance just because the user interface is different (web browser vs. SSH 
client).
Adding a take on this, there are kids born after 9/11, with IP allocations and 
ASNs experimenting in the DFZ right now. If they can make it work, and not 
cause harm to other members in this community, it clearly demonstrates a lack 
of knowledge, or honest human error (which will never go away).
Anything that can be used, can be misused. With that said, why shouldn't ALL 
BGP software implementations encourage best practice? They decided RPKI 
validation was a good thing.
Ryan
On Aug 1 2020, at 4:12 pm, Matt Erculiani  wrote:
> Ryan,
>
> The reason Noction is being singled out here as opposed to other BGP speakers 
> is that it inherently breaks several BGP protection mechanisms as a means to 
> achieve its purpose. BGP was never intended to be "optimized", it was 
> intended to be stable and scalable. While i'm sure there are hundreds of 
> operators that use these optimizers without incident, they are a significant 
> paint point for the rest of the internet.
>
> They have created a platform that has the ease of use of a residential CPE, 
> but with the consequences of misuse of any DFZ platform. This allows users 
> who have little experience speaking BGP with the world to make these mistakes 
> because they don't know any better, whereas the other platforms you mention 
> require some knowledge to configure. It's not a perfect filter, but it does 
> create a barrier for the inept.
>
> Since Noction has made it easy enough to configure their software so that 
> anyone can do it, with or without experience on the DFZ, they have SOME 
> responsibility to keep their software from accidentally breaking the internet.
>
> -Matt
>
>
> On Sat, Aug 1, 2020 at 2:30 PM Ryan Hamel  (mailto:r...@rkhtech.org)> wrote:
> > Job,
> >
> > I disagree on the fact that it is not fair to the BGP implementation 
> > ecosystem, to enforce a single piece of software to activate the no-export 
> > community by default, due to ignorance from the engineer(s) implementing 
> > the solution. It should be common sense that certain routes that should be 
> > advertised beyond the local AS, just like RFC1918 routes, and more. Also, 
> > wasn't it you that said Cisco routers had a bug in ignoring NO_EXPORT? 
> > Would you go on a rant with Cisco, even if Noction add that enabled 
> > checkbox by default?
> > Why are you not on your soap box about BIRD, FRrouting, OpenBGPd, Cisco, 
> > Juniper, etc... about how they can possibly allow every day screw ups to 
> > happen, but the same options like the NO_EXPORT community are available for 
> > the engineer to use? One solution would be to implement "BGP Group/Session 
> > Profiles" (ISP/RTBH/DDOS Filtering/Route Optimizers/etc) or a "BGP Session 
> > Wizard" (ask the operator questions about their intentions), then 
> > automatically generate import and export policies based on known accepted 
> > practices.
> > Another solution could be having the BGP daemon disclose the make, model 
> > family, and exact model of hardware it is running on, to BGP peers, and add 
> > more knobs into policy creation to match said values, and take action 
> > appropriately. That would be useful in getting around vendor specific 
> > issues, as well as belt & suspenders protection.
> > Ryan
> > On Aug 1 2020, at 9:58 am, Job Snijders  > (mailto:j...@instituut.net)> wrote:
> > > On Sat, Aug 01, 2020 at 06:50:55AM -0700, Ca By wrote:
> > > > I am not normally supporting a heavy hand in regulation, but i think it 
> > > > is
> > > > fair to say Noction and similar BGP optimizers are unsafe at any speed 
> > > > and
> > > > the FTC or similar should ban them in the USA. They harm consumers and 
> > > > are
> > > > a risk to national security / critical infrastructure
> > > >
> > > > Noction and similar could have set basic defaults (no-export, only 
> > > > create
> > > > /25 bogus routes to limit scope), but they have been clear that their 
> > > > greed
> > > > to suck up traffic does not benefit from these defaults and they wont do
> > > > it.
> > >
> > > Following a large scale BGP incident in March 2015, noction made it
> > > possible to optionally set the well-known NO_EXPORT community on route
> > > advertisements originated by IRP instances.
> > >
> > > "In order to further reduce the likelihood of these problems
> > > occurring in the future, we will be adding a feature within Noction
> > > IRP to give an option to tag all the more specific prefixes that it
> > > generates with the BGP NO_EXPORT community. This will not 

Re: Issue with Noction IRP default setting (Was: BGP route hijack by AS10990)

[Mike Hammett]

Was Tulix using Noction, or was it something else that caused their particular 
issue? 




- 
Mike Hammett 
Intelligent Computing Solutions 
http://www.ics-il.com 

Midwest-IX 
http://www.midwest-ix.com 

- Original Message -

From: "Job Snijders"  
To: nanog@nanog.org 
Sent: Saturday, August 1, 2020 11:58:12 AM 
Subject: Issue with Noction IRP default setting (Was: BGP route hijack by 
AS10990) 

On Sat, Aug 01, 2020 at 06:50:55AM -0700, Ca By wrote: 
> I am not normally supporting a heavy hand in regulation, but i think it is 
> fair to say Noction and similar BGP optimizers are unsafe at any speed and 
> the FTC or similar should ban them in the USA. They harm consumers and are 
> a risk to national security / critical infrastructure 
> 
> Noction and similar could have set basic defaults (no-export, only create 
> /25 bogus routes to limit scope), but they have been clear that their greed 
> to suck up traffic does not benefit from these defaults and they wont do 
> it. 

Following a large scale BGP incident in March 2015, noction made it 
possible to optionally set the well-known NO_EXPORT community on route 
advertisements originated by IRP instances. 

"In order to further reduce the likelihood of these problems 
occurring in the future, we will be adding a feature within Noction 
IRP to give an option to tag all the more specific prefixes that it 
generates with the BGP NO_EXPORT community. This will not be enabled 
by default [snip]" 
https://www.noction.com/blog/route-optimizers 
Mar 27, 2015 

Due to NO_EXPORT not being set in the default configuration, there are 
probably if not certainly many unsuspecting network engineers who end up 
deploying this software - without ever even considering - to change that 
one setting in the configuration. 

Fast forward a few years and a few incidents, on the topic of default 
settings, following the Cloudflare/DQE/Verizon incident: 

"We do have no export community support and have done for many 
years. The use of more specifics is also optional. Neither replaces 
the need for filters." 
https://twitter.com/noction/status/1143177562191011840 
Jun 24, 2019 

Community members responded: 

"Noction have been facilitating Internet outages for years and 
years and the best thing they can say in response is that it is 
technically possible to use their product responsibly, they just 
don't ship it that way." 
https://twitter.com/PowerDNS_Bert/status/1143252745257979905 
June 24, 2019 

Last year Noction stated: 

"Nobody found this leak pleasant." 
https://www.noction.com/news/incident-response 
June 26, 2019 

Sentiment we all can agree with, change is needed! 

As far as I know, Noction IRP is the ONLY commercially available 
off-the-shelf BGP route manipulation software which - as default - does 
NOT set the BGP well-known NO_EXPORT community on the product's route 
advertisements. This is a product design decision which causes 
collateral damage. 

I would like to urge Noction to reconsider their position. Seek to 
migrate the existing users to use NO_EXPORT, and release a new version 
of the IRP software which sets NO_EXPORT BY DEFAULT on all generated 
routes. 

Kind regards, 

Job 

Re: Issue with Noction IRP default setting (Was: BGP route hijack by AS10990)

[Matt Erculiani]

Ryan,

The reason Noction is being singled out here as opposed to other BGP
speakers is that it inherently breaks several BGP protection mechanisms as
a means to achieve its purpose. BGP was never intended to be "optimized",
it was intended to be stable and scalable. While i'm sure there are
hundreds of operators that use these optimizers without incident, they are
a significant paint point for the rest of the internet.

They have created a platform that has the ease of use of a residential CPE,
but with the consequences of misuse of any DFZ platform. This allows users
who have little experience speaking BGP with the world to make these
mistakes because they don't know any better, whereas the other platforms
you mention require some knowledge to configure. It's not a perfect filter,
but it does create a barrier for the inept.

Since Noction has made it easy enough to configure their software so that
anyone can do it, with or without experience on the DFZ, they have SOME
responsibility to keep their software from accidentally breaking the
internet.

-Matt


On Sat, Aug 1, 2020 at 2:30 PM Ryan Hamel  wrote:

> Job,
>
> I disagree on the fact that it is not fair to the BGP implementation
> ecosystem, to enforce a single piece of software to activate the no-export
> community by default, due to ignorance from the engineer(s) implementing
> the solution. It should be common sense that certain routes that should be
> advertised beyond the local AS, just like RFC1918 routes, and more. Also,
> wasn't it you that said Cisco routers had a bug in ignoring NO_EXPORT?
> Would you go on a rant with Cisco, even if Noction add that enabled
> checkbox by default?
>
> Why are you not on your soap box about BIRD, FRrouting, OpenBGPd, Cisco,
> Juniper, etc... about how they can possibly allow every day screw ups to
> happen, but the same options like the NO_EXPORT community are available for
> the engineer to use? One solution would be to implement "BGP Group/Session
> Profiles" (ISP/RTBH/DDOS Filtering/Route Optimizers/etc) or a "BGP Session
> Wizard" (ask the operator questions about their intentions), then
> automatically generate import and export policies based on known accepted
> practices.
>
> Another solution could be having the BGP daemon disclose the make, model
> family, and exact model of hardware it is running on, to BGP peers, and add
> more knobs into policy creation to match said values, and take action
> appropriately. That would be useful in getting around vendor specific
> issues, as well as belt & suspenders protection.
>
> Ryan
> On Aug 1 2020, at 9:58 am, Job Snijders  wrote:
>
> On Sat, Aug 01, 2020 at 06:50:55AM -0700, Ca By wrote:
> > I am not normally supporting a heavy hand in regulation, but i think it
> is
> > fair to say Noction and similar BGP optimizers are unsafe at any speed
> and
> > the FTC or similar should ban them in the USA. They harm consumers and
> are
> > a risk to national security / critical infrastructure
> >
> > Noction and similar could have set basic defaults (no-export, only create
> > /25 bogus routes to limit scope), but they have been clear that their
> greed
> > to suck up traffic does not benefit from these defaults and they wont do
> > it.
>
> Following a large scale BGP incident in March 2015, noction made it
> possible to optionally set the well-known NO_EXPORT community on route
> advertisements originated by IRP instances.
>
> "In order to further reduce the likelihood of these problems
> occurring in the future, we will be adding a feature within Noction
> IRP to give an option to tag all the more specific prefixes that it
> generates with the BGP NO_EXPORT community. This will not be enabled
> by default [snip]"
> https://www.noction.com/blog/route-optimizers
> Mar 27, 2015
>
> Due to NO_EXPORT not being set in the default configuration, there are
> probably if not certainly many unsuspecting network engineers who end up
> deploying this software - without ever even considering - to change that
> one setting in the configuration.
>
> Fast forward a few years and a few incidents, on the topic of default
> settings, following the Cloudflare/DQE/Verizon incident:
>
> "We do have no export community support and have done for many
> years. The use of more specifics is also optional. Neither replaces
> the need for filters."
> https://twitter.com/noction/status/1143177562191011840
> Jun 24, 2019
>
> Community members responded:
>
> "Noction have been facilitating Internet outages for years and
> years and the best thing they can say in response is that it is
> technically possible to use their product responsibly, they just
> don't ship it that way."
> https://twitter.com/PowerDNS_Bert/status/1143252745257979905
> June 24, 2019
>
> Last year Noction stated:
>
> "Nobody found this leak pleasant."
> https://www.noction.com/news/incident-response
> June 26, 2019
>
> Sentiment we all can agree with, change is needed!
>
> As far as I know, Noction IRP is the ONLY 

Re: Has virtualization become obsolete in 5G?

[John Lee]

The short answer is that the "Cloud Native Computing" folks need to talk to
the Intel Embedded Systems Application engineers to discover that micro
services have been running on Intel hardware in (non-standard) containers
for years. We call it real time computing, process control,... Current
multi Terabit Ethernet interfaces require specialized hardware and
interfaces that will connect fiber optics to clouds but cannot be run on
clouds.

Some comments on Software Controlled Telecomm (/datacom) networking. When
DTMF was invented the telco used in band signaling for call control. Kevin
Mitnick et. al. designed red and black boxes to control the telco systems
so the telcos moved call control out of band. They created SIgnal Control
Points which managed the actual circuit switch hardware to route calls or
eventually 64kbps digital paths and this protocol was SS#7. There were six
to seven volumes of CLASS services that were enabled by SS#7 which ran on
UNIX systems developed by Bell Labs. In the mid seventies, I worked on VM
systems from DEC and Apollo of which Apollo had the better virtualization
that worked across the network and was the first "cloud" system that I
worked on.

In the mid nineties, I had worked on large Gigabit/Terabit routers but
again the control plane was part of the data plane until ATM based
networks  could use out of band control to setup a SVC between input port
and output port and switch the IP packets instead of routing them achieving
network end to end delays of less than milliseconds. VLAN and MPLS
protocols were developed to switch packets in the backbone of the networks
and not to route them.

In 2000 we put our first pre-standard cloud together with multi Gigabit
routers and Sun workstations at 45 PoPs in the US, 3 in Asia and 6 in
Europe and implemented a "cloud" O/S. Our fastest links were 10 Gbps. Now
we can have 2-50 Tbps per fiber using Superchannel DWDM technology between
PoP, data centers or cell towers. Network control functions can dynamically
change by using Dynamic Reprogrammable EPROMs from companies like Xilinx
and Intel to repurpose firmware control and device functions.

Embedded systems have implemented "micro services" for years as that is how
you handle interrupt driven real time control. We call this a context
switch which is still hardware CPU dependent. As far as I know, current
standard containers do not handle real time CPU interrupts or do they allow
very tight timing reponse loops within the standard containers?

Certain 5G proposals are discussing network slicing et al to virtualize
control functions that can work better without virtualization. Current 5G
protocol submissions that I have reviewed are way too complex to work out
in the real world on real networks, maintained by union labor. (This is not
a dig at union labor, as they are some of the best trained techs.) :)

On Sat, Aug 1, 2020 at 8:35 AM Mark Tinka  wrote:

>
>
> On 1/Aug/20 11:23, Etienne-Victor Depasquale wrote:
>
> Over the past few weeks, I've attended webinars and watched videos
> organized by Intel.
> These activities have centred on 5G and examined applications (like
> "visual cloud" and "gaming"),
> as well as segment-oriented aspects (like edge networks, 5G RAN and 5G
> Core).
>
> I am stunned (no hyperbole) by the emphasis on Kubernetes in particular,
> and cloud-native computing in general.
> Equally stunning (for me), public telecommunications networks have been
> portrayed
> as having a history that moved from integrated software and hardware,
> to virtualization and now to cloud-native computing.
> See, for example Alex Quach, here
> 
>  @10:30).
> I reason that Intel's implication is that virtualization is becoming
> obsolete.
>
> Would anyone care to let me know his thoughts on this prediction?
>
>
> In the early dawn of SDN, where it was cool to have the RP's in Beirut and
> the line cards in Lagos, the industry quickly realized that was not
> entirely feasible.
>
> If you are looking at over-the-top services, so-called cloud-native
> computing makes sense in order to deliver that value accordingly, and with
> agility. But as it pertains to actual network transport, I'm not yet sure
> the industry is at the stage where we are confident enough to decompose
> packet forwarding through a cloud.
>
> Network operators are more likely to keep using kit that integrates
> forwarding hardware as well as a NOS, as no amount of cloud architecting is
> going to rival a 100Gbps purpose-built port, for example.
>
> Suffice it to say, there was a time when folk were considering running
> their critical infrastructure (such as your route reflectors) in AWS or
> similar. I'm not quite sure public clouds are at that level of confidence
> yet. So if some kind of cloud-native infrastructure is to be considered for
> critical infrastructure, I highly suspect it will be 

Re: BGP route hijack by AS10990

[Owen DeLong]

> On Aug 1, 2020, at 12:59 PM, Sabri Berisha  wrote:
> 
> - On Aug 1, 2020, at 12:50 PM, Nick Hilliard n...@foobar.org wrote:
> 
> Hi,
> 
>> Sabri Berisha wrote on 01/08/2020 20:03:
>>> but because Noction's decision to not enable NO_EXPORT by default
>> 
>> the primary problem is not this but that Noction reinjects prefixes into
>> the local ibgp mesh with the as-path stripped and then prioritises these
>> prefixes so that they're learned as the best path.
> 
> Yeah, but that's not problem as far as I'm concerned. Their network, 
> their rules. I've done weirder stuff than that, in tightly controlled
> environments.

Your network, your rules is fine as far as your border. When you start 
announcing crap to the rest of the world, then the rest of the world has a 
right to object.

When your product makes it easy for your customers to accidentally announce 
crap to the rest of the world, then it’s the moral equivalent of building a car 
without a seatbelt. Sure, before the technology was widely known and its life 
saving capabilities well understood, it was legitimate to dismiss it as an 
unnecessary added cost. Today, there’s no excuse for such an action. The 
hazards of BGP optimizers are pretty well known and it’s not unreasonable to 
expect vendors to implement appropriate safeguards into their products and/or 
recommend appropriate safeguards by their customers in their other routing 
devices. Certainly no-export by default is an example of something that there’s 
really no reason not to do in any BGP optimizer.

>> The as-path is the primary loop detection mechanism in eBGP.  Removing
>> this is like hot-wiring your electrical distribution board because you
>> found out you could get more power if you bypass those stupid RCDs.
> 
> Well, let's be honest. Sometimes we need to get rid of that pesky mechanism.
> For example, when using BGP-as-IGP, the "allowas-in" disregards the as-path,
> in a controlled manner (and yes, I know, different use case).

Also a much more constrained case… allowas-in (which I still argue is a poor 
substitute
for getting different ASNs for your different non-backboned sites) only allows 
you to loop your own AS and only at your own sites. It doesn’t support allowing 
you to feed crap to the internet.

> My point is that there can be operational reasons to do so, and whatever
> they wish to do on their network is perfectly fine. As long as they don't
> bother the rest of the world with it. 

But the whole reason we’re having this conversation is that they _DID_ bother 
the rest of the world with it. Kind of takes the wind out of that particular 
argument, wouldn’t you say?

Owen

Re: Has virtualization become obsolete in 5G?

[Scott Weeks]

--- ed...@ieee.org wrote:
From: Etienne-Victor Depasquale 

See, for example, Azhar Sayeed's (Red Hat) contribution here
@15:33.



Don't send links to this list that require one to register 
to read the article and then say, "By registering for our 
site, your email will be added to our promotions list" and 
"Occasionally our trusted partners may want to send you 
information about exciting new products and services"

No one's going to click on that!

scott

Re: Has virtualization become obsolete in 5G?

[Mark Tinka]

On 1/Aug/20 18:23, Robert Raszuk wrote:

> Virtualization is not becoming obsolete ... quite reverse in fact in
> all types of deployments I can see around. 
>
> The point is that VM provides hardware virtualization while kubernetes
> with containers virtualize OS apps and services are running on in
> isolation. 
>
> Clearly to virtualize operating systems as long as your level of
> virtualization mainly in terms of security and resource consumption
> isolation & reservation is satisfactory is a much better and lighter
> option.

I see cloud-native as NFV++. It requires some adjustment to how classic
NFV has been deployed, and that comes down to whether operators
(especially those who err on the side of network operations rather than
services) see value in upgrading their stack to cloud-native.

If you're a Netflix or an Uber, sure, a cloud-native architecture is
probably the only way you can scale. But if you are simple network
operators who focus more on pushing packets than over-the-top services,
particularly if you already have some NFV, making the move to
cloud-native/NFV++ is a whole consideration.

Mark.

Re: BGP route hijack by AS10990

[Mark Tinka]

On 1/Aug/20 21:31, Owen DeLong wrote:

> I disagree. I think Noction and Telia are both culpable here. Most of the top 
> 200 providers
> manage to do prefix filtering at the customer edge, so I don’t see any reason 
> to give
> Telia a free pass here.

Both Noction and Telia are culpable, because they both (should) know
about past incidents, and how to do their part in protecting against them.

I mean, this is what we talk about on and at *NOG everyday of the year.
It's not like they've been living under a rock.

Mark.

Re: BGP route hijack by AS10990

[Mark Tinka]

On 1/Aug/20 21:20, Owen DeLong wrote:

> IP Prefix level filtering at the customer edge is not that hard, no
> matter how large of a transit
> provider you are. Customer edge filtration by Telia in this case would
> have prevented this
> problem from spreading beyond the misconfigured ASN.

+1.

There's simply no excuse - even if 100% of your eBGP sessions may be
customers :-).

Mark.

Re: BGP route hijack by AS10990

[Mark Tinka]

On 1/Aug/20 21:03, Sabri Berisha wrote:

> The same can be said here. Noction and/or its operators appear to not 
> understand
> how BGP works, and/or what safety measures must be deployed to ensure that the
> larger internet will not be hurt by misconfiguration.

I think the latter would be more appropriate. Their implementation of
BGP is likely correct, but they aren't putting any emphasis on what the
deployment of their use-case can do to global BGP security and
performance. This where I'd say they can add more focus.


> I also agree with Job, that Noction has some responsibility here. And as I
> understand more and more about it, I must now agree with Mark T that this
> was an avoidable incident (although not because of Telia, but because 
> Noction's
> decision to not enable NO_EXPORT by default).

I see it differently.

The chain is only as strong as its weakest actor. It is not unreasonable
to expect that global actors of significant scale have enough clue to
make sure any mistakes committed downstream are not propagated by them
to the rest of the Internet.

So while I do not absolve Noction (and their customer) of any
responsibility here, I'd apportion the blame as:

    - Telia 51%
    - Noction 30%
    - Noction's customer 19%

When the weaker chains of the link fail, we should be able to count on
the strongest chain in that link to be the last line of defence...
Telia, in this case. Simply for no other reason than they "know best",
and have such global scope which comes with significant responsibility.

But that isn't to say that neither Noction nor their customer cannot do
better either. After all, BGP security and performance only works well
when we all do our part, and not just some of us.

Mark.

Re: BGP route hijack by AS10990

[Mark Tinka]

On 1/Aug/20 20:14, Hank Nussbacher wrote:

> AS  level filtering is easy.  IP prefix level filtering is hard. 
> Especially when you are in the top 200:
>
> https://asrank.caida.org/
>

Doesn't immediately make sense to me why prefix filtering is hard.


>
> That being said, and due to these BGP "polluters" constantly doing the
> same thing, wouldn't an easy fix be to use the max-prefix/prefix-limit
> option:
>
> https://www.cisco.com/c/en/us/support/docs/ip/border-gateway-protocol-bgp/25160-bgp-maximum-prefix.html
>
> https://www.juniper.net/documentation/en_US/junos/topics/reference/configuration-statement/prefix-limit-edit-protocols-bgp.html
>
>
> For every BGP peer,  the ISP determines what the current max-prefix
> currently is.  Then add in 2% and set the max-prefix. 
>
> An errant BGP polluter would then only have limited damage to the
> Internet routing table.
>
> Not the greatest solution, but easy to implement via a one line change
> on every BGP peer.
>

It's about combining multiple solutions to ensure several catch-points.
AS_PATH filtering, prefix filtering and max-prefix.


>
> Smaller ISPs can easily do it on their 10 BGP peers so as to limit
> damage as to what they will hear from their neighbors.
>

All ISP's should do this. All ISP's can.

Mark.

Re: Issue with Noction IRP default setting (Was: BGP route hijack by AS10990)

[Ryan Hamel]

Job,

I disagree on the fact that it is not fair to the BGP implementation ecosystem, 
to enforce a single piece of software to activate the no-export community by 
default, due to ignorance from the engineer(s) implementing the solution. It 
should be common sense that certain routes that should be advertised beyond the 
local AS, just like RFC1918 routes, and more. Also, wasn't it you that said 
Cisco routers had a bug in ignoring NO_EXPORT? Would you go on a rant with 
Cisco, even if Noction add that enabled checkbox by default?
Why are you not on your soap box about BIRD, FRrouting, OpenBGPd, Cisco, 
Juniper, etc... about how they can possibly allow every day screw ups to 
happen, but the same options like the NO_EXPORT community are available for the 
engineer to use? One solution would be to implement "BGP Group/Session 
Profiles" (ISP/RTBH/DDOS Filtering/Route Optimizers/etc) or a "BGP Session 
Wizard" (ask the operator questions about their intentions), then automatically 
generate import and export policies based on known accepted practices.
Another solution could be having the BGP daemon disclose the make, model 
family, and exact model of hardware it is running on, to BGP peers, and add 
more knobs into policy creation to match said values, and take action 
appropriately. That would be useful in getting around vendor specific issues, 
as well as belt & suspenders protection.
Ryan
On Aug 1 2020, at 9:58 am, Job Snijders  wrote:
> On Sat, Aug 01, 2020 at 06:50:55AM -0700, Ca By wrote:
> > I am not normally supporting a heavy hand in regulation, but i think it is
> > fair to say Noction and similar BGP optimizers are unsafe at any speed and
> > the FTC or similar should ban them in the USA. They harm consumers and are
> > a risk to national security / critical infrastructure
> >
> > Noction and similar could have set basic defaults (no-export, only create
> > /25 bogus routes to limit scope), but they have been clear that their greed
> > to suck up traffic does not benefit from these defaults and they wont do
> > it.
>
> Following a large scale BGP incident in March 2015, noction made it
> possible to optionally set the well-known NO_EXPORT community on route
> advertisements originated by IRP instances.
>
> "In order to further reduce the likelihood of these problems
> occurring in the future, we will be adding a feature within Noction
> IRP to give an option to tag all the more specific prefixes that it
> generates with the BGP NO_EXPORT community. This will not be enabled
> by default [snip]"
> https://www.noction.com/blog/route-optimizers
> Mar 27, 2015
>
> Due to NO_EXPORT not being set in the default configuration, there are
> probably if not certainly many unsuspecting network engineers who end up
> deploying this software - without ever even considering - to change that
> one setting in the configuration.
>
> Fast forward a few years and a few incidents, on the topic of default
> settings, following the Cloudflare/DQE/Verizon incident:
>
> "We do have no export community support and have done for many
> years. The use of more specifics is also optional. Neither replaces
> the need for filters."
> https://twitter.com/noction/status/1143177562191011840
> Jun 24, 2019
>
> Community members responded:
> "Noction have been facilitating Internet outages for years and
> years and the best thing they can say in response is that it is
> technically possible to use their product responsibly, they just
> don't ship it that way."
> https://twitter.com/PowerDNS_Bert/status/1143252745257979905
> June 24, 2019
>
> Last year Noction stated:
> "Nobody found this leak pleasant."
> https://www.noction.com/news/incident-response
> June 26, 2019
>
> Sentiment we all can agree with, change is needed!
> As far as I know, Noction IRP is the ONLY commercially available
> off-the-shelf BGP route manipulation software which - as default - does
> NOT set the BGP well-known NO_EXPORT community on the product's route
> advertisements. This is a product design decision which causes
> collateral damage.
>
> I would like to urge Noction to reconsider their position. Seek to
> migrate the existing users to use NO_EXPORT, and release a new version
> of the IRP software which sets NO_EXPORT BY DEFAULT on all generated
> routes.
>
> Kind regards,
> Job

Re: BGP route hijack by AS10990

[Nick Hilliard]

Sabri Berisha wrote on 01/08/2020 20:59:

My point is that there can be operational reasons to do so, and whatever
they wish to do on their network is perfectly fine. As long as they don't
bother the rest of the world with it.


I get what you're saying, and am a big fan of personal responsibility, 
but when a vendor ships a product like a BGP optimiser, it requires that 
you run your network with the safety controls removed.


It's no different in principle to shipping guns with the safety welded 
to off, or hot-wiring 20kW cables to bypass your RCDs.  It can produce 
some great results, no doubt about it, but sooner or later you're 
guaranteed that there's going to be a nasty accident.


In any individual case, it's understandable to assign blame to an 
operator for messing up their configs. In the general case, shipping 
products with dangerous-by-default configurations is going lead to more 
accidents happening.


At this point, a large proportion of the major routing leaks on the 
internet can be associated with bgp optimisers and Noction's name 
appears with disturbing regularity.  This is an appalling record, not 
least because it's almost entirely preventable.


Nick

Re: BGP route hijack by AS10990

[Sabri Berisha]

- On Aug 1, 2020, at 12:50 PM, Nick Hilliard n...@foobar.org wrote:

Hi,

> Sabri Berisha wrote on 01/08/2020 20:03:
>> but because Noction's decision to not enable NO_EXPORT by default
> 
> the primary problem is not this but that Noction reinjects prefixes into
> the local ibgp mesh with the as-path stripped and then prioritises these
> prefixes so that they're learned as the best path.

Yeah, but that's not problem as far as I'm concerned. Their network, 
their rules. I've done weirder stuff than that, in tightly controlled
environments.

> The as-path is the primary loop detection mechanism in eBGP.  Removing
> this is like hot-wiring your electrical distribution board because you
> found out you could get more power if you bypass those stupid RCDs.

Well, let's be honest. Sometimes we need to get rid of that pesky mechanism.
For example, when using BGP-as-IGP, the "allowas-in" disregards the as-path,
in a controlled manner (and yes, I know, different use case).

My point is that there can be operational reasons to do so, and whatever
they wish to do on their network is perfectly fine. As long as they don't
bother the rest of the world with it. 

Thanks,

Sabri

Re: BGP route hijack by AS10990

[Nick Hilliard]

Sabri Berisha wrote on 01/08/2020 20:03:

but because Noction's decision to not enable NO_EXPORT by default


the primary problem is not this but that Noction reinjects prefixes into 
the local ibgp mesh with the as-path stripped and then prioritises these 
prefixes so that they're learned as the best path.


The as-path is the primary loop detection mechanism in eBGP.  Removing 
this is like hot-wiring your electrical distribution board because you 
found out you could get more power if you bypass those stupid RCDs.


Once you strip off the as-path in the local view, it's like the AS7007 
incident desperately begging to happen all over again.


As long as route optimiser vendors ship their products with such deeply 
harmful defaults, we're going to continue to see these problems ad nauseam.


Nick

Re: BGP route hijack by AS10990

[Owen DeLong]

> On Aug 1, 2020, at 12:03 , Sabri Berisha  wrote:
> 
> Hi,
> 
> - On Aug 1, 2020, at 8:49 AM, Owen DeLong o...@delong.com wrote:
> 
>> In fact, there are striking parallels between Asiana 214 and this incident.
> 
> Yes. Children of the magenta line. Depending on automation, and no clue what 
> to
> do when the Instrument Landing System goes down.

This wasn’t a case of the ILS going down. This was a case where the automation 
was
put in the wrong mode (accidentally) without any of the pilots in the cockpit 
noticing
it until it was too late. The problem was discovered and power applied 8 
seconds before impact.
It takes 19 seconds for the engines on a 777 to spool up to adequate power for 
a go-around
at the airspeed and in the configuration that existed at the time.

> But, the most important parallel is (hopefully) yet to come. One major 
> outcome of
> the Asiana investigation was the call for more training, as the crew did not
> properly understand how the aircraft worked.

That’s true in virtually every human factors accident, but in reality, failure 
to understand
the automation was a tiny contributing factor in this accident. Every pilot is 
taught
early in their ab initio training that they must monitor the approach carefully 
and make
sure not to bleed off too much energy (airspeed) in the process.

There’s a very common and easily identifiable pattern to an under-powered 
approach
on autopilot that all of the pilots in the cockpit should have readily 
recognized if they
were even paying the slightest attention to the approach…

1. Airplane begins to dip below glide slope.
2. Autopilot raises nose to reduce descent rate and recapture glide slope.
3. Increased pitch = greater induced drag = lower airspeed.
4. Lower airspeed = less lift = goto 1.

Until power is applied, this process will repeat until one of the following 
events occurs:
1.  Landing short of the runway (as in the case of Asiana 214)
2.  Power is applied and the approach is stabilized
3.  The pitch attitude exceeds the critical angle of attack and the 
wings stall,
causing an abrupt pitch down.

This cycle is well understood by every student pilot before they can be endorsed
for their first solo flight.

No amount of training will make up for the utter and complete failure to pay 
attention
to the approach. This is one of the reasons US carriers have a “sterile 
cockpit” rule.

In most cases, the sterile cockpit rule is approximately this: “Below 10,000 
feet or in
other critical phases of flight (emergency situations, unusual climbs or 
descents,
mechanical difficulties, etc.), cockpit communications are limited to those 
related to
the safe operation of the aircraft.”

> The same can be said here. Noction and/or its operators appear to not 
> understand
> how BGP works, and/or what safety measures must be deployed to ensure that the
> larger internet will not be hurt by misconfiguration.

On one level, there’s validity to your claim here. On the other hand, there’s a 
certain
extent to which your telling hammer manufacturers that they have to make it 
impossible
for a carpenter to injure his thumb by missing the nail.

> I also agree with Job, that Noction has some responsibility here. And as I
> understand more and more about it, I must now agree with Mark T that this
> was an avoidable incident (although not because of Telia, but because 
> Noction's
> decision to not enable NO_EXPORT by default).

I disagree. I think Noction and Telia are both culpable here. Most of the top 
200 providers
manage to do prefix filtering at the customer edge, so I don’t see any reason 
to give
Telia a free pass here.

Owen

Re: BGP route hijack by AS10990

[Owen DeLong]

> On Aug 1, 2020, at 11:14 , Hank Nussbacher  wrote:
> 
> On 01/08/2020 00:50, Mark Tinka wrote:
>> On 31/Jul/20 23:38, Sabri Berisha wrote:
>> 
>>> Kudos to Telia for admitting their mistakes, and fixing their processes.
>> Considering Telia's scope and "experience", that is one thing. But for
>> the general good of the Internet, the number of intended or
>> unintentional route hijacks in recent years, and all the noise that
>> rises on this and other lists each time we have such incidents (this
>> won't be the last), Telia should not have waited to be called out in
>> order to get this fixed.
>> 
>> Do we know if they are fixing this on just this customer of theirs, or
>> all their customers? I know this has been their filtering policy with us
>> (SEACOM) since 2014, as I pointed out earlier today. There has not been
>> a shortage of similar incidents between now and then, where the
>> community has consistently called for more deliberate and effective
>> route filtering across inter-AS arrangements.
>> 
>> 
> AS  level filtering is easy.  IP prefix level filtering is hard.  Especially 
> when you are in the top 200:
> https://asrank.caida.org/ 
IP Prefix level filtering at backbone<->backbone connections is hard (and 
mostly pointless).

IP Prefix level filtering at the customer edge is not that hard, no matter how 
large of a transit
provider you are. Customer edge filtration by Telia in this case would have 
prevented this
problem from spreading beyond the misconfigured ASN.

> That being said, and due to these BGP "polluters" constantly doing the same 
> thing, wouldn't an easy fix be to use the max-prefix/prefix-limit option:
> https://www.cisco.com/c/en/us/support/docs/ip/border-gateway-protocol-bgp/25160-bgp-maximum-prefix.html
>  
> 
> https://www.juniper.net/documentation/en_US/junos/topics/reference/configuration-statement/prefix-limit-edit-protocols-bgp.html
>  
> 
That’s a decent pair of suspenders to go with the belt of prefix filtration at 
the edge, but it’s no substitute.

> For every BGP peer,  the ISP determines what the current max-prefix currently 
> is.  Then add in 2% and set the max-prefix. 
> An errant BGP polluter would then only have limited damage to the Internet 
> routing table.
> Not the greatest solution, but easy to implement via a one line change on 
> every BGP peer.

To the best of my knowledge, that’s already fairly common practice. It’s 
usually more like 10% (2% would require way
too much active change and create churn and risk).

Owen

Re: BGP route hijack by AS10990

[Sabri Berisha]

Hi,

- On Aug 1, 2020, at 8:49 AM, Owen DeLong o...@delong.com wrote:

> In fact, there are striking parallels between Asiana 214 and this incident.

Yes. Children of the magenta line. Depending on automation, and no clue what to
do when the Instrument Landing System goes down.

But, the most important parallel is (hopefully) yet to come. One major outcome 
of
the Asiana investigation was the call for more training, as the crew did not
properly understand how the aircraft worked.

The same can be said here. Noction and/or its operators appear to not understand
how BGP works, and/or what safety measures must be deployed to ensure that the
larger internet will not be hurt by misconfiguration.

I also agree with Job, that Noction has some responsibility here. And as I
understand more and more about it, I must now agree with Mark T that this
was an avoidable incident (although not because of Telia, but because Noction's
decision to not enable NO_EXPORT by default).

Thanks,

Sabri

Re: BGP route hijack by AS10990

[Mark Tinka]

On 1/Aug/20 18:46, Owen DeLong wrote:

> ROFLMAO, if you truly believe this, you have no concept of life in the
> cockpit.

I was born into aviation, with both my mom and dad licensed ATPL pilots
for several decades. So I know my way around a number of different
cockpits.

The goal wasn't to turn this thread into an aviation one, but to focus
on what we can do better in Internet operations for more accountability.

Let's stay on-topic, please. Thanks.

Mark.

Re: Has virtualization become obsolete in 5G?

[Mel Beckman]

An operating system is just a high-level machine. That the M-plane in VM is 
implemented in software isn’t relevant, as pretty much all hardware CPUs are 
implemented in software as well, so VM is just virtualizing software already.

Containerization is VM, but using the OS as the M-plane As long as the OS 
delivers all the functions needed by applications, it’s a perfectly reasonable, 
and even preferable, plane to virtualize.

 -mel

On Aug 1, 2020, at 11:12 AM, Etienne-Victor Depasquale  wrote:


Clearly to virtualize operating systems as long as your level of virtualization 
mainly in terms of security and resource consumption isolation & reservation is 
satisfactory is a much better and lighter option.

That pretty much sums up Intel's view.

To quote an Intel executive I was corresponding with:

"The purpose of the paper was to showcase how Communication Service Providers 
can move to a more nimble and future proof microservices based network 
architecture with cloud native functions, via container deployment 
methodologies versus virtual machines.  The paper cites many benefits of moving 
to a microservices architecture beyond whether it is done in a VM environment 
or cloud native. We believe the 5G networks of the future will benefit greatly 
by implementing such an approach to deploying new services."

The paper referred to is this 
one.

Cheers,

Etienne

On Sat, Aug 1, 2020 at 6:23 PM Robert Raszuk 
mailto:rob...@raszuk.net>> wrote:
I reason that Intel's implication is that virtualization is becoming obsolete.
Would anyone care to let me know his thoughts on this prediction?

Virtualization is not becoming obsolete ... quite reverse in fact in all types 
of deployments I can see around.

The point is that VM provides hardware virtualization while kubernetes with 
containers virtualize OS apps and services are running on in isolation.

Clearly to virtualize operating systems as long as your level of virtualization 
mainly in terms of security and resource consumption isolation & reservation is 
satisfactory is a much better and lighter option.

Thx,
R.



--
Ing. Etienne-Victor Depasquale
Assistant Lecturer
Department of Communications & Computer Engineering
Faculty of Information & Communication Technology
University of Malta
Web. https://www.um.edu.mt/profile/etiennedepasquale

Re: BGP route hijack by AS10990

[Hank Nussbacher]

  
  
On 01/08/2020 00:50, Mark Tinka wrote:


  

On 31/Jul/20 23:38, Sabri Berisha wrote:


  
Kudos to Telia for admitting their mistakes, and fixing their processes.

  
  
Considering Telia's scope and "experience", that is one thing. But for
the general good of the Internet, the number of intended or
unintentional route hijacks in recent years, and all the noise that
rises on this and other lists each time we have such incidents (this
won't be the last), Telia should not have waited to be called out in
order to get this fixed.

Do we know if they are fixing this on just this customer of theirs, or
all their customers? I know this has been their filtering policy with us
(SEACOM) since 2014, as I pointed out earlier today. There has not been
a shortage of similar incidents between now and then, where the
community has consistently called for more deliberate and effective
route filtering across inter-AS arrangements.




AS  level filtering is easy.  IP prefix level filtering is hard. 
  Especially when you are in the top 200:
https://asrank.caida.org/


That being said, and due to these BGP "polluters" constantly
  doing the same thing, wouldn't an easy fix be to use the
  max-prefix/prefix-limit option:
https://www.cisco.com/c/en/us/support/docs/ip/border-gateway-protocol-bgp/25160-bgp-maximum-prefix.html
https://www.juniper.net/documentation/en_US/junos/topics/reference/configuration-statement/prefix-limit-edit-protocols-bgp.html


For every BGP peer,  the ISP determines what the current
  max-prefix currently is.  Then add in 2% and set the max-prefix. 

An errant BGP polluter would then only have limited damage to the
  Internet routing table.
Not the greatest solution, but easy to implement via a one line
  change on every BGP peer.


Smaller ISPs can easily do it on their 10 BGP peers so as to
  limit damage as to what they will hear from their neighbors.



-Hank


Caveat: The views expressed above are solely my own and do not
  express the views or opinions of my employer 



  

Re: Has virtualization become obsolete in 5G?

[Etienne-Victor Depasquale]

>
> Clearly to virtualize operating systems as long as your level of
> virtualization mainly in terms of security and resource consumption
> isolation & reservation is satisfactory is a much better and lighter
> option.
>

That pretty much sums up Intel's view.

To quote an Intel executive I was corresponding with:

"The purpose of the paper was to showcase how Communication Service
Providers can move to a more nimble and future proof microservices based
network architecture with cloud native functions, via container deployment
methodologies versus virtual machines.  The paper cites many benefits of
moving to a microservices architecture beyond whether it is done in a VM
environment or cloud native. We believe the 5G networks of the future will
benefit greatly by implementing such an approach to deploying new services."

The paper referred to is this one

.

Cheers,

Etienne

On Sat, Aug 1, 2020 at 6:23 PM Robert Raszuk  wrote:

> I reason that Intel's implication is that virtualization is becoming
>> obsolete.
>> Would anyone care to let me know his thoughts on this prediction?
>>
>
> Virtualization is not becoming obsolete ... quite reverse in fact in all
> types of deployments I can see around.
>
> The point is that VM provides hardware virtualization while kubernetes
> with containers virtualize OS apps and services are running on in
> isolation.
>
> Clearly to virtualize operating systems as long as your level of
> virtualization mainly in terms of security and resource consumption
> isolation & reservation is satisfactory is a much better and lighter
> option.
>
> Thx,
> R.
>
>


-- 
Ing. Etienne-Victor Depasquale
Assistant Lecturer
Department of Communications & Computer Engineering
Faculty of Information & Communication Technology
University of Malta
Web. https://www.um.edu.mt/profile/etiennedepasquale

Issue with Noction IRP default setting (Was: BGP route hijack by AS10990)

[Job Snijders]

On Sat, Aug 01, 2020 at 06:50:55AM -0700, Ca By wrote:
> I am not normally supporting a heavy hand in regulation, but i think it is
> fair to say Noction and similar BGP optimizers are unsafe at any speed and
> the FTC or similar should ban them in the USA. They harm consumers and are
> a risk to national security / critical infrastructure
> 
> Noction and similar could have set basic defaults (no-export, only create
> /25 bogus routes to limit scope), but they have been clear that their greed
> to suck up traffic does not benefit from these defaults and they wont do
> it.

Following a large scale BGP incident in March 2015, noction made it
possible to optionally set the well-known NO_EXPORT community on route
advertisements originated by IRP instances.

"In order to further reduce the likelihood of these problems
occurring in the future, we will be adding a feature within Noction
IRP to give an option to tag all the more specific prefixes that it
generates with the BGP NO_EXPORT community. This will not be enabled
by default [snip]"
https://www.noction.com/blog/route-optimizers
Mar 27, 2015

Due to NO_EXPORT not being set in the default configuration, there are
probably if not certainly many unsuspecting network engineers who end up
deploying this software - without ever even considering - to change that
one setting in the configuration.

Fast forward a few years and a few incidents, on the topic of default
settings, following the Cloudflare/DQE/Verizon incident:

"We do have no export community support and have done for many
years. The use of more specifics is also optional. Neither replaces
the need for filters."
https://twitter.com/noction/status/1143177562191011840
Jun 24, 2019

Community members responded:

"Noction have been facilitating Internet outages for years and
years and the best thing they can say in response is that it is
technically possible to use their product responsibly, they just
don't ship it that way."
https://twitter.com/PowerDNS_Bert/status/1143252745257979905
June 24, 2019

Last year Noction stated:

"Nobody found this leak pleasant."
https://www.noction.com/news/incident-response
June 26, 2019

Sentiment we all can agree with, change is needed!

As far as I know, Noction IRP is the ONLY commercially available
off-the-shelf BGP route manipulation software which - as default - does
NOT set the BGP well-known NO_EXPORT community on the product's route
advertisements. This is a product design decision which causes
collateral damage.

I would like to urge Noction to reconsider their position. Seek to
migrate the existing users to use NO_EXPORT, and release a new version
of the IRP software which sets NO_EXPORT BY DEFAULT on all generated
routes.

Kind regards,

Job

Re: BGP route hijack by AS10990

[Owen DeLong]

> On Aug 1, 2020, at 09:09 , Mark Tinka  wrote:
> 
> 
> 
> On 1/Aug/20 17:49, Owen DeLong wrote:
> 
>> Aviation makes a strong effort in this area, perhaps stronger than any other
>> human endeavor, especially when you’re talking about the fraction of
>> Aviation known in the US as “Part 121 Scheduled Air Carrier Services”.
>> 
>> However, as noted above, there are exceptions.
>> 
>> In fact, there are striking parallels between Asiana 214 and this incident.
>> 
>> The tools to avoid the accident in question automatically were available to 
>> the
>> pilots, but they failed to turn them on (autothrottle).
>> 
>> The tools to avoid this incident were available to Telia, but they
>> failed to turn them on.
> 
> Agreed, the leading cause of aircraft incidents is human error. When
> human errors in aeroplane accidents are repeated, it's usually because
> of poor crew resource management, poor training, low experience, poor
> situational awareness, crew fatigue, crew disorientation, not following
> checklists... that sort of thing.

Let’s be clear… This was not an incident, it was an accident.

In the US at least, under 49CFR§830.2 the two are specifically defined
as follows:

Aircraft accident means an occurrence associated with the operation of an 
aircraft
which takes place between the time any person boards the aircraft with the 
intention
of flight and all such persons have disembarked, and in which any person 
suffers death
or serious injury, or in which the aircraft receives substantial damage. For 
purposes of
this part, the definition of “aircraft accident” includes “unmanned aircraft 
accident,”
as defined herein.

Serious injury is defined in my previous message (reference to the same code 
section)

Substantial damage is defined as “damage or failure which adversely affects the
structural strength, performance, or flight characteristics of the aircraft and 
which
would normally require major repair or replacement of the affected component. 
Engine failure or damage limited
to an engine if only one engine fails or is damaged, bent fairings or cowling, 
dented skin, small punctured holses
in the skin or fabric, ground damage to rotor or propeller blade, and damage to 
landing gear, wheels, tires, flaps,
engine accessories, brakes, or wingtips are not considered “substantial damage” 
for the purpose of this part.

An “Incident” is an occurrence other than an accident, associated with the 
operation of an aircraft,
which affects or could affect the safety of operations.

> We've made a whole hymn out of "do proper filtering at eBGP hand-off
> points" over the years. Network operators are not always working under
> pressure like airline pilots do. On a quiet, calm afternoon, an engineer
> can comb the network to make sure all potential mistakes that have been
> shouted about for years within our community are plugged, especially
> when working at an "experienced" operation such as Telia and similar.

Airline pilots are not always under pressure, either. In fact, airline flying is
90% boredom, 9+% routine operations (procedures for preparation and
departure, departure and climb-out, preparations for approach and landing,
descent, approach, and landing) and <1% actual pressure (IROPS,
in-flight emergencies, etc.).

I say this not only as someone who’s spent a lot of time as a passenger,
but also as a commercial instrument-rated pilot.

> It's almost a "do once and forget, and watch it repeat" type-thing, vs.
> airline pilots who need to be on it 110%, every second of every flight,
> even if they've got 25,000hrs under their epaulettes.

ROFLMAO, if you truly believe this, you have no concept of life in the
cockpit.

Yes, airline pilots need to be paying attention even in the most routine
phases of flight, but in reality, 90+% of every flight is routine monitoring
of systems, essentially checking the “Ts”…
Time:   Is the flight progressing as expected
Are we where we expected to be at this time?
Is the fuel consumption in line with our expectations?
Turn:   Are we on course?
How far to the next heading change?
Throttle:   Is our performance correct for this point in the flight?
Are we at the desired altitude, attitude, power, and airspeed?
If applicable, are the auto throttles in the correct mode?
Twist:  Are any adjustments or preparations on Radios/Navigation/FMS 
needed?
This is where you check to make sure that not only are you on 
the correct
frequency now (com, navigation, etc.), but that you 
also have things set
for the next change.
It’s also where you make those changes (e.g. flip to the next 
VOR) if that’s
due.
Track:  How does our track compare to our intended course.
This is where the heading is adjusted, if necessary, to achieve 
the desired course.
 

Re: Has virtualization become obsolete in 5G?

[Robert Raszuk]

>
> I reason that Intel's implication is that virtualization is becoming
> obsolete.
> Would anyone care to let me know his thoughts on this prediction?
>

Virtualization is not becoming obsolete ... quite reverse in fact in all
types of deployments I can see around.

The point is that VM provides hardware virtualization while kubernetes with
containers virtualize OS apps and services are running on in isolation.

Clearly to virtualize operating systems as long as your level of
virtualization mainly in terms of security and resource consumption
isolation & reservation is satisfactory is a much better and lighter
option.

Thx,
R.

Re: BGP route hijack by AS10990

[Mark Tinka]

On 1/Aug/20 17:49, Owen DeLong wrote:

> Aviation makes a strong effort in this area, perhaps stronger than any other
> human endeavor, especially when you’re talking about the fraction of
> Aviation known in the US as “Part 121 Scheduled Air Carrier Services”.
>
> However, as noted above, there are exceptions.
>
> In fact, there are striking parallels between Asiana 214 and this incident.
>
> The tools to avoid the accident in question automatically were available to 
> the
> pilots, but they failed to turn them on (autothrottle).
>
> The tools to avoid this incident were available to Telia, but they
> failed to turn them on.

Agreed, the leading cause of aircraft incidents is human error. When
human errors in aeroplane accidents are repeated, it's usually because
of poor crew resource management, poor training, low experience, poor
situational awareness, crew fatigue, crew disorientation, not following
checklists... that sort of thing.

We've made a whole hymn out of "do proper filtering at eBGP hand-off
points" over the years. Network operators are not always working under
pressure like airline pilots do. On a quiet, calm afternoon, an engineer
can comb the network to make sure all potential mistakes that have been
shouted about for years within our community are plugged, especially
when working at an "experienced" operation such as Telia and similar.

It's almost a "do once and forget, and watch it repeat" type-thing, vs.
airline pilots who need to be on it 110%, every second of every flight,
even if they've got 25,000hrs under their epaulettes.

It shouldn't be this hard...

Mark.

Re: BGP route hijack by AS10990

[Owen DeLong]

> On Aug 1, 2020, at 04:20 , Mark Tinka  wrote:
> 
> 
> 
> On 1/Aug/20 02:17, Sabri Berisha wrote:
> 
>> I'm not sure if you read their entire Mea Culpa, but they did indicate that
>> the root cause of this issue was the provisioning of a legacy filter that
>> they are no longer using. So effectively, that makes it a human error.
>> 
>> We're going to a point where a single error is no longer causing outages,
>> something very similar to my favorite analogy: avation. Pretty much every
>> major air disaster was caused by a combination of factors. Pretty much
>> every major outage these days is caused by a combination of factors.
>> 
>> The manual provisioning of an inadequate filter, combined with an
>> automation error on the side of a customer (which by itself was probably
>> caused by a combination of factors), caused this issue.
>> 
>> We learn from every outage. And instead of radio silence, they fessed up
>> and fixed the issue. Have a look at the ASRS program :)
> 
> What I meant by "TOTALLY avoidable" is that "this particular plane
> crash" has happened in the exact same way, for the exact same reasons,
> over and over again.

That’s also true of Asiana 214. 
(Root cause: 5 pilots failed to pay attention to the approach)

https://www.ntsb.gov/investigations/AccidentReports/Reports/AAR1401.pdf

(The full report probably only interests pilots, but the executive summary on
pages xi - xv is a good read).

Worth noting, contrary to the public perception of airline accidents, despite 
the
near total destruction of the airframe in this incident, 288 of 291 passengers
and all of the crew survived. Of the 307 people on board, only 49 suffered
serious injuries. (serious is defined as an injury requiring >48 hours of
hospitalization within 7 days of the accident in which the injury was 
sustained).
(49CFR§830.2)

For those that find 5 pages of type TL;DR, the key findings are in the first
paragraph after the last bullet point on page xv.

> Aviation learns from mistakes that don't generally recur in the exact
> same way for the exact same reasons.

Aviation makes a strong effort in this area, perhaps stronger than any other
human endeavor, especially when you’re talking about the fraction of
Aviation known in the US as “Part 121 Scheduled Air Carrier Services”.

However, as noted above, there are exceptions.

In fact, there are striking parallels between Asiana 214 and this incident.

The tools to avoid the accident in question automatically were available to the
pilots, but they failed to turn them on (autothrottle).

The tools to avoid this incident were available to Telia, but they
failed to turn them on.

Owen

Re: Has virtualization become obsolete in 5G?

[Mark Tinka]

On 1/Aug/20 16:52, Etienne-Victor Depasquale wrote:
> But the point is just that: how serious is this progression towards
> cloud-native, if so much effort was put in to virtualization?

I suspect that if a significant amount of investment has already gone
into classic NFV, and for the most part, it's working reasonably well,
an operation would need to be seriously bored or have tons of cash and
time around to uproot all of that work and change things around without
some compelling technical or commercial reason to do so.

Despite the NFV world being well bedded in, it's still an evolving piece
of tech., and this is one field where operators are prone to spending
multiple times on the same thing, as they realize the previous decision
fell out of favour with the community or their favorite vendor.

I've seen it happen right here in South Africa, when a company built an
"SDN" platform 7 different times in 3 years as the industry kept
oscillating; going through whatever "SDN" platform vendors pushed, what
the open community was putting out, OpenStack, e.t.c.

They eventually closed down that side of the business, this year.

So for greenfield sites, maybe. But for existing installations that have
been around a while, I guess the transition to "cloud-native" might be a
bit of an ask, given the industry's history on this.

Mark.

Re: BGP route hijack by AS10990

[Mark Tinka]

On 1/Aug/20 16:44, Nick Hilliard wrote:

> ... so once again, route optimisers were at the heart of another
> serious route leaking incident.
>
> BGP is designed to prevent loops from happening, and has tools like
> no-export to help prevent inadvertent leaks.
>
> When people build "BGP optimisers" which reinject a prefix into a
> routing mesh with the entire as-path stripped and then they refuse to
> apply the basic minimum of common sense by refusing point blank to tag
> prefixes with no-export, it's a matter of certainty that leaks are
> going to happen, and that when they do, they'll cause damage.
>
> It's about as responsible as shipping a shotgun with the safety
> disabled and then handing it to a newbie.  After all, the safety makes
> it more difficult to operate and if the newbie shoots themselves, it
> was their fault.  And if they shot someone else, they shouldn't have
> got in the way, right?

All in all, agreed.

While gun ownership and use is highly regulated (and penalized if
violated) in almost all countries, it suffers the same problem as folk
that have access to and drive cars without a valid license.

In our case, we don't really have anything beyond person-to-person trust
in doing their part to not only adhere to global BCOP's for BGP
operation, but to also understand what they are doing with the equipment
they have, as well as the BGP protocol itself.

Without some plan in place to make sure BGP actors do so with sufficient
knowledge and care, these problems are only going to worsen as the next
crop of network engineers prefer a BGP optimizer with a point & click
GUI to actually understanding BGP Multi-Homing principles and techniques.

I'm not opposed to Cameron's suggestion on how to deal with BGP
optimizers :-).

The issue of correctly filtering at eBGP hand-off points has been beaten
to death probably longer than I have been a member of this mailing list.
So...

Mark.

Re: Has virtualization become obsolete in 5G?

[Mark Tinka]

On 1/Aug/20 16:33, Ca By wrote:

>
> Be careful not to confuse vendors pumping stuff with whats actually
> deployed.

Words of wisdom.

Mark.

Re: BGP route hijack by AS10990

[Mark Tinka]

On 1/Aug/20 15:50, Ca By wrote:

>
> Aviation is regulated.

Which is my point. While, like you, I am not in support in heavy-handed
regulation like most life & death industries require, we also can't be
leaving our industry open for any actor to do as they please.


>
> I am not normally supporting a heavy hand in regulation, but i think
> it is fair to say Noction and similar BGP optimizers are unsafe at any
> speed and the FTC or similar should ban them in the USA. They harm
> consumers and are a risk to national security / critical infrastructure
>
> Noction and similar could have set basic defaults (no-export, only
> create /25 bogus routes to limit scope), but they have been clear that
> their greed to suck up traffic does not benefit from these defaults
> and they wont do it. 
>
> Tar and feather them. FTC, do your job. 
>
> FTC has done good work before 
> https://www.ftc.gov/news-events/press-releases/2017/01/ftc-charges-d-link-put-consumers-privacy-risk-due-inadequate
>
> Noction — delete your account

+1.

Mark.

Re: Has virtualization become obsolete in 5G?

[Etienne-Victor Depasquale]

>
> Be careful not to confuse vendors pumping stuff with whats actually
> deployed.
>
Well yes, there's always the hype factor to discount. The reason why I'm
asking this forum is to separate hype from hope.

Also, AT has been doing virtualization for nearly 10 years now, so
> perhaps you were just not paying attention

But the point is just that: how serious is this progression towards
cloud-native, if so much effort was put in to virtualization?

Incidentally, AT's Brian Bearden was present here
: just
listen to how he defended Intel's containerization drive @24:56.

>
>
On Sat, Aug 1, 2020 at 4:33 PM Ca By  wrote:

>
>
> On Sat, Aug 1, 2020 at 7:21 AM Etienne-Victor Depasquale 
> wrote:
>
>> The surprise for me regards Intel's (and the entire Cloud Native
>> Computing Foundation's?) readiness to move past network functions run on
>> VMs
>> and towards network functions run as microservices in containers.
>>
>> See, for example, Azhar Sayeed's (Red Hat) contribution here
>> @15:33.
>>
>
> Be careful not to confuse vendors pumping stuff with whats actually
> deployed.
>
> Also, AT has been doing virtualization for nearly 10 years now, so
> perhaps you were just not paying attention
>
>
> https://www.fiercetelecom.com/telecom/at-t-target-for-virtualizing-75-its-network-by-2020
>
> Not sure it has helped ATT in any meaningful way, their stock price  is
> the same it was in 2015.
>
>
>> Cheers,
>>
>> Etienne
>>
>> On Sat, Aug 1, 2020 at 2:35 PM Mark Tinka  wrote:
>>
>>>
>>>
>>> On 1/Aug/20 11:23, Etienne-Victor Depasquale wrote:
>>>
>>> Over the past few weeks, I've attended webinars and watched videos
>>> organized by Intel.
>>> These activities have centred on 5G and examined applications (like
>>> "visual cloud" and "gaming"),
>>> as well as segment-oriented aspects (like edge networks, 5G RAN and 5G
>>> Core).
>>>
>>> I am stunned (no hyperbole) by the emphasis on Kubernetes in particular,
>>> and cloud-native computing in general.
>>> Equally stunning (for me), public telecommunications networks have been
>>> portrayed
>>> as having a history that moved from integrated software and hardware,
>>> to virtualization and now to cloud-native computing.
>>> See, for example Alex Quach, here
>>> 
>>>  @10:30).
>>> I reason that Intel's implication is that virtualization is becoming
>>> obsolete.
>>>
>>> Would anyone care to let me know his thoughts on this prediction?
>>>
>>>
>>> In the early dawn of SDN, where it was cool to have the RP's in Beirut
>>> and the line cards in Lagos, the industry quickly realized that was not
>>> entirely feasible.
>>>
>>> If you are looking at over-the-top services, so-called cloud-native
>>> computing makes sense in order to deliver that value accordingly, and with
>>> agility. But as it pertains to actual network transport, I'm not yet sure
>>> the industry is at the stage where we are confident enough to decompose
>>> packet forwarding through a cloud.
>>>
>>> Network operators are more likely to keep using kit that integrates
>>> forwarding hardware as well as a NOS, as no amount of cloud architecting is
>>> going to rival a 100Gbps purpose-built port, for example.
>>>
>>> Suffice it to say, there was a time when folk were considering running
>>> their critical infrastructure (such as your route reflectors) in AWS or
>>> similar. I'm not quite sure public clouds are at that level of confidence
>>> yet. So if some kind of cloud-native infrastructure is to be considered for
>>> critical infrastructure, I highly suspect it will be in-house.
>>>
>>> On the other hand, for any new budding entrepreneurs that want to get
>>> into the mobile game with as little cost as possible, there is a huge
>>> opportunity to do so by building all that infrastructure in an on-prem
>>> cloud-native architecture, and offer packet forwarding using
>>> general-purpose hardware provided they don't exceed their expectations.
>>> This way, they wouldn't have to deal with the high costs traditional
>>> vendors (Ericsson, Nokia, Huawei, Siemens, ZTE, e.t.c.) impose. Granted, it
>>> would be small scale, but maybe that is the business model. And in an
>>> industry where capex is fast out-pacing revenue, it would be the mobile
>>> network equivalent of low-cost carrier airlines.
>>>
>>> I very well could be talking out the side of my neck, but my prediction
>>> is mobile operators will be optimistic but cautious. I reckon a healthy mix
>>> between cloud-native and tried & tested practices.
>>>
>>> Mark.
>>>
>>
>>
>> --
>> Ing. Etienne-Victor Depasquale
>> Assistant Lecturer
>> Department of Communications & Computer Engineering
>> Faculty of Information & Communication Technology
>> University of Malta
>> Web. https://www.um.edu.mt/profile/etiennedepasquale
>>
>

-- 

Re: BGP route hijack by AS10990

[Nick Hilliard]

Mark Tinka wrote on 01/08/2020 12:20:

The difference between us and aviation is that fundamental flaws or
mistakes that impact safety are required to be fixed and checked if you
want to keep operating in the industry. We don't have that, so...


... so once again, route optimisers were at the heart of another serious 
route leaking incident.


BGP is designed to prevent loops from happening, and has tools like 
no-export to help prevent inadvertent leaks.


When people build "BGP optimisers" which reinject a prefix into a 
routing mesh with the entire as-path stripped and then they refuse to 
apply the basic minimum of common sense by refusing point blank to tag 
prefixes with no-export, it's a matter of certainty that leaks are going 
to happen, and that when they do, they'll cause damage.


It's about as responsible as shipping a shotgun with the safety disabled 
and then handing it to a newbie.  After all, the safety makes it more 
difficult to operate and if the newbie shoots themselves, it was their 
fault.  And if they shot someone else, they shouldn't have got in the 
way, right?


Nick

Re: Has virtualization become obsolete in 5G?

[Ca By]

On Sat, Aug 1, 2020 at 7:21 AM Etienne-Victor Depasquale 
wrote:

> The surprise for me regards Intel's (and the entire Cloud Native Computing
> Foundation's?) readiness to move past network functions run on VMs
> and towards network functions run as microservices in containers.
>
> See, for example, Azhar Sayeed's (Red Hat) contribution here
> @15:33.
>

Be careful not to confuse vendors pumping stuff with whats actually
deployed.

Also, AT has been doing virtualization for nearly 10 years now, so
perhaps you were just not paying attention

https://www.fiercetelecom.com/telecom/at-t-target-for-virtualizing-75-its-network-by-2020

Not sure it has helped ATT in any meaningful way, their stock price  is the
same it was in 2015.


> Cheers,
>
> Etienne
>
> On Sat, Aug 1, 2020 at 2:35 PM Mark Tinka  wrote:
>
>>
>>
>> On 1/Aug/20 11:23, Etienne-Victor Depasquale wrote:
>>
>> Over the past few weeks, I've attended webinars and watched videos
>> organized by Intel.
>> These activities have centred on 5G and examined applications (like
>> "visual cloud" and "gaming"),
>> as well as segment-oriented aspects (like edge networks, 5G RAN and 5G
>> Core).
>>
>> I am stunned (no hyperbole) by the emphasis on Kubernetes in particular,
>> and cloud-native computing in general.
>> Equally stunning (for me), public telecommunications networks have been
>> portrayed
>> as having a history that moved from integrated software and hardware,
>> to virtualization and now to cloud-native computing.
>> See, for example Alex Quach, here
>> 
>>  @10:30).
>> I reason that Intel's implication is that virtualization is becoming
>> obsolete.
>>
>> Would anyone care to let me know his thoughts on this prediction?
>>
>>
>> In the early dawn of SDN, where it was cool to have the RP's in Beirut
>> and the line cards in Lagos, the industry quickly realized that was not
>> entirely feasible.
>>
>> If you are looking at over-the-top services, so-called cloud-native
>> computing makes sense in order to deliver that value accordingly, and with
>> agility. But as it pertains to actual network transport, I'm not yet sure
>> the industry is at the stage where we are confident enough to decompose
>> packet forwarding through a cloud.
>>
>> Network operators are more likely to keep using kit that integrates
>> forwarding hardware as well as a NOS, as no amount of cloud architecting is
>> going to rival a 100Gbps purpose-built port, for example.
>>
>> Suffice it to say, there was a time when folk were considering running
>> their critical infrastructure (such as your route reflectors) in AWS or
>> similar. I'm not quite sure public clouds are at that level of confidence
>> yet. So if some kind of cloud-native infrastructure is to be considered for
>> critical infrastructure, I highly suspect it will be in-house.
>>
>> On the other hand, for any new budding entrepreneurs that want to get
>> into the mobile game with as little cost as possible, there is a huge
>> opportunity to do so by building all that infrastructure in an on-prem
>> cloud-native architecture, and offer packet forwarding using
>> general-purpose hardware provided they don't exceed their expectations.
>> This way, they wouldn't have to deal with the high costs traditional
>> vendors (Ericsson, Nokia, Huawei, Siemens, ZTE, e.t.c.) impose. Granted, it
>> would be small scale, but maybe that is the business model. And in an
>> industry where capex is fast out-pacing revenue, it would be the mobile
>> network equivalent of low-cost carrier airlines.
>>
>> I very well could be talking out the side of my neck, but my prediction
>> is mobile operators will be optimistic but cautious. I reckon a healthy mix
>> between cloud-native and tried & tested practices.
>>
>> Mark.
>>
>
>
> --
> Ing. Etienne-Victor Depasquale
> Assistant Lecturer
> Department of Communications & Computer Engineering
> Faculty of Information & Communication Technology
> University of Malta
> Web. https://www.um.edu.mt/profile/etiennedepasquale
>

Re: Has virtualization become obsolete in 5G?

[Etienne-Victor Depasquale]

The surprise for me regards Intel's (and the entire Cloud Native Computing
Foundation's?) readiness to move past network functions run on VMs
and towards network functions run as microservices in containers.

See, for example, Azhar Sayeed's (Red Hat) contribution here
@15:33.

Cheers,

Etienne

On Sat, Aug 1, 2020 at 2:35 PM Mark Tinka  wrote:

>
>
> On 1/Aug/20 11:23, Etienne-Victor Depasquale wrote:
>
> Over the past few weeks, I've attended webinars and watched videos
> organized by Intel.
> These activities have centred on 5G and examined applications (like
> "visual cloud" and "gaming"),
> as well as segment-oriented aspects (like edge networks, 5G RAN and 5G
> Core).
>
> I am stunned (no hyperbole) by the emphasis on Kubernetes in particular,
> and cloud-native computing in general.
> Equally stunning (for me), public telecommunications networks have been
> portrayed
> as having a history that moved from integrated software and hardware,
> to virtualization and now to cloud-native computing.
> See, for example Alex Quach, here
> 
>  @10:30).
> I reason that Intel's implication is that virtualization is becoming
> obsolete.
>
> Would anyone care to let me know his thoughts on this prediction?
>
>
> In the early dawn of SDN, where it was cool to have the RP's in Beirut and
> the line cards in Lagos, the industry quickly realized that was not
> entirely feasible.
>
> If you are looking at over-the-top services, so-called cloud-native
> computing makes sense in order to deliver that value accordingly, and with
> agility. But as it pertains to actual network transport, I'm not yet sure
> the industry is at the stage where we are confident enough to decompose
> packet forwarding through a cloud.
>
> Network operators are more likely to keep using kit that integrates
> forwarding hardware as well as a NOS, as no amount of cloud architecting is
> going to rival a 100Gbps purpose-built port, for example.
>
> Suffice it to say, there was a time when folk were considering running
> their critical infrastructure (such as your route reflectors) in AWS or
> similar. I'm not quite sure public clouds are at that level of confidence
> yet. So if some kind of cloud-native infrastructure is to be considered for
> critical infrastructure, I highly suspect it will be in-house.
>
> On the other hand, for any new budding entrepreneurs that want to get into
> the mobile game with as little cost as possible, there is a huge
> opportunity to do so by building all that infrastructure in an on-prem
> cloud-native architecture, and offer packet forwarding using
> general-purpose hardware provided they don't exceed their expectations.
> This way, they wouldn't have to deal with the high costs traditional
> vendors (Ericsson, Nokia, Huawei, Siemens, ZTE, e.t.c.) impose. Granted, it
> would be small scale, but maybe that is the business model. And in an
> industry where capex is fast out-pacing revenue, it would be the mobile
> network equivalent of low-cost carrier airlines.
>
> I very well could be talking out the side of my neck, but my prediction is
> mobile operators will be optimistic but cautious. I reckon a healthy mix
> between cloud-native and tried & tested practices.
>
> Mark.
>


-- 
Ing. Etienne-Victor Depasquale
Assistant Lecturer
Department of Communications & Computer Engineering
Faculty of Information & Communication Technology
University of Malta
Web. https://www.um.edu.mt/profile/etiennedepasquale

Re: BGP route hijack by AS10990

[Ca By]

On Sat, Aug 1, 2020 at 4:21 AM Mark Tinka  wrote:

>
>
> What I meant by "TOTALLY avoidable" is that "this particular plane
> crash" has happened in the exact same way, for the exact same reasons,
> over and over again.
>
> Aviation learns from mistakes that don't generally recur in the exact
> same way for the exact same reasons.


Aviation is regulated.

I am not normally supporting a heavy hand in regulation, but i think it is
fair to say Noction and similar BGP optimizers are unsafe at any speed and
the FTC or similar should ban them in the USA. They harm consumers and are
a risk to national security / critical infrastructure

Noction and similar could have set basic defaults (no-export, only create
/25 bogus routes to limit scope), but they have been clear that their greed
to suck up traffic does not benefit from these defaults and they wont do
it.

Tar and feather them. FTC, do your job.

FTC has done good work before
https://www.ftc.gov/news-events/press-releases/2017/01/ftc-charges-d-link-put-consumers-privacy-risk-due-inadequate

Noction — delete your account

Re: Has virtualization become obsolete in 5G?

[Mark Tinka]

On 1/Aug/20 11:23, Etienne-Victor Depasquale wrote:
> Over the past few weeks, I've attended webinars and watched videos
> organized by Intel. 
> These activities have centred on 5G and examined applications (like
> "visual cloud" and "gaming"), 
> as well as segment-oriented aspects (like edge networks, 5G RAN and 5G
> Core).
>
> I am stunned (no hyperbole) by the emphasis on Kubernetes in particular,
> and cloud-native computing in general. 
> Equally stunning (for me), public telecommunications networks have
> been portrayed 
> as having a history that moved from integrated software and hardware, 
> to virtualization and now to cloud-native computing. 
> See, for example Alex Quach, here
> 
>  @10:30).
> I reason that Intel's implication is that virtualization is becoming
> obsolete.
>
> Would anyone care to let me know his thoughts on this prediction?

In the early dawn of SDN, where it was cool to have the RP's in Beirut
and the line cards in Lagos, the industry quickly realized that was not
entirely feasible.

If you are looking at over-the-top services, so-called cloud-native
computing makes sense in order to deliver that value accordingly, and
with agility. But as it pertains to actual network transport, I'm not
yet sure the industry is at the stage where we are confident enough to
decompose packet forwarding through a cloud.

Network operators are more likely to keep using kit that integrates
forwarding hardware as well as a NOS, as no amount of cloud architecting
is going to rival a 100Gbps purpose-built port, for example.

Suffice it to say, there was a time when folk were considering running
their critical infrastructure (such as your route reflectors) in AWS or
similar. I'm not quite sure public clouds are at that level of
confidence yet. So if some kind of cloud-native infrastructure is to be
considered for critical infrastructure, I highly suspect it will be
in-house.

On the other hand, for any new budding entrepreneurs that want to get
into the mobile game with as little cost as possible, there is a huge
opportunity to do so by building all that infrastructure in an on-prem
cloud-native architecture, and offer packet forwarding using
general-purpose hardware provided they don't exceed their expectations.
This way, they wouldn't have to deal with the high costs traditional
vendors (Ericsson, Nokia, Huawei, Siemens, ZTE, e.t.c.) impose. Granted,
it would be small scale, but maybe that is the business model. And in an
industry where capex is fast out-pacing revenue, it would be the mobile
network equivalent of low-cost carrier airlines.

I very well could be talking out the side of my neck, but my prediction
is mobile operators will be optimistic but cautious. I reckon a healthy
mix between cloud-native and tried & tested practices.

Mark.

Re: BGP route hijack by AS10990

[Mark Tinka]

On 1/Aug/20 02:17, Sabri Berisha wrote:

> I'm not sure if you read their entire Mea Culpa, but they did indicate that
> the root cause of this issue was the provisioning of a legacy filter that
> they are no longer using. So effectively, that makes it a human error.
>
> We're going to a point where a single error is no longer causing outages,
> something very similar to my favorite analogy: avation. Pretty much every
> major air disaster was caused by a combination of factors. Pretty much
> every major outage these days is caused by a combination of factors.
>
> The manual provisioning of an inadequate filter, combined with an
> automation error on the side of a customer (which by itself was probably
> caused by a combination of factors), caused this issue.
>
> We learn from every outage. And instead of radio silence, they fessed up
> and fixed the issue. Have a look at the ASRS program :)

What I meant by "TOTALLY avoidable" is that "this particular plane
crash" has happened in the exact same way, for the exact same reasons,
over and over again.

Aviation learns from mistakes that don't generally recur in the exact
same way for the exact same reasons.

Telia and others have known about these issues from them happening to
other operators. When we see these issues, we go back and look at our
own networks to implement the fixes that solve the problem the last time
it happened. That's the idea.

The difference between us and aviation is that fundamental flaws or
mistakes that impact safety are required to be fixed and checked if you
want to keep operating in the industry. We don't have that, so...

Mark.

Has virtualization become obsolete in 5G?

[Etienne-Victor Depasquale]

Hi folks,

Over the past few weeks, I've attended webinars and watched videos
organized by Intel.
These activities have centred on 5G and examined applications (like "visual
cloud" and "gaming"),
as well as segment-oriented aspects (like edge networks, 5G RAN and 5G
Core).

I am stunned (no hyperbole) by the emphasis on Kubernetes in particular,
and cloud-native computing in general.
Equally stunning (for me), public telecommunications networks have been
portrayed
as having a history that moved from integrated software and hardware,
to virtualization and now to cloud-native computing.
See, for example Alex Quach, here

@10:30).
I reason that Intel's implication is that virtualization is becoming
obsolete.

Would anyone care to let me know his thoughts on this prediction?


Cheers all,

Etienne

-- 
Ing. Etienne-Victor Depasquale
Assistant Lecturer
Department of Communications & Computer Engineering
Faculty of Information & Communication Technology
University of Malta
Web. https://www.um.edu.mt/profile/etiennedepasqualeI