NANOG 88 Peering Forum + Sponsorships + More

[Nanog News]

*Have You Signed Up For the Peering Forum Yet?*
*A 90-minute Session to be Held on 12-Jun During NANOG 88 Conference*

NANOG 88 Peering Coordination Forum applications open!

The forum allows attendees to meet and network with others in the peering
community present at NANOG. Applications will remain open until 20
applications are received or 5-Jun-2023, whichever is first.

*MORE INFO * 

*Seattle Sponsorships Still Available! *
*Invest in the Strength of the Community We've Built*

   - Showcase your newest technologies + solutions
   - Increase your brand’s visibility + reach
   - Amplify your organization’s message
   - Connect with industry influencers + decision-makers
   - Empower people + inspire change

*MORE INFO
*

*Did You Know There is a Running Affinity Group at NANOG?*
*Find Community in our Online Discussion Threads*

The NANOG Affinity Groups are part of the Community Forum and allow our
members to connect over shared interests. We need your voice! Connect with
others in the tech industry and find mentorship and support while sharing
your experiences.

*MORE INFO * 

[NANOG-announce] NANOG 88 Peering Forum + Sponsorships + More

[Nanog News]

*Have You Signed Up For the Peering Forum Yet?*
*A 90-minute Session to be Held on 12-Jun During NANOG 88 Conference*

NANOG 88 Peering Coordination Forum applications open!

The forum allows attendees to meet and network with others in the peering
community present at NANOG. Applications will remain open until 20
applications are received or 5-Jun-2023, whichever is first.

*MORE INFO * 

*Seattle Sponsorships Still Available! *
*Invest in the Strength of the Community We've Built*

   - Showcase your newest technologies + solutions
   - Increase your brand’s visibility + reach
   - Amplify your organization’s message
   - Connect with industry influencers + decision-makers
   - Empower people + inspire change

*MORE INFO
*

*Did You Know There is a Running Affinity Group at NANOG?*
*Find Community in our Online Discussion Threads*

The NANOG Affinity Groups are part of the Community Forum and allow our
members to connect over shared interests. We need your voice! Connect with
others in the tech industry and find mentorship and support while sharing
your experiences.

*MORE INFO * 
___
NANOG-announce mailing list
NANOG-announce@nanog.org
https://mailman.nanog.org/mailman/listinfo/nanog-announce

Re: Best Linux (or BSD) hosted BGP?

[Mark Tinka]

On 5/11/23 07:28, Blake Dunlap wrote:

I'm confused here, are you intentionally running larger MTU interfaces 
than the packet filter can handle with default config, and not wanting 
to change the tunable to fix the config for buffer size for the packet 
filter, or am I misreading?


So I've gone ahead and tested "net.bpf.bufsize" and 
"net.bpf.maxbufsize", and none of them are helping any.


I'll send one to the FreeBSD mailing list and ask them if they can 
explain this. I saw something a little similar for DHCP on pfSense, 
although it has quite a few differences in that case:


https://forum.netgate.com/topic/130219/dhcpv6-client-broken-in-latest-snapshot/45

I'll advise when I know more.

Mark.

Re: Routed optical networks

[Etienne-Victor Depasquale via NANOG]

>
> Why you are so sure that they have access to traffic data for many access
> networks?

Well, no, I didn't say that I'm sure, actually I agree with you on
everything you've said.

But I can't say that it's purely speculation, as that would imply that I
know more than I actually do.

The overview of the method (from the 2016 - 2021 report) states the
following:

"The Cisco Visual Networking Index Forecast methodology has been developed
based on a combination of analyst projections, in-house estimates and
forecasts, and direct data collection. The analyst projections for
broadband connections, video subscribers, mobile connections, and Internet
application adoption come from SNL Kagan, Ovum, Informa Telecoms & Media,
Infonetics, IDC, Gartner, AMI, Verto Analytics, Ookla Speedtest.net,
Strategy Analytics, Screen Digest, Dell’Oro Group, Synergy, comScore,
Nielsen, Maravedis, Machina Research, ACG Research, ABI Research, Media
Partners Asia, IHS, International Telecommunications Union (ITU), CTIA, UN,
telecommunications regulators, and others. Upon this foundation are layered
Cisco’s own estimates for application adoption, minutes of use, and
kilobytes per minute. The adoption, usage, and bit-rate assumptions are
tied to fundamental enablers such as broadband speed and computing speed.
All usage and traffic results are then validated using data shared with
Cisco from service providers. Figure 1 shows the forecast methodology."

Link to Figure 1 here

.

Cheers,

Etienne

On Thu, May 11, 2023 at 5:18 PM Vasilenko Eduard <
vasilenko.edu...@huawei.com> wrote:

> Why you are so sure that they have access to traffic data for many access
> networks?
>
> Their BRASes market share is far from 100%. DSLAMs finished 20 years ago.
>
> For the cases where they support BRASes they could collect statistics. But
> are they doing it? Carriers have not given permission (not even requested).
>
> I do not have a clue about VNI arrangement – it is magic.
>
> PS: Everything above and below in this thread is just my personal opinion.
>
> Eduard
>
> *From:* Etienne-Victor Depasquale [mailto:ed...@ieee.org]
> *Sent:* Thursday, May 11, 2023 6:03 PM
> *To:* Vasilenko Eduard 
> *Cc:* Dave Taht ; Phil Bedard ;
> NANOG 
> *Subject:* Re: Routed optical networks
>
>
>
> Eduard, you know the answer as well as I do, right :) ?
>
>
>
> Here's my answer: I think that Cisco can only estimate (let's not say
> speculate, it has pretty bad connotations) what comes out of access
> networks.
>
>
>
> No offence meant, I hope none is taken.
>
>
>
> Cheers,
>
>
>
> Etienne
>
>
>
> On Thu, May 11, 2023 at 3:47 PM Vasilenko Eduard <
> vasilenko.edu...@huawei.com> wrote:
>
> Hi Etienne,
>
> Look carefully what you have shown to me. It is a only speculation again
> (“predictions”). It is just a table with a collection of all predictions in
> the past. Moreover, averaged between years.
>
> I was asking for real data from the past 5 years. Are you sure that VNI
> has it?
>
>
>
> If you would find real historical data in VNI, then we would be capable to
> check the table that you have shown: was the guessing right?
>
> I strongly suspect an answer.
>
>
>
> Eduard
>
> *From:* Etienne-Victor Depasquale [mailto:ed...@ieee.org]
> *Sent:* Thursday, May 11, 2023 2:46 PM
> *To:* Vasilenko Eduard 
> *Cc:* Dave Taht ; Phil Bedard ;
> NANOG 
> *Subject:* Re: Routed optical networks
>
>
>
> To clarify the table I linked to in the previous email:
>
>
>
> Cisco estimates IP traffic exchanged over the access network by both
> businesses and consumers with:
>
>
> • endpoints over managed networks and
> • endpoints over unmanaged networks (“Internet traffic”).
>
>
> Both the mobile access network and the fixed access network are
> considered.
>
>
>
> Cisco considers IP traffic over managed networks to be characterized by
> passage through a single service provider.
>
> Without explicitly referring to quality of service (QoS),
>
> the implication is clearly that the traffic is controlled to meet the QoS
> demanded by the service level agreement (SLA).
>
>
>
> In contrast, “Internet traffic” crosses provider domains;
>
> typically, this traffic is delivered on the basis of providers’ best
> effort.
>
> These two kinds of traffic complement one another and collectively are
> referred to as total global IP traffic.
>
>
>
> Cheers,
>
>
>
> Etienne
>
>
>
> On Thu, May 11, 2023 at 1:37 PM Etienne-Victor Depasquale 
> wrote:
>
> Historically, this is what VNI has claimed
> 
> .
>
>
>
> Cheers,
>
>
> Etienne
>
>
>
> On Thu, May 11, 2023 at 1:25 PM Vasilenko Eduard <
> vasilenko.edu...@huawei.com> wrote:
>
> I did investigate traffic for every Carrier while dealing with it as a
> consultant (repeated many dozens of times).
>
> I have seen over a decade how traffic growth dropped year-over-year (from
> 60% to 25% in 

Re: Routed optical networks

[Mark Tinka]

On 5/11/23 17:26, Jared Mauch wrote:


And as I’ve seen, they continue to become a bit more divergent.  As someone who 
has an access network I see where the majority of my bits go, which is to the 
content folks.  There’s some to the other part, but mostly people want their 
MTV, but since it’s 2023, it’s not MTV, but people want their TikTok, 
Metaverse, Game downloads, Streaming service, Email and cloud ramps 
(enterprise).


100% - and if the trend continues, Telegeography et al will have less 
raw growth to report on unless the content folk willingly open their 
skirts up to them for a peek - which they won't do.


So yes, predicting stable or negative growth for the public Internet is 
not without merit. But that does not translate to what the content folk 
are recording.


It might make more sense to start reporting on traffic growth in the 
edge, specifically, the peering edge, as content networks continue to 
become major on-ramp/off-ramp paths for telco's. But that prediction can 
be extrapolated from submarine cable builds, submarine cable upgrades as 
well as DWDM vendor sales, with some degree of reliability.


Mark.

Re: Routed optical networks

[Jared Mauch]

> On May 11, 2023, at 11:11 AM, Mark Tinka  wrote:
> 
> 
> 
> On 5/11/23 15:50, Vasilenko Eduard via NANOG wrote:
> 
>> Hi Jared,
>> Could I make a conclusion from your comments: "only Carrier itself 
>> understand the traffic - see many examples in the text".
>> I would very agree to this.
> 
> I wouldn't say only carriers understand the traffic as much as I would say 
> carrier's traffic is more transparent, and perhaps, even more readily 
> available.
> 
> I just think that it is not relevant to try and lump network and content 
> traffic into one growth pattern. For all intents & purposes, there are two 
> Internets running side-by-side between network and content. They converge at 
> some point, but really, they are very different.

And as I’ve seen, they continue to become a bit more divergent.  As someone who 
has an access network I see where the majority of my bits go, which is to the 
content folks.  There’s some to the other part, but mostly people want their 
MTV, but since it’s 2023, it’s not MTV, but people want their TikTok, 
Metaverse, Game downloads, Streaming service, Email and cloud ramps 
(enterprise).

- jared

RE: Routed optical networks

[Vasilenko Eduard via NANOG]

Why you are so sure that they have access to traffic data for many access 
networks?
Their BRASes market share is far from 100%. DSLAMs finished 20 years ago.
For the cases where they support BRASes they could collect statistics. But are 
they doing it? Carriers have not given permission (not even requested).
I do not have a clue about VNI arrangement – it is magic.
PS: Everything above and below in this thread is just my personal opinion.
Eduard
From: Etienne-Victor Depasquale [mailto:ed...@ieee.org]
Sent: Thursday, May 11, 2023 6:03 PM
To: Vasilenko Eduard 
Cc: Dave Taht ; Phil Bedard ; NANOG 

Subject: Re: Routed optical networks

Eduard, you know the answer as well as I do, right :) ?

Here's my answer: I think that Cisco can only estimate (let's not say 
speculate, it has pretty bad connotations) what comes out of access networks.

No offence meant, I hope none is taken.

Cheers,

Etienne

On Thu, May 11, 2023 at 3:47 PM Vasilenko Eduard 
mailto:vasilenko.edu...@huawei.com>> wrote:
Hi Etienne,
Look carefully what you have shown to me. It is a only speculation again 
(“predictions”). It is just a table with a collection of all predictions in the 
past. Moreover, averaged between years.
I was asking for real data from the past 5 years. Are you sure that VNI has it?

If you would find real historical data in VNI, then we would be capable to 
check the table that you have shown: was the guessing right?
I strongly suspect an answer.

Eduard
From: Etienne-Victor Depasquale [mailto:ed...@ieee.org]
Sent: Thursday, May 11, 2023 2:46 PM
To: Vasilenko Eduard 
mailto:vasilenko.edu...@huawei.com>>
Cc: Dave Taht mailto:dave.t...@gmail.com>>; Phil Bedard 
mailto:bedard.p...@gmail.com>>; NANOG 
mailto:nanog@nanog.org>>
Subject: Re: Routed optical networks

To clarify the table I linked to in the previous email:

Cisco estimates IP traffic exchanged over the access network by both businesses 
and consumers with:

• endpoints over managed networks and
• endpoints over unmanaged networks (“Internet traffic”).

Both the mobile access network and the fixed access network are considered.

Cisco considers IP traffic over managed networks to be characterized by passage 
through a single service provider.
Without explicitly referring to quality of service (QoS),
the implication is clearly that the traffic is controlled to meet the QoS 
demanded by the service level agreement (SLA).

In contrast, “Internet traffic” crosses provider domains;
typically, this traffic is delivered on the basis of providers’ best effort.
These two kinds of traffic complement one another and collectively are referred 
to as total global IP traffic.

Cheers,

Etienne

On Thu, May 11, 2023 at 1:37 PM Etienne-Victor Depasquale 
mailto:ed...@ieee.org>> wrote:
Historically, this is what VNI has 
claimed.

Cheers,

Etienne

On Thu, May 11, 2023 at 1:25 PM Vasilenko Eduard 
mailto:vasilenko.edu...@huawei.com>> wrote:
I did investigate traffic for every Carrier while dealing with it as a 
consultant (repeated many dozens of times).
I have seen over a decade how traffic growth dropped year-over-year (from 60% 
to 25% in 2019 when I dropped this activity in 2020 – covid blocked travel).
Sometimes I talk to old connections and they confirm that it is even less now.
In rear cases, It is typically possible to find this information on the public 
Internet (I remember the case when Google disclosed traffic for Pakistan at the 
conference with the explanation that 30% is attributed to new subscribers, and 
an additional +30% is to more heavy content per subscriber).
But mostly, it was confidential information from a discussion with Carriers – 
they all know very well their traffic growth.
In general, traffic stat is pretty confidential. I did not have the motivation 
to aggregate it.

Sandvine is not representative of global traffic because DPI is installed 
mostly for Mobiles. But Mobile subscriber is 10x less than fixed on traffic – 
it is not the biggest source. Moreover, Mobiles would look better growing 
because the limiting factor was on technology (5G proposed more than 4G, 4G 
proposed much more than 3G) – it would probably would less disruptive in the 
future.
Fixed Carriers do not pay DPI premiums. And rarely share their traffic 
publicly. Sandvine could not see it.

VNI is claiming so many things. Please show where exactly they show traffic 
growth (I am not interested in prediction speculations). Is it possible to 
understand CAGR for the 5 last years? Is it declining or growing? (traffic 
itself is for sure still growing)

Of course, the disruption could come at any year and add a new S-curve 
(Metaverse?). But disruption is by definition not predictable.

PS: Everything above and below in this thread is just my personal opinion.

Eduard
From: Etienne-Victor Depasquale [mailto:ed...@ieee.org]
Sent: Thursday, May 11, 2023 

Re: Routed optical networks

[Mark Tinka]

On 5/11/23 15:50, Vasilenko Eduard via NANOG wrote:


Hi Jared,
Could I make a conclusion from your comments: "only Carrier itself 
understand the traffic - see many examples in the text".

I would very agree to this.


I wouldn't say only carriers understand the traffic as much as I would 
say carrier's traffic is more transparent, and perhaps, even more 
readily available.


I just think that it is not relevant to try and lump network and content 
traffic into one growth pattern. For all intents & purposes, there are 
two Internets running side-by-side between network and content. They 
converge at some point, but really, they are very different.


Mark.

Re: Routed optical networks

[Mark Tinka]

On 5/11/23 14:15, Jared Mauch wrote


We have seen a continued trend of the privatization of traffic and localization 
of that over time.


CDN's, submarine cables and exchange points are decentralizing the "core 
of the Internet", and relegating IP Transit providers to whom CDN's 
subscribe as sources of origin content, if they are unable to build 
their own backbones.


Only in markets where CDN's are not as rife do you still see huge growth 
and stable pricing for IP Transit traffic.




   I’ve watched all the big carriers retreat from their global network reaches 
to be more of regionalized networks.  A decade ago you would have seen European 
national incumbents peering and with market in Asia, and the complete global 
networks continue to shrink.


Yeah - unless you serve a very tight niche of Enterprise customers that 
require stable (not necessarily fast) connectivity to some part of the 
word, it makes little sense, nowadays, for the large telco's of old to 
go poking in other markets that aren't theirs, something the top global 
exchange points are going to learn the hard way.




Meanwhile you have a mix of the content and cloud providers continue to build 
their business-purpose networks expanding into markets that the uppercase 
Internet may not need to reach.


At the moment, the only motivation I see for large telco's trying to 
enter markets they can't be strong in is their "global brand". Often 
times, they can't do very well in that market because a) there is 
sufficient local content and peering, b) they can't build a network as 
robust as what the local telco's can, and c) all they have to offer is 
access to traffic several milliseconds away that is not any faster than 
what the local telco's can, which forms less than 20% of what the local 
customers chase after anyway.


Such telco's usually, then, turn to targeting global Enterprise 
customers with "worldwide service contracts" accompanied by marked-up 
pricing for "private global branch connectivity". It always sounds and 
looks good on PowerPoint slides, but 12 months after the big launch, a 
broken champagne bottle and some photos, the PoP is gathering quite a 
bit of dust in the data centre.




You can look at the proposals in the EU about fees, and I have dual thoughts on 
this which are MY OWN and don’t represent my employer or otherwise, but if you 
read this post from Petra Arts 
-https://blog.cloudflare.com/eu-network-usage-fees/  - it speaks around major 
interconnection points like Frankfurt, which are important but double as 
problematic.  The number of people that need to go to the near market (eg: 
Chicago, while I’m in Detroit area) for good connectivity is an issue, 
meanwhile there’s a robust need to keep traffic within the state of Michigan 
and a halfway decent ecosystem for that via Detroit IX - (disclaimer, I’m on 
the board).  There need to be some aggregation points, so not everyone needs to 
be in Detroit, but also not everyone needs to be in Frankfurt - and content 
localization needs to continue to happen, but is also very regionalized in 
popularity.


Us telco's didn't learn our lesson 23 years ago when the content folk 
tried to do a deal with us. This will be deja vu, although this time, 
with some help from governments that we rather won't like.


Telco's continue to strong-arm content because we own the customer. But, 
it's 2023... I'm not sure we want to see a battle of the customer 
between content and network. It might not end well. We already see 
glimpses of it with QUIC, so...




How to do this all and not have it all route via Chicago or Frankfurt is a 
challenge, but also not everyone will be in Berlin, Munich or these other 
markets.  This is where having a robust optical network capability (or 
backbone) can come into play, that you can deliver deeper from those hub 
points, but at the same time, I’ve been in meetings where companies have their 
own challenges accepting that content in those downstream locations as their 
network was also built to get to/from the major hub cities, or IP space wasn’t 
allocated in a way that can provide consistent routing results or behaviors.  
(This is where IPv6 can be super helpful, it gives the chance to possibly 
Greenfield, aka not screw it up - at least initially).


As content continues to grow, PoP's built in far-flung locations away 
from a telco's core area of business will only remain as relevant as 
traffic that has not yet migrated to a syndicated cloud. As that tapers 
off- which it will - there will be more pressure to either shut those 
PoP's down, or compete for local IP Transit. The latter is much less likely.




There’s huge volumes of IP traffic exchanged, but the largest volumes are being 
moved over private interconnects or a localized IX to those eyeball networks 
with the historical global backbones playing more of the long-distance carrier 
role, which is critical as you want a path to deliver those bits, without it 

Re: Routed optical networks

[Mark Tinka]

On 5/11/23 15:50, Vasilenko Eduard via NANOG wrote:


Hi Jared,
Could I make a conclusion from your comments: "only Carrier itself understand the 
traffic - see many examples in the text".
I would very agree to this.


I wouldn't say only carriers understand the traffic as much as I would 
say carrier's traffic is more transparent, and perhaps, even more 
readily available.


I just think that it is not relevant to try and lump network and content 
traffic into one growth pattern. For all intents & purposes, there are 
two Internets running side-by-side between network and content. They 
converge at some point, but really, they are very different.


Mark.

Re: Routed optical networks

[Etienne-Victor Depasquale via NANOG]

Eduard, you know the answer as well as I do, right :) ?

Here's my answer: I think that Cisco can only estimate (let's not say
speculate, it has pretty bad connotations) what comes out of access
networks.

No offence meant, I hope none is taken.

Cheers,

Etienne

On Thu, May 11, 2023 at 3:47 PM Vasilenko Eduard <
vasilenko.edu...@huawei.com> wrote:

> Hi Etienne,
>
> Look carefully what you have shown to me. It is a only speculation again
> (“predictions”). It is just a table with a collection of all predictions in
> the past. Moreover, averaged between years.
>
> I was asking for real data from the past 5 years. Are you sure that VNI
> has it?
>
>
>
> If you would find real historical data in VNI, then we would be capable to
> check the table that you have shown: was the guessing right?
>
> I strongly suspect an answer.
>
>
>
> Eduard
>
> *From:* Etienne-Victor Depasquale [mailto:ed...@ieee.org]
> *Sent:* Thursday, May 11, 2023 2:46 PM
> *To:* Vasilenko Eduard 
> *Cc:* Dave Taht ; Phil Bedard ;
> NANOG 
> *Subject:* Re: Routed optical networks
>
>
>
> To clarify the table I linked to in the previous email:
>
>
>
> Cisco estimates IP traffic exchanged over the access network by both
> businesses and consumers with:
>
>
> • endpoints over managed networks and
> • endpoints over unmanaged networks (“Internet traffic”).
>
>
> Both the mobile access network and the fixed access network are
> considered.
>
>
>
> Cisco considers IP traffic over managed networks to be characterized by
> passage through a single service provider.
>
> Without explicitly referring to quality of service (QoS),
>
> the implication is clearly that the traffic is controlled to meet the QoS
> demanded by the service level agreement (SLA).
>
>
>
> In contrast, “Internet traffic” crosses provider domains;
>
> typically, this traffic is delivered on the basis of providers’ best
> effort.
>
> These two kinds of traffic complement one another and collectively are
> referred to as total global IP traffic.
>
>
>
> Cheers,
>
>
>
> Etienne
>
>
>
> On Thu, May 11, 2023 at 1:37 PM Etienne-Victor Depasquale 
> wrote:
>
> Historically, this is what VNI has claimed
> 
> .
>
>
>
> Cheers,
>
>
> Etienne
>
>
>
> On Thu, May 11, 2023 at 1:25 PM Vasilenko Eduard <
> vasilenko.edu...@huawei.com> wrote:
>
> I did investigate traffic for every Carrier while dealing with it as a
> consultant (repeated many dozens of times).
>
> I have seen over a decade how traffic growth dropped year-over-year (from
> 60% to 25% in 2019 when I dropped this activity in 2020 – covid blocked
> travel).
>
> Sometimes I talk to old connections and they confirm that it is even less
> now.
>
> In rear cases, It is typically possible to find this information on the
> public Internet (I remember the case when Google disclosed traffic for
> Pakistan at the conference with the explanation that 30% is attributed to
> new subscribers, and an additional +30% is to more heavy content per
> subscriber).
>
> But mostly, it was confidential information from a discussion with
> Carriers – they all know very well their traffic growth.
>
> In general, traffic stat is pretty confidential. I did not have the
> motivation to aggregate it.
>
>
>
> Sandvine is not representative of global traffic because DPI is installed
> mostly for Mobiles. But Mobile subscriber is 10x less than fixed on traffic
> – it is not the biggest source. Moreover, Mobiles would look better growing
> because the limiting factor was on technology (5G proposed more than 4G, 4G
> proposed much more than 3G) – it would probably would less disruptive in
> the future.
>
> Fixed Carriers do not pay DPI premiums. And rarely share their traffic
> publicly. Sandvine could not see it.
>
>
>
> VNI is claiming so many things. Please show where exactly they show
> traffic growth (I am not interested in prediction speculations). Is it
> possible to understand CAGR for the 5 last years? Is it declining or
> growing? (traffic itself is for sure still growing)
>
>
>
> Of course, the disruption could come at any year and add a new S-curve
> (Metaverse?). But disruption is by definition not predictable.
>
>
>
> PS: Everything above and below in this thread is just my personal opinion.
>
>
>
> Eduard
>
> *From:* Etienne-Victor Depasquale [mailto:ed...@ieee.org]
> *Sent:* Thursday, May 11, 2023 12:48 PM
> *To:* Vasilenko Eduard 
> *Cc:* Dave Taht ; Phil Bedard ;
> NANOG 
> *Subject:* Re: Routed optical networks
>
>
>
> Eduard, academics cite the VNI (and the Sandvine Global reports).
>
>
>
> Do you know of alternative sources that show traffic growth data you're
> more comfortable with?
>
>
>
> Cheers,
>
>
>
> Etienne
>
>
>
> On Thu, May 11, 2023 at 9:34 AM Vasilenko Eduard <
> vasilenko.edu...@huawei.com> wrote:
>
> But it is speculation, not a trend yet.
>
> I remember 10y ago every presentation started from the claim that 100B of
> IoT would drive XXX traffic. 

Re: Best Linux (or BSD) hosted BGP?

[Brandon Zhi]

I use bird2 with Debian11 sometimes, I'm curious, what is the usual
hardware for using Linux as a router? In addition, the Linux ip rule seems
to have a problem with the matching of the ipv4 source address. . .
*Brandon Zhi*
HUIZE LTD
www.huize.asia  | www.ixp.su | Twitter

This e-mail and any attachments or any reproduction of this e-mail in
whatever manner are confidential and for the use of the addressee(s) only.
HUIZE LTD can’t take any liability and guarantee of the text of the email
message and virus.


On Thu, 11 May 2023 at 13:29, Blake Dunlap  wrote:

> I'm confused here, are you intentionally running larger MTU interfaces
> than the packet filter can handle with default config, and not wanting to
> change the tunable to fix the config for buffer size for the packet filter,
> or am I misreading?
>
> On Wed, May 10, 2023 at 11:51 PM Mark Tinka  wrote:
>
>>
>>
>> On 5/10/23 15:55, Tom Beecher wrote:
>>
>> >  That could just as easily happen today. Every OS release has all
>> > kinds of changes to defaults, and frequently don't get caught until
>> > they break something. Even if today's FreeBSD defaults worked for this
>> > scenario, the next release could change to a value that doesn't.
>>
>> We implement a lot of user-defined changes to FreeBSD defaults via
>> "/etc/sysctl.conf", as an example, whose unexpected change would not
>> necessarily break anything as they would reduce scaled performance. We
>> can live with that, because we can afford a reduction in performance
>> until the fault is found, not an outright outage.
>>
>> The problem with doing this with something like a routing protocol - and
>> in this specific case with FRR on FreeBSD for IS-IS - is that it would
>> not be a reduction in performance if an unexpected change were to find
>> its way into future revisions of FreeBSD... it would, in all likelihood,
>> be a complete outage. That is a steeper price to pay, for us anyway.
>>
>> It's just about weighing the risks for one's particular operating
>> environment, and for us, that risk is too high for a routing protocol.
>>
>> Mark.
>>
>

Re: Routed optical networks

[Mark Tinka]

On 5/11/23 13:25, Vasilenko Eduard via NANOG wrote:

Sandvine is not representative of global traffic because DPI is 
installed mostly for Mobiles. But Mobile subscriber is 10x less than 
fixed on traffic – it is not the biggest source. Moreover, Mobiles 
would look better growing because the limiting factor was on 
technology (5G proposed more than 4G, 4G proposed much more than 3G) – 
it would probably would less disruptive in the future.


Fixed Carriers do not pay DPI premiums. And rarely share their traffic 
publicly. Sandvine could not see it.




And QUIC is making the the DPI model so loved by MNO's quickly irrelevant.

Mark.

Re: Routed optical networks

[Mark Tinka]

On 5/11/23 09:33, Vasilenko Eduard via NANOG wrote:


But it is speculation, not a trend yet.

I remember 10y ago every presentation started from the claim that 100B 
of IoT would drive XXX traffic. It did not happen.


Now we see presentations that AI would be talking to AI that generates 
 traffic.


Maybe some technology would push traffic next S-curve, maybe not. It 
is still speculation.


The traffic growth was stimulated (despite all VNIs) by 1) new 
subscribers, 2) video quality for subscribers. Nothing else yet.


It is almost finished for both trends. We are close to the plateau of 
these S-curves.


For some years (2013-2020) I was carefully looking at numbers for many 
countries: it was always possible to split CAGR for these 2 
components. The video part was extremely consistent between countries. 
The subscriber part was 100% proportional to subscriber CAGR.


Everything else up to now was “marketing” to say it mildly.

Reminder: nothing in nature could grow indefinitely. The limit always 
exists. It is only a question of when.


PS: Of course, marketing people could draw you any traffic growth – it 
depends just on the marketing budget.




I think the place where we are seeing continued growth is in the content 
provider private network, for that DC-to-DC traffic, even across continents.


For everything else, as you say, it's nice to predict. But what analysts 
and marketing departments say and what end users actually do are often 
vastly different.


As long as the majority of the Internet's traffic is "invisible", we 
can't really ever know. We can only tell by how much kit the DWDM 
vendors are selling to non-telco customers (content folk), as well as 
how many submarine cables are being built by non-telco consortia 
(content folk).


Mark.

Re: Routed optical networks

[Mark Tinka]

On 5/11/23 13:45, Etienne-Victor Depasquale via NANOG wrote:


To clarify the table I linked to in the previous email:

Cisco estimates IP traffic exchanged over the access network by both 
businesses and consumers with:


• endpoints over managed networks and
• endpoints over unmanaged networks (“Internet traffic”).

Both the mobile access network and the fixed access network are 
considered.


Cisco considers IP traffic over managed networks to be characterized 
by passage through a single service provider.

Without explicitly referring to quality of service (QoS),
the implication is clearly that the traffic is controlled to meet the 
QoS demanded by the service level agreement (SLA).


So either Cisco think/though that the only operators worth 
surveying/predicting were the large ones (NTT, Telia, Tata, Lumen, 
Cogent, e.t.c.), or that on-net MPLS/VPN traffic was more significant 
than public IP Transit both in terms of revenue and strategic direction 
of the operators they surveyed/predicated.


Either way, I'd imagine any results based on those data points would be 
incomplete, at least from a real-world standpoint.




In contrast, “Internet traffic” crosses provider domains;
typically, this traffic is delivered on the basis of providers’ best 
effort.
These two kinds of traffic complement one another and collectively are 
referred to as total global IP traffic.


In the IP space, this traffic type is quickly exceeding any historically 
significant MPLS/VPN traffic, if it hasn't already.


Mark.

RE: Routed optical networks

[Vasilenko Eduard via NANOG]

Hi Jared,
Could I make a conclusion from your comments: "only Carrier itself understand 
the traffic - see many examples in the text".
I would very agree to this.
Eduard
-Original Message-
From: Jared Mauch [mailto:ja...@puck.nether.net] 
Sent: Thursday, May 11, 2023 3:16 PM
To: Etienne-Victor Depasquale 
Cc: Vasilenko Eduard ; NANOG 
Subject: Re: Routed optical networks



> On May 11, 2023, at 7:45 AM, Etienne-Victor Depasquale via NANOG 
>  wrote:
> 
> To clarify the table I linked to in the previous email:
> 
> Cisco estimates IP traffic exchanged over the access network by both 
> businesses and consumers with:
> 
> • endpoints over managed networks and • endpoints over unmanaged 
> networks (“Internet traffic”).
> 
> Both the mobile access network and the fixed access network are considered. 
> 
> Cisco considers IP traffic over managed networks to be characterized by 
> passage through a single service provider. 
> Without explicitly referring to quality of service (QoS), the 
> implication is clearly that the traffic is controlled to meet the QoS 
> demanded by the service level agreement (SLA).
> 
> In contrast, “Internet traffic” crosses provider domains; typically, 
> this traffic is delivered on the basis of providers’ best effort.
> These two kinds of traffic complement one another and collectively are 
> referred to as total global IP traffic.


I think there’s a lot of problems here.  While places like my employer will 
periodically disclose our traffic numbers, and DDoS providers, mitigation 
platforms and otherwise will disclose the peaks they see, much of this data is 
a bit opaque, and tools like AI that do in-metro or cross-metro 
datacenter-datacenter remote DMA type activities, those all count differently.

We have seen a continued trend of the privatization of traffic and localization 
of that over time.  I’ve watched all the big carriers retreat from their global 
network reaches to be more of regionalized networks.  A decade ago you would 
have seen European national incumbents peering and with market in Asia, and the 
complete global networks continue to shrink.

Meanwhile you have a mix of the content and cloud providers continue to build 
their business-purpose networks expanding into markets that the uppercase 
Internet may not need to reach.

You can look at the proposals in the EU about fees, and I have dual thoughts on 
this which are MY OWN and don’t represent my employer or otherwise, but if you 
read this post from Petra Arts - 
https://blog.cloudflare.com/eu-network-usage-fees/ - it speaks around major 
interconnection points like Frankfurt, which are important but double as 
problematic.  The number of people that need to go to the near market (eg: 
Chicago, while I’m in Detroit area) for good connectivity is an issue, 
meanwhile there’s a robust need to keep traffic within the state of Michigan 
and a halfway decent ecosystem for that via Detroit IX - (disclaimer, I’m on 
the board).  There need to be some aggregation points, so not everyone needs to 
be in Detroit, but also not everyone needs to be in Frankfurt - and content 
localization needs to continue to happen, but is also very regionalized in 
popularity.

How to do this all and not have it all route via Chicago or Frankfurt is a 
challenge, but also not everyone will be in Berlin, Munich or these other 
markets.  This is where having a robust optical network capability (or 
backbone) can come into play, that you can deliver deeper from those hub 
points, but at the same time, I’ve been in meetings where companies have their 
own challenges accepting that content in those downstream locations as their 
network was also built to get to/from the major hub cities, or IP space wasn’t 
allocated in a way that can provide consistent routing results or behaviors.  
(This is where IPv6 can be super helpful, it gives the chance to possibly 
Greenfield, aka not screw it up - at least initially).

There’s huge volumes of IP traffic exchanged, but the largest volumes are being 
moved over private interconnects or a localized IX to those eyeball networks 
with the historical global backbones playing more of the long-distance carrier 
role, which is critical as you want a path to deliver those bits, without it 
following the ITU-style sender pays model, as the majority of IP traffic is 
actually requested by the customer of the end-user network.  (All of it if you 
remove network scans, ddos, web bots/crawlers).

Most networks have no SLA once things cross an unpaid boundary (SFI, or even 
private peering) - and if they are a customer and that path is congested, it’s 
up to the customer to upgrade that path.

- Jared (many hats)

RE: Routed optical networks

[Vasilenko Eduard via NANOG]

Hi Etienne,
Look carefully what you have shown to me. It is a only speculation again 
(“predictions”). It is just a table with a collection of all predictions in the 
past. Moreover, averaged between years.
I was asking for real data from the past 5 years. Are you sure that VNI has it?

If you would find real historical data in VNI, then we would be capable to 
check the table that you have shown: was the guessing right?
I strongly suspect an answer.

Eduard
From: Etienne-Victor Depasquale [mailto:ed...@ieee.org]
Sent: Thursday, May 11, 2023 2:46 PM
To: Vasilenko Eduard 
Cc: Dave Taht ; Phil Bedard ; NANOG 

Subject: Re: Routed optical networks

To clarify the table I linked to in the previous email:

Cisco estimates IP traffic exchanged over the access network by both businesses 
and consumers with:

• endpoints over managed networks and
• endpoints over unmanaged networks (“Internet traffic”).

Both the mobile access network and the fixed access network are considered.

Cisco considers IP traffic over managed networks to be characterized by passage 
through a single service provider.
Without explicitly referring to quality of service (QoS),
the implication is clearly that the traffic is controlled to meet the QoS 
demanded by the service level agreement (SLA).

In contrast, “Internet traffic” crosses provider domains;
typically, this traffic is delivered on the basis of providers’ best effort.
These two kinds of traffic complement one another and collectively are referred 
to as total global IP traffic.

Cheers,

Etienne

On Thu, May 11, 2023 at 1:37 PM Etienne-Victor Depasquale 
mailto:ed...@ieee.org>> wrote:
Historically, this is what VNI has 
claimed.

Cheers,

Etienne

On Thu, May 11, 2023 at 1:25 PM Vasilenko Eduard 
mailto:vasilenko.edu...@huawei.com>> wrote:
I did investigate traffic for every Carrier while dealing with it as a 
consultant (repeated many dozens of times).
I have seen over a decade how traffic growth dropped year-over-year (from 60% 
to 25% in 2019 when I dropped this activity in 2020 – covid blocked travel).
Sometimes I talk to old connections and they confirm that it is even less now.
In rear cases, It is typically possible to find this information on the public 
Internet (I remember the case when Google disclosed traffic for Pakistan at the 
conference with the explanation that 30% is attributed to new subscribers, and 
an additional +30% is to more heavy content per subscriber).
But mostly, it was confidential information from a discussion with Carriers – 
they all know very well their traffic growth.
In general, traffic stat is pretty confidential. I did not have the motivation 
to aggregate it.

Sandvine is not representative of global traffic because DPI is installed 
mostly for Mobiles. But Mobile subscriber is 10x less than fixed on traffic – 
it is not the biggest source. Moreover, Mobiles would look better growing 
because the limiting factor was on technology (5G proposed more than 4G, 4G 
proposed much more than 3G) – it would probably would less disruptive in the 
future.
Fixed Carriers do not pay DPI premiums. And rarely share their traffic 
publicly. Sandvine could not see it.

VNI is claiming so many things. Please show where exactly they show traffic 
growth (I am not interested in prediction speculations). Is it possible to 
understand CAGR for the 5 last years? Is it declining or growing? (traffic 
itself is for sure still growing)

Of course, the disruption could come at any year and add a new S-curve 
(Metaverse?). But disruption is by definition not predictable.

PS: Everything above and below in this thread is just my personal opinion.

Eduard
From: Etienne-Victor Depasquale [mailto:ed...@ieee.org]
Sent: Thursday, May 11, 2023 12:48 PM
To: Vasilenko Eduard 
mailto:vasilenko.edu...@huawei.com>>
Cc: Dave Taht mailto:dave.t...@gmail.com>>; Phil Bedard 
mailto:bedard.p...@gmail.com>>; NANOG 
mailto:nanog@nanog.org>>
Subject: Re: Routed optical networks

Eduard, academics cite the VNI (and the Sandvine Global reports).

Do you know of alternative sources that show traffic growth data you're more 
comfortable with?

Cheers,

Etienne

On Thu, May 11, 2023 at 9:34 AM Vasilenko Eduard 
mailto:vasilenko.edu...@huawei.com>> wrote:
But it is speculation, not a trend yet.
I remember 10y ago every presentation started from the claim that 100B of IoT 
would drive XXX traffic. It did not happen.
Now we see presentations that AI would be talking to AI that generates  
traffic.
Maybe some technology would push traffic next S-curve, maybe not. It is still 
speculation.

The traffic growth was stimulated (despite all VNIs) by 1) new subscribers, 2) 
video quality for subscribers. Nothing else yet.
It is almost finished for both trends. We are close to the plateau of these 
S-curves.
For some years (2013-2020) I was carefully looking at numbers for many 
countries: it 

Re: Routed optical networks

[Jared Mauch]

> On May 11, 2023, at 7:45 AM, Etienne-Victor Depasquale via NANOG 
>  wrote:
> 
> To clarify the table I linked to in the previous email:
> 
> Cisco estimates IP traffic exchanged over the access network by both 
> businesses and consumers with:
> 
> • endpoints over managed networks and 
> • endpoints over unmanaged networks (“Internet traffic”).
> 
> Both the mobile access network and the fixed access network are considered. 
> 
> Cisco considers IP traffic over managed networks to be characterized by 
> passage through a single service provider. 
> Without explicitly referring to quality of service (QoS), 
> the implication is clearly that the traffic is controlled to meet the QoS 
> demanded by the service level agreement (SLA). 
> 
> In contrast, “Internet traffic” crosses provider domains; 
> typically, this traffic is delivered on the basis of providers’ best effort. 
> These two kinds of traffic complement one another and collectively are 
> referred to as total global IP traffic.


I think there’s a lot of problems here.  While places like my employer will 
periodically disclose our traffic numbers, and DDoS providers, mitigation 
platforms and otherwise will disclose the peaks they see, much of this data is 
a bit opaque, and tools like AI that do in-metro or cross-metro 
datacenter-datacenter remote DMA type activities, those all count differently.

We have seen a continued trend of the privatization of traffic and localization 
of that over time.  I’ve watched all the big carriers retreat from their global 
network reaches to be more of regionalized networks.  A decade ago you would 
have seen European national incumbents peering and with market in Asia, and the 
complete global networks continue to shrink.

Meanwhile you have a mix of the content and cloud providers continue to build 
their business-purpose networks expanding into markets that the uppercase 
Internet may not need to reach.

You can look at the proposals in the EU about fees, and I have dual thoughts on 
this which are MY OWN and don’t represent my employer or otherwise, but if you 
read this post from Petra Arts - 
https://blog.cloudflare.com/eu-network-usage-fees/ - it speaks around major 
interconnection points like Frankfurt, which are important but double as 
problematic.  The number of people that need to go to the near market (eg: 
Chicago, while I’m in Detroit area) for good connectivity is an issue, 
meanwhile there’s a robust need to keep traffic within the state of Michigan 
and a halfway decent ecosystem for that via Detroit IX - (disclaimer, I’m on 
the board).  There need to be some aggregation points, so not everyone needs to 
be in Detroit, but also not everyone needs to be in Frankfurt - and content 
localization needs to continue to happen, but is also very regionalized in 
popularity.

How to do this all and not have it all route via Chicago or Frankfurt is a 
challenge, but also not everyone will be in Berlin, Munich or these other 
markets.  This is where having a robust optical network capability (or 
backbone) can come into play, that you can deliver deeper from those hub 
points, but at the same time, I’ve been in meetings where companies have their 
own challenges accepting that content in those downstream locations as their 
network was also built to get to/from the major hub cities, or IP space wasn’t 
allocated in a way that can provide consistent routing results or behaviors.  
(This is where IPv6 can be super helpful, it gives the chance to possibly 
Greenfield, aka not screw it up - at least initially).

There’s huge volumes of IP traffic exchanged, but the largest volumes are being 
moved over private interconnects or a localized IX to those eyeball networks 
with the historical global backbones playing more of the long-distance carrier 
role, which is critical as you want a path to deliver those bits, without it 
following the ITU-style sender pays model, as the majority of IP traffic is 
actually requested by the customer of the end-user network.  (All of it if you 
remove network scans, ddos, web bots/crawlers).

Most networks have no SLA once things cross an unpaid boundary (SFI, or even 
private peering) - and if they are a customer and that path is congested, it’s 
up to the customer to upgrade that path.

- Jared (many hats)

Re: Routed optical networks

[Etienne-Victor Depasquale via NANOG]

To clarify the table I linked to in the previous email:

Cisco estimates IP traffic exchanged over the access network by both
businesses and consumers with:

• endpoints over managed networks and
• endpoints over unmanaged networks (“Internet traffic”).

Both the mobile access network and the fixed access network are considered.

Cisco considers IP traffic over managed networks to be characterized by
passage through a single service provider.
Without explicitly referring to quality of service (QoS),
the implication is clearly that the traffic is controlled to meet the QoS
demanded by the service level agreement (SLA).

In contrast, “Internet traffic” crosses provider domains;
typically, this traffic is delivered on the basis of providers’ best
effort.
These two kinds of traffic complement one another and collectively are
referred to as total global IP traffic.

Cheers,

Etienne

On Thu, May 11, 2023 at 1:37 PM Etienne-Victor Depasquale 
wrote:

> Historically, this is what VNI has claimed
> 
> .
>
> Cheers,
>
> Etienne
>
> On Thu, May 11, 2023 at 1:25 PM Vasilenko Eduard <
> vasilenko.edu...@huawei.com> wrote:
>
>> I did investigate traffic for every Carrier while dealing with it as a
>> consultant (repeated many dozens of times).
>>
>> I have seen over a decade how traffic growth dropped year-over-year (from
>> 60% to 25% in 2019 when I dropped this activity in 2020 – covid blocked
>> travel).
>>
>> Sometimes I talk to old connections and they confirm that it is even less
>> now.
>>
>> In rear cases, It is typically possible to find this information on the
>> public Internet (I remember the case when Google disclosed traffic for
>> Pakistan at the conference with the explanation that 30% is attributed to
>> new subscribers, and an additional +30% is to more heavy content per
>> subscriber).
>>
>> But mostly, it was confidential information from a discussion with
>> Carriers – they all know very well their traffic growth.
>>
>> In general, traffic stat is pretty confidential. I did not have the
>> motivation to aggregate it.
>>
>>
>>
>> Sandvine is not representative of global traffic because DPI is installed
>> mostly for Mobiles. But Mobile subscriber is 10x less than fixed on traffic
>> – it is not the biggest source. Moreover, Mobiles would look better growing
>> because the limiting factor was on technology (5G proposed more than 4G, 4G
>> proposed much more than 3G) – it would probably would less disruptive in
>> the future.
>>
>> Fixed Carriers do not pay DPI premiums. And rarely share their traffic
>> publicly. Sandvine could not see it.
>>
>>
>>
>> VNI is claiming so many things. Please show where exactly they show
>> traffic growth (I am not interested in prediction speculations). Is it
>> possible to understand CAGR for the 5 last years? Is it declining or
>> growing? (traffic itself is for sure still growing)
>>
>>
>>
>> Of course, the disruption could come at any year and add a new S-curve
>> (Metaverse?). But disruption is by definition not predictable.
>>
>>
>>
>> PS: Everything above and below in this thread is just my personal opinion.
>>
>>
>>
>> Eduard
>>
>> *From:* Etienne-Victor Depasquale [mailto:ed...@ieee.org]
>> *Sent:* Thursday, May 11, 2023 12:48 PM
>> *To:* Vasilenko Eduard 
>> *Cc:* Dave Taht ; Phil Bedard ;
>> NANOG 
>> *Subject:* Re: Routed optical networks
>>
>>
>>
>> Eduard, academics cite the VNI (and the Sandvine Global reports).
>>
>>
>>
>> Do you know of alternative sources that show traffic growth data you're
>> more comfortable with?
>>
>>
>>
>> Cheers,
>>
>>
>>
>> Etienne
>>
>>
>>
>> On Thu, May 11, 2023 at 9:34 AM Vasilenko Eduard <
>> vasilenko.edu...@huawei.com> wrote:
>>
>> But it is speculation, not a trend yet.
>>
>> I remember 10y ago every presentation started from the claim that 100B of
>> IoT would drive XXX traffic. It did not happen.
>>
>> Now we see presentations that AI would be talking to AI that generates
>>  traffic.
>>
>> Maybe some technology would push traffic next S-curve, maybe not. It is
>> still speculation.
>>
>>
>>
>> The traffic growth was stimulated (despite all VNIs) by 1) new
>> subscribers, 2) video quality for subscribers. Nothing else yet.
>>
>> It is almost finished for both trends. We are close to the plateau of
>> these S-curves.
>>
>> For some years (2013-2020) I was carefully looking at numbers for many
>> countries: it was always possible to split CAGR for these 2 components. The
>> video part was extremely consistent between countries. The subscriber part
>> was 100% proportional to subscriber CAGR.
>>
>> Everything else up to now was “marketing” to say it mildly.
>>
>>
>>
>> Reminder: nothing in nature could grow indefinitely. The limit always
>> exists. It is only a question of when.
>>
>>
>>
>> PS: Of course, marketing people could draw you any traffic growth – it
>> depends just on the marketing budget.
>>
>>
>>
>> Eduard
>>
>> *From:* 

Re: Routed optical networks

[Etienne-Victor Depasquale via NANOG]

Historically, this is what VNI has claimed

.

Cheers,

Etienne

On Thu, May 11, 2023 at 1:25 PM Vasilenko Eduard <
vasilenko.edu...@huawei.com> wrote:

> I did investigate traffic for every Carrier while dealing with it as a
> consultant (repeated many dozens of times).
>
> I have seen over a decade how traffic growth dropped year-over-year (from
> 60% to 25% in 2019 when I dropped this activity in 2020 – covid blocked
> travel).
>
> Sometimes I talk to old connections and they confirm that it is even less
> now.
>
> In rear cases, It is typically possible to find this information on the
> public Internet (I remember the case when Google disclosed traffic for
> Pakistan at the conference with the explanation that 30% is attributed to
> new subscribers, and an additional +30% is to more heavy content per
> subscriber).
>
> But mostly, it was confidential information from a discussion with
> Carriers – they all know very well their traffic growth.
>
> In general, traffic stat is pretty confidential. I did not have the
> motivation to aggregate it.
>
>
>
> Sandvine is not representative of global traffic because DPI is installed
> mostly for Mobiles. But Mobile subscriber is 10x less than fixed on traffic
> – it is not the biggest source. Moreover, Mobiles would look better growing
> because the limiting factor was on technology (5G proposed more than 4G, 4G
> proposed much more than 3G) – it would probably would less disruptive in
> the future.
>
> Fixed Carriers do not pay DPI premiums. And rarely share their traffic
> publicly. Sandvine could not see it.
>
>
>
> VNI is claiming so many things. Please show where exactly they show
> traffic growth (I am not interested in prediction speculations). Is it
> possible to understand CAGR for the 5 last years? Is it declining or
> growing? (traffic itself is for sure still growing)
>
>
>
> Of course, the disruption could come at any year and add a new S-curve
> (Metaverse?). But disruption is by definition not predictable.
>
>
>
> PS: Everything above and below in this thread is just my personal opinion.
>
>
>
> Eduard
>
> *From:* Etienne-Victor Depasquale [mailto:ed...@ieee.org]
> *Sent:* Thursday, May 11, 2023 12:48 PM
> *To:* Vasilenko Eduard 
> *Cc:* Dave Taht ; Phil Bedard ;
> NANOG 
> *Subject:* Re: Routed optical networks
>
>
>
> Eduard, academics cite the VNI (and the Sandvine Global reports).
>
>
>
> Do you know of alternative sources that show traffic growth data you're
> more comfortable with?
>
>
>
> Cheers,
>
>
>
> Etienne
>
>
>
> On Thu, May 11, 2023 at 9:34 AM Vasilenko Eduard <
> vasilenko.edu...@huawei.com> wrote:
>
> But it is speculation, not a trend yet.
>
> I remember 10y ago every presentation started from the claim that 100B of
> IoT would drive XXX traffic. It did not happen.
>
> Now we see presentations that AI would be talking to AI that generates
>  traffic.
>
> Maybe some technology would push traffic next S-curve, maybe not. It is
> still speculation.
>
>
>
> The traffic growth was stimulated (despite all VNIs) by 1) new
> subscribers, 2) video quality for subscribers. Nothing else yet.
>
> It is almost finished for both trends. We are close to the plateau of
> these S-curves.
>
> For some years (2013-2020) I was carefully looking at numbers for many
> countries: it was always possible to split CAGR for these 2 components. The
> video part was extremely consistent between countries. The subscriber part
> was 100% proportional to subscriber CAGR.
>
> Everything else up to now was “marketing” to say it mildly.
>
>
>
> Reminder: nothing in nature could grow indefinitely. The limit always
> exists. It is only a question of when.
>
>
>
> PS: Of course, marketing people could draw you any traffic growth – it
> depends just on the marketing budget.
>
>
>
> Eduard
>
> *From:* Dave Taht [mailto:dave.t...@gmail.com]
> *Sent:* Tuesday, May 9, 2023 11:41 PM
> *To:* Vasilenko Eduard 
> *Cc:* Phil Bedard ; Etienne-Victor Depasquale <
> ed...@ieee.org>; NANOG 
> *Subject:* Re: Routed optical networks
>
>
>
> Up until this moment I was feeling that my take on the decline of traffic
> growth was somewhat isolated, in that I have long felt that we are nearing
> the top of the S curve of the data we humans can create and consume. About
> the only source of future traffic growth I can think of comes from getting
> more humans online, and that is a mere another doubling.
>
>
>
> On the other hand, predictions such as 640k should be enough for everyone
> did not pan out.
>
>
>
> On the gripping hand, there has been an explosion of LLM stuff of late,
> with enormous models being widely distributed in just the past month:
>
>
>
> https://lwn.net/Articles/930939/
>
>
>
> Could the AIs takeoff lead to a resumption of traffic growth? I still
> don´t think so...
>
>
>
>
>
> On Thu, May 4, 2023 at 10:59 PM Vasilenko Eduard via NANOG <
> nanog@nanog.org> wrote:
>
> Disclaimer: 

RE: Routed optical networks

[Vasilenko Eduard via NANOG]

I did investigate traffic for every Carrier while dealing with it as a 
consultant (repeated many dozens of times).
I have seen over a decade how traffic growth dropped year-over-year (from 60% 
to 25% in 2019 when I dropped this activity in 2020 – covid blocked travel).
Sometimes I talk to old connections and they confirm that it is even less now.
In rear cases, It is typically possible to find this information on the public 
Internet (I remember the case when Google disclosed traffic for Pakistan at the 
conference with the explanation that 30% is attributed to new subscribers, and 
an additional +30% is to more heavy content per subscriber).
But mostly, it was confidential information from a discussion with Carriers – 
they all know very well their traffic growth.
In general, traffic stat is pretty confidential. I did not have the motivation 
to aggregate it.

Sandvine is not representative of global traffic because DPI is installed 
mostly for Mobiles. But Mobile subscriber is 10x less than fixed on traffic – 
it is not the biggest source. Moreover, Mobiles would look better growing 
because the limiting factor was on technology (5G proposed more than 4G, 4G 
proposed much more than 3G) – it would probably would less disruptive in the 
future.
Fixed Carriers do not pay DPI premiums. And rarely share their traffic 
publicly. Sandvine could not see it.

VNI is claiming so many things. Please show where exactly they show traffic 
growth (I am not interested in prediction speculations). Is it possible to 
understand CAGR for the 5 last years? Is it declining or growing? (traffic 
itself is for sure still growing)

Of course, the disruption could come at any year and add a new S-curve 
(Metaverse?). But disruption is by definition not predictable.

PS: Everything above and below in this thread is just my personal opinion.

Eduard
From: Etienne-Victor Depasquale [mailto:ed...@ieee.org]
Sent: Thursday, May 11, 2023 12:48 PM
To: Vasilenko Eduard 
Cc: Dave Taht ; Phil Bedard ; NANOG 

Subject: Re: Routed optical networks

Eduard, academics cite the VNI (and the Sandvine Global reports).

Do you know of alternative sources that show traffic growth data you're more 
comfortable with?

Cheers,

Etienne

On Thu, May 11, 2023 at 9:34 AM Vasilenko Eduard 
mailto:vasilenko.edu...@huawei.com>> wrote:
But it is speculation, not a trend yet.
I remember 10y ago every presentation started from the claim that 100B of IoT 
would drive XXX traffic. It did not happen.
Now we see presentations that AI would be talking to AI that generates  
traffic.
Maybe some technology would push traffic next S-curve, maybe not. It is still 
speculation.

The traffic growth was stimulated (despite all VNIs) by 1) new subscribers, 2) 
video quality for subscribers. Nothing else yet.
It is almost finished for both trends. We are close to the plateau of these 
S-curves.
For some years (2013-2020) I was carefully looking at numbers for many 
countries: it was always possible to split CAGR for these 2 components. The 
video part was extremely consistent between countries. The subscriber part was 
100% proportional to subscriber CAGR.
Everything else up to now was “marketing” to say it mildly.

Reminder: nothing in nature could grow indefinitely. The limit always exists. 
It is only a question of when.

PS: Of course, marketing people could draw you any traffic growth – it depends 
just on the marketing budget.

Eduard
From: Dave Taht [mailto:dave.t...@gmail.com]
Sent: Tuesday, May 9, 2023 11:41 PM
To: Vasilenko Eduard 
mailto:vasilenko.edu...@huawei.com>>
Cc: Phil Bedard mailto:bedard.p...@gmail.com>>; 
Etienne-Victor Depasquale mailto:ed...@ieee.org>>; NANOG 
mailto:nanog@nanog.org>>
Subject: Re: Routed optical networks

Up until this moment I was feeling that my take on the decline of traffic 
growth was somewhat isolated, in that I have long felt that we are nearing the 
top of the S curve of the data we humans can create and consume. About the only 
source of future traffic growth I can think of comes from getting more humans 
online, and that is a mere another doubling.

On the other hand, predictions such as 640k should be enough for everyone did 
not pan out.

On the gripping hand, there has been an explosion of LLM stuff of late, with 
enormous models being widely distributed in just the past month:

https://lwn.net/Articles/930939/

Could the AIs takeoff lead to a resumption of traffic growth? I still don´t 
think so...


On Thu, May 4, 2023 at 10:59 PM Vasilenko Eduard via NANOG 
mailto:nanog@nanog.org>> wrote:
Disclaimer: Metaverse has not changed Metro traffic yet. Then …

I am puzzled when people talk about 400GE and Tbps in the Mero context.
For historical reasons, Metro is still about 2*2*10GE (one “2” for redundancy, 
another “2” for capacity) in the majority of cases worldwide.
How many BRASes serve more than 4/1.5=27k users in the busy hour?
It means that 50GE is the best interface now for the 

Re: Routed optical networks

[Etienne-Victor Depasquale via NANOG]

Eduard, academics cite the VNI (and the Sandvine Global reports).

Do you know of alternative sources that show traffic growth data you're
more comfortable with?

Cheers,

Etienne

On Thu, May 11, 2023 at 9:34 AM Vasilenko Eduard <
vasilenko.edu...@huawei.com> wrote:

> But it is speculation, not a trend yet.
>
> I remember 10y ago every presentation started from the claim that 100B of
> IoT would drive XXX traffic. It did not happen.
>
> Now we see presentations that AI would be talking to AI that generates
>  traffic.
>
> Maybe some technology would push traffic next S-curve, maybe not. It is
> still speculation.
>
>
>
> The traffic growth was stimulated (despite all VNIs) by 1) new
> subscribers, 2) video quality for subscribers. Nothing else yet.
>
> It is almost finished for both trends. We are close to the plateau of
> these S-curves.
>
> For some years (2013-2020) I was carefully looking at numbers for many
> countries: it was always possible to split CAGR for these 2 components. The
> video part was extremely consistent between countries. The subscriber part
> was 100% proportional to subscriber CAGR.
>
> Everything else up to now was “marketing” to say it mildly.
>
>
>
> Reminder: nothing in nature could grow indefinitely. The limit always
> exists. It is only a question of when.
>
>
>
> PS: Of course, marketing people could draw you any traffic growth – it
> depends just on the marketing budget.
>
>
>
> Eduard
>
> *From:* Dave Taht [mailto:dave.t...@gmail.com]
> *Sent:* Tuesday, May 9, 2023 11:41 PM
> *To:* Vasilenko Eduard 
> *Cc:* Phil Bedard ; Etienne-Victor Depasquale <
> ed...@ieee.org>; NANOG 
> *Subject:* Re: Routed optical networks
>
>
>
> Up until this moment I was feeling that my take on the decline of traffic
> growth was somewhat isolated, in that I have long felt that we are nearing
> the top of the S curve of the data we humans can create and consume. About
> the only source of future traffic growth I can think of comes from getting
> more humans online, and that is a mere another doubling.
>
>
>
> On the other hand, predictions such as 640k should be enough for everyone
> did not pan out.
>
>
>
> On the gripping hand, there has been an explosion of LLM stuff of late,
> with enormous models being widely distributed in just the past month:
>
>
>
> https://lwn.net/Articles/930939/
>
>
>
> Could the AIs takeoff lead to a resumption of traffic growth? I still
> don´t think so...
>
>
>
>
>
> On Thu, May 4, 2023 at 10:59 PM Vasilenko Eduard via NANOG <
> nanog@nanog.org> wrote:
>
> Disclaimer: Metaverse has not changed Metro traffic yet. Then …
>
>
>
> I am puzzled when people talk about 400GE and Tbps in the Mero context.
>
> For historical reasons, Metro is still about 2*2*10GE (one “2” for
> redundancy, another “2” for capacity) in the majority of cases worldwide.
>
> How many BRASes serve more than 4/1.5=27k users in the busy hour?
>
> It means that 50GE is the best interface now for the majority of cases.
> 2*50GE=100Gbps is good room for growth.
>
> Of course, exceptions could be. I know BRAS that handles 86k subscribers
> (do not recommend anybody to push the limits – it was so painful).
>
>
>
> We have just 2 eyes and look at video content about 22h per week (on
> average). Our eyes do not permit us to see resolution better than
> particular for chosen distance (4k for typical TV, HD for smartphones, and
> so on). Color depth 10bits is enough for the majority, 12bits is sure
> enough for everybody. 120 frames/sec is enough for everybody. It would
> never change – it is our genetics.
>
> Fortunately for Carriers, the traffic has a limit. You have probably seen
> that every year traffic growth % is decreasing. The Internet is stabilizing
> and approaching the plateau.
>
> How much growth is still awaiting us? 1.5? 1.4? It needs separate
> research. The result would be tailored for whom would pay for the research.
>
> IMHO: It is not mandatory that 100GE would become massive in the metro. (I
> know that 100GE is already massive in the DC CLOS)
>
>
>
> Additionally, who would pay for this traffic growth? It also limits
> traffic at some point.
>
> I hope it would happen after we would get our 22h/4k/12bit/120hz.
>
>
>
> Now, you could argue that Metaverse would jump and multiply traffic by an
> additional 2x or 3x. Then 400GE may be needed.
>
> Sorry, but it is speculation yet. It is not a trend like the current
> (declining) traffic growth.
>
>
>
> Ed/
>
> *From:* NANOG [mailto:nanog-bounces+vasilenko.eduard=huawei@nanog.org]
> *On Behalf Of *Phil Bedard
> *Sent:* Thursday, May 4, 2023 8:32 PM
> *To:* Etienne-Victor Depasquale ; NANOG 
> *Subject:* Re: Routed optical networks
>
>
>
> It’s not necessarily metro specific although the metro networks could lend
> themselves to overall optimizations.
>
>
>
> The adoption of ZR/ZR+ IPoWDM currently somewhat corresponds with your
> adoption of 400G since today they require a QDD port.   There are 100G QDD
> ports but that’s not 

RE: Routed optical networks

[Vasilenko Eduard via NANOG]

But it is speculation, not a trend yet.
I remember 10y ago every presentation started from the claim that 100B of IoT 
would drive XXX traffic. It did not happen.
Now we see presentations that AI would be talking to AI that generates  
traffic.
Maybe some technology would push traffic next S-curve, maybe not. It is still 
speculation.

The traffic growth was stimulated (despite all VNIs) by 1) new subscribers, 2) 
video quality for subscribers. Nothing else yet.
It is almost finished for both trends. We are close to the plateau of these 
S-curves.
For some years (2013-2020) I was carefully looking at numbers for many 
countries: it was always possible to split CAGR for these 2 components. The 
video part was extremely consistent between countries. The subscriber part was 
100% proportional to subscriber CAGR.
Everything else up to now was “marketing” to say it mildly.

Reminder: nothing in nature could grow indefinitely. The limit always exists. 
It is only a question of when.

PS: Of course, marketing people could draw you any traffic growth – it depends 
just on the marketing budget.

Eduard
From: Dave Taht [mailto:dave.t...@gmail.com]
Sent: Tuesday, May 9, 2023 11:41 PM
To: Vasilenko Eduard 
Cc: Phil Bedard ; Etienne-Victor Depasquale 
; NANOG 
Subject: Re: Routed optical networks

Up until this moment I was feeling that my take on the decline of traffic 
growth was somewhat isolated, in that I have long felt that we are nearing the 
top of the S curve of the data we humans can create and consume. About the only 
source of future traffic growth I can think of comes from getting more humans 
online, and that is a mere another doubling.

On the other hand, predictions such as 640k should be enough for everyone did 
not pan out.

On the gripping hand, there has been an explosion of LLM stuff of late, with 
enormous models being widely distributed in just the past month:

https://lwn.net/Articles/930939/

Could the AIs takeoff lead to a resumption of traffic growth? I still don´t 
think so...


On Thu, May 4, 2023 at 10:59 PM Vasilenko Eduard via NANOG 
mailto:nanog@nanog.org>> wrote:
Disclaimer: Metaverse has not changed Metro traffic yet. Then …

I am puzzled when people talk about 400GE and Tbps in the Mero context.
For historical reasons, Metro is still about 2*2*10GE (one “2” for redundancy, 
another “2” for capacity) in the majority of cases worldwide.
How many BRASes serve more than 4/1.5=27k users in the busy hour?
It means that 50GE is the best interface now for the majority of cases. 
2*50GE=100Gbps is good room for growth.
Of course, exceptions could be. I know BRAS that handles 86k subscribers (do 
not recommend anybody to push the limits – it was so painful).

We have just 2 eyes and look at video content about 22h per week (on average). 
Our eyes do not permit us to see resolution better than particular for chosen 
distance (4k for typical TV, HD for smartphones, and so on). Color depth 10bits 
is enough for the majority, 12bits is sure enough for everybody. 120 frames/sec 
is enough for everybody. It would never change – it is our genetics.
Fortunately for Carriers, the traffic has a limit. You have probably seen that 
every year traffic growth % is decreasing. The Internet is stabilizing and 
approaching the plateau.
How much growth is still awaiting us? 1.5? 1.4? It needs separate research. The 
result would be tailored for whom would pay for the research.
IMHO: It is not mandatory that 100GE would become massive in the metro. (I know 
that 100GE is already massive in the DC CLOS)

Additionally, who would pay for this traffic growth? It also limits traffic at 
some point.
I hope it would happen after we would get our 22h/4k/12bit/120hz.

Now, you could argue that Metaverse would jump and multiply traffic by an 
additional 2x or 3x. Then 400GE may be needed.
Sorry, but it is speculation yet. It is not a trend like the current 
(declining) traffic growth.

Ed/
From: NANOG 
[mailto:nanog-bounces+vasilenko.eduard=huawei@nanog.org]
 On Behalf Of Phil Bedard
Sent: Thursday, May 4, 2023 8:32 PM
To: Etienne-Victor Depasquale mailto:ed...@ieee.org>>; NANOG 
mailto:nanog@nanog.org>>
Subject: Re: Routed optical networks

It’s not necessarily metro specific although the metro networks could lend 
themselves to overall optimizations.

The adoption of ZR/ZR+ IPoWDM currently somewhat corresponds with your adoption 
of 400G since today they require a QDD port.   There are 100G QDD ports but 
that’s not all that popular yet.   Of course there is work to do something 
similar in QSFP28 if the power can be reduced to what is supported by an 
existing QSFP28 port in most devices.   In larger networks with higher speed 
requirements and moving to 400G with QDD, using the DCO optics for connecting 
routers is kind of a no-brainer vs. a traditional muxponder.   Whether that’s 
over a ROADM based optical network or not, especially