CDN, Steam, Origin and NAT.

[Laurent Dumont]

Hi,

We are running a small-ish LAN event in Toronto where we have to use a 
single IP address to NAT between 250-350 players. I have been made aware 
of possible issues with different services like Steam, Origin and Twitch 
who can run into issues when a large number of connections seem to 
originate from a single IP address. I just wanted to poke the list to 
see if anyone can chime him on their experiences with NATing customers 
and the impact it might have on public services. I am usually using 
public IP address space for players when designing most large LAN 
events. Dealing with NAT for a medium-ish amount of customers is not 
something I am used to do.


It feels silly to worry about that when you assume that WISP 
sometimes(mostly?) use CGN when providing internet to customers. The 
same could be said of most large office buildings around the world.


I appreciate any input on the matter!

Thanks

Laurent

Re: DOCSIS 3.1 upstream

[Jean-Francois Mezei]

On 2016-04-20 13:09, Rob Seastrom wrote:

> Going to D3.1 in a meaningful way means migrating to either a mid-split at 85 
> MHz or a high split at 200 MHz 

Thanks. This is what I expected. But in the past, the canadian cablecos
had argued that removing the 42mhz upstream limitation was a huge
endeavour (they have to convicne CRTC to keep wholesale rates up, so
create artificial scarcity by claiming that replacing all those 42mhz
repeaters would cost a fortune, so they have to do node splits instead.

Arguing at CRTC is all about finding out what incumbent statements are
just spin and which are true.

Thanks for the links as well.é

> RFoG is its own kettle of fish.  Getting more than one channel on upstream 
> for RFoG is hard. 

But they can allocate a single very big channel, right ?  Or did you
mean a single traditional NTSC 6mhz channel ?

Re: Mobile providers in the US for backup access

[Eric Kuhnke]

ting is owned/run by tucows, who are now also doing a 1Gb (GPON?)
residential single home FTTH project...

http://www.fiercetelecom.com/europe/tags/tucows

On Wed, Apr 20, 2016 at 2:18 PM, Josh Reynolds  wrote:

> Ting's support is the BEST support I've ever had in the IT industry. I
> event ended up in a long discussion with one of the reps about custom
> roms :P
>
> On Wed, Apr 20, 2016 at 3:59 PM, Mike Hammett  wrote:
> > *shrugs* Seems to work here, though if Ting uses T-Mo and Sprint, I
> suppose Ting's more likely to have a good signal.
> >
> > I don't expect much support on a $6 mobile wireless service.
> >
> >
> >
> >
> > -
> > Mike Hammett
> > Intelligent Computing Solutions
> > http://www.ics-il.com
> >
> >
> >
> > Midwest Internet Exchange
> > http://www.midwest-ix.com
> >
> >
> > - Original Message -
> >
> > From: "Owen DeLong" 
> > To: "Mike Hammett" 
> > Cc: "NANOG" 
> > Sent: Wednesday, April 20, 2016 1:42:44 PM
> > Subject: Re: Mobile providers in the US for backup access
> >
> > I had horrible experience when I tried to use Freedom POP many years ago.
> >
> > Their customer service is awful and completely uncooperative. Their
> equipment did not work well
> > in my environment at all.
> >
> > I would not wish them on my worst enemy.
> >
> > Owen
> >
> >> On Apr 20, 2016, at 1:35 PM, Mike Hammett  wrote:
> >>
> >> I'd look at FreedomPOP's Netgear 341U. $20 - $50 NRC, single digit MRC
> for low usage.
> >>
> >>
> >>
> >>
> >> -
> >> Mike Hammett
> >> Intelligent Computing Solutions
> >> http://www.ics-il.com
> >>
> >>
> >>
> >> Midwest Internet Exchange
> >> http://www.midwest-ix.com
> >>
> >>
> >> - Original Message -
> >>
> >> From: "Dovid Bender" 
> >> To: "NANOG" 
> >> Sent: Wednesday, April 20, 2016 1:16:56 PM
> >> Subject: Mobile providers in the US for backup access
> >>
> >> A while ago some people mentioned that some US carriers have basic
> internet
> >> plans for backup access to their equipment. A few questions:
> >> 1) Do they give you a public IP per connection or do you tunnel back to
> a
> >> central location and then connect via the tunnel?
> >> 2) Which carriers offer this and what kind of devices do you use to
> >> connect? Is it simply a GSM card on a "MyFi" like device? We have lots
> of
> >> Pi's out there that we want backup access to.
> >> 3) Can you send off list contacts and pricing that you have gotten in
> the
> >> past?
> >>
> >> TIA.
> >>
> >> Dovid
> >
> >
>

Re: Mobile providers in the US for backup access

[Josh Reynolds]

Ting's support is the BEST support I've ever had in the IT industry. I
event ended up in a long discussion with one of the reps about custom
roms :P

On Wed, Apr 20, 2016 at 3:59 PM, Mike Hammett  wrote:
> *shrugs* Seems to work here, though if Ting uses T-Mo and Sprint, I suppose 
> Ting's more likely to have a good signal.
>
> I don't expect much support on a $6 mobile wireless service.
>
>
>
>
> -
> Mike Hammett
> Intelligent Computing Solutions
> http://www.ics-il.com
>
>
>
> Midwest Internet Exchange
> http://www.midwest-ix.com
>
>
> - Original Message -
>
> From: "Owen DeLong" 
> To: "Mike Hammett" 
> Cc: "NANOG" 
> Sent: Wednesday, April 20, 2016 1:42:44 PM
> Subject: Re: Mobile providers in the US for backup access
>
> I had horrible experience when I tried to use Freedom POP many years ago.
>
> Their customer service is awful and completely uncooperative. Their equipment 
> did not work well
> in my environment at all.
>
> I would not wish them on my worst enemy.
>
> Owen
>
>> On Apr 20, 2016, at 1:35 PM, Mike Hammett  wrote:
>>
>> I'd look at FreedomPOP's Netgear 341U. $20 - $50 NRC, single digit MRC for 
>> low usage.
>>
>>
>>
>>
>> -
>> Mike Hammett
>> Intelligent Computing Solutions
>> http://www.ics-il.com
>>
>>
>>
>> Midwest Internet Exchange
>> http://www.midwest-ix.com
>>
>>
>> - Original Message -
>>
>> From: "Dovid Bender" 
>> To: "NANOG" 
>> Sent: Wednesday, April 20, 2016 1:16:56 PM
>> Subject: Mobile providers in the US for backup access
>>
>> A while ago some people mentioned that some US carriers have basic internet
>> plans for backup access to their equipment. A few questions:
>> 1) Do they give you a public IP per connection or do you tunnel back to a
>> central location and then connect via the tunnel?
>> 2) Which carriers offer this and what kind of devices do you use to
>> connect? Is it simply a GSM card on a "MyFi" like device? We have lots of
>> Pi's out there that we want backup access to.
>> 3) Can you send off list contacts and pricing that you have gotten in the
>> past?
>>
>> TIA.
>>
>> Dovid
>
>

Re: Mobile providers in the US for backup access

[Mike Hammett]

*shrugs* Seems to work here, though if Ting uses T-Mo and Sprint, I suppose 
Ting's more likely to have a good signal. 

I don't expect much support on a $6 mobile wireless service. 




- 
Mike Hammett 
Intelligent Computing Solutions 
http://www.ics-il.com 



Midwest Internet Exchange 
http://www.midwest-ix.com 


- Original Message -

From: "Owen DeLong"  
To: "Mike Hammett"  
Cc: "NANOG"  
Sent: Wednesday, April 20, 2016 1:42:44 PM 
Subject: Re: Mobile providers in the US for backup access 

I had horrible experience when I tried to use Freedom POP many years ago. 

Their customer service is awful and completely uncooperative. Their equipment 
did not work well 
in my environment at all. 

I would not wish them on my worst enemy. 

Owen 

> On Apr 20, 2016, at 1:35 PM, Mike Hammett  wrote: 
> 
> I'd look at FreedomPOP's Netgear 341U. $20 - $50 NRC, single digit MRC for 
> low usage. 
> 
> 
> 
> 
> - 
> Mike Hammett 
> Intelligent Computing Solutions 
> http://www.ics-il.com 
> 
> 
> 
> Midwest Internet Exchange 
> http://www.midwest-ix.com 
> 
> 
> - Original Message - 
> 
> From: "Dovid Bender"  
> To: "NANOG"  
> Sent: Wednesday, April 20, 2016 1:16:56 PM 
> Subject: Mobile providers in the US for backup access 
> 
> A while ago some people mentioned that some US carriers have basic internet 
> plans for backup access to their equipment. A few questions: 
> 1) Do they give you a public IP per connection or do you tunnel back to a 
> central location and then connect via the tunnel? 
> 2) Which carriers offer this and what kind of devices do you use to 
> connect? Is it simply a GSM card on a "MyFi" like device? We have lots of 
> Pi's out there that we want backup access to. 
> 3) Can you send off list contacts and pricing that you have gotten in the 
> past? 
> 
> TIA. 
> 
> Dovid 

Re: Mobile providers in the US for backup access

[Dovid Bender]

Yang,

Thanks. I will check them out.


On Wed, Apr 20, 2016 at 4:02 PM, Yang Yu  wrote:

> On Wed, Apr 20, 2016 at 1:49 PM, Dovid Bender  wrote:
> > Thank you everyone for your feedback. I also wanted to know if any
> > providers offered unlimited 2g since in some cases they want to stream
> back
> > some audio as well.
>
> 4gantennashop has T-Mobile business with LTE data and unlimited 2G
> afterwards
>

Re: Mobile providers in the US for backup access

[Yang Yu]

On Wed, Apr 20, 2016 at 1:49 PM, Dovid Bender  wrote:
> Thank you everyone for your feedback. I also wanted to know if any
> providers offered unlimited 2g since in some cases they want to stream back
> some audio as well.

4gantennashop has T-Mobile business with LTE data and unlimited 2G afterwards

Re: Mobile providers in the US for backup access

[Donn Lasher via NANOG]

As a 3+ year “customer” of freedom-pop, I agree.

Their IP service was a bargain until the WiMax->LTE migration. Now the service 
is useless.
Their technical support continually redefines lack of effort.




On 4/20/16, 11:42 AM, "NANOG on behalf of Owen DeLong"  wrote:

>I had horrible experience when I tried to use Freedom POP many years ago.
>
>Their customer service is awful and completely uncooperative. Their equipment 
>did not work well
>in my environment at all.
>
>I would not wish them on my worst enemy.
>
>Owen
>
>> On Apr 20, 2016, at 1:35 PM, Mike Hammett  wrote:
>> 
>> I'd look at FreedomPOP's Netgear 341U. $20 - $50 NRC, single digit MRC for 
>> low usage. 
>> 
>> 
>> 
>> 
>> - 
>> Mike Hammett 
>> Intelligent Computing Solutions 
>> http://www.ics-il.com 
>> 
>> 
>> 
>> Midwest Internet Exchange 
>> http://www.midwest-ix.com 
>> 
>> 

Re: Mobile providers in the US for backup access

[Dovid Bender]

Thank you everyone for your feedback. I also wanted to know if any
providers offered unlimited 2g since in some cases they want to stream back
some audio as well.


On Wed, Apr 20, 2016 at 2:16 PM, Dovid Bender  wrote:

> A while ago some people mentioned that some US carriers have basic
> internet plans for backup access to their equipment. A few questions:
> 1) Do they give you a public IP per connection or do you tunnel back to a
> central location and then connect via the tunnel?
> 2) Which carriers offer this and what kind of devices do you use to
> connect? Is it simply a GSM card on a "MyFi" like device? We have lots of
> Pi's out there that we want backup access to.
> 3) Can you send off list contacts and pricing that you have gotten in the
> past?
>
> TIA.
>
> Dovid
>
>

Re: Mobile providers in the US for backup access

[Owen DeLong]

I had horrible experience when I tried to use Freedom POP many years ago.

Their customer service is awful and completely uncooperative. Their equipment 
did not work well
in my environment at all.

I would not wish them on my worst enemy.

Owen

> On Apr 20, 2016, at 1:35 PM, Mike Hammett  wrote:
> 
> I'd look at FreedomPOP's Netgear 341U. $20 - $50 NRC, single digit MRC for 
> low usage. 
> 
> 
> 
> 
> - 
> Mike Hammett 
> Intelligent Computing Solutions 
> http://www.ics-il.com 
> 
> 
> 
> Midwest Internet Exchange 
> http://www.midwest-ix.com 
> 
> 
> - Original Message -
> 
> From: "Dovid Bender"  
> To: "NANOG"  
> Sent: Wednesday, April 20, 2016 1:16:56 PM 
> Subject: Mobile providers in the US for backup access 
> 
> A while ago some people mentioned that some US carriers have basic internet 
> plans for backup access to their equipment. A few questions: 
> 1) Do they give you a public IP per connection or do you tunnel back to a 
> central location and then connect via the tunnel? 
> 2) Which carriers offer this and what kind of devices do you use to 
> connect? Is it simply a GSM card on a "MyFi" like device? We have lots of 
> Pi's out there that we want backup access to. 
> 3) Can you send off list contacts and pricing that you have gotten in the 
> past? 
> 
> TIA. 
> 
> Dovid 

Re: Mobile providers in the US for backup access

[Eric Kuhnke]

Look into Ting if all you want is a backup OOB path:

https://ting.com/rates?ab=1

$6/month per active SIM card. Plus billing for actual data usage. Use it in
your choice of HSPA+/LTE modem equipment. They're an MVNO using, if I
remember right, a combination of T-Mobile and Sprint.

On Wed, Apr 20, 2016 at 11:16 AM, Dovid Bender  wrote:

> A while ago some people mentioned that some US carriers have basic internet
> plans for backup access to their equipment. A few questions:
> 1) Do they give you a public IP per connection or do you tunnel back to a
> central location and then connect via the tunnel?
> 2) Which carriers offer this and what kind of devices do you use to
> connect? Is it simply a GSM card on a "MyFi" like device? We have lots of
> Pi's out there that we want backup access to.
> 3) Can you send off list contacts and pricing that you have gotten in the
> past?
>
> TIA.
>
> Dovid
>

Re: Mobile providers in the US for backup access

[Mike Hammett]

I'd look at FreedomPOP's Netgear 341U. $20 - $50 NRC, single digit MRC for low 
usage. 




- 
Mike Hammett 
Intelligent Computing Solutions 
http://www.ics-il.com 



Midwest Internet Exchange 
http://www.midwest-ix.com 


- Original Message -

From: "Dovid Bender"  
To: "NANOG"  
Sent: Wednesday, April 20, 2016 1:16:56 PM 
Subject: Mobile providers in the US for backup access 

A while ago some people mentioned that some US carriers have basic internet 
plans for backup access to their equipment. A few questions: 
1) Do they give you a public IP per connection or do you tunnel back to a 
central location and then connect via the tunnel? 
2) Which carriers offer this and what kind of devices do you use to 
connect? Is it simply a GSM card on a "MyFi" like device? We have lots of 
Pi's out there that we want backup access to. 
3) Can you send off list contacts and pricing that you have gotten in the 
past? 

TIA. 

Dovid 

Mobile providers in the US for backup access

[Dovid Bender]

A while ago some people mentioned that some US carriers have basic internet
plans for backup access to their equipment. A few questions:
1) Do they give you a public IP per connection or do you tunnel back to a
central location and then connect via the tunnel?
2) Which carriers offer this and what kind of devices do you use to
connect? Is it simply a GSM card on a "MyFi" like device? We have lots of
Pi's out there that we want backup access to.
3) Can you send off list contacts and pricing that you have gotten in the
past?

TIA.

Dovid

Re: DOCSIS 3.1 upstream

[Rob Seastrom]

> On Apr 14, 2016, at 10:43 PM, Jean-Francois Mezei 
>  wrote:
> 
> Also, have cablecos with such limits for upstream begun to upgrade the
> cable plant to increase the upstream bandwidth ? Canadian cablecos have
> told the regulator it would be prohibitively expensive to do so, but
> incumbents tend to exagerate these things when it is convenient. (they
> can then claim higher costs/congestion/need for node splits which
> increates regulated wholesale rates).

Going to D3.1 in a meaningful way means migrating to either a mid-split at 85 
MHz or a high split at 200 MHz (117 MHz is in the spec but I've never heard 
anyone talk about it).  It is not uncommon to see space (both for the upstream 
and downstream) freed up by sunsetting analog video channels.  Yes, one has to 
do a truck roll and swap out amplifiers etc. but that is relatively 
straightforward.  The "guts" pop out of the enclosure that hangs from the 
messenger wire and are then replaced.  You don't need to actually put a wrench 
on a coax connector in order to do this.  There may need to be plant 
rebalancing (checking and possibly replacing tilt compensators) but that's 
something that should be happening on an annual basis or perhaps more often, 
depending on local practice.

Fiber nodes are similar in terms of work to swap them out, though they may be 
more modular inside.

Amplifier insides:  
https://www.arris.com/globalassets/resources/data-sheets/starline-series-ble100-1-ghz-line-extender-data-sheet.pdf
Fiber node insides:  
https://www.arris.com/globalassets/resources/data-sheets/sg4000-modular-4x4-node-data-sheet.pdf

Passives (splitters, directional taps, terminators, and the like) are 
bidirectional and typically do not need to be replaced.

Possibly useful reading for folks who want an overview of how it all goes 
together:  
http://www.cablelabs.com/wp-content/uploads/2014/12/DOCSIS3-1_Pocket_Guide_2014.pdf

Without having read the Canadian cable providers' representations to the CRTC I 
am ill-equipped to pass judgemenent on them, but in my personal opinion any 
discussion of "D3.1 deployment" that doesn't plan for a refactoring of splits 
is a bit dishonest.


> And would it be correct that in RFoG deployment, the 42mhz limit
> disapears as the customer equipment talks directly tothe CMTS over fibre
> all the way ?

RFoG is its own kettle of fish.  Getting more than one channel on upstream for 
RFoG is hard.  There's a scheduler for transmitting on individual RF channels, 
but not for the upstream laser, so you could have two lasers coming on at the 
same time when two cablemodems (assume legacy D3.0 for a moment) transmit 
simultaneously on 19.40 MHz and 30.60 MHz - in an HFC plant where the mixing 
happens between two different radio frequencies in a copper wire and then feeds 
an single upstream fiber node, one doesn't have this problem.

-r

Re: phone fun, was GeoIP database issues and the real world consequences

[Dan Lacey]

Great explanation!

Remember that LECs (Local Exchange Carrier, CenturyLink, Verizon, etc.) 
typically get to decide how this all works...
ATT is still an 800 pound gorilla and a couple years ago stopped ALL 
payments to CLECs (Competitive Local Exchange Carrier, buy wholesale 
from LECs), took them all to court (which for a CLEC, it is almost 
impossible to find a good lawyer not on retainer to a LEC) and basically 
just told everyone what they would pay...


Since all the LECs started offering unlimited long distance, they could 
not afford the termination fees.

So... They changed them!!!

Telco is very different from data, not in the physical aspects, but in 
the business and political areas.


On 4/20/16 9:20 AM, John Levine wrote:

For the most part, “long distance” calls within the US are a thing of the
past and at least one mobile carrier now treats US/CA/MX as a single
local calling area

Is this a case of telcos having switched to IP trunks and can reach
other carriers for "free"

No, it's because fiber bandwidth is so cheap.  It's equally cheap whether
the framing is ATM or IP.


Or are wholesale long distance still billed between carriers but at
prices so low that they can afford to offer "free" long distance at
retail level ?

Some of each.  Some carriers do reciprocal compensation at very low
rates, small fractions of a cent per minute, some do bill and keep
with no settlements at all.

The history of settlements is closely tied to the history of the
Internet.  Before the Bell breakup separations (within Bell) and
settlements (between Bell and independents) were uncontentious, moving
money around to make the rate of return on invested capital at each
carrier come out right.

Then when cell phones were new, the Bell companies observed that
traffic was highly imbalanced, far more cell->landline than the other
way, so they demanded high reciprocal compensation, and the cellcos
were willing to pay since it gave the Bells the incentive to build the
interconnecting trunks.  One of Verizon's predecessors famously
derided "bilk and keep."

Then the dialup Internet became a big thing, the Bells ignored it as a
passing fad (which it was, but not for the reasons they thought), and
CLECs realized they could build modem banks and make a lot of money
from the incoming calls from Bell customers to the modems.  So the
Bells did a pirouette and suddenly discovered that bill and keep was a
law of nature and recip comp was a quaint artifact that needed to be
snuffed out as fast as possible.

These days the FCC likes to see cost justifications for settlements,
and the actual per-minute cost of calls is tiny compared to the fixed
costs of the links and equipment.  The main place where you see
settlements is to tiny local telcos with very high costs, with the per
minute payments a deliberate subsidy to them.  Then some greedy little
telcos added conference call lines to pump up their incoming traffic ...

R's,
John

Re: Latency, TCP ACKs and upload needs

[Tony Finch]

Leo Bicknell  wrote:
>
> 1460 byte payloads down, maybe 64 byte acks on the return, and with SACK
> which is widely deployed an ACK every 2-4 packets.  You would see about
> 2,140 packets/sec downstream (25Mbps/1460), and perhaps send 1070 ACKs
> back upstream, at 64 bytes each, or about 68Kbps.  Well under the 1Mbps
> upstream bandwidth.

Note that with delayed ACKs (RFC 1122) there is an ACK for every other
packet; SACK should do better than that.

Tony.
-- 
f.anthony.n.finch    http://dotat.at/  -  I xn--zr8h punycode
Humber, Thames: Northwest, veering north or northeast, 4 or 5. Slight or
moderate. Fair. Good.

Re: Latency, TCP ACKs and upload needs

[Mikael Abrahamsson]

On Tue, 19 Apr 2016, Jean-Francois Mezei wrote:

Considering a single download TCP connection. I am aware that modern TCP 
stacks will rationalize ACKs and send 1 ACK for every x packets 
received, thus reducing upload bandwidth requirements. Is this basically 
widespread and assumed that everyone has that ?


Typically you'll see one ACK per 2 packets, so you need approximately 1:50 
bytes up/down ratio for the ACKs.


It's possible to have middle boxes suppress some ACKs, please see thread 
here:


https://www.ietf.org/mail-archive/web/aqm/current/msg01482.html

--
Mikael Abrahamssonemail: swm...@swm.pp.se

Re: Latency, TCP ACKs and upload needs

[Lee]

On 4/20/16, Leo Bicknell wrote:
>
> Others have already answered with the technical details.  Let me take a
> stab at some more, uh, variable items.

  [.. snip lots ..]

> 90%+ of the stacks deployed will be too small.  Modern Unix generally
> has "autotuning" TCP stacks, but I don't think Windows or OS X has
> those features yet (but I'd be very happy to be wrong on that point).

Windows has had an autotuning stack since at least Vista.

Regards,
Lee

Re: Latency, TCP ACKs and upload needs

[Jean-Francois Mezei]

Thanks to all for the sanity check.

Always depressing when you think you may have a good argument but after
much reading, you find out you don't :-(

BTW, in case someone knows. With the recent "beam" satellites having a
lot of different focused antennas, how does the uplink work ?

Does all traffic pass through a central "switch" which then directs
packets to the approperiate antenna ?

Would a each beam directed at a served area be paired with its own
dedicated beam directed at ground station ? Or does the uplink from
ground station carry traffic for multiple beams and thus becomes the
bottleneck ?

(Xplornet bragged about its next satellite having 20gbps capacity, but
IF the uplink from ground station is also at 20gps and serves 5 beams
aimed at Canada, then on average each beam only gets 4gbps ?)


With regards to the "dream" of having 350 low orbit satellites covering
the globe for Internet, does anyone know how the uplink will be done?
Won't there be a bottleneck if in serving Canada's north, satellites
currently speeding over the region have to use satellite-to-satellite
links to carry information until it reaches a satellite that is over a
ground station in the south ?

or is it expected that ground stations will be built "near" each area to
be served ?

(Am trying to justify that satellite should be reserved for people truly
isolated and that Nunavut communities should get undersea fibre and work
need to start now because of long construction times during which
satellites will fall further back in terms of capacity needs).

Re: phone fun, was GeoIP database issues and the real world consequences

[John Levine]

>> For the most part, “long distance” calls within the US are a thing of the
>> past and at least one mobile carrier now treats US/CA/MX as a single
>> local calling area 
>
>Is this a case of telcos having switched to IP trunks and can reach
>other carriers for "free"

No, it's because fiber bandwidth is so cheap.  It's equally cheap whether
the framing is ATM or IP.

>Or are wholesale long distance still billed between carriers but at
>prices so low that they can afford to offer "free" long distance at
>retail level ?

Some of each.  Some carriers do reciprocal compensation at very low
rates, small fractions of a cent per minute, some do bill and keep
with no settlements at all.

The history of settlements is closely tied to the history of the
Internet.  Before the Bell breakup separations (within Bell) and
settlements (between Bell and independents) were uncontentious, moving
money around to make the rate of return on invested capital at each
carrier come out right.

Then when cell phones were new, the Bell companies observed that
traffic was highly imbalanced, far more cell->landline than the other
way, so they demanded high reciprocal compensation, and the cellcos
were willing to pay since it gave the Bells the incentive to build the
interconnecting trunks.  One of Verizon's predecessors famously
derided "bilk and keep."

Then the dialup Internet became a big thing, the Bells ignored it as a
passing fad (which it was, but not for the reasons they thought), and
CLECs realized they could build modem banks and make a lot of money
from the incoming calls from Bell customers to the modems.  So the
Bells did a pirouette and suddenly discovered that bill and keep was a
law of nature and recip comp was a quaint artifact that needed to be
snuffed out as fast as possible.

These days the FCC likes to see cost justifications for settlements,
and the actual per-minute cost of calls is tiny compared to the fixed
costs of the links and equipment.  The main place where you see
settlements is to tiny local telcos with very high costs, with the per
minute payments a deliberate subsidy to them.  Then some greedy little
telcos added conference call lines to pump up their incoming traffic ...

R's,
John

Re: phone fun, was GeoIP database issues and the real world consequences

[Owen DeLong]

> On Apr 20, 2016, at 7:59 AM, Jean-Francois Mezei 
>  wrote:
> 
> On 2016-04-20 10:52, Owen DeLong wrote:
> 
>> For the most part, “long distance” calls within the US are a thing of the
>> past and at least one mobile carrier now treats US/CA/MX as a single
>> local calling area 
> 
> 
> Is this a case of telcos having switched to IP trunks and can reach
> other carriers for "free"
> 
> Or are wholesale long distance still billed between carriers but at
> prices so low that they can afford to offer "free" long distance at
> retail level ?

I think it boiled down to a recognition that the costs of billing were 
beginning to account for something like $0.99 of every $1 billed.

Owen

Re: Latency, TCP ACKs and upload needs

[Lee]

On 4/19/16, Jean-Francois Mezei  wrote:
> As part of the ongoing CRTC hearings, the incumbents' claim that
> continued implementation of the current 5/1 standard would make Canada a
> world leader for broadband in the future.
>
> A satellite company who currently can't even deliver its advertised 5/1
> now brags its next satellite will deliver 25/1.

Take a look at https://www.rfc-editor.org/rfc/rfc3449.txt
  TCP Performance Implications of Network Path Asymmetry

> So I have a few questions:
>
> Considering a single download TCP connection. I am aware that modern TCP
> stacks will rationalize ACKs and send 1 ACK for every x packets
> received, thus reducing upload bandwidth requirements. Is this basically
> widespread and assumed that everyone has that ?

The usual defaults are to ack every other packet & wait 200ms before
acking a single packet.


> Also, as you split available bandwidth between multiple streams, won't
> ack upload requirements increase because ACK rationalisation happens far
> less often sicne each TCP connection has its own context for ACKs?

Yes.  And multiple streams will interfere with each other.
It used to be popular to split a download into multiple streams but I
thought that went out of style now that tcp window scaling is
generally enabled by default.

> When one considers the added latency of satellite links, does 25/1 make
> sense ?  (I need a sanity check to distinguish between marketing spin
> presented to the regulator and real life)
>
> I noticed that in the USA, EXEDE Satellite advertises 12/3 plans and
> they are also on a VIA Sat satellite, presumably the same vehicle that
> Xplornet tries to deliver its measly 5/1 on. Would all beams be
> identical on a satellite or can they be configured differently with a
> ISP adjustable rate of upload/download inside the same spectrum ?
>
>
>
>
> Also, when you establish a TCP connection, do most stacks have a default
> window size that gives the sender enough "patience" to wait long enough
> for the ACK ?

There's an initial timeout that's on the order of 3-5 seconds.  Once
the xfer starts ... I've forgotten the details, but the retransmission
timer is based on the "smoothed round trip time" (srtt).

> If sender sends packet 457, doesn't get ACK in time and resends 457,
> doesn't that also result in reduction in window size (the very opposite
> of what would be needed in high latency links) ?

The window size is what the receiver advertises, the congestion window
(cwnd) is what the sender computes based on the advertised window size
& packet loss.  cwnd is what gets reduced when there's packet loss.
& yes, reducing cwnd is bad for thruput as is not having selective
acks (sack) enabled on the receiver.

> And when the first ACK finally arrives, won't the sender assume this ACK
> was for the resent 457 ?

The sender keeps track of the 'smoothed round trip time' (srtt).
Since it can't tell if the ack is for the original or retransmitted
pkt, it's not supposed to use that ack for updating the rtt

>   Or is satellite latency low enough that
> stacks all have enough default "patience" to wait for ACKs and this is a
> non issue ?

should be a non-issue

> (Note Xplornet refused to answer questions on whether they operate
> special proxies at their gound stations to manage TCP connections to
> appear "close").
>
>
>
> What i am trying to get at here is whether 25/1 on satellite, in real
> life with a few apps exchanging data, would actually be able to make use
> of the 25 download speed or whether the limited 1mbps upload would choke
> the downloads ?

dunno.  Assuming the bandwidth is available, I suspect you could get
25Mb/s doing something like downloading a movie from archive.org but
for anything interactive like web surfing / gaming I'd bet no - but
because of latency, not the 1Mb/s uplink speed.

Regards,
Lee

Re: phone fun, was GeoIP database issues and the real world consequences

[Jean-Francois Mezei]

On 2016-04-20 10:52, Owen DeLong wrote:

> For the most part, “long distance” calls within the US are a thing of the
> past and at least one mobile carrier now treats US/CA/MX as a single
> local calling area 


Is this a case of telcos having switched to IP trunks and can reach
other carriers for "free"

Or are wholesale long distance still billed between carriers but at
prices so low that they can afford to offer "free" long distance at
retail level ?

Re: phone fun, was GeoIP database issues and the real world consequences

[Owen DeLong]

> On Apr 15, 2016, at 2:21 PM, Mark Andrews  wrote:
> 
> 
> In message , David Barak 
> writes
> :
>>> On Apr 15, 2016, at 3:09 PM, Mark Andrews  wrote:
>>> 
>>> Australia is about the area as the US and has always had caller
>>> pays and seperate area codes for mobiles.
>> 
>> Australia has fewer people than Texas, and is more than an order of
>> magnitude smaller than the US by population.  Effects of scale apply here
>> in terms of path dependence for solutions.
>> 
>> David Barak
>> Sent from mobile device, please excuse autocorrection artifacts
> 
> NA has a 10 digit scheme (3 area code - 7 local) though most of the
> time you end up dialing the 10 digits.

Not an entirely accurate description. In fact, in the US, it’s more of
a 3-tier mechanism… 3 area code, 3 prefix, 4 local.

As a general rule, a prefix exists within a single CO (modulo cutouts
for LNP, etc.). There are usually multiple prefixes per CO since most
COs serve significantly more than 10,000 numbers.

In the US, Area codes do not cross state lines and in most cases do
not cross LATA boundaries, either.

For the most part, “long distance” calls within the US are a thing of the
past and at least one mobile carrier now treats US/CA/MX as a single
local calling area (calls to/from anywhere in those three countries are
the same price (generally included/free) as calls between two phones
standing next to each other.

> 
> Australia has a 9 digit scheme (1 area code - 8 local)
> 
> Yes the area codes are huge (multi-state) and some "local" calls
> are sometimes long distance.  In my lifetime local calls have gone
> from 6 digits to 7 and then 8 digits.  The last change got rid of
> lots of area codes and expanded all the local numbers to 8 digits.
> This allows you to use what was a Canberra number in Sydney as they
> are now all in the same area code.  Canberra and Sydney are a 3
> hour drive apart.
> 
> We are no longer in a age where we need to route calls on a digit
> by digit basis.

While this is true, there are still significant differences in scale and cost
structures between AU and US.

Owen

Arista Routing Solutions

[Colton Conor]

NANOG,

I know Arista is typically a switch manufacturer, but with their recently
announced Arista 7500R Series and soon to be announced but already shipping
7280R Series Arista is officially getting into the routing game. The fixed
1U 7280R Series looks quite impressive. The 7500R series is your
traditional chassis and line card based solution.

Both of these products have the ability to hold the full internet routing
table, and Arista is working on MPLS features. Both of these new products
use the latest Broadcom Jerico chipsets.

I would like to know how viable of a product NANOG thinks these Arista
routers are compared to service provider grade routers from Cisco, Juniper,
ALU, and Brocade?

Cost wise, Arista seems to be much, much less per port. For example, the
1U Arista 7280R with 48x10GbE (SFP+) & 6x100GbE QSFP cost about the same as
what Juniper sells a MX104 with only four 10G ports for (Under 20K).

Can the Arista EOS software combine with their hardware based on the
Broadcom Jericho chipset truly compete  with the custom chipsets and
accompanying software from the big guys?

Re: Latency, TCP ACKs and upload needs

[Leo Bicknell]

Others have already answered with the technical details.  Let me take a 
stab at some more, uh, variable items.

In a message written on Tue, Apr 19, 2016 at 09:29:12PM -0400, Jean-Francois 
Mezei wrote:
> Also, when you establish a TCP connection, do most stacks have a default
> window size that gives the sender enough "patience" to wait long enough
> for the ACK ?

Your question is phrased backwards.  All will wait for the ACK, the
timeouts are long (30-120 seconds).  The issue is that you only get
one window of data per RTT, so if the window is too small, it will
choke the connection.

90%+ of the stacks deployed will be too small.  Modern Unix generally
has "autotuning" TCP stacks, but I don't think Windows or OS X has
those features yet (but I'd be very happy to be wrong on that point).
Regardless of satellite uplink/downlink speeds, boxes generally need
to be tuned to get maximum performance on satellite.

> What i am trying to get at here is whether 25/1 on satellite, in real
> life with a few apps exchanging data, would actually be able to make use
> of the 25 download speed or whether the limited 1mbps upload would choke
> the downloads ?

With a properly tuned stack what you're describing is not a problem.

1460 byte payloads down, maybe 64 byte acks on the return, and with SACK
which is widely deployed an ACK every 2-4 packets.  You would see about
2,140 packets/sec downstream (25Mbps/1460), and perhaps send 1070 ACKs
back upstream, at 64 bytes each, or about 68Kbps.  Well under the 1Mbps
upstream bandwidth.

-- 
Leo Bicknell - bickn...@ufp.org
PGP keys at http://www.ufp.org/~bicknell/


pgpRCAH4V8jii.pgp
Description: PGP signature

Re: ASR-9K CPU troubleshooting

[Rukka Pal]

This document (BRKARC-2017) turned out to be very useful to determine the
possible root-cause. The utilization of the spp and netio processes
increase if the router/line-card is software switching traffic, in our case
ICMP. We will test the policing feature and implement it.


On Wed, Apr 20, 2016 at 1:17 AM, Micah Croff  wrote:

> I've experienced similar behavior on other platforms as well.  Sometimes
> the output of the box is not correct.  We were able to prove this to the
> vendor by conducting experiments and graphing the CPU.  One of the
> protocols they said "couldn't possibly be causing this" turned out to be
> the root of the problem.
>
> I live by one rule when troubleshooting:
> The box is a lie.
>
> Micah
>
>
> On Tue, Apr 19, 2016 at 4:06 PM, Laurent Dumont 
> wrote:
>
> > It coincides with nothing else? More traffic? CPU increasing at regular
> > intervals every day without any obvious reasons is probably something
> worth
> > looking into!
> >
> > On 4/18/2016 2:14 PM, Scott Weeks wrote:
> >
> >>
> >> --- rege...@gmail.com wrote:
> >> From: Rukka Pal 
> >>
> >> How do you guys troubleshoot high CPU utilization on the ASR-9K
> platform?
> >> Detailed guides are available for IOS platforms, but I can't seem to
> find
> >> anything useful for the ASR.
> >>
> >> The average line-card (0/0/CPU0: A9K-24x10GE-TR) CPU utilization of my
> >> routers is about 10%, however recently I have noticed that 3-5 times a
> day
> >> it increases to 40% and stays there for about an hour (20% spp + 10%
> netio
> >> + the rest).
> >>
> >> I know this is well withing the acceptable range, but I am the kind of
> >> person who likes to understand every change in his network and during
> the
> >> investigation I had to realize that I simply don't have the tools to
> >> troubleshoot the ASR CPU.
> >> ---
> >>
> >>
> >> On cisco: sho proc cpu
> >>
> >> scott
> >>
> >
> >
>