Re: Apple Catalina Appears to Introduce Massive Jitter - SOLVED! - Update!

[Dan Halperin via NANOG]

FWIW, in this really interesting read

about
the latest vulnerability released by Project Zero, they talk about AWDL and
how if this is in use (AirDrop enabled?) then you're going to get tons of
jitter/packet loss.

In this way [TDM hopping between Wi-Fi Access Point and AWDL Mesh] the
> device can appear to be connected to the access point whilst also
> participating in the AWDL mesh at the same time. Of course, frames might be
> missed from both the AP and the AWDL mesh but the protocols are treating
> radio as an unreliable transport anyway so this only really has an impact
> on throughput. A large part of the AWDL protocol involves trying to
> synchronize the channel switching between peers to improve throughput.


Dan

On Sun, Nov 22, 2020 at 2:30 PM d...@darwincosta.com 
wrote:

>
>
> On 22 Nov 2020, at 20:43, J. Hellenthal  wrote:
>
> You can supposedly still use 4.5 4.6 on Big Sur if you do the following
> but I have not tested it on Little Snotch, works fine for personal software
> and others ...
>
> codesign -dvvv littlesnitch.package name
> Save the team identifier
> Boot into recovery mode
> Open terminal and type the following...
> spctl kext-consent add 
> Reboot into normal user mode and install version 4
>
> Thanks for the hint. Will have a look into it.
>
>
>
>
> --
>  J. Hellenthal
>
> The fact that there's a highway to Hell but only a stairway to Heaven says
> a lot about anticipated traffic volume.
>
> On Nov 22, 2020, at 11:55, d...@darwincosta.com wrote:
>
> 
>
> On 22 Nov 2020, at 10:17, Mark Tinka  wrote:
>
>  So after installing Little Snitch and basically denying "trustd" any
> kind of Internet access, I have been seeing reasonably normal jitter with
> Bluetooth enabled.
>
> I actually “saw the same” on Catalina while using little snitch.
>
> “Saw the same” after installing yesterday Big Sur and suddenly received a
> notification “this version of little snitch is no longer supported by
> macOS. It’s looks like I have to pay 25€ for a new compatible version.
>
>
> It's not that Bluetooth stops scanning, but it's not scanning as
> aggressively. So after a few minutes, there will be very high jitter when
> Bluetooth scans the environment, but it would affect only a single packet.
> It's easily reduced its chattiness by 99%.
>
> I don't have any empirical data to support the claim that Little Snitch
> has anything to do with it (and I am too lazy to dig further into it), but
> the reduction in jitter is massively noticeable since Little Snitch. Which
> means I can now run Catalina with Bluetooth enabled and not have any wi-fi
> problems.
>
> Just FYI, for the archives :-).
>
> Mark.
>
> Cheers,
> Darwin-.
>
>

Re: Apple Catalina Appears to Introduce Massive Jitter - SOLVED! - Update!

[d...@darwincosta.com]

> On 22 Nov 2020, at 20:43, J. Hellenthal  wrote:
> 
> You can supposedly still use 4.5 4.6 on Big Sur if you do the following but 
> I have not tested it on Little Snotch, works fine for personal software and 
> others ...
> 
> codesign -dvvv littlesnitch.package name
> Save the team identifier
> Boot into recovery mode
> Open terminal and type the following...
> spctl kext-consent add 
> Reboot into normal user mode and install version 4
Thanks for the hint. Will have a look into it.
> 
> 
> 
> -- 
>  J. Hellenthal
> 
> The fact that there's a highway to Hell but only a stairway to Heaven says a 
> lot about anticipated traffic volume.
> 
>>> On Nov 22, 2020, at 11:55, d...@darwincosta.com wrote:
>>> 
>> 
 On 22 Nov 2020, at 10:17, Mark Tinka  wrote:
 
>>>  So after installing Little Snitch and basically denying "trustd" any kind 
>>> of Internet access, I have been seeing reasonably normal jitter with 
>>> Bluetooth enabled.
>> I actually “saw the same” on Catalina while using little snitch. 
>> 
>> “Saw the same” after installing yesterday Big Sur and suddenly received a 
>> notification “this version of little snitch is no longer supported by macOS. 
>> It’s looks like I have to pay 25€ for a new compatible version. 
>>> 
>>> It's not that Bluetooth stops scanning, but it's not scanning as 
>>> aggressively. So after a few minutes, there will be very high jitter when 
>>> Bluetooth scans the environment, but it would affect only a single packet. 
>>> It's easily reduced its chattiness by 99%.
>>> 
>>> I don't have any empirical data to support the claim that Little Snitch has 
>>> anything to do with it (and I am too lazy to dig further into it), but the 
>>> reduction in jitter is massively noticeable since Little Snitch. Which 
>>> means I can now run Catalina with Bluetooth enabled and not have any wi-fi 
>>> problems.
>>> 
>>> Just FYI, for the archives :-).
>>> 
>>> Mark.
>> 
>> Cheers,
>> Darwin-. 

Re: Apple Catalina Appears to Introduce Massive Jitter - SOLVED! - Update!

[J. Hellenthal via NANOG]

You can supposedly still use 4.5 4.6 on Big Sur if you do the following but I 
have not tested it on Little Snotch, works fine for personal software and 
others ...

codesign -dvvv littlesnitch.package name
Save the team identifier
Boot into recovery mode
Open terminal and type the following...
spctl kext-consent add 
Reboot into normal user mode and install version 4



-- 
 J. Hellenthal

The fact that there's a highway to Hell but only a stairway to Heaven says a 
lot about anticipated traffic volume.

> On Nov 22, 2020, at 11:55, d...@darwincosta.com wrote:
> 
> 
>>> On 22 Nov 2020, at 10:17, Mark Tinka  wrote:
>>> 
>>  So after installing Little Snitch and basically denying "trustd" any kind 
>> of Internet access, I have been seeing reasonably normal jitter with 
>> Bluetooth enabled.
> I actually “saw the same” on Catalina while using little snitch. 
> 
> “Saw the same” after installing yesterday Big Sur and suddenly received a 
> notification “this version of little snitch is no longer supported by macOS. 
> It’s looks like I have to pay 25€ for a new compatible version. 
>> 
>> It's not that Bluetooth stops scanning, but it's not scanning as 
>> aggressively. So after a few minutes, there will be very high jitter when 
>> Bluetooth scans the environment, but it would affect only a single packet. 
>> It's easily reduced its chattiness by 99%.
>> 
>> I don't have any empirical data to support the claim that Little Snitch has 
>> anything to do with it (and I am too lazy to dig further into it), but the 
>> reduction in jitter is massively noticeable since Little Snitch. Which means 
>> I can now run Catalina with Bluetooth enabled and not have any wi-fi 
>> problems.
>> 
>> Just FYI, for the archives :-).
>> 
>> Mark.
> 
> Cheers,
> Darwin-. 

Re: Apple Catalina Appears to Introduce Massive Jitter - SOLVED! - Update!

[Randy Bush]

> “Saw the same” after installing yesterday Big Sur and suddenly
> received a notification “this version of little snitch is no longer
> supported by macOS. It’s looks like I have to pay 25€ for a new
> compatible version.

and big slur bypasses it for some nefarious uses, e.g. [un]trustd

i am sticking with catatonic

randy

Re: Apple Catalina Appears to Introduce Massive Jitter - SOLVED! - Update!

[d...@darwincosta.com]

> On 22 Nov 2020, at 10:17, Mark Tinka  wrote:
> 
>  So after installing Little Snitch and basically denying "trustd" any kind 
> of Internet access, I have been seeing reasonably normal jitter with 
> Bluetooth enabled.
I actually “saw the same” on Catalina while using little snitch. 

“Saw the same” after installing yesterday Big Sur and suddenly received a 
notification “this version of little snitch is no longer supported by macOS. 
It’s looks like I have to pay 25€ for a new compatible version. 
> 
> It's not that Bluetooth stops scanning, but it's not scanning as 
> aggressively. So after a few minutes, there will be very high jitter when 
> Bluetooth scans the environment, but it would affect only a single packet. 
> It's easily reduced its chattiness by 99%.
> 
> I don't have any empirical data to support the claim that Little Snitch has 
> anything to do with it (and I am too lazy to dig further into it), but the 
> reduction in jitter is massively noticeable since Little Snitch. Which means 
> I can now run Catalina with Bluetooth enabled and not have any wi-fi problems.
> 
> Just FYI, for the archives :-).
> 
> Mark.

Cheers,
Darwin-. 

Re: Apple Catalina Appears to Introduce Massive Jitter - SOLVED! - Update!

[Mark Tinka]

On 11/22/20 12:25, d...@darwincosta.com wrote:

“Saw the same” after installing yesterday Big Sur and suddenly 
received a notification “this version of little snitch is no longer 
supported by macOS. It’s looks like I have to pay 25€ for a new 
compatible version.


My advice would be to keep Catalina :-).

Mark.

Re: Apple Catalina Appears to Introduce Massive Jitter - SOLVED! - Update!

[Mark Tinka]

So after installing Little Snitch and basically denying "trustd" any 
kind of Internet access, I have been seeing reasonably normal jitter 
with Bluetooth enabled.


It's not that Bluetooth stops scanning, but it's not scanning as 
aggressively. So after a few minutes, there will be very high jitter 
when Bluetooth scans the environment, but it would affect only a single 
packet. It's easily reduced its chattiness by 99%.


I don't have any empirical data to support the claim that Little Snitch 
has anything to do with it (and I am too lazy to dig further into it), 
but the reduction in jitter is massively noticeable since Little Snitch. 
Which means I can now run Catalina with Bluetooth enabled and not have 
any wi-fi problems.


Just FYI, for the archives :-).

Mark.

Re: Apple Catalina Appears to Introduce Massive Jitter - SOLVED!

[David Hubbard]

The leaking past the VPN thing is pretty obnoxious.  There are people who may 
be subject to policy and/or regulatory requirements that don’t permit split 
tunnels (even if supposedly not in userspace), so it will be interesting to see 
what burdens the use of an OS that intentionally leaks data will place on 
certain companies.  In contrast, it’s pretty funny that while they let their 
own data collection apps leak past a tunnel to call home, they do not let the 
link local ipv6 traffic that Sidecar uses leak past a non-split VPN; i.e. if 
I’m on corporate VPN, I can no longer connect my tablet as a Sidecar monitor to 
my Macbook because that traffic is blocked.


From: NANOG  on behalf 
of Mark Tinka 
Organization: SEACOM
Date: Tuesday, November 17, 2020 at 2:37 AM
To: Saku Ytti 
Cc: North American Network Operators Group 
Subject: Re: Apple Catalina Appears to Introduce Massive Jitter - SOLVED!


On 11/17/20 09:26, Saku Ytti wrote:
https://support.apple.com/en-us/HT202491

I am not trying to make any argument, just wanted to add context.

Yes, saw that too, and that post by Apple is also highlighted (and explained) 
in the same report.

The Gatekeeper OCSP checks remain unencrypted.

It still leave two glaring issues:

  *   Apple are still not saying anything about their OS apps bypassing local 
firewalls and leaking our IP address and location past any VPN's we may be 
running on Big Sur.

  *   The backdoor in iMessage's encryption that allows Apple and other 
"interested parties" to view our iMessage texts.
Mark.

Re: Apple Catalina Appears to Introduce Massive Jitter - SOLVED!

[Saku Ytti]

On Tue, 17 Nov 2020 at 09:35, Mark Tinka  wrote:

> Yes, saw that too, and that post by Apple is also highlighted (and explained) 
> in the same report.

Aah, I had not seen the updated version of it, thanks.

-- 
  ++ytti

Re: Apple Catalina Appears to Introduce Massive Jitter - SOLVED!

[Mark Tinka]

On 11/17/20 09:26, Saku Ytti wrote:


https://support.apple.com/en-us/HT202491
I am not trying to make any argument, just wanted to add context.


Yes, saw that too, and that post by Apple is also highlighted (and 
explained) in the same report.


The Gatekeeper OCSP checks remain unencrypted.

It still leave two glaring issues:

 * Apple are still not saying anything about their OS apps bypassing
   local firewalls and leaking our IP address and location past any
   VPN's we may be running on Big Sur.

 * The backdoor in iMessage's encryption that allows Apple and other
   "interested parties" to view our iMessage texts.

Mark.

Re: Apple Catalina Appears to Introduce Massive Jitter - SOLVED!

[Saku Ytti]

On Tue, 17 Nov 2020 at 09:17, Mark Tinka  wrote:

> https://sneak.berlin/20201112/your-computer-isnt-yours/
https://support.apple.com/en-us/HT202491

I am not trying to make any argument, just wanted to add context.

-- 
  ++ytti

Re: Apple Catalina Appears to Introduce Massive Jitter - SOLVED!

[Mark Tinka]

I'm not generally into conspiracy, but as I keep trying to work out the 
issue I described in this thread, I came across this:


    https://sneak.berlin/20201112/your-computer-isnt-yours/

Might explain quite a lot, actually, (particularly at the FAQ section 
under "When did this start?") and why it feels like macOS has taken a 
huge dive (that I couldn't explain) since Mojave.


I still have High Sierra installed on my 2017 15-inch Mac, and will 
likely keep it that way forever... if for nothing else, but as a 
reference of what once was.


Staying away from Big Sur for as long as I can.

Mark.

Re: Apple Catalina Appears to Introduce Massive Jitter - SOLVED!

[Mark Tinka]

Thanks, J.

So I did test this a few times as well. The only thing I had enabled 
(during the first test) in "System Services" was "Find My Mac". 
Everything else was turned off, and the issue remained.


Just to confirm that this was a clean install of Catalina, so the only 
wi-fi AP in my history is my home one, as I haven't worked in a cafe, 
airport, or hotel since then :-).


Since disabling Bluetooth and the system stabilizing, I've re-enabled 
"Location-Based Suggestions" and "Significant Locations" only. 
Everything else under "System Services" is still turned off, including 
"Wi-Fi Networking".


That said, you did mention that you've fixed this on Big Sur. I am still 
running Catalina. Considering Apple's history of dodgy initial releases 
of a new OS, I'll give Big Sur a few months (or a year) before I feel 
it's safe to upgrade. I'm already having to deal with Catalina as it is, 
which is why I have High Sierra installed on my old laptop for my 
weekend DJ streaming habit :-). OBS Studio seems to like High Sierra 
better than Catalina, hehe.


Many thanks for working on this - it's most appreciated!

Mark.

On 11/10/20 16:30, J. Hellenthal wrote:

Hey Mark,

Went through a bunch of tests here. Seems I’ve cleared up the matter on this macOS[1] Big 
Sur at least by disabling Wi-Fi Networking under “Location Services -> System Services 
-> Wi-Fi Networking [2]”. It seems at least from perspective that something changed 
there and causes the Mac to scan more aggressively when more than one access point 
(generally speaking your SSID & SSID + 5G) has been logged at a location as 
accessible. This same thing could be observed at least on my system while having those 
settings turned off, bluetooth on and location services enabled and opening (Wi-Fi 
Explorer[3]) which puts the interface into “monitor mode” which seems to be causing the 
contention somewhere. After those changes keep in mind I had to restart from a full 
shutdown to get to some real clean ms traffic to the router and I prefer to be connected 
to 5Ghz before 2.4Ghz.


1. Darwin Kernel Version 20.1.0: Thu Oct 29 05:35:40 PDT 2020; 
root:xnu-7195.50.5~4/RELEASE_X86_64
2. 
https://www.dropbox.com/s/m3xm3fpoziwe01d/Screen%20Shot%202020-11-10%20at%2008.20.08.png?dl=0
3. https://apps.apple.com/us/app/wifi-explorer/id494803304?mt=12



On Nov 5, 2020, at 00:43, Mark Tinka  wrote:

Just an update on this re: the Bluetooth.

I had my AirPods paired previously for single use. I don't use them on the 
laptop (there is some latency), so I prefer the wired earphones. But it seems 
like Bluetooth was aggressively scanning for them. After removing them from the 
system, the scanning remained, but reduced significantly.

So looking at Console again, every so often, Bluetooth is scanning the network on behalf 
of the "sharingd" process.

sharingd is a sharing daemon that supports features such as AirDrop, Handoff, 
Instant Hotspot, Shared Computers and Remote Disc in Finder.

Still keeping Bluetooth off, however.

Mark.

Re: Apple Catalina Appears to Introduce Massive Jitter - SOLVED!

[J. Hellenthal via NANOG]

Hey Mark,

Went through a bunch of tests here. Seems I’ve cleared up the matter on this 
macOS[1] Big Sur at least by disabling Wi-Fi Networking under “Location 
Services -> System Services -> Wi-Fi Networking [2]”. It seems at least from 
perspective that something changed there and causes the Mac to scan more 
aggressively when more than one access point (generally speaking your SSID & 
SSID + 5G) has been logged at a location as accessible. This same thing could 
be observed at least on my system while having those settings turned off, 
bluetooth on and location services enabled and opening (Wi-Fi Explorer[3]) 
which puts the interface into “monitor mode” which seems to be causing the 
contention somewhere. After those changes keep in mind I had to restart from a 
full shutdown to get to some real clean ms traffic to the router and I prefer 
to be connected to 5Ghz before 2.4Ghz.


1. Darwin Kernel Version 20.1.0: Thu Oct 29 05:35:40 PDT 2020; 
root:xnu-7195.50.5~4/RELEASE_X86_64
2. 
https://www.dropbox.com/s/m3xm3fpoziwe01d/Screen%20Shot%202020-11-10%20at%2008.20.08.png?dl=0
3. https://apps.apple.com/us/app/wifi-explorer/id494803304?mt=12


> On Nov 5, 2020, at 00:43, Mark Tinka  wrote:
> 
> Just an update on this re: the Bluetooth.
> 
> I had my AirPods paired previously for single use. I don't use them on the 
> laptop (there is some latency), so I prefer the wired earphones. But it seems 
> like Bluetooth was aggressively scanning for them. After removing them from 
> the system, the scanning remained, but reduced significantly.
> 
> So looking at Console again, every so often, Bluetooth is scanning the 
> network on behalf of the "sharingd" process.
> 
> sharingd is a sharing daemon that supports features such as AirDrop, Handoff, 
> Instant Hotspot, Shared Computers and Remote Disc in Finder.
> 
> Still keeping Bluetooth off, however.
> 
> Mark.


-- 

J. Hellenthal

The fact that there's a highway to Hell but only a stairway to Heaven says a 
lot about anticipated traffic volume.

Re: Apple Catalina Appears to Introduce Massive Jitter - SOLVED!

[Mark Tinka]

Just an update on this re: the Bluetooth.

I had my AirPods paired previously for single use. I don't use them on 
the laptop (there is some latency), so I prefer the wired earphones. But 
it seems like Bluetooth was aggressively scanning for them. After 
removing them from the system, the scanning remained, but reduced 
significantly.


So looking at Console again, every so often, Bluetooth is scanning the 
network on behalf of the "sharingd" process.


sharingd is a sharing daemon that supports features such as AirDrop, 
Handoff, Instant Hotspot, Shared Computers and Remote Disc in Finder.


Still keeping Bluetooth off, however.

Mark.

Re: Apple Catalina Appears to Introduce Massive Jitter - SOLVED!

[Karl Auerbach]

Let me jump in and add a bit more information.

I am not an RF guy - I stopped playing with radios [and TV] in the days 
when they used vacuum tubes (yes, really.)


Many laptops share radio and antenna resources between WiFi and bluetooth.

Bluetooth lives on the 2.4ghz band.  Wifi presently uses both that band 
and also a 5ghz band. Different antennas might be used for each.


I encountered Wi-Fi/Bluetooth contention issues a couple of years back

My home wifi has (or rather had) distinct SSIDs for Wifi on the 2.4 and 
5ghz bands.  It was a rough attempt at manual load and distance balancing.


(Our house is in a relatively quiet area, RF wise, so there's not really 
any seriously competing wi-fi - or for that matter cell signal, 
broadcast TV, or FM radio.)


I began to notice that when I had one of my laptops on the 5ghz WiFi and 
was listening to music via some bluetooth speakers that my remote 
terminal keystrokes sometimes had that sluggish feel that is familiar 
when doing remote terminal command-line stuff over long paths with a lot 
of latency/jitter.  And at the same time the music via Bluetooth often 
broke up or stuttered.  There was a clear correlation between the two 
problems.


I had heard from some Linux kernel developers that deep down in the 
Linux kernel the simultaneous use of Wifi on a 5ghz channel and 
bluetooth on 2.4 causes a lot of thrashing and flogging of the the radio 
system.  I don't know, but I suspect that as a result there are queues 
of outbound traffic waiting for the radio or antennas to become 
operational on the channel they need.  I have no idea what happens to 
inbound frames when the radio system is tuned elsewhere - I never 
measured whether the frames are lost or delayed.


I suspect similar issues are present in *BSD, MacOS, and Windows kernels.

So I did some simple empirical testing to compare life with the laptop 
coerced to use an SSID present only on the 2.4ghz band. The problems 
went away.


I went back to the laptop, but coerced onto the 5ghz band for WiFi and, 
voila, there was trouble.


I've done this with a MacBook Pro (circa 2015 model) using various 
versions of MacOS and with my rather newer Linux laptops (mostly Dell 
XPS units with Fedora.)  Same sorts of behavior.


These were all i5 based units with 2 or 4 cores - plenty of CPU power to 
simultaneously handle an SSH remote console client and a music player.


I did not test with mobile phone or tablet platforms.

I do not know if the single radio issue is the result of cost savings or 
some radio-engineering or antenna issue.  I do suspect that these things 
could become more troublesome as WiFi 6 and/or 5G start to use some of 
the higher frequency allocations around 5.9 and 6ghz.)


(A few weeks ago we switched our home WiFi to a WiFi 6 [Netgear Orbi-6] 
mesh system that does not appear to allow separate SSIDs for the 2.4 and 
5ghz bands, so I can not repeat these tests without constructing a test 
network with the now unused access points.  BTW, I did encounter the 
hell that is known as "reconfiguring dozens upon dozens of different 
kinds of IoT devices to use a different SSID".)


Looking somewhat off topic - it is my sense that we will be seeing a lot 
more latency/jitter (and packet resequencing) issues in the future as 
radio systems become more agile and as we begin to use shorter 
(millimeter) wavelength frequencies with reduced ability to penetrate 
walls that, in turn, cause more frequent access-point transitions (with 
possibly distinctly different backhaul characteristics).  I've observed 
that these things can cause trouble for some TCP stacks and some non-TCP 
based VoIP and streaming applications.


        --karl--

On 10/30/20 12:08 PM, Mark Tinka wrote:

Hi all.

So I may have fixed this for my end, and hopefully others may be able 
to use the same fix.


After a tip from Karl Auerbach and this link:

https://developer.apple.com/forums/thread/97805

... I was able to fix the problem by disabling Bluetooth.

Re: Apple Catalina Appears to Introduce Massive Jitter - SOLVED!

[Mark Tinka]

Thanks for the input, Karl.

Hopefully someone from Apple is around here and can get some ideas on 
how to fix this particular problem set.


Mark.

On 10/31/20 11:37, Karl Auerbach wrote:


Let me jump in and add a bit more information.

I am not an RF guy - I stopped playing with radios [and TV] in the 
days when they used vacuum tubes (yes, really.)


Many laptops share radio and antenna resources between WiFi and bluetooth.

Bluetooth lives on the 2.4ghz band.  Wifi presently uses both that 
band and also a 5ghz band. Different antennas might be used for each.


I encountered Wi-Fi/Bluetooth contention issues a couple of years back

My home wifi has (or rather had) distinct SSIDs for Wifi on the 2.4 
and 5ghz bands.  It was a rough attempt at manual load and distance 
balancing.


(Our house is in a relatively quiet area, RF wise, so there's not 
really any seriously competing wi-fi - or for that matter cell signal, 
broadcast TV, or FM radio.)


I began to notice that when I had one of my laptops on the 5ghz WiFi 
and was listening to music via some bluetooth speakers that my remote 
terminal keystrokes sometimes had that sluggish feel that is familiar 
when doing remote terminal command-line stuff over long paths with a 
lot of latency/jitter.  And at the same time the music via Bluetooth 
often broke up or stuttered.  There was a clear correlation between 
the two problems.


I had heard from some Linux kernel developers that deep down in the 
Linux kernel the simultaneous use of Wifi on a 5ghz channel and 
bluetooth on 2.4 causes a lot of thrashing and flogging of the the 
radio system.  I don't know, but I suspect that as a result there are 
queues of outbound traffic waiting for the radio or antennas to become 
operational on the channel they need.  I have no idea what happens to 
inbound frames when the radio system is tuned elsewhere - I never 
measured whether the frames are lost or delayed.


I suspect similar issues are present in *BSD, MacOS, and Windows kernels.

So I did some simple empirical testing to compare life with the laptop 
coerced to use an SSID present only on the 2.4ghz band. The problems 
went away.


I went back to the laptop, but coerced onto the 5ghz band for WiFi 
and, voila, there was trouble.


I've done this with a MacBook Pro (circa 2015 model) using various 
versions of MacOS and with my rather newer Linux laptops (mostly Dell 
XPS units with Fedora.)  Same sorts of behavior.


These were all i5 based units with 2 or 4 cores - plenty of CPU power 
to simultaneously handle an SSH remote console client and a music player.


I did not test with mobile phone or tablet platforms.

I do not know if the single radio issue is the result of cost savings 
or some radio-engineering or antenna issue.  I do suspect that these 
things could become more troublesome as WiFi 6 and/or 5G start to use 
some of the higher frequency allocations around 5.9 and 6ghz.)


(A few weeks ago we switched our home WiFi to a WiFi 6 [Netgear 
Orbi-6] mesh system that does not appear to allow separate SSIDs for 
the 2.4 and 5ghz bands, so I can not repeat these tests without 
constructing a test network with the now unused access points.  BTW, I 
did encounter the hell that is known as "reconfiguring dozens upon 
dozens of different kinds of IoT devices to use a different SSID".)


Looking somewhat off topic - it is my sense that we will be seeing a 
lot more latency/jitter (and packet resequencing) issues in the future 
as radio systems become more agile and as we begin to use shorter 
(millimeter) wavelength frequencies with reduced ability to penetrate 
walls that, in turn, cause more frequent access-point transitions 
(with possibly distinctly different backhaul characteristics).  I've 
observed that these things can cause trouble for some TCP stacks and 
some non-TCP based VoIP and streaming applications.


        --karl--

On 10/30/20 12:08 PM, Mark Tinka wrote:

Hi all.

So I may have fixed this for my end, and hopefully others may be able 
to use the same fix.


After a tip from Karl Auerbach and this link:

https://developer.apple.com/forums/thread/97805

... I was able to fix the problem by disabling Bluetooth.

Re: Apple Catalina Appears to Introduce Massive Jitter - SOLVED!

[Mark Tinka]

On 10/30/20 23:57, Doug Barton wrote:

I would hesitate to blame BT. I have a macbook pro from ~1 year ago, 
on Catalina, and I use BT extensively ... mouse, keyboard, and 
headset. I do have location services trimmed down to just find my mac.


I ran: ping -c 1000 -i 0.1 

1000 packets transmitted, 998 packets received, 0.2% packet loss
round-trip min/avg/max/stddev = 1.255/2.378/9.095/0.634 ms

One thing that may contribute to blaming BT however is if you are 
using wifi on 2.4G only, and/or preferring it, as BT operates in the 
same frequency range neighborhood. My macbook is connected using 5G.


Happy to compare other settings if there is interest.


What I do know is that there has been more than one fix for most people 
that have had this issue. For some, it has been a dodgy app, and for 
others, it has just been disabling some Location Services.


In my case, what worked was disabling Bluetooth, in combination with as 
many Location Services as I could afford to not use.


I am able to reliably reproduce this issue, so in my case, I can 
certainly blame Bluetooth. Pity, since I do use some Handoff services 
with my other Apple devices, but I won't give up wi-fi stability for that.


And yes, this happens both on 2.4GHz and 5GHz. As you may know, macOS 
will default to 5GHz provided RSSI is at -68dB or better. My laptop only 
ever connects on 2.4GHz if I am outside my house (which is hardly). 
Inside the house, there is always a 5GHz transmitter within -38dB of me.


Mark.

Re: Apple Catalina Appears to Introduce Massive Jitter - SOLVED!

[Aaron Atac via NANOG]

Hi Mark,

I'm running a MacBook Pro (Retina, 15-inch, Mid 2015) with Mojave 10.14.6 
(latest).
I've always had location services off (including all system services within).

I haven't seen any jitter issues on my end.

Along with, no matter if I have bluetooth turned on with my wireless mouse and 
keyboard connected or not, I see no consistent change in jitter.

Even further, enabling location services (including all system services within) 
and bluetooth enabled, I'm not seeing any change in jitter.

I know you said earlier this issue seems to have started somewhere _around_ 
Mojave so just my two cents.

-Aaron

Oct 30, 2020, 12:08 by mark.ti...@seacom.com:

> Hi all.
>  
>  So I may have fixed this for my end, and hopefully others may be  able 
> to use the same fix.
>  
>  After a tip from Karl Auerbach and this link:
>  
>      > https://developer.apple.com/forums/thread/97805
>  
>  ... I was able to fix the problem by disabling Bluetooth. 
>  
>  However, disabling Bluetooth was not enough. I also had to disable  all 
> Location Services.
>  
>  After that, I re-enabled Location Services and only allowed for  two 
> features:
>  
>      - NetSpot
>      - Find My Mac
>  
>  With just those two location services, as well as Bluetooth  disabled, I 
> have no more high jitter.
>  
>  App performance like Zoom and Youtube uploads are now crisp, with  0.0% 
> packet loss.
>  
>  So looks like that Bluetooth is a huge problem. Confirmed by  opening 
> the "Console" app, and adding "scan" in the filter bar,  top right.
>  
>  A peak latency of 13.5ms after 300 packets:
>  
>  Marks-MacBook-Pro.local(172.16.0.239)
>   
>   
> 2020-10-30T21:06:05+0200
>  Keys:  Help   Display mode   Restart statistics   Order of  fields   quit
>   
>   
>    Packets   
> Pings
>   Host
>   
>      Loss%   Snt   Last   Avg 
>  Best  Wrst StDev
>   1.172.16.0.254  
>   
>      0.0%   300    3.1   4.8  
>  2.2  13.5   1.9
>  
>  Mark.
>  
>  

Re: Apple Catalina Appears to Introduce Massive Jitter - SOLVED!

[J. Hellenthal via NANOG]

Heya ! Doug!

Yeah I wouldn’t put this on BT either. On the other hand it seems that whether 
the scheduler is newreno or cubic that this situation persists pasts my 
previous suggestions. Seems tho that when you put strain on an upload that the 
jitter gets considerably worse... 90m out of a 100m link it starts to get to 
1000+ms

-- 
 J. Hellenthal

The fact that there's a highway to Hell but only a stairway to Heaven says a 
lot about anticipated traffic volume.

> On Oct 30, 2020, at 16:59, Doug Barton  wrote:
> I would hesitate to blame BT. I have a macbook pro from ~1 year ago, on 
> Catalina, and I use BT extensively ... mouse, keyboard, and headset. I do 
> have location services trimmed down to just find my mac.
> 
> I ran: ping -c 1000 -i 0.1 
> 
> 1000 packets transmitted, 998 packets received, 0.2% packet loss
> round-trip min/avg/max/stddev = 1.255/2.378/9.095/0.634 ms
> 
> One thing that may contribute to blaming BT however is if you are using wifi 
> on 2.4G only, and/or preferring it, as BT operates in the same frequency 
> range neighborhood. My macbook is connected using 5G.
> 
> Happy to compare other settings if there is interest.
> 
> Doug
> 
> 
>> On 10/30/20 12:08 PM, Mark Tinka wrote:
>> Hi all.
>> So I may have fixed this for my end, and hopefully others may be able to use 
>> the same fix.
>> After a tip from Karl Auerbach and this link:
>> https://developer.apple.com/forums/thread/97805
>> ... I was able to fix the problem by disabling Bluetooth.
>> However, disabling Bluetooth was not enough. I also had to disable all 
>> Location Services.
>> After that, I re-enabled Location Services and only allowed for two features:
>> - NetSpot
>> - Find My Mac
>> With just those two location services, as well as Bluetooth disabled, I have 
>> no more high jitter.
>> App performance like Zoom and Youtube uploads are now crisp, with 0.0% 
>> packet loss.
>> So looks like that Bluetooth is a huge problem. Confirmed by opening the 
>> "Console" app, and adding "scan" in the filter bar, top right.
>> A peak latency of 13.5ms after 300 packets:
>> Marks-MacBook-Pro.local (172.16.0.239) 2020-10-30T21:06:05+0200
>> Keys:  Help   Display mode   Restart statistics   Order of fields   quit
>> Packets   Pings
>>  Host Loss%   Snt   Last   Avg  Best  Wrst StDev
>>  1. 172.16.0.254 0.0%   3003.1   4.8   2.2  13.5   1.9
>> Mark.


smime.p7s
Description: S/MIME cryptographic signature

Re: Apple Catalina Appears to Introduce Massive Jitter - SOLVED!

[Doug Barton]

I would hesitate to blame BT. I have a macbook pro from ~1 year ago, on 
Catalina, and I use BT extensively ... mouse, keyboard, and headset. I 
do have location services trimmed down to just find my mac.


I ran: ping -c 1000 -i 0.1 

1000 packets transmitted, 998 packets received, 0.2% packet loss
round-trip min/avg/max/stddev = 1.255/2.378/9.095/0.634 ms

One thing that may contribute to blaming BT however is if you are using 
wifi on 2.4G only, and/or preferring it, as BT operates in the same 
frequency range neighborhood. My macbook is connected using 5G.


Happy to compare other settings if there is interest.

Doug


On 10/30/20 12:08 PM, Mark Tinka wrote:

Hi all.

So I may have fixed this for my end, and hopefully others may be able to 
use the same fix.


After a tip from Karl Auerbach and this link:

https://developer.apple.com/forums/thread/97805

... I was able to fix the problem by disabling Bluetooth.

However, disabling Bluetooth was not enough. I also had to disable all 
Location Services.


After that, I re-enabled Location Services and only allowed for two 
features:


     - NetSpot
     - Find My Mac

With just those two location services, as well as Bluetooth disabled, I 
have no more high jitter.


App performance like Zoom and Youtube uploads are now crisp, with 0.0% 
packet loss.


So looks like that Bluetooth is a huge problem. Confirmed by opening the 
"Console" app, and adding "scan" in the filter bar, top right.


A peak latency of 13.5ms after 300 packets:

Marks-MacBook-Pro.local (172.16.0.239) 2020-10-30T21:06:05+0200
Keys:  Help   Display mode   Restart statistics   Order of fields   quit
Packets   Pings
  Host Loss%   Snt   Last   Avg  Best  Wrst StDev
  1. 172.16.0.254 0.0%   300    3.1   4.8   2.2  13.5   1.9

Mark.

Re: Apple Catalina Appears to Introduce Massive Jitter - SOLVED!

[Mark Tinka]

Hi all.

So I may have fixed this for my end, and hopefully others may be able to 
use the same fix.


After a tip from Karl Auerbach and this link:

    https://developer.apple.com/forums/thread/97805

... I was able to fix the problem by disabling Bluetooth.

However, disabling Bluetooth was not enough. I also had to disable all 
Location Services.


After that, I re-enabled Location Services and only allowed for two 
features:


    - NetSpot
    - Find My Mac

With just those two location services, as well as Bluetooth disabled, I 
have no more high jitter.


App performance like Zoom and Youtube uploads are now crisp, with 0.0% 
packet loss.


So looks like that Bluetooth is a huge problem. Confirmed by opening the 
"Console" app, and adding "scan" in the filter bar, top right.


A peak latency of 13.5ms after 300 packets:

Marks-MacBook-Pro.local (172.16.0.239) 2020-10-30T21:06:05+0200
Keys:  Help   Display mode   Restart statistics   Order of fields   quit
Packets   Pings
 Host Loss%   Snt   Last   Avg  Best  Wrst StDev
 1. 172.16.0.254 0.0%   300    3.1   4.8   2.2  13.5   1.9

Mark.