Re: Performance issue on macOS 10.15 obtaining display name for ~/Desktop, ~/Documents, and ~/Downloads

2020-05-22 Thread Allan Odgaard via Cocoa-dev 

On 22 May 2020, at 21:59, Aandi Inston wrote:


1. I wonder if your tests are in a non-English language system.


No, running on a non-localized system, and the evidence is overwhelming 
that this is about SIP / AMFI (based on inspecting the stack trace, 
which clearly show communication between the process and Apple’s 
various security policy daemons).


3. I wonder if the other testers who could not reproduce this were 
using an

English language system.


There is plenty of people who have confirmed the issue. There was one on 
this list who said he couldn’t reproduce, but he also said he changed 
the code to NSLog, so I wonder if he ran from Xcode, in which case the 
new process probably inherited Xcode’s privileges, and I have just 
learned that some users have a Developer Tools under System Preferences 
→ Security & Privacy. Adding something to this group seems to disable 
a lot of security checks (and all child processes).


I currently do not see this category on my system though, and based on 
comments, I am not alone.

___

Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)

Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com

Help/Unsubscribe/Update your Subscription:
https://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com

This email sent to arch...@mail-archive.com

Re: Performance issue on macOS 10.15 obtaining display name for ~/Desktop, ~/Documents, and ~/Downloads

2020-05-22 Thread Rob Petrovec via Cocoa-dev 


> On May 22, 2020, at 8:42 AM, Allan Odgaard via Cocoa-dev 
>  wrote:
> 
> On 23 Apr 2020, at 21:15, Rob Petrovec wrote:
> 
>> If what you say is correct then everyone would be seeing a delay since most 
>> people don’t have blazing fast internet connections.  I do not think this is 
>> the normal behavior.  I think it is specific to your system, otherwise there 
>> would be TONS of people complaining about slowness.
> 
> Episode 379 of ATP has just been released and both Marco Arment and John 
> Siracusa are complaining about significant delays after upgrading to macOS 
> 10.15, though for John Siracusa I think only one of his machines has the 
> problem.
> 
> Also, I think I already mentioned it, but I had a friend of mine run tests on 
> his machine in another geographical zone, and he saw delays as well.
> 
>> Either way, did you file a bug with a sysdiagnose taken during the delay?  
>> If so, do you have the bug number?
> 
> Several people in this thread were asking for bug numbers: I do not 
> understand this, no-one outside of Apple can read these bugs, right?
Correct, however, you are presuming that no apple engineers reads this 
mailing list. That would be an incorrect assumption.

—Rob




___

Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)

Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com

Help/Unsubscribe/Update your Subscription:
https://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com

This email sent to arch...@mail-archive.com

Re: Performance issue on macOS 10.15 obtaining display name for ~/Desktop, ~/Documents, and ~/Downloads

2020-05-22 Thread Aandi Inston via Cocoa-dev 
Your report is interesting. I have been following it but may have missed a
part of the discussion, so here is one
thought I had. You say that getting the display name of ~/Documents may
result in a delay. This is localizable, so
that in Portugal (for example) the display name is Documentos. So here are
thoughts
1. I wonder if your tests are in a non-English language system.
2. I wonder if in an English language system there is a fast path that
tells the system to bypass localization tests.
3. I wonder if the other testers who could not reproduce this were using an
English language system.
I observe that the internet says (so it must be true) that the test for
localized folders is a hidden file in the folder
called ".localized". To check for this in any obvious way is going to need
read access to the folder, and hence may
trigger security checking.

On Fri, 22 May 2020 at 15:43, Allan Odgaard via Cocoa-dev <
cocoa-dev@lists.apple.com> wrote:

> On 23 Apr 2020, at 21:15, Rob Petrovec wrote:
>
> > If what you say is correct then everyone would be seeing a delay since
> > most people don’t have blazing fast internet connections.  I do not
> > think this is the normal behavior.  I think it is specific to your
> > system, otherwise there would be TONS of people complaining about
> > slowness.
>
> Episode 379 of ATP has just been released and both Marco Arment and John
> Siracusa are complaining about significant delays after upgrading to
> macOS 10.15, though for John Siracusa I think only one of his machines
> has the problem.
>
> Also, I think I already mentioned it, but I had a friend of mine run
> tests on his machine in another geographical zone, and he saw delays as
> well.
>
> > Either way, did you file a bug with a sysdiagnose taken during the
> > delay?  If so, do you have the bug number?
>
> Several people in this thread were asking for bug numbers: I do not
> understand this, no-one outside of Apple can read these bugs, right?
>
> But now I wrote up a summary of what I have found so far with bug
> numbers included:
> https://sigpipe.macromates.com/2020/macos-catalina-slow-by-design/
> ___
>
> Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)
>
> Please do not post admin requests or moderator comments to the list.
> Contact the moderators at cocoa-dev-admins(at)lists.apple.com
>
> Help/Unsubscribe/Update your Subscription:
> https://lists.apple.com/mailman/options/cocoa-dev/aandi%40quite.com
>
> This email sent to aa...@quite.com
>
___

Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)

Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com

Help/Unsubscribe/Update your Subscription:
https://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com

This email sent to arch...@mail-archive.com

Re: Performance issue on macOS 10.15 obtaining display name for ~/Desktop, ~/Documents, and ~/Downloads

2020-05-22 Thread Allan Odgaard via Cocoa-dev 

On 23 Apr 2020, at 21:15, Rob Petrovec wrote:

If what you say is correct then everyone would be seeing a delay since 
most people don’t have blazing fast internet connections.  I do not 
think this is the normal behavior.  I think it is specific to your 
system, otherwise there would be TONS of people complaining about 
slowness.


Episode 379 of ATP has just been released and both Marco Arment and John 
Siracusa are complaining about significant delays after upgrading to 
macOS 10.15, though for John Siracusa I think only one of his machines 
has the problem.


Also, I think I already mentioned it, but I had a friend of mine run 
tests on his machine in another geographical zone, and he saw delays as 
well.


Either way, did you file a bug with a sysdiagnose taken during the 
delay?  If so, do you have the bug number?


Several people in this thread were asking for bug numbers: I do not 
understand this, no-one outside of Apple can read these bugs, right?


But now I wrote up a summary of what I have found so far with bug 
numbers included: 
https://sigpipe.macromates.com/2020/macos-catalina-slow-by-design/

___

Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)

Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com

Help/Unsubscribe/Update your Subscription:
https://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com

This email sent to arch...@mail-archive.com

Re: Performance issue on macOS 10.15 obtaining display name for ~/Desktop, ~/Documents, and ~/Downloads

2020-04-24 Thread Gary L. Wade via Cocoa-dev 
Also, did you take advantage of one of your free tech support incidents?
--
Gary L. Wade
http://www.garywade.com/

> On Apr 24, 2020, at 8:26 AM, Gary L. Wade via Cocoa-dev 
>  wrote:
> 
> That’s a very narrow view of reality, which I know to be far broader.
> 
> What’s the feedback number?
> --
> Gary
> 
>>> On Apr 24, 2020, at 8:01 AM, Allan Odgaard via Cocoa-dev 
>>>  wrote:
>> That said, I *have* filed a report about this, but I still seek more 
>> information about the issue, which I had hoped to get from this list, 
>> because I sure as hell am not going to get it via Apple’s bug reporting 
>> system.

___

Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)

Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com

Help/Unsubscribe/Update your Subscription:
https://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com

This email sent to arch...@mail-archive.com

Re: Performance issue on macOS 10.15 obtaining display name for ~/Desktop, ~/Documents, and ~/Downloads

2020-04-24 Thread Gary L. Wade via Cocoa-dev 
That’s a very narrow view of reality, which I know to be far broader.

What’s the feedback number?
--
Gary

> On Apr 24, 2020, at 8:01 AM, Allan Odgaard via Cocoa-dev 
>  wrote:
> 
> That said, I *have* filed a report about this, but I still seek more 
> information about the issue, which I had hoped to get from this list, because 
> I sure as hell am not going to get it via Apple’s bug reporting system.

___

Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)

Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com

Help/Unsubscribe/Update your Subscription:
https://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com

This email sent to arch...@mail-archive.com

Re: Performance issue on macOS 10.15 obtaining display name for ~/Desktop, ~/Documents, and ~/Downloads

2020-04-24 Thread Allan Odgaard via Cocoa-dev 

On 24 Apr 2020, at 21:33, Gary L. Wade wrote:

Here’s two web sites that should help you get the answer you want. 
Try one or both:


https://feedbackassistant.apple.com/welcome
https://www.apple.com/jobs/us/


You don’t get answers from filing bug reports with Apple, at best, the 
issue gets closed as a duplicate, which seems to be acknowledgement that 
others have run into the same problem, but then you never hear more 
about it. Alternatively the bug just stays open for years, maybe at some 
future OS release, you get an email saying to please test if the issue 
is still present.


That said, I *have* filed a report about this, but I still seek more 
information about the issue, which I had hoped to get from this list, 
because I sure as hell am not going to get it via Apple’s bug 
reporting system.


For the records, I asked a friend to run the test on his system, he got 
a delay of 160-170 ms, he is in Europe where Apple have local data 
centers, so it’s not unreasonable to think his ping time to 
gs.apple.com is 30-50 ms, hence it still seems plausible that it does a 
connection to Apple’s server for a call to getxattr().


Though I’m waiting for more data from him.
___

Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)

Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com

Help/Unsubscribe/Update your Subscription:
https://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com

This email sent to arch...@mail-archive.com

Re: Performance issue on macOS 10.15 obtaining display name for ~/Desktop, ~/Documents, and ~/Downloads

2020-04-24 Thread Gary L. Wade via Cocoa-dev 
Here’s two web sites that should help you get the answer you want. Try one or 
both:

https://feedbackassistant.apple.com/welcome
https://www.apple.com/jobs/us/
--
Gary
___

Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)

Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com

Help/Unsubscribe/Update your Subscription:
https://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com

This email sent to arch...@mail-archive.com

Re: Performance issue on macOS 10.15 obtaining display name for ~/Desktop, ~/Documents, and ~/Downloads

2020-04-24 Thread Sandor Szatmari via Cocoa-dev 
Allan,

> On Apr 24, 2020, at 00:14, Allan Odgaard via Cocoa-dev 
>  wrote:
> 
> On 24 Apr 2020, at 9:51, Gary L. Wade wrote:
> 
>> Have you tried a speed check with just iCloud turned off but internet on?
> 
> I have tried with iCloud disabled, internet disabled, and SIP disabled.

Have you tried something like the Little Snitch app to Confirm there is network 
access to Apple and determine what network activity is going on?

Sandor
> 
> Only the latter two removes the delay. Also, the issue happens for 
> ~/Downloads which is not an iCloud folder.
> 
> Furthermore, the stack dump seems fairly clear about what goes on.
> 
> For the records, I’m reposting my findings here with stack dump.
> 
> This is actually from Transmission.app, in Preferences you can configure 
> download folders, so I selected ~/Documents, ~/Downloads, and ~/Desktop, 
> which are the 3 “protected” folders. I then ran a spindump when opening 
> Preferences, which has to obtain display name and icon for these folders (to 
> show in the UI), which results in 620 ms spent in NSWorkspace’s iconForFile: 
> (getattrlist).
> 
>62   -[NSWorkspace iconForFile:] + 80 (AppKit + 3229058) [0x7fff32ee1582]
>62   -[NSWorkspace _iconForURL:] + 100 (AppKit + 3229205) [0x7fff32ee1615]
>62   _GetIconRefFromURL + 26 (LaunchServices + 233122) [0x7fff370edea2]
>62   BindingManager::CreateWithURL(__CFURL const*, bool) + 28 
> (LaunchServices + 233204) [0x7fff370edef4]
>62   BindingBlueprint::BindingBlueprint(__CFURL const*) + 78 
> (LaunchServices + 233350) [0x7fff370edf86]
>62   BindingBlueprint::copyURLProperties(__CFURL const*) + 50 
> (LaunchServices + 233496) [0x7fff370ee018]
>62   CFURLCopyResourcePropertiesForKeys + 111 (CoreFoundation + 555324) 
> [0x7fff3599493c]
>62   _FSURLCopyResourcePropertiesForKeysInternal(__CFURL const*, __CFArray 
> const*, void*, __CFError**, unsigned char) + 1110 (CoreServicesInternal + 
> 60765) [0x7fff4ecb0d5d]
>62   prepareValuesForBitmap(__CFURL const*, __FileCache*, 
> _FilePropertyBitmap*, __CFError**) + 363 (CoreServicesInternal + 17604) 
> [0x7fff4eca64c4]
>62   __getattrlist + 10 (libsystem_kernel.dylib + 5746) [0x7fff6fa19672]
>*62  hndl_unix_scall64 + 22 (kernel + 819718) [0xff80002c8206]
>*62  unix_syscall64 + 647 (kernel + 7894519) [0xff80009875f7]
>*62  getattrlist + 136 (kernel + 3512472) [0xff8000559898]
>*62  ??? (kernel + 3512820) [0xff80005599f4]
>*62  ??? (kernel + 3501529) [0xff8000556dd9]
>*62  ??? (kernel + 3490836) [0xff8000554414]
>*62  mac_vnode_check_access + 154 (kernel + 9093082) [0xff8000aabfda]
>*62  hook_vnode_check_access + 167 (Sandbox + 39552) [0xff7f823a7a80]
>*62  sb_evaluate + 5004 (Sandbox + 67658) [0xff7f823ae84a]
>*62  approval_response_wait + 142 (Sandbox + 154851) [0xff7f823c3ce3]
>*62  __WAITING_ON_APPROVAL_FROM_SANDBOXD__ + 23 (Sandbox + 155134) 
> [0xff7f823c3dfe]
>*62  ??? (kernel + 6855402) [0xff8000889aea]
>*62  thread_block_reason + 175 (kernel + 1317935) [0xff8000341c2f]
>*62  ??? (kernel + 1324017) [0xff80003433f1]
>*62  machine_switch_context + 200 (kernel + 2388456) [0xff80004471e8]
> 
> Looking at sandboxd, it has 3 queues spending respectively 150, 160, and 310 
> ms in TCCAccessPreflightWithAuditToken (so total time spent there is 620 ms), 
> which I would guess is processing Transmission’s request for reading the 
> extended attributes for the 3 paths.
> 
> But sandboxd itself does tccd_send_message, so it seems to call upon the tccd 
> daemon.
> 
> There are two tccd instances running on my system, each have two queues with 
> 150 ms spent in SecCodeCheckValidityWithErrors (so a total of 600 ms spent in 
> tccd).
> 
> This code though again seems rely on another process by calling: 
> securityd_send_sync_and_do (which does XPC).
> 
> But I can’t find out which process it communicates with. I don’t know if this 
> could be launched-on-demand and therefore not part of the spindump report?
> 
> But to summarize, first call of getattrlist/getxattr for a protected folder 
> seems to trigger application signature validation (makes sense), which, at 
> least on my system, appears to involve network access, reminiscent of what 
> goes on with execve().
> 
> Given the above, I don’t think anyone can dispute that this is what goes on, 
> and it would be extremely strange if this is not “by design”.
> 
> What might be a bug is that the check does network access, but you don’t just 
> put such code in by mistake, so obviously there must be certain times where 
> it needs network access, and as demonstrated with execve(), Apple is not shy 
> of making low-level system APIs contact Apple’s servers as part of their SIP.
> 
> The million dollar question is now: What are the conditions under which it 
> must have network access? and/or why does it seem to trigger all the time on 
> my system?
>

Re: Performance issue on macOS 10.15 obtaining display name for ~/Desktop, ~/Documents, and ~/Downloads

2020-04-23 Thread Allan Odgaard via Cocoa-dev 

On 24 Apr 2020, at 11:49, Saagar Jha wrote:


GateKeeper is basically Safari adding a quarantine flag […]
Nit: not just Safari; other applications do this to at their 
discretion when appropriate (for example, if they too download files 
from the internet). Quarantine is just one part of GateKeeper.


Right, I said “basically” to indicate I was simplifying the 
description.


But at least it was a user-space feature with co-operation from (amongst 
others) Finder and Safari, without the need for any network access.


This is different from the issue being discussed, which is kernel-space 
feature that does network access even for locally created files.


GateKeeper and XProtect though are probably more umbrella marketing 
terms than actual technologies, so they may cover more today, for 
example app translocation probably also falls under GateKeeper, and the 
recent uninstall of the Zoom web server I think was said to fall under 
XProtect.

___

Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)

Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com

Help/Unsubscribe/Update your Subscription:
https://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com

This email sent to arch...@mail-archive.com

Re: Performance issue on macOS 10.15 obtaining display name for ~/Desktop, ~/Documents, and ~/Downloads

2020-04-23 Thread Saagar Jha via Cocoa-dev 

Saagar Jha

> On Apr 23, 2020, at 21:26, Allan Odgaard via Cocoa-dev 
>  wrote:
> 
> On 24 Apr 2020, at 9:57, Rob Petrovec wrote:
> 
>>> Also weird, why would it phone home for a shell script which has neither 
>>> been stapled nor even code-signed?
>> I think you answered the question just then…  a "shell script which has 
>> neither been stapled nor even code-signed”.  Google XProtect & Gatekeeper.
> 
> GateKeeper is basically Safari adding a quarantine flag (via extended 
> attributes)

Nit: not just Safari; other applications do this to at their discretion when 
appropriate (for example, if they too download files from the internet). 
Quarantine is just one part of GateKeeper.

> to files downloaded form the internet, and then having Finder check this 
> flag, throwing up a dialog if the flag is set, and recently, also checking if 
> the code signature is from a verified developer (possibly refusing to launch 
> at all, if not).
> 
> XProtect is basically a blacklist that applications are checked against. If 
> an application matches, it’s considered malware. The blacklist is a local 
> file on your system but updated by Apple.
> 
> These things operate very differently than having low-level system calls 
> potentially contact Apple’s servers every time a process is launched on your 
> system (or, as it appears to be the case on my system, when processes are 
> accessing certain locations in the file system).
> ___
> 
> Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)
> 
> Please do not post admin requests or moderator comments to the list.
> Contact the moderators at cocoa-dev-admins(at)lists.apple.com
> 
> Help/Unsubscribe/Update your Subscription:
> https://lists.apple.com/mailman/options/cocoa-dev/saagar%40saagarjha.com
> 
> This email sent to saa...@saagarjha.com

___

Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)

Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com

Help/Unsubscribe/Update your Subscription:
https://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com

This email sent to arch...@mail-archive.com

Re: Performance issue on macOS 10.15 obtaining display name for ~/Desktop, ~/Documents, and ~/Downloads

2020-04-23 Thread Allan Odgaard via Cocoa-dev 

On 24 Apr 2020, at 9:57, Rob Petrovec wrote:

Also weird, why would it phone home for a shell script which has 
neither been stapled nor even code-signed?
I think you answered the question just then…  a "shell script which 
has neither been stapled nor even code-signed”.  Google XProtect & 
Gatekeeper.


GateKeeper is basically Safari adding a quarantine flag (via extended 
attributes) to files downloaded form the internet, and then having 
Finder check this flag, throwing up a dialog if the flag is set, and 
recently, also checking if the code signature is from a verified 
developer (possibly refusing to launch at all, if not).


XProtect is basically a blacklist that applications are checked against. 
If an application matches, it’s considered malware. The blacklist is a 
local file on your system but updated by Apple.


These things operate very differently than having low-level system calls 
potentially contact Apple’s servers every time a process is launched 
on your system (or, as it appears to be the case on my system, when 
processes are accessing certain locations in the file system).

___

Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)

Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com

Help/Unsubscribe/Update your Subscription:
https://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com

This email sent to arch...@mail-archive.com

Re: Performance issue on macOS 10.15 obtaining display name for ~/Desktop, ~/Documents, and ~/Downloads

2020-04-23 Thread Allan Odgaard via Cocoa-dev 

On 24 Apr 2020, at 9:51, Gary L. Wade wrote:

Have you tried a speed check with just iCloud turned off but internet 
on?


I have tried with iCloud disabled, internet disabled, and SIP disabled.

Only the latter two removes the delay. Also, the issue happens for 
~/Downloads which is not an iCloud folder.


Furthermore, the stack dump seems fairly clear about what goes on.

For the records, I’m reposting my findings here with stack dump.

This is actually from Transmission.app, in Preferences you can configure 
download folders, so I selected ~/Documents, ~/Downloads, and ~/Desktop, 
which are the 3 “protected” folders. I then ran a spindump when 
opening Preferences, which has to obtain display name and icon for these 
folders (to show in the UI), which results in 620 ms spent in 
NSWorkspace’s iconForFile: (getattrlist).


	62   -[NSWorkspace iconForFile:] + 80 (AppKit + 3229058) 
[0x7fff32ee1582]
	62   -[NSWorkspace _iconForURL:] + 100 (AppKit + 3229205) 
[0x7fff32ee1615]

62   _GetIconRefFromURL + 26 (LaunchServices + 233122) [0x7fff370edea2]
	62   BindingManager::CreateWithURL(__CFURL const*, bool) + 28 
(LaunchServices + 233204) [0x7fff370edef4]
	62   BindingBlueprint::BindingBlueprint(__CFURL const*) + 78 
(LaunchServices + 233350) [0x7fff370edf86]
	62   BindingBlueprint::copyURLProperties(__CFURL const*) + 50 
(LaunchServices + 233496) [0x7fff370ee018]
	62   CFURLCopyResourcePropertiesForKeys + 111 (CoreFoundation + 555324) 
[0x7fff3599493c]
	62   _FSURLCopyResourcePropertiesForKeysInternal(__CFURL const*, 
__CFArray const*, void*, __CFError**, unsigned char) + 1110 
(CoreServicesInternal + 60765) [0x7fff4ecb0d5d]
	62   prepareValuesForBitmap(__CFURL const*, __FileCache*, 
_FilePropertyBitmap*, __CFError**) + 363 (CoreServicesInternal + 17604) 
[0x7fff4eca64c4]
	62   __getattrlist + 10 (libsystem_kernel.dylib + 5746) 
[0x7fff6fa19672]

*62  hndl_unix_scall64 + 22 (kernel + 819718) [0xff80002c8206]
*62  unix_syscall64 + 647 (kernel + 7894519) [0xff80009875f7]
*62  getattrlist + 136 (kernel + 3512472) [0xff8000559898]
*62  ??? (kernel + 3512820) [0xff80005599f4]
*62  ??? (kernel + 3501529) [0xff8000556dd9]
*62  ??? (kernel + 3490836) [0xff8000554414]
	*62  mac_vnode_check_access + 154 (kernel + 9093082) 
[0xff8000aabfda]
	*62  hook_vnode_check_access + 167 (Sandbox + 39552) 
[0xff7f823a7a80]

*62  sb_evaluate + 5004 (Sandbox + 67658) [0xff7f823ae84a]
	*62  approval_response_wait + 142 (Sandbox + 154851) 
[0xff7f823c3ce3]
	*62  __WAITING_ON_APPROVAL_FROM_SANDBOXD__ + 23 (Sandbox + 155134) 
[0xff7f823c3dfe]

*62  ??? (kernel + 6855402) [0xff8000889aea]
*62  thread_block_reason + 175 (kernel + 1317935) [0xff8000341c2f]
*62  ??? (kernel + 1324017) [0xff80003433f1]
	*62  machine_switch_context + 200 (kernel + 2388456) 
[0xff80004471e8]


Looking at sandboxd, it has 3 queues spending respectively 150, 160, and 
310 ms in TCCAccessPreflightWithAuditToken (so total time spent there is 
620 ms), which I would guess is processing Transmission’s request for 
reading the extended attributes for the 3 paths.


But sandboxd itself does tccd_send_message, so it seems to call upon the 
tccd daemon.


There are two tccd instances running on my system, each have two queues 
with 150 ms spent in SecCodeCheckValidityWithErrors (so a total of 600 
ms spent in tccd).


This code though again seems rely on another process by calling: 
securityd_send_sync_and_do (which does XPC).


But I can’t find out which process it communicates with. I don’t 
know if this could be launched-on-demand and therefore not part of the 
spindump report?


But to summarize, first call of getattrlist/getxattr for a protected 
folder seems to trigger application signature validation (makes sense), 
which, at least on my system, appears to involve network access, 
reminiscent of what goes on with execve().


Given the above, I don’t think anyone can dispute that this is what 
goes on, and it would be extremely strange if this is not “by 
design”.


What might be a bug is that the check does network access, but you 
don’t just put such code in by mistake, so obviously there must be 
certain times where it needs network access, and as demonstrated with 
execve(), Apple is not shy of making low-level system APIs contact 
Apple’s servers as part of their SIP.


The million dollar question is now: What are the conditions under which 
it must have network access? and/or why does it seem to trigger all the 
time on my system?

___

Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)

Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com

Help/Unsubscribe/Update your Subscription:
https://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com

This email sent to arch...@mail-ar

Re: Performance issue on macOS 10.15 obtaining display name for ~/Desktop, ~/Documents, and ~/Downloads

2020-04-23 Thread Marco S Hyman via Cocoa-dev 
>> Also weird, why would it phone home for a shell script which has neither 
>> been stapled nor even code-signed?
>   I think you answered the question just then…  a "shell script which has 
> neither been stapled nor even code-signed”.  Google XProtect & Gatekeeper.

1) The executable part of a shell script is the shell.
2) The shell is signed by apple as can be seen from the output of, for example, 
   codesign --verify --display --verbose=4 /bin/sh
   or bash, or zsh, etc.
3) network monitoring shows no traffic due to running a shell script on my 
machine

We’re getting pretty far afield from the original subject, no?
___

Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)

Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com

Help/Unsubscribe/Update your Subscription:
https://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com

This email sent to arch...@mail-archive.com

Re: Performance issue on macOS 10.15 obtaining display name for ~/Desktop, ~/Documents, and ~/Downloads

2020-04-23 Thread Rob Petrovec via Cocoa-dev 


> On Apr 23, 2020, at 8:35 PM, Allan Odgaard via Cocoa-dev 
>  wrote:
> 
> On 24 Apr 2020, at 2:28, Gabriel Zachmann via Cocoa-dev wrote:
> 
>>> I believe that is why you are supposed to staple notarization tickets to 
>>> your apps.
>> Then, why would it "phone home" in case there is an internet connection?
> 
> Also weird, why would it phone home for a shell script which has neither been 
> stapled nor even code-signed?
I think you answered the question just then…  a "shell script which has 
neither been stapled nor even code-signed”.  Google XProtect & Gatekeeper.

—Rob




___

Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)

Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com

Help/Unsubscribe/Update your Subscription:
https://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com

This email sent to arch...@mail-archive.com

Re: Performance issue on macOS 10.15 obtaining display name for ~/Desktop, ~/Documents, and ~/Downloads

2020-04-23 Thread Gary L. Wade via Cocoa-dev 
Have you tried a speed check with just iCloud turned off but internet on?
--
Gary
___

Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)

Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com

Help/Unsubscribe/Update your Subscription:
https://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com

This email sent to arch...@mail-archive.com

Re: Performance issue on macOS 10.15 obtaining display name for ~/Desktop, ~/Documents, and ~/Downloads

2020-04-23 Thread Allan Odgaard via Cocoa-dev 

On 24 Apr 2020, at 2:28, Gabriel Zachmann via Cocoa-dev wrote:

I believe that is why you are supposed to staple notarization tickets 
to your apps.
Then, why would it "phone home" in case there is an internet 
connection?


Also weird, why would it phone home for a shell script which has neither 
been stapled nor even code-signed?


Actually, weird is probably the wrong word: This is concerning!
___

Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)

Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com

Help/Unsubscribe/Update your Subscription:
https://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com

This email sent to arch...@mail-archive.com

Re: Performance issue on macOS 10.15 obtaining display name for ~/Desktop, ~/Documents, and ~/Downloads

2020-04-23 Thread Rob Petrovec via Cocoa-dev 


> On Apr 23, 2020, at 7:30 PM, Allan Odgaard via Cocoa-dev 
>  wrote:
> 
> On 24 Apr 2020, at 2:18, Rob Petrovec wrote:
> 
>> I get a 1 second time for the first run and then a much quicker time for the 
>> second.  I did some sampling and the longer time due to is Apple’s check for 
>> malware on first run of a process.  This is a known, documented and 
>> advertised behavior.
> 
> I would be very interested in documentation about what low-level APIs (like 
> execve) do malware checks (network access), under which conditions they are 
> performed, what servers are contacted, and what sort of caching of good/bad 
> results are done.
> 
> Is any of that documented?
Here is some from a quick Google search.  I think the feature in 
question is XProtect.  With a little more time I could probably find more 
in-depth docs.

https://www.apple.com/macos/security/  See the 'Protection starts at 
the core’ section

https://support.apple.com/guide/mac-help/protect-your-mac-from-malware-mh40596/mac

https://www.howtogeek.com/217043/xprotect-explained-how-your-macs-built-in-anti-malware-works/


> There is also blacklisting going on: I can get an executable locally 
> blacklisted which will cause it to terminate instantly when executed. This 
> seems to be about some run-time code signature validation, and when it 
> happens, it appears to be the inode that gets blacklisted until next reboot, 
> but more info about this would be nice.
Depending on where the app is being terminated, I would suspect it is 
the same “Allow apps downloaded from” feature in the General section of the 
Security & Privacy Pref pane.


>> […] So I don’t think this test is analogous to your initial issue of a delay 
>> opening a file every time.
> 
> I said I get a similar delay the first time my app obtains URL properties¹ 
> for ~/Desktop, ~/Documents, and, ~/Downloads, and I included sample code for 
> this issue.
Sorry I forgot what your initial problem was.  However, my statement 
still applies.  Getting the localized string for a folder is completely 
different then the launching app.


> Perhaps you would be willing to add this sample code to a GUI application and 
> see if you can reproduce? I re-attached it below, and have the result written 
> to /tmp/duration.txt so you don’t have to fiddle with capturing log output.
I tried it (although I changed it from writing a file to disk to 
NSLog() and it spit out:

default 19:58:53.343324-0600Test FooDuration 0.003

—Rob


___

Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)

Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com

Help/Unsubscribe/Update your Subscription:
https://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com

This email sent to arch...@mail-archive.com

Re: Performance issue on macOS 10.15 obtaining display name for ~/Desktop, ~/Documents, and ~/Downloads

2020-04-23 Thread Allan Odgaard via Cocoa-dev 

On 24 Apr 2020, at 2:18, Rob Petrovec wrote:

I get a 1 second time for the first run and then a much quicker time 
for the second.  I did some sampling and the longer time due to is 
Apple’s check for malware on first run of a process.  This is a 
known, documented and advertised behavior.


I would be very interested in documentation about what low-level APIs 
(like execve) do malware checks (network access), under which conditions 
they are performed, what servers are contacted, and what sort of caching 
of good/bad results are done.


Is any of that documented?

There is also blacklisting going on: I can get an executable locally 
blacklisted which will cause it to terminate instantly when executed. 
This seems to be about some run-time code signature validation, and when 
it happens, it appears to be the inode that gets blacklisted until next 
reboot, but more info about this would be nice.


[…] So I don’t think this test is analogous to your initial issue 
of a delay opening a file every time.


I said I get a similar delay the first time my app obtains URL 
properties¹ for ~/Desktop, ~/Documents, and, ~/Downloads, and I 
included sample code for this issue.


The actual delay appears to be in getxattr() which communicates with 
sandboxd, and the delay disappears when disabling WiFi, hence why it 
appears to also be a “phone home” issue like execve() above. But 
where execve() only does it on “first launch”, this check seems to 
be performed on first call after each new launch (for each different 
“protected” file path).


Perhaps you would be willing to add this sample code to a GUI 
application and see if you can reproduce? I re-attached it below, and 
have the result written to /tmp/duration.txt so you don’t have to 
fiddle with capturing log output.


Please note, it should be stand-alone GUI app launched from Finder, as 
the delay does not (always) happen when launched from the terminal.


¹ I later said I have seen many delays on my system since upgrading to 
macOS 10.15: But so far, I have only tracked down reproducible issues 
with execve() and getxattr(), but those do not explain all my delays, so 
there might be more low-level API that does “phone home”, I’m 
still in the data collection phase.


--8<--

void test ()
{
   NSTimeInterval startTime = NSDate.timeIntervalSinceReferenceDate;

   NSArray* urls = @[
	  [NSFileManager.defaultManager URLForDirectory:NSDesktopDirectory 
inDomain:NSUserDomainMask appropriateForURL:nil create:NO error:nil],
	  [NSFileManager.defaultManager URLForDirectory:NSDocumentDirectory 
inDomain:NSUserDomainMask appropriateForURL:nil create:NO error:nil],
	  [NSFileManager.defaultManager 
URLForDirectory:NSDownloadsDirectory inDomain:NSUserDomainMask 
appropriateForURL:nil create:NO error:nil],

   ];

   NSString* localizedName;
   for(NSURL* url in urls)
	  [url getResourceValue:&localizedName forKey:NSURLLocalizedNameKey 
error:nil];


	   NSString* duration = [NSString stringWithFormat:@"Duration %.03f", 
NSDate.timeIntervalSinceReferenceDate - startTime];
	   [duration writeToFile:@"/tmp/duration.txt" atomically:NO 
encoding:NSUTF8StringEncoding error:nil];

}
___

Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)

Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com

Help/Unsubscribe/Update your Subscription:
https://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com

This email sent to arch...@mail-archive.com

Re: Performance issue on macOS 10.15 obtaining display name for ~/Desktop, ~/Documents, and ~/Downloads

2020-04-23 Thread Gabriel Zachmann via Cocoa-dev 
> 
> I believe that is why you are supposed to staple notarization tickets to your 
> apps.
> 

Then, why would it "phone home" in case there is an internet connection?


Best regards, Gabriel


___

Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)

Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com

Help/Unsubscribe/Update your Subscription:
https://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com

This email sent to arch...@mail-archive.com

Re: Performance issue on macOS 10.15 obtaining display name for ~/Desktop, ~/Documents, and ~/Downloads

2020-04-23 Thread Saagar Jha via Cocoa-dev 
I believe that is why you are supposed to staple notarization tickets to your 
apps.

Saagar Jha

> On Apr 23, 2020, at 12:12, Gabriel Zachmann via Cocoa-dev 
>  wrote:
> 
>> 
>> It appears the problem is not with a local service, but that Apple 
>> actually ?phones home? when a program asks for display name.
>> 
>> I don?t know if this is common knowledge, but with notarization, Apple 
>> now validates executables on your system before they are executed, and 
>> it does so in calls like execve(), where it will actually stall 
>> execution, contact Apple?s servers, and then proceed once the 
>> executable got validated.
> 
> 
> I am just curious: what does it when there is *no* internet connection?
> (Suppose, someone downloads the app, then disconnects from internet, then 
> executes it;
> or copies the app via USB drive to the machine without internet connection.)
> And what is it *supposed* to do in that case?
> 
> 
> Best regards, Gabriel
> 
> 
> ___
> 
> Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)
> 
> Please do not post admin requests or moderator comments to the list.
> Contact the moderators at cocoa-dev-admins(at)lists.apple.com
> 
> Help/Unsubscribe/Update your Subscription:
> https://lists.apple.com/mailman/options/cocoa-dev/saagar%40saagarjha.com
> 
> This email sent to saa...@saagarjha.com

___

Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)

Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com

Help/Unsubscribe/Update your Subscription:
https://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com

This email sent to arch...@mail-archive.com

Re: Performance issue on macOS 10.15 obtaining display name for ~/Desktop, ~/Documents, and ~/Downloads

2020-04-23 Thread Rob Petrovec via Cocoa-dev 


> On Apr 23, 2020, at 9:10 AM, Allan Odgaard via Cocoa-dev 
>  wrote:
> 
> On 23 Apr 2020, at 21:15, Rob Petrovec wrote:
> 
>> If what you say is correct then everyone would be seeing a delay since most 
>> people don’t have blazing fast internet connections.  I do not think this is 
>> the normal behavior.
> 
> Please try run this in a terminal and report the times:
> 
>rm -f /tmp/test.sh && echo $'#!/bin/sh\necho hello' > /tmp/test.sh && 
> chmod a+x /tmp/test.sh && time /tmp/test.sh && time /tmp/test.sh
> 
> For this particular issue, it appears the lookup is cached by inode, so only 
> first run should take > 50ms, where second one would be < 5ms.
> 
> Then maybe also try it with Apple’s Network Link Conditioner and set it to 
> simulate lousy internet, and try it also with WiFi disabled.
> 
> I have seen the issue on multiple systems, so I do not think this is limited 
> to my system, but more data would be great.
I get a 1 second time for the first run and then a much quicker time 
for the second.  I did some sampling and the longer time due to is Apple’s 
check for malware on first run of a process.  This is a known, documented and 
advertised behavior.  It is a one time cost and only effects applications, not 
regular files.  For a developer though, yeah you get the cost after ever 
rebuild.  So I don’t think this test is analogous to your initial issue of a 
delay opening a file every time.



>> Sending an email to a developer mailing list doesn’t count.  Just sayin’...
> 
> My email was to ask for help, i.e. whether others have seen this issue, can 
> reproduce, have any idea what could cause this, etc.
> 
> Of course I am aware that this is not a place to report bugs to Apple…
So did you file a bug?  Would you mind posting the bug number?  Thanks.

—Rob

___

Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)

Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com

Help/Unsubscribe/Update your Subscription:
https://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com

This email sent to arch...@mail-archive.com

Re: Performance issue on macOS 10.15 obtaining display name for ~/Desktop, ~/Documents, and ~/Downloads

2020-04-23 Thread Gabriel Zachmann via Cocoa-dev 
> 
> It appears the problem is not with a local service, but that Apple 
> actually ?phones home? when a program asks for display name.
> 
> I don?t know if this is common knowledge, but with notarization, Apple 
> now validates executables on your system before they are executed, and 
> it does so in calls like execve(), where it will actually stall 
> execution, contact Apple?s servers, and then proceed once the 
> executable got validated.


I am just curious: what does it when there is *no* internet connection?
(Suppose, someone downloads the app, then disconnects from internet, then 
executes it;
or copies the app via USB drive to the machine without internet connection.)
And what is it *supposed* to do in that case?


Best regards, Gabriel


___

Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)

Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com

Help/Unsubscribe/Update your Subscription:
https://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com

This email sent to arch...@mail-archive.com

Re: Performance issue on macOS 10.15 obtaining display name for ~/Desktop, ~/Documents, and ~/Downloads

2020-04-23 Thread Allan Odgaard via Cocoa-dev 

On 23 Apr 2020, at 21:15, Rob Petrovec wrote:

If what you say is correct then everyone would be seeing a delay since 
most people don’t have blazing fast internet connections.  I do not 
think this is the normal behavior.


Please try run this in a terminal and report the times:

rm -f /tmp/test.sh && echo $'#!/bin/sh\necho hello' > /tmp/test.sh 
&& chmod a+x /tmp/test.sh && time /tmp/test.sh && time /tmp/test.sh


For this particular issue, it appears the lookup is cached by inode, so 
only first run should take > 50ms, where second one would be < 5ms.


Then maybe also try it with Apple’s Network Link Conditioner and set 
it to simulate lousy internet, and try it also with WiFi disabled.


I have seen the issue on multiple systems, so I do not think this is 
limited to my system, but more data would be great.


Sending an email to a developer mailing list doesn’t count.  Just 
sayin’...


My email was to ask for help, i.e. whether others have seen this issue, 
can reproduce, have any idea what could cause this, etc.


Of course I am aware that this is not a place to report bugs to Apple…

btw: today I took the drastic step of disabling SIP: My system now feels 
*much* better. Maybe some of the delays are bugs, but I’m quite sure 
the execve() issue is by design, it’s just a pretty developer 
unfriendly design, because the caching only lasts until the next 
rebuild, or incase of a script, the next (atomic) save, or incase of 
dynamically created helper scripts: no caching.

___

Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)

Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com

Help/Unsubscribe/Update your Subscription:
https://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com

This email sent to arch...@mail-archive.com

Re: Performance issue on macOS 10.15 obtaining display name for ~/Desktop, ~/Documents, and ~/Downloads

2020-04-23 Thread Rob Petrovec via Cocoa-dev 
If what you say is correct then everyone would be seeing a delay since most 
people don’t have blazing fast internet connections.  I do not think this is 
the normal behavior.  I think it is specific to your system, otherwise there 
would be TONS of people complaining about slowness.  A couple second delay 
opening a file or app like you describe would be all over the internet.  I 
don’t remember from your previous emails on this subject, but did you try 
creating a new partition and installing a fresh OS with a brand new user on it 
(nothing migrated) to see if the problem reproduced?  That would be an 
interesting data point.

Either way, did you file a bug with a sysdiagnose taken during the delay?  If 
so, do you have the bug number?  Something like this doesn’t get fixed if you 
don’t report it.  Sending an email to a developer mailing list doesn’t count.  
Just sayin’...

—Rob


> On Apr 22, 2020, at 11:16 PM, Allan Odgaard via Cocoa-dev 
>  wrote:
> 
> On 20 Apr 2020, at 0:11, Allan Odgaard via Cocoa-dev wrote:
> 
>> Unfortunately though I can’t figure out *what* the problem is; running 
>> `tccutil reset All` (and rebooting) did not fix it.
> 
> It appears the problem is not with a local service, but that Apple actually 
> “phones home” when a program asks for display name.
> 
> I don’t know if this is common knowledge, but with notarization, Apple now 
> validates executables on your system before they are executed, and it does so 
> in calls like execve(), where it will actually stall execution, contact 
> Apple’s servers, and then proceed once the executable got validated.
> 
> I *thought* this was the only place it did it, and that the result got cached 
> (based on inode).
> 
> But it seems Apple added this to other places, because since I have upgraded 
> to macOS 10.15, I see *many* delays.
> 
> This is because I am currently in South East Asia where the connection to 
> Apple’s servers is not good.
> 
> For example I have a script that takes a video file as argument, it launches 
> VLC with this video file, and then deletes the file when VLC terminates.
> 
> It can take more than 5 seconds just until VLC is launched, and then VLC will 
> be “thinking” for another 5 seconds, before the video actually starts.
> 
> Today the delays were extra bad, so it was easy to reproduce the VLC issue, 
> obtaining display name (which today took 7 seconds for 3 names), and a few 
> other things.
> 
> Now, if I disable internet, no delays at all!!!
> 
> Enable it again, and all the delays are back.
> 
> It is so utterly frustrating that Apple is not only going down this path of 
> locking down our machines, but they do it in ways that are so crippling for 
> our productivity :(
> ___
> 
> Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)
> 
> Please do not post admin requests or moderator comments to the list.
> Contact the moderators at cocoa-dev-admins(at)lists.apple.com
> 
> Help/Unsubscribe/Update your Subscription:
> https://lists.apple.com/mailman/options/cocoa-dev/petrock%40mac.com
> 
> This email sent to petr...@mac.com

___

Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)

Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com

Help/Unsubscribe/Update your Subscription:
https://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com

This email sent to arch...@mail-archive.com

Re: Performance issue on macOS 10.15 obtaining display name for ~/Desktop, ~/Documents, and ~/Downloads

2020-04-22 Thread Allan Odgaard via Cocoa-dev 

On 20 Apr 2020, at 0:11, Allan Odgaard via Cocoa-dev wrote:

Unfortunately though I can’t figure out *what* the problem is; 
running `tccutil reset All` (and rebooting) did not fix it.


It appears the problem is not with a local service, but that Apple 
actually “phones home” when a program asks for display name.


I don’t know if this is common knowledge, but with notarization, Apple 
now validates executables on your system before they are executed, and 
it does so in calls like execve(), where it will actually stall 
execution, contact Apple’s servers, and then proceed once the 
executable got validated.


I *thought* this was the only place it did it, and that the result got 
cached (based on inode).


But it seems Apple added this to other places, because since I have 
upgraded to macOS 10.15, I see *many* delays.


This is because I am currently in South East Asia where the connection 
to Apple’s servers is not good.


For example I have a script that takes a video file as argument, it 
launches VLC with this video file, and then deletes the file when VLC 
terminates.


It can take more than 5 seconds just until VLC is launched, and then VLC 
will be “thinking” for another 5 seconds, before the video actually 
starts.


Today the delays were extra bad, so it was easy to reproduce the VLC 
issue, obtaining display name (which today took 7 seconds for 3 names), 
and a few other things.


Now, if I disable internet, no delays at all!!!

Enable it again, and all the delays are back.

It is so utterly frustrating that Apple is not only going down this path 
of locking down our machines, but they do it in ways that are so 
crippling for our productivity :(

___

Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)

Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com

Help/Unsubscribe/Update your Subscription:
https://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com

This email sent to arch...@mail-archive.com

Re: Performance issue on macOS 10.15 obtaining display name for ~/Desktop, ~/Documents, and ~/Downloads

2020-04-19 Thread Gary L. Wade via Cocoa-dev 
Regardless of whatever workaround you find, I would second Rob’s suggestion to 
go ahead and file a bug with a sysdiagnose and/or spindump along with a sample 
app that reproduces it.  This isn’t expected behavior, and the teams at Apple 
are still working and would be very interested in seeing this.  Once filed, if 
you do find a workaround, you can append that to the bug.
--
Gary L. Wade
http://www.garywade.com/ 

> On Apr 19, 2020, at 10:58 AM, Allan Odgaard via Cocoa-dev 
>  wrote:
> 
> On 20 Apr 2020, at 0:37, Rob Petrovec wrote:
> 
>> >> I think you are right about this being a permission / “sandbox” issue, 
>> >> because the 3 folders in question are all folders that macOS 10.15 now 
>> >> require special permission to read (even though in my case, I just 
>> >> request their display name).
>>  Yes, this is because of iCloud.  Log out of iCloud and it should go 
>> away as I suggested in my first reply.  You could also grant your app full 
>> disk access and that might ‘fix’ it for you too.  But your users would 
>> probably not want to do that.
> 
> As written previously, there is no delay when running from terminal, and 
> ~/Downloads is also causing a delay, which is not stored on iCloud.
> 
> So this very much seems to be about Apple’s file system access policies where 
> permission must be obtained to access certain folders.
> 
> But as mentioned, I have given my app full disk access, reset permissions, 
> and I have even signed and notarized the app: Still no fix for the delay.
> 
> To indulge you, I have now also tried to log out of iCloud: Still a delay.
> 
> Now I would very much appreciate if someone would try the code I included in 
> my original email to see if this is a general problem, or something limited 
> to my system.
> 
> As I think I mentioned, I am running a fairly clean install of macOS 10.15.

___

Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)

Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com

Help/Unsubscribe/Update your Subscription:
https://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com

This email sent to arch...@mail-archive.com

Re: Performance issue on macOS 10.15 obtaining display name for ~/Desktop, ~/Documents, and ~/Downloads

2020-04-19 Thread Allan Odgaard via Cocoa-dev 

On 20 Apr 2020, at 0:37, Rob Petrovec wrote:

>> I think you are right about this being a permission / “sandbox” 
issue, because the 3 folders in question are all folders that macOS 
10.15 now require special permission to read (even though in my case, 
I just request their display name).
	Yes, this is because of iCloud.  Log out of iCloud and it should go 
away as I suggested in my first reply.  You could also grant your app 
full disk access and that might ‘fix’ it for you too.  But your 
users would probably not want to do that.


As written previously, there is no delay when running from terminal, and 
~/Downloads is also causing a delay, which is not stored on iCloud.


So this very much seems to be about Apple’s file system access 
policies where permission must be obtained to access certain folders.


But as mentioned, I have given my app full disk access, reset 
permissions, and I have even signed and notarized the app: Still no fix 
for the delay.


To indulge you, I have now also tried to log out of iCloud: Still a 
delay.


Now I would very much appreciate if someone would try the code I 
included in my original email to see if this is a general problem, or 
something limited to my system.


As I think I mentioned, I am running a fairly clean install of macOS 
10.15.

___

Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)

Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com

Help/Unsubscribe/Update your Subscription:
https://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com

This email sent to arch...@mail-archive.com

Re: Performance issue on macOS 10.15 obtaining display name for ~/Desktop, ~/Documents, and ~/Downloads

2020-04-19 Thread Rob Petrovec via Cocoa-dev 
>> I think you are right about this being a permission / “sandbox” issue, 
>> because the 3 folders in question are all folders that macOS 10.15 now 
>> require special permission to read (even though in my case, I just request 
>> their display name).
Yes, this is because of iCloud.  Log out of iCloud and it should go 
away as I suggested in my first reply.  You could also grant your app full disk 
access and that might ‘fix’ it for you too.  But your users would probably not 
want to do that.


>> That’s 181 milliseconds spent __WAITING_ON_APPROVAL_FROM_SANDBOXD__ in a 
>> non-sandboxed app obtaining display name for 3 folders in the user’s home 
>> directory.
Its actually worse then that.  At the top of the spindump file it says 
was the numbers represent.  Something like:

Duration: 10.00s
Steps:1001 (10ms sampling interval)

So 181 actually represents 1810ms, or 1.81 seconds.  So this is a sandbox 
performance issue.  Please file a bug and include a sysdiagnose as I suggested 
in my first reply.

—Rob



> On Apr 19, 2020, at 11:11 AM, Allan Odgaard via Cocoa-dev 
>  wrote:
> 
> On 19 Apr 2020, at 22:54, David M. Cotter wrote:
> 
>> i have discovered it may have to do with permissions / entitlements that 
>> have been granted the app by the user, and that resetting all perms to 
>> default will "fix" the problem
> 
> I think you are right about this being a permission / “sandbox” issue, 
> because the 3 folders in question are all folders that macOS 10.15 now 
> require special permission to read (even though in my case, I just request 
> their display name).
> 
> Furthermore, it could explain why there is no delay when launched from 
> terminal, as the process will inherit the entitlements of the terminal, which 
> may have some special entitlements for the full disk access.
> 
> Unfortunately though I can’t figure out *what* the problem is; running 
> `tccutil reset All` (and rebooting) did not fix it.
> 
> I have tried the code in question in a signed and notarized app that has been 
> granted full disk access, yet it still has the delay.
> 
> A spindump does indicate it is a “sandbox” issue:
> 
>   *182   getattrlist + 136 (kernel + 3512472) [0xff8000559898]
> *182   ??? (kernel + 3512820) [0xff80005599f4]
>   *181   ??? (kernel + 3501529) [0xff8000556dd9]
> *181   ??? (kernel + 3490836) [0xff8000554414]
>   *181   mac_vnode_check_access + 154 (kernel + 9093082) 
> [0xff8000aabfda]
> *181   hook_vnode_check_access + 167 (Sandbox + 39552) 
> [0xff7f823a7a80]
>   *181   sb_evaluate + 5004 (Sandbox + 67658) [0xff7f823ae84a]
> *181   approval_response_wait + 142 (Sandbox + 154851) 
> [0xff7f823c3ce3]
>   *181   __WAITING_ON_APPROVAL_FROM_SANDBOXD__ + 23 (Sandbox 
> + 155134) [0xff7f823c3dfe]
> *181   ??? (kernel + 6855402) [0xff8000889aea]
>   *181   thread_block_reason + 175 (kernel + 1317935) 
> [0xff8000341c2f]
> *181   ??? (kernel + 1324017) [0xff80003433f1]
>   *181   machine_switch_context + 200 (kernel + 
> 2388456) [0xff80004471e8]
> 
> That’s 181 milliseconds spent __WAITING_ON_APPROVAL_FROM_SANDBOXD__ in a 
> non-sandboxed app obtaining display name for 3 folders in the user’s home 
> directory.
> ___
> 
> Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)
> 
> Please do not post admin requests or moderator comments to the list.
> Contact the moderators at cocoa-dev-admins(at)lists.apple.com
> 
> Help/Unsubscribe/Update your Subscription:
> https://lists.apple.com/mailman/options/cocoa-dev/petrock%40mac.com
> 
> This email sent to petr...@mac.com

___

Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)

Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com

Help/Unsubscribe/Update your Subscription:
https://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com

This email sent to arch...@mail-archive.com

Re: Performance issue on macOS 10.15 obtaining display name for ~/Desktop, ~/Documents, and ~/Downloads

2020-04-19 Thread Allan Odgaard via Cocoa-dev 

On 19 Apr 2020, at 22:54, David M. Cotter wrote:

i have discovered it may have to do with permissions / entitlements 
that have been granted the app by the user, and that resetting all 
perms to default will "fix" the problem


I think you are right about this being a permission / “sandbox” 
issue, because the 3 folders in question are all folders that macOS 
10.15 now require special permission to read (even though in my case, I 
just request their display name).


Furthermore, it could explain why there is no delay when launched from 
terminal, as the process will inherit the entitlements of the terminal, 
which may have some special entitlements for the full disk access.


Unfortunately though I can’t figure out *what* the problem is; running 
`tccutil reset All` (and rebooting) did not fix it.


I have tried the code in question in a signed and notarized app that has 
been granted full disk access, yet it still has the delay.


A spindump does indicate it is a “sandbox” issue:

   *182   getattrlist + 136 (kernel + 3512472) [0xff8000559898]
 *182   ??? (kernel + 3512820) [0xff80005599f4]
   *181   ??? (kernel + 3501529) [0xff8000556dd9]
 *181   ??? (kernel + 3490836) [0xff8000554414]
   *181   mac_vnode_check_access + 154 (kernel + 9093082) 
[0xff8000aabfda]
 *181   hook_vnode_check_access + 167 (Sandbox + 39552) 
[0xff7f823a7a80]
   *181   sb_evaluate + 5004 (Sandbox + 67658) 
[0xff7f823ae84a]
 *181   approval_response_wait + 142 (Sandbox + 154851) 
[0xff7f823c3ce3]
   *181   __WAITING_ON_APPROVAL_FROM_SANDBOXD__ + 23 
(Sandbox + 155134) [0xff7f823c3dfe]

 *181   ??? (kernel + 6855402) [0xff8000889aea]
   *181   thread_block_reason + 175 (kernel + 
1317935) [0xff8000341c2f]
 *181   ??? (kernel + 1324017) 
[0xff80003433f1]
   *181   machine_switch_context + 200 (kernel 
+ 2388456) [0xff80004471e8]


That’s 181 milliseconds spent __WAITING_ON_APPROVAL_FROM_SANDBOXD__ in 
a non-sandboxed app obtaining display name for 3 folders in the user’s 
home directory.

___

Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)

Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com

Help/Unsubscribe/Update your Subscription:
https://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com

This email sent to arch...@mail-archive.com

Re: Performance issue on macOS 10.15 obtaining display name for ~/Desktop, ~/Documents, and ~/Downloads

2020-04-19 Thread Rob Petrovec via Cocoa-dev 
I assume you have iCloud enabled.  If so, these three folders are ’special’.  
Try logging out of iCloud & rebooting and see if the problem persists.

I would also recommend filing a bug with Apple and include a sysdiagnose taken 
while the problem was reproducing (sudo sysdiagnose).  Or at least a spindump 
(sudo spindump) while the problem is reproducing.

—Rob



> On Apr 19, 2020, at 9:54 AM, David M. Cotter via Cocoa-dev 
>  wrote:
> 
> this may be difficult for other to repro
> 
> i have discovered it may have to do with permissions / entitlements that have 
> been granted the app by the user, and that resetting all perms to default 
> will "fix" the problem
> 
> in the terminal, do this:
>> tccutil reset All
> i expect that after that, your delay will disappear.
> 
> however, you'll have to manually re-authorize all apps in the system 
> prefs->security & privacy->privacy tab
> 
> i realize this is a "sweep the problem under the rug" rather than "fix the 
> problem" approach
> but hopefully this should give someone else a clue as to WHY this is 
> happening?
> 
> -dave
> 
>> On Apr 19, 2020, at 8:08 AM, Allan Odgaard via Cocoa-dev 
>>  wrote:
>> 
>> Starting with macOS 10.15 I have noticed that obtaining 
>> NSURLLocalizedNameKey, NSURLTagNamesKey, and even calling getxattr(), can 
>> cause a significant delay.
>> 
>> The code below, which gets NSURLLocalizedNameKey for 3 folders, takes 1.9 
>> seconds to execute on my system.
>> 
>> It appears though that it is only these 3 specific folders that has the 
>> problem.
>> 
>> Furthermore, the problem only happens when I include the code below in a GUI 
>> application which is launched via launch services. If I run the executable 
>> directly from a terminal then the code is near instant.
>> 
>> I tried to sample the code, and it appears that it calls out to some OS 
>> service/daemon. Presumably it never gets a response, and a timeout is 
>> triggered.
>> 
>> I wonder if anyone else has come across this? And maybe have discovered more 
>> info?
>> 
>> This is btw macOS 10.15.4 and a fairly stock system (new computer without 
>> migrating old data), and I noticed the problem some time ago, so it 
>> definitely existed before 10.15.4, but I never saw anything like this before 
>> upgrading to Catelina.
>> 
>>   NSTimeInterval startTime = NSDate.timeIntervalSinceReferenceDate;
>> 
>>   NSArray* urls = @[]
>>   [NSFileManager.defaultManager URLForDirectory:NSDesktopDirectory 
>> inDomain:NSUserDomainMask appropriateForURL:nil create:NO error:nil],,
>>   [NSFileManager.defaultManager URLForDirectory:NSDocumentDirectory 
>> inDomain:NSUserDomainMask appropriateForURL:nil create:NO error:nil]
>>   [NSFileManager.defaultManager URLForDirectory:NSDownloadsDirectory 
>> inDomain:NSUserDomainMask appropriateForURL:nil create:NO error:nil,
>>   ];
>> 
>>   NSString* localizedName;
>>   for(NSURL* url in urls)
>>   [url getResourceValue:&localizedName forKey:NSURLLocalizedNameKey 
>> error:nil];
>> 
>>   NSLog(@"Duration %.03f", NSDate.timeIntervalSinceReferenceDate - 
>> startTime);
> 
> ___
> 
> Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)
> 
> Please do not post admin requests or moderator comments to the list.
> Contact the moderators at cocoa-dev-admins(at)lists.apple.com
> 
> Help/Unsubscribe/Update your Subscription:
> https://lists.apple.com/mailman/options/cocoa-dev/petrock%40mac.com
> 
> This email sent to petr...@mac.com

___

Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)

Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com

Help/Unsubscribe/Update your Subscription:
https://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com

This email sent to arch...@mail-archive.com

Re: Performance issue on macOS 10.15 obtaining display name for ~/Desktop, ~/Documents, and ~/Downloads

2020-04-19 Thread David M. Cotter via Cocoa-dev 
this may be difficult for other to repro

i have discovered it may have to do with permissions / entitlements that have 
been granted the app by the user, and that resetting all perms to default will 
"fix" the problem

in the terminal, do this:
> tccutil reset All
i expect that after that, your delay will disappear.

however, you'll have to manually re-authorize all apps in the system 
prefs->security & privacy->privacy tab

i realize this is a "sweep the problem under the rug" rather than "fix the 
problem" approach
but hopefully this should give someone else a clue as to WHY this is happening?

-dave

> On Apr 19, 2020, at 8:08 AM, Allan Odgaard via Cocoa-dev 
>  wrote:
> 
> Starting with macOS 10.15 I have noticed that obtaining 
> NSURLLocalizedNameKey, NSURLTagNamesKey, and even calling getxattr(), can 
> cause a significant delay.
> 
> The code below, which gets NSURLLocalizedNameKey for 3 folders, takes 1.9 
> seconds to execute on my system.
> 
> It appears though that it is only these 3 specific folders that has the 
> problem.
> 
> Furthermore, the problem only happens when I include the code below in a GUI 
> application which is launched via launch services. If I run the executable 
> directly from a terminal then the code is near instant.
> 
> I tried to sample the code, and it appears that it calls out to some OS 
> service/daemon. Presumably it never gets a response, and a timeout is 
> triggered.
> 
> I wonder if anyone else has come across this? And maybe have discovered more 
> info?
> 
> This is btw macOS 10.15.4 and a fairly stock system (new computer without 
> migrating old data), and I noticed the problem some time ago, so it 
> definitely existed before 10.15.4, but I never saw anything like this before 
> upgrading to Catelina.
> 
>NSTimeInterval startTime = NSDate.timeIntervalSinceReferenceDate;
> 
>NSArray* urls = @[]
>[NSFileManager.defaultManager URLForDirectory:NSDesktopDirectory 
> inDomain:NSUserDomainMask appropriateForURL:nil create:NO error:nil],,
>[NSFileManager.defaultManager URLForDirectory:NSDocumentDirectory 
> inDomain:NSUserDomainMask appropriateForURL:nil create:NO error:nil]
>[NSFileManager.defaultManager URLForDirectory:NSDownloadsDirectory 
> inDomain:NSUserDomainMask appropriateForURL:nil create:NO error:nil,
>];
> 
>NSString* localizedName;
>for(NSURL* url in urls)
>[url getResourceValue:&localizedName forKey:NSURLLocalizedNameKey 
> error:nil];
> 
>NSLog(@"Duration %.03f", NSDate.timeIntervalSinceReferenceDate - 
> startTime);

___

Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)

Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com

Help/Unsubscribe/Update your Subscription:
https://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com

This email sent to arch...@mail-archive.com

Performance issue on macOS 10.15 obtaining display name for ~/Desktop, ~/Documents, and ~/Downloads

2020-04-19 Thread Allan Odgaard via Cocoa-dev 
Starting with macOS 10.15 I have noticed that obtaining 
NSURLLocalizedNameKey, NSURLTagNamesKey, and even calling getxattr(), 
can cause a significant delay.


The code below, which gets NSURLLocalizedNameKey for 3 folders, takes 
1.9 seconds to execute on my system.


It appears though that it is only these 3 specific folders that has the 
problem.


Furthermore, the problem only happens when I include the code below in a 
GUI application which is launched via launch services. If I run the 
executable directly from a terminal then the code is near instant.


I tried to sample the code, and it appears that it calls out to some OS 
service/daemon. Presumably it never gets a response, and a timeout is 
triggered.


I wonder if anyone else has come across this? And maybe have discovered 
more info?


This is btw macOS 10.15.4 and a fairly stock system (new computer 
without migrating old data), and I noticed the problem some time ago, so 
it definitely existed before 10.15.4, but I never saw anything like this 
before upgrading to Catelina.


NSTimeInterval startTime = NSDate.timeIntervalSinceReferenceDate;

NSArray* urls = @[]
[NSFileManager.defaultManager 
URLForDirectory:NSDesktopDirectory inDomain:NSUserDomainMask 
appropriateForURL:nil create:NO error:nil],,
[NSFileManager.defaultManager 
URLForDirectory:NSDocumentDirectory inDomain:NSUserDomainMask 
appropriateForURL:nil create:NO error:nil]
[NSFileManager.defaultManager 
URLForDirectory:NSDownloadsDirectory inDomain:NSUserDomainMask 
appropriateForURL:nil create:NO error:nil,

];

NSString* localizedName;
for(NSURL* url in urls)
[url getResourceValue:&localizedName 
forKey:NSURLLocalizedNameKey error:nil];


NSLog(@"Duration %.03f", NSDate.timeIntervalSinceReferenceDate - 
startTime);

___

Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)

Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com

Help/Unsubscribe/Update your Subscription:
https://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com

This email sent to arch...@mail-archive.com