Re: Performance issue on macOS 10.15 obtaining display name for ~/Desktop, ~/Documents, and ~/Downloads

[Allan Odgaard via Cocoa-dev]

On 22 May 2020, at 21:59, Aandi Inston wrote:


1. I wonder if your tests are in a non-English language system.


No, running on a non-localized system, and the evidence is overwhelming 
that this is about SIP / AMFI (based on inspecting the stack trace, 
which clearly show communication between the process and Apple’s 
various security policy daemons).


3. I wonder if the other testers who could not reproduce this were 
using an

English language system.


There is plenty of people who have confirmed the issue. There was one on 
this list who said he couldn’t reproduce, but he also said he changed 
the code to NSLog, so I wonder if he ran from Xcode, in which case the 
new process probably inherited Xcode’s privileges, and I have just 
learned that some users have a Developer Tools under System Preferences 
→ Security & Privacy. Adding something to this group seems to disable 
a lot of security checks (and all child processes).


I currently do not see this category on my system though, and based on 
comments, I am not alone.

___

Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)

Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com

Help/Unsubscribe/Update your Subscription:
https://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com

This email sent to arch...@mail-archive.com

Re: Performance issue on macOS 10.15 obtaining display name for ~/Desktop, ~/Documents, and ~/Downloads

[Rob Petrovec via Cocoa-dev]

> On May 22, 2020, at 8:42 AM, Allan Odgaard via Cocoa-dev 
>  wrote:
> 
> On 23 Apr 2020, at 21:15, Rob Petrovec wrote:
> 
>> If what you say is correct then everyone would be seeing a delay since most 
>> people don’t have blazing fast internet connections.  I do not think this is 
>> the normal behavior.  I think it is specific to your system, otherwise there 
>> would be TONS of people complaining about slowness.
> 
> Episode 379 of ATP has just been released and both Marco Arment and John 
> Siracusa are complaining about significant delays after upgrading to macOS 
> 10.15, though for John Siracusa I think only one of his machines has the 
> problem.
> 
> Also, I think I already mentioned it, but I had a friend of mine run tests on 
> his machine in another geographical zone, and he saw delays as well.
> 
>> Either way, did you file a bug with a sysdiagnose taken during the delay?  
>> If so, do you have the bug number?
> 
> Several people in this thread were asking for bug numbers: I do not 
> understand this, no-one outside of Apple can read these bugs, right?
Correct, however, you are presuming that no apple engineers reads this 
mailing list. That would be an incorrect assumption.

—Rob




___

Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)

Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com

Help/Unsubscribe/Update your Subscription:
https://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com

This email sent to arch...@mail-archive.com

Re: Performance issue on macOS 10.15 obtaining display name for ~/Desktop, ~/Documents, and ~/Downloads

[Aandi Inston via Cocoa-dev]

Your report is interesting. I have been following it but may have missed a
part of the discussion, so here is one
thought I had. You say that getting the display name of ~/Documents may
result in a delay. This is localizable, so
that in Portugal (for example) the display name is Documentos. So here are
thoughts
1. I wonder if your tests are in a non-English language system.
2. I wonder if in an English language system there is a fast path that
tells the system to bypass localization tests.
3. I wonder if the other testers who could not reproduce this were using an
English language system.
I observe that the internet says (so it must be true) that the test for
localized folders is a hidden file in the folder
called ".localized". To check for this in any obvious way is going to need
read access to the folder, and hence may
trigger security checking.

On Fri, 22 May 2020 at 15:43, Allan Odgaard via Cocoa-dev <
cocoa-dev@lists.apple.com> wrote:

> On 23 Apr 2020, at 21:15, Rob Petrovec wrote:
>
> > If what you say is correct then everyone would be seeing a delay since
> > most people don’t have blazing fast internet connections.  I do not
> > think this is the normal behavior.  I think it is specific to your
> > system, otherwise there would be TONS of people complaining about
> > slowness.
>
> Episode 379 of ATP has just been released and both Marco Arment and John
> Siracusa are complaining about significant delays after upgrading to
> macOS 10.15, though for John Siracusa I think only one of his machines
> has the problem.
>
> Also, I think I already mentioned it, but I had a friend of mine run
> tests on his machine in another geographical zone, and he saw delays as
> well.
>
> > Either way, did you file a bug with a sysdiagnose taken during the
> > delay?  If so, do you have the bug number?
>
> Several people in this thread were asking for bug numbers: I do not
> understand this, no-one outside of Apple can read these bugs, right?
>
> But now I wrote up a summary of what I have found so far with bug
> numbers included:
> https://sigpipe.macromates.com/2020/macos-catalina-slow-by-design/
> ___
>
> Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)
>
> Please do not post admin requests or moderator comments to the list.
> Contact the moderators at cocoa-dev-admins(at)lists.apple.com
>
> Help/Unsubscribe/Update your Subscription:
> https://lists.apple.com/mailman/options/cocoa-dev/aandi%40quite.com
>
> This email sent to aa...@quite.com
>
___

Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)

Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com

Help/Unsubscribe/Update your Subscription:
https://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com

This email sent to arch...@mail-archive.com

Re: Performance issue on macOS 10.15 obtaining display name for ~/Desktop, ~/Documents, and ~/Downloads

[Allan Odgaard via Cocoa-dev]

On 23 Apr 2020, at 21:15, Rob Petrovec wrote:

If what you say is correct then everyone would be seeing a delay since 
most people don’t have blazing fast internet connections.  I do not 
think this is the normal behavior.  I think it is specific to your 
system, otherwise there would be TONS of people complaining about 
slowness.


Episode 379 of ATP has just been released and both Marco Arment and John 
Siracusa are complaining about significant delays after upgrading to 
macOS 10.15, though for John Siracusa I think only one of his machines 
has the problem.


Also, I think I already mentioned it, but I had a friend of mine run 
tests on his machine in another geographical zone, and he saw delays as 
well.


Either way, did you file a bug with a sysdiagnose taken during the 
delay?  If so, do you have the bug number?


Several people in this thread were asking for bug numbers: I do not 
understand this, no-one outside of Apple can read these bugs, right?


But now I wrote up a summary of what I have found so far with bug 
numbers included: 
https://sigpipe.macromates.com/2020/macos-catalina-slow-by-design/

___

Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)

Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com

Help/Unsubscribe/Update your Subscription:
https://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com

This email sent to arch...@mail-archive.com

Re: Performance issue on macOS 10.15 obtaining display name for ~/Desktop, ~/Documents, and ~/Downloads

[Gary L. Wade via Cocoa-dev]

Also, did you take advantage of one of your free tech support incidents?
--
Gary L. Wade
http://www.garywade.com/

> On Apr 24, 2020, at 8:26 AM, Gary L. Wade via Cocoa-dev 
>  wrote:
> 
> That’s a very narrow view of reality, which I know to be far broader.
> 
> What’s the feedback number?
> --
> Gary
> 
>>> On Apr 24, 2020, at 8:01 AM, Allan Odgaard via Cocoa-dev 
>>>  wrote:
>> That said, I *have* filed a report about this, but I still seek more 
>> information about the issue, which I had hoped to get from this list, 
>> because I sure as hell am not going to get it via Apple’s bug reporting 
>> system.

___

Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)

Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com

Help/Unsubscribe/Update your Subscription:
https://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com

This email sent to arch...@mail-archive.com

Re: Performance issue on macOS 10.15 obtaining display name for ~/Desktop, ~/Documents, and ~/Downloads

[Gary L. Wade via Cocoa-dev]

That’s a very narrow view of reality, which I know to be far broader.

What’s the feedback number?
--
Gary

> On Apr 24, 2020, at 8:01 AM, Allan Odgaard via Cocoa-dev 
>  wrote:
> 
> That said, I *have* filed a report about this, but I still seek more 
> information about the issue, which I had hoped to get from this list, because 
> I sure as hell am not going to get it via Apple’s bug reporting system.

___

Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)

Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com

Help/Unsubscribe/Update your Subscription:
https://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com

This email sent to arch...@mail-archive.com

Re: Performance issue on macOS 10.15 obtaining display name for ~/Desktop, ~/Documents, and ~/Downloads

[Allan Odgaard via Cocoa-dev]

On 24 Apr 2020, at 21:33, Gary L. Wade wrote:

Here’s two web sites that should help you get the answer you want. 
Try one or both:


https://feedbackassistant.apple.com/welcome
https://www.apple.com/jobs/us/


You don’t get answers from filing bug reports with Apple, at best, the 
issue gets closed as a duplicate, which seems to be acknowledgement that 
others have run into the same problem, but then you never hear more 
about it. Alternatively the bug just stays open for years, maybe at some 
future OS release, you get an email saying to please test if the issue 
is still present.


That said, I *have* filed a report about this, but I still seek more 
information about the issue, which I had hoped to get from this list, 
because I sure as hell am not going to get it via Apple’s bug 
reporting system.


For the records, I asked a friend to run the test on his system, he got 
a delay of 160-170 ms, he is in Europe where Apple have local data 
centers, so it’s not unreasonable to think his ping time to 
gs.apple.com is 30-50 ms, hence it still seems plausible that it does a 
connection to Apple’s server for a call to getxattr().


Though I’m waiting for more data from him.
___

Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)

Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com

Help/Unsubscribe/Update your Subscription:
https://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com

This email sent to arch...@mail-archive.com

Re: Performance issue on macOS 10.15 obtaining display name for ~/Desktop, ~/Documents, and ~/Downloads

[Gary L. Wade via Cocoa-dev]

Here’s two web sites that should help you get the answer you want. Try one or 
both:

https://feedbackassistant.apple.com/welcome
https://www.apple.com/jobs/us/
--
Gary
___

Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)

Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com

Help/Unsubscribe/Update your Subscription:
https://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com

This email sent to arch...@mail-archive.com

Re: Performance issue on macOS 10.15 obtaining display name for ~/Desktop, ~/Documents, and ~/Downloads

[Sandor Szatmari via Cocoa-dev]

Allan,

> On Apr 24, 2020, at 00:14, Allan Odgaard via Cocoa-dev 
>  wrote:
> 
> On 24 Apr 2020, at 9:51, Gary L. Wade wrote:
> 
>> Have you tried a speed check with just iCloud turned off but internet on?
> 
> I have tried with iCloud disabled, internet disabled, and SIP disabled.

Have you tried something like the Little Snitch app to Confirm there is network 
access to Apple and determine what network activity is going on?

Sandor
> 
> Only the latter two removes the delay. Also, the issue happens for 
> ~/Downloads which is not an iCloud folder.
> 
> Furthermore, the stack dump seems fairly clear about what goes on.
> 
> For the records, I’m reposting my findings here with stack dump.
> 
> This is actually from Transmission.app, in Preferences you can configure 
> download folders, so I selected ~/Documents, ~/Downloads, and ~/Desktop, 
> which are the 3 “protected” folders. I then ran a spindump when opening 
> Preferences, which has to obtain display name and icon for these folders (to 
> show in the UI), which results in 620 ms spent in NSWorkspace’s iconForFile: 
> (getattrlist).
> 
>62   -[NSWorkspace iconForFile:] + 80 (AppKit + 3229058) [0x7fff32ee1582]
>62   -[NSWorkspace _iconForURL:] + 100 (AppKit + 3229205) [0x7fff32ee1615]
>62   _GetIconRefFromURL + 26 (LaunchServices + 233122) [0x7fff370edea2]
>62   BindingManager::CreateWithURL(__CFURL const*, bool) + 28 
> (LaunchServices + 233204) [0x7fff370edef4]
>62   BindingBlueprint::BindingBlueprint(__CFURL const*) + 78 
> (LaunchServices + 233350) [0x7fff370edf86]
>62   BindingBlueprint::copyURLProperties(__CFURL const*) + 50 
> (LaunchServices + 233496) [0x7fff370ee018]
>62   CFURLCopyResourcePropertiesForKeys + 111 (CoreFoundation + 555324) 
> [0x7fff3599493c]
>62   _FSURLCopyResourcePropertiesForKeysInternal(__CFURL const*, __CFArray 
> const*, void*, __CFError**, unsigned char) + 1110 (CoreServicesInternal + 
> 60765) [0x7fff4ecb0d5d]
>62   prepareValuesForBitmap(__CFURL const*, __FileCache*, 
> _FilePropertyBitmap*, __CFError**) + 363 (CoreServicesInternal + 17604) 
> [0x7fff4eca64c4]
>62   __getattrlist + 10 (libsystem_kernel.dylib + 5746) [0x7fff6fa19672]
>*62  hndl_unix_scall64 + 22 (kernel + 819718) [0xff80002c8206]
>*62  unix_syscall64 + 647 (kernel + 7894519) [0xff80009875f7]
>*62  getattrlist + 136 (kernel + 3512472) [0xff8000559898]
>*62  ??? (kernel + 3512820) [0xff80005599f4]
>*62  ??? (kernel + 3501529) [0xff8000556dd9]
>*62  ??? (kernel + 3490836) [0xff8000554414]
>*62  mac_vnode_check_access + 154 (kernel + 9093082) [0xff8000aabfda]
>*62  hook_vnode_check_access + 167 (Sandbox + 39552) [0xff7f823a7a80]
>*62  sb_evaluate + 5004 (Sandbox + 67658) [0xff7f823ae84a]
>*62  approval_response_wait + 142 (Sandbox + 154851) [0xff7f823c3ce3]
>*62  __WAITING_ON_APPROVAL_FROM_SANDBOXD__ + 23 (Sandbox + 155134) 
> [0xff7f823c3dfe]
>*62  ??? (kernel + 6855402) [0xff8000889aea]
>*62  thread_block_reason + 175 (kernel + 1317935) [0xff8000341c2f]
>*62  ??? (kernel + 1324017) [0xff80003433f1]
>*62  machine_switch_context + 200 (kernel + 2388456) [0xff80004471e8]
> 
> Looking at sandboxd, it has 3 queues spending respectively 150, 160, and 310 
> ms in TCCAccessPreflightWithAuditToken (so total time spent there is 620 ms), 
> which I would guess is processing Transmission’s request for reading the 
> extended attributes for the 3 paths.
> 
> But sandboxd itself does tccd_send_message, so it seems to call upon the tccd 
> daemon.
> 
> There are two tccd instances running on my system, each have two queues with 
> 150 ms spent in SecCodeCheckValidityWithErrors (so a total of 600 ms spent in 
> tccd).
> 
> This code though again seems rely on another process by calling: 
> securityd_send_sync_and_do (which does XPC).
> 
> But I can’t find out which process it communicates with. I don’t know if this 
> could be launched-on-demand and therefore not part of the spindump report?
> 
> But to summarize, first call of getattrlist/getxattr for a protected folder 
> seems to trigger application signature validation (makes sense), which, at 
> least on my system, appears to involve network access, reminiscent of what 
> goes on with execve().
> 
> Given the above, I don’t think anyone can dispute that this is what goes on, 
> and it would be extremely strange if this is not “by design”.
> 
> What might be a bug is that the check does network access, but you don’t just 
> put such code in by mistake, so obviously there must be certain times where 
> it needs network access, and as demonstrated with execve(), Apple is not shy 
> of making low-level system APIs contact Apple’s servers as part of their SIP.
> 
> The million dollar question is now: What are the conditions under which it 
> must have network access? and/or why does it seem to trigger all the time on 
> my system?
> 

Re: Performance issue on macOS 10.15 obtaining display name for ~/Desktop, ~/Documents, and ~/Downloads

[Allan Odgaard via Cocoa-dev]

On 24 Apr 2020, at 11:49, Saagar Jha wrote:


GateKeeper is basically Safari adding a quarantine flag […]
Nit: not just Safari; other applications do this to at their 
discretion when appropriate (for example, if they too download files 
from the internet). Quarantine is just one part of GateKeeper.


Right, I said “basically” to indicate I was simplifying the 
description.


But at least it was a user-space feature with co-operation from (amongst 
others) Finder and Safari, without the need for any network access.


This is different from the issue being discussed, which is kernel-space 
feature that does network access even for locally created files.


GateKeeper and XProtect though are probably more umbrella marketing 
terms than actual technologies, so they may cover more today, for 
example app translocation probably also falls under GateKeeper, and the 
recent uninstall of the Zoom web server I think was said to fall under 
XProtect.

___

Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)

Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com

Help/Unsubscribe/Update your Subscription:
https://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com

This email sent to arch...@mail-archive.com

Re: Performance issue on macOS 10.15 obtaining display name for ~/Desktop, ~/Documents, and ~/Downloads

[Saagar Jha via Cocoa-dev]

Saagar Jha

> On Apr 23, 2020, at 21:26, Allan Odgaard via Cocoa-dev 
>  wrote:
> 
> On 24 Apr 2020, at 9:57, Rob Petrovec wrote:
> 
>>> Also weird, why would it phone home for a shell script which has neither 
>>> been stapled nor even code-signed?
>> I think you answered the question just then…  a "shell script which has 
>> neither been stapled nor even code-signed”.  Google XProtect & Gatekeeper.
> 
> GateKeeper is basically Safari adding a quarantine flag (via extended 
> attributes)

Nit: not just Safari; other applications do this to at their discretion when 
appropriate (for example, if they too download files from the internet). 
Quarantine is just one part of GateKeeper.

> to files downloaded form the internet, and then having Finder check this 
> flag, throwing up a dialog if the flag is set, and recently, also checking if 
> the code signature is from a verified developer (possibly refusing to launch 
> at all, if not).
> 
> XProtect is basically a blacklist that applications are checked against. If 
> an application matches, it’s considered malware. The blacklist is a local 
> file on your system but updated by Apple.
> 
> These things operate very differently than having low-level system calls 
> potentially contact Apple’s servers every time a process is launched on your 
> system (or, as it appears to be the case on my system, when processes are 
> accessing certain locations in the file system).
> ___
> 
> Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)
> 
> Please do not post admin requests or moderator comments to the list.
> Contact the moderators at cocoa-dev-admins(at)lists.apple.com
> 
> Help/Unsubscribe/Update your Subscription:
> https://lists.apple.com/mailman/options/cocoa-dev/saagar%40saagarjha.com
> 
> This email sent to saa...@saagarjha.com

___

Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)

Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com

Help/Unsubscribe/Update your Subscription:
https://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com

This email sent to arch...@mail-archive.com

Re: Performance issue on macOS 10.15 obtaining display name for ~/Desktop, ~/Documents, and ~/Downloads

[Allan Odgaard via Cocoa-dev]

On 24 Apr 2020, at 9:57, Rob Petrovec wrote:

Also weird, why would it phone home for a shell script which has 
neither been stapled nor even code-signed?
I think you answered the question just then…  a "shell script which 
has neither been stapled nor even code-signed”.  Google XProtect & 
Gatekeeper.


GateKeeper is basically Safari adding a quarantine flag (via extended 
attributes) to files downloaded form the internet, and then having 
Finder check this flag, throwing up a dialog if the flag is set, and 
recently, also checking if the code signature is from a verified 
developer (possibly refusing to launch at all, if not).


XProtect is basically a blacklist that applications are checked against. 
If an application matches, it’s considered malware. The blacklist is a 
local file on your system but updated by Apple.


These things operate very differently than having low-level system calls 
potentially contact Apple’s servers every time a process is launched 
on your system (or, as it appears to be the case on my system, when 
processes are accessing certain locations in the file system).

___

Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)

Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com

Help/Unsubscribe/Update your Subscription:
https://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com

This email sent to arch...@mail-archive.com

Re: Performance issue on macOS 10.15 obtaining display name for ~/Desktop, ~/Documents, and ~/Downloads

[Allan Odgaard via Cocoa-dev]

On 24 Apr 2020, at 9:51, Gary L. Wade wrote:

Have you tried a speed check with just iCloud turned off but internet 
on?


I have tried with iCloud disabled, internet disabled, and SIP disabled.

Only the latter two removes the delay. Also, the issue happens for 
~/Downloads which is not an iCloud folder.


Furthermore, the stack dump seems fairly clear about what goes on.

For the records, I’m reposting my findings here with stack dump.

This is actually from Transmission.app, in Preferences you can configure 
download folders, so I selected ~/Documents, ~/Downloads, and ~/Desktop, 
which are the 3 “protected” folders. I then ran a spindump when 
opening Preferences, which has to obtain display name and icon for these 
folders (to show in the UI), which results in 620 ms spent in 
NSWorkspace’s iconForFile: (getattrlist).


	62   -[NSWorkspace iconForFile:] + 80 (AppKit + 3229058) 
[0x7fff32ee1582]
	62   -[NSWorkspace _iconForURL:] + 100 (AppKit + 3229205) 
[0x7fff32ee1615]

62   _GetIconRefFromURL + 26 (LaunchServices + 233122) [0x7fff370edea2]
	62   BindingManager::CreateWithURL(__CFURL const*, bool) + 28 
(LaunchServices + 233204) [0x7fff370edef4]
	62   BindingBlueprint::BindingBlueprint(__CFURL const*) + 78 
(LaunchServices + 233350) [0x7fff370edf86]
	62   BindingBlueprint::copyURLProperties(__CFURL const*) + 50 
(LaunchServices + 233496) [0x7fff370ee018]
	62   CFURLCopyResourcePropertiesForKeys + 111 (CoreFoundation + 555324) 
[0x7fff3599493c]
	62   _FSURLCopyResourcePropertiesForKeysInternal(__CFURL const*, 
__CFArray const*, void*, __CFError**, unsigned char) + 1110 
(CoreServicesInternal + 60765) [0x7fff4ecb0d5d]
	62   prepareValuesForBitmap(__CFURL const*, __FileCache*, 
_FilePropertyBitmap*, __CFError**) + 363 (CoreServicesInternal + 17604) 
[0x7fff4eca64c4]
	62   __getattrlist + 10 (libsystem_kernel.dylib + 5746) 
[0x7fff6fa19672]

*62  hndl_unix_scall64 + 22 (kernel + 819718) [0xff80002c8206]
*62  unix_syscall64 + 647 (kernel + 7894519) [0xff80009875f7]
*62  getattrlist + 136 (kernel + 3512472) [0xff8000559898]
*62  ??? (kernel + 3512820) [0xff80005599f4]
*62  ??? (kernel + 3501529) [0xff8000556dd9]
*62  ??? (kernel + 3490836) [0xff8000554414]
	*62  mac_vnode_check_access + 154 (kernel + 9093082) 
[0xff8000aabfda]
	*62  hook_vnode_check_access + 167 (Sandbox + 39552) 
[0xff7f823a7a80]

*62  sb_evaluate + 5004 (Sandbox + 67658) [0xff7f823ae84a]
	*62  approval_response_wait + 142 (Sandbox + 154851) 
[0xff7f823c3ce3]
	*62  __WAITING_ON_APPROVAL_FROM_SANDBOXD__ + 23 (Sandbox + 155134) 
[0xff7f823c3dfe]

*62  ??? (kernel + 6855402) [0xff8000889aea]
*62  thread_block_reason + 175 (kernel + 1317935) [0xff8000341c2f]
*62  ??? (kernel + 1324017) [0xff80003433f1]
	*62  machine_switch_context + 200 (kernel + 2388456) 
[0xff80004471e8]


Looking at sandboxd, it has 3 queues spending respectively 150, 160, and 
310 ms in TCCAccessPreflightWithAuditToken (so total time spent there is 
620 ms), which I would guess is processing Transmission’s request for 
reading the extended attributes for the 3 paths.


But sandboxd itself does tccd_send_message, so it seems to call upon the 
tccd daemon.


There are two tccd instances running on my system, each have two queues 
with 150 ms spent in SecCodeCheckValidityWithErrors (so a total of 600 
ms spent in tccd).


This code though again seems rely on another process by calling: 
securityd_send_sync_and_do (which does XPC).


But I can’t find out which process it communicates with. I don’t 
know if this could be launched-on-demand and therefore not part of the 
spindump report?


But to summarize, first call of getattrlist/getxattr for a protected 
folder seems to trigger application signature validation (makes sense), 
which, at least on my system, appears to involve network access, 
reminiscent of what goes on with execve().


Given the above, I don’t think anyone can dispute that this is what 
goes on, and it would be extremely strange if this is not “by 
design”.


What might be a bug is that the check does network access, but you 
don’t just put such code in by mistake, so obviously there must be 
certain times where it needs network access, and as demonstrated with 
execve(), Apple is not shy of making low-level system APIs contact 
Apple’s servers as part of their SIP.


The million dollar question is now: What are the conditions under which 
it must have network access? and/or why does it seem to trigger all the 
time on my system?

___

Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)

Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com

Help/Unsubscribe/Update your Subscription:
https://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com

This email sent to arch...@mail-ar

Re: Performance issue on macOS 10.15 obtaining display name for ~/Desktop, ~/Documents, and ~/Downloads

[Marco S Hyman via Cocoa-dev]

>> Also weird, why would it phone home for a shell script which has neither 
>> been stapled nor even code-signed?
>   I think you answered the question just then…  a "shell script which has 
> neither been stapled nor even code-signed”.  Google XProtect & Gatekeeper.

1) The executable part of a shell script is the shell.
2) The shell is signed by apple as can be seen from the output of, for example, 
   codesign --verify --display --verbose=4 /bin/sh
   or bash, or zsh, etc.
3) network monitoring shows no traffic due to running a shell script on my 
machine

We’re getting pretty far afield from the original subject, no?
___

Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)

Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com

Help/Unsubscribe/Update your Subscription:
https://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com

This email sent to arch...@mail-archive.com

Re: Performance issue on macOS 10.15 obtaining display name for ~/Desktop, ~/Documents, and ~/Downloads

[Rob Petrovec via Cocoa-dev]

> On Apr 23, 2020, at 8:35 PM, Allan Odgaard via Cocoa-dev 
>  wrote:
> 
> On 24 Apr 2020, at 2:28, Gabriel Zachmann via Cocoa-dev wrote:
> 
>>> I believe that is why you are supposed to staple notarization tickets to 
>>> your apps.
>> Then, why would it "phone home" in case there is an internet connection?
> 
> Also weird, why would it phone home for a shell script which has neither been 
> stapled nor even code-signed?
I think you answered the question just then…  a "shell script which has 
neither been stapled nor even code-signed”.  Google XProtect & Gatekeeper.

—Rob




___

Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)

Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com

Help/Unsubscribe/Update your Subscription:
https://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com

This email sent to arch...@mail-archive.com

Re: Performance issue on macOS 10.15 obtaining display name for ~/Desktop, ~/Documents, and ~/Downloads

[Gary L. Wade via Cocoa-dev]

Have you tried a speed check with just iCloud turned off but internet on?
--
Gary
___

Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)

Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com

Help/Unsubscribe/Update your Subscription:
https://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com

This email sent to arch...@mail-archive.com

Re: Performance issue on macOS 10.15 obtaining display name for ~/Desktop, ~/Documents, and ~/Downloads

[Allan Odgaard via Cocoa-dev]

On 24 Apr 2020, at 2:28, Gabriel Zachmann via Cocoa-dev wrote:

I believe that is why you are supposed to staple notarization tickets 
to your apps.
Then, why would it "phone home" in case there is an internet 
connection?


Also weird, why would it phone home for a shell script which has neither 
been stapled nor even code-signed?


Actually, weird is probably the wrong word: This is concerning!
___

Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)

Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com

Help/Unsubscribe/Update your Subscription:
https://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com

This email sent to arch...@mail-archive.com

Re: Performance issue on macOS 10.15 obtaining display name for ~/Desktop, ~/Documents, and ~/Downloads

[Rob Petrovec via Cocoa-dev]

> On Apr 23, 2020, at 7:30 PM, Allan Odgaard via Cocoa-dev 
>  wrote:
> 
> On 24 Apr 2020, at 2:18, Rob Petrovec wrote:
> 
>> I get a 1 second time for the first run and then a much quicker time for the 
>> second.  I did some sampling and the longer time due to is Apple’s check for 
>> malware on first run of a process.  This is a known, documented and 
>> advertised behavior.
> 
> I would be very interested in documentation about what low-level APIs (like 
> execve) do malware checks (network access), under which conditions they are 
> performed, what servers are contacted, and what sort of caching of good/bad 
> results are done.
> 
> Is any of that documented?
Here is some from a quick Google search.  I think the feature in 
question is XProtect.  With a little more time I could probably find more 
in-depth docs.

https://www.apple.com/macos/security/  See the 'Protection starts at 
the core’ section

https://support.apple.com/guide/mac-help/protect-your-mac-from-malware-mh40596/mac

https://www.howtogeek.com/217043/xprotect-explained-how-your-macs-built-in-anti-malware-works/


> There is also blacklisting going on: I can get an executable locally 
> blacklisted which will cause it to terminate instantly when executed. This 
> seems to be about some run-time code signature validation, and when it 
> happens, it appears to be the inode that gets blacklisted until next reboot, 
> but more info about this would be nice.
Depending on where the app is being terminated, I would suspect it is 
the same “Allow apps downloaded from” feature in the General section of the 
Security & Privacy Pref pane.


>> […] So I don’t think this test is analogous to your initial issue of a delay 
>> opening a file every time.
> 
> I said I get a similar delay the first time my app obtains URL properties¹ 
> for ~/Desktop, ~/Documents, and, ~/Downloads, and I included sample code for 
> this issue.
Sorry I forgot what your initial problem was.  However, my statement 
still applies.  Getting the localized string for a folder is completely 
different then the launching app.


> Perhaps you would be willing to add this sample code to a GUI application and 
> see if you can reproduce? I re-attached it below, and have the result written 
> to /tmp/duration.txt so you don’t have to fiddle with capturing log output.
I tried it (although I changed it from writing a file to disk to 
NSLog() and it spit out:

default 19:58:53.343324-0600Test FooDuration 0.003

—Rob


___

Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)

Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com

Help/Unsubscribe/Update your Subscription:
https://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com

This email sent to arch...@mail-archive.com

Re: Performance issue on macOS 10.15 obtaining display name for ~/Desktop, ~/Documents, and ~/Downloads

[Allan Odgaard via Cocoa-dev]

On 24 Apr 2020, at 2:18, Rob Petrovec wrote:

I get a 1 second time for the first run and then a much quicker time 
for the second.  I did some sampling and the longer time due to is 
Apple’s check for malware on first run of a process.  This is a 
known, documented and advertised behavior.


I would be very interested in documentation about what low-level APIs 
(like execve) do malware checks (network access), under which conditions 
they are performed, what servers are contacted, and what sort of caching 
of good/bad results are done.


Is any of that documented?

There is also blacklisting going on: I can get an executable locally 
blacklisted which will cause it to terminate instantly when executed. 
This seems to be about some run-time code signature validation, and when 
it happens, it appears to be the inode that gets blacklisted until next 
reboot, but more info about this would be nice.


[…] So I don’t think this test is analogous to your initial issue 
of a delay opening a file every time.


I said I get a similar delay the first time my app obtains URL 
properties¹ for ~/Desktop, ~/Documents, and, ~/Downloads, and I 
included sample code for this issue.


The actual delay appears to be in getxattr() which communicates with 
sandboxd, and the delay disappears when disabling WiFi, hence why it 
appears to also be a “phone home” issue like execve() above. But 
where execve() only does it on “first launch”, this check seems to 
be performed on first call after each new launch (for each different 
“protected” file path).


Perhaps you would be willing to add this sample code to a GUI 
application and see if you can reproduce? I re-attached it below, and 
have the result written to /tmp/duration.txt so you don’t have to 
fiddle with capturing log output.


Please note, it should be stand-alone GUI app launched from Finder, as 
the delay does not (always) happen when launched from the terminal.


¹ I later said I have seen many delays on my system since upgrading to 
macOS 10.15: But so far, I have only tracked down reproducible issues 
with execve() and getxattr(), but those do not explain all my delays, so 
there might be more low-level API that does “phone home”, I’m 
still in the data collection phase.


--8<--

void test ()
{
   NSTimeInterval startTime = NSDate.timeIntervalSinceReferenceDate;

   NSArray* urls = @[
	  [NSFileManager.defaultManager URLForDirectory:NSDesktopDirectory 
inDomain:NSUserDomainMask appropriateForURL:nil create:NO error:nil],
	  [NSFileManager.defaultManager URLForDirectory:NSDocumentDirectory 
inDomain:NSUserDomainMask appropriateForURL:nil create:NO error:nil],
	  [NSFileManager.defaultManager 
URLForDirectory:NSDownloadsDirectory inDomain:NSUserDomainMask 
appropriateForURL:nil create:NO error:nil],

   ];

   NSString* localizedName;
   for(NSURL* url in urls)
	  [url getResourceValue:&localizedName forKey:NSURLLocalizedNameKey 
error:nil];


	   NSString* duration = [NSString stringWithFormat:@"Duration %.03f", 
NSDate.timeIntervalSinceReferenceDate - startTime];
	   [duration writeToFile:@"/tmp/duration.txt" atomically:NO 
encoding:NSUTF8StringEncoding error:nil];

}
___

Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)

Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com

Help/Unsubscribe/Update your Subscription:
https://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com

This email sent to arch...@mail-archive.com

Re: Performance issue on macOS 10.15 obtaining display name for ~/Desktop, ~/Documents, and ~/Downloads

[Gabriel Zachmann via Cocoa-dev]

> 
> I believe that is why you are supposed to staple notarization tickets to your 
> apps.
> 

Then, why would it "phone home" in case there is an internet connection?


Best regards, Gabriel


___

Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)

Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com

Help/Unsubscribe/Update your Subscription:
https://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com

This email sent to arch...@mail-archive.com

Re: Performance issue on macOS 10.15 obtaining display name for ~/Desktop, ~/Documents, and ~/Downloads

[Saagar Jha via Cocoa-dev]

I believe that is why you are supposed to staple notarization tickets to your 
apps.

Saagar Jha

> On Apr 23, 2020, at 12:12, Gabriel Zachmann via Cocoa-dev 
>  wrote:
> 
>> 
>> It appears the problem is not with a local service, but that Apple 
>> actually ?phones home? when a program asks for display name.
>> 
>> I don?t know if this is common knowledge, but with notarization, Apple 
>> now validates executables on your system before they are executed, and 
>> it does so in calls like execve(), where it will actually stall 
>> execution, contact Apple?s servers, and then proceed once the 
>> executable got validated.
> 
> 
> I am just curious: what does it when there is *no* internet connection?
> (Suppose, someone downloads the app, then disconnects from internet, then 
> executes it;
> or copies the app via USB drive to the machine without internet connection.)
> And what is it *supposed* to do in that case?
> 
> 
> Best regards, Gabriel
> 
> 
> ___
> 
> Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)
> 
> Please do not post admin requests or moderator comments to the list.
> Contact the moderators at cocoa-dev-admins(at)lists.apple.com
> 
> Help/Unsubscribe/Update your Subscription:
> https://lists.apple.com/mailman/options/cocoa-dev/saagar%40saagarjha.com
> 
> This email sent to saa...@saagarjha.com

___

Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)

Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com

Help/Unsubscribe/Update your Subscription:
https://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com

This email sent to arch...@mail-archive.com

Re: Performance issue on macOS 10.15 obtaining display name for ~/Desktop, ~/Documents, and ~/Downloads

[Rob Petrovec via Cocoa-dev]

> On Apr 23, 2020, at 9:10 AM, Allan Odgaard via Cocoa-dev 
>  wrote:
> 
> On 23 Apr 2020, at 21:15, Rob Petrovec wrote:
> 
>> If what you say is correct then everyone would be seeing a delay since most 
>> people don’t have blazing fast internet connections.  I do not think this is 
>> the normal behavior.
> 
> Please try run this in a terminal and report the times:
> 
>rm -f /tmp/test.sh && echo $'#!/bin/sh\necho hello' > /tmp/test.sh && 
> chmod a+x /tmp/test.sh && time /tmp/test.sh && time /tmp/test.sh
> 
> For this particular issue, it appears the lookup is cached by inode, so only 
> first run should take > 50ms, where second one would be < 5ms.
> 
> Then maybe also try it with Apple’s Network Link Conditioner and set it to 
> simulate lousy internet, and try it also with WiFi disabled.
> 
> I have seen the issue on multiple systems, so I do not think this is limited 
> to my system, but more data would be great.
I get a 1 second time for the first run and then a much quicker time 
for the second.  I did some sampling and the longer time due to is Apple’s 
check for malware on first run of a process.  This is a known, documented and 
advertised behavior.  It is a one time cost and only effects applications, not 
regular files.  For a developer though, yeah you get the cost after ever 
rebuild.  So I don’t think this test is analogous to your initial issue of a 
delay opening a file every time.



>> Sending an email to a developer mailing list doesn’t count.  Just sayin’...
> 
> My email was to ask for help, i.e. whether others have seen this issue, can 
> reproduce, have any idea what could cause this, etc.
> 
> Of course I am aware that this is not a place to report bugs to Apple…
So did you file a bug?  Would you mind posting the bug number?  Thanks.

—Rob

___

Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)

Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com

Help/Unsubscribe/Update your Subscription:
https://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com

This email sent to arch...@mail-archive.com

Re: Performance issue on macOS 10.15 obtaining display name for ~/Desktop, ~/Documents, and ~/Downloads

[Gabriel Zachmann via Cocoa-dev]

> 
> It appears the problem is not with a local service, but that Apple 
> actually ?phones home? when a program asks for display name.
> 
> I don?t know if this is common knowledge, but with notarization, Apple 
> now validates executables on your system before they are executed, and 
> it does so in calls like execve(), where it will actually stall 
> execution, contact Apple?s servers, and then proceed once the 
> executable got validated.


I am just curious: what does it when there is *no* internet connection?
(Suppose, someone downloads the app, then disconnects from internet, then 
executes it;
or copies the app via USB drive to the machine without internet connection.)
And what is it *supposed* to do in that case?


Best regards, Gabriel


___

Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)

Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com

Help/Unsubscribe/Update your Subscription:
https://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com

This email sent to arch...@mail-archive.com

Re: Performance issue on macOS 10.15 obtaining display name for ~/Desktop, ~/Documents, and ~/Downloads

[Allan Odgaard via Cocoa-dev]

On 23 Apr 2020, at 21:15, Rob Petrovec wrote:

If what you say is correct then everyone would be seeing a delay since 
most people don’t have blazing fast internet connections.  I do not 
think this is the normal behavior.


Please try run this in a terminal and report the times:

rm -f /tmp/test.sh && echo $'#!/bin/sh\necho hello' > /tmp/test.sh 
&& chmod a+x /tmp/test.sh && time /tmp/test.sh && time /tmp/test.sh


For this particular issue, it appears the lookup is cached by inode, so 
only first run should take > 50ms, where second one would be < 5ms.


Then maybe also try it with Apple’s Network Link Conditioner and set 
it to simulate lousy internet, and try it also with WiFi disabled.


I have seen the issue on multiple systems, so I do not think this is 
limited to my system, but more data would be great.


Sending an email to a developer mailing list doesn’t count.  Just 
sayin’...


My email was to ask for help, i.e. whether others have seen this issue, 
can reproduce, have any idea what could cause this, etc.


Of course I am aware that this is not a place to report bugs to Apple…

btw: today I took the drastic step of disabling SIP: My system now feels 
*much* better. Maybe some of the delays are bugs, but I’m quite sure 
the execve() issue is by design, it’s just a pretty developer 
unfriendly design, because the caching only lasts until the next 
rebuild, or incase of a script, the next (atomic) save, or incase of 
dynamically created helper scripts: no caching.

___

Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)

Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com

Help/Unsubscribe/Update your Subscription:
https://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com

This email sent to arch...@mail-archive.com

Re: Performance issue on macOS 10.15 obtaining display name for ~/Desktop, ~/Documents, and ~/Downloads

[Rob Petrovec via Cocoa-dev]

If what you say is correct then everyone would be seeing a delay since most 
people don’t have blazing fast internet connections.  I do not think this is 
the normal behavior.  I think it is specific to your system, otherwise there 
would be TONS of people complaining about slowness.  A couple second delay 
opening a file or app like you describe would be all over the internet.  I 
don’t remember from your previous emails on this subject, but did you try 
creating a new partition and installing a fresh OS with a brand new user on it 
(nothing migrated) to see if the problem reproduced?  That would be an 
interesting data point.

Either way, did you file a bug with a sysdiagnose taken during the delay?  If 
so, do you have the bug number?  Something like this doesn’t get fixed if you 
don’t report it.  Sending an email to a developer mailing list doesn’t count.  
Just sayin’...

—Rob


> On Apr 22, 2020, at 11:16 PM, Allan Odgaard via Cocoa-dev 
>  wrote:
> 
> On 20 Apr 2020, at 0:11, Allan Odgaard via Cocoa-dev wrote:
> 
>> Unfortunately though I can’t figure out *what* the problem is; running 
>> `tccutil reset All` (and rebooting) did not fix it.
> 
> It appears the problem is not with a local service, but that Apple actually 
> “phones home” when a program asks for display name.
> 
> I don’t know if this is common knowledge, but with notarization, Apple now 
> validates executables on your system before they are executed, and it does so 
> in calls like execve(), where it will actually stall execution, contact 
> Apple’s servers, and then proceed once the executable got validated.
> 
> I *thought* this was the only place it did it, and that the result got cached 
> (based on inode).
> 
> But it seems Apple added this to other places, because since I have upgraded 
> to macOS 10.15, I see *many* delays.
> 
> This is because I am currently in South East Asia where the connection to 
> Apple’s servers is not good.
> 
> For example I have a script that takes a video file as argument, it launches 
> VLC with this video file, and then deletes the file when VLC terminates.
> 
> It can take more than 5 seconds just until VLC is launched, and then VLC will 
> be “thinking” for another 5 seconds, before the video actually starts.
> 
> Today the delays were extra bad, so it was easy to reproduce the VLC issue, 
> obtaining display name (which today took 7 seconds for 3 names), and a few 
> other things.
> 
> Now, if I disable internet, no delays at all!!!
> 
> Enable it again, and all the delays are back.
> 
> It is so utterly frustrating that Apple is not only going down this path of 
> locking down our machines, but they do it in ways that are so crippling for 
> our productivity :(
> ___
> 
> Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)
> 
> Please do not post admin requests or moderator comments to the list.
> Contact the moderators at cocoa-dev-admins(at)lists.apple.com
> 
> Help/Unsubscribe/Update your Subscription:
> https://lists.apple.com/mailman/options/cocoa-dev/petrock%40mac.com
> 
> This email sent to petr...@mac.com

___

Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)

Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com

Help/Unsubscribe/Update your Subscription:
https://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com

This email sent to arch...@mail-archive.com

Re: Performance issue on macOS 10.15 obtaining display name for ~/Desktop, ~/Documents, and ~/Downloads

[Allan Odgaard via Cocoa-dev]

On 20 Apr 2020, at 0:11, Allan Odgaard via Cocoa-dev wrote:

Unfortunately though I can’t figure out *what* the problem is; 
running `tccutil reset All` (and rebooting) did not fix it.


It appears the problem is not with a local service, but that Apple 
actually “phones home” when a program asks for display name.


I don’t know if this is common knowledge, but with notarization, Apple 
now validates executables on your system before they are executed, and 
it does so in calls like execve(), where it will actually stall 
execution, contact Apple’s servers, and then proceed once the 
executable got validated.


I *thought* this was the only place it did it, and that the result got 
cached (based on inode).


But it seems Apple added this to other places, because since I have 
upgraded to macOS 10.15, I see *many* delays.


This is because I am currently in South East Asia where the connection 
to Apple’s servers is not good.


For example I have a script that takes a video file as argument, it 
launches VLC with this video file, and then deletes the file when VLC 
terminates.


It can take more than 5 seconds just until VLC is launched, and then VLC 
will be “thinking” for another 5 seconds, before the video actually 
starts.


Today the delays were extra bad, so it was easy to reproduce the VLC 
issue, obtaining display name (which today took 7 seconds for 3 names), 
and a few other things.


Now, if I disable internet, no delays at all!!!

Enable it again, and all the delays are back.

It is so utterly frustrating that Apple is not only going down this path 
of locking down our machines, but they do it in ways that are so 
crippling for our productivity :(

___

Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)

Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com

Help/Unsubscribe/Update your Subscription:
https://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com

This email sent to arch...@mail-archive.com

Re: Performance issue on macOS 10.15 obtaining display name for ~/Desktop, ~/Documents, and ~/Downloads

[Gary L. Wade via Cocoa-dev]

Regardless of whatever workaround you find, I would second Rob’s suggestion to 
go ahead and file a bug with a sysdiagnose and/or spindump along with a sample 
app that reproduces it.  This isn’t expected behavior, and the teams at Apple 
are still working and would be very interested in seeing this.  Once filed, if 
you do find a workaround, you can append that to the bug.
--
Gary L. Wade
http://www.garywade.com/ 

> On Apr 19, 2020, at 10:58 AM, Allan Odgaard via Cocoa-dev 
>  wrote:
> 
> On 20 Apr 2020, at 0:37, Rob Petrovec wrote:
> 
>> >> I think you are right about this being a permission / “sandbox” issue, 
>> >> because the 3 folders in question are all folders that macOS 10.15 now 
>> >> require special permission to read (even though in my case, I just 
>> >> request their display name).
>>  Yes, this is because of iCloud.  Log out of iCloud and it should go 
>> away as I suggested in my first reply.  You could also grant your app full 
>> disk access and that might ‘fix’ it for you too.  But your users would 
>> probably not want to do that.
> 
> As written previously, there is no delay when running from terminal, and 
> ~/Downloads is also causing a delay, which is not stored on iCloud.
> 
> So this very much seems to be about Apple’s file system access policies where 
> permission must be obtained to access certain folders.
> 
> But as mentioned, I have given my app full disk access, reset permissions, 
> and I have even signed and notarized the app: Still no fix for the delay.
> 
> To indulge you, I have now also tried to log out of iCloud: Still a delay.
> 
> Now I would very much appreciate if someone would try the code I included in 
> my original email to see if this is a general problem, or something limited 
> to my system.
> 
> As I think I mentioned, I am running a fairly clean install of macOS 10.15.

___

Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)

Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com

Help/Unsubscribe/Update your Subscription:
https://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com

This email sent to arch...@mail-archive.com

Re: Performance issue on macOS 10.15 obtaining display name for ~/Desktop, ~/Documents, and ~/Downloads

[Allan Odgaard via Cocoa-dev]

On 20 Apr 2020, at 0:37, Rob Petrovec wrote:

>> I think you are right about this being a permission / “sandbox” 
issue, because the 3 folders in question are all folders that macOS 
10.15 now require special permission to read (even though in my case, 
I just request their display name).
	Yes, this is because of iCloud.  Log out of iCloud and it should go 
away as I suggested in my first reply.  You could also grant your app 
full disk access and that might ‘fix’ it for you too.  But your 
users would probably not want to do that.


As written previously, there is no delay when running from terminal, and 
~/Downloads is also causing a delay, which is not stored on iCloud.


So this very much seems to be about Apple’s file system access 
policies where permission must be obtained to access certain folders.


But as mentioned, I have given my app full disk access, reset 
permissions, and I have even signed and notarized the app: Still no fix 
for the delay.


To indulge you, I have now also tried to log out of iCloud: Still a 
delay.


Now I would very much appreciate if someone would try the code I 
included in my original email to see if this is a general problem, or 
something limited to my system.


As I think I mentioned, I am running a fairly clean install of macOS 
10.15.

___

Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)

Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com

Help/Unsubscribe/Update your Subscription:
https://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com

This email sent to arch...@mail-archive.com

Re: Performance issue on macOS 10.15 obtaining display name for ~/Desktop, ~/Documents, and ~/Downloads

[Rob Petrovec via Cocoa-dev]

>> I think you are right about this being a permission / “sandbox” issue, 
>> because the 3 folders in question are all folders that macOS 10.15 now 
>> require special permission to read (even though in my case, I just request 
>> their display name).
Yes, this is because of iCloud.  Log out of iCloud and it should go 
away as I suggested in my first reply.  You could also grant your app full disk 
access and that might ‘fix’ it for you too.  But your users would probably not 
want to do that.


>> That’s 181 milliseconds spent __WAITING_ON_APPROVAL_FROM_SANDBOXD__ in a 
>> non-sandboxed app obtaining display name for 3 folders in the user’s home 
>> directory.
Its actually worse then that.  At the top of the spindump file it says 
was the numbers represent.  Something like:

Duration: 10.00s
Steps:1001 (10ms sampling interval)

So 181 actually represents 1810ms, or 1.81 seconds.  So this is a sandbox 
performance issue.  Please file a bug and include a sysdiagnose as I suggested 
in my first reply.

—Rob



> On Apr 19, 2020, at 11:11 AM, Allan Odgaard via Cocoa-dev 
>  wrote:
> 
> On 19 Apr 2020, at 22:54, David M. Cotter wrote:
> 
>> i have discovered it may have to do with permissions / entitlements that 
>> have been granted the app by the user, and that resetting all perms to 
>> default will "fix" the problem
> 
> I think you are right about this being a permission / “sandbox” issue, 
> because the 3 folders in question are all folders that macOS 10.15 now 
> require special permission to read (even though in my case, I just request 
> their display name).
> 
> Furthermore, it could explain why there is no delay when launched from 
> terminal, as the process will inherit the entitlements of the terminal, which 
> may have some special entitlements for the full disk access.
> 
> Unfortunately though I can’t figure out *what* the problem is; running 
> `tccutil reset All` (and rebooting) did not fix it.
> 
> I have tried the code in question in a signed and notarized app that has been 
> granted full disk access, yet it still has the delay.
> 
> A spindump does indicate it is a “sandbox” issue:
> 
>   *182   getattrlist + 136 (kernel + 3512472) [0xff8000559898]
> *182   ??? (kernel + 3512820) [0xff80005599f4]
>   *181   ??? (kernel + 3501529) [0xff8000556dd9]
> *181   ??? (kernel + 3490836) [0xff8000554414]
>   *181   mac_vnode_check_access + 154 (kernel + 9093082) 
> [0xff8000aabfda]
> *181   hook_vnode_check_access + 167 (Sandbox + 39552) 
> [0xff7f823a7a80]
>   *181   sb_evaluate + 5004 (Sandbox + 67658) [0xff7f823ae84a]
> *181   approval_response_wait + 142 (Sandbox + 154851) 
> [0xff7f823c3ce3]
>   *181   __WAITING_ON_APPROVAL_FROM_SANDBOXD__ + 23 (Sandbox 
> + 155134) [0xff7f823c3dfe]
> *181   ??? (kernel + 6855402) [0xff8000889aea]
>   *181   thread_block_reason + 175 (kernel + 1317935) 
> [0xff8000341c2f]
> *181   ??? (kernel + 1324017) [0xff80003433f1]
>   *181   machine_switch_context + 200 (kernel + 
> 2388456) [0xff80004471e8]
> 
> That’s 181 milliseconds spent __WAITING_ON_APPROVAL_FROM_SANDBOXD__ in a 
> non-sandboxed app obtaining display name for 3 folders in the user’s home 
> directory.
> ___
> 
> Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)
> 
> Please do not post admin requests or moderator comments to the list.
> Contact the moderators at cocoa-dev-admins(at)lists.apple.com
> 
> Help/Unsubscribe/Update your Subscription:
> https://lists.apple.com/mailman/options/cocoa-dev/petrock%40mac.com
> 
> This email sent to petr...@mac.com

___

Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)

Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com

Help/Unsubscribe/Update your Subscription:
https://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com

This email sent to arch...@mail-archive.com

Re: Performance issue on macOS 10.15 obtaining display name for ~/Desktop, ~/Documents, and ~/Downloads

[Allan Odgaard via Cocoa-dev]

On 19 Apr 2020, at 22:54, David M. Cotter wrote:

i have discovered it may have to do with permissions / entitlements 
that have been granted the app by the user, and that resetting all 
perms to default will "fix" the problem


I think you are right about this being a permission / “sandbox” 
issue, because the 3 folders in question are all folders that macOS 
10.15 now require special permission to read (even though in my case, I 
just request their display name).


Furthermore, it could explain why there is no delay when launched from 
terminal, as the process will inherit the entitlements of the terminal, 
which may have some special entitlements for the full disk access.


Unfortunately though I can’t figure out *what* the problem is; running 
`tccutil reset All` (and rebooting) did not fix it.


I have tried the code in question in a signed and notarized app that has 
been granted full disk access, yet it still has the delay.


A spindump does indicate it is a “sandbox” issue:

   *182   getattrlist + 136 (kernel + 3512472) [0xff8000559898]
 *182   ??? (kernel + 3512820) [0xff80005599f4]
   *181   ??? (kernel + 3501529) [0xff8000556dd9]
 *181   ??? (kernel + 3490836) [0xff8000554414]
   *181   mac_vnode_check_access + 154 (kernel + 9093082) 
[0xff8000aabfda]
 *181   hook_vnode_check_access + 167 (Sandbox + 39552) 
[0xff7f823a7a80]
   *181   sb_evaluate + 5004 (Sandbox + 67658) 
[0xff7f823ae84a]
 *181   approval_response_wait + 142 (Sandbox + 154851) 
[0xff7f823c3ce3]
   *181   __WAITING_ON_APPROVAL_FROM_SANDBOXD__ + 23 
(Sandbox + 155134) [0xff7f823c3dfe]

 *181   ??? (kernel + 6855402) [0xff8000889aea]
   *181   thread_block_reason + 175 (kernel + 
1317935) [0xff8000341c2f]
 *181   ??? (kernel + 1324017) 
[0xff80003433f1]
   *181   machine_switch_context + 200 (kernel 
+ 2388456) [0xff80004471e8]


That’s 181 milliseconds spent __WAITING_ON_APPROVAL_FROM_SANDBOXD__ in 
a non-sandboxed app obtaining display name for 3 folders in the user’s 
home directory.

___

Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)

Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com

Help/Unsubscribe/Update your Subscription:
https://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com

This email sent to arch...@mail-archive.com

Re: Performance issue on macOS 10.15 obtaining display name for ~/Desktop, ~/Documents, and ~/Downloads

[Rob Petrovec via Cocoa-dev]

I assume you have iCloud enabled.  If so, these three folders are ’special’.  
Try logging out of iCloud & rebooting and see if the problem persists.

I would also recommend filing a bug with Apple and include a sysdiagnose taken 
while the problem was reproducing (sudo sysdiagnose).  Or at least a spindump 
(sudo spindump) while the problem is reproducing.

—Rob



> On Apr 19, 2020, at 9:54 AM, David M. Cotter via Cocoa-dev 
>  wrote:
> 
> this may be difficult for other to repro
> 
> i have discovered it may have to do with permissions / entitlements that have 
> been granted the app by the user, and that resetting all perms to default 
> will "fix" the problem
> 
> in the terminal, do this:
>> tccutil reset All
> i expect that after that, your delay will disappear.
> 
> however, you'll have to manually re-authorize all apps in the system 
> prefs->security & privacy->privacy tab
> 
> i realize this is a "sweep the problem under the rug" rather than "fix the 
> problem" approach
> but hopefully this should give someone else a clue as to WHY this is 
> happening?
> 
> -dave
> 
>> On Apr 19, 2020, at 8:08 AM, Allan Odgaard via Cocoa-dev 
>>  wrote:
>> 
>> Starting with macOS 10.15 I have noticed that obtaining 
>> NSURLLocalizedNameKey, NSURLTagNamesKey, and even calling getxattr(), can 
>> cause a significant delay.
>> 
>> The code below, which gets NSURLLocalizedNameKey for 3 folders, takes 1.9 
>> seconds to execute on my system.
>> 
>> It appears though that it is only these 3 specific folders that has the 
>> problem.
>> 
>> Furthermore, the problem only happens when I include the code below in a GUI 
>> application which is launched via launch services. If I run the executable 
>> directly from a terminal then the code is near instant.
>> 
>> I tried to sample the code, and it appears that it calls out to some OS 
>> service/daemon. Presumably it never gets a response, and a timeout is 
>> triggered.
>> 
>> I wonder if anyone else has come across this? And maybe have discovered more 
>> info?
>> 
>> This is btw macOS 10.15.4 and a fairly stock system (new computer without 
>> migrating old data), and I noticed the problem some time ago, so it 
>> definitely existed before 10.15.4, but I never saw anything like this before 
>> upgrading to Catelina.
>> 
>>   NSTimeInterval startTime = NSDate.timeIntervalSinceReferenceDate;
>> 
>>   NSArray* urls = @[]
>>   [NSFileManager.defaultManager URLForDirectory:NSDesktopDirectory 
>> inDomain:NSUserDomainMask appropriateForURL:nil create:NO error:nil],,
>>   [NSFileManager.defaultManager URLForDirectory:NSDocumentDirectory 
>> inDomain:NSUserDomainMask appropriateForURL:nil create:NO error:nil]
>>   [NSFileManager.defaultManager URLForDirectory:NSDownloadsDirectory 
>> inDomain:NSUserDomainMask appropriateForURL:nil create:NO error:nil,
>>   ];
>> 
>>   NSString* localizedName;
>>   for(NSURL* url in urls)
>>   [url getResourceValue:&localizedName forKey:NSURLLocalizedNameKey 
>> error:nil];
>> 
>>   NSLog(@"Duration %.03f", NSDate.timeIntervalSinceReferenceDate - 
>> startTime);
> 
> ___
> 
> Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)
> 
> Please do not post admin requests or moderator comments to the list.
> Contact the moderators at cocoa-dev-admins(at)lists.apple.com
> 
> Help/Unsubscribe/Update your Subscription:
> https://lists.apple.com/mailman/options/cocoa-dev/petrock%40mac.com
> 
> This email sent to petr...@mac.com

___

Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)

Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com

Help/Unsubscribe/Update your Subscription:
https://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com

This email sent to arch...@mail-archive.com

Re: Performance issue on macOS 10.15 obtaining display name for ~/Desktop, ~/Documents, and ~/Downloads

[David M. Cotter via Cocoa-dev]

this may be difficult for other to repro

i have discovered it may have to do with permissions / entitlements that have 
been granted the app by the user, and that resetting all perms to default will 
"fix" the problem

in the terminal, do this:
> tccutil reset All
i expect that after that, your delay will disappear.

however, you'll have to manually re-authorize all apps in the system 
prefs->security & privacy->privacy tab

i realize this is a "sweep the problem under the rug" rather than "fix the 
problem" approach
but hopefully this should give someone else a clue as to WHY this is happening?

-dave

> On Apr 19, 2020, at 8:08 AM, Allan Odgaard via Cocoa-dev 
>  wrote:
> 
> Starting with macOS 10.15 I have noticed that obtaining 
> NSURLLocalizedNameKey, NSURLTagNamesKey, and even calling getxattr(), can 
> cause a significant delay.
> 
> The code below, which gets NSURLLocalizedNameKey for 3 folders, takes 1.9 
> seconds to execute on my system.
> 
> It appears though that it is only these 3 specific folders that has the 
> problem.
> 
> Furthermore, the problem only happens when I include the code below in a GUI 
> application which is launched via launch services. If I run the executable 
> directly from a terminal then the code is near instant.
> 
> I tried to sample the code, and it appears that it calls out to some OS 
> service/daemon. Presumably it never gets a response, and a timeout is 
> triggered.
> 
> I wonder if anyone else has come across this? And maybe have discovered more 
> info?
> 
> This is btw macOS 10.15.4 and a fairly stock system (new computer without 
> migrating old data), and I noticed the problem some time ago, so it 
> definitely existed before 10.15.4, but I never saw anything like this before 
> upgrading to Catelina.
> 
>NSTimeInterval startTime = NSDate.timeIntervalSinceReferenceDate;
> 
>NSArray* urls = @[]
>[NSFileManager.defaultManager URLForDirectory:NSDesktopDirectory 
> inDomain:NSUserDomainMask appropriateForURL:nil create:NO error:nil],,
>[NSFileManager.defaultManager URLForDirectory:NSDocumentDirectory 
> inDomain:NSUserDomainMask appropriateForURL:nil create:NO error:nil]
>[NSFileManager.defaultManager URLForDirectory:NSDownloadsDirectory 
> inDomain:NSUserDomainMask appropriateForURL:nil create:NO error:nil,
>];
> 
>NSString* localizedName;
>for(NSURL* url in urls)
>[url getResourceValue:&localizedName forKey:NSURLLocalizedNameKey 
> error:nil];
> 
>NSLog(@"Duration %.03f", NSDate.timeIntervalSinceReferenceDate - 
> startTime);

___

Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)

Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com

Help/Unsubscribe/Update your Subscription:
https://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com

This email sent to arch...@mail-archive.com

Performance issue on macOS 10.15 obtaining display name for ~/Desktop, ~/Documents, and ~/Downloads

[Allan Odgaard via Cocoa-dev]

Starting with macOS 10.15 I have noticed that obtaining 
NSURLLocalizedNameKey, NSURLTagNamesKey, and even calling getxattr(), 
can cause a significant delay.


The code below, which gets NSURLLocalizedNameKey for 3 folders, takes 
1.9 seconds to execute on my system.


It appears though that it is only these 3 specific folders that has the 
problem.


Furthermore, the problem only happens when I include the code below in a 
GUI application which is launched via launch services. If I run the 
executable directly from a terminal then the code is near instant.


I tried to sample the code, and it appears that it calls out to some OS 
service/daemon. Presumably it never gets a response, and a timeout is 
triggered.


I wonder if anyone else has come across this? And maybe have discovered 
more info?


This is btw macOS 10.15.4 and a fairly stock system (new computer 
without migrating old data), and I noticed the problem some time ago, so 
it definitely existed before 10.15.4, but I never saw anything like this 
before upgrading to Catelina.


NSTimeInterval startTime = NSDate.timeIntervalSinceReferenceDate;

NSArray* urls = @[]
[NSFileManager.defaultManager 
URLForDirectory:NSDesktopDirectory inDomain:NSUserDomainMask 
appropriateForURL:nil create:NO error:nil],,
[NSFileManager.defaultManager 
URLForDirectory:NSDocumentDirectory inDomain:NSUserDomainMask 
appropriateForURL:nil create:NO error:nil]
[NSFileManager.defaultManager 
URLForDirectory:NSDownloadsDirectory inDomain:NSUserDomainMask 
appropriateForURL:nil create:NO error:nil,

];

NSString* localizedName;
for(NSURL* url in urls)
[url getResourceValue:&localizedName 
forKey:NSURLLocalizedNameKey error:nil];


NSLog(@"Duration %.03f", NSDate.timeIntervalSinceReferenceDate - 
startTime);

___

Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)

Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com

Help/Unsubscribe/Update your Subscription:
https://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com

This email sent to arch...@mail-archive.com

Re: dequeueReusableCellWithIdentifier bad performance in case of less number of cells?

[Alex Zavatone]

What is the string you are using for the identifier?

All the code I have seen from India that I have seen (from maybe 5 different 
developers) using a cell identifier - for SOME REASON - makes the identifier 
for each cell a different value.

Are you using one value for the identifier or is it a random value each time 
you create the cell?


> On Jan 10, 2018, at 11:56 AM, David Duncan  wrote:
> 
> 
>> On Jan 10, 2018, at 3:48 AM, Devarshi Kulshreshtha 
>>  wrote:
>> 
>> I had a discussion with my colleagues regarding use
>> of dequeueReusableCellWithIdentifier, few of them are under assumption that
>> if there are less number of cells it is better to directly alloc init a
>> UITableViewCell in place of using dequeueReusableCellWithIdentifier. They
>> are saying that since at a time almost all the cells will be displayed
>> there is no need to use dequeueReusableCellWithIdentifier because it
>> degrades performance in comparison to alloc init, in such scenarios. My
>> understanding is that there is no any performance cost as such and behind
>> the scenes dequeueReusableCellWithIdentifier is doing the same thing
>> (allocating and initializing) if a cell cannot be reused.
>> 
>> Is there any way I can compare the performance of using
>> dequeueReusableCellWithIdentifier versus using directly alloc init and
>> share with them to come to a conclusion? Please suggest.
> 
> The overhead of calling dequeueReusableCellWithIdentifier: should be 
> extremely minimal. The static/short table case is actually more likely to be 
> a case where breaking the standard pattern is only going to lead to future 
> maintenance headaches vs achieving any significant performance improvements.
> 
> If you want to measure the difference, you would probably just call 
> alloc/init vs dequeue in a tight loop and measure the total time taken. If 
> you find a case that seems problematic, it would be worth a bug report.
> 
>> 
>> -- 
>> Thanks,
>> 
>> Devarshi
>> ___
>> 
>> Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)
>> 
>> Please do not post admin requests or moderator comments to the list.
>> Contact the moderators at cocoa-dev-admins(at)lists.apple.com
>> 
>> Help/Unsubscribe/Update your Subscription:
>> https://lists.apple.com/mailman/options/cocoa-dev/david.duncan%40apple.com
>> 
>> This email sent to david.dun...@apple.com
> 
> --
> David Duncan
> 
> ___
> 
> Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)
> 
> Please do not post admin requests or moderator comments to the list.
> Contact the moderators at cocoa-dev-admins(at)lists.apple.com
> 
> Help/Unsubscribe/Update your Subscription:
> https://lists.apple.com/mailman/options/cocoa-dev/zav%40mac.com
> 
> This email sent to z...@mac.com

___

Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)

Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com

Help/Unsubscribe/Update your Subscription:
https://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com

This email sent to arch...@mail-archive.com

Re: dequeueReusableCellWithIdentifier bad performance in case of less number of cells?

[David Duncan]

> On Jan 10, 2018, at 3:48 AM, Devarshi Kulshreshtha 
>  wrote:
> 
> I had a discussion with my colleagues regarding use
> of dequeueReusableCellWithIdentifier, few of them are under assumption that
> if there are less number of cells it is better to directly alloc init a
> UITableViewCell in place of using dequeueReusableCellWithIdentifier. They
> are saying that since at a time almost all the cells will be displayed
> there is no need to use dequeueReusableCellWithIdentifier because it
> degrades performance in comparison to alloc init, in such scenarios. My
> understanding is that there is no any performance cost as such and behind
> the scenes dequeueReusableCellWithIdentifier is doing the same thing
> (allocating and initializing) if a cell cannot be reused.
> 
> Is there any way I can compare the performance of using
> dequeueReusableCellWithIdentifier versus using directly alloc init and
> share with them to come to a conclusion? Please suggest.

The overhead of calling dequeueReusableCellWithIdentifier: should be extremely 
minimal. The static/short table case is actually more likely to be a case where 
breaking the standard pattern is only going to lead to future maintenance 
headaches vs achieving any significant performance improvements.

If you want to measure the difference, you would probably just call alloc/init 
vs dequeue in a tight loop and measure the total time taken. If you find a case 
that seems problematic, it would be worth a bug report.

> 
> -- 
> Thanks,
> 
> Devarshi
> ___
> 
> Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)
> 
> Please do not post admin requests or moderator comments to the list.
> Contact the moderators at cocoa-dev-admins(at)lists.apple.com
> 
> Help/Unsubscribe/Update your Subscription:
> https://lists.apple.com/mailman/options/cocoa-dev/david.duncan%40apple.com
> 
> This email sent to david.dun...@apple.com

--
David Duncan

___

Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)

Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com

Help/Unsubscribe/Update your Subscription:
https://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com

This email sent to arch...@mail-archive.com

Re: dequeueReusableCellWithIdentifier bad performance in case of less number of cells?

[Igor Ranieri Elland]

Easiest way is to write both solutions and analyse them with Instruments. 

Best,
ir

> Am 10.01.2018 um 12:48 schrieb Devarshi Kulshreshtha 
> :
> 
> I had a discussion with my colleagues regarding use
> of dequeueReusableCellWithIdentifier, few of them are under assumption that
> if there are less number of cells it is better to directly alloc init a
> UITableViewCell in place of using dequeueReusableCellWithIdentifier. They
> are saying that since at a time almost all the cells will be displayed
> there is no need to use dequeueReusableCellWithIdentifier because it
> degrades performance in comparison to alloc init, in such scenarios. My
> understanding is that there is no any performance cost as such and behind
> the scenes dequeueReusableCellWithIdentifier is doing the same thing
> (allocating and initializing) if a cell cannot be reused.
> 
> Is there any way I can compare the performance of using
> dequeueReusableCellWithIdentifier versus using directly alloc init and
> share with them to come to a conclusion? Please suggest.
> 
> -- 
> Thanks,
> 
> Devarshi
> ___
> 
> Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)
> 
> Please do not post admin requests or moderator comments to the list.
> Contact the moderators at cocoa-dev-admins(at)lists.apple.com
> 
> Help/Unsubscribe/Update your Subscription:
> https://lists.apple.com/mailman/options/cocoa-dev/igor%40elland.me
> 
> This email sent to i...@elland.me

___

Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)

Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com

Help/Unsubscribe/Update your Subscription:
https://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com

This email sent to arch...@mail-archive.com

dequeueReusableCellWithIdentifier bad performance in case of less number of cells?

[Devarshi Kulshreshtha]

I had a discussion with my colleagues regarding use
of dequeueReusableCellWithIdentifier, few of them are under assumption that
if there are less number of cells it is better to directly alloc init a
UITableViewCell in place of using dequeueReusableCellWithIdentifier. They
are saying that since at a time almost all the cells will be displayed
there is no need to use dequeueReusableCellWithIdentifier because it
degrades performance in comparison to alloc init, in such scenarios. My
understanding is that there is no any performance cost as such and behind
the scenes dequeueReusableCellWithIdentifier is doing the same thing
(allocating and initializing) if a cell cannot be reused.

Is there any way I can compare the performance of using
dequeueReusableCellWithIdentifier versus using directly alloc init and
share with them to come to a conclusion? Please suggest.

-- 
Thanks,

Devarshi
___

Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)

Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com

Help/Unsubscribe/Update your Subscription:
https://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com

This email sent to arch...@mail-archive.com

Re: Requires High Performance Graphic Card - Why?

[Jens Alfke]

… and by total coincidence, I just received the iOS Dev Weekly newsletter, 
which links to a blog post written a few days ago describing this in great 
detail:
http://supermegaultragroovy.com//2016/12/10/auto-graphics-switching 


"Oddly, many years after this feature was introduced, it still doesn’t appear 
by default in the Info.plist. It doesn’t even show up in the default Info.plist 
generated for new apps by Xcode. Is this a big deal? Not really, but it seems 
like an oversight this many years later.”

—Jens
___

Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)

Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com

Help/Unsubscribe/Update your Subscription:
https://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com

This email sent to arch...@mail-archive.com

Re: Requires High Performance Graphic Card - Why?

[Jens Alfke]

> On Dec 15, 2016, at 11:19 PM, Alastair Houghton 
>  wrote:
> 
> This is one of those things that you just need to know.  You need to add the 
> key NSSupportsAutomaticGraphicsSwitching to your app’s Info.plist; see
> 
>  https://developer.apple.com/library/content/qa/qa1734/_index.html 
> <https://developer.apple.com/library/content/qa/qa1734/_index.html>

Huh, I did not know this. That document says:
"By default, once your application creates an OpenGL context (by either 
calling OpenGL directly or an API that relies on OpenGL such as Core Animation, 
Core Image, etc), the MacBook Pro automatically switches to the higher-end 
discrete GPU for performance concerns and won't switch back until the 
application quits. … Starting with OS X 10.7, you may utilize the integrated 
GPU on the MacBook Pros if you want to, for example, to save battery life.”

So … any app that uses Core Animation will trigger a switch to the high-end 
(and power hungry) graphics card, unless it adds that plist key? Even if all 
the developer did was click that IB checkbox that enables layers?

—Jens
___

Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)

Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com

Help/Unsubscribe/Update your Subscription:
https://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com

This email sent to arch...@mail-archive.com

Re: Requires High Performance Graphic Card - Why?

[Alastair Houghton]

On 16 Dec 2016, at 04:11, Gerriet M. Denkmann  wrote:
> 
> macOS 12.2; MacBook Pro (Retina, Mid 2012).
> 
> Activity Monitor → Energy tells me that my app requires a "High Performance 
> Graphic Card”.
> 
> The problem: it has absolutely no reason to do so.
> The app does some WiFi stuff and displays the result in a window. There is 
> almost no graphics (except from some sliders, buttons, a colour well, etc.) - 
> and there definitely is no need for “High Performance Graphics” of any sort.
> 
> How to debug this?

This is one of those things that you just need to know.  You need to add the 
key NSSupportsAutomaticGraphicsSwitching to your app’s Info.plist; see

  https://developer.apple.com/library/content/qa/qa1734/_index.html

Kind regards,

Alastair.

--
http://alastairs-place.net


___

Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)

Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com

Help/Unsubscribe/Update your Subscription:
https://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com

This email sent to arch...@mail-archive.com

Requires High Performance Graphic Card - Why?

[Gerriet M. Denkmann]

macOS 12.2; MacBook Pro (Retina, Mid 2012).

Activity Monitor → Energy tells me that my app requires a "High Performance 
Graphic Card”.

The problem: it has absolutely no reason to do so.
The app does some WiFi stuff and displays the result in a window. There is 
almost no graphics (except from some sliders, buttons, a colour well, etc.) - 
and there definitely is no need for “High Performance Graphics” of any sort.

How to debug this?

Gerriet.


___

Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)

Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com

Help/Unsubscribe/Update your Subscription:
https://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com

This email sent to arch...@mail-archive.com

Re: undomanger performance

[Graham Cox]

> On 3 May 2016, at 6:10 AM, Martin Wierschin  wrote:
> 
>> My app deals with an object tree that can have millions of leaves. It is 
>> possible to run an operation on all of them. Each will register its own 
>> change with the undo manager. 
> 
> Is it absolutely necessary for each leaf object to register its own undo 
> invocation? I don’t know if that’s the actual cause of your slowdown, but it 
> seems like a design to be avoided if possible. Can you instead capture a 
> single undo invocation for all affected objects? Even if you have to bundle 
> up and collect state information for all your millions of objects, having a 
> single undo invocation would likely be more efficient than millions of undo 
> invocations each with their own state information.



+1

The Undo Manager creates an NSInvocation object for every action that it 
records. That could add up to a massive memory load for a big tree with each 
node recording an action, as well as a significant performance hit.

As Martin suggests, you probably want to disable undo altogether around the big 
tree operation, enable it again afterwards, then record ONE undo task that can 
undo the entire tree operation. Many big structure designs include a “bulk 
load” operation for this sort of purpose.

—Graham



___

Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)

Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com

Help/Unsubscribe/Update your Subscription:
https://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com

This email sent to arch...@mail-archive.com

Re: undomanger performance

[Martin Wierschin]

> My app deals with an object tree that can have millions of leaves. It is 
> possible to run an operation on all of them. Each will register its own 
> change with the undo manager. 

Is it absolutely necessary for each leaf object to register its own undo 
invocation? I don’t know if that’s the actual cause of your slowdown, but it 
seems like a design to be avoided if possible. Can you instead capture a single 
undo invocation for all affected objects? Even if you have to bundle up and 
collect state information for all your millions of objects, having a single 
undo invocation would likely be more efficient than millions of undo 
invocations each with their own state information.

~Martin Wierschin


___

Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)

Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com

Help/Unsubscribe/Update your Subscription:
https://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com

This email sent to arch...@mail-archive.com

Re: undomanger performance

[Jean-Daniel Dupas]

> Le 1 mai 2016 à 01:42, Quincey Morris  a 
> écrit :
> 
> On Apr 30, 2016, at 12:49 , Jean-Daniel Dupas  > wrote:
>> 
>> Maybe registering the changes is not executed immediately but deferred until 
>> the end of the current event loop cycle
> 
> There’s no “maybe” about it. By default, the undo manager groups all undo 
> actions registered between iterations of the run loop into a single top-level 
> group.

There is other way to group action that deferring the registration until the 
end of the event loop. That’s why I said maybe.


___

Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)

Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com

Help/Unsubscribe/Update your Subscription:
https://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com

This email sent to arch...@mail-archive.com

Re: undomanger performance

[Quincey Morris]

On Apr 30, 2016, at 12:49 , Jean-Daniel Dupas  wrote:
> 
> Maybe registering the changes is not executed immediately but deferred until 
> the end of the current event loop cycle

There’s no “maybe” about it. By default, the undo manager groups all undo 
actions registered between iterations of the run loop into a single top-level 
group.
___

Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)

Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com

Help/Unsubscribe/Update your Subscription:
https://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com

This email sent to arch...@mail-archive.com

Re: undomanger performance

[Jean-Daniel Dupas]

Just my 2 cents. 

Maybe registering the changes is not executed immediately but deferred until 
the end of the current event loop cycle, so the undo manager can group them 
into a single operation.

In such case, it would mean that what you think is the undo registration is 
just a call to schedule the real operation and is relatively fast, but the real 
works take much more time and is deferred until the end of the current event 
handling.

> Le 30 avr. 2016 à 21:39, Georg Seifert  a écrit :
> 
> Hi
> 
> My app deals with an object tree that can have millions of leaves. It is 
> possible to run an operation on all of them. Each will register its own 
> change with the undo manager. The hole operation might tale a few second (the 
> last line in the trace below). But after my operation is finished, there is 
> another operation triggered by the system that takes much longer than the 
> original task and it blocks the UI. It is directly linked to the undo 
> registration, if I disable it, this does not happen. 
> 
> Running Time  Self (ms)   Symbol Name
> 26258.0ms   81.3% 0,0 _dispatch_mgr_thread  0x4ef18d
> 26258.0ms   81.3% 0,0  _dispatch_mgr_thread
> 26258.0ms   81.3% 0,0   _dispatch_mgr_invoke
> 26250.0ms   81.2% 0,0_dispatch_mgr_queue_drain
> 26248.0ms   81.2% 48,0_dispatch_queue_drain
> 26168.0ms   81.0% 1,0  _dispatch_client_callout
> 26141.0ms   80.9% 97,0  _dispatch_source_set_timer3
> 25561.0ms   79.1% 25561,0_dispatch_timers_update
> 481.0ms1.4%   8,0_dispatch_resume_slow
> 2.0ms0.0% 2,0_dispatch_queue_wakeup
> 8.0ms0.0% 0,0   free
> 6.0ms0.0% 6,0   dispatch_release
> 6.0ms0.0% 6,0   _os_object_release
> 4.0ms0.0% 4,0   dispatch_resume
> 1.0ms0.0% 1,0   nano_free_definite_size
> 1.0ms0.0% 0,0   
> 12.0ms0.0%12,0 OSAtomicEnqueue
> 9.0ms0.0% 4,0  gcd_queue_item_complete_hook
> 6.0ms0.0% 1,0  _dispatch_continuation_free_to_cache_limit
> 3.0ms0.0% 2,0  
> _dispatch_introspection_queue_item_dequeue_hook
> 1.0ms0.0% 1,0  DYLD-STUB$$free
> 1.0ms0.0% 0,0  
> 2.0ms0.0% 2,0 _dispatch_introspection_callout_return
> 7.0ms0.0% 6,0_dispatch_cache_cleanup
> 1.0ms0.0% 0,0_dispatch_timers_run
> 4162.0ms   12.8%  0,0 Main Thread  0x4ef176
> 
> Does anyone has an idea what’s going on?
> 
> Thanks
> Georg Seifert
> Xcode 7.2 on MacOS 10.10.5, 27" Retina-iMac.
> 
> 
> ___
> 
> Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)
> 
> Please do not post admin requests or moderator comments to the list.
> Contact the moderators at cocoa-dev-admins(at)lists.apple.com
> 
> Help/Unsubscribe/Update your Subscription:
> https://lists.apple.com/mailman/options/cocoa-dev/mailing%40xenonium.com
> 
> This email sent to mail...@xenonium.com


___

Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)

Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com

Help/Unsubscribe/Update your Subscription:
https://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com

This email sent to arch...@mail-archive.com

undomanger performance

[Georg Seifert]

Hi

My app deals with an object tree that can have millions of leaves. It is 
possible to run an operation on all of them. Each will register its own change 
with the undo manager. The hole operation might tale a few second (the last 
line in the trace below). But after my operation is finished, there is another 
operation triggered by the system that takes much longer than the original task 
and it blocks the UI. It is directly linked to the undo registration, if I 
disable it, this does not happen. 

Running TimeSelf (ms)   Symbol Name
26258.0ms   81.3%   0,0 _dispatch_mgr_thread  0x4ef18d
26258.0ms   81.3%   0,0  _dispatch_mgr_thread
26258.0ms   81.3%   0,0   _dispatch_mgr_invoke
26250.0ms   81.2%   0,0_dispatch_mgr_queue_drain
26248.0ms   81.2%   48,0_dispatch_queue_drain
26168.0ms   81.0%   1,0  _dispatch_client_callout
26141.0ms   80.9%   97,0  _dispatch_source_set_timer3
25561.0ms   79.1%   25561,0_dispatch_timers_update
481.0ms1.4% 8,0_dispatch_resume_slow
2.0ms0.0%   2,0_dispatch_queue_wakeup
8.0ms0.0%   0,0   free
6.0ms0.0%   6,0   dispatch_release
6.0ms0.0%   6,0   _os_object_release
4.0ms0.0%   4,0   dispatch_resume
1.0ms0.0%   1,0   nano_free_definite_size
1.0ms0.0%   0,0   
12.0ms0.0%  12,0 OSAtomicEnqueue
9.0ms0.0%   4,0  gcd_queue_item_complete_hook
6.0ms0.0%   1,0  _dispatch_continuation_free_to_cache_limit
3.0ms0.0%   2,0  
_dispatch_introspection_queue_item_dequeue_hook
1.0ms0.0%   1,0  DYLD-STUB$$free
1.0ms0.0%   0,0  
2.0ms0.0%   2,0 _dispatch_introspection_callout_return
7.0ms0.0%   6,0_dispatch_cache_cleanup
1.0ms0.0%   0,0_dispatch_timers_run
4162.0ms   12.8%0,0 Main Thread  0x4ef176

Does anyone has an idea what’s going on?

Thanks
Georg Seifert
Xcode 7.2 on MacOS 10.10.5, 27" Retina-iMac.


___

Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)

Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com

Help/Unsubscribe/Update your Subscription:
https://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com

This email sent to arch...@mail-archive.com

Re: Improve performance of data structure saved to disk

[Uli Kusterer]

On 06 Aug 2015, at 15:36, Juanjo Conti  wrote:
> I've checked the number of entries and is only 350. They are regular
> cookies for well known sites like google, new relic, twitter...

 That should not be a performance bottleneck. How often are you calling this 
(let's call it saveAllCookies for lack of a better term) on your entire cookie 
storage per whatever unit of time makes sense? Like, when a new cookie comes 
in, is there more than one call to saveAllCookies? 

> This is part of a CookieStorage class replacement, so I read/write the hole
> storage each time a cookie is set or deleted.

 So, how large is the biggest cookie? What's the average size? If your cookies 
are large, it might be better to just individually serialize each cookie to its 
own file and then only write out ones that have actually changed.

Cheers,
-- Uli Kusterer
"The Witnesses of TeachText are everywhere..."
http://stacksmith.org





___

Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)

Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com

Help/Unsubscribe/Update your Subscription:
https://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com

This email sent to arch...@mail-archive.com

Re: Improve performance of data structure saved to disk

[Juanjo Conti]

I was calling archive a lot of times! Changing that really improve
performance.

Thanks!

On Thu, Aug 6, 2015 at 9:09 PM, Graham Cox  wrote:

>
> > With only 350 objects you should be fine using a ‘dumb’ archived
> dictionary. I’ve used that approach for several thousand objects that were
> more complex than cookies; this was on a Mac, but it was back in 2004 so it
> was probably slower than today’s iPhones ;-)
> >
> >> I detect the performance issue using Instruments to mesure CPU time. The
> >> heaviest call from my call resulted to [CookieKey encodeWithCoder:]
> >
> > That method should be pretty quick, so if it’s taking a lot of time
> there may be too many calls to it.
> >
> > My guess is that you’re saving the ‘database’ to disk after every single
> change. That’s going to take O(n^2) time to add n entries, and I can
> believe it would be noticeably slow to add 350. Try doing what I said in my
> last message, using a time delay before saving.
>
>
> I would agree with what Jens says here - the problem isn’t using keyed
> archiving, but the way you’re using keyed archiving.
>
> I sometimes save tens of thousands of objects using keyed archiving and
> it’s not slow. While that’s on Mac not iOS, the difference shouldn’t be
> important. At one time I did run into performance issues with very large
> numbers of objects, but that turned out to be mostly to do with dearchiving
> which called a property setter for each property (some of which had
> side-effects) rather than directly setting the backing store for the
> property. If you’re using automatic backing store synthesis, there’s
> probably not much you can do about that, but you’d want to be definitely
> sure that was the problem before doing anything about it anyway.
>
> Sounds like there is plenty of scope for improving performance in your
> design without changing from keyed archving or needing to look at direct
> propery access. The problem seems to be in your higher-level code that is
> managing the saving to disk of the archives you create.
>
> —Graham
>
>
>


-- 

Juanjo Conti http://goog_2023646312>@carouselapps.com
>

Software Engineer - Carousel Apps <https://carouselapps.com>

-- 
Carousel Apps Limited, registered in England & Wales with registered number 
7689440 and registered office Unit 2 Artbrand Studios, 7 Leathermarket 
Street, London SE1 3HN. Any communication sent by or on behalf of Carousel 
App Ltd or any of its subsidiary, holding or affiliated companies or 
entities (together "Watu") is confidential and may be privileged or 
otherwise protected. If you receive it in error please inform us and then 
delete it from your system. You should not copy it or disclose its contents 
to anyone. Messages sent to and from Watu may be monitored to ensure 
compliance with our internal policies and to protect our business. Emails 
are not secure and cannot be guaranteed to be error free. Anyone who 
communicates with us by email is taken to accept these risks.
___

Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)

Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com

Help/Unsubscribe/Update your Subscription:
https://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com

This email sent to arch...@mail-archive.com

Re: Improve performance of data structure saved to disk

[Graham Cox]

> With only 350 objects you should be fine using a ‘dumb’ archived dictionary. 
> I’ve used that approach for several thousand objects that were more complex 
> than cookies; this was on a Mac, but it was back in 2004 so it was probably 
> slower than today’s iPhones ;-)
> 
>> I detect the performance issue using Instruments to mesure CPU time. The
>> heaviest call from my call resulted to [CookieKey encodeWithCoder:]
> 
> That method should be pretty quick, so if it’s taking a lot of time there may 
> be too many calls to it.
> 
> My guess is that you’re saving the ‘database’ to disk after every single 
> change. That’s going to take O(n^2) time to add n entries, and I can believe 
> it would be noticeably slow to add 350. Try doing what I said in my last 
> message, using a time delay before saving.


I would agree with what Jens says here - the problem isn’t using keyed 
archiving, but the way you’re using keyed archiving.

I sometimes save tens of thousands of objects using keyed archiving and it’s 
not slow. While that’s on Mac not iOS, the difference shouldn’t be important. 
At one time I did run into performance issues with very large numbers of 
objects, but that turned out to be mostly to do with dearchiving which called a 
property setter for each property (some of which had side-effects) rather than 
directly setting the backing store for the property. If you’re using automatic 
backing store synthesis, there’s probably not much you can do about that, but 
you’d want to be definitely sure that was the problem before doing anything 
about it anyway.

Sounds like there is plenty of scope for improving performance in your design 
without changing from keyed archving or needing to look at direct propery 
access. The problem seems to be in your higher-level code that is managing the 
saving to disk of the archives you create.

—Graham



___

Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)

Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com

Help/Unsubscribe/Update your Subscription:
https://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com

This email sent to arch...@mail-archive.com

Re: Improve performance of data structure saved to disk

[Sixten Otto]

On Thu, Aug 6, 2015 at 9:31 AM, Jens Alfke  wrote:

> As far as I know, there’s no good Cocoa solution for super-simple
> persistence — something like a persistent NSDictionary that can efficiently
> store any number of keys. This would be pretty easy to implement using a
> bare-bones key/value store like Berkeley DB, Tokyo Cabinet, LevelDB, LMDB,
> ForestDB, etc. (You can even use SQLite with a very simple key/value
> schema.)
>

It's not Apple-provided, but YapDatabase (
https://github.com/yapstudios/YapDatabase) is a key-value store implemented
on top of SQLite that's usable on iOS and OS X, and which I've seen
recommended a lot in this sort of conversation, and which should be much
easier to adopt that another cross-platform binary store. It also has lots
of extensions to support things like querying and change notifications.

For the record, I agree that it's probably overkill in this instance.

(Also, I'll note for the record that Tokyo/Kyoto Cabinet aren't going to be
usable in an App Store app without a commercial license.)

Sixten
___

Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)

Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com

Help/Unsubscribe/Update your Subscription:
https://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com

This email sent to arch...@mail-archive.com

Re: Improve performance of data structure saved to disk

[Jens Alfke]

> On Aug 6, 2015, at 6:36 AM, Juanjo Conti  wrote:
> 
> I've checked the number of entries and is only 350. They are regular
> cookies for well known sites like google, new relic, twitter...

With only 350 objects you should be fine using a ‘dumb’ archived dictionary. 
I’ve used that approach for several thousand objects that were more complex 
than cookies; this was on a Mac, but it was back in 2004 so it was probably 
slower than today’s iPhones ;-)

> I detect the performance issue using Instruments to mesure CPU time. The
> heaviest call from my call resulted to [CookieKey encodeWithCoder:]

That method should be pretty quick, so if it’s taking a lot of time there may 
be too many calls to it.

My guess is that you’re saving the ‘database’ to disk after every single 
change. That’s going to take O(n^2) time to add n entries, and I can believe it 
would be noticeably slow to add 350. Try doing what I said in my last message, 
using a time delay before saving.

—Jens
___

Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)

Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com

Help/Unsubscribe/Update your Subscription:
https://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com

This email sent to arch...@mail-archive.com

Re: Improve performance of data structure saved to disk

[Jens Alfke]

> On Aug 5, 2015, at 8:42 PM, Quincey Morris 
>  wrote:
> 
> IMO, Core Data is a terribly painful technology that will make you very, very 
> miserable, not to mention adding many months to your project.

I’m not _quite_ as down on it, but my attempts to use it circa 2006-07 weren’t 
as successful as I’d have liked. It took a lot of work, a lot of things were 
unintuitive, and it didn’t scale well. (I’m sure there were ways to redesign my 
data or queries to scale better, but I couldn’t figure them out.)

> As an alternative, if your use case is really a kind of database, you could 
> try using something like sqlite or couchDB (whatever it’s called now, I’m 
> sure Jens would be happy to fill you in). If it’s not, write a custom data 
> file and be done with it.

Yeah, I work on a mobile database called Couchbase Lite. It has an 
object-modeling layer that’s somewhat inspired by Core Data but with less 
complexity. On the other hand, Couchbase Lite’s primary feature is 
sync/replication, which ends up adding complexity of its own. I’d say that if 
all you want to do is locally persist a bunch of objects, it’s overkill.

As far as I know, there’s no good Cocoa solution for super-simple persistence — 
something like a persistent NSDictionary that can efficiently store any number 
of keys. This would be pretty easy to implement using a bare-bones key/value 
store like Berkeley DB, Tokyo Cabinet, LevelDB, LMDB, ForestDB, etc. (You can 
even use SQLite with a very simple key/value schema.) Just use the top-level 
dictionary key as the database key, and use NSCoding to persist the values as 
blobs. You wouldn’t get any query support, but for a lot of use cases you don’t 
need it.

(If anything like this does exist, let me know!)

Anyway, back to Juanjo’s question — I think I have to echo Uli and ask what the 
bottleneck is. Most likely you’ve got a lot of objects in your collection and 
it’s slow to read and write it. To some degree you can work around this by 
being careful not to save changes too often — throttle it so you wait a second 
after the change is made, and then you’ll be able to make one save for multiple 
changes. You can also do the write on a background thread/queue. But that’s 
just a band-aid.

—Jens
___

Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)

Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com

Help/Unsubscribe/Update your Subscription:
https://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com

This email sent to arch...@mail-archive.com

Re: Improve performance of data structure saved to disk

[Juanjo Conti]

I've checked the number of entries and is only 350. They are regular
cookies for well known sites like google, new relic, twitter...

I detect the performance issue using Instruments to mesure CPU time. The
heaviest call from my call resulted to [CookieKey encodeWithCoder:] which
current implementation is:

- (void)encodeWithCoder:(NSCoder *)encoder {
[encoder encodeObject:self.domain forKey:COOKIE_KEY_DOMAIN_KEY];
[encoder encodeObject:self.path forKey:COOKIE_KEY_PATH_KEY];
[encoder encodeObject:self.name forKey:COOKIE_KEY_NAME_KEY];
}

This is part of a CookieStorage class replacement, so I read/write the hole
storage each time a cookie is set or deleted.

CookieKey do implement the methods you mention.

- (NSUInteger)hash {
return self.domain.hash ^ self.path.hash ^ self.name.hash;
}

- (BOOL)isEqual:(id)other {
if (![other isKindOfClass:[CookieKey class]]) {
return NO;
}
CookieKey *otherCookieKey = (CookieKey *)other;
return [self.domain isEqual: otherCookieKey.domain] && [self.path
isEqual: otherCookieKey.path] && [self.name isEqual: otherCookieKey.name];
}

Thanks,


On Thu, Aug 6, 2015 at 7:53 AM, Uli Kusterer 
wrote:

> On 06 Aug 2015, at 05:17, Juanjo Conti  wrote:
> > At the moment I'm using Keyed-Archiving, but after detecting performance
> > issues and read I'm changing to Core-Data.
>
>  How did you detect these performance issues, and where exactly did it
> show you that keyed archiving is at fault?
>
> > The data structure is a NSMutableDictionary in which keys are instantness
> > of a custom class CookieKey and values and instances of NSHTTPCookie.
> > CookieKey has 3 string properties (domain, path and name) and implements
> > NSCoding protocol.
>
>  You are storing browser cookies and are seeing performance issues with
> keyed archivers? How many cookies do you have? Or are they somehow insanely
> large? How often do you read/write the whole storage?
>
>  Given keyed archiving is a generic, and forward-compatible way of storing
> objects, there are of course ways to create a more efficient implementation
> for a particular narrower case, but they're reasonably efficient for at
> least a couple hundred, maybe even thousands of objects, so I'm a bit
> skeptical that this really is your performance bottleneck.
>
> > In the new implementation CookieKey will extend NSManagedObject and have
> > the 3 string properties + a NSData one called cookie. And I'll work with
> a
> > NSMutableArray.
> >
> > Is it a good idea? Another approach to use Core Data and improve
> > performance?
>
>  It depends on your cookies and what makes them slow. If you have many
> cookies and you generally look them up by URL, CoreData might be the right
> approach. If your cookies are simply ridiculously large for a cookie,
> CoreData will probably make things worse. Storing NSData in a managed
> object is usually not the best idea, as you can't index the data, and the
> sqlite database underlying a CoreData store isn't really intended to store
> large blobs of data.
>
>  Without knowing how you diagnosed your performance issue and what part is
> actually slow (writing changed objects? Reading the list at startup?
> Looking up an entry?) we can't really make good suggestions.
>
>  My wild guess from what you've said so far is that you might have picked
> an unsuitable algorithm to look up your objects and that is what's slow.
> Does CookieKey (I hope that class has a prefix on its name in your actual
> code!) implement -compare:, -isEqual: and -hash? Does it implement
> NSCopying like a key should? How did you implement those? How are you
> looking up your objects (if that is what's slow)?
>
> Cheers,
> -- Uli Kusterer
> "The Witnesses of TeachText are everywhere..."
> http://stacksmith.org
>
>
>
>
>


-- 

Juanjo Conti http://goog_2023646312>@carouselapps.com
>

Software Engineer - Carousel Apps <https://carouselapps.com>

-- 
Carousel Apps Limited, registered in England & Wales with registered number 
7689440 and registered office Unit 2 Artbrand Studios, 7 Leathermarket 
Street, London SE1 3HN. Any communication sent by or on behalf of Carousel 
App Ltd or any of its subsidiary, holding or affiliated companies or 
entities (together "Watu") is confidential and may be privileged or 
otherwise protected. If you receive it in error please inform us and then 
delete it from your system. You should not copy it or disclose its contents 
to anyone. Messages sent to and from Watu may be monitored to ensure 
compliance with our internal policies and to protect our business. Emails 
are not secure and cannot be guaranteed to be error free. Anyone who 
communicates w

Re: Improve performance of data structure saved to disk

[Uli Kusterer]

On 06 Aug 2015, at 05:17, Juanjo Conti  wrote:
> At the moment I'm using Keyed-Archiving, but after detecting performance
> issues and read I'm changing to Core-Data.

 How did you detect these performance issues, and where exactly did it show you 
that keyed archiving is at fault?

> The data structure is a NSMutableDictionary in which keys are instantness
> of a custom class CookieKey and values and instances of NSHTTPCookie.
> CookieKey has 3 string properties (domain, path and name) and implements
> NSCoding protocol.

 You are storing browser cookies and are seeing performance issues with keyed 
archivers? How many cookies do you have? Or are they somehow insanely large? 
How often do you read/write the whole storage?

 Given keyed archiving is a generic, and forward-compatible way of storing 
objects, there are of course ways to create a more efficient implementation for 
a particular narrower case, but they're reasonably efficient for at least a 
couple hundred, maybe even thousands of objects, so I'm a bit skeptical that 
this really is your performance bottleneck.

> In the new implementation CookieKey will extend NSManagedObject and have
> the 3 string properties + a NSData one called cookie. And I'll work with a
> NSMutableArray.
> 
> Is it a good idea? Another approach to use Core Data and improve
> performance?

 It depends on your cookies and what makes them slow. If you have many cookies 
and you generally look them up by URL, CoreData might be the right approach. If 
your cookies are simply ridiculously large for a cookie, CoreData will probably 
make things worse. Storing NSData in a managed object is usually not the best 
idea, as you can't index the data, and the sqlite database underlying a 
CoreData store isn't really intended to store large blobs of data.

 Without knowing how you diagnosed your performance issue and what part is 
actually slow (writing changed objects? Reading the list at startup? Looking up 
an entry?) we can't really make good suggestions.

 My wild guess from what you've said so far is that you might have picked an 
unsuitable algorithm to look up your objects and that is what's slow. Does 
CookieKey (I hope that class has a prefix on its name in your actual code!) 
implement -compare:, -isEqual: and -hash? Does it implement NSCopying like a 
key should? How did you implement those? How are you looking up your objects 
(if that is what's slow)?

Cheers,
-- Uli Kusterer
"The Witnesses of TeachText are everywhere..."
http://stacksmith.org





___

Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)

Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com

Help/Unsubscribe/Update your Subscription:
https://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com

This email sent to arch...@mail-archive.com

Re: Improve performance of data structure saved to disk

[Quincey Morris]

On Aug 5, 2015, at 20:17 , Juanjo Conti  wrote:
> 
> At the moment I'm using Keyed-Archiving, but after detecting performance
> issues and read I'm changing to Core-Data.

What quantity of entries/records are you talking about here? It’s not going to 
make a big difference to performance (as opposed to a smaller, custom solution 
that doesn’t involved archiving, if that is in fact the bottleneck) unless you 
have tens of thousands of records to store.

IMO, Core Data is a terribly painful technology that will make you very, very 
miserable, not to mention adding many months to your project.

As an alternative, if your use case is really a kind of database, you could try 
using something like sqlite or couchDB (whatever it’s called now, I’m sure Jens 
would be happy to fill you in). If it’s not, write a custom data file and be 
done with it.

I don’t know for sure, but my impression is that the things that are slow in 
archiving are:

— Following links in a non-hierarchical object graph.

— Uniquing values stored as objects, such as NSString and NSNumber.

___

Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)

Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com

Help/Unsubscribe/Update your Subscription:
https://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com

This email sent to arch...@mail-archive.com

Improve performance of data structure saved to disk

[Juanjo Conti]

Hi there!

At the moment I'm using Keyed-Archiving, but after detecting performance
issues and read I'm changing to Core-Data.

The data structure is a NSMutableDictionary in which keys are instantness
of a custom class CookieKey and values and instances of NSHTTPCookie.
CookieKey has 3 string properties (domain, path and name) and implements
NSCoding protocol.

In the new implementation CookieKey will extend NSManagedObject and have
the 3 string properties + a NSData one called cookie. And I'll work with a
NSMutableArray.

Is it a good idea? Another approach to use Core Data and improve
performance?

Thanks in advance!

-- 

Juanjo Conti http://goog_2023646312>@carouselapps.com
>

Software Engineer - Carousel Apps <https://carouselapps.com>

-- 
Carousel Apps Limited, registered in England & Wales with registered number 
7689440 and registered office Unit 2 Artbrand Studios, 7 Leathermarket 
Street, London SE1 3HN. Any communication sent by or on behalf of Carousel 
App Ltd or any of its subsidiary, holding or affiliated companies or 
entities (together "Watu") is confidential and may be privileged or 
otherwise protected. If you receive it in error please inform us and then 
delete it from your system. You should not copy it or disclose its contents 
to anyone. Messages sent to and from Watu may be monitored to ensure 
compliance with our internal policies and to protect our business. Emails 
are not secure and cannot be guaranteed to be error free. Anyone who 
communicates with us by email is taken to accept these risks.
___

Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)

Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com

Help/Unsubscribe/Update your Subscription:
https://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com

This email sent to arch...@mail-archive.com

Re: CoreData regression or performance problem, anyone?

[Vincent Habchi]

Jens,

> If you're using CoreData I'm not sure if you'll be able to fix your database 
> schema, since presumably CoreData manages it and not you. So I'm not sure how 
> you'd work around the problem. Good luck!

In fact, the Macports search engine is not, so I guess we’ll figure out a way 
to fix this correctly. But I wanted also to warn every possible user of Core 
Data that they might experience regression without any evident reason.

Cheers!
Vincent


___

Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)

Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com

Help/Unsubscribe/Update your Subscription:
https://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com

This email sent to arch...@mail-archive.com

Re: CoreData regression or performance problem, anyone?

[Jens Alfke]

> On Nov 12, 2014, at 6:51 AM, Vincent Habchi  wrote:
> 
> the sqlite version installed on OS X machines has been bumped to 3.8.x in OS 
> X 10.10. This apparently had the unfortunate side (or collateral) effect of 
> plummeting performance of some requests (specifically, some Macports related 
> database operations now take forever while they were fairly quick on 10.9).

Yes, I ran into this in September and tracked down the problem. Yosemite is 
using SQLite 3.8.5. This includes the new query planner[1] that was added in 
3.8. Which is great, but comes with a caveat:

>> The [Next Generation Query Planner] is almost always better than the legacy 
>> query planner. However, there may exist legacy applications that unknowingly 
>> depend on undefined and/or suboptimal behavior in the legacy query planner, 
>> and upgrading to the NGQP on those legacy applications could cause 
>> performance regressions.

They have a checklist[2] of how to troubleshoot performance problems. One of 
the causes is indexes whose primary keys have lots of repeated values, so they 
have to be linearly scanned — in my case I had an index whose primary key was a 
boolean value. Ouch. Improving the index fixed the problem.

If you're using CoreData I'm not sure if you'll be able to fix your database 
schema, since presumably CoreData manages it and not you. So I'm not sure how 
you'd work around the problem. Good luck!

—Jens

[1] https://www.sqlite.org/queryplanner-ng.html
[2] https://www.sqlite.org/queryplanner-ng.html#howtofix

smime.p7s
Description: S/MIME cryptographic signature
___

Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)

Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com

Help/Unsubscribe/Update your Subscription:
https://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com

This email sent to arch...@mail-archive.com

CoreData regression or performance problem, anyone?

[Vincent Habchi]

Folks,

the sqlite version installed on OS X machines has been bumped to 3.8.x in OS X 
10.10. This apparently had the unfortunate side (or collateral) effect of 
plummeting performance of some requests (specifically, some Macports related 
database operations now take forever while they were fairly quick on 10.9).

I was just wondering if some of you, in their applications using SQLite/Core 
Data, had noticed such a severe loss of performance.

Thanks,
Vincent


___

Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)

Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com

Help/Unsubscribe/Update your Subscription:
https://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com

This email sent to arch...@mail-archive.com

Re: string literals and performance

[Jens Alfke]

On May 25, 2014, at 3:18 AM, 2551 <2551p...@gmail.com> wrote:

> That's what I understood from the stackexchange discussion I linked to. As I 
> said, those issues aren't relevant in this case, so I was wondering if there 
> were others. From the replies thus far, it seems not, and I'm inclined to go 
> to style 2 for reasons more or less outlined by Quincey.

I’ve seen people use style 1, and to be honest it looks nuts to me, a ‘code 
smell’ of a newbie programmer. It’s identical to style 2 to the compiler, but 
harder for a human to read and understand.

The SO answer is a typical case of over-answering the question. The linker will 
combine all instances of the same string literal in one binary into a single 
object, so it doesn’t matter how many times you put @"foo” in your source code. 
And it makes absolutely no difference whether that literal is assigned to a 
variable or used directly as a method parameter.

What _is_ important is the DRY principle (Don’t Repeat Yourself). If  @“foo” is 
an important string, like a key in a dictionary where you store some value, 
then you shouldn’t sprinkle it all over your code, but instead declare it once 
as a global constant. That way the compiler can catch typos, and it’s easy to 
change the string later if you every need to. This has nothing to do with 
performance, and everything to do with maintainability and reliability.

—Jens
___

Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)

Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com

Help/Unsubscribe/Update your Subscription:
https://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com

This email sent to arch...@mail-archive.com

Re: string literals and performance

[2551]

> On 25 May 2014, at 15:11, Gary L. Wade  wrote:
> The performance benefit for choosing the first style over the second style 
> comes in if you need to debug your app or change the contents of a string 
> literal.

That's what I understood from the stackexchange discussion I linked to. As I 
said, those issues aren't relevant in this case, so I was wondering if there 
were others. From the replies thus far, it seems not, and I'm inclined to go to 
style 2 for reasons more or less outlined by Quincey.

Thanks to all for your thoughts.


Best

P




signature.asc
Description: Message signed with OpenPGP using GPGMail
___

Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)

Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com

Help/Unsubscribe/Update your Subscription:
https://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com

This email sent to arch...@mail-archive.com

Re: string literals and performance

[Gary L. Wade]

The performance benefit for choosing the first style over the second style 
comes in if you need to debug your app or change the contents of a string 
literal.

If you hard code the same string everywhere you use it, as with the second 
case, you are either going to copy/paste or type it all over again. If you type 
it, this can lead to misspellings which can be frustrating to debug especially 
if the string is a key or a format.

If you use a named constant like the first case, the compiler will tell you 
when you make a misspelling with the variable.

Similarly, if you decide to change the contents of string, a named constant 
improves your performance by you only changing it one place rather than having 
to change it in multiple places. Even a global search and replace requires some 
proofing to be sure you don't change too many instances.

For example, if any of these strings are localizable, using separate named 
constants per meaning will keep you safe when dealing with false friends. As an 
example of one app I worked on, we had two strings that were identical in 
English but different in Spanish: State as in California and State as in On/Off 
but Estados for the former and Condicion for the latter.

While it is true you will see little, if any, performance difference in the 
execution of your code, you will find your coding and debugging performance 
improved by doing these safer and best practices of software development.
--
Gary L. Wade (Sent from my iPhone)
http://www.garywade.com/

> On May 24, 2014, at 9:08 PM, 2551 <2551p...@gmail.com> wrote:
> 
> Are there any performance implications that would suggest preferring one or 
> the other of these different styles?
> 
> NSString *s = @"sing me a song";
> [myClass aMethod: s];
> 
> and
> 
> [myClass aMethod: @"sing me a song"];

___

Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)

Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com

Help/Unsubscribe/Update your Subscription:
https://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com

This email sent to arch...@mail-archive.com

Re: string literals and performance

[Quincey Morris]

On May 24, 2014, at 21:08 , 2551 <2551p...@gmail.com> wrote:

> Are there any performance implications that would suggest preferring one or 
> the other of these different styles?
> 
> NSString *s = @"sing me a song";
> [myClass aMethod: s];
> 
> and
> 
> [myClass aMethod: @"sing me a song”];

Basically — if not literally — no.

> I have a lot of the first kind in my code, and I'm thinking of simplifying it 
> by turning them all into the second kind.

There’s a good chance that the compiler already optimizes the first form into 
the second form.

> A discussion on stackoverflow here:
> 
> http://stackoverflow.com/questions/25746/whats-the-difference-between-a-string-constant-and-a-string-literal
> 
> seems to suggest (if I've understood it correctly) that I'd be better off 
> sticking with the first style to avoid errors and to speed up comparisons.

There is no difference in performance that you should consider *unless* you 
have documented evidence of certain string comparisons causing performance 
problem. Any other stance is unnecessary premature optimization (and you don’t 
know that it will make any kind of difference at all).

The case discussed on SO where you use a global string variable is also 
unnecessary premature optimization, again unless you have some reason to 
believe you have a problem.

If this really worries you, you can *try* to write a sample app where the 
source coding style leads to a measurable performance difference. If you can do 
it, post the code here, because I’m sure we’d all be flabbergasted to see it. ;)

> However, since in my code they're all one-off 'throw away" strings that don't 
> get used elsewhere, that doesn't seem relevant. Is there any other reason why 
> I should stick with the first style rather than the second?

No, the second style is better IMO:

a. It’s less keystrokes to type.

b. It’s clearer when you read the source code again later.

OTOH, if you’ve got a method invocation that is very long, it could be be 
argued that splitting it into multiple lines (the first style) makes it easier 
to read. Since it’s almost entirely a stylistic matter, use the form that looks 
less ugly to you.

___

Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)

Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com

Help/Unsubscribe/Update your Subscription:
https://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com

This email sent to arch...@mail-archive.com

Re: string literals and performance

[Stephen J. Butler]

You misunderstand the point of the Stackoverflow answer. In the first
example you've given it looks like "s" is just a stack local variable. In
that case there is no difference between the two examples. Actually, in the
face of optimization I bet the assembly turns out exactly the same. Use the
later because it is more clear what's going on.

If you did use the same string literal in multiple .m files then you might
get a tiny performance benefit from the "extern NSString* const". But I'd
argue that the real benefit to that is if you ever decide to change the
value then you don't have to worry about updating multiple locations and/or
projects (in the case of a framework).


On Sat, May 24, 2014 at 11:08 PM, 2551 <2551p...@gmail.com> wrote:

> Are there any performance implications that would suggest preferring one
> or the other of these different styles?
>
> NSString *s = @"sing me a song";
> [myClass aMethod: s];
>
> and
>
> [myClass aMethod: @"sing me a song"];
>
> I have a lot of the first kind in my code, and I'm thinking of simplifying
> it by turning them all into the second kind. A discussion on stackoverflow
> here:
>
>
> http://stackoverflow.com/questions/25746/whats-the-difference-between-a-string-constant-and-a-string-literal
>
> seems to suggest (if I've understood it correctly) that I'd be better off
> sticking with the first style to avoid errors and to speed up comparisons.
> However, since in my code they're all one-off 'throw away" strings that
> don't get used elsewhere, that doesn't seem relevant. Is there any other
> reason why I should stick with the first style rather than the second?
>
>
> Thanks for your thoughts.
>
> P
>
> ___
>
> Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)
>
> Please do not post admin requests or moderator comments to the list.
> Contact the moderators at cocoa-dev-admins(at)lists.apple.com
>
> Help/Unsubscribe/Update your Subscription:
>
> https://lists.apple.com/mailman/options/cocoa-dev/stephen.butler%40gmail.com
>
> This email sent to stephen.but...@gmail.com
>
___

Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)

Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com

Help/Unsubscribe/Update your Subscription:
https://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com

This email sent to arch...@mail-archive.com

string literals and performance

[2551]

Are there any performance implications that would suggest preferring one or the 
other of these different styles?

NSString *s = @"sing me a song";
[myClass aMethod: s];

and

[myClass aMethod: @"sing me a song"];

I have a lot of the first kind in my code, and I'm thinking of simplifying it 
by turning them all into the second kind. A discussion on stackoverflow here:

http://stackoverflow.com/questions/25746/whats-the-difference-between-a-string-constant-and-a-string-literal

seems to suggest (if I've understood it correctly) that I'd be better off 
sticking with the first style to avoid errors and to speed up comparisons. 
However, since in my code they're all one-off 'throw away" strings that don't 
get used elsewhere, that doesn't seem relevant. Is there any other reason why I 
should stick with the first style rather than the second?


Thanks for your thoughts.

P


signature.asc
Description: Message signed with OpenPGP using GPGMail
___

Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)

Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com

Help/Unsubscribe/Update your Subscription:
https://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com

This email sent to arch...@mail-archive.com

Re: GCD killed my performance

[David Duncan]

Just because you are using Core Foundation interfaces doesn’t mean you aren’t 
getting Objective-C code behind the scenes.

On Apr 26, 2014, at 2:44 PM, ChanMaxthon  wrote:

> Also when you are using custom run loop sources (which is required here) 
> pretty much all code that is interfacing the run loop is CF code, so no 
> Objective-C method calls here (and the few remained ones can be IMP-cached so 
> they are just as fast.)
> 
> Sent from my iPad
> 
>> On Apr 27, 2014, at 5:36 AM, Jens Alfke  wrote:
>> 
>> 
>>> On Apr 26, 2014, at 2:10 PM, ChanMaxthon  wrote:
>>> 
>>> Since you are interfacing with database maybe you can use a little 
>>> transaction interface which is its own thread and run loop. That may be 
>>> able to cut down your amount of syscalls. That is, not using GCD but old 
>>> fashioned NSThread, NSRunLoop (and CFRunLoop) and NSCondition.
>> 
>> I’m pretty sure NSRunLoop has more overhead than a dispatch_queue does. It’s 
>> doing more work and it’s using Obj-C messaging.
>> 
>>> OS X implemented GCD at kernel level which introduced lots of syscalls 
>>> which are super expensive. The old school method is largely user land so it 
>>> may help a little by keeping syscalls to a minimum. @synchronized uses 
>>> Objective-C runtime functions which was once code behind those old school 
>>> classes.
>> 
>> That's the opposite of what the docs say: "A dispatch semaphore works like a 
>> traditional semaphore, except that when the resource is available, it takes 
>> less time to acquire a dispatch semaphore. The reason is that Grand Central 
>> Dispatch does not call into the kernel for this particular case. It calls 
>> into the kernel only when the resource is not available and the system needs 
>> to park your thread until the semaphore is signaled.” —Concurrency 
>> Programming Guide
>> 
>> —Jens
> ___
> 
> Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)
> 
> Please do not post admin requests or moderator comments to the list.
> Contact the moderators at cocoa-dev-admins(at)lists.apple.com
> 
> Help/Unsubscribe/Update your Subscription:
> https://lists.apple.com/mailman/options/cocoa-dev/david.duncan%40apple.com
> 
> This email sent to david.dun...@apple.com

--
David Duncan


___

Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)

Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com

Help/Unsubscribe/Update your Subscription:
https://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com

This email sent to arch...@mail-archive.com

Re: GCD killed my performance

[Dave Fernandes]

On Apr 26, 2014, at 5:36 PM, Jens Alfke  wrote:

>> OS X implemented GCD at kernel level which introduced lots of syscalls which 
>> are super expensive. The old school method is largely user land so it may 
>> help a little by keeping syscalls to a minimum. @synchronized uses 
>> Objective-C runtime functions which was once code behind those old school 
>> classes.
> 
> That's the opposite of what the docs say: "A dispatch semaphore works like a 
> traditional semaphore, except that when the resource is available, it takes 
> less time to acquire a dispatch semaphore. The reason is that Grand Central 
> Dispatch does not call into the kernel for this particular case. It calls 
> into the kernel only when the resource is not available and the system needs 
> to park your thread until the semaphore is signaled.” —Concurrency 
> Programming Guide

They say this for dispatch semaphores, but it doesn’t necessarily apply to 
dispatch queues. (If I were less lazy, I suppose I could check the source.)
___

Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)

Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com

Help/Unsubscribe/Update your Subscription:
https://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com

This email sent to arch...@mail-archive.com

Re: GCD killed my performance

[ChanMaxthon]

Also when you are using custom run loop sources (which is required here) pretty 
much all code that is interfacing the run loop is CF code, so no Objective-C 
method calls here (and the few remained ones can be IMP-cached so they are just 
as fast.)

Sent from my iPad

> On Apr 27, 2014, at 5:36 AM, Jens Alfke  wrote:
> 
> 
>> On Apr 26, 2014, at 2:10 PM, ChanMaxthon  wrote:
>> 
>> Since you are interfacing with database maybe you can use a little 
>> transaction interface which is its own thread and run loop. That may be able 
>> to cut down your amount of syscalls. That is, not using GCD but old 
>> fashioned NSThread, NSRunLoop (and CFRunLoop) and NSCondition.
> 
> I’m pretty sure NSRunLoop has more overhead than a dispatch_queue does. It’s 
> doing more work and it’s using Obj-C messaging.
> 
>> OS X implemented GCD at kernel level which introduced lots of syscalls which 
>> are super expensive. The old school method is largely user land so it may 
>> help a little by keeping syscalls to a minimum. @synchronized uses 
>> Objective-C runtime functions which was once code behind those old school 
>> classes.
> 
> That's the opposite of what the docs say: "A dispatch semaphore works like a 
> traditional semaphore, except that when the resource is available, it takes 
> less time to acquire a dispatch semaphore. The reason is that Grand Central 
> Dispatch does not call into the kernel for this particular case. It calls 
> into the kernel only when the resource is not available and the system needs 
> to park your thread until the semaphore is signaled.” —Concurrency 
> Programming Guide
> 
> —Jens
___

Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)

Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com

Help/Unsubscribe/Update your Subscription:
https://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com

This email sent to arch...@mail-archive.com

Re: GCD killed my performance

[ChanMaxthon]

Weird. My old code used to be GCD heavy and even deadlocked once but new 
projects that used old school methods are faster and never locked up. My 
projects are heavy in network IO.

I built a transactional network interface there, which is somehow a clone of 
Distributed Objects that is based on HTTP.

Sent from my iPad

> On Apr 27, 2014, at 5:36 AM, Jens Alfke  wrote:
> 
> 
>> On Apr 26, 2014, at 2:10 PM, ChanMaxthon  wrote:
>> 
>> Since you are interfacing with database maybe you can use a little 
>> transaction interface which is its own thread and run loop. That may be able 
>> to cut down your amount of syscalls. That is, not using GCD but old 
>> fashioned NSThread, NSRunLoop (and CFRunLoop) and NSCondition.
> 
> I’m pretty sure NSRunLoop has more overhead than a dispatch_queue does. It’s 
> doing more work and it’s using Obj-C messaging.
> 
>> OS X implemented GCD at kernel level which introduced lots of syscalls which 
>> are super expensive. The old school method is largely user land so it may 
>> help a little by keeping syscalls to a minimum. @synchronized uses 
>> Objective-C runtime functions which was once code behind those old school 
>> classes.
> 
> That's the opposite of what the docs say: "A dispatch semaphore works like a 
> traditional semaphore, except that when the resource is available, it takes 
> less time to acquire a dispatch semaphore. The reason is that Grand Central 
> Dispatch does not call into the kernel for this particular case. It calls 
> into the kernel only when the resource is not available and the system needs 
> to park your thread until the semaphore is signaled.” —Concurrency 
> Programming Guide
> 
> —Jens
___

Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)

Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com

Help/Unsubscribe/Update your Subscription:
https://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com

This email sent to arch...@mail-archive.com

Re: GCD killed my performance

[Jens Alfke]

On Apr 26, 2014, at 2:10 PM, ChanMaxthon  wrote:

> Since you are interfacing with database maybe you can use a little 
> transaction interface which is its own thread and run loop. That may be able 
> to cut down your amount of syscalls. That is, not using GCD but old fashioned 
> NSThread, NSRunLoop (and CFRunLoop) and NSCondition.

I’m pretty sure NSRunLoop has more overhead than a dispatch_queue does. It’s 
doing more work and it’s using Obj-C messaging.

> OS X implemented GCD at kernel level which introduced lots of syscalls which 
> are super expensive. The old school method is largely user land so it may 
> help a little by keeping syscalls to a minimum. @synchronized uses 
> Objective-C runtime functions which was once code behind those old school 
> classes.

That's the opposite of what the docs say: "A dispatch semaphore works like a 
traditional semaphore, except that when the resource is available, it takes 
less time to acquire a dispatch semaphore. The reason is that Grand Central 
Dispatch does not call into the kernel for this particular case. It calls into 
the kernel only when the resource is not available and the system needs to park 
your thread until the semaphore is signaled.” —Concurrency Programming Guide

—Jens
___

Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)

Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com

Help/Unsubscribe/Update your Subscription:
https://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com

This email sent to arch...@mail-archive.com

Re: GCD killed my performance

[ChanMaxthon]

You can write your own dispatch queue by using a CFRunLoopSource and add an 
NSCondition to it when you need it to be synchronous (implementing 
dispatch_sync)

Sent from my iPad

> On Apr 27, 2014, at 4:55 AM, Quincey Morris 
>  wrote:
> 
>> On Apr 26, 2014, at 12:02 , Kyle Sluder  wrote:
>> 
>> FWIW, I’ve been of the opinion for a while that including dispatch_sync in 
>> the API was a mistake.  Too often it becomes a crutch used without 
>> understanding, resulting in stop-start behavior across threads or cores.
> 
> I don’t know if you’ll agree, but it seems to me that there’s a distinction 
> between a *locking* mechanism such as @synchronized, and a *queuing* 
> mechanism, which Jens seems to have demonstrated dispatch_sync to be.
> 
> I understand that both mechanisms may at a lower level depend on both queues 
> and locks, but the distinction I’m making is that a locking mechanism is used 
> when we hope that the lock will generally be granted without contention, 
> while a queuing mechanism is used when we expect there will generally be some 
> contending operation in progress.
> 
> ___
> 
> Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)
> 
> Please do not post admin requests or moderator comments to the list.
> Contact the moderators at cocoa-dev-admins(at)lists.apple.com
> 
> Help/Unsubscribe/Update your Subscription:
> https://lists.apple.com/mailman/options/cocoa-dev/xcvista%40me.com
> 
> This email sent to xcvi...@me.com

___

Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)

Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com

Help/Unsubscribe/Update your Subscription:
https://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com

This email sent to arch...@mail-archive.com

Re: GCD killed my performance

[ChanMaxthon]

Since you are interfacing with database maybe you can use a little transaction 
interface which is its own thread and run loop. That may be able to cut down 
your amount of syscalls. That is, not using GCD but old fashioned NSThread, 
NSRunLoop (and CFRunLoop) and NSCondition.

OS X implemented GCD at kernel level which introduced lots of syscalls which 
are super expensive. The old school method is largely user land so it may help 
a little by keeping syscalls to a minimum. @synchronized uses Objective-C 
runtime functions which was once code behind those old school classes.

Sent from my iPad

> On Apr 27, 2014, at 4:55 AM, Quincey Morris 
>  wrote:
> 
>> On Apr 26, 2014, at 12:02 , Kyle Sluder  wrote:
>> 
>> FWIW, I’ve been of the opinion for a while that including dispatch_sync in 
>> the API was a mistake.  Too often it becomes a crutch used without 
>> understanding, resulting in stop-start behavior across threads or cores.
> 
> I don’t know if you’ll agree, but it seems to me that there’s a distinction 
> between a *locking* mechanism such as @synchronized, and a *queuing* 
> mechanism, which Jens seems to have demonstrated dispatch_sync to be.
> 
> I understand that both mechanisms may at a lower level depend on both queues 
> and locks, but the distinction I’m making is that a locking mechanism is used 
> when we hope that the lock will generally be granted without contention, 
> while a queuing mechanism is used when we expect there will generally be some 
> contending operation in progress.
> 
> ___
> 
> Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)
> 
> Please do not post admin requests or moderator comments to the list.
> Contact the moderators at cocoa-dev-admins(at)lists.apple.com
> 
> Help/Unsubscribe/Update your Subscription:
> https://lists.apple.com/mailman/options/cocoa-dev/xcvista%40me.com
> 
> This email sent to xcvi...@me.com

___

Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)

Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com

Help/Unsubscribe/Update your Subscription:
https://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com

This email sent to arch...@mail-archive.com

Re: GCD killed my performance

[Quincey Morris]

On Apr 26, 2014, at 12:02 , Kyle Sluder  wrote:

> FWIW, I’ve been of the opinion for a while that including dispatch_sync in 
> the API was a mistake.  Too often it becomes a crutch used without 
> understanding, resulting in stop-start behavior across threads or cores.

I don’t know if you’ll agree, but it seems to me that there’s a distinction 
between a *locking* mechanism such as @synchronized, and a *queuing* mechanism, 
which Jens seems to have demonstrated dispatch_sync to be.

I understand that both mechanisms may at a lower level depend on both queues 
and locks, but the distinction I’m making is that a locking mechanism is used 
when we hope that the lock will generally be granted without contention, while 
a queuing mechanism is used when we expect there will generally be some 
contending operation in progress.

___

Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)

Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com

Help/Unsubscribe/Update your Subscription:
https://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com

This email sent to arch...@mail-archive.com

Re: GCD killed my performance

[Kyle Sluder]

> On Apr 25, 2014, at 10:35 AM, Seth Willits  wrote:
> 
>> On Apr 25, 2014, at 8:08 AM, Jens Alfke  wrote:
>> 
>> I’m ending up at the opposite of the received wisdom, namely:
>> * dispatch_sync is a lot cheaper than dispatch_async
>> * only use dispatch_async if you really need to, or for an expensive 
>> operation, because it will slow down all your dispatch_sync calls
> 
> Saying "dispatch_async will slow down *all* dispatch_sync calls" throws up 
> red flags for me.
> 
> In my mind there's still a possibility that You're Doing It Wrong.

FWIW, I’ve been of the opinion for a while that including dispatch_sync in the 
API was a mistake.  Too often it becomes a crutch used without understanding, 
resulting in stop-start behavior across threads or cores. The correct solution 
is usually to rewrite the code in continuation-passing style, even if that 
means a cascade of blocks. Microsoft realized this when designing WinRT and 
_removed all the synchronous APIs_. They also added the async/await keywords to 
C# to avoid the seventeen-levels-of-indentation problem that plagues ObjC code.

Some people at Apple agree that use of dispatch_sync is a code smell, but that 
if they didn't include it a hundred different re-implementations would arise, 
all differently broken.

--Kyle Sluder
___

Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)

Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com

Help/Unsubscribe/Update your Subscription:
https://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com

This email sent to arch...@mail-archive.com

Re: GCD killed my performance

[Jens Alfke]

On Apr 25, 2014, at 5:42 PM, Roland King  wrote:

> They can be very useful finding places where everything blocks waiting for 
> one piece of code to execute, or you ping madly from thread-to-thread, 
> queue-to-queue. 

Thanks, that sounds very useful. I’lll give it a try when I dive back into this.

Today I got the GCD-enabled code almost back up to the performance it had on 
iOS before it was thread-safe and parallel. Still counts as a victory, because 
with more cores (as on my Mac) it’s a lot faster. Some of the speed came from 
not using dispatch_async for trivial blocks, some from fixing bugs in my code, 
some from allocating fewer objects… Slowly chipping away at it and making 
little gains one at a time.

—Jens
___

Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)

Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com

Help/Unsubscribe/Update your Subscription:
https://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com

This email sent to arch...@mail-archive.com

Re: GCD killed my performance

[Roland King]

Have you clicked the 'strategy' buttons at the top left of instruments, I for 
some reason can only do this after I've recorded a trace, not during? They are 
easily overlooked but show per-core or per-thread traces. They can be very 
useful finding places where everything blocks waiting for one piece of code to 
execute, or you ping madly from thread-to-thread, queue-to-queue. There are 
stack traces available at each event. It's a good tool for understanding how 
dispatch queues are being used and finding out whether your dispatch_sync 
really is staying on the same CPU or not. 


On 25 Apr, 2014, at 11:08 pm, Jens Alfke  wrote:

> 
> On Apr 25, 2014, at 1:11 AM, Jonathan Taylor  
> wrote:
> 
>> Have you looked at the output from System Trace on both systems? I often 
>> find that to be informative.
> 
> OK, I tried this and it did turn out to be very informative :) even though I 
> don’t know how to interpret any of the numbers. But just the pretty charts 
> alone told the story:
> - With @synchronized there was very little activity in the System Calls or 
> Scheduling tracks.
> - With GCD there was a whole ton of activity.
> I was surprised there’s this much of a difference, because there’s no actual 
> concurrency in the code at this point! In the commit I’ve rolled back to, all 
> I’ve done is taken my existing single-threaded code and wrapped the C calls 
> with either @synchronized or dispatch_sync. My understanding is that while 
> dispatch_sync is technically switching to a different dispatch queue, if 
> there isn’t any contention it will just do some bookkeeping and run the block 
> on the same thread’s stack. So in this case I wouldn’t expect there to be any 
> actual thread switching going on; except there is.
> 
> … So then I searched the project for “dispatch_async” and found that there 
> was actually _one_ call to it, so my statement about “no actual concurrency” 
> above was a lie. The block it runs doesn’t really need to be async; I was 
> just running it that way because I didn’t need it to complete right away. I 
> changed that call to dispatch_sync, and voila! Almost all the thread 
> scheduling and system calls went away; the system trace now looks like the 
> @synchronized one, and the benchmark times are now slightly better than 
> @synchronized!
> 
> I guess this makes sense: dispatch_sync is super cheap in the uncontended 
> case, but if there’s a dispatch_async pending, then that one obviously has to 
> run first, and it’s probably been scheduled onto another thread, so the 
> dispatch_sync has to either queue onto that thread or at least do some 
> more-expensive locking to wait for the other thread to finish the async call.
> 
> I’m ending up at the opposite of the received wisdom, namely:
> * dispatch_sync is a lot cheaper than dispatch_async
> * only use dispatch_async if you really need to, or for an expensive 
> operation, because it will slow down all your dispatch_sync calls
> 
> I wish there were a big fat super-dense O’Reilly or Big Nerd Ranch book about 
> GCD so I didn’t have to figure all this out on my own...
> 
> —Jens
> ___
> 
> Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)
> 
> Please do not post admin requests or moderator comments to the list.
> Contact the moderators at cocoa-dev-admins(at)lists.apple.com
> 
> Help/Unsubscribe/Update your Subscription:
> https://lists.apple.com/mailman/options/cocoa-dev/rols%40rols.org
> 
> This email sent to r...@rols.org


___

Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)

Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com

Help/Unsubscribe/Update your Subscription:
https://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com

This email sent to arch...@mail-archive.com

Re: GCD killed my performance

[Seth Willits]

On Apr 25, 2014, at 8:08 AM, Jens Alfke  wrote:

> I’m ending up at the opposite of the received wisdom, namely:
> * dispatch_sync is a lot cheaper than dispatch_async
> * only use dispatch_async if you really need to, or for an expensive 
> operation, because it will slow down all your dispatch_sync calls

Saying "dispatch_async will slow down *all* dispatch_sync calls" throws up red 
flags for me.

In my mind there's still a possibility that You're Doing It Wrong. You haven't 
completely outlined exactly what you've been doing, so it's very hard to offer 
any solid advice. For example, you didn't mention which queues you're 
dispatching too, how often your syncs are happen relative to the async, which 
threads they're being dispatch from, what in particular is happening within 
them, didn't show any instrument traces… 

I don't doubt you've found something, but your conclusion doesn't paint the 
whole picture. And as a matter of fact, I think your first email shows this:

"On my MacBook Pro this gave me a nice speedup of 50% or more."

If it was all down to async universally slowing down all dispatch_sync calls, 
then wouldn't you expect it to be slower there too? It seems to me you need a 
better theory as to why the change you made worked. But really, we're flying 
blind here.



--
Seth Willits




___

Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)

Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com

Help/Unsubscribe/Update your Subscription:
https://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com

This email sent to arch...@mail-archive.com

Re: GCD killed my performance

[Jens Alfke]

On Apr 25, 2014, at 1:11 AM, Jonathan Taylor  
wrote:

> Have you looked at the output from System Trace on both systems? I often find 
> that to be informative.

OK, I tried this and it did turn out to be very informative :) even though I 
don’t know how to interpret any of the numbers. But just the pretty charts 
alone told the story:
- With @synchronized there was very little activity in the System Calls or 
Scheduling tracks.
- With GCD there was a whole ton of activity.
I was surprised there’s this much of a difference, because there’s no actual 
concurrency in the code at this point! In the commit I’ve rolled back to, all 
I’ve done is taken my existing single-threaded code and wrapped the C calls 
with either @synchronized or dispatch_sync. My understanding is that while 
dispatch_sync is technically switching to a different dispatch queue, if there 
isn’t any contention it will just do some bookkeeping and run the block on the 
same thread’s stack. So in this case I wouldn’t expect there to be any actual 
thread switching going on; except there is.

… So then I searched the project for “dispatch_async” and found that there was 
actually _one_ call to it, so my statement about “no actual concurrency” above 
was a lie. The block it runs doesn’t really need to be async; I was just 
running it that way because I didn’t need it to complete right away. I changed 
that call to dispatch_sync, and voila! Almost all the thread scheduling and 
system calls went away; the system trace now looks like the @synchronized one, 
and the benchmark times are now slightly better than @synchronized!

I guess this makes sense: dispatch_sync is super cheap in the uncontended case, 
but if there’s a dispatch_async pending, then that one obviously has to run 
first, and it’s probably been scheduled onto another thread, so the 
dispatch_sync has to either queue onto that thread or at least do some 
more-expensive locking to wait for the other thread to finish the async call.

I’m ending up at the opposite of the received wisdom, namely:
* dispatch_sync is a lot cheaper than dispatch_async
* only use dispatch_async if you really need to, or for an expensive operation, 
because it will slow down all your dispatch_sync calls

I wish there were a big fat super-dense O’Reilly or Big Nerd Ranch book about 
GCD so I didn’t have to figure all this out on my own...

—Jens
___

Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)

Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com

Help/Unsubscribe/Update your Subscription:
https://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com

This email sent to arch...@mail-archive.com

Re: GCD killed my performance

[Jonathan Taylor]

Have you looked at the output from System Trace on both systems? I often find 
that to be informative.

That might or might not be the way to tell, but have you considered that the 
very different CPU characteristics might mean that the actual timing and 
pattern of database commands is different on the iPhone, resulting in e.g. a 
different level of contention for the queues? Suppose dispatch_sync is fast on 
an empty queue but has additional overhead on a full queue - resulting in 
different performance on one platform to another?


On 25 Apr 2014, at 04:42, cocoa-dev-requ...@lists.apple.com wrote:

> I’m writing an Objective-C API around a database library, and trying to add 
> some optimizations. There’s a lot of room for parallelizing, since tasks like 
> indexing involve a combination of I/O-bound and CPU-bound operations. As a 
> first step, I made my API thread-safe by creating a dispatch queue and 
> wrapping the C database calls in dispatch_sync blocks. Then I did some 
> reorganization of the code so different parts run on different queues, 
> allowing I/O and computation to run in parallel.
> 
> On my MacBook Pro this gave me a nice speedup of 50% or more.
> 
> But when I tested the code on my iPhone 5 today, I found performance had 
> dropped by about a third. Profiling shows that most of the time is being 
> spent in thread/queue management or Objective-C refcount bookkeeping. It 
> looks as though adding GCD introduced a lot of CPU overhead, and the two 
> cores on my iPhone aren’t enough to make up for that, while the eight cores 
> in my MacBook Pro make it worthwhile.
> 
> I tried backing out all the restructuring of my code, so there’s no actual 
> parallelism going on, just the dispatch_sync calls. Predictably, performance 
> is even worse; slightly more than half as fast as without them.
> 
> So, I’m pretty disappointed. I know that dispatch queues aren’t free, but I 
> wasn’t expecting them to be this expensive! I’m not doing anything silly like 
> wrapping dispatch_sync around trivial calls. The APIs I’m using it on do 
> things like reading and writing values from the persistent store. I was 
> expecting the cost of thread-safety to be lost in the noise compared to that.

___

Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)

Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com

Help/Unsubscribe/Update your Subscription:
https://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com

This email sent to arch...@mail-archive.com

Re: GCD killed my performance

[Quincey Morris]

On Apr 24, 2014, at 22:49 , Jens Alfke  wrote:

> It is, but most of it appears to be memory management _caused_ by GCD, since 
> it goes away when I replace the dispatch calls with @synchronized. GCD is 
> apparently causing a lot of blocks to get copied to the heap.

Well, you know what you’re seeing in Instruments, but this characterization 
seems improbable. Ignoring the GCD-specific entries, if block copy was causing 
a large number of retains and releases (if that’s what you meant by “refcount 
bookkeeping”), I’d expect there to be a large number of block copies, too. If 
block copies (as I’d also expect) are much more time-consuming than the 
retains/releases, by comparison you’d barely notice the retain/release times in 
Instruments. If the retains/releases caused by block copies do dominate, that 
suggests the block copies are comparatively very cheap, which in turn suggests 
a horrible bug in block copies.

With luck someone might jump in with a plausible answer, but this is starting 
to sound TSI-worthy.

___

Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)

Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com

Help/Unsubscribe/Update your Subscription:
https://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com

This email sent to arch...@mail-archive.com

Re: GCD killed my performance

[Jens Alfke]

On Apr 24, 2014, at 10:30 PM, Quincey Morris 
 wrote:

> Approaching this naively, this result suggests that the block content, while 
> not trivial, is too fine-grained — is divided too finely. For example, if 
> you’re putting (essentially) one database read/write operation (or even a 
> handful) in each block, perhaps that’s too small a unit of work for GCD.

I’m coming to that conclusion. I thought that a file-based b-tree lookup would 
be complex enough to drown out that overhead, but maybe not.

> a physical disk access is presumably much slower on average than an access to 
> iOS persistent memory. (Of course, if your MacBook Pro is purely SSD, that 
> might blow that idea out of the water. I don’t know how SSD access speeds 
> compare to iOS memory.)

No, you’ve got it backwards. My 2012 MacBook Pro has a damn fast SSD (one of 
the Apple on-the-motherboard ones), but the flash storage on iOS devices is 
really slow, slower than a decent hard disk.

I interpret the difference in performance as showing that, while there’s a lot 
of extra CPU overhead to using GCD, it’s still a net win when it lets you 
spread your code out over eight cores instead of one. But when there are only 
two cores, it seems to be not worth the expense. 

> Unless I missed something, all of the responses in this thread went to the 
> GCD issue. But if memory management is showing up “hot” like that too, there 
> may be something else to investigate.


It is, but most of it appears to be memory management _caused_ by GCD, since it 
goes away when I replace the dispatch calls with @synchronized. GCD is 
apparently causing a lot of blocks to get copied to the heap.

—Jens
___

Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)

Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com

Help/Unsubscribe/Update your Subscription:
https://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com

This email sent to arch...@mail-archive.com

Re: GCD killed my performance

[Quincey Morris]

On Apr 24, 2014, at 20:14 , Jens Alfke  wrote:

> On my MacBook Pro this gave me a nice speedup of 50% or more.
> 
> But when I tested the code on my iPhone 5 today, I found performance had 
> dropped by about a third.

> I know that dispatch queues aren’t free, but I wasn’t expecting them to be 
> this expensive! I’m not doing anything silly like wrapping dispatch_sync 
> around trivial calls. The APIs I’m using it on do things like reading and 
> writing values from the persistent store.

Approaching this naively, this result suggests that the block content, while 
not trivial, is too fine-grained — is divided too finely. For example, if 
you’re putting (essentially) one database read/write operation (or even a 
handful) in each block, perhaps that’s too small a unit of work for GCD. That 
idea is given *some* plausibility by the different outcomes on Mac and iPhone — 
a physical disk access is presumably much slower on average than an access to 
iOS persistent memory. (Of course, if your MacBook Pro is purely SSD, that 
might blow that idea out of the water. I don’t know how SSD access speeds 
compare to iOS memory.)

> Profiling shows that most of the time is being spent in thread/queue 
> management or Objective-C refcount bookkeeping. 

Unless I missed something, all of the responses in this thread went to the GCD 
issue. But if memory management is showing up “hot” like that too, there may be 
something else to investigate.

One option might be to look at what Instruments reports for the hotspots in 
terms of *counts* rather than times, and compare OS X with iOS. This would 
necessitate arranging things so you could get counts for known increments of 
work. If iOS gives comparable counts but much longer times, you will have one 
way of proceeding; if it gives much larger counts but similar or smaller times 
per count, you will have another way of proceeding.

___

Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)

Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com

Help/Unsubscribe/Update your Subscription:
https://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com

This email sent to arch...@mail-archive.com

Re: GCD killed my performance

[Dave Fernandes]

Is there any way to batch up more work to do in each block? Then your ratio of 
real work to overhead would go up.

On Apr 25, 2014, at 12:35 AM, Jens Alfke  wrote:

> 
> On Apr 24, 2014, at 9:04 PM, Dave Fernandes  
> wrote:
> 
>> What’s the CPU utilization? Are you actually getting full use of them, or 
>> are your threads blocked waiting for something?
> 
> Fairly high — I think 175% or so (out of 200% possible). The problem is that 
> a large fraction of that is taken up with busywork. In the CPU profile list 
> from Instruments, the top 25 or so stack frames are all system 
> infrastructure; you have to read down quite a ways to find any application 
> code at all.
> 
> —Jens

___

Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)

Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com

Help/Unsubscribe/Update your Subscription:
https://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com

This email sent to arch...@mail-archive.com

Re: GCD killed my performance

[Jens Alfke]

On Apr 24, 2014, at 9:04 PM, Dave Fernandes  wrote:

> What’s the CPU utilization? Are you actually getting full use of them, or are 
> your threads blocked waiting for something?

Fairly high — I think 175% or so (out of 200% possible). The problem is that a 
large fraction of that is taken up with busywork. In the CPU profile list from 
Instruments, the top 25 or so stack frames are all system infrastructure; you 
have to read down quite a ways to find any application code at all.

—Jens
___

Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)

Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com

Help/Unsubscribe/Update your Subscription:
https://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com

This email sent to arch...@mail-archive.com

Re: GCD killed my performance

[Dave Fernandes]

What’s the CPU utilization? Are you actually getting full use of them, or are 
your threads blocked waiting for something?

On Apr 24, 2014, at 11:14 PM, Jens Alfke  wrote:

> I’m writing an Objective-C API around a database library, and trying to add 
> some optimizations. There’s a lot of room for parallelizing, since tasks like 
> indexing involve a combination of I/O-bound and CPU-bound operations. As a 
> first step, I made my API thread-safe by creating a dispatch queue and 
> wrapping the C database calls in dispatch_sync blocks. Then I did some 
> reorganization of the code so different parts run on different queues, 
> allowing I/O and computation to run in parallel.
> 
> On my MacBook Pro this gave me a nice speedup of 50% or more.
> 
> But when I tested the code on my iPhone 5 today, I found performance had 
> dropped by about a third. Profiling shows that most of the time is being 
> spent in thread/queue management or Objective-C refcount bookkeeping. It 
> looks as though adding GCD introduced a lot of CPU overhead, and the two 
> cores on my iPhone aren’t enough to make up for that, while the eight cores 
> in my MacBook Pro make it worthwhile.
> 
> I tried backing out all the restructuring of my code, so there’s no actual 
> parallelism going on, just the dispatch_sync calls. Predictably, performance 
> is even worse; slightly more than half as fast as without them.
> 
> So, I’m pretty disappointed. I know that dispatch queues aren’t free, but I 
> wasn’t expecting them to be this expensive! I’m not doing anything silly like 
> wrapping dispatch_sync around trivial calls. The APIs I’m using it on do 
> things like reading and writing values from the persistent store. I was 
> expecting the cost of thread-safety to be lost in the noise compared to that.
> 
> Any suggestions on what to try next?
> 
> —Jens
> ___
> 
> Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)
> 
> Please do not post admin requests or moderator comments to the list.
> Contact the moderators at cocoa-dev-admins(at)lists.apple.com
> 
> Help/Unsubscribe/Update your Subscription:
> https://lists.apple.com/mailman/options/cocoa-dev/dave.fernandes%40utoronto.ca
> 
> This email sent to dave.fernan...@utoronto.ca


___

Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)

Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com

Help/Unsubscribe/Update your Subscription:
https://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com

This email sent to arch...@mail-archive.com

Re: GCD killed my performance

[Jens Alfke]

On Apr 24, 2014, at 8:42 PM, Ken Thomases  wrote:

> You may be aware of this, but dispatch_sync() is not necessary or even 
> particularly relevant to thread-safety.  The use of a serial queue or, 
> possibly, a reader/write mechanism using barriers, is what achieves thread 
> safety.

Initial experimentation showed that dispatch_async was significantly slower 
than dispatch_sync. This makes sense because dispatch_async has to copy the 
block (thus allocating an object on the heap and retaining any captured object 
variables) while dispatch_sync can get away with running the block before the 
call returns, which avoids all that overhead.

> Using a synchronous call is only necessary if your API has synchronous 
> semantics.  For example, if a call provides immediate results to the caller.  
> Reading from a database would typically have to be synchronous, but writing 
> to it can often be asynchronous.

Yeah, I was torn about making the write calls async. But in the underlying C 
API both the read and write calls return error codes, since there could be disk 
or memory errors, and I didn’t want to ignore the return codes on the write 
functions. (My mama didn’t raise no boys to skip proper error handling.)

—Jens
___

Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)

Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com

Help/Unsubscribe/Update your Subscription:
https://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com

This email sent to arch...@mail-archive.com

Re: GCD killed my performance

[Ken Thomases]

On Apr 24, 2014, at 10:14 PM, Jens Alfke wrote:

> I’m writing an Objective-C API around a database library, and trying to add 
> some optimizations. There’s a lot of room for parallelizing, since tasks like 
> indexing involve a combination of I/O-bound and CPU-bound operations. As a 
> first step, I made my API thread-safe by creating a dispatch queue and 
> wrapping the C database calls in dispatch_sync blocks.

You may be aware of this, but dispatch_sync() is not necessary or even 
particularly relevant to thread-safety.  The use of a serial queue or, 
possibly, a reader/write mechanism using barriers, is what achieves thread 
safety.

Using a synchronous call is only necessary if your API has synchronous 
semantics.  For example, if a call provides immediate results to the caller.  
Reading from a database would typically have to be synchronous, but writing to 
it can often be asynchronous.  All you care about is that future reads will 
always see what was previously written, which the serial queue or barriers will 
guarantee.

That doesn't necessarily have any bearing on the overhead of GCD vs. the 
resources of an iPhone, but I thought I'd point it out.

Regards,
Ken


___

Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)

Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com

Help/Unsubscribe/Update your Subscription:
https://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com

This email sent to arch...@mail-archive.com

Re: GCD killed my performance

[Jens Alfke]

Follow-up: I tried replacing every instance of
dispatch_sync(_queue, ^{ … });
with
@synchronized(self) { … }

Things got faster again — looks like @synchronized is a few percent slower than 
no thread-safety, but _significantly_ faster than dispatch_sync. Which seems to 
contradict what the GCD overview says about dispatch queues being faster than 
regular locking techniques. I looked at the disassembly, and @synchronized 
compiles into calls to objc_sync_enter() and objc_sync_exit(), which in turn 
call pthread_mutex_lock and pthread_mutex_unlock; Instruments shows all these 
functions consuming nearly zero CPU time during my benchmark. As opposed to 
with GCD, where the dispatch-queue runtime calls were most of the hottest code 
in the entire run.

I’m not sure what’s going on here. GCD seems to be pretty well respected by 
people I trust (I read Mike Ash’s blog posts about it pretty thoroughly while 
doing my refactoring, for example) and yet my experience with it so far is that 
the overhead is too high to make all the fun queue-and-block-based programming 
worthwhile, at least on iOS. :(

—Jens
___

Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)

Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com

Help/Unsubscribe/Update your Subscription:
https://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com

This email sent to arch...@mail-archive.com

GCD killed my performance

[Jens Alfke]

I’m writing an Objective-C API around a database library, and trying to add 
some optimizations. There’s a lot of room for parallelizing, since tasks like 
indexing involve a combination of I/O-bound and CPU-bound operations. As a 
first step, I made my API thread-safe by creating a dispatch queue and wrapping 
the C database calls in dispatch_sync blocks. Then I did some reorganization of 
the code so different parts run on different queues, allowing I/O and 
computation to run in parallel.

On my MacBook Pro this gave me a nice speedup of 50% or more.

But when I tested the code on my iPhone 5 today, I found performance had 
dropped by about a third. Profiling shows that most of the time is being spent 
in thread/queue management or Objective-C refcount bookkeeping. It looks as 
though adding GCD introduced a lot of CPU overhead, and the two cores on my 
iPhone aren’t enough to make up for that, while the eight cores in my MacBook 
Pro make it worthwhile.

I tried backing out all the restructuring of my code, so there’s no actual 
parallelism going on, just the dispatch_sync calls. Predictably, performance is 
even worse; slightly more than half as fast as without them.

So, I’m pretty disappointed. I know that dispatch queues aren’t free, but I 
wasn’t expecting them to be this expensive! I’m not doing anything silly like 
wrapping dispatch_sync around trivial calls. The APIs I’m using it on do things 
like reading and writing values from the persistent store. I was expecting the 
cost of thread-safety to be lost in the noise compared to that.

Any suggestions on what to try next?

—Jens
___

Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)

Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com

Help/Unsubscribe/Update your Subscription:
https://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com

This email sent to arch...@mail-archive.com

Re: IOS floating point performance

[Uli Kusterer]

On Aug 8, 2013, at 15:29, Trygve Inda  wrote:
> [[NSCalendar alloc] initWithCalendarIdentifier:NSGregorianCalendar]
> 
> This is called from C (not Cocoa) so I am looking at the best way to do this
> once and pass the NSCalendar object to where it is needed.

A common thing for NSCalendar and NSDateFormatter objects is to keep them in 
static variables. That way you pay the overhead once, per function/module and 
then it gets re-used.

Cheers,
-- Uli Kusterer
“The Witnesses of TeachText are everywhere...”
http://stacksmith.org


___

Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)

Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com

Help/Unsubscribe/Update your Subscription:
https://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com

This email sent to arch...@mail-archive.com

Re: IOS floating point performance

[David Rowland]

Fascinating (Spock - Star Trek, 1967)

I added just this to the loop

for (int jj = 0; jj<100;++jj)
{
  T += (float)jj/1000.00;
……..

The time for 100 iterations is 0.033 sec. about ten times slower than before. I 
guess some optimizations remain. 

Still, that works out at 330 microseconds for one iteration, pretty good.

D




On Aug 8, 2013, at 1:14 PM, Thomas Wetmore  wrote:

> David,
> 
> Why don't you increment T by a little bit in each iteration, say by jj/1000., 
> to prove that no optimization is occurring? I would do it but I develop for 
> Mac OSX only.
> 
> Tom Wetmore
> 
> On Aug 8, 2013, at 3:59 PM, David Rowland  wrote:
> 
>> I ran it in Debug mode which should turn off most optimizations. I ran the 
>> loop 100 times and then 200 times. The latter took almost exactly twice the 
>> time as the former. The results are saved in instance variables of the C++ 
>> class this belongs to.
>> 
>> 
>> On Aug 8, 2013, at 12:06 PM, Sandy McGuffog  wrote:
>> 
>>> Be careful using that code as a test; a good optimizing compiler could pick 
>>> up that sin is a library function without side effects, and no result is 
>>> saved, and optimize that loop to two calls to adjustValueRadians. 
>>> 
>>> Sandy
>>> 
>>> On Aug 8, 2013, at 8:17 PM, Thomas Wetmore  wrote:
>>> 
 David,
 
 Those are lightening speeds. So I agree with you wholeheartedly -- there 
 is no sense in working on a custom table-driven approach. The current 
 approach must already be table-based with speeds like that.
 
 Tom Wetmore
 
 
 On Aug 8, 2013, at 1:26 PM, David Rowland  wrote:
 
> I wrote an app that calculates the positions of Sun and Moon and other 
> information as well. The heart of the Moon calculation is this. I added a 
> loop around it and called a stopwatch at the beginning and end.
> 
> startTime();
> for (int jj = 0; jj<100;++jj)
> {
> //in radians
> double lambda = 3.81040282295402 + 8399.70910754626 * T
> + 0.109781209950443 * sin(adjustValueRadians(2.35619449019234 + 
> 8328.69146829639 * T))
> - 0.022165681500328 * sin(adjustValueRadians(4.5256387504213 - 
> 7214.06294691607 * T))
> 
> One hundred times through this loop, on my iPhone 5, took about 0.0028 
> seconds. Two hundred times took about 0.0056 sec.
> 
> I infer that one pass takes about 0.28 seconds, or 28.0 microseconds.
> 
> The functions are probably very carefully written and could not be 
> improved by table lookups, vector libraries, etc. That is barking up the 
> wrong tree. 
> 
> David

___

Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)

Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com

Help/Unsubscribe/Update your Subscription:
https://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com

This email sent to arch...@mail-archive.com

Re: IOS floating point performance

[Greg Parker]

On Aug 8, 2013, at 10:26 AM, David Rowland  wrote:
> One hundred times through this loop, on my iPhone 5, took about 0.0028 
> seconds. Two hundred times took about 0.0056 sec.

Those times are way too small to be reliable. You're down at the level where OS 
scheduler effects are significant.

Make the test longer, say between 1 and 10 seconds. That should give you more 
trustworthy numbers.

(Running the test too long is also counterproductive: the OS is likely to slow 
your process to save power or decrease heat.)


-- 
Greg Parker gpar...@apple.com Runtime Wrangler



___

Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)

Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com

Help/Unsubscribe/Update your Subscription:
https://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com

This email sent to arch...@mail-archive.com

Re: IOS floating point performance

[Thomas Wetmore]

David,

Why don't you increment T by a little bit in each iteration, say by jj/1000., 
to prove that no optimization is occurring? I would do it but I develop for Mac 
OSX only.

Tom Wetmore

On Aug 8, 2013, at 3:59 PM, David Rowland  wrote:

> I ran it in Debug mode which should turn off most optimizations. I ran the 
> loop 100 times and then 200 times. The latter took almost exactly twice the 
> time as the former. The results are saved in instance variables of the C++ 
> class this belongs to.
> 
> 
> On Aug 8, 2013, at 12:06 PM, Sandy McGuffog  wrote:
> 
>> Be careful using that code as a test; a good optimizing compiler could pick 
>> up that sin is a library function without side effects, and no result is 
>> saved, and optimize that loop to two calls to adjustValueRadians. 
>> 
>> Sandy
>> 
>> On Aug 8, 2013, at 8:17 PM, Thomas Wetmore  wrote:
>> 
>>> David,
>>> 
>>> Those are lightening speeds. So I agree with you wholeheartedly -- there is 
>>> no sense in working on a custom table-driven approach. The current approach 
>>> must already be table-based with speeds like that.
>>> 
>>> Tom Wetmore
>>> 
>>> 
>>> On Aug 8, 2013, at 1:26 PM, David Rowland  wrote:
>>> 
 I wrote an app that calculates the positions of Sun and Moon and other 
 information as well. The heart of the Moon calculation is this. I added a 
 loop around it and called a stopwatch at the beginning and end.
 
 startTime();
 for (int jj = 0; jj<100;++jj)
 {
 //in radians
 double lambda = 3.81040282295402 + 8399.70910754626 * T
 + 0.109781209950443 * sin(adjustValueRadians(2.35619449019234 + 
 8328.69146829639 * T))
 - 0.022165681500328 * sin(adjustValueRadians(4.5256387504213 - 
 7214.06294691607 * T))
 
 One hundred times through this loop, on my iPhone 5, took about 0.0028 
 seconds. Two hundred times took about 0.0056 sec.
 
 I infer that one pass takes about 0.28 seconds, or 28.0 microseconds.
 
 The functions are probably very carefully written and could not be 
 improved by table lookups, vector libraries, etc. That is barking up the 
 wrong tree. 
 
 David

___

Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)

Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com

Help/Unsubscribe/Update your Subscription:
https://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com

This email sent to arch...@mail-archive.com

Re: IOS floating point performance

[David Rowland]

I ran it in Debug mode which should turn off most optimizations. I ran the loop 
100 times and then 200 times. The latter took almost exactly twice the time as 
the former. The results are saved in instance variables of the C++ class this 
belongs to.


On Aug 8, 2013, at 12:06 PM, Sandy McGuffog  wrote:

> Be careful using that code as a test; a good optimizing compiler could pick 
> up that sin is a library function without side effects, and no result is 
> saved, and optimize that loop to two calls to adjustValueRadians. 
> 
> Sandy
> 
> On Aug 8, 2013, at 8:17 PM, Thomas Wetmore  wrote:
> 
>> David,
>> 
>> Those are lightening speeds. So I agree with you wholeheartedly -- there is 
>> no sense in working on a custom table-driven approach. The current approach 
>> must already be table-based with speeds like that.
>> 
>> Tom Wetmore
>> 
>> 
>> On Aug 8, 2013, at 1:26 PM, David Rowland  wrote:
>> 
>>> I wrote an app that calculates the positions of Sun and Moon and other 
>>> information as well. The heart of the Moon calculation is this. I added a 
>>> loop around it and called a stopwatch at the beginning and end.
>>> 
>>> startTime();
>>> for (int jj = 0; jj<100;++jj)
>>> {
>>> //in radians
>>> double lambda = 3.81040282295402 + 8399.70910754626 * T
>>> + 0.109781209950443 * sin(adjustValueRadians(2.35619449019234 + 
>>> 8328.69146829639 * T))
>>> - 0.022165681500328 * sin(adjustValueRadians(4.5256387504213 - 
>>> 7214.06294691607 * T))
>>> 
>>> One hundred times through this loop, on my iPhone 5, took about 0.0028 
>>> seconds. Two hundred times took about 0.0056 sec.
>>> 
>>> I infer that one pass takes about 0.28 seconds, or 28.0 microseconds.
>>> 
>>> The functions are probably very carefully written and could not be improved 
>>> by table lookups, vector libraries, etc. That is barking up the wrong tree. 
>>> 
>>> David
>> 
>> ___
>> 
>> Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)
>> 
>> Please do not post admin requests or moderator comments to the list.
>> Contact the moderators at cocoa-dev-admins(at)lists.apple.com
>> 
>> Help/Unsubscribe/Update your Subscription:
>> https://lists.apple.com/mailman/options/cocoa-dev/mcguffogl%40gmail.com
>> 
>> This email sent to mcguff...@gmail.com
> 


___

Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)

Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com

Help/Unsubscribe/Update your Subscription:
https://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com

This email sent to arch...@mail-archive.com

Re: IOS floating point performance

[Sandy McGuffog]

Be careful using that code as a test; a good optimizing compiler could pick up 
that sin is a library function without side effects, and no result is saved, 
and optimize that loop to two calls to adjustValueRadians. 

Sandy

On Aug 8, 2013, at 8:17 PM, Thomas Wetmore  wrote:

> David,
> 
> Those are lightening speeds. So I agree with you wholeheartedly -- there is 
> no sense in working on a custom table-driven approach. The current approach 
> must already be table-based with speeds like that.
> 
> Tom Wetmore
> 
> 
> On Aug 8, 2013, at 1:26 PM, David Rowland  wrote:
> 
>> I wrote an app that calculates the positions of Sun and Moon and other 
>> information as well. The heart of the Moon calculation is this. I added a 
>> loop around it and called a stopwatch at the beginning and end.
>> 
>>  startTime();
>>  for (int jj = 0; jj<100;++jj)
>>  {
>>  //in radians
>>  double lambda = 3.81040282295402 + 8399.70910754626 * T
>>  + 0.109781209950443 * sin(adjustValueRadians(2.35619449019234 + 
>> 8328.69146829639 * T))
>>  - 0.022165681500328 * sin(adjustValueRadians(4.5256387504213 - 
>> 7214.06294691607 * T))
>> 
>> One hundred times through this loop, on my iPhone 5, took about 0.0028 
>> seconds. Two hundred times took about 0.0056 sec.
>> 
>> I infer that one pass takes about 0.28 seconds, or 28.0 microseconds.
>> 
>> The functions are probably very carefully written and could not be improved 
>> by table lookups, vector libraries, etc. That is barking up the wrong tree. 
>> 
>> David
> 
> ___
> 
> Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)
> 
> Please do not post admin requests or moderator comments to the list.
> Contact the moderators at cocoa-dev-admins(at)lists.apple.com
> 
> Help/Unsubscribe/Update your Subscription:
> https://lists.apple.com/mailman/options/cocoa-dev/mcguffogl%40gmail.com
> 
> This email sent to mcguff...@gmail.com


___

Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)

Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com

Help/Unsubscribe/Update your Subscription:
https://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com

This email sent to arch...@mail-archive.com

Re: IOS floating point performance

[John McCall]

On Aug 8, 2013, at 11:04 AM, Scott Ribe  wrote:
> On Aug 8, 2013, at 11:50 AM, Jens Alfke  wrote:
>> I can’t quote you from the C language spec, but I’d be very surprised if 
>> this were true. The result of (type)+(type) is the same (type), for all 
>> numeric types; same for the other built-in operators.
> 
> On Aug 8, 2013, at 11:19 AM, John McCall  wrote:
> 
>> If all the operands to an operator are floats, the operation occurs in 
>> float, and
>> the type of the result will be float.
> 
> Good god, I am old, and you two are making me feel it today. I usually turn 
> to Harbison & Steele for plain C questions, and here's what the 5th Edition 
> (2002) says:
> 
> "Prior to Standard C, implementations were required to convert all values of 
> type float to type double before any operations were performed… In Standard 
> C, operations can now be performed using type float…"
> 
> So, in summary: GET OFF MY LAWN YOU DAMN KIDS!!!

To which the obvious rejoinder is: maybe you should get out a little more, 
grandpa. :)

When a rule hasn’t been in force for over twenty years, it might be time to 
realize that it’s not the rule anymore.

John.
___

Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)

Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com

Help/Unsubscribe/Update your Subscription:
https://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com

This email sent to arch...@mail-archive.com

Re: IOS floating point performance

[Thomas Wetmore]

David,

Those are lightening speeds. So I agree with you wholeheartedly -- there is no 
sense in working on a custom table-driven approach. The current approach must 
already be table-based with speeds like that.

Tom Wetmore


On Aug 8, 2013, at 1:26 PM, David Rowland  wrote:

> I wrote an app that calculates the positions of Sun and Moon and other 
> information as well. The heart of the Moon calculation is this. I added a 
> loop around it and called a stopwatch at the beginning and end.
> 
>   startTime();
>   for (int jj = 0; jj<100;++jj)
>   {
>   //in radians
>   double lambda = 3.81040282295402 + 8399.70910754626 * T
>   + 0.109781209950443 * sin(adjustValueRadians(2.35619449019234 + 
> 8328.69146829639 * T))
>   - 0.022165681500328 * sin(adjustValueRadians(4.5256387504213 - 
> 7214.06294691607 * T))
> 
> One hundred times through this loop, on my iPhone 5, took about 0.0028 
> seconds. Two hundred times took about 0.0056 sec.
> 
> I infer that one pass takes about 0.28 seconds, or 28.0 microseconds.
> 
> The functions are probably very carefully written and could not be improved 
> by table lookups, vector libraries, etc. That is barking up the wrong tree. 
> 
> David

___

Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)

Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com

Help/Unsubscribe/Update your Subscription:
https://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com

This email sent to arch...@mail-archive.com

Re: IOS floating point performance

[Dave Fernandes]

On 2013-08-08, at 1:26 PM, David Rowland  wrote:

> The functions are probably very carefully written and could not be improved 
> by table lookups, vector libraries, etc. That is barking up the wrong tree. 

"vForce is a library of highly optimized transcendental functions (e.g. sin, 
cos, exp). In cases where a large amount of data needs to be processed by 
applying the same transcendental function, vForce routines can provide 
significant performance enhancements."

https://developer.apple.com/library/mac/#featuredarticles/AccelerateFrameworkData/_index.html
___

Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)

Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com

Help/Unsubscribe/Update your Subscription:
https://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com

This email sent to arch...@mail-archive.com

Re: IOS floating point performance

[Scott Ribe]

On Aug 8, 2013, at 11:50 AM, Jens Alfke  wrote:

> I can’t quote you from the C language spec, but I’d be very surprised if this 
> were true. The result of (type)+(type) is the same (type), for all numeric 
> types; same for the other built-in operators.
> 

On Aug 8, 2013, at 11:19 AM, John McCall  wrote:

> If all the operands to an operator are floats, the operation occurs in float, 
> and
> the type of the result will be float.

Good god, I am old, and you two are making me feel it today. I usually turn to 
Harbison & Steele for plain C questions, and here's what the 5th Edition (2002) 
says:

"Prior to Standard C, implementations were required to convert all values of 
type float to type double before any operations were performed… In Standard C, 
operations can now be performed using type float…"

So, in summary: GET OFF MY LAWN YOU DAMN KIDS!!!

-- 
Scott Ribe
scott_r...@elevated-dev.com
http://www.elevated-dev.com/
(303) 722-0567 voice





___

Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)

Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com

Help/Unsubscribe/Update your Subscription:
https://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com

This email sent to arch...@mail-archive.com