Re: Spinning the busy indicator
On 2 May 2015, at 2:42 am, Kyle Sluder k...@ksluder.com wrote: Well that’s an interesting result - creating my own NSOperation with a QoS of NSOperationQualityOfServiceBackground and adding it to my queue produces a perfectly smooth and non-blocking app. If I use -addOperationWithBlock: things are screwed up. Please file a radar! --Kyle Sluder Radar: #20789132 I attached a simple demo app that reproduces the issue: http://apptree.net/code/NSOperationQueueBug.zip Radar report: Summary: NSOperationQueue and related (NSOperation) documentation claims that its default Quality Of Service (QoS) should be set to NSOperationQualityOfServiceBackground. However, it ends up set much higher, which under some circumstances can cause the main thread to become blocked and unresponsive for potentially long periods. The user will experience this as a hang. The problem is especially acute if the work performed by the operation is lengthy and CPU intensive. Steps to Reproduce: The attached demo code quickly displays this problem. It can be opened and compiled in XCode. 1. Uncheck demonstrate bug and click Run Tasks. A number of CPU-intensive tasks are added to an operation queue. These take many seconds to complete, but while they are executing, the main thread runs normally. This can be verified by clicking and dragging in the menubar to show the menus, and also observing the spinning busy indicator. 2. Check demonstrate bug and click 'Run Tasks. The same tasks are run, but this time with the default setting of NSOperationQueue and NSBlockOperation for the QoS property. Click in the menubar and the main thread will block until the tasks complete. Observe also that the spinning indicator does not animate as its thread never gets scheduled. Expected Results: According to documentation, the default QoS property for NSOperationQueue and NSOperation is NSOperationQualityOfServiceBackground. This should give the main thread and other threads time even if the operations are CPU intensive. The documentation goes on to say that developers should not usually change the default QoS setting from this value. Actual Results: In reality, the default setting for QoS ends up as such a high priority that no other threads, including the main thread, can get time and can end up blocked. Version: 10.10.3 Notes: A workaround is to deliberately set the QoS to NSOperationQualityOfServiceBackground rather than leave it at the default value. However, this is at odds with current documentation and expectations. Configuration: This was observed on 3.2 GHz Intel Core i3 CPU (iMac mid 2010 model) but was also verified by other developers on various current models. Attachments: 'NSOperationQueueBug.zip' was successfully uploaded. ___ Cocoa-dev mailing list (Cocoa-dev@lists.apple.com) Please do not post admin requests or moderator comments to the list. Contact the moderators at cocoa-dev-admins(at)lists.apple.com Help/Unsubscribe/Update your Subscription: https://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com This email sent to arch...@mail-archive.com
Re: Spinning the busy indicator
On Apr 30, 2015, at 22:53 , Graham Cox graham@bigpond.com wrote: It looks as if to be sure I’m going to have to drop down a level and create my own NSOperations. You can create your own (non-serial) GCD queue with any desired QoS, then set your NSOperationQueue to use it. ___ Cocoa-dev mailing list (Cocoa-dev@lists.apple.com) Please do not post admin requests or moderator comments to the list. Contact the moderators at cocoa-dev-admins(at)lists.apple.com Help/Unsubscribe/Update your Subscription: https://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com This email sent to arch...@mail-archive.com
Re: Spinning the busy indicator
On 1 May 2015, at 3:53 pm, Graham Cox graham@bigpond.com wrote: It looks as if to be sure I’m going to have to drop down a level and create my own NSOperations. Well that’s an interesting result - creating my own NSOperation with a QoS of NSOperationQualityOfServiceBackground and adding it to my queue produces a perfectly smooth and non-blocking app. If I use -addOperationWithBlock: things are screwed up. For [NSOperation qualityOfService] Apple says: The default value of this property is NSOperationQualityOfServiceBackground and you should leave that value in place whenever possible. But if I leave it at the default, the same blocking problem is apparent. Logging the default QoS, I see it’s -1, which equates to NSQualityOfServiceDefault. Setting it to NSOperationQualityOfServiceBackground things are fine. Therefore at the very least the documentation is incorrect. I do wonder if this isn’t actually a bug though - if something somewhere is interpreting -1 to mean a very high priority instead of a low one, that’s going to screw things up pretty badly all over the place. I’d be interested to know if this has changed from 10.9 or earlier (I’m on 10.10). —Graham ___ Cocoa-dev mailing list (Cocoa-dev@lists.apple.com) Please do not post admin requests or moderator comments to the list. Contact the moderators at cocoa-dev-admins(at)lists.apple.com Help/Unsubscribe/Update your Subscription: https://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com This email sent to arch...@mail-archive.com
Re: Spinning the busy indicator
On Apr 30, 2015, at 23:41 , Graham Cox graham@bigpond.com wrote: I’d be interested to know if this has changed from 10.9 or earlier (I’m on 10.10). Oh, QoS is 10.10+ only. Before that there was threadPriority and queuePriority for NSOperation, and the old GCD dispatch queue priorities correspond to the new QoS levels. ___ Cocoa-dev mailing list (Cocoa-dev@lists.apple.com) Please do not post admin requests or moderator comments to the list. Contact the moderators at cocoa-dev-admins(at)lists.apple.com Help/Unsubscribe/Update your Subscription: https://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com This email sent to arch...@mail-archive.com
Re: Spinning the busy indicator
On Apr 30, 2015, at 23:41 , Graham Cox graham@bigpond.com wrote: But if I leave it at the default, the same blocking problem is apparent. Logging the default QoS, I see it’s -1, which equates to NSQualityOfServiceDefault. Setting it to NSOperationQualityOfServiceBackground things are fine. Therefore at the very least the documentation is incorrect. Look here: https://developer.apple.com/library/mac/releasenotes/Foundation/RN-Foundation/#10_10QoS https://developer.apple.com/library/mac/releasenotes/Foundation/RN-Foundation/#10_10QoS It depends on the NSOperationQueue and the underlying dispatch queue. (Keep in mind that these are beta release notes. I don’t know that there were ever any final release notes for 10.10.) ___ Cocoa-dev mailing list (Cocoa-dev@lists.apple.com) Please do not post admin requests or moderator comments to the list. Contact the moderators at cocoa-dev-admins(at)lists.apple.com Help/Unsubscribe/Update your Subscription: https://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com This email sent to arch...@mail-archive.com
Re: Spinning the busy indicator
In addition to GCD which is a good idea, you might look into the Accelerate framework to see if it offers something. There's a fairly recent WWDC video about it. Sent from my iPhone On 2015/05/01, at 14:53, Graham Cox graham@bigpond.com wrote: On 1 May 2015, at 3:28 pm, Quincey Morris quinceymor...@rivergatesoftware.com wrote: ― I don’t see anything really wrong at any point, other than it looks unresponsive because it’s very busy for a while. Well, thanks for having a look and taking an interest - and apologies for the rather scrappy coding. I would suggest you use QoS to lower the priority of your calculations. This will let UI updates and other important main thread stuff go through ahead of the calculations. The calculations should probably be at the lowest or second-lowest QoS level. I must admit I hadn’t given any thought to the QoS - just learning my way around this stuff. So, the docs say (ha! here we go again…) that the default QoS is NSOperationQualityOfServiceBackground. This appears to be the LOWEST QoS constant. However, it also states that it is only used if the NSOperation itself doesn’t set this value, but because addOperationWithBlock: creates its own NSOperation internally (that we mere mortals don’t get to access), it might be assigning a value that overrides this. Unhelpfully, the simple CPU usage view in XCode only states QoS unavailable”. It looks as if to be sure I’m going to have to drop down a level and create my own NSOperations. Lowering the QoS might take care of your problem, but if it doesn’t, you’re faced with some decisions about “number of threads to use”. If it comes to this, I wonder if it’s a mistake in this case to let the processing fall through to GCD like you’re doing. You have so much processing to do, it’s going to impact the system badly if you tell the system to get it done as fast as possible. This *may* be a case where you create your own pool of NSThreads (either the number of logical CPUs or one less than that), and parcel the work out to them, so that it can’t get out of hand. (However, you might still use NSOperationQueue to manage things for you, but instead of an individual NSOperation doing the work, it would give the work to a NSThread in your pool.) I reckon if it comes to that there’s probably going to be very little value left in NSOperationQueue so I may as well just use the GCD API directly. But I’ll experiment a bit more… ―Graham ___ Cocoa-dev mailing list (Cocoa-dev@lists.apple.com) Please do not post admin requests or moderator comments to the list. Contact the moderators at cocoa-dev-admins(at)lists.apple.com Help/Unsubscribe/Update your Subscription: https://lists.apple.com/mailman/options/cocoa-dev/dangerwillrobinsondanger%40gmail.com This email sent to dangerwillrobinsondan...@gmail.com ___ Cocoa-dev mailing list (Cocoa-dev@lists.apple.com) Please do not post admin requests or moderator comments to the list. Contact the moderators at cocoa-dev-admins(at)lists.apple.com Help/Unsubscribe/Update your Subscription: https://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com This email sent to arch...@mail-archive.com
Re: Spinning the busy indicator
On Fri, May 1, 2015, at 01:41 AM, Graham Cox wrote: On 1 May 2015, at 3:53 pm, Graham Cox graham@bigpond.com wrote: It looks as if to be sure I’m going to have to drop down a level and create my own NSOperations. Well that’s an interesting result - creating my own NSOperation with a QoS of NSOperationQualityOfServiceBackground and adding it to my queue produces a perfectly smooth and non-blocking app. If I use -addOperationWithBlock: things are screwed up. Please file a radar! --Kyle Sluder ___ Cocoa-dev mailing list (Cocoa-dev@lists.apple.com) Please do not post admin requests or moderator comments to the list. Contact the moderators at cocoa-dev-admins(at)lists.apple.com Help/Unsubscribe/Update your Subscription: https://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com This email sent to arch...@mail-archive.com
Re: Spinning the busy indicator
On 1 May 2015, at 12:53 AM, Graham Cox graham@bigpond.com wrote: So, the docs say (ha! here we go again…) that the default QoS is NSOperationQualityOfServiceBackground. This appears to be the LOWEST QoS constant. However, it also states that it is only used if the NSOperation itself doesn’t set this value, but because addOperationWithBlock: creates its own NSOperation internally (that we mere mortals don’t get to access), it might be assigning a value that overrides this. Unhelpfully, the simple CPU usage view in XCode only states QoS unavailable”. It looks as if to be sure I’m going to have to drop down a level and create my own NSOperations. Idle curiosity — if your operations queue is all your own, is there a point in your process in which you can iterate its .operations array and set QoS then? There’s no explicit promise that the property is mutable after the operation is enqueued, though, and you might be Too Late already. This may be moot. You have a workaround, and Kyle accepts that it may be a bug. — F ___ Cocoa-dev mailing list (Cocoa-dev@lists.apple.com) Please do not post admin requests or moderator comments to the list. Contact the moderators at cocoa-dev-admins(at)lists.apple.com Help/Unsubscribe/Update your Subscription: https://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com This email sent to arch...@mail-archive.com
Re: Spinning the busy indicator
On 01 May 2015, at 06:46, Graham Cox graham@bigpond.com wrote: I use NSOperationQueue with the default -maxConcurrentOperationCount which is NSOperationQueueDefaultMaxConcurrentOperationCount, i.e. let the system figure it out. That appears to create 4 threads for my particular machine, which has a Core i3 (2 cores) CPU. My peak CPU usage is reported as 418%. I’m not sure how it achieves that from a dual core CPU, but one would assume that Apple know what they’re doing [/irony]. Does your Core i3 do HyperThreading? In that case, there are a second set of virtual CPUs, so that when one blocks, a second thread can get swapped in *at the hardware level* and get cycles. Cheers, -- Uli Kusterer “The Witnesses of TeachText are everywhere...” http://zathras.de ___ Cocoa-dev mailing list (Cocoa-dev@lists.apple.com) Please do not post admin requests or moderator comments to the list. Contact the moderators at cocoa-dev-admins(at)lists.apple.com Help/Unsubscribe/Update your Subscription: https://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com This email sent to arch...@mail-archive.com
Re: Spinning the busy indicator
On 01 May 2015, at 04:15, Graham Cox graham@bigpond.com wrote: When the calculation for a tile is finished, it calls its delegate to tell it it has finished. The delegate is the original view. I use -performSelectorOnMainThread: to notify the delegate. Idea: Have you tried using performSelectorOnMainThread:withObject:waitUntilDone:modes: and making sure to not just pass NSDefaultRunLoopMode but also NSEventTrackingRunLoopMode (and maybe even NSModalPanelRunLoopMode)? The run loop that runs while the menus are pulled down is the tracking mode, so your messages may not fire because of that, and maybe you have an issue where, after a while, somehow they don’t fire at all even after tracking? Cheers, -- Uli Kusterer “The Witnesses of TeachText are everywhere...” http://zathras.de ___ Cocoa-dev mailing list (Cocoa-dev@lists.apple.com) Please do not post admin requests or moderator comments to the list. Contact the moderators at cocoa-dev-admins(at)lists.apple.com Help/Unsubscribe/Update your Subscription: https://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com This email sent to arch...@mail-archive.com
Re: Spinning the busy indicator
On 1 May 2015, at 19:13, Uli Kusterer witness.of.teacht...@gmx.net wrote: On 01 May 2015, at 04:15, Graham Cox graham@bigpond.com wrote: When the calculation for a tile is finished, it calls its delegate to tell it it has finished. The delegate is the original view. I use -performSelectorOnMainThread: to notify the delegate. Idea: Have you tried using performSelectorOnMainThread:withObject:waitUntilDone:modes: and making sure to not just pass NSDefaultRunLoopMode but also NSEventTrackingRunLoopMode (and maybe even NSModalPanelRunLoopMode)? The run loop that runs while the menus are pulled down is the tracking mode, so your messages may not fire because of that, and maybe you have an issue where, after a while, somehow they don’t fire at all even after tracking? I don’t think that theory fits the observations, not as I understand them. I didn’t see any comment that Graham had issues with his messages not firing back to the main thread when the menu was up, or in fact at all, but just that the progress indicator stopped spinning when menus were opened, indicating not progress but a jammed main thread; and the stack trace seems to indicate that the main thread is indeed blocked on a synchronous call to launch services during the opening of the menu item, not running the event loop in any mode, in fact not running anything at all. ___ Cocoa-dev mailing list (Cocoa-dev@lists.apple.com) Please do not post admin requests or moderator comments to the list. Contact the moderators at cocoa-dev-admins(at)lists.apple.com Help/Unsubscribe/Update your Subscription: https://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com This email sent to arch...@mail-archive.com
Re: Spinning the busy indicator
On May 1, 2015, at 00:08:33, Graham Cox graham@bigpond.com wrote: Well, it’s always gratifying to find I’m not alone ;) How did you figure out a value that leaves one for the main thread? The actual value returned is -1 for NSOperationQueueDefaultMaxConcurrentOperationCount, not the actual number it calculated. I guess there’s a way to get the number of cores, but that isn’t necessarily the whole story…? I just dinked around until I found something that worked for my app (which can create many more ops than there are cores). I ended up with: const NSUIntegernumCores = [[NSProcessInfo processInfo] processorCount] * 2 - 1; [self.searchQueue setMaxConcurrentOperationCount:(NSInteger)numCores]; -- Steve Mills Drummer, Mac geek ___ Cocoa-dev mailing list (Cocoa-dev@lists.apple.com) Please do not post admin requests or moderator comments to the list. Contact the moderators at cocoa-dev-admins(at)lists.apple.com Help/Unsubscribe/Update your Subscription: https://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com This email sent to arch...@mail-archive.com
Re: Spinning the busy indicator
On Apr 30, 2015, at 21:46 , Graham Cox graham@bigpond.com wrote: If anyone’s interested in having a look at what’s happening, I’ve put the project sources up here: http://apptree.net/code/Gingerbread.zip http://apptree.net/code/Gingerbread.zip Here’s what I see: — I took out your 3-thread limitation. — I’m running on a 4-core i7, so there are 8 logical CPUs, and it uses 8 threads for each “run”. — I see the progress indicator appear, stop spinning for a while, spin for a while and disappear. It seems clear that it stops spinning simply because the CPUs are busy doing other things. I’m almost certain this is a red herring (though a UI issue, a point I’ll come back to). — It takes several seconds to recalculate/redraw the view. Then it takes another few seconds for the CPUs to go idle. It looks like there’s a lot of catching up/cleaning up going on after its finished. Perhaps there’s a pile of delegate notifications pending? You really should look into this via Instruments. — I don’t see anything really wrong at any point, other than it looks unresponsive because it’s very busy for a while. I would suggest you use QoS to lower the priority of your calculations. This will let UI updates and other important main thread stuff go through ahead of the calculations. The calculations should probably be at the lowest or second-lowest QoS level. Lowering the QoS might take care of your problem, but if it doesn’t, you’re faced with some decisions about “number of threads to use”. If it comes to this, I wonder if it’s a mistake in this case to let the processing fall through to GCD like you’re doing. You have so much processing to do, it’s going to impact the system badly if you tell the system to get it done as fast as possible. This *may* be a case where you create your own pool of NSThreads (either the number of logical CPUs or one less than that), and parcel the work out to them, so that it can’t get out of hand. (However, you might still use NSOperationQueue to manage things for you, but instead of an individual NSOperation doing the work, it would give the work to a NSThread in your pool.) ___ Cocoa-dev mailing list (Cocoa-dev@lists.apple.com) Please do not post admin requests or moderator comments to the list. Contact the moderators at cocoa-dev-admins(at)lists.apple.com Help/Unsubscribe/Update your Subscription: https://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com This email sent to arch...@mail-archive.com
Re: Spinning the busy indicator
On Apr 30, 2015, at 23:46:53, Graham Cox graham@bigpond.com wrote: Well, here’s a huge clue. I use NSOperationQueue with the default -maxConcurrentOperationCount which is NSOperationQueueDefaultMaxConcurrentOperationCount, i.e. let the system figure it out. That appears to create 4 threads for my particular machine, which has a Core i3 (2 cores) CPU. My peak CPU usage is reported as 418%. I’m not sure how it achieves that from a dual core CPU, but one would assume that Apple know what they’re doing [/irony]. So, I tried setting that NSOperationQueue property directly to something less, and lo-and-behold, the problem goes away with a limit of 3 or less. A limit of 4 produces the same problem. This appears to suggest that if you let NSOperationQueue take all of the CPU resources, the main thread is left high and dry if it needs to create another thread, e.g. for an XPC call. Maybe this is intentional, but it’s a bloody nuisance - surely a better outcome for the “let the system figure it out” would be to say “whatever the max is -1”, so it leaves some capacity in hand for the main thread. The way it works now might give slightly better performance overall, but it causes an app to hang unresponsively for some indeterminate period in some cases. Of course I can set the limit to 3 myself, but if I found myself on a 16 core machine, that's very suboptimal. I’m tempted to report this as a bug, but I’d like to hear what others think about this first. I’ve run into this too, where letting the OS figure out how many operations to queue at once doesn’t always work like a human wants it to work. I ended up doing what you did - leave one for the main thread. NSOperationQueueDefaultMaxConcurrentOperationCount really isn’t smart enough, or they need to add another constant NSOperationQueueDefaultMaxConcurrentButDontBeAJerkOperationCount. -- Steve Mills Drummer, Mac geek ___ Cocoa-dev mailing list (Cocoa-dev@lists.apple.com) Please do not post admin requests or moderator comments to the list. Contact the moderators at cocoa-dev-admins(at)lists.apple.com Help/Unsubscribe/Update your Subscription: https://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com This email sent to arch...@mail-archive.com
Re: Spinning the busy indicator
On 1 May 2015, at 3:28 pm, Quincey Morris quinceymor...@rivergatesoftware.com wrote: — I don’t see anything really wrong at any point, other than it looks unresponsive because it’s very busy for a while. Well, thanks for having a look and taking an interest - and apologies for the rather scrappy coding. I would suggest you use QoS to lower the priority of your calculations. This will let UI updates and other important main thread stuff go through ahead of the calculations. The calculations should probably be at the lowest or second-lowest QoS level. I must admit I hadn’t given any thought to the QoS - just learning my way around this stuff. So, the docs say (ha! here we go again…) that the default QoS is NSOperationQualityOfServiceBackground. This appears to be the LOWEST QoS constant. However, it also states that it is only used if the NSOperation itself doesn’t set this value, but because addOperationWithBlock: creates its own NSOperation internally (that we mere mortals don’t get to access), it might be assigning a value that overrides this. Unhelpfully, the simple CPU usage view in XCode only states QoS unavailable”. It looks as if to be sure I’m going to have to drop down a level and create my own NSOperations. Lowering the QoS might take care of your problem, but if it doesn’t, you’re faced with some decisions about “number of threads to use”. If it comes to this, I wonder if it’s a mistake in this case to let the processing fall through to GCD like you’re doing. You have so much processing to do, it’s going to impact the system badly if you tell the system to get it done as fast as possible. This *may* be a case where you create your own pool of NSThreads (either the number of logical CPUs or one less than that), and parcel the work out to them, so that it can’t get out of hand. (However, you might still use NSOperationQueue to manage things for you, but instead of an individual NSOperation doing the work, it would give the work to a NSThread in your pool.) I reckon if it comes to that there’s probably going to be very little value left in NSOperationQueue so I may as well just use the GCD API directly. But I’ll experiment a bit more… —Graham ___ Cocoa-dev mailing list (Cocoa-dev@lists.apple.com) Please do not post admin requests or moderator comments to the list. Contact the moderators at cocoa-dev-admins(at)lists.apple.com Help/Unsubscribe/Update your Subscription: https://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com This email sent to arch...@mail-archive.com
Re: Spinning the busy indicator
Doing a little bit of googling on some of the stuff in your stack trace, InstallEventHander and GlobalRegistryEventRegistered all seem to be part of HIToolBox. I just picked a random app of my own and stuck a few breakpoints in it to find every menu invocation in my app goes down pretty much the same path, including the synchronous xpc call following HIToolbox adding and removing event handlers, more than one actually. Every one. Just sticking logging on the synchronous xpc calls generates a lot of guff when you open a menu. All this seems to be calling out to some piece of LaunchServices with ‘addasn’ and ‘removeasn’ messages (looking up the stack you can see the strings put into the xpc dictionary). So the act of opening a menu appears to be registering for notifications from launchservices, I’m guessing so that it knows if the app ceases to be frontmost or similar. So I don’t think this has anything to do with your code at all, apart from the fact your code makes the machine really busy. It seems the menu system does a synchronous call out to an XPC service which doesn’t return in a timely manner when the cpu is busy. This is all in the OS code (I’m on Yosemite by the way). Noting the documented warning for using xpc_connection_send_message_with_reply_sync() If the response will be constructed with data that exists in-memory in the server, it is usually safe to make the API synchronous. But if constructing the response requires I/O, and it is likely to be called from the main thread (or a thread which synchronizes with the main thread), we highly encourage that you take the asynchronous route to avoid the risk of blocking the UI.” On 1 May 2015, at 10:59, Roland King r...@rols.org wrote: Stumped. —Graham I don’t have a lot more ideas than you, having been reading this thread (no pun intended) for 2 days. Is this only happening when you click to bring up a menu when your app is running or at other times too? I can’t currently think of a good reason why opening a menu ends up resulting in an XPC call, worse, an XPC call which is synchronous and waits for a reply, on the main thread. Do the classes/methods in the trace, InstallEventHandler(), TEventTypeIndex, GlobalRegistryEventRegistered and LSNotificationReceiver mean anything to you, or anyone else? Might be time to grit teeth and break out Instruments. That can quite usefully show you breakdowns by queue, by thread or by processor. Sometimes just looking at the pattern gives you a clue what’s happening. It will surely tell you if you have the CPUs all spun up to 100%. The xpc call which is hanging. Are xpc services always separate processes, I thought so but someone can correct me if wrong. If so, it sounds like the xpc service being used is starved of resources because the CPUs are busy mandelbrotting, so it doesn’t reply very quickly and blocks your main thread. If you open and close menus on a barebones system running just your app, can you find another process which keeps spinning up? It looks to me that opening a menu calls some xpc service which is expected to return quickly but doesn’t because the machine is swamped (Instruments should be able to tell you if it is). Whether that xpc call is something the OS does all the time or if there’s something in your app which is causing it, can’t say. Synchronous calls to xpc services from the main thread would seem to be something Apple is always telling devs not to do, so it’s probably them. ___ Cocoa-dev mailing list (Cocoa-dev@lists.apple.com) Please do not post admin requests or moderator comments to the list. Contact the moderators at cocoa-dev-admins(at)lists.apple.com Help/Unsubscribe/Update your Subscription: https://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com This email sent to arch...@mail-archive.com
Re: Spinning the busy indicator
On 1 May 2015, at 1:12 pm, Quincey Morris
quinceymor...@rivergatesoftware.com wrote:
Yes, they do say that. But as it happens I went to IB (6.3.1). The
“indeterminate” checkbox is right there, and if you uncheck it you get a
clock-style progress indicator. I guess the docs are out of date, though only
by about 4 or 5 major OS X versions. ;)
You’re right - there is another style of circular spinner you get with
indeterminate = NO. Never noticed that before. And you’re right - the docs are
out of date.
I don’t have any answers here, just some random thoughts:
— Looking at your code, my intuition says that this smells like a reference
cycle problem, one that leads to queue blockage.
— Don’t use ‘performSelector…’ with ARC, since it typically can’t preserve
memory management correctness across that boundary. It’s uglier but
preferable to get back to the main thread via ‘dispatch_async
(dispatch_get_main_queue (), ^{ … });’. There’s an autocompletion snippet for
most of this.
— I’m not a big fan of NSOperationQueue. A simple GCD dispatch_async to the
default non-main queue seems more obvious to me. The only reason for using
NSOperationQueue is if you want a parallel operation count other than 1 (i.e.
serial) or “let the system decide” (i.e. the GCD default), provided that the
individual operations don’t block internally, which tends to mess GCD up. Or
if you want to be able to cancel operations, since there’s no API for that in
GCD itself. Or if you want selector-based operations rather than the
block-based ones you’re using.
— For anything to do with blocks running on background GCD threads, you need
to pay attention to the top level autorelease pool, and the place where
exceptions are caught. IIRC, in vanilla GCD, you have to provide these
yourself, but I don’t remember the NSOperation semantics.
Well, here’s a huge clue.
I use NSOperationQueue with the default -maxConcurrentOperationCount which is
NSOperationQueueDefaultMaxConcurrentOperationCount, i.e. let the system figure
it out. That appears to create 4 threads for my particular machine, which has a
Core i3 (2 cores) CPU. My peak CPU usage is reported as 418%. I’m not sure how
it achieves that from a dual core CPU, but one would assume that Apple know
what they’re doing [/irony].
So, I tried setting that NSOperationQueue property directly to something less,
and lo-and-behold, the problem goes away with a limit of 3 or less. A limit of
4 produces the same problem.
This appears to suggest that if you let NSOperationQueue take all of the CPU
resources, the main thread is left high and dry if it needs to create another
thread, e.g. for an XPC call. Maybe this is intentional, but it’s a bloody
nuisance - surely a better outcome for the “let the system figure it out” would
be to say “whatever the max is -1”, so it leaves some capacity in hand for the
main thread. The way it works now might give slightly better performance
overall, but it causes an app to hang unresponsively for some indeterminate
period in some cases. Of course I can set the limit to 3 myself, but if I found
myself on a 16 core machine, that's very suboptimal.
On 1 May 2015, at 2:20 pm, Roland King r...@rols.org wrote:
So I don’t think this has anything to do with your code at all, apart from
the fact your code makes the machine really busy. It seems the menu system
does a synchronous call out to an XPC service which doesn’t return in a
timely manner when the cpu is busy. This is all in the OS code (I’m on
Yosemite by the way).
I’m tempted to report this as a bug, but I’d like to hear what others think
about this first.
If anyone’s interested in having a look at what’s happening, I’ve put the
project sources up here: http://apptree.net/code/Gingerbread.zip As it’s only a
toy/experimental project, there’s nothing in it that matters but it’s a bit
rough and ready. Line 124 in GCMandelGenerator.m is where I manipulate the
queue max count. Commenting that out gives the default behaviour, which shows
the problem.
—Graham
___
Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)
Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com
Help/Unsubscribe/Update your Subscription:
https://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com
This email sent to arch...@mail-archive.com
Re: Spinning the busy indicator
On 1 May 2015, at 3:02 pm, Steve Mills sjmi...@mac.com wrote: I’ve run into this too, where letting the OS figure out how many operations to queue at once doesn’t always work like a human wants it to work. I ended up doing what you did - leave one for the main thread. NSOperationQueueDefaultMaxConcurrentOperationCount really isn’t smart enough, or they need to add another constant NSOperationQueueDefaultMaxConcurrentButDontBeAJerkOperationCount. Well, it’s always gratifying to find I’m not alone ;) How did you figure out a value that leaves one for the main thread? The actual value returned is -1 for NSOperationQueueDefaultMaxConcurrentOperationCount, not the actual number it calculated. I guess there’s a way to get the number of cores, but that isn’t necessarily the whole story…? —Graham ___ Cocoa-dev mailing list (Cocoa-dev@lists.apple.com) Please do not post admin requests or moderator comments to the list. Contact the moderators at cocoa-dev-admins(at)lists.apple.com Help/Unsubscribe/Update your Subscription: https://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com This email sent to arch...@mail-archive.com
Re: Spinning the busy indicator
On 1 May 2015, at 10:18 am, Quincey Morris
quinceymor...@rivergatesoftware.com wrote:
On Apr 30, 2015, at 16:39 , Graham Cox graham@bigpond.com wrote:
As I mentioned it’s the spinning busy indicator, which is always
indeterminate.
The circular style isn’t always indeterminate, though (I forgot) it looks
different when it’s not indeterminate.
Well, the docs say:
This method only has an effect if style returns NSProgressIndicatorBarStyle.
If style returns NSProgressIndicatorSpinningStyle, the indicator is always
indeterminate, regardless of what you pass to this method.
However, I’m now fairly convinced that this is a red herring, and is a side
effect of the more serious bug.
My app also hits a breakpoint at _dispatch_semaphore_wait_slow when I click
on a menu bar menu, also in the middle of talking to another process via XPC,
so I don’t think that’s a cause of your problem.
Instead, I suspect you have a memory management error that’s corrupting
something. Or a thread safety error that’s causing this as a side effect.
I’m starting to think that as well.
I’m using ARC - I keep trying to get on board with ARC, and every time I do I
seem to run into this sort of difficult to track down issue. At this stage I’ll
try and persist with it, because I’m not getting any problems when I use any of
the memory managment debugging aids such as zombies and guard malloc, etc. So
perhaps as you suggest it’s more of a thread safety issue. I can well believe
it could be, as I’m just exploring using NSOperationQueue for this kind of
processing, and maybe I’ve just done it wrong.
The app here is only an experiment - if I can get it to work properly it’ll be
useful experience that I can apply to other situations. So I’m drawing a
portion of the Mandelbrot set in a view. I tile the view into a number of tiles
and each one is given the appropriate coordinates and goes away and does the
calculation work in a block added to an NSOperationQueue. Plotting the
Mandelbrot set is a good experiment because it is eminently parallisable
without any co-dependencies between tiles - it just needs to keep track of
which part of the overall view it belongs to.
When the calculation for a tile is finished, it calls its delegate to tell it
it has finished. The delegate is the original view. I use
-performSelectorOnMainThread: to notify the delegate. In response, it takes a
bitmap that the tile created and filled in, and draws it in the view (well,
it’s a little more complex. When the delegate of the tile gets the ‘finished’
notification, it adds the tile to a dictionary of tiles that can be drawn,
keyed by a string version of the tile’s coordinates, then it calls
-setNeedsDisplayInRect: for that tile. When -drawRect is called, it iterates
through the dictionary drawing tiles that intersect the dirty region of the
view.) That all works fine - as each tile completes, I see the view plot that
tile, so gradually the view fills with the complete image.
Pursuing the corruption/thread safety angle, if I set it to just use 1 tile for
the view, all is well. The spinner spins and the menubar doesn’t block. If I go
to 2x2 tiles, it’s still OK. Any more than that and the menubar click blocks
the main thread and the spinner doesn’t animate. Thus the problem is sensitive
to the number of tasks I add to the NSOperationQueue.
The code that schedules the work is:
if( !self.busy )
{
mBusy = YES;
[[[self class] mandelGeneratorQueue] addOperationWithBlock:^
{
[self createBitmapIfNeeded];
if([self.delegate
respondsToSelector:@selector(mandelGeneratorWillStartCalculation:)])
[self.delegate
performSelectorOnMainThread:@selector(mandelGeneratorWillStartCalculation:)
withObject:self waitUntilDone:NO];
NSUInteger x, y, xm, ym;
size_t index;
NSRect dr = self.destinationRect;
x = xm = NSWidth( dr );
y = ym = NSHeight( dr );
while( y-- )
{
x = xm;
while( x-- )
{
CGFloat ev = [self
calculateEscapeValueAtX:x y:y];
[self plotEscapeValue:ev atX:x y:y];
}
}
mBusy = NO;
if([self.delegate
respondsToSelector:@selector(mandelGeneratorDidFinishCalculation:)])
[self.delegate
Re: Spinning the busy indicator
On Apr 30, 2015, at 19:15 , Graham Cox graham@bigpond.com wrote:
Well, the docs say:
This method only has an effect if style returns NSProgressIndicatorBarStyle.
If style returns NSProgressIndicatorSpinningStyle, the indicator is always
indeterminate, regardless of what you pass to this method.
Yes, they do say that. But as it happens I went to IB (6.3.1). The
“indeterminate” checkbox is right there, and if you uncheck it you get a
clock-style progress indicator. I guess the docs are out of date, though only
by about 4 or 5 major OS X versions. ;)
The code that schedules the work is:
I don’t have any answers here, just some random thoughts:
— Looking at your code, my intuition says that this smells like a reference
cycle problem, one that leads to queue blockage.
— Don’t use ‘performSelector…’ with ARC, since it typically can’t preserve
memory management correctness across that boundary. It’s uglier but preferable
to get back to the main thread via ‘dispatch_async (dispatch_get_main_queue (),
^{ … });’. There’s an autocompletion snippet for most of this.
— I’m not a big fan of NSOperationQueue. A simple GCD dispatch_async to the
default non-main queue seems more obvious to me. The only reason for using
NSOperationQueue is if you want a parallel operation count other than 1 (i.e.
serial) or “let the system decide” (i.e. the GCD default), provided that the
individual operations don’t block internally, which tends to mess GCD up. Or if
you want to be able to cancel operations, since there’s no API for that in GCD
itself. Or if you want selector-based operations rather than the block-based
ones you’re using.
— For anything to do with blocks running on background GCD threads, you need to
pay attention to the top level autorelease pool, and the place where exceptions
are caught. IIRC, in vanilla GCD, you have to provide these yourself, but I
don’t remember the NSOperation semantics.
___
Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)
Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com
Help/Unsubscribe/Update your Subscription:
https://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com
This email sent to arch...@mail-archive.com
Re: Spinning the busy indicator
Stumped. —Graham I don’t have a lot more ideas than you, having been reading this thread (no pun intended) for 2 days. Is this only happening when you click to bring up a menu when your app is running or at other times too? I can’t currently think of a good reason why opening a menu ends up resulting in an XPC call, worse, an XPC call which is synchronous and waits for a reply, on the main thread. Do the classes/methods in the trace, InstallEventHandler(), TEventTypeIndex, GlobalRegistryEventRegistered and LSNotificationReceiver mean anything to you, or anyone else? Might be time to grit teeth and break out Instruments. That can quite usefully show you breakdowns by queue, by thread or by processor. Sometimes just looking at the pattern gives you a clue what’s happening. It will surely tell you if you have the CPUs all spun up to 100%. The xpc call which is hanging. Are xpc services always separate processes, I thought so but someone can correct me if wrong. If so, it sounds like the xpc service being used is starved of resources because the CPUs are busy mandelbrotting, so it doesn’t reply very quickly and blocks your main thread. If you open and close menus on a barebones system running just your app, can you find another process which keeps spinning up? It looks to me that opening a menu calls some xpc service which is expected to return quickly but doesn’t because the machine is swamped (Instruments should be able to tell you if it is). Whether that xpc call is something the OS does all the time or if there’s something in your app which is causing it, can’t say. Synchronous calls to xpc services from the main thread would seem to be something Apple is always telling devs not to do, so it’s probably them. ___ Cocoa-dev mailing list (Cocoa-dev@lists.apple.com) Please do not post admin requests or moderator comments to the list. Contact the moderators at cocoa-dev-admins(at)lists.apple.com Help/Unsubscribe/Update your Subscription: https://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com This email sent to arch...@mail-archive.com
Re: Spinning the busy indicator
Did you set your progress indicator to indeterminate? If not, it’s not going to animate. As I mentioned it’s the spinning busy indicator, which is always indeterminate. Something odd is happening, which may or may not have an effect on the busy indicator (which is a minor annoyance, but possibly an indication of a much more major problem). When I start my threaded operations, my main loop generally remains responsive and as each operation block completes, it updates a view (via the main thread) and I see those updates arriving as they complete as intended. I can also interact with the view while this is happening, so I can see that events are being processed and handled by the view as normal. But when I click in the menubar, the main thread suddenly blocks as given by the stack trace in my previous message. It remains blocked until all of the pending operations complete. My menubar does not have any hooks into my app’s code at all - I’m not even implementing menu item validation at the moment. What could the menubar click be waiting on that matters? If it’s “normal” that clicking in the menubar blocks the main thread while there is an operation queue with unfinished tasks that has nothing to do with it, then I think that’s got to be a bug in the OS - it just doesn’t make sense. This is a much more serious issue than the busy indicator animation, but something tells me it may be related in some way, though I’m not really seeing what it is at the moment. A general question - given a blocked lock (semaphore_wait_trap), how can I discover in the debugger what other threads or code is holding that lock? —Graham ___ Cocoa-dev mailing list (Cocoa-dev@lists.apple.com) Please do not post admin requests or moderator comments to the list. Contact the moderators at cocoa-dev-admins(at)lists.apple.com Help/Unsubscribe/Update your Subscription: https://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com This email sent to arch...@mail-archive.com
Re: Spinning the busy indicator
On Apr 30, 2015, at 16:39 , Graham Cox graham@bigpond.com wrote: As I mentioned it’s the spinning busy indicator, which is always indeterminate. The circular style isn’t always indeterminate, though (I forgot) it looks different when it’s not indeterminate. I just tried forcing an app with a spinning indeterminate progress indicator to block its own main thread, and the indicator doesn’t stop animating, regardless of whether usesThreadedAnimation is set. But when I click in the menubar, the main thread suddenly blocks as given by the stack trace in my previous message. It remains blocked until all of the pending operations complete. My app also hits a breakpoint at _dispatch_semaphore_wait_slow when I click on a menu bar menu, also in the middle of talking to another process via XPC, so I don’t think that’s a cause of your problem. Instead, I suspect you have a memory management error that’s corrupting something. Or a thread safety error that’s causing this as a side effect. ___ Cocoa-dev mailing list (Cocoa-dev@lists.apple.com) Please do not post admin requests or moderator comments to the list. Contact the moderators at cocoa-dev-admins(at)lists.apple.com Help/Unsubscribe/Update your Subscription: https://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com This email sent to arch...@mail-archive.com
Re: Spinning the busy indicator
On 30 Apr 2015, at 11:22 am, Graham Cox graham@bigpond.com wrote: My guess is that the busy indicator is animated by a low-priority thread of its own, and the work being done by my queue has a higher priority so the animation thread never gets any time. Does that sound like a reasonable explanation? If so, is there any way to make sure that the spinner actually spins? OK, I’m a lot less sure about this now. I noticed that my main thread is blocking for some reason after I kick off a number of operations on my operation queue. This remains blocked for some time - it appears as if it’s waiting on some of the operations to complete, which is defeating the entire purpose of using the queue. I’ve checked and none of the queued tasks end up on the main thread, which is in line with my expectations having read the docs. My main thread looks like this: Thread 1Queue : com.apple.main-thread (serial) #0 0x7fff8dd7851a in semaphore_wait_trap () #1 0x0001000477e0 in _dispatch_semaphore_wait_slow () #2 0x7fff90ee in xpc_connection_send_message_with_reply_sync () #3 0x7fff91d76150 in LSNotificationReceiver::modify(unsigned long, LSNotificationCode const*, unsigned long, LSNotificationCode const*, void const*, void const*) () #4 0x7fff91d76ac4 in _LSModifyNotification () #5 0x7fff99fad0b4 in HandlerStatusChanged () #6 0x7fff99faa1a0 in GlobalRegistryEventRegistered(unsigned int, unsigned int) () #7 0x7fff99fa9ea8 in TEventTypeIndex::AddTypes(unsigned long, EventTypeSpec const*) () #8 0x7fff99fa9adf in AddHandlersToIndex(EventTargetRec*, HandlerRec*, unsigned long, EventTypeSpec const*) () #9 0x7fff9a129e1e in PushEventHandler(EventTargetRec*, int (*)(OpaqueEventHandlerCallRef*, OpaqueEventRef*, void*), unsigned char, unsigned long, EventTypeSpec const*, void*, OpaqueEventHandlerRef**) () #10 0x7fff99faca07 in InstallEventHandler () #11 0x7fff9a015af1 in SetupMenuTracking(MenuSelectData, unsigned char, Point, double, MenuData*, unsigned int, unsigned short, Rect const*, Rect const*, __CFDictionary const*, unsigned int, Rect const*, __CFString const*) () #12 0x7fff9a0392cf in MenuSelectCore(MenuData*, Point, double, unsigned int, OpaqueMenuRef**, unsigned short*) () #13 0x7fff9a0390fe in _HandleMenuSelection2 () #14 0x7fff8fab2de0 in _NSHandleCarbonMenuEvent () #15 0x7fff8f9e9d0d in _DPSNextEvent () #16 0x7fff8f9e8f68 in -[NSApplication nextEventMatchingMask:untilDate:inMode:dequeue:] () #17 0x7fff8f9debf3 in -[NSApplication run] () #18 0x7fff8f95b354 in NSApplicationMain () (This was caught when I clicked in the menu bar after starting my tasks). The remainder of my threads are in various stages of execution, and appear to be what I’d expect. I’m pretty sure this wasn’t happening earlier on in the development of this, so something I’ve done has caused this to change. I’m not explicitly using any locks anywhere, nor any @synchronized blocks. I add my tasks to be executed to my queue using -addOperationWithBlock: The object that hosts that code has a couple of atomic properties because they can be set either from the main thread or read by the task block. So there’s an implied lock there, but I’m not accessing those properties at the time the main thread is blocked here. Probably without posting my entire code it’s likely going to be hard for anyone to pinpoint the issue, but what’s weird is that my main thread really should just be sitting there doing nothing, not waiting on a lock like this. Is there any way to tell in the debugger what other thread it’s waiting on? That might help me to figure out where the hold up is arising at least. —Graham ___ Cocoa-dev mailing list (Cocoa-dev@lists.apple.com) Please do not post admin requests or moderator comments to the list. Contact the moderators at cocoa-dev-admins(at)lists.apple.com Help/Unsubscribe/Update your Subscription: https://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com This email sent to arch...@mail-archive.com
Re: Spinning the busy indicator
On Apr 29, 2015, at 18:22 , Graham Cox graham@bigpond.com wrote: The indicator shows and hides correctly, but most of the time it doesn’t actually spin. It does sometimes, but mostly it doesn’t. I’m wondering if there’s something I need to do to keep it going that I’m not doing (I’m not doing anything other than the above). I believe it’s been quite a long time since an app’s threading structure had any effect on progress animation. Did you set your progress indicator to indeterminate? If not, it’s not going to animate. ___ Cocoa-dev mailing list (Cocoa-dev@lists.apple.com) Please do not post admin requests or moderator comments to the list. Contact the moderators at cocoa-dev-admins(at)lists.apple.com Help/Unsubscribe/Update your Subscription: https://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com This email sent to arch...@mail-archive.com
