Re: Debugging memory leak in NSURLSession with ARC
On 13 Jan 2015, at 01:00, Graham Cox graham@bigpond.com wrote:
I'm still having an issue with this - I think.
I've exhaustively hunted down every leak and memory allocation in my app -
luckily it's a fairly small one, though one that can create many threads -
and have eliminated everything I have control over*
My heap space is still growing over time. I'm having a LOT of trouble
understanding how to use Allocations Instrument effectively - I just can't
really understand what it's trying to tell me. I have read the documentation
for Allocations but it's not really much help, because it's hard for me to
'mark generations' when a 'generation' is something that happens as a result
of external factors, not a user-interface action.
I've satisfied myself that possible issues to do with blocks causing
potential retain cycles are definitely not the issue, nor is fumbling my way
using ARC for the first time**.
What I'm left with is either a genuine memory leak that occurs inside the
CFNetwork framework, or an apparent memory leak that isn't one really that I
can safely ignore. I'm just not sure which.
To recap, what my app does is to record chunks of video data to a file using
NSURLSession and NSURLSessionDataTask. These chunks are obtained by parsing a
.m3u8 playlist file. Each 'chunk' is a unique URL which is generated by the
server and persists for a fairly short time. By concatenating the data
returned by each fetch of these URLs, a complete capture of a live stream is
achieved. Overall, this process works great with the nagging problem of
growing memory usage. This growth appears to be somewhat related (but not
exactly correlated with) the amount of data downloaded and recorded. For
example, at first the leak is somewhat larger than what I record, but over
time it becomes quite a bit smaller. However if I record nothing, there is no
leak. What concerns me is that the leak can get large - after a day or so
of running, it's getting up to 3GB for example.
In my NSURLSessionConfiguration, I have turned off the URLCache (set it to
nil) - there's no purpose to caching data for the chunks because they are
only ever accessed once. Similarly I've disabled cookies. What I should be
looking at is a straightforward uncached download of a URL, write that data
to disk and move on. Here's my config setup:
self.configuration = [NSURLSessionConfiguration
ephemeralSessionConfiguration];
NSDictionary* additionalHeaders =
@{@User-Agent:XViPadUserAgentString};
self.configuration.HTTPAdditionalHeaders = additionalHeaders;
self.configuration.HTTPCookieAcceptPolicy =
NSHTTPCookieAcceptPolicyNever;
self.configuration.HTTPCookieStorage = nil;
self.configuration.URLCache = nil;
self.session = [NSURLSession
sessionWithConfiguration:self.configuration];
One small suggestion: I think the frameworks would prefer it if you copied the
ephemeralSessionConfiguration, and then modified the copy. Modifying this
shared config is probably not supported, and while it might work now, might not
in future.
___
Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)
Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com
Help/Unsubscribe/Update your Subscription:
https://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com
This email sent to arch...@mail-archive.com
Re: Debugging memory leak in NSURLSession with ARC
leaks, not bugs. Kevin On 13 Jan 2015, at 10:21, Kevin Meaney k...@yvs.eu.com wrote: On 13 Jan 2015, at 02:51, Graham Cox graham@bigpond.com wrote: On 13 Jan 2015, at 12:21 pm, Roland King r...@rols.org wrote: Did you read the devforums thread I pointed you at a couple of weeks ago? Umm, not sure Roland. I read the blog post by bbum about using Allocations, which is the one you linked in this thread. Did you mean something else? Forgive me, I can't locate the link if so. If you're referring to bbum's post, I read that. I'm assuming that heapshot is now labelled mark generations but otherwise is the same thing. The problem with this in my case is that a generation is a new URL download and that's fired off automatically by either the previous one completing or a timer that's set to a variable time based on the target time of the playlist entry. There's no clear means for me to hit mark generation at exactly the right time. That might not matter all that much in that the process is continuous, so as long as I'm downloading a stream at a fairly steady rate, and hit the button at regular intervals, there should be a reasonable similarity between runs. Doing that, I get inconclusive results. Most of the memory that is left is like this: Snapshot Timestamp Growth # Persistent Generation B 01:32.780.375 2.09 MB 38 VM: Performance tool data2.08 MB 4 0x116816000 01:13.421.801 532.00 KB 0x1162ed000 01:32.145.259 532.00 KB 0x11614 01:23.051.011 532.00 KB 0x1161e5000 01:02.847.030 532.00 KB Which suggests it's memory allocated by Allocations itself. But where I'm checking this over longer time periods isn't in Instruments at all, but in Xcode's memory viewer. Unfortunately that doesn't give me a breakdown, just an overall usage. I went through a similar painful process in early November hunting down bugs. I'm not using NSURLConnection so I can't really comment about that API and I'm working on OS X. This comment and the remarks a little later in this thread referring to Quinn triggered a recollection. In desperation I looked at things using Activity Monitor's memory section and turned on the columns Real Mem, Shared Mem, and Purgeable Mem. In my case what I was seeing as a leak matched with the memory marked as purgeable. Exactly what is meant by purgeable I don't know beyond the obvious. Kevin ___ Cocoa-dev mailing list (Cocoa-dev@lists.apple.com) Please do not post admin requests or moderator comments to the list. Contact the moderators at cocoa-dev-admins(at)lists.apple.com Help/Unsubscribe/Update your Subscription: https://lists.apple.com/mailman/options/cocoa-dev/ktam%40yvs.eu.com This email sent to k...@yvs.eu.com ___ Cocoa-dev mailing list (Cocoa-dev@lists.apple.com) Please do not post admin requests or moderator comments to the list. Contact the moderators at cocoa-dev-admins(at)lists.apple.com Help/Unsubscribe/Update your Subscription: https://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com This email sent to arch...@mail-archive.com
Re: Debugging memory leak in NSURLSession with ARC
On 13 Jan 2015, at 02:51, Graham Cox graham@bigpond.com wrote: On 13 Jan 2015, at 12:21 pm, Roland King r...@rols.org wrote: Did you read the devforums thread I pointed you at a couple of weeks ago? Umm, not sure Roland. I read the blog post by bbum about using Allocations, which is the one you linked in this thread. Did you mean something else? Forgive me, I can't locate the link if so. If you're referring to bbum's post, I read that. I'm assuming that heapshot is now labelled mark generations but otherwise is the same thing. The problem with this in my case is that a generation is a new URL download and that's fired off automatically by either the previous one completing or a timer that's set to a variable time based on the target time of the playlist entry. There's no clear means for me to hit mark generation at exactly the right time. That might not matter all that much in that the process is continuous, so as long as I'm downloading a stream at a fairly steady rate, and hit the button at regular intervals, there should be a reasonable similarity between runs. Doing that, I get inconclusive results. Most of the memory that is left is like this: Snapshot Timestamp Growth # Persistent Generation B 01:32.780.375 2.09 MB 38 VM: Performance tool data 2.08 MB 4 0x116816000 01:13.421.801 532.00 KB 0x1162ed000 01:32.145.259 532.00 KB 0x11614 01:23.051.011 532.00 KB 0x1161e5000 01:02.847.030 532.00 KB Which suggests it's memory allocated by Allocations itself. But where I'm checking this over longer time periods isn't in Instruments at all, but in Xcode's memory viewer. Unfortunately that doesn't give me a breakdown, just an overall usage. I went through a similar painful process in early November hunting down bugs. I'm not using NSURLConnection so I can't really comment about that API and I'm working on OS X. This comment and the remarks a little later in this thread referring to Quinn triggered a recollection. In desperation I looked at things using Activity Monitor's memory section and turned on the columns Real Mem, Shared Mem, and Purgeable Mem. In my case what I was seeing as a leak matched with the memory marked as purgeable. Exactly what is meant by purgeable I don't know beyond the obvious. Kevin ___ Cocoa-dev mailing list (Cocoa-dev@lists.apple.com) Please do not post admin requests or moderator comments to the list. Contact the moderators at cocoa-dev-admins(at)lists.apple.com Help/Unsubscribe/Update your Subscription: https://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com This email sent to arch...@mail-archive.com
Re: Debugging memory leak in NSURLSession with ARC
On 13 Jan 2015, at 8:11 pm, Mike Abdullah mabdul...@karelia.com wrote: One small suggestion: I think the frameworks would prefer it if you copied the ephemeralSessionConfiguration, and then modified the copy. Modifying this shared config is probably not supported, and while it might work now, might not in future. My reading of the docs is that these convenience methods create new objects, not return shared ones. I'll check again, though. --Graham ___ Cocoa-dev mailing list (Cocoa-dev@lists.apple.com) Please do not post admin requests or moderator comments to the list. Contact the moderators at cocoa-dev-admins(at)lists.apple.com Help/Unsubscribe/Update your Subscription: https://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com This email sent to arch...@mail-archive.com
Re: Debugging memory leak in NSURLSession with ARC
On 13 Jan 2015, at 13:05, Graham Cox graham@bigpond.com wrote: On 13 Jan 2015, at 8:11 pm, Mike Abdullah mabdul...@karelia.com wrote: One small suggestion: I think the frameworks would prefer it if you copied the ephemeralSessionConfiguration, and then modified the copy. Modifying this shared config is probably not supported, and while it might work now, might not in future. My reading of the docs is that these convenience methods create new objects, not return shared ones. I'll check again, though. Upon a re-read, I think you’re probably right. The docs for +defaultSessionConfiguration are certainly clear on the subject, and it seems to reasonable to assume +ephemeralSessionConfiguration follows the same behaviour. ___ Cocoa-dev mailing list (Cocoa-dev@lists.apple.com) Please do not post admin requests or moderator comments to the list. Contact the moderators at cocoa-dev-admins(at)lists.apple.com Help/Unsubscribe/Update your Subscription: https://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com This email sent to arch...@mail-archive.com
Re: Debugging memory leak in NSURLSession with ARC
May I just jump in to say that you probably should be discussing this on the macnetworkprog list? There are people on that list (esp. Quinn) who are experts in this area. —Jens ___ Cocoa-dev mailing list (Cocoa-dev@lists.apple.com) Please do not post admin requests or moderator comments to the list. Contact the moderators at cocoa-dev-admins(at)lists.apple.com Help/Unsubscribe/Update your Subscription: https://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com This email sent to arch...@mail-archive.com
Re: Debugging memory leak in NSURLSession with ARC
I'm still having an issue with this - I think.
I've exhaustively hunted down every leak and memory allocation in my app -
luckily it's a fairly small one, though one that can create many threads - and
have eliminated everything I have control over*
My heap space is still growing over time. I'm having a LOT of trouble
understanding how to use Allocations Instrument effectively - I just can't
really understand what it's trying to tell me. I have read the documentation
for Allocations but it's not really much help, because it's hard for me to
'mark generations' when a 'generation' is something that happens as a result of
external factors, not a user-interface action.
I've satisfied myself that possible issues to do with blocks causing potential
retain cycles are definitely not the issue, nor is fumbling my way using ARC
for the first time**.
What I'm left with is either a genuine memory leak that occurs inside the
CFNetwork framework, or an apparent memory leak that isn't one really that I
can safely ignore. I'm just not sure which.
To recap, what my app does is to record chunks of video data to a file using
NSURLSession and NSURLSessionDataTask. These chunks are obtained by parsing a
.m3u8 playlist file. Each 'chunk' is a unique URL which is generated by the
server and persists for a fairly short time. By concatenating the data returned
by each fetch of these URLs, a complete capture of a live stream is achieved.
Overall, this process works great with the nagging problem of growing memory
usage. This growth appears to be somewhat related (but not exactly correlated
with) the amount of data downloaded and recorded. For example, at first the
leak is somewhat larger than what I record, but over time it becomes quite a
bit smaller. However if I record nothing, there is no leak. What concerns me is
that the leak can get large - after a day or so of running, it's getting up
to 3GB for example.
In my NSURLSessionConfiguration, I have turned off the URLCache (set it to nil)
- there's no purpose to caching data for the chunks because they are only ever
accessed once. Similarly I've disabled cookies. What I should be looking at is
a straightforward uncached download of a URL, write that data to disk and move
on. Here's my config setup:
self.configuration = [NSURLSessionConfiguration
ephemeralSessionConfiguration];
NSDictionary* additionalHeaders =
@{@User-Agent:XViPadUserAgentString};
self.configuration.HTTPAdditionalHeaders = additionalHeaders;
self.configuration.HTTPCookieAcceptPolicy =
NSHTTPCookieAcceptPolicyNever;
self.configuration.HTTPCookieStorage = nil;
self.configuration.URLCache = nil;
self.session = [NSURLSession
sessionWithConfiguration:self.configuration];
So my question is, has anyone used these classes and seen something similar
occurring? Is it just one of those things, or am I still doing something
wrong? At this point I'm just not sure what else I can do. Is some of that URL
download being cached anyway, despite me telling it not to bother? Is it just
cache growth I'm seeing and that's being managed elsewhere? Maybe 3GB after a
day or two is acceptable?
*One of the things I tried in desperation is to turn off ARC and go back to
manual memory management. Fact is, I'm just more comfortable with MM at the
moment. I'm not claiming ARC is bad or anything like that, but in trying to
hunt down the issue, I wanted to have as familiar territory as possible. In
fact the manual memory management did allow me to discover a few other leaks
where I'd obviously not told ARC the right thing (e.g. something was not being
released because it created a timer, and the timer was invalidated in -dealloc,
which of course doesn't work, even in MM land), but not related to this leak
problem, which persists.
** See above - there is no ARC now.
On 2 Jan 2015, at 2:45 pm, Graham Cox graham@bigpond.com wrote:
What appears to be amassing are 132KB malloc'd blocks (by the hundreds).
These are created by
HTTPNetStreamInfo::_readStreamClientCallBack(__CFReadStream*, unsigned long),
down in CFNetwork. The stack trace is:
0 libsystem_malloc.dylib malloc_zone_malloc
1 libsystem_malloc.dylib malloc
2 CFNetwork HTTPNetStreamInfo::_readStreamClientCallBack(__CFReadStream*,
unsigned long)
3 CFNetwork
CFNetworkReadStream::_readStreamClientCallBackCallBack(__CFReadStream*,
unsigned long, void*)
4 CoreFoundation _signalEventSync
5 CoreFoundation _cfstream_shared_signalEventSync
6 CoreFoundation __CFRUNLOOP_IS_CALLING_OUT_TO_A_SOURCE0_PERFORM_FUNCTION__
7 CoreFoundation __CFRunLoopDoSources0
8 CoreFoundation __CFRunLoopRun
9 CoreFoundation CFRunLoopRunSpecific
10 CFNetwork +[NSURLConnection(Loader) _resourceLoadLoop:]
11 Foundation __NSThread__main__
12 libsystem_pthread.dylib
Re: Debugging memory leak in NSURLSession with ARC
Did you read the devforums thread I pointed you at a couple of weeks ago? I
noted it was iOS not OSX however my general belief is as time goes by, more and
more code is common to the platforms so if there’s a bug in iOS at 8.x (and
there is) it may also exist on OSX at some recent version. On that thread
someone was doing what you’re doing, scheduling new NSURL requests at the
completion of the previous one and they were getting constant memory growth.
I’d also suggest at this point after tearing things apart for quite a while and
putting them back together again, you might be best served asking DTS for help.
If you’re lucky Quinn will pick up the ticket, but even if he doesn’t someone
should actually be able to help you make sense of your allocations traces and
tell you if there is a current, open bug in NSURLSession.
On 13 Jan 2015, at 09:00, Graham Cox graham@bigpond.com wrote:
I'm still having an issue with this - I think.
I've exhaustively hunted down every leak and memory allocation in my app -
luckily it's a fairly small one, though one that can create many threads -
and have eliminated everything I have control over*
My heap space is still growing over time. I'm having a LOT of trouble
understanding how to use Allocations Instrument effectively - I just can't
really understand what it's trying to tell me. I have read the documentation
for Allocations but it's not really much help, because it's hard for me to
'mark generations' when a 'generation' is something that happens as a result
of external factors, not a user-interface action.
I've satisfied myself that possible issues to do with blocks causing
potential retain cycles are definitely not the issue, nor is fumbling my way
using ARC for the first time**.
What I'm left with is either a genuine memory leak that occurs inside the
CFNetwork framework, or an apparent memory leak that isn't one really that I
can safely ignore. I'm just not sure which.
To recap, what my app does is to record chunks of video data to a file using
NSURLSession and NSURLSessionDataTask. These chunks are obtained by parsing a
.m3u8 playlist file. Each 'chunk' is a unique URL which is generated by the
server and persists for a fairly short time. By concatenating the data
returned by each fetch of these URLs, a complete capture of a live stream is
achieved. Overall, this process works great with the nagging problem of
growing memory usage. This growth appears to be somewhat related (but not
exactly correlated with) the amount of data downloaded and recorded. For
example, at first the leak is somewhat larger than what I record, but over
time it becomes quite a bit smaller. However if I record nothing, there is no
leak. What concerns me is that the leak can get large - after a day or so
of running, it's getting up to 3GB for example.
In my NSURLSessionConfiguration, I have turned off the URLCache (set it to
nil) - there's no purpose to caching data for the chunks because they are
only ever accessed once. Similarly I've disabled cookies. What I should be
looking at is a straightforward uncached download of a URL, write that data
to disk and move on. Here's my config setup:
self.configuration = [NSURLSessionConfiguration
ephemeralSessionConfiguration];
NSDictionary* additionalHeaders =
@{@User-Agent:XViPadUserAgentString};
self.configuration.HTTPAdditionalHeaders = additionalHeaders;
self.configuration.HTTPCookieAcceptPolicy =
NSHTTPCookieAcceptPolicyNever;
self.configuration.HTTPCookieStorage = nil;
self.configuration.URLCache = nil;
self.session = [NSURLSession
sessionWithConfiguration:self.configuration];
So my question is, has anyone used these classes and seen something similar
occurring? Is it just one of those things, or am I still doing something
wrong? At this point I'm just not sure what else I can do. Is some of that
URL download being cached anyway, despite me telling it not to bother? Is it
just cache growth I'm seeing and that's being managed elsewhere? Maybe 3GB
after a day or two is acceptable?
*One of the things I tried in desperation is to turn off ARC and go back to
manual memory management. Fact is, I'm just more comfortable with MM at the
moment. I'm not claiming ARC is bad or anything like that, but in trying to
hunt down the issue, I wanted to have as familiar territory as possible. In
fact the manual memory management did allow me to discover a few other leaks
where I'd obviously not told ARC the right thing (e.g. something was not
being released because it created a timer, and the timer was invalidated in
-dealloc, which of course doesn't work, even in MM land), but not related to
this leak problem, which persists.
** See above - there is no ARC now.
On 2 Jan 2015, at 2:45 pm,
Re: Debugging memory leak in NSURLSession with ARC
https://devforums.apple.com/message/1056669#1056669 No that one from the same mail. On 13 Jan 2015, at 10:51, Graham Cox graham@bigpond.com wrote: On 13 Jan 2015, at 12:21 pm, Roland King r...@rols.org wrote: Did you read the devforums thread I pointed you at a couple of weeks ago? Umm, not sure Roland. I read the blog post by bbum about using Allocations, which is the one you linked in this thread. Did you mean something else? Forgive me, I can't locate the link if so. If you're referring to bbum's post, I read that. I'm assuming that heapshot is now labelled mark generations but otherwise is the same thing. The problem with this in my case is that a generation is a new URL download and that's fired off automatically by either the previous one completing or a timer that's set to a variable time based on the target time of the playlist entry. There's no clear means for me to hit mark generation at exactly the right time. That might not matter all that much in that the process is continuous, so as long as I'm downloading a stream at a fairly steady rate, and hit the button at regular intervals, there should be a reasonable similarity between runs. Doing that, I get inconclusive results. Most of the memory that is left is like this: SnapshotTimestampGrowth# Persistent Generation B01:32.780.3752.09 MB38 VM: Performance tool data2.08 MB4 0x11681600001:13.421.801532.00 KB 0x1162ed00001:32.145.259532.00 KB 0x1161401:23.051.011532.00 KB 0x1161e500001:02.847.030532.00 KB Which suggests it's memory allocated by Allocations itself. But where I'm checking this over longer time periods isn't in Instruments at all, but in Xcode's memory viewer. Unfortunately that doesn't give me a breakdown, just an overall usage. --Graham ___ Cocoa-dev mailing list (Cocoa-dev@lists.apple.com) Please do not post admin requests or moderator comments to the list. Contact the moderators at cocoa-dev-admins(at)lists.apple.com Help/Unsubscribe/Update your Subscription: https://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com This email sent to arch...@mail-archive.com
Re: Debugging memory leak in NSURLSession with ARC
On 13 Jan 2015, at 12:21 pm, Roland King r...@rols.org wrote: Did you read the devforums thread I pointed you at a couple of weeks ago? Umm, not sure Roland. I read the blog post by bbum about using Allocations, which is the one you linked in this thread. Did you mean something else? Forgive me, I can't locate the link if so. If you're referring to bbum's post, I read that. I'm assuming that heapshot is now labelled mark generations but otherwise is the same thing. The problem with this in my case is that a generation is a new URL download and that's fired off automatically by either the previous one completing or a timer that's set to a variable time based on the target time of the playlist entry. There's no clear means for me to hit mark generation at exactly the right time. That might not matter all that much in that the process is continuous, so as long as I'm downloading a stream at a fairly steady rate, and hit the button at regular intervals, there should be a reasonable similarity between runs. Doing that, I get inconclusive results. Most of the memory that is left is like this: SnapshotTimestamp Growth # Persistent Generation B01:32.780.375 2.09 MB 38 VM: Performance tool data 2.08 MB 4 0x116816000 01:13.421.801 532.00 KB 0x1162ed000 01:32.145.259 532.00 KB 0x11614 01:23.051.011 532.00 KB 0x1161e5000 01:02.847.030 532.00 KB Which suggests it's memory allocated by Allocations itself. But where I'm checking this over longer time periods isn't in Instruments at all, but in Xcode's memory viewer. Unfortunately that doesn't give me a breakdown, just an overall usage. --Graham ___ Cocoa-dev mailing list (Cocoa-dev@lists.apple.com) Please do not post admin requests or moderator comments to the list. Contact the moderators at cocoa-dev-admins(at)lists.apple.com Help/Unsubscribe/Update your Subscription: https://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com This email sent to arch...@mail-archive.com
Re: Debugging memory leak in NSURLSession with ARC
Thanks - sorry I missed it in the first mail for some reason.
An interesting thread. This remark from Quinn stood out for me:
If you stop issuing new requests, NSURL{Session,Connection} quickly recovers
this memory to the point where, at the end of a cycle like this, the memory use
(as shown by Allocations) is pretty much the same as when you started (a
generational analysis shows just a few KiB of growth). So the problem here is
not a leak, or even abandoned memory, but rather a delay in recovering memory.
This could be what I'm seeing as well, since as a stream is downloaded, each
chunk is pretty much requested as soon as the previous one completes, and is
triggered by that completion (though I am rescheduling it on the main thread,
which in turn adds it to the session's operation queue).
What's not clear is what is needed to cause the memory recovery to occur. I
presume that invalidating the session will do that, but that isn't appropriate
for in-between chunks.
I'll look into this angle and see if I can prove anything. If so, I'll file a
radar and hope it comes back as dupe, fix imminent :) (though I won't be
holding my breath)
--Graham
On 13 Jan 2015, at 2:01 pm, Roland King r...@rols.org wrote:
https://devforums.apple.com/message/1056669#1056669
No that one from the same mail.
On 13 Jan 2015, at 10:51, Graham Cox graham@bigpond.com wrote:
On 13 Jan 2015, at 12:21 pm, Roland King r...@rols.org wrote:
Did you read the devforums thread I pointed you at a couple of weeks ago?
Umm, not sure Roland. I read the blog post by bbum about using Allocations,
which is the one you linked in this thread. Did you mean something else?
Forgive me, I can't locate the link if so.
If you're referring to bbum's post, I read that. I'm assuming that
heapshot is now labelled mark generations but otherwise is the same
thing. The problem with this in my case is that a generation is a new URL
download and that's fired off automatically by either the previous one
completing or a timer that's set to a variable time based on the target
time of the playlist entry. There's no clear means for me to hit mark
generation at exactly the right time. That might not matter all that much
in that the process is continuous, so as long as I'm downloading a stream at
a fairly steady rate, and hit the button at regular intervals, there should
be a reasonable similarity between runs.
Doing that, I get inconclusive results. Most of the memory that is left is
like this:
SnapshotTimestampGrowth# Persistent
Generation B01:32.780.3752.09 MB38
VM: Performance tool data2.08 MB4
0x11681600001:13.421.801532.00 KB
0x1162ed00001:32.145.259532.00 KB
0x1161401:23.051.011532.00 KB
0x1161e500001:02.847.030532.00 KB
Which suggests it's memory allocated by Allocations itself.
But where I'm checking this over longer time periods isn't in Instruments at
all, but in Xcode's memory viewer. Unfortunately that doesn't give me a
breakdown, just an overall usage.
--Graham
___
Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)
Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com
Help/Unsubscribe/Update your Subscription:
https://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com
This email sent to arch...@mail-archive.com
Re: Debugging memory leak in NSURLSession with ARC
On 13 Jan 2015, at 11:20, Graham Cox graham@bigpond.com wrote:
Thanks - sorry I missed it in the first mail for some reason.
An interesting thread. This remark from Quinn stood out for me:
If you stop issuing new requests, NSURL{Session,Connection} quickly recovers
this memory to the point where, at the end of a cycle like this, the memory
use (as shown by Allocations) is pretty much the same as when you started (a
generational analysis shows just a few KiB of growth). So the problem here
is not a leak, or even abandoned memory, but rather a delay in recovering
memory.
This could be what I'm seeing as well, since as a stream is downloaded, each
chunk is pretty much requested as soon as the previous one completes, and is
triggered by that completion (though I am rescheduling it on the main thread,
which in turn adds it to the session's operation queue).
Yes I saw that in the code you posted way up there in the thread sometime and
that’s what made me think of that devforums thread which I have no idea why I
was reading in the first place. But indeed the recollection I had was that the
poster in that thread was also chaining new requests to the completion of old
ones and that appeared to be triggering the memory growth behaviour in his app.
What's not clear is what is needed to cause the memory recovery to occur. I
presume that invalidating the session will do that, but that isn't
appropriate for in-between chunks.
I think ‘taking a break’ is probably what’s necessary to cause the memory
recovery to occur. Whether that means delaying 1 second between every request,
or doing a batch of 50 and then not issuing another request for 1/2 a minute I
have no idea, but it’s probably pretty simple to put in code which does N
requests and then waits T seconds before doing the next one then trying some
values of N and T to see if it does anything at all.
I'll look into this angle and see if I can prove anything. If so, I'll file a
radar and hope it comes back as dupe, fix imminent :) (though I won't be
holding my breath)
well you might get one of your hopes there :)
___
Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)
Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com
Help/Unsubscribe/Update your Subscription:
https://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com
This email sent to arch...@mail-archive.com
Re: Debugging memory leak in NSURLSession with ARC
On Jan 1, 2015, at 3:38 PM, Graham Cox graham@bigpond.com wrote: Hi all, I know I'm very, very late to the party, but I'm building my first ever project with ARC instead of manual retain/release (which I was always very comfortable with). Frankly, I'm finding it frustrating because it seems much harder to know how memory is managed. Anyway. I will plough on - no doubt I'll get used to it. The first thing to keep in mind is that unless otherwise qualified, all object references are owned - locals, ivars, array entries, etc. The places where this isn’t possible (such as structs) are flagged as compiler errors. The lifetime of ownership can typically be considered as “usage scoped” - that is, the object is owned up until its last reference, where by default you can think of it as released. There are exceptions to both of these rules, but they are generally minor and many likely line up with your expectations as well. The most prominent exception has to do with inner pointers to things that are not Obj-C objects (such as -[UIColor CGColor]) where ARC doesn’t take ownership of the returned object (because it isn’t an Obj-C object) and your last reference to the outer object (a UIColor in this case) ends before your usage of the inner non-object. But thats just to help you understand/get used to ARC, it doesn’t likely apply to your issue at hand :). I'm using NSURLSession and NSURLSessionDataTask to fetch chunks of data from a .m3u8 playlist. It works pretty well, but I'm seeing a gradual increase in memory usage as my app runs. Under classical memory management, I would probably have little difficulty in isolating the problem, but with ARC I'm finding it much harder to track down because it's unclear when certain objects come to the end of their lives. Anyway, running in Instruments shows a constant list of calls to malloc() from HTTPNetStreamInfo::_readStreamClientCallBack(__CFReadStream*, unsigned long) in the CFNetwork library, allocating 132KB. These blocks are allocated at a high rate (over 100x per second) as the app runs, and as far as I can see are never freed. I'm not directly using this API, it must be something internal to NSURLSession. How can I ensure that these blocks are freed? I'm assuming that a leak in the OS of this sort would have been spotted, so my assumption here is that I'm doing something wrong that prevents proper freeing. Generally the answer would be to stop referencing objects when you don’t need them anymore, but Instruments’s Allocations tool would be the most useful thing to use to debug this - in particular retain/release tracking would likely be the most useful way to debug this issue. I would suspect the memory blocks at hand are being allocated for receiving the downloaded data and possibly wrapped as NSData objects. The way I'm using NSURLSession seems to be normal, as far as I can tell. I have an object that allocates a NSURLSession and this session exists for as long as the task runs, which can be a very long time. Each chunk of data is added to the session as a NSURLSessionDataTask and I supply a completion block. The completion block simply appends the data it receives to a file and fires off some notifications. The completion block may or may not schedule the download of the following chunk, depending on various factors, but it doesn't do anything special to tell the task it has finished - as far as I can tell from documentation, the fact the completion block is called is because the task has finished, so I can just forget about it. The session itself is invalidated when the owning object is dealloced, but that might be hours or days later when the user decides to discard the task. Otherwise it stays alive indefinitely. I'm wondering if that's the right thing to do? The docs suggest that a NSURLSession is somewhat like a tab in a web browser, so it lives as long as that tab exists. Any clues as to how I can track down the leak or whether there's an obvious missing step that should free temporary buffers used by NSURLSessionDataTask would be gratefully received at this point. --Graham ___ Cocoa-dev mailing list (Cocoa-dev@lists.apple.com) Please do not post admin requests or moderator comments to the list. Contact the moderators at cocoa-dev-admins(at)lists.apple.com Help/Unsubscribe/Update your Subscription: https://lists.apple.com/mailman/options/cocoa-dev/david.duncan%40apple.com This email sent to david.dun...@apple.com -- David Duncan ___ Cocoa-dev mailing list (Cocoa-dev@lists.apple.com) Please do not post admin requests or moderator comments to the list. Contact the moderators at cocoa-dev-admins(at)lists.apple.com Help/Unsubscribe/Update your Subscription: https://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com
Re: Debugging memory leak in NSURLSession with ARC
On Jan 1, 2015, at 17:20 , Graham Cox graham@bigpond.com wrote: I'm using Allocations, but I'm finding the volume of data a bit overwhelming. As an aside, Leaks shows nothing at all. Does that mean I'm not actually leaking anything? Forget Leaks, it’s an utter waste of time*. Use marked generations. The persistent increase in each generation is your real “leak”. — In code where things happen repeatedly in the background, it can be hard to get your app into a quiescent state (in terms of allocations) so that you can find a good place to mark a new generation in Instruments. You may have temporarily add code to create and/or prolong such a quiescent state. — It’s generally not useful to *start* by looking at the retain/release history. You’ll find problem objects easily, but you won’t know where they came from or why they didn’t get released. — In the list of objects incrementally leaked in a generation**, look for the ones that you actually created (either objects of your own classes, or objects that were obviously created at a particular known place in your code). Out of those objects, try to eliminate the ones that appear to be “children” of others. — Ideally, you’ll find two sorta-kinda top-level objects remain. The presumption is then that there’s a reference cycle between them. For example, you might find that one is a window controller and the other is something in your data model***. Or, you might find that one is a block, and the other is whatever created the block. — Audit the relevant source code. It’s often easiest and quickest to find the cycle just by inspecting the declarations/blocks. — Only if that goes nowhere is it time to start analyzing the retain/release history of a generation in Instruments. By now, though, you should have a vague idea of what you’re looking for. — Start by getting Instruments to pair as many retains and releases as it can automatically. — If that’s unproductive, start over and compare them all yourself. (But I should add that I haven’t spent much time doing this in Yosemite yet. The UI may be a bit different since Mavericks, and the auto-pairing may work a bit differently, too.) ** Create multiple generations, and look not only at what was leaked in each generation, but what’s repetitive in the pattern of leaks. Your real culprit should produce an identical pattern every time. Leaks unique to a generation aren’t what you’re looking for here. (I think this is the debugging equivalent of the second derivative.) *** That usually means a window delegate reference needs to be manually nil’ed when the window closes, because we learned how to do bad memory management in the bad old days. That usually means the block and the ‘self’ it captured mutually refer to each other. I’m betting this is what’s wrong in your case. * I should say, I’m not scorning Leaks. It’s just that experienced Obj-C developers tend not to make the kinds of mistakes that Leaks can detect. Experienced developers make other, better mistakes. ___ Cocoa-dev mailing list (Cocoa-dev@lists.apple.com) Please do not post admin requests or moderator comments to the list. Contact the moderators at cocoa-dev-admins(at)lists.apple.com Help/Unsubscribe/Update your Subscription: https://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com This email sent to arch...@mail-archive.com
Re: Debugging memory leak in NSURLSession with ARC
On 2 Jan 2015, at 10:52 am, David Duncan david.dun...@apple.com wrote: The first thing to keep in mind is that unless otherwise qualified, all object references are owned - locals, ivars, array entries, etc. The places where this isn’t possible (such as structs) are flagged as compiler errors. The lifetime of ownership can typically be considered as “usage scoped” - that is, the object is owned up until its last reference, where by default you can think of it as released. There are exceptions to both of these rules, but they are generally minor and many likely line up with your expectations as well. Thanks David, that does agree with the way I've been thinking about memory management with ARC, so I guess I'm probably not too far wrong. I don't think I'm running into any of those corner cases - everything is an NSObject, no CF stuff (in my code), and no structs, etc. Generally the answer would be to stop referencing objects when you don’t need them anymore, but Instruments’s Allocations tool would be the most useful thing to use to debug this - in particular retain/release tracking would likely be the most useful way to debug this issue. I would suspect the memory blocks at hand are being allocated for receiving the downloaded data and possibly wrapped as NSData objects. I'm using Allocations, but I'm finding the volume of data a bit overwhelming. As an aside, Leaks shows nothing at all. Does that mean I'm not actually leaking anything? What I'm seeing sort of agrees with expectations - the memory usage graph has a broadly sawtooth waveform as each download task is created, builds up, is processed and released. However the underlying memory usage continues to climb, with the sawtooth on top. Breaking down the allocations shows that the 132KB bufers are the vast majority of the allocations, but maybe my interpretation that they're not being freed is wrong - they are listed but maybe that doesn't mean they're not being freed? I don't know - there's the problem, I'm not terribly clear on how to interpret the data. Regarding the NSData I'm passed in my NSURLSessionDataTask completion block (which are the biggest blocks I handle), I write that data to an open file handle and that's the last time I reference it. I don't own that NSData anyway, so even under classic MM I wouldn't do anything special about it. I guess NSFileHandle could be retaining that NSData. I leave that file handle open continually as long as a download stream is available, so it might be that it's not releasing NSData blocks passed to -writeData: until it is closed. That could explain what I'm seeing, though thinking about *why* NSFileHandle would retain that data is not so obvious - why indeed? Wouldn't it just copy that data to the file and release it? --Graham ___ Cocoa-dev mailing list (Cocoa-dev@lists.apple.com) Please do not post admin requests or moderator comments to the list. Contact the moderators at cocoa-dev-admins(at)lists.apple.com Help/Unsubscribe/Update your Subscription: https://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com This email sent to arch...@mail-archive.com
Re: Debugging memory leak in NSURLSession with ARC
A while back I had a similar sounding issue in an iOS project I was working on. The memory usage kept growing, although I could find no obvious culprits in my code after hours of exploration. My solution probably doesn’t pertain to you, but I thought I’d pass it along just in case it’s something you might have overlooked. I showed my project to another more experienced developer seeking help. He agreed with my assessment that there weren’t any apparent major problems with my code. But he discovered that I had left Zombies detection turned on in the project scheme settings (left on from exploring another issue some time back). So of course Instruments was showing me the memory that Zombies was holding onto. Turning off Zombies detection gave me a very steady memory usage pattern. On Jan 1, 2015, at 5:20 PM, Graham Cox graham@bigpond.com wrote: I'm using Allocations, but I'm finding the volume of data a bit overwhelming. As an aside, Leaks shows nothing at all. Does that mean I'm not actually leaking anything? What I'm seeing sort of agrees with expectations - the memory usage graph has a broadly sawtooth waveform as each download task is created, builds up, is processed and released. However the underlying memory usage continues to climb, with the sawtooth on top. Breaking down the allocations shows that the 132KB bufers are the vast majority of the allocations, but maybe my interpretation that they're not being freed is wrong - they are listed but maybe that doesn't mean they're not being freed? I don't know - there's the problem, I'm not terribly clear on how to interpret the data. Regarding the NSData I'm passed in my NSURLSessionDataTask completion block (which are the biggest blocks I handle), I write that data to an open file handle and that's the last time I reference it. I don't own that NSData anyway, so even under classic MM I wouldn't do anything special about it. I guess NSFileHandle could be retaining that NSData. I leave that file handle open continually as long as a download stream is available, so it might be that it's not releasing NSData blocks passed to -writeData: until it is closed. That could explain what I'm seeing, though thinking about *why* NSFileHandle would retain that data is not so obvious - why indeed? Wouldn't it just copy that data to the file and release it? ___ Cocoa-dev mailing list (Cocoa-dev@lists.apple.com) Please do not post admin requests or moderator comments to the list. Contact the moderators at cocoa-dev-admins(at)lists.apple.com Help/Unsubscribe/Update your Subscription: https://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com This email sent to arch...@mail-archive.com
Re: Debugging memory leak in NSURLSession with ARC
On 1 jan 2015, at 18:26, Graham Cox graham@bigpond.com wrote:
On 2 Jan 2015, at 12:48 pm, Quincey Morris
quinceymor...@rivergatesoftware.com wrote:
That usually means the block and the ‘self’ it captured mutually refer
to each other. I’m betting this is what’s wrong in your case.
Quincey, thanks for your lengthy and well-thought-out reply (as usual) :)
I think you've hit the nail straight on the head (as usual), with the
block/self problem. It's one I knew about before, but had forgotten again in
my excitement at getting the code running. The question is what to do about
it.
My handler block refers to 'self' quite extensively - it calls other methods
of self and also refers to properties such as self.delegate. I'm not quite
sure how I can rework it not to refer to self. Maybe I just need to not use
the completion block approach and use a delegate callback instead. I need to
go away and think about this... thanks for the slap about the head.
You can always employ the weakself+strongself pattern:
__weak Foo *weakSelf = self;
AsyncBar(^{
Foo *strongSelf = weakSelf;
if (nil != strongSelf) {
[strongSelf doStuff];
// more stuff
}
});
That said, if an object is going away it's typically better to use explicit
cleanup / teardown, than relying on __weak. Tear down bindings, KVO, timers,
delayed runloop invocations, outstanding async work, etc. More direct, more
predictable, easier to troubleshoot/debug.
Joar
___
Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)
Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com
Help/Unsubscribe/Update your Subscription:
https://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com
This email sent to arch...@mail-archive.com
Re: Debugging memory leak in NSURLSession with ARC
On 1 jan 2015, at 18:22, Roland King r...@rols.org wrote: +1 for all of this. I wouldn't call leaks an utter waste of time, but it really does only find pure retain cycles (which it then annotates very nicely) and not memory which is really is pinned by a real reference which is more often the case. Also, if you're using KVO anywhere, this tends to entirely defeat leaks even though KVO isn't a strong reference and shouldn't be treated like one, it is. That makes leaks less useful than it could be for me because I'm always KVO'ing something. I don't think leaks would find cycles. I could be wrong, but I don't think it's sophisticated enough to perform that additional level of analysis. I think it only finds stuff that's truly referenced from nowhere else on the heap. There's an old blog by bbum which covered using generational analysis for finding leaks which aren't leaks. Xcode has changed quite a lot since then but some of the screens look quite similar even now. I routinely put my code through this torture test, it's so easy to run and the results are often very illuminating. http://www.friday.com/bbum/2010/10/17/when-is-a-leak-not-a-leak-using-heapshot-analysis-to-find-undesirable-memory-growth/http://www.friday.com/bbum/2010/10/17/when-is-a-leak-not-a-leak-using-heapshot-analysis-to-find-undesirable-memory-growth/ That functionality has since been integrated into Instruments, so it's easier to use. Joar ___ Cocoa-dev mailing list (Cocoa-dev@lists.apple.com) Please do not post admin requests or moderator comments to the list. Contact the moderators at cocoa-dev-admins(at)lists.apple.com Help/Unsubscribe/Update your Subscription: https://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com This email sent to arch...@mail-archive.com
Re: Debugging memory leak in NSURLSession with ARC
On 2 Jan 2015, at 10:52, Joar Wingfors j...@joar.com wrote: On 1 jan 2015, at 18:22, Roland King r...@rols.org wrote: +1 for all of this. I wouldn't call leaks an utter waste of time, but it really does only find pure retain cycles (which it then annotates very nicely) and not memory which is really is pinned by a real reference which is more often the case. Also, if you're using KVO anywhere, this tends to entirely defeat leaks even though KVO isn't a strong reference and shouldn't be treated like one, it is. That makes leaks less useful than it could be for me because I'm always KVO'ing something. I don't think leaks would find cycles. I could be wrong, but I don't think it's sophisticated enough to perform that additional level of analysis. I think it only finds stuff that's truly referenced from nowhere else on the heap. Perhaps we're talking cross purposes because it definitely finds what I'm calling cycles, where object A references B strongly and B references A strongly (or a longer chain of objects which eventually forms an equivalent loop). If that loop of objects isn't strongly referenced by anything else and is thus 'leaked', leaks not only finds it but gives you a pretty diagram showing how all the objects are strongly referencing each other, with property names etc. IIRC that facility arrived the same year as ARC. It's occasionally useful. ___ Cocoa-dev mailing list (Cocoa-dev@lists.apple.com) Please do not post admin requests or moderator comments to the list. Contact the moderators at cocoa-dev-admins(at)lists.apple.com Help/Unsubscribe/Update your Subscription: https://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com This email sent to arch...@mail-archive.com
Re: Debugging memory leak in NSURLSession with ARC
On Jan 1, 2015, at 18:26, Graham Cox graham@bigpond.com wrote:
On 2 Jan 2015, at 12:48 pm, Quincey Morris
quinceymor...@rivergatesoftware.com wrote:
That usually means the block and the ‘self’ it captured mutually refer
to each other. I’m betting this is what’s wrong in your case.
Quincey, thanks for your lengthy and well-thought-out reply (as usual) :)
I think you've hit the nail straight on the head (as usual), with the
block/self problem. It's one I knew about before, but had forgotten again in
my excitement at getting the code running. The question is what to do about
it.
My handler block refers to 'self' quite extensively - it calls other methods
of self and also refers to properties such as self.delegate. I'm not quite
sure how I can rework it not to refer to self. Maybe I just need to not use
the completion block approach and use a delegate callback instead. I need to
go away and think about this... thanks for the slap about the head.
—Graham
One of the first things I define in any ARC codebase I’m going to be working on
is:
#define DECLARE_WEAK_SELF __weak __typeof__(self) weak_self = self;
#define DECLARE_STRONG_SELF__strong __typeof__(weak_self) self = weak_self;
And I use it like so:
- (void)method {
DECLARE_WEAK_SELF
self.someBlock = ^{
DECLARE_STRONG_SELF;
//Use “self” as usual here
};
self.someOtherBlock = ^{
DECLARE_STRONG_SELF;
//Use “self” as usual here
};
}
--
Clark Smith Cox III
clarkc...@gmail.com
smime.p7s
Description: S/MIME cryptographic signature
___
Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)
Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com
Help/Unsubscribe/Update your Subscription:
https://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com
This email sent to arch...@mail-archive.com
Re: Debugging memory leak in NSURLSession with ARC
My handler block refers to 'self' quite extensively - it calls other methods
of self and also refers to properties such as self.delegate. I'm not quite
sure how I can rework it not to refer to self. Maybe I just need to not use
the completion block approach and use a delegate callback instead. I need to
go away and think about this... thanks for the slap about the head.
--Graham
Having a handler block which refers to self is not in and of itself a problem,
very many blocks implicitly do. The block retains self, however in most cases
something else retains the block and the self reference goes away when the
block is released. The problem usually comes when the handler block which
refers to self is also a property of the object, eg myObj.completion = ^{ ..
block referring to myobj }. Xcode normally warns you if you even get close to
doing that however.
If that's happening then allocations should show you are amassing whatever
objects those are and never releasing them, does it? Or you can be old skool
about it and NSLog() dealloc() to see if it's getting called.
Two ways around block retain cycles are
1) If the block is a property of the object, eg a callback block, when you've
called the block, nil the property, break the cycle.
2) The strong/weak dance. You make a weak pointer to self which is actually
captured in the block, then in the block you assign it to a strong pointer,
then check for nil, then use that pointer.
MyObj __weak *weakSelf = self;
^{
MyObj *strongSelf = weakSelf;
if( strongSelf )
{
// do work using strongSelf explicitly
}
}
___
Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)
Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com
Help/Unsubscribe/Update your Subscription:
https://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com
This email sent to arch...@mail-archive.com
Re: Debugging memory leak in NSURLSession with ARC
On 2 Jan 2015, at 12:48 pm, Quincey Morris quinceymor...@rivergatesoftware.com wrote: That usually means the block and the ‘self’ it captured mutually refer to each other. I’m betting this is what’s wrong in your case. Quincey, thanks for your lengthy and well-thought-out reply (as usual) :) I think you've hit the nail straight on the head (as usual), with the block/self problem. It's one I knew about before, but had forgotten again in my excitement at getting the code running. The question is what to do about it. My handler block refers to 'self' quite extensively - it calls other methods of self and also refers to properties such as self.delegate. I'm not quite sure how I can rework it not to refer to self. Maybe I just need to not use the completion block approach and use a delegate callback instead. I need to go away and think about this... thanks for the slap about the head. --Graham ___ Cocoa-dev mailing list (Cocoa-dev@lists.apple.com) Please do not post admin requests or moderator comments to the list. Contact the moderators at cocoa-dev-admins(at)lists.apple.com Help/Unsubscribe/Update your Subscription: https://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com This email sent to arch...@mail-archive.com
Re: Debugging memory leak in NSURLSession with ARC
+1 for all of this. I wouldn't call leaks an utter waste of time, but it really does only find pure retain cycles (which it then annotates very nicely) and not memory which is really is pinned by a real reference which is more often the case. Also, if you're using KVO anywhere, this tends to entirely defeat leaks even though KVO isn't a strong reference and shouldn't be treated like one, it is. That makes leaks less useful than it could be for me because I'm always KVO'ing something. There's an old blog by bbum which covered using generational analysis for finding leaks which aren't leaks. Xcode has changed quite a lot since then but some of the screens look quite similar even now. I routinely put my code through this torture test, it's so easy to run and the results are often very illuminating. http://www.friday.com/bbum/2010/10/17/when-is-a-leak-not-a-leak-using-heapshot-analysis-to-find-undesirable-memory-growth/ http://www.friday.com/bbum/2010/10/17/when-is-a-leak-not-a-leak-using-heapshot-analysis-to-find-undesirable-memory-growth/ I do remember there was a recent discussion on the dev forums about NSURLConnection 'leaking' in iOS8. It appears it wasn't actually leaking, but it did amass memory which it gave up in a delayed manner only when you stopped making requests. Again that's iOS, but I do wonder if the same code is in OSX and whether the same effect is there too. That discussion, which doesn't quite reach a conclusion, is here .. https://devforums.apple.com/message/1056669#1056669 https://devforums.apple.com/message/1056669#1056669 On 2 Jan 2015, at 09:48, Quincey Morris quinceymor...@rivergatesoftware.com wrote: On Jan 1, 2015, at 17:20 , Graham Cox graham@bigpond.com wrote: I'm using Allocations, but I'm finding the volume of data a bit overwhelming. As an aside, Leaks shows nothing at all. Does that mean I'm not actually leaking anything? Forget Leaks, it’s an utter waste of time*. Use marked generations. The persistent increase in each generation is your real “leak”. — In code where things happen repeatedly in the background, it can be hard to get your app into a quiescent state (in terms of allocations) so that you can find a good place to mark a new generation in Instruments. You may have temporarily add code to create and/or prolong such a quiescent state. — It’s generally not useful to *start* by looking at the retain/release history. You’ll find problem objects easily, but you won’t know where they came from or why they didn’t get released. — In the list of objects incrementally leaked in a generation**, look for the ones that you actually created (either objects of your own classes, or objects that were obviously created at a particular known place in your code). Out of those objects, try to eliminate the ones that appear to be “children” of others. — Ideally, you’ll find two sorta-kinda top-level objects remain. The presumption is then that there’s a reference cycle between them. For example, you might find that one is a window controller and the other is something in your data model***. Or, you might find that one is a block, and the other is whatever created the block. — Audit the relevant source code. It’s often easiest and quickest to find the cycle just by inspecting the declarations/blocks. — Only if that goes nowhere is it time to start analyzing the retain/release history of a generation in Instruments. By now, though, you should have a vague idea of what you’re looking for. — Start by getting Instruments to pair as many retains and releases as it can automatically. — If that’s unproductive, start over and compare them all yourself. (But I should add that I haven’t spent much time doing this in Yosemite yet. The UI may be a bit different since Mavericks, and the auto-pairing may work a bit differently, too.) ** Create multiple generations, and look not only at what was leaked in each generation, but what’s repetitive in the pattern of leaks. Your real culprit should produce an identical pattern every time. Leaks unique to a generation aren’t what you’re looking for here. (I think this is the debugging equivalent of the second derivative.) *** That usually means a window delegate reference needs to be manually nil’ed when the window closes, because we learned how to do bad memory management in the bad old days. That usually means the block and the ‘self’ it captured mutually refer to each other. I’m betting this is what’s wrong in your case. * I should say, I’m not scorning Leaks. It’s just that experienced Obj-C developers tend not to make the kinds of mistakes that Leaks can detect. Experienced developers make other, better mistakes. ___ Cocoa-dev mailing list (Cocoa-dev@lists.apple.com) Please do not post admin requests or moderator comments to the list.
Re: Debugging memory leak in NSURLSession with ARC
On 2 Jan 2015, at 1:46 pm, Roland King r...@rols.org wrote:
Having a handler block which refers to self is not in and of itself a
problem, very many blocks implicitly do. The block retains self, however in
most cases something else retains the block and the self reference goes away
when the block is released. The problem usually comes when the handler block
which refers to self is also a property of the object, eg myObj.completion =
^{ .. block referring to myobj }. Xcode normally warns you if you even get
close to doing that however.
Right. Well, my block refers to self, but it's not a property of self. Quickly
trying it with a weak version of self shows no change of behaviour regarding
memory usage, so I guess that was a red herring.
Just to be sure, I have:
my
object...[owns]...NSURLSession...[owns]...NSURLSessionDataTask...[owns]...completion
block...[refers to]...my object
I don't think this amounts to a retain cycle, because when the session has
finished the task, it will release it, and its block, and those references to
self (my object) that it has.
If that's happening then allocations should show you are amassing whatever
objects those are and never releasing them, does it? Or you can be old skool
about it and NSLog() dealloc() to see if it's getting called.
What appears to be amassing are 132KB malloc'd blocks (by the hundreds). These
are created by HTTPNetStreamInfo::_readStreamClientCallBack(__CFReadStream*,
unsigned long), down in CFNetwork. The stack trace is:
0 libsystem_malloc.dylib malloc_zone_malloc
1 libsystem_malloc.dylib malloc
2 CFNetwork HTTPNetStreamInfo::_readStreamClientCallBack(__CFReadStream*,
unsigned long)
3 CFNetwork
CFNetworkReadStream::_readStreamClientCallBackCallBack(__CFReadStream*,
unsigned long, void*)
4 CoreFoundation _signalEventSync
5 CoreFoundation _cfstream_shared_signalEventSync
6 CoreFoundation __CFRUNLOOP_IS_CALLING_OUT_TO_A_SOURCE0_PERFORM_FUNCTION__
7 CoreFoundation __CFRunLoopDoSources0
8 CoreFoundation __CFRunLoopRun
9 CoreFoundation CFRunLoopRunSpecific
10 CFNetwork +[NSURLConnection(Loader) _resourceLoadLoop:]
11 Foundation __NSThread__main__
12 libsystem_pthread.dylib _pthread_body
13 libsystem_pthread.dylib _pthread_start
14 libsystem_pthread.dylib thread_start
Which actually doesn't directly link it to anything in my code, though I think
it's safe to say it's something set up by NSURLSession or NSURLSessionDataTask,
which I do create in my code.
BUT, and this is a big but, I'm not sure if I'm interpreting Allocations
correctly. I have it set to created and persistent, and graphing these blocks
shows a very close corrlation between them and the overall memory usage
profile, so my interpretation is that these are predominantly responsible for
the memory usage of my app. However, the shape of the graph does show that a
lot of these blocks are freed, but not all, leading to the gradual increase in
baseline allocation.
Here's the rough outline of the code I'm using to run the
NSURLSessionDataTasks. This is a method of my object, and it is called once
when there is data to begin downloading, and runs itself as long as there are
items in the queue to be processed.
- (BOOL)dequeueNextChunk
{
// removes next URL from the chunks queue and starts it asynchronously
downloading as a session. Returns YES if a chunk was dequeued, NO if not (queue
empty).
XVMediaChunk* chunk = [queuedChunkURLs firstObject];
if( chunk )
{
// dequeue
[queuedChunkURLs removeObjectAtIndex:0];
// schedule download and start it
[[self.session dataTaskWithURL:chunk.url
completionHandler:^(NSData *data, NSURLResponse *response, NSError *error)
{
NSHTTPURLResponse* httpResponse =
(NSHTTPURLResponse*)response;
if( httpResponse.statusCode == 200 )
{
// got a valid response. If it's a valid chunk
add the data to the chunk object and pass it along to the next step.
if([httpResponse.MIMEType
isEqualToString:@video/mp2t])
{
chunk.data = data;
[self processVideoChunk:chunk];
// and schedule the next chunk in the
queue
[self
performSelectorOnMainThread:@selector(dequeueNextChunk) withObject:nil
waitUntilDone:NO];
}
}
else
