Re: Core Data memory not freed after reset
On 9/21/09 4:21 PM, Ben Trumbull said: If you're using an NSArrayController in Entity mode, you can turn on Use Lazy Fetching. You'll want to disable auto-rearrange content. Ben, May I ask, why turn off 'auto-rearrange content'? Is it not compatible with 'use lazy fetching'? Or does 'auto-rearrange content' on its own degrade performance somehow? Thanks, -- Sean McBride, B. Eng s...@rogue-research.com Rogue Researchwww.rogue-research.com Mac Software Developer Montréal, Québec, Canada ___ Cocoa-dev mailing list (Cocoa-dev@lists.apple.com) Please do not post admin requests or moderator comments to the list. Contact the moderators at cocoa-dev-admins(at)lists.apple.com Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com This email sent to arch...@mail-archive.com
Re: Core Data memory not freed after reset
On Sep 22, 2009, at 8:54 AM, Sean McBride wrote: On 9/21/09 4:21 PM, Ben Trumbull said: If you're using an NSArrayController in Entity mode, you can turn on Use Lazy Fetching. You'll want to disable auto-rearrange content. Ben, May I ask, why turn off 'auto-rearrange content'? Is it not compatible with 'use lazy fetching'? Or does 'auto-rearrange content' on its own degrade performance somehow? auto-rearrange content is very expensive. Preserve selection can also be expensive, although not nearly as bad. But if we're talking about a million object table view (which is a little odd, btw, most UIs have ... and more instead) then extraneous layout options will add up fast. I have a vague and hazy memory that auto-rearrange content is not compatible with use lazy fetching. If you run into trouble, file a bug, and disable auto-rearrange content as a workaround. - Ben ___ Cocoa-dev mailing list (Cocoa-dev@lists.apple.com) Please do not post admin requests or moderator comments to the list. Contact the moderators at cocoa-dev-admins(at)lists.apple.com Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com This email sent to arch...@mail-archive.com
Re: Core Data memory not freed after reset
On 9/22/09 1:59 PM, Ben Trumbull said: If you're using an NSArrayController in Entity mode, you can turn on Use Lazy Fetching. You'll want to disable auto-rearrange content. Ben, May I ask, why turn off 'auto-rearrange content'? Is it not compatible with 'use lazy fetching'? Or does 'auto-rearrange content' on its own degrade performance somehow? auto-rearrange content is very expensive. Preserve selection can also be expensive, although not nearly as bad. But if we're talking about a million object table view (which is a little odd, btw, most UIs have ... and more instead) then extraneous layout options will add up fast. Thanks for this info. I have a vague and hazy memory that auto-rearrange content is not compatible with use lazy fetching. If you run into trouble, file a bug, and disable auto-rearrange content as a workaround. I asked because I've just recently turned 'auto-rearrange content' ON in most of my array controllers (so that my tableviews stay sorted when the user changes some string). My tables rarely have more that 10 rows though. I've not yet tried lazy fetching but am now forewarned. :) Thanks, -- Sean McBride, B. Eng s...@rogue-research.com Rogue Researchwww.rogue-research.com Mac Software Developer Montréal, Québec, Canada ___ Cocoa-dev mailing list (Cocoa-dev@lists.apple.com) Please do not post admin requests or moderator comments to the list. Contact the moderators at cocoa-dev-admins(at)lists.apple.com Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com This email sent to arch...@mail-archive.com
Re: Core Data memory not freed after reset
On Sep 21, 2009, at 05:40, Sebastian Morsch wrote: in my SQLite backed Core Data app, a search action fetches from a large number of objects (1.000.000) only to show them in a table. When the user exits search mode (search string empty), I'd like to free the managed objects to restore the app's normal memory footprint. I do that by resetting the managed context, but it doesn't seem to work, physical memory stays where it was when the fetch was completed. Strangely, in ObjectAlloc instrument I can see the fetched objects being deallocated, but the physical memory still peaks. Core Data has (or, I should say, had, since I haven't investigated the behavior in Snow Leopard) its own internal in-memory cache of object and attribute data, which means that, up to a point, data from a persistent store is in memory twice. AFAICT there's no way of unloading or controlling this cache, which has a maximum size that Core Data chooses. So you can get back the memory occupied by the objects themselves (and their attribute value objects), but your memory footprint could stick at a couple of hundred MB. You could just ignore the issue (the footprint won't grow beyond certain point, so it's sort of harmless), or try some of the fetch performance optimization techniques described in the Core Data documentation to see if you can keep unwanted information out of the cache from the beginning. ___ Cocoa-dev mailing list (Cocoa-dev@lists.apple.com) Please do not post admin requests or moderator comments to the list. Contact the moderators at cocoa-dev-admins(at)lists.apple.com Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com This email sent to arch...@mail-archive.com
re: Core Data memory not freed after reset
in my SQLite backed Core Data app, a search action fetches from a large number of objects (1.000.000) only to show them in a table. When the user exits search mode (search string empty), I'd like to free the managed objects to restore the app's normal memory footprint. I do that by resetting the managed context, but it doesn't seem to work, physical memory stays where it was when the fetch was completed. Strangely, in ObjectAlloc instrument I can see the fetched objects being deallocated, but the physical memory still peaks. If ObjectAlloc shows the fetched objects being deallocated, but top shows a large RSIZE, and the heap command shows a very large heap thats most unused, then you're doing everything you can on the deallocation side. There is a difference between heap size, and VM allocated address space. This issue is caught at the boundary, where the malloc system won't aggressively deallocate address space after the memory using it has been deallocated. It doesn't, because as a general rule that's a net loss. So your only choice is to either (a) not worry about it or (b) reduce peak memory. (b) reducing the heap high watermark is a separate problem than simply freeing all the memory you allocate. Reducing peak memory will have many other performance benefits besides making your RSIZE look pretty. If you're using an NSArrayController in Entity mode, you can turn on Use Lazy Fetching. You'll want to disable auto-rearrange content. This works on 10.5. On 10.6 and iPhoneOS, you have more options, including using Batched Fetching on the fetch request with - setFetchBatchSize. This will make a vast reduction in peak memory when working with such a large result set. Working with 1 million objects like this will take ~16MB of RAM. - Ben ___ Cocoa-dev mailing list (Cocoa-dev@lists.apple.com) Please do not post admin requests or moderator comments to the list. Contact the moderators at cocoa-dev-admins(at)lists.apple.com Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com This email sent to arch...@mail-archive.com
Re: Core Data memory not freed after reset
Core Data has (or, I should say, had, since I haven't investigated the behavior in Snow Leopard) its own internal in-memory cache of object and attribute data, which means that, up to a point, data from a persistent store is in memory twice. AFAICT there's no way of unloading or controlling this cache, which has a maximum size that Core Data chooses. So you can get back the memory occupied by the objects themselves (and their attribute value objects), but your memory footprint could stick at a couple of hundred MB. You could just ignore the issue (the footprint won't grow beyond certain point, so it's sort of harmless), or try some of the fetch performance optimization techniques described in the Core Data documentation to see if you can keep unwanted information out of the cache from the beginning. The caching is happening at the NSPersistentStoreCoordinator level, and is shared copy-on-write with the managed objects themselves (e.g. faulting the same object in multiple MOCs will reuse the same string data). The raw column data is not duplicated in memory. The cache entries are deallocated after the managed object representing that row is deallocated, and at certain other times involving faulting. Simply releasing the last managed object representing that row is enough. You may need to call -processPendingChanges to purge anything that as accumulated as ready to be disposed to encourage this to happen sooner if you find it's taking too long. - Ben ___ Cocoa-dev mailing list (Cocoa-dev@lists.apple.com) Please do not post admin requests or moderator comments to the list. Contact the moderators at cocoa-dev-admins(at)lists.apple.com Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com This email sent to arch...@mail-archive.com
