Re: Continuously running daemon process CFConstantStringRefs build up over time
> CFStringRef hexStringRef = CFStringCreateWithFormat( NULL, NULL, > CFSTR("%x"), versionValue ); <- Instruments is hi-lighting this line as > an allocation. It sounds like you found the cause of your persisting string objects so the details of CFSTR is irrelevant, but to clarify anyway: the result of CFStringCreateWithFormat() is of course an allocation as Instruments reported, but the format string that you supplied CFStringCreateWithFormat() – that is, the CFString whose value is "%x" – exists statically within your executable (in the __DATA/__cfstring segment/section) and is therefore not an allocation. ___ Cocoa-dev mailing list (Cocoa-dev@lists.apple.com) Please do not post admin requests or moderator comments to the list. Contact the moderators at cocoa-dev-admins(at)lists.apple.com Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com This email sent to arch...@mail-archive.com
Re: Continuously running daemon process CFConstantStringRefs build up over time
Thanks a million Scott, I've started reading the sqlite documentation and it looks like I can adjust the PRAGMA cache_size to let me tweak the size of the cache. I am only writing to the database with this daemon and I'll try to dig up any other options that might help optimize it for this scenario. Those were the largest allocations in my testing and setting the cache to 10 seems to eliminate the outstanding malloc'ed objects. I still have some outstanding allocations from -[NSAutoeleasePool init] and +[NSRunLoop(NSRunLoop) currentRunLoop] that Foundation must be caching, but the memory overhead has drastically reduced under stress-testing. While I'm on the subject, are there any caveats to calling [[NSRunLoop currentRunLoop] run]; in the -(void)main of my NSOperation subclass? In my testing it works wonderfully but I haven't read anything in the docs that say to avoid it. Thanks again for your help, Kevin Ross On Oct 1, 2010, at 1:20 PM, Scott Ribe wrote: > On Oct 1, 2010, at 2:15 PM, Kevin Ross wrote: > >> libsqlite3.dylib mallocs 35 objects that are still considered "live" > > Sqlite manages its own cache. > > -- > Scott Ribe > scott_r...@elevated-dev.com > http://www.elevated-dev.com/ > (303) 722-0567 voice > > > > > ___ Cocoa-dev mailing list (Cocoa-dev@lists.apple.com) Please do not post admin requests or moderator comments to the list. Contact the moderators at cocoa-dev-admins(at)lists.apple.com Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com This email sent to arch...@mail-archive.com
Re: Continuously running daemon process CFConstantStringRefs build up over time
On Oct 1, 2010, at 2:15 PM, Kevin Ross wrote: > libsqlite3.dylib mallocs 35 objects that are still considered "live" Sqlite manages its own cache. -- Scott Ribe scott_r...@elevated-dev.com http://www.elevated-dev.com/ (303) 722-0567 voice ___ Cocoa-dev mailing list (Cocoa-dev@lists.apple.com) Please do not post admin requests or moderator comments to the list. Contact the moderators at cocoa-dev-admins(at)lists.apple.com Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com This email sent to arch...@mail-archive.com
Re: Continuously running daemon process CFConstantStringRefs build up over time
Thanks for your insight Dave, In my current implementation I do have an autorelease pool being created and released at the end of main, as well as inside each posted notification that NSFileHandle posts on each background thread. Upon further inspection the largest culprits are not CFConstantStringRefs but instead non-object allocations inside libsqlite3.dylib being called by sqlite3MemMalloc. The constant strings are indeed building but are not the largest allocations. With one save of the managedObjectContext libsqlite3.dylib mallocs 35 objects that are still considered "live" even after the NSOperation that is the root of the call stack has since been freed. Here is the call-stack for one of the allocations since they all seem to follow the same call stack: 0 libsqlite3.dylib sqlite3MemMalloc 1 libsqlite3.dylib mallocWithAlarm 2 libsqlite3.dylib pcache1Fetch 3 libsqlite3.dylib sqlite3PcacheFetch 4 libsqlite3.dylib sqlite3PagerAcquire 5 libsqlite3.dylib sqlite3BtreeBeginTrans 6 libsqlite3.dylib btreeCursor 7 libsqlite3.dylib sqlite3InitOne 8 libsqlite3.dylib sqlite3Init 9 libsqlite3.dylib sqlite3Pragma 10 libsqlite3.dylib yy_reduce 11 libsqlite3.dylib sqlite3Parser 12 libsqlite3.dylib sqlite3RunParser 13 libsqlite3.dylib sqlite3LockAndPrepare 14 CoreData -[NSSQLiteConnection _executeSQLString:] 15 CoreData -[NSSQLiteConnection _configurePragmaOptions] 16 CoreData -[NSSQLiteConnection connect] 17 CoreData -[NSSQLCore _loadOrSetMetadata] 18 CoreData -[NSSQLCore _ensureMetadataLoaded] 19 CoreData -[NSSQLCore initWithPersistentStoreCoordinator:configurationName:URL:options:] 20 CoreData -[NSPersistentStoreCoordinator addPersistentStoreWithType:configuration:URL:options:error:] 21 MAID -[MIMaidServer persistentStoreCoordinator] /Users/kevin/Desktop/MAID_Workspace/MAID/MAID/Source/MIMaidServer.m:466 22 MAID -[MIMaidServer managedObjectContext] /Users/kevin/Desktop/MAID_Workspace/MAID/MAID/Source/MIMaidServer.m:492 23 MAID -[MIMaidOperation managedObjectContext] /Users/kevin/Desktop/MAID_Workspace/MAID/MAID/Source/MIMaidOperation.m:432 24 MAID -[MIMaidOperation logClientData:] /Users/kevin/Desktop/MAID_Workspace/MAID/MAID/Source/MIMaidOperation.m:242 25 MAID -[MIMaidOperation handleIncommingData:] /Users/kevin/Desktop/MAID_Workspace/MAID/MAID/Source/MIMaidOperation.m:230 26 MAID -[MIMaidOperation dataReadFromClient:] /Users/kevin/Desktop/MAID_Workspace/MAID/MAID/Source/MIMaidOperation.m:210 27 Foundation _nsnote_callback 28 CoreFoundation __CFXNotificationPost 29 CoreFoundation _CFXNotificationPostNotification 30 Foundation -[NSNotificationCenter postNotificationName:object:userInfo:] 31 Foundation _performFileHandleSource 32 CoreFoundation __CFRunLoopDoSources0 33 CoreFoundation __CFRunLoopRun 34 CoreFoundation CFRunLoopRunSpecific 35 Foundation -[NSRunLoop(NSRunLoop) runMode:beforeDate:] 36 Foundation -[NSRunLoop(NSRunLoop) run] 37 MAID -[MIMaidOperation main] /Users/kevin/Desktop/MAID_Workspace/MAID/MAID/Source/MIMaidOperation.m:165 38 Foundation __NSThread__main__ 39 libSystem.B.dylib _pthread_start 40 libSystem.B.dylib thread_start With regard to the CFSTR being static within the executable CFStringRef MICreateVersionStringFromVersionValue( UInt32 versionValue ) { UniChar upperBase = 0; UniChar lowerBase = 0; UniChar major = 0; UniChar minor = 0; CFStringRef hexStringRef = CFStringCreateWithFormat( NULL, NULL, CFSTR("%x"), versionValue ); <- Instruments is hi-lighting this line as an allocation. CFIndex sizeMatters = CFStringGetLength( hexStringRef ); //CFIndex charactersReturned = 0; if ( sizeMatters > 3 ) { CFStringGetCharacters( hexStringRef, CFRangeMake( sizeMatters - 4, 1 ), &upperBase ); } CFStringGetCharacters( hexStringRef, CFRangeMake( sizeMatters - 3, 1 ), &lowerBase ); CFStringGetCharacters( hexStringRef, CFRangeMake( sizeMatters - 2, 1 ), &major ); CFStringGetCharacters( hexStringRef, CFRangeMake( sizeMatters - 1, 1 ), &minor ); CFStringRef versionStringRef = CFStringCreateWithFormat( NULL, NULL, CFSTR("%c%c.%c.%c"), upperBase, lowerBase, major, minor ); <- Instruments is hi-lighting this line as an allocation even though it is being released by the caller of the function. CFRelease( hexStringRef ); return versionStringRef; } The returned versionStringRef is being released from the caller of the function and none of the other tools are reporting this as a leak so I'm not sure why the system is still considers any of the memory "live". Thanks for your help in this matter, it's been driving me crazy for a few days. Sincerely, Kevin Ross On Oct 1, 2010, at 1:00 AM, Dave Keck wrote: > Since you're writing a daemon, you'll need to handle autorelease-pool > creation and draini
Re: Continuously running daemon process CFConstantStringRefs build up over time
Since you're writing a daemon, you'll need to handle autorelease-pool creation and draining manually (something that's normally handled by NSApplication in standard AppKit apps.) Perhaps objects are autoreleased and placed in the root autorelease pool (that you might be creating in main() or the like) which is never drained? Also, how many strings are leaking? I know the frameworks cache NSNumber instances; I'm not sure about immutable strings. > I have run the daemon through the clang static analyzer and the Instruments > leaks tool but none are reporting any leaks. I have even downloaded a fresh > copy of valgrind from svn and it too is not finding anything. The > instruments allocation monitor is reporting that there are > CFConstantStringRefs that Foundation is allocating from internal methods and > CFSTR macros that I am using in some functions. I'm happy to provide more > details of the actual call-stacks and code if necessary. A pedantic detail: note that strings created with CFSTR exist statically within your executable (they aren't dynamically allocated) and therefore aren't leaks. ___ Cocoa-dev mailing list (Cocoa-dev@lists.apple.com) Please do not post admin requests or moderator comments to the list. Contact the moderators at cocoa-dev-admins(at)lists.apple.com Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com This email sent to arch...@mail-archive.com
Continuously running daemon process CFConstantStringRefs build up over time
Greetings Cocoa Developers, I have run into a snag while writing my background daemon and I was hoping that there might be some people on this list that could point me in the right direction or suggest a workaround. I am writing a background daemon to handle to field incoming software update requests for several client applications running in the wild. My implementation is working great, but I seem to have a slow buildup of CFConstantStringRefs that are being created by internal Foundation calls and CFSTR macros and are never released. This slowly builds up over time and there doesn't appear to be any way to release these strings. I have run the daemon through the clang static analyzer and the Instruments leaks tool but none are reporting any leaks. I have even downloaded a fresh copy of valgrind from svn and it too is not finding anything. The instruments allocation monitor is reporting that there are CFConstantStringRefs that Foundation is allocating from internal methods and CFSTR macros that I am using in some functions. I'm happy to provide more details of the actual call-stacks and code if necessary. Does anyone have any idea about what could be causing this or what I can do about it? While this isn't a big deal in the short term, I have seen it build up fairly quickly during stress testing. I have even tried setting hard and soft resource limits using the launchd plist by setting the MemoryLock and ResidentSetSize but these are not having the expected effect of limiting the memory buildup. A work around I was contemplating was to have an idle timer that kills the app after a time if there are no active or incoming connections so it will then be restarted by launchd and reset the memory. This seems like a hack to get around the problem and I think I must be overlooking something. Any help would be greatly appreciated as I've tried googling and re-reading the background daemon tech notes and have turned up nothing. Thank you for your help. Sincerely, Kevin Ross ___ Cocoa-dev mailing list (Cocoa-dev@lists.apple.com) Please do not post admin requests or moderator comments to the list. Contact the moderators at cocoa-dev-admins(at)lists.apple.com Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com This email sent to arch...@mail-archive.com