On Feb 5, 2010, at 5:49 PM, Greg Robertson wrote: > I am trying to parse a large CSV (5MB) file using some code I found here: > http://www.macresearch.org/cocoa-scientists-part-xxvi-parsing-csv-data > > but I am getting what I think is an out of memory error part way through, the > error is: > > DataImport(1324,0xa09f3500) malloc: *** mmap(size=16777216) failed (error > code=12) > *** error: can't allocate region > > I'm fairly certain I do not have a memory leak it is just that the NSString > is very large. Is there anyway to increase the memory allocated or clean up > the code to reduce its memory usage?
Did you try running it with one of the memory profiling tools to see what's happening? I wouldn't consider a 5MB csv file large, so I'd guess that you're blowing up the autorelease pool; this code looks really inefficient in that respect. You end up creating a mutable array and at least one mutable string per row, and in addition calling -[NSScanner scanCharactersFromSet:] is extremely inefficient (it inverts and autoreleases a copy of the character set). You can invert the set and use scanUpToCharactersFromSet: to work around that bug. Additionally, using immutable character sets should be faster. As a bandaid, you could try creating an autorelease pool in the outer while loop and drain it on each pass (once per row if I read the code correctly). Alternately, you could use alloc/init/release instead of the convenience methods, or even rewrite using CFStringInlineBuffer if you have lots of time to kill :). hth, Adam _______________________________________________ Cocoa-dev mailing list (Cocoa-dev@lists.apple.com) Please do not post admin requests or moderator comments to the list. Contact the moderators at cocoa-dev-admins(at)lists.apple.com Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com This email sent to arch...@mail-archive.com
