Re: UIDocument + NSFileWrapper crashes on opening
On Jun 21, 2013, at 03:40 , Markus Spoettl wrote: > can it be that a wrapper which I hold on to from the time I first created it > switches it's backing from in-memory to memory mapped disk once it has been > saved? There are actually three possible states for a regular file wrapper: 1. 'regularFileContents' is a NSData instance whose data is in memory in the normal way 2. 'regularFileContents' is a NSData instance whose data is memory-mapped to a file 3. 'regularFileContents' is nil When you create a regular file wrapper with 'initRegularFileWithContents:', the initial state is #1. There is no documented API that will change this state to #2 or #3, other than releasing the file wrapper and creating a new one. When you create a file wrapper with 'initWithURL:options:error:', the initial state is #3, unless you use the 'NSFileWrapperReadingImmediate' option. This gets changed to #1 or #2 when you access the 'regularFileContents' property. When a save occurs, 'contentsForType:error:' returns a package wrapper that may be the one you were given originally, or may be one that you construct at save time. I believe, therefore, that the frameworks do nothing to sub-wrappers after you return the package wrapper for the save. For files that were edited, the sub-wrapper will be in state #1 (with the new data), and for files that weren't, the sub-wrapper will be in state #2 or #3 (depending on whether this particular instance was used to read the old data or not). The net effect is that only changed file data is hogging RAM, and the only way to "switch" the sub-wrappers back to state #3 is the recreate the package wrapper yourself. What's confusing about this is that it's hard to tell the difference between the ideal (state #3 except for sub-wrappers with freshly edited data) and the typical (state #2 for sub-wrappers whose data has been read). ___ Cocoa-dev mailing list (Cocoa-dev@lists.apple.com) Please do not post admin requests or moderator comments to the list. Contact the moderators at cocoa-dev-admins(at)lists.apple.com Help/Unsubscribe/Update your Subscription: https://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com This email sent to arch...@mail-archive.com
Re: UIDocument + NSFileWrapper crashes on opening
On 6/21/13 8:19 AM, Quincey Morris wrote: What is interesting too is that my application was able to create the whole wrapper structure when it first created the package. It would have required the same amount of memory to hold the wrappers in memory before they get written to the disk by UIDocument. Maybe it has to do with the fact that it is creating the structure incrementally and relatively slowly while downloading objects over the network. Reading the UIDocument from disk on the other hand requires it to load everything instantly. Maybe there's some system safeguard in place that kills apps whose memory usage increases too rapidly. I think the short answer is that your files are being memory-mapped rather than read via I/O. That means the result is dependent on the VM swapping caused by your pattern of access. When a new file wrapper is added to an existing wrapper UIDocument/the framework hasn't had a chance to memory map it because I didn't hand over the root file wrapper yet. That inspired the thought that saving - even incrementally - will use all memory necessary to load the whole thing. I clearly don't know enough about the inner workings of NSFileWrapper but can it be that a wrapper which I hold on to from the time I first created it switches it's backing from in-memory to memory mapped disk once it has been saved? That would explain why incremental saving worked but loading didn't. Regards Markus -- __ Markus Spoettl ___ Cocoa-dev mailing list (Cocoa-dev@lists.apple.com) Please do not post admin requests or moderator comments to the list. Contact the moderators at cocoa-dev-admins(at)lists.apple.com Help/Unsubscribe/Update your Subscription: https://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com This email sent to arch...@mail-archive.com
Re: UIDocument + NSFileWrapper crashes on opening
On Jun 20, 2013, at 22:05 , Markus Spoettl wrote: > It seems that the default implementation of UIDocument uses an NSFileWrapper > initialized with the NSFileWrapperReadingImmediate reading option. > > When I create a NSFileWrapper not using that option it works, meaning it now > opens and loads the package document it previously crashed on. I wonder what > surprises I will run into with lazy wrapper loading like that. Initial tests > show no problems when changing the document and saving the whole thing. For > now it seems to work exactly like on OSX. > > What is interesting too is that my application was able to create the whole > wrapper structure when it first created the package. It would have required > the same amount of memory to hold the wrappers in memory before they get > written to the disk by UIDocument. Maybe it has to do with the fact that it > is creating the structure incrementally and relatively slowly while > downloading objects over the network. Reading the UIDocument from disk on the > other hand requires it to load everything instantly. Maybe there's some > system safeguard in place that kills apps whose memory usage increases too > rapidly. I think the short answer is that your files are being memory-mapped rather than read via I/O. That means the result is dependent on the VM swapping caused by your pattern of access. That's assuming you actually read all of the files. If you don't access the data, it won't have any performance impact. You might get away with this on iOS, but on OS X it's all going to come unstuck if the document file is on a disk that doesn't allow memory mapping, such as a network volume. One of these days I'm going to write an experimental app that tries to figure out what happens when. ___ Cocoa-dev mailing list (Cocoa-dev@lists.apple.com) Please do not post admin requests or moderator comments to the list. Contact the moderators at cocoa-dev-admins(at)lists.apple.com Help/Unsubscribe/Update your Subscription: https://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com This email sent to arch...@mail-archive.com
Re: UIDocument + NSFileWrapper crashes on opening
On 6/20/13 11:52 PM, Quincey Morris wrote: OK, but isn't NSFileWrapper supposed to facilitate exactly that by providing sub-wrappers instead of actual data of contained files/folders, which can be read on demand when needed? No, NSFileWrapper provides *lazy* loading, in the sense that you don't need to load a file until/unless it's actually needed. It doesn't provide any mechanism for evicting file contents from memory -- not that I can find, and I looked hard. If you need to read all the files, they will all eventually be in memory. Yes, but the important word is "eventually". It seems that the default implementation of UIDocument uses an NSFileWrapper initialized with the NSFileWrapperReadingImmediate reading option. When I create a NSFileWrapper not using that option it works, meaning it now opens and loads the package document it previously crashed on. I wonder what surprises I will run into with lazy wrapper loading like that. Initial tests show no problems when changing the document and saving the whole thing. For now it seems to work exactly like on OSX. What is interesting too is that my application was able to create the whole wrapper structure when it first created the package. It would have required the same amount of memory to hold the wrappers in memory before they get written to the disk by UIDocument. Maybe it has to do with the fact that it is creating the structure incrementally and relatively slowly while downloading objects over the network. Reading the UIDocument from disk on the other hand requires it to load everything instantly. Maybe there's some system safeguard in place that kills apps whose memory usage increases too rapidly. This is doubly irritating, because if the file contents is an archive, or otherwise needs to be converted or expanded, then both the raw data and unarchived data are in memory.** This is distinct from *incremental* loading, which loads the raw data in pieces and only temporarily. This cannot -- in practical way -- be done with NSFileWrapper. While you can't have a filewrapper load one file incrementally, you sure can use NSFileWrapper to load the contents of a folder structure's files incrementally, one file at a time. Regards Markus -- __ Markus Spoettl ___ Cocoa-dev mailing list (Cocoa-dev@lists.apple.com) Please do not post admin requests or moderator comments to the list. Contact the moderators at cocoa-dev-admins(at)lists.apple.com Help/Unsubscribe/Update your Subscription: https://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com This email sent to arch...@mail-archive.com
Re: UIDocument + NSFileWrapper crashes on opening
You just saved me days of work. Thanks very much! Regards Markus On 6/20/13 11:52 PM, Luke the Hiesterman wrote: By default, UIDocument does eager reading. You can override that in -readFromURL:error:. Luke On Jun 20, 2013, at 2:37 PM, Markus Spoettl wrote: On 6/20/13 11:16 PM, Luke the Hiesterman wrote: Probably exactly as you said. Try overriding -readFromURL:error: to implement incremental reading. OK, but isn't NSFileWrapper supposed to facilitate exactly that by providing sub-wrappers instead of actual data of contained files/folders, which can be read on demand when needed? Is NSFileWrapper on iOS doing things differently than on OSX in that regard? What kind of folder hierarchy/size can it read? 140MB doesn't seem enormously big (of course it does depend on what it does with it). Regards Markus -- __ Markus Spoettl -- __ Markus Spoettl ___ Cocoa-dev mailing list (Cocoa-dev@lists.apple.com) Please do not post admin requests or moderator comments to the list. Contact the moderators at cocoa-dev-admins(at)lists.apple.com Help/Unsubscribe/Update your Subscription: https://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com This email sent to arch...@mail-archive.com
Re: UIDocument + NSFileWrapper crashes on opening
By default, UIDocument does eager reading. You can override that in -readFromURL:error:. Luke On Jun 20, 2013, at 2:37 PM, Markus Spoettl wrote: > On 6/20/13 11:16 PM, Luke the Hiesterman wrote: >> Probably exactly as you said. Try overriding -readFromURL:error: to >> implement incremental reading. > > OK, but isn't NSFileWrapper supposed to facilitate exactly that by providing > sub-wrappers instead of actual data of contained files/folders, which can be > read on demand when needed? > > Is NSFileWrapper on iOS doing things differently than on OSX in that regard? > > What kind of folder hierarchy/size can it read? 140MB doesn't seem enormously > big (of course it does depend on what it does with it). > > Regards > Markus > -- > __ > Markus Spoettl ___ Cocoa-dev mailing list (Cocoa-dev@lists.apple.com) Please do not post admin requests or moderator comments to the list. Contact the moderators at cocoa-dev-admins(at)lists.apple.com Help/Unsubscribe/Update your Subscription: https://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com This email sent to arch...@mail-archive.com
Re: UIDocument + NSFileWrapper crashes on opening
On Jun 20, 2013, at 14:37 , Markus Spoettl wrote: > OK, but isn't NSFileWrapper supposed to facilitate exactly that by providing > sub-wrappers instead of actual data of contained files/folders, which can be > read on demand when needed? No, NSFileWrapper provides *lazy* loading, in the sense that you don't need to load a file until/unless it's actually needed. It doesn't provide any mechanism for evicting file contents from memory -- not that I can find, and I looked hard. If you need to read all the files, they will all eventually be in memory. This is doubly irritating, because if the file contents is an archive, or otherwise needs to be converted or expanded, then both the raw data and unarchived data are in memory.** This is distinct from *incremental* loading, which loads the raw data in pieces and only temporarily. This cannot -- in practical way -- be done with NSFileWrapper. I spent a lot of time going round in circles about this, so I'm speaking up here in case I've missed something obvious, in which case someone will jump in and correct me. But AFAIK NSFileWrapper isn't the way to go for managed, incremental loading. ** What makes this acceptable in a lot of cases is that NSFileWrapper may read files by memory mapping them. This can perform as well as, or better than, a properly functioning cache, at the expense of using up additional VM address space. ___ Cocoa-dev mailing list (Cocoa-dev@lists.apple.com) Please do not post admin requests or moderator comments to the list. Contact the moderators at cocoa-dev-admins(at)lists.apple.com Help/Unsubscribe/Update your Subscription: https://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com This email sent to arch...@mail-archive.com
Re: UIDocument + NSFileWrapper crashes on opening
On 6/20/13 11:16 PM, Luke the Hiesterman wrote: Probably exactly as you said. Try overriding -readFromURL:error: to implement incremental reading. OK, but isn't NSFileWrapper supposed to facilitate exactly that by providing sub-wrappers instead of actual data of contained files/folders, which can be read on demand when needed? Is NSFileWrapper on iOS doing things differently than on OSX in that regard? What kind of folder hierarchy/size can it read? 140MB doesn't seem enormously big (of course it does depend on what it does with it). Regards Markus -- __ Markus Spoettl ___ Cocoa-dev mailing list (Cocoa-dev@lists.apple.com) Please do not post admin requests or moderator comments to the list. Contact the moderators at cocoa-dev-admins(at)lists.apple.com Help/Unsubscribe/Update your Subscription: https://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com This email sent to arch...@mail-archive.com
Re: UIDocument + NSFileWrapper crashes on opening
Probably exactly as you said. Try overriding -readFromURL:error: to implement incremental reading. Luke On Jun 20, 2013, at 2:01 PM, Markus Spoettl wrote: > Hello everyone, > > I have an iOS 6 app that uses UIDocument to implement loading and saving of > my app's data. The document data is loaded from and saved to a NSFileWrapper > (representing a file package containing many files and folders), handed from > and to the document in -loadFromContents::: and -contentsForType::. > > This all works well for small documents but I now have a case where a > document the app created in a previous session apparently is too big to be > loaded. > > My app vanishes about 1-2 seconds after UIDocument calls -readFromURL:error: > before my implementation of -loadFromContents::: is even called. There is no > crash and no useful information (to me anyway) logged to the console. It just > logs that my app was killed but gives no reason. > > I *suspect* UIDocument tries to load the whole package containing its 140MB > of data in about 1000 files and couple of 100 folders and then chokes on it. > > The system had no problem creating the file package and saving it to the > "disk", but it sure can't load it. > > I have tried on the simulator and using the same data as on the device it > works, the data loads just fine. > > I'm pasting the console output of a typical session from the time the app is > started via Springboard until it crashes/vanishes. Maybe it contains a clue I > don't see. > > Does someone know what may be going on? I'm using Xcode 4.6.3 and iOS 6.1.3 > (iPad Retina). > > Thanks! > Regards > Markus > > ___ > > Cocoa-dev mailing list (Cocoa-dev@lists.apple.com) > > Please do not post admin requests or moderator comments to the list. > Contact the moderators at cocoa-dev-admins(at)lists.apple.com > > Help/Unsubscribe/Update your Subscription: > https://lists.apple.com/mailman/options/cocoa-dev/luketheh%40apple.com > > This email sent to luket...@apple.com ___ Cocoa-dev mailing list (Cocoa-dev@lists.apple.com) Please do not post admin requests or moderator comments to the list. Contact the moderators at cocoa-dev-admins(at)lists.apple.com Help/Unsubscribe/Update your Subscription: https://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com This email sent to arch...@mail-archive.com
Re: UIDocument + NSFileWrapper crashes on opening
Seems the log didn't make it in the initial post, not sure why. On 6/20/13 11:01 PM, Markus Spoettl wrote: I'm pasting the console output of a typical session from the time the app is started via Springboard until it crashes/vanishes. Maybe it contains a clue I don't see. Jun 20 20:52:08 Flatfish kernel[0] : launchd[136] Builtin profile: container (sandbox) Jun 20 20:52:08 Flatfish kernel[0] : launchd[136] Container: /private/var/mobile/Applications/2AB076E0-7F97-424E-81DB-C2701B968CCC (sandbox) Jun 20 20:52:08 Flatfish librariand[106] : client process 136 does not have a valid com.apple.developer.ubiquity-container-identifiers entitlement Jun 20 20:52:09 Flatfish com.apple.launchd[1] (com.apple.accountsd[112]) : (com.apple.accountsd) Idle-exit job was jettisoned. Will bypass throttle interval for next on-demand launch. Jun 20 20:52:09 Flatfish com.apple.launchd[1] (com.apple.accountsd[112]) : (com.apple.accountsd) Exited: Killed: 9 Jun 20 20:52:09 Flatfish kernel[0] : vnode: table is full Jun 20 20:52:09 Flatfish kernel[0] : 1250 desired, 1250 numvnodes, 0 free, 0 dead, 0 rage Jun 20 20:52:09 Flatfish com.apple.launchd[1] (com.apple.mobile.installd[109]) : (com.apple.mobile.installd) Idle-exit job was jettisoned. Will bypass throttle interval for next on-demand launch. Jun 20 20:52:09 Flatfish com.apple.launchd[1] (com.apple.mobile.installd[109]) : (com.apple.mobile.installd) Exited: Killed: 9 Jun 20 20:52:10 Flatfish com.apple.launchd[1] (com.apple.afcd[118]) : (com.apple.afcd) Idle-exit job was jettisoned. Will bypass throttle interval for next on-demand launch. Jun 20 20:52:10 Flatfish com.apple.launchd[1] (com.apple.afcd[118]) : (com.apple.afcd) Exited: Killed: 9 Jun 20 20:52:10 Flatfish com.apple.launchd[1] (com.apple.crashreportcopymobile[133]) : (com.apple.crashreportcopymobile) Idle-exit job was jettisoned. Will bypass throttle interval for next on-demand launch. Jun 20 20:52:10 Flatfish com.apple.launchd[1] (com.apple.crashreportcopymobile[133]) : (com.apple.crashreportcopymobile) Exited: Killed: 9 Jun 20 20:52:10 Flatfish kernel[0] : vnode: table is full Jun 20 20:52:10 Flatfish kernel[0] : 1250 desired, 1250 numvnodes, 0 free, 0 dead, 0 rage Jun 20 20:52:10 Flatfish kernel[0] : vnode: table is full Jun 20 20:52:10 Flatfish kernel[0] : 1250 desired, 1250 numvnodes, 0 free, 0 dead, 0 rage Jun 20 20:52:10 Flatfish kernel[0] : vnode: table is full Jun 20 20:52:10 Flatfish kernel[0] : 1250 desired, 1250 numvnodes, 0 free, 0 dead, 0 rage Jun 20 20:52:10 Flatfish com.apple.launchd[1] (com.apple.lsd[130]) : (com.apple.lsd) Idle-exit job was jettisoned. Will bypass throttle interval for next on-demand launch. Jun 20 20:52:10 Flatfish com.apple.launchd[1] (com.apple.lsd[130]) : (com.apple.lsd) Exited: Killed: 9 Jun 20 20:52:10 Flatfish kernel[0] : vnode: table is full Jun 20 20:52:10 Flatfish kernel[0] : 1250 desired, 1250 numvnodes, 0 free, 0 dead, 0 rage Jun 20 20:52:10 Flatfish com.apple.launchd[1] (UIKitApplication:com.apple.mobilemail[0x1dc][108]) : (UIKitApplication:com.apple.mobilemail[0x1dc]) Exited: Killed: 9 Jun 20 20:52:10 Flatfish backboardd[26] : Application 'UIKitApplication:com.apple.mobilemail[0x1dc]' exited abnormally with signal 9: Killed: 9 Jun 20 20:52:10 Flatfish com.apple.launchd[1] (com.apple.mediastream.mstreamd[111]) : (com.apple.mediastream.mstreamd) Exited: Killed: 9 Jun 20 20:52:10 Flatfish com.apple.launchd[1] (UIKitApplication:com.apple.mobilephone[0xebcb][107]) : (UIKitApplication:com.apple.mobilephone[0xebcb]) Exited: Killed: 9 Jun 20 20:52:10 Flatfish backboardd[26] : Application 'UIKitApplication:com.apple.mobilephone[0xebcb]' exited abnormally with signal 9: Killed: 9 Jun 20 20:52:10 Flatfish MyApp[136] : shared connection error: Connection interrupted Jun 20 20:52:10 Flatfish com.apple.launchd[1] (com.apple.tccd[110]) : (com.apple.tccd) Exited: Killed: 9 Jun 20 20:52:10 Flatfish com.apple.launchd[1] (com.apple.librariand[106]) : (com.apple.librariand) Exited: Killed: 9 Jun 20 20:52:10 Flatfish com.apple.launchd[1] (com.apple.mediastream.mstreamd[139]) : (com.apple.mediastream.mstreamd) Exited: Killed: 9 Jun 20 20:52:10 Flatfish com.apple.launchd[1] (com.apple.mediastream.mstreamd) : (com.apple.mediastream.mstreamd) Throttling respawn: Will start in 1 seconds Jun 20 20:52:10 Flatfish com.apple.launchd[1] (com.apple.accountsd[138]) : (com.apple.accountsd) Idle-exit job was jettisoned. Will bypass throttle interval for next on-demand launch. Jun 20 20:52:10 Flatfish com.apple.launchd[1] (com.apple.accountsd[138]) : (com.apple.accountsd) Exited: Killed: 9 Jun 20 20:52:10 Flatfish com.apple.launchd[1] (UIKitApplication:com.apple.mobilemail[0x21ae][137]) : (UIKitApplication:com.apple.mobilemail[0x21ae]) Exited: Killed: 9 Jun 20 20:52:10 Flatfish com.apple.launchd[1] (UIKitApplication:com.apple.mobilemail[0x21ae]) : (UIKitApplication:com.apple.mobilemail[0x21ae]) Throttling respawn: Will start in 2147483647 seconds
UIDocument + NSFileWrapper crashes on opening
Hello everyone, I have an iOS 6 app that uses UIDocument to implement loading and saving of my app's data. The document data is loaded from and saved to a NSFileWrapper (representing a file package containing many files and folders), handed from and to the document in -loadFromContents::: and -contentsForType::. This all works well for small documents but I now have a case where a document the app created in a previous session apparently is too big to be loaded. My app vanishes about 1-2 seconds after UIDocument calls -readFromURL:error: before my implementation of -loadFromContents::: is even called. There is no crash and no useful information (to me anyway) logged to the console. It just logs that my app was killed but gives no reason. I *suspect* UIDocument tries to load the whole package containing its 140MB of data in about 1000 files and couple of 100 folders and then chokes on it. The system had no problem creating the file package and saving it to the "disk", but it sure can't load it. I have tried on the simulator and using the same data as on the device it works, the data loads just fine. I'm pasting the console output of a typical session from the time the app is started via Springboard until it crashes/vanishes. Maybe it contains a clue I don't see. Does someone know what may be going on? I'm using Xcode 4.6.3 and iOS 6.1.3 (iPad Retina). Thanks! Regards Markus ___ Cocoa-dev mailing list (Cocoa-dev@lists.apple.com) Please do not post admin requests or moderator comments to the list. Contact the moderators at cocoa-dev-admins(at)lists.apple.com Help/Unsubscribe/Update your Subscription: https://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com This email sent to arch...@mail-archive.com
