How to manage a separate data file while saving an NSPersistentDocument or during schema migration
Following the advice in http://developer.apple.com/documentation/Cocoa/Conceptual/CoreData/Articles/cdPerformance.html#//apple_ref/doc/uid/TP40003468-SW5 regarding storage of BLOBs, I've chosen to store binary data for my NSPersistenDocument-based app (Leopard-only, if that makes a difference) in an external file (separate from the document Core Data data store) and keeping an archived NDAlias reference to that file in my document's data store. Ultimately, I would like the external data file to end up in the same directory as the document (data store) file. Since I may need to write data to the external data file before the document is saved (this is a scientific data acquisition app and losing data due to a crash is to be avoided if possible), I've taken the following approach: I would like to encapsulate the management of this external file in the model-related classes since I will need the same functionality during schema migration. Thus managing the external data file in my NSPersistentDocument's subclass seems wrong. Thus, in the object model's root object, I create the external file in NSTemporaryDirectory() in the root objects awakeFromInsert method and store the NDAlias referencing the data file. I would then like to move the external data file to the same directory as the saved data store, when a save occurs. I thought didSave would be the appropriate place to do it, but it looks like during invocation of the didSave method, the persistent store is still in a temporary directory (presumably before being FSExchangeObjects'd to create an atomic save operation). I plan to factor the logic in awakeFromInsert and didSave into class methods so that they can be called during schema migration in a custom entity policy's createDestinationInstancesForSourceInstance:entityMapping:manager:error: and endInstanceCreationForEntityMapping:manager:error: respectively. So, my question: when during NSPersistentDocument save and during schema migration can I be assured that my root object's objectID.persistentStore.URL is the 'final' URL of the save (e.g. where the user chose to save the file for document save)? thanks, Barry ___ Cocoa-dev mailing list (Cocoa-dev@lists.apple.com) Please do not post admin requests or moderator comments to the list. Contact the moderators at cocoa-dev-admins(at)lists.apple.com Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com This email sent to arch...@mail-archive.com
Re: How to manage a separate data file while saving an NSPersistentDocument or during schema migration
On Jan 16, 2009, at 10:00, Barry Wark wrote: I would like to encapsulate the management of this external file in the model-related classes since I will need the same functionality during schema migration. Thus managing the external data file in my NSPersistentDocument's subclass seems wrong. Thus, in the object model's root object, I create the external file in NSTemporaryDirectory() in the root objects awakeFromInsert method and store the NDAlias referencing the data file. I would then like to move the external data file to the same directory as the saved data store, when a save occurs. I thought didSave would be the appropriate place to do it, but it looks like during invocation of the didSave method, the persistent store is still in a temporary directory (presumably before being FSExchangeObjects'd to create an atomic save operation). FWIW, I spent quite some time trying to devise a way of getting something like this to work, and eventually came to this conclusion: That Way Madness Lies but YMMV (your madness may vary). There are two sets of problems to deal with. First, NSDocument doesn't guarantee any particular file handling behavior at Save time, and you can expect that the implementation may change in different versions of OS X. Also, the implementation may well be different on different file systems, so the sequence of events you observe on a local HFS+ volume may not apply to (say) network volumes. Anything you choose to rely on may break at any time, unless the documentation says otherwise. Second, NSDocument is fundamentally invested in the concept of save-by- copying for reasons of data integrity. As soon as you try to *preserve* unchanged files across a save, the integrity of the save metaphor is thrown into doubt. I'm certain that it's solvable (though I'm also certain I'm not smart enough to do it myself), but only if you have a solution to the first problem. Think about NSPersistentDocument, and how it shoehorns Core Data into a NSDocument environment. This was done by people at Apple who were certainly smart enough, but even so there's no save-to (i.e. Save a Copy As ...), no autosave, and save-as is implemented (AFAIK) as a Core Data migration. So I think you have four options: 1. Go ahead with your original plan and hope it doesn't break on future Mac OS releases or on different file systems. 2. Put the BLOBs in the Core Data object graph after all. (That's what I chose. However, since you can't fault out individual properties, instead of making each BLOB an attribute of its owning object, I made it an attribute of an object owned by the owning object. That way I could in effect fault out the BLOBs at will.) 3. Make your own NSDocument subclass that uses Core Data internally (but not NSPersistentDocument), and replace the entire saving mechanism by your own implementation. 4. Decide that your database is not really a document, and abandon the entire NSDocument metaphor. You can still have housekeeping functions for backing up versions of your database or snapshotting or cloning it, without following the document model exactly. ___ Cocoa-dev mailing list (Cocoa-dev@lists.apple.com) Please do not post admin requests or moderator comments to the list. Contact the moderators at cocoa-dev-admins(at)lists.apple.com Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com This email sent to arch...@mail-archive.com
Re: How to manage a separate data file while saving an NSPersistentDocument or during schema migration
On Fri, Jan 16, 2009 at 3:00 PM, Quincey Morris quinceymor...@earthlink.net wrote: On Jan 16, 2009, at 10:00, Barry Wark wrote: I would like to encapsulate the management of this external file in the model-related classes since I will need the same functionality during schema migration. Thus managing the external data file in my NSPersistentDocument's subclass seems wrong. Thus, in the object model's root object, I create the external file in NSTemporaryDirectory() in the root objects awakeFromInsert method and store the NDAlias referencing the data file. I would then like to move the external data file to the same directory as the saved data store, when a save occurs. I thought didSave would be the appropriate place to do it, but it looks like during invocation of the didSave method, the persistent store is still in a temporary directory (presumably before being FSExchangeObjects'd to create an atomic save operation). FWIW, I spent quite some time trying to devise a way of getting something like this to work, and eventually came to this conclusion: That Way Madness Lies but YMMV (your madness may vary). There are two sets of problems to deal with. First, NSDocument doesn't guarantee any particular file handling behavior at Save time, and you can expect that the implementation may change in different versions of OS X. Also, the implementation may well be different on different file systems, so the sequence of events you observe on a local HFS+ volume may not apply to (say) network volumes. Anything you choose to rely on may break at any time, unless the documentation says otherwise. Second, NSDocument is fundamentally invested in the concept of save-by-copying for reasons of data integrity. As soon as you try to *preserve* unchanged files across a save, the integrity of the save metaphor is thrown into doubt. I'm certain that it's solvable (though I'm also certain I'm not smart enough to do it myself), but only if you have a solution to the first problem. Think about NSPersistentDocument, and how it shoehorns Core Data into a NSDocument environment. This was done by people at Apple who were certainly smart enough, but even so there's no save-to (i.e. Save a Copy As ...), no autosave, and save-as is implemented (AFAIK) as a Core Data migration. I was hoping that those same smart engineers (and they were very smart) had come up with a solution for this use case as well. Ah, well. So I think you have four options: 1. Go ahead with your original plan and hope it doesn't break on future Mac OS releases or on different file systems. I haven't even gotten this solution working on the current OS. Grr. 2. Put the BLOBs in the Core Data object graph after all. (That's what I chose. However, since you can't fault out individual properties, instead of making each BLOB an attribute of its owning object, I made it an attribute of an object owned by the owning object. That way I could in effect fault out the BLOBs at will.) In fact, that's how I started. It works fine (using a separate entity for the data), and performance was fine. Unfortunately, the backup system at the deployment site backs up entire changed files (rather than binary diffs), so the continuous changes to flags and metadata in the data model during analysis force a back up of all the original BLOB data as well... order 100GB/day. Thus, we've decided to go with the separate file solution. Since this is, in fact, the Apple recommended approach, I was surprised it wasn't easier to implement. 3. Make your own NSDocument subclass that uses Core Data internally (but not NSPersistentDocument), and replace the entire saving mechanism by your own implementation. 4. Decide that your database is not really a document, and abandon the entire NSDocument metaphor. You can still have housekeeping functions for backing up versions of your database or snapshotting or cloning it, without following the document model exactly. We considered both of these, but the application is well modeled by the document architecture, so it seems a shame to throw all that out. I guess it's time to file a ticket on this one... ___ Cocoa-dev mailing list (Cocoa-dev@lists.apple.com) Please do not post admin requests or moderator comments to the list. Contact the moderators at cocoa-dev-admins(at)lists.apple.com Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com This email sent to arch...@mail-archive.com
Re: How to manage a separate data file while saving an NSPersistentDocument or during schema migration
On Jan 16, 2009, at 17:26, Barry Wark wrote: In fact, that's how I started. It works fine (using a separate entity for the data), and performance was fine. Unfortunately, the backup system at the deployment site backs up entire changed files (rather than binary diffs), so the continuous changes to flags and metadata in the data model during analysis force a back up of all the original BLOB data as well... order 100GB/day. Thus, we've decided to go with the separate file solution. Since this is, in fact, the Apple recommended approach, I was surprised it wasn't easier to implement. If the BLOBs themselves are not being changed (at least by this application on a per-document basis), then you could keep a library of BLOB sets in the Application Support folder or somewhere shared. Or, even if the BLOBs are changed sometimes, you could try putting them in a separate persistent store that you add to your managed object context, and use cross-store links. It's *possible* that a simple Save will not touch the BLOB store if the BLOBs haven't changed (though I have no idea if that's so or not). In that case, your backup system won't try to backup the unchanged store. You'd have to try this out for feasibility, and it means messing with cross-store links, but the big win is of course that you don't have to get your hands dirty in the save process at all (I think). ___ Cocoa-dev mailing list (Cocoa-dev@lists.apple.com) Please do not post admin requests or moderator comments to the list. Contact the moderators at cocoa-dev-admins(at)lists.apple.com Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com This email sent to arch...@mail-archive.com
