Re: More Core Data Questions
On Oct 13, 2009, at 3:17 AM, Jon Hull wrote: You don't need proxies or NSProxy for this. You can just use a delegate. Off the cuff, I might consider implementing a protocol on your immutable objects for methods like "currentLocation" and "currentPlayer" which vector through a semi-global context object not unlike the app delegate. Then you can use keypaths to naturally write currentPlayer.name and so forth. NSProxy is unnecessarily heavy weight for simple delegation. They use memory too, ja know. And the semi-global context allows you to avoid making all your objects larger. You might even model the "current state" as a formal entity. That buys you change tracking, undo, multi-writer conflict resolutions and the ability to easily persistent and search for it. It also allows other managed objects to work with it naturally. Hmm... something to consider. The proxies do allow some powerful runtime state effects. The currentLocation was just a simple (and often used) example, but it is easy to have proxies which represent "The sister of the person I am currently talking to" or the location of that person or the item which that person holds in their hand. They can also be used to represent groups of characters. This is very powerful, and has come in very useful. Still, it might be possible to get the same functionality in another way. Okay. Typically the "Cocoa" way of doing that is by name and using keypaths instead of by pointer value and NSProxy. and then call off to the manager with the id whenever they need to run an event. Inside the manager I would either call off to a small sql database with blobs holding freeze-dried (keyedArchiving) objects & an id column, not a great plan. No searching into blobs, no partially loading or saving the larger freeze dried objects. hmm... The largest blob would probably be an object with an array of 20 or so pointers/ids. Not sure I need to search into them... mostly I just need to grab them by id. I had considered just using core data for everything, but as I mentioned in a previous post, I *need* to have consistently ordered arrays of these immutable objects (which can be in multiple places in a single array, and can be in multiple arrays). This is apparently difficult using core data :-( It's not difficult, although it is a bit tedious. Ordered relationships require you model the join table between two entities yourself (for many-to-manys), and add an ordering attribute. For one- to-many, you can put the ordering attribute on the destination entity (no join table necessary). Although, now that I think about it, perhaps I can store *just* the array of ids as a binary property, and have everything else defined in the entity. I will have to do some experiments. Yes, you could do that. Also, you could do that trivially with Core Data and just be done. Do you mean store the blobs in a core data managedObject instead of a SQL database? yes or avoid the blobs entirely using core data? Probably Unfortunately, it sounds like you don't have a ready alternative besides to spend a considerable amount of your own engineering resources. You'll have to decide if learning Core Data, and tuning your app performance fits within your schedule requirements, and whether implementing, debugging, and tuning all this yourself will really take any less time. You might consider mocking up a Core Data version over a few days and seeing how far you get. Yes, I think I will try that. Any advice on how to handle the 2 different types of graphs mentioned in my earlier post? Ideally I should have 2 files. One holding the event tree and one holding the rest (i.e. the stuff that changes). They main problem there is that they have connections between each other (e.g. an event might take a location or character as a parameter) Just create two stores and add them to the same NSPersistentStoreCoordinator. The connections between them can be represented via keypaths (bound dynamically) or persistently by stashing the objectID's URI away and materializing it in a transient relationship in -awakeFromFetch - Ben ___ Cocoa-dev mailing list (Cocoa-dev@lists.apple.com) Please do not post admin requests or moderator comments to the list. Contact the moderators at cocoa-dev-admins(at)lists.apple.com Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com This email sent to arch...@mail-archive.com
Re: More Core Data Questions
The project is a game engine which has 2 graphs. The first is a tree of events that represent the story. Each "event" in the story is an immutable object, and there is a special event which represents a series of events to run in order and one which represents branches that the player has to choose from. All of these are immutable, and the main goal is to avoid loading the whole graph which will consist of 10k-50k events. This graph will be the same for every player. Okay. If you build this immutable store on your dev machine and include it in your project resources, you should see excellent performance. Creating a store this size on the device would take a while. Just don't go crazy with to-many relationships or database normalization (for this many objects, on a phone). Of course, you don't want to search across all the events all the time. You'll want some criteria (NSPredicate) to evaluate just the subset of relevant elements. Yes, I plan to have an editor which creates the file on my Mac, and then include it with my resources for the iPhone project. The only real search-ish type thing I need to do at the moment is finding an Event by it's id. I also use proxies as context switches to objects in this graph. A concrete example is the current location, which is a proxy that pretends to be a location, and forwards all the messages it receives to the location object where the player is currently located. This allows events which reference locations/players/etc that change depending on state to remain immutable. You don't need proxies or NSProxy for this. You can just use a delegate. Off the cuff, I might consider implementing a protocol on your immutable objects for methods like "currentLocation" and "currentPlayer" which vector through a semi-global context object not unlike the app delegate. Then you can use keypaths to naturally write currentPlayer.name and so forth. NSProxy is unnecessarily heavy weight for simple delegation. They use memory too, ja know. And the semi-global context allows you to avoid making all your objects larger. You might even model the "current state" as a formal entity. That buys you change tracking, undo, multi-writer conflict resolutions and the ability to easily persistent and search for it. It also allows other managed objects to work with it naturally. Hmm... something to consider. The proxies do allow some powerful runtime state effects. The currentLocation was just a simple (and often used) example, but it is easy to have proxies which represent "The sister of the person I am currently talking to" or the location of that person or the item which that person holds in their hand. They can also be used to represent groups of characters. This is very powerful, and has come in very useful. Still, it might be possible to get the same functionality in another way. Luckily, the architecture of the engine has allowed me to make major changes rather painlessly so far. Part of the reason that I used proxies in the first place is that it allowed me to keep proper encapsulation. I will have to think on other ways to get the same behavior while maintaining the overall architecture/extensibility. I have a controller set up which communicates with the main thread, and the background thread blocks until the controller gets a valid response, at which point it starts unwinding again until it needs another response or reaches the end. There is no run loop for this thread. sure. don't forget an autorelease pool. Yes, I create/destroy a pool for each nested array of events and then call off to the manager with the id whenever they need to run an event. Inside the manager I would either call off to a small sql database with blobs holding freeze-dried (keyedArchiving) objects & an id column, not a great plan. No searching into blobs, no partially loading or saving the larger freeze dried objects. hmm... The largest blob would probably be an object with an array of 20 or so pointers/ids. Not sure I need to search into them... mostly I just need to grab them by id. I had considered just using core data for everything, but as I mentioned in a previous post, I *need* to have consistently ordered arrays of these immutable objects (which can be in multiple places in a single array, and can be in multiple arrays). This is apparently difficult using core data :-( Although, now that I think about it, perhaps I can store *just* the array of ids as a binary property, and have everything else defined in the entity. I will have to do some experiments. Also, you could do that trivially with Core Data and just be done. Do you mean store the blobs in a core data managedObject instead of a SQL database? or avoid the blobs entirely using core data? or save each event as a file with the id in the filename. Each event file would have to be > 8K before this
Re: More Core Data Questions
The project is a game engine which has 2 graphs. The first is a tree of events that represent the story. Each "event" in the story is an immutable object, and there is a special event which represents a series of events to run in order and one which represents branches that the player has to choose from. All of these are immutable, and the main goal is to avoid loading the whole graph which will consist of 10k-50k events. This graph will be the same for every player. Okay. If you build this immutable store on your dev machine and include it in your project resources, you should see excellent performance. Creating a store this size on the device would take a while. Just don't go crazy with to-many relationships or database normalization (for this many objects, on a phone). Of course, you don't want to search across all the events all the time. You'll want some criteria (NSPredicate) to evaluate just the subset of relevant elements. I also use proxies as context switches to objects in this graph. A concrete example is the current location, which is a proxy that pretends to be a location, and forwards all the messages it receives to the location object where the player is currently located. This allows events which reference locations/players/etc that change depending on state to remain immutable. You don't need proxies or NSProxy for this. You can just use a delegate. Off the cuff, I might consider implementing a protocol on your immutable objects for methods like "currentLocation" and "currentPlayer" which vector through a semi-global context object not unlike the app delegate. Then you can use keypaths to naturally write currentPlayer.name and so forth. NSProxy is unnecessarily heavy weight for simple delegation. They use memory too, ja know. And the semi-global context allows you to avoid making all your objects larger. You might even model the "current state" as a formal entity. That buys you change tracking, undo, multi-writer conflict resolutions and the ability to easily persistent and search for it. It also allows other managed objects to work with it naturally. The event graph is traversed from a background thread which basically runs through events in order until it needs a user response. Surely most of the events are not relevant most of the time, no ? By location, if nothing else like pre-requisiste events. If you can keep a cookie into the player's choices tree, then you can do an indexed query on eligible events and only evaluate those in memory. I have a controller set up which communicates with the main thread, and the background thread blocks until the controller gets a valid response, at which point it starts unwinding again until it needs another response or reaches the end. There is no run loop for this thread. sure. don't forget an autorelease pool. All of this works fantastically for the game, but now I have reached the point where I can no longer put off save & load. When I started the project Core Data was not yet available on the iPhone, so the design did not take it into account. I was thinking that core data might be able to help me save & load these graphs, but now I am not so sure. My new plan is as follows: Create a manager singleton for events which returns an event object for a given id, and then modify the special events mentioned above to hold an event's id instead of actual event objects, You could do that. Probably easier to just use an NSManagedObjectContext and the delegate I mentioned above. and then call off to the manager with the id whenever they need to run an event. Inside the manager I would either call off to a small sql database with blobs holding freeze-dried (keyedArchiving) objects & an id column, not a great plan. No searching into blobs, no partially loading or saving the larger freeze dried objects. Also, you could do that trivially with Core Data and just be done. or save each event as a file with the id in the filename. Each event file would have to be > 8K before this even begins its distant journey to approach making any sense at all from a performance perspective. I would also have a cache in the manager to avoid reallocating objects unnecessarily. I wouldn't worry about caching just yet. Bigger fish. I would prefer to have something like core data do this lazy loading for me, but it may not work in this case. For the objects that store state, I will just archive them into a save file. still no lazy loading or incremental saving. Thoughts? Does this sound like a reasonable approach, Not really, no. Maybe you need to do something quick & very dirty for deadlines. c'est la vi or will core data actually work for me? It's not clear to me why you think it wouldn't.So far, all I get is that you haven't used Core Data before and don't know it. Which, btw, is a perfectly valid issue to be having as you
Re: More Core Data Questions
Yes, well it is fairly complex... but it looks like core data is not a good fit since a basic requirement is that I need it to store nested arrays of immutable objects and keep the order (I also need to allow those objects to be inserted at multiple points in the array). The immutability and the possibility of multiple entries make an order property unworkable. Possibly, but you might be pleasantly surprised (not a knowing hint, just an honest possibility :-)). If you can, try to explain your needs. Someone might just have a good suggestion (or solid reasons why not). The project is a game engine which has 2 graphs. The first is a tree of events that represent the story. Each "event" in the story is an immutable object, and there is a special event which represents a series of events to run in order and one which represents branches that the player has to choose from. All of these are immutable, and the main goal is to avoid loading the whole graph which will consist of 10k-50k events. This graph will be the same for every player. The second graph represents in-game objects (characters, locations, items, etc...) and stores any state for the story. These should be saved off to a separate file, and will be different for each player/ saved game. I also use proxies as context switches to objects in this graph. A concrete example is the current location, which is a proxy that pretends to be a location, and forwards all the messages it receives to the location object where the player is currently located. This allows events which reference locations/players/etc that change depending on state to remain immutable. The event graph is traversed from a background thread which basically runs through events in order until it needs a user response. I have a controller set up which communicates with the main thread, and the background thread blocks until the controller gets a valid response, at which point it starts unwinding again until it needs another response or reaches the end. There is no run loop for this thread. All of this works fantastically for the game, but now I have reached the point where I can no longer put off save & load. When I started the project Core Data was not yet available on the iPhone, so the design did not take it into account. I was thinking that core data might be able to help me save & load these graphs, but now I am not so sure. My new plan is as follows: Create a manager singleton for events which returns an event object for a given id, and then modify the special events mentioned above to hold an event's id instead of actual event objects, and then call off to the manager with the id whenever they need to run an event. Inside the manager I would either call off to a small sql database with blobs holding freeze-dried (keyedArchiving) objects & an id column, or save each event as a file with the id in the filename. I would also have a cache in the manager to avoid reallocating objects unnecessarily. I would prefer to have something like core data do this lazy loading for me, but it may not work in this case. For the objects that store state, I will just archive them into a save file. Thoughts? Does this sound like a reasonable approach, or will core data actually work for me? Thanks, Jon ___ Cocoa-dev mailing list (Cocoa-dev@lists.apple.com) Please do not post admin requests or moderator comments to the list. Contact the moderators at cocoa-dev-admins(at)lists.apple.com Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com This email sent to arch...@mail-archive.com
Re: More Core Data Questions
On Oct 12, 2009, at 1:31 PM, Jon Hull wrote: 3) What is the best way of connecting objects from different stores? I am considering giving them UUIDs and then storing that as a reference. Then setting a transient property based on that in - awakeFromFetch. Alternatively, I could store it as a fetched property, but I want it to be a 1:1 correspondence. This might help. "Let's merge managed object models!" by Chris Hanson http://chanson.livejournal.com/187540.html Richard ___ Cocoa-dev mailing list (Cocoa-dev@lists.apple.com) Please do not post admin requests or moderator comments to the list. Contact the moderators at cocoa-dev-admins(at)lists.apple.com Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com This email sent to arch...@mail-archive.com
re: More Core Data Questions
Ok, let me ask some more specific questions and see if that gets a response... Feel free to respond if you only know the answer to 1 or 2 of these. 1) Can I count on a to-many relationship keeping the order of the managedObjects it points to? The order is very important in this case, and I need a way to ensure that the order does not change when the object is saved and reloaded. No. You'll have to model order as an attribute yourself. 2) Does core data require a run-loop to work? No. 3) What is the best way of connecting objects from different stores? I am considering giving them UUIDs and then storing that as a reference. Then setting a transient property based on that in - awakeFromFetch. Alternatively, I could store it as a fetched property, but I want it to be a 1:1 correspondence. That's fine, although you can just use the URI representation of the destination object instead of creating your own separate UUID. You might look at /Developer/Examples/CoreData/iClass 4) Is there a better way to get this lazy loading? What was the first way ? My main goal is to keep only those objects from this large (>1000) object graph in memory that are needed (since the iPhone has limited memory). You may find the NSFetchedResultsController useful, as well as the options on NSFetchRequest like -setFetchBatchSize: They are very aggressive about memory optimizations. Basically, I want the behavior of the old resource manager from the olden days (that is I can act as if my full object graph is in memory, but only those that are needed actually are... and they are fetched just in time). Core Data always does that. That's the default with its SQLite persistent store. Are you making this more complicated than it needs to be for performance issues you have yet to measure ? I wouldn't add multiple stores to work around a performance issue before actually trying it. - Ben ___ Cocoa-dev mailing list (Cocoa-dev@lists.apple.com) Please do not post admin requests or moderator comments to the list. Contact the moderators at cocoa-dev-admins(at)lists.apple.com Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com This email sent to arch...@mail-archive.com
Re: More Core Data Questions
On Oct 12, 2009, at 4:32 PM, Jon Hull wrote: I have spent the last 48 hours (re)reading core data docs. My head is swimming in docs. Understandable. It's a complicated technology (especially when you consider its interaction with Bindings). The short answer is yes, I understand to give each thread it's own context, but I don't know if it will break without a run loop. Maybe I'm being dense - a distinct possibility - but I don't get what you mean by this. It might be good to start a separate (mailing list) thread with a detailed explanation of what you intend to do. Yes, although I also read a warning about using these cross-stores somewhere, so I was thinking of rolling my own. If I encountered the same warning, I honestly can't remember it. :-) The core data UUID would probably also be temporary with the way I am currently creating my objects (because I wouldn't have saved yet), so I would need something persistent. I don't think adding a UUID hurts anything, honestly, so if it simplifies things for you, go for it. Yes, well it is fairly complex... but it looks like core data is not a good fit since a basic requirement is that I need it to store nested arrays of immutable objects and keep the order (I also need to allow those objects to be inserted at multiple points in the array). The immutability and the possibility of multiple entries make an order property unworkable. Possibly, but you might be pleasantly surprised (not a knowing hint, just an honest possibility :-)). If you can, try to explain your needs. Someone might just have a good suggestion (or solid reasons why not). -- I.S. ___ Cocoa-dev mailing list (Cocoa-dev@lists.apple.com) Please do not post admin requests or moderator comments to the list. Contact the moderators at cocoa-dev-admins(at)lists.apple.com Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com This email sent to arch...@mail-archive.com
Re: More Core Data Questions
On Oct 12, 2009, at 12:46 PM, I. Savant wrote: On Oct 12, 2009, at 3:31 PM, Jon Hull wrote: 1) Can I count on a to-many relationship keeping the order of the managedObjects it points to? The order is very important in this case, and I need a way to ensure that the order does not change when the object is saved and reloaded. No. This is stated in the documentation and has come up on this list many times. You'll want to add a property to keep track of the desired order. I must have missed this. Thank you. 2) Does core data require a run-loop to work? I'm not quite sure how to answer that, specifically ... All of the work takes place in a background thread, and that is the only thread which would be accessing these objects ... so I'll answer this (what you appear to be asking). Multithreading with Core Data gets its own section in the documentation - each thread needs its own context. Read the documentation. I have spent the last 48 hours (re)reading core data docs. My head is swimming in docs. The short answer is yes, I understand to give each thread it's own context, but I don't know if it will break without a run loop. 3) What is the best way of connecting objects from different stores? I am considering giving them UUIDs and then storing that as a reference. Then setting a transient property based on that in -awakeFromFetch. Alternatively, I could store it as a fetched property, but I want it to be a 1:1 correspondence. If the objects are already saved to a store, they already have a stable, unique ID. This is also covered in the documentation and in the list archives. Yes, although I also read a warning about using these cross-stores somewhere, so I was thinking of rolling my own. The core data UUID would probably also be temporary with the way I am currently creating my objects (because I wouldn't have saved yet), so I would need something persistent. 4) Is there a better way to get this lazy loading? My main goal is to keep only those objects from this large (>1000) object graph in memory that are needed (since the iPhone has limited memory). You're going to need to be specific about the relevant parts of your data model fora "best approach" suggestion. General guidelines are in the docs. Yes, well it is fairly complex... but it looks like core data is not a good fit since a basic requirement is that I need it to store nested arrays of immutable objects and keep the order (I also need to allow those objects to be inserted at multiple points in the array). The immutability and the possibility of multiple entries make an order property unworkable. You can always have a separate entity for the large data (to act as a BLOB). IE, an "Image" entity that holds lots of metadata, possibly a small thumbnail, and a reference to an instance of "ImageData" that holds the actual data. Yes, I did this in the last core data app I wrote. • Use core data in a rather complex way What complex way? Multiple stores + some way of keeping ordered arrays • Roll my own SQL with a single table that stores blobs by that identifier (see above) and then returns a freeze dried object from the blob. Not possible (or at least remotely sane) with a Core Data created store. Don't edit the store. Its implementation details (ie, the schema) are private and subject to change. This is a Bad Idea. Oh no... I mean instead of core data • Store each object as a small file in a folder and use the identifier in the filename Also possible if you don't want the data inside the store. You can create (and archive) an FSRef for more robust "locate-the-moved-file- between-sessions" behavior. again instead of core data, but useful to know. • Something with proxies that go grab it just in time (combined with SQL or the files above) I'm not sure what this means. I could use a proxy to implement lazy loading (from a file or out of SQL) • Something I haven't thought of... You were on the right track with the idea of a BLOB if you don't mind saving the BLOB in the store. The alternative - which you also guessed - is to save a reference to an external file. The choice is yours: neither is better than the other in the general case. Thank you. I might consider something like this instead of rolling my own SQL. Any help is greatly appreciated! You can help yourself better than anybody else by (re-)reading the Core Data Programming Guide. It contains answers to pretty much everything you've asked here. Very specific answers. Yes, I have been living those docs for a couple of days now. It now looks like core data is not a good match for my particular problem. Thanks for your help. Thanks, Jon___ Cocoa-dev mailing list (Cocoa-dev@lists.apple.com) Please do not post admin requests or moderator comments to the list. Contact the moderators a
Re: More Core Data Questions
On Oct 12, 2009, at 3:31 PM, Jon Hull wrote: 1) Can I count on a to-many relationship keeping the order of the managedObjects it points to? The order is very important in this case, and I need a way to ensure that the order does not change when the object is saved and reloaded. No. This is stated in the documentation and has come up on this list many times. You'll want to add a property to keep track of the desired order. 2) Does core data require a run-loop to work? I'm not quite sure how to answer that, specifically ... All of the work takes place in a background thread, and that is the only thread which would be accessing these objects ... so I'll answer this (what you appear to be asking). Multithreading with Core Data gets its own section in the documentation - each thread needs its own context. Read the documentation. 3) What is the best way of connecting objects from different stores? I am considering giving them UUIDs and then storing that as a reference. Then setting a transient property based on that in - awakeFromFetch. Alternatively, I could store it as a fetched property, but I want it to be a 1:1 correspondence. If the objects are already saved to a store, they already have a stable, unique ID. This is also covered in the documentation and in the list archives. 4) Is there a better way to get this lazy loading? My main goal is to keep only those objects from this large (>1000) object graph in memory that are needed (since the iPhone has limited memory). You're going to need to be specific about the relevant parts of your data model fora "best approach" suggestion. General guidelines are in the docs. You can always have a separate entity for the large data (to act as a BLOB). IE, an "Image" entity that holds lots of metadata, possibly a small thumbnail, and a reference to an instance of "ImageData" that holds the actual data. • Use core data in a rather complex way What complex way? • Roll my own SQL with a single table that stores blobs by that identifier (see above) and then returns a freeze dried object from the blob. Not possible (or at least remotely sane) with a Core Data created store. Don't edit the store. Its implementation details (ie, the schema) are private and subject to change. This is a Bad Idea. • Store each object as a small file in a folder and use the identifier in the filename Also possible if you don't want the data inside the store. You can create (and archive) an FSRef for more robust "locate-the-moved-file- between-sessions" behavior. • Something with proxies that go grab it just in time (combined with SQL or the files above) I'm not sure what this means. • Something I haven't thought of... You were on the right track with the idea of a BLOB if you don't mind saving the BLOB in the store. The alternative - which you also guessed - is to save a reference to an external file. The choice is yours: neither is better than the other in the general case. Any help is greatly appreciated! You can help yourself better than anybody else by (re-)reading the Core Data Programming Guide. It contains answers to pretty much everything you've asked here. Very specific answers. -- I.S. ___ Cocoa-dev mailing list (Cocoa-dev@lists.apple.com) Please do not post admin requests or moderator comments to the list. Contact the moderators at cocoa-dev-admins(at)lists.apple.com Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com This email sent to arch...@mail-archive.com
