.. oh and one other thing, there's a core data instruments tool in XCode, well there is for OSX, not for iPhoneOS which I develop for which may be why I never saw it before. You could try that.
On 13-Feb-2010, at 9:36 PM, Roland King wrote: > ok, I don't see anything wrong with the predicate code, but I'm no core data > expert. > > I'll make one totally challengable statement. Assuming that core data uses > sqllite in a rational way to store objects (eg not storing everything as > blobs of opaque data) for instance one table per entity where each column of > the table is an attribute and evaluating the predicate does what you would > expect it to do, ie uses SQL to do as much of the heavy lifting on a fetch > request as possible, that column is indexed in the table and sqllite is using > the index; taking multi-minutes to find one row out of 20,000 just doesn't > make any sense, it should take seconds at most. > > I believe core data does use table-per-entity. I think that partly because > the documentation hints at it, partly because it makes sense and partly > because I looked at the implementation of one data model that I have. > > I can't see the point of making indexes if the predicate code doesn't > generate SQL which doesn't use them, but it's possible. It's possible that > core data goes and loads all the entity rows and inspects their attributes by > hand and filters them in code, but this is apple not microsoft. > > So that leaves column isn't indexed as the most likely. But you've checked > the 'indexed' box. Here's another wild assed guess, does coredata only create > a store when you have no current store? It certainly checks to see if the > store is compatible with the model but as the indexed property is just a hint > anyway, that store is compatible, just non-optimal .. it's possible if you > created the store with the property defined as not-indexed and have just > checked that box later, without regenerating the whole store, the index was > never added. Did you do that, just check it later? Have you regenerated a > complete new store since or are you using a store you've been populating for > a while. > > Here's a particularly ugly idea, purists please stop reading now. We can look > at the store and see if it has an index on that property ... first get up a > terminal window and go to the path where your store is. I'm assuming you have > sqlite3 installed like I do .. it came with the OS as far as I know. > > Your store should be called something.sqlite, let's say it's Foo. Type > > sqlite3 Foo.sqlite > > and that should open the store and give you a prompt. First you want to find > the tables in the store, so type > > .tables > > as far as I can see they are called Z<YOUR ENTITY NAME>, so for you I'd > expect to see one of the tables called ZMCARTICLE. If there is one, you can > find out what indices are on it > > .indices ZMCARTICLE > > I believe again the indices are called Z<YOUR ENTITY NAME>_Z<YOUR ATTRIBUTE > NAME>_INDEX, so you'd expect to find ZMCARTICLE_ZMESSAGEID_INDEX in that > list. If you don't have it, the store wasn't created with that index. If none > of those tables exist at all, my rudimentary reverse engineering of the whole > coredata thing is flawed (or I'm using some entirely different version from > you). > > If the tables and indices exist, including the one on ZMESSAGEID, I'm out of > ideas unless someone knows of a way to put coredata into a form of debug mode > and see the SQL generated to figure out if it's doing anything smart. > > If either none of the above works or it does work but you don't have the > index, you have a couple of options. The right one is to delete your whole > message store and run your app and make a brand new one to see if that then > adds the indexed property with an index. Depending on how you've populated > the store, that might be a real pain, perhaps you can force a migration or > something. The other really stupid idea would be to just add the index and > hope that doesn't break everything entirely which is entirely possible at > which point you delete the store and start over. You would do that by running > > CREATE INDEX ZMCARTICLE_ZMESSAGEID_INDEX ON ZMCARTICLE (ZMESSAGEID); > > Here's another useful thing I just came across, I would certainly run this to > see if the SQL being executed makes sense. > > > With Mac OS X version 10.4.3 and later, you can use the user default > com.apple.CoreData.SQLDebug to log to stderr the actual SQL sent to SQLite. > (Note that user default names are case sensitive.) For example, you can pass > the following as an argument to the application: > > -com.apple.CoreData.SQLDebug 1 > Higher levels of debug numbers produce more information, although using > higher numbers is likely to be of diminishing utility. > > > > I'd love to hear about any other ways people have to debug coredata. I sort > of trust apple has done a good job with it and for it to break down > performance wise on looking for a row in 20,000 with a certain attribute > doesn't make sense to me. If you really can't get it to work, I'd write a > short project which inserts 20,000 simple objects into a store and another > one which opened the store and goes looking for one by attribute in the way > you have. If it takes multi-minutes, I'd sent it to apple as a bug. > > > On 13-Feb-2010, at 6:32 PM, daniele malcom wrote: > >> >> >> On Sat, Feb 13, 2010 at 4:06 AM, Roland King <r...@rols.org> wrote: >> That's not a horrible solution, except for the feeling that core data ought >> to let you do what you want without having to implement your own UUID cache. >> I'm still a bit surprised that a lookup for an object by one attribute is >> taking so long, over just 30,000 objects. You do have the uuid attribute >> marked as indexed right? >> >> I found >> http://cocoawithlove.com/2008/03/testing-core-data-with-very-big.html whilst >> hunting around for some examples of core data with big data sets. This guy >> was working on sets of 1 million objects and doing fetches with indexed >> properties was taking about 2 seconds, vs non-indexed, 600 seconds. There >> are some comments at the bottom from an apple engineer too. >> >> Hi Roland, >> Uhmmm probability there is something wrong with my code, because with >> indexed messageid property it takes a very long time too (I can't see any >> visible difference, 20k messages took minutes to be completed) >> Here you will found my simple storage data model: >> http://img197.imageshack.us/img197/4388/screenshot20100213at111.jpg >> >> This is my code: >> for (NSString *cMessage in messagedata) { >> NSString *idd = [cMessage >> headerValueFor:ARTICLE_TAG_ID]; // takes the MESSAGE UUID >> if ([thegroup articleForID:idd inCtx:ctx] == nil) { // >> CHECK IF MESSAGE UUID IS ALREADY ON DB >> MCArticle *c = [MCArticle newArticleWithID:idd >> context:ctx]; // WE CAN MAKE THE MESSAGE AND ADD IT TO COREDATA >> >> // WE WANT TO CHECK FOR PARENT >> NSString *parentidms = [[[f >> headerValueFor:ARTICLE_TAG_REFS] reverseOrderedReferences] objectAtIndex:0]; >> MCArticle *parent = [thegroup >> articleForID:parentidms inCtx:ctx]; // WE QUERY FOR PARENT UUID >> if (parent != nil) { >> // LINK BOTH PARENT AND CHILD >> } >> } >> } >> >> ArticleForID:inCtx: method follow (inCtx is required becase these functions >> works in a multithreading environment) >> (I've also tried to re-use fetchRequest object but it's not the main >> problem...) >> >> - (MCArticle *) articleForID:(NSString*) _msgid >> inCtx:(NSManagedObjectContext*) ctx { >> if (_msgid == nil) return nil; >> NSFetchRequest *fetchRequest = [[NSFetchRequest alloc] init]; >> [fetchRequest setFetchLimit:1]; >> [fetchRequest setEntity: [NSEntityDescription >> entityForName:@"MCArticle" inManagedObjectContext:ctx]]; >> NSPredicate *predicate = [NSPredicate predicateWithFormat:@"messageid >> == %@",_msgid]; >> [fetchRequest setPredicate: predicate]; >> NSArray *results = [ctx executeFetchRequest:fetchRequest error:nil]; >> [fetchRequest release]; >> if ([results count]==0)return nil; >> return [results objectAtIndex:0]; >> } >> >> Finally the init method for Message object: >> >> + (MCArticle *) newArticleWithID:(NSString *) _messageID >> context:(NSManagedObjectContext *) _context { >> MCArticle *oj = (MCArticle*)[[NSManagedObject alloc] >> initWithEntity:[NSEntityDescription entityForName:@"MCArticle" >> >> >> inManagedObjectContext:_context] >> >> insertIntoManagedObjectContext:_context]; >> [oj setValue:_messageID forKey:@"messageid"]; >> return oj; >> } > > _______________________________________________ > > Cocoa-dev mailing list (Cocoa-dev@lists.apple.com) > > Please do not post admin requests or moderator comments to the list. > Contact the moderators at cocoa-dev-admins(at)lists.apple.com > > Help/Unsubscribe/Update your Subscription: > http://lists.apple.com/mailman/options/cocoa-dev/rols%40rols.org > > This email sent to r...@rols.org _______________________________________________ Cocoa-dev mailing list (Cocoa-dev@lists.apple.com) Please do not post admin requests or moderator comments to the list. Contact the moderators at cocoa-dev-admins(at)lists.apple.com Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com This email sent to arch...@mail-archive.com
