You can page using the reference query. Since it is a query object, just be sure that you order in a "stable" way, then you can use one of the methods suggested in the app engine articles to page. Have you checked to see if as the size of the list property grows get_by_id()'s performance gets closer to the query's performance?
Otherwise you might consider implementing something like Nick's distributed transactions: http://blog.notdot.net/2009/9/Distributed-Transactions-on-App-Engine Robert On Sat, Jan 23, 2010 at 9:14 PM, dburns <drrnb...@gmail.com> wrote: > Thanks for the reply. Using a ReferenceProperty from the SS to the > User does at first sound ideal, but the problem is that the back- > reference that this gives you is actually a Query object. So that > would essentially put me back to where I was initially (as I described > in the second-last paragraph). Using the Query seems to be a bit > slower than fetching all the objects by a list of ids, but the more > important difference is that the list of ids lends itself to paging (I > simply chop the list of ids into groups of 20 or whatever). > > One other optimization I get from using lists of ids (fav_ids and > owned_ids) is that I can optimize the two get_by_id calls into a > single call by concatenating the lists into one, then splitting the > returned list since I know how many are favourites and how many are > owned. There are a lot of things I like about the current set-up I > have. The only thing I don't like is the lack of atomic operations. > If I can't find a way out, I'll try to make the code more resilient to > "corrupted" data just in case it ever happens. > > I appreciate the feedback. > > > On Jan 23, 9:51 pm, Wooble <geoffsp...@gmail.com> wrote: >> Why not use a ReferenceProperty pointing to a User in the SS model >> instead of an unindexed StringProperty? The User model can then use >> the backreference collection to get a list of photos owned by the >> user. >> >> I don't see a problem with a simple ListProperty of favorites, >> although making this a list of db.Keys instead of a list of integer >> IDs is probably preferable, because if you do decide to make photos >> children of another entity the IDs won't be globally unique while the >> db.Keys still will be. >> >> On Jan 23, 7:29 pm, dburns <drrnb...@gmail.com> wrote: >> >> > I'd appreciate any insight into the best design for this problem. I >> > have a photo-sharing app. I want people to be able to mark a photo as >> > a "favourite", just as YouTube does with videos. >> >> > I have two kinds: Snapshots and Users. Given a user, I need to be >> > able to get a list of their favourites, and a list of the snapshots >> > they created. I need the inverse too, i.e. given a snapshot, I need >> > to know who made it. >> >> > The problem I have is ensuring consistency across these two kinds, but >> > I don't think I can put them into the same entity group. First, here >> > they are: >> >> > # The snapshot class. >> > class SS(db.Model): >> > owner = db.StringProperty(indexed=False) # Who created this (user >> > id). >> > #Other data about the snapshot here >> >> > # The user class (the key_name is the user id) >> > class User(db.Model): >> > owned_ids = db.ListProperty(int, indexed=False) # IDs of owned >> > snapshots (i.e. created by this user) >> > fav_ids = db.ListProperty(int, indexed=False) # IDs of >> > favourite snapshots >> >> > The issue is that when a user either creates or deletes a snapshot, >> > there's a potential for those to get out of sync if an exception >> > happens just at the wrong moment (e.g. a snapshot could exist where >> > the creating user doesn't have it in the owned_ids list). >> >> > Making a User instance the parent of a SS (snapshot) instance seems >> > like a natural fit, except then I can't fetch all the favourites via: >> > favs = SS.get_by_id(user.fav_ids). The reason is that all parents >> > have to be the same to use SS.get_by_id (according >> > tohttp://code.google.com/appengine/docs/python/datastore/modelclass.htm...), >> > but those favourites may have been created by various users (hence the >> > parents would be different). >> >> > Originally I had no owned_ids in User, and did a query to find that >> > user's snapshots (owner in SS was indexed). But that was slow and >> > didn't lend itself to paging. So I switched to get_by_id. >> >> > Documentation note: run_in_transaction >> > athttp://code.google.com/appengine/docs/python/datastore/functions.html >> > doesn't mention the restriction that entities have to be in the same >> > group. I discovered it by seeing the exception, then read up in more >> > detail elsewhere. I naturally started there so it probably should be >> > mentioned. >> >> > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to google-appeng...@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
