All depends on what you are doing. I ended up denormalizing my data model. So I could retrieve my Images by a key or id. I use another table to link stuff together.
class ThumbStore(db.Model): thumbId=db.IntegerProperty() imageId=db.IntegerProperty() thumbnail=db.BlobProperty() filename=db.StringProperty() # just for ref in dataviewer class StoryIdx(db.Model): storyId=db.IntegerProperty() thumbId=db.IntegerProperty() imageId=db.IntegerProperty() textId = db.IntegerProperty() So instead of querying the ThumbStore that will hit a dead line error. I search the StoryIdx table first to find the keys I need in the Thumb table. see my site: http://www.hikejournal.com On Thursday, August 23, 2012 2:26:00 PM UTC-5, Phil wrote: > > In some initialization work my app needs to run through the all of the > datastore entities of a given kind. I have a lot of these entities (80k > currently) and it's increasing rapidly. I'm currently trying to read these > in using a single datastore query, but running up against the default > datastore timeout of 30 seconds. > > Is there a good practice for sharding this or otherwise breaking this up > so that I won't hit these deadlines? I was thinking I would do a keyOnly > query and then break up the keys into a number of reasonably sized > sub-queries, but perhaps there is a better approach out there? > > Thanks, > Phil > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/z4-1VTymnjUJ. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.