On Sat, Oct 10, 2009 at 6:27 PM, Jason Smith <j...@proven-corporation.com>wrote:
> > Thanks for the help guys. I think this is an important matter to have > cleared up. > > It's bedtime here (GMT+7) however tomorrow I think I will do some > benchmarks along the lines of the example I wrote up in the SO > question. > > At this point I would think the safest thing would be to completely > change the model name, thereby guaranteeing that you will be writing > entities with fresh keys. However I suspect it's not necessary to go > that far. I'm thinking that on the production datastore, changing the > model definition and then re-put()ing the entity will be what's > required to realize a speed benefit when reducing the number of > properties on a model. But the facts will speak for themselves. > There's no need to use a new model name: You can simply create new entities to replace the old ones, under the current model name. If you're using key names, you can construct a new entity with the same values as the old ones, and store that. You can also use the low-level API in google.appengine.api.datastore; this provides a dict-like interface from which you can delete unwanted fields. -Nick Johnson > On Oct 11, 12:17 am, Andy Freeman <ana...@earthlink.net> wrote: > > > In other words: if I want to reduce the size of my entities, is > > > it necessary to migrate the old entities to ones with the new > > > definition? > > > > I'm pretty sure that the answer to that is yes. > > > > > If so, is it sufficient to re-put() the entity, or must I > > > save under a wholly new key? > > > > I think that it should be sufficient re-put() but decided to test that > > hypothesis. > > > > It isn't sufficient in the SDK - the SDK admin console continues to > > show values for properties that you've deleted from the model > > definition after the re-put(). Yes, I checked to make sure that those > > properties didn't have values before the re-put(). > > > > I did the get and re-put() in a transaction, namely: > > > > def txn(key): > > obj = Model.get(key) > > obj.put() > > assert db.run_in_transaction(txn, key) > > > > I tried two things to get around this problem. The first was to add > > db.delete(obj.key()) right before obj.put(). (You can't do obj.delete > > because that trashes the obj.) > > > > The second was to add "obj.old_property = None" right before the > > obj.put() (old_property is the name of the property that I deleted > > from Model's definition.) > > > > Neither one worked. According to the SDK's datastore viewer, existing > > instances of Model continued to have values for old_property after I > > updated them with that transaction even with the two changes, together > > or separately. > > > > If this is also true of the production datastore, this is a big deal. > > > > On Oct 10, 4:44 am, Jason Smith <j...@proven-corporation.com> wrote: > > > > > > > > > Hi, group. My app's main cost (in dollars and response time) is in the > > > db.get([list, of, keys, here]) call in some very high-trafficked code. > > > I want to pare down the size of that model to the bare minimum with > > > the hope of reducing the time and CPU fee for this very common > > > activity. Many users who are experiencing growth in the app popularity > > > probably have this objective as well. > > > > > I have two questions that hopefully others are thinking about too. > > > > > 1. Can I expect the API time of a db.get() with several hundred keys > > > to reduce roughly linearly as I reduce the size of the entity? > > > Currently the entity has the following data attached: 9 String, 9 > > > Boolean, 8 Integer, 1 GeoPt, 2 DateTime, 1 Text (avg size ~100 bytes > > > FWIW), 1 Reference, 1 StringList (avg size 500 bytes). The goal is to > > > move the vast majority of this data to related classes so that the > > > core fetch of the main model will be quick. > > > > > 2. If I do not change the name of the entity (i.e. just delete all the > > > db.*Property definitions in the model), will I still incur the same > > > high cost fetching existing entities? The documentation says that all > > > properties of a model are fetched simultaneously. Will the old > > > unneeded properties still transfer over RPC on my dime and while users > > > wait? In other words: if I want to reduce the size of my entities, is > > > it necessary to migrate the old entities to ones with the new > > > definition? If so, is it sufficient to re-put() the entity, or must I > > > save under a wholly new key? > > > > > Thanks very much to anyone who knows about this matter! > > > -- Nick Johnson, Developer Programs Engineer, App Engine Google Ireland Ltd. :: Registered in Dublin, Ireland, Registration Number: 368047 --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~----------~----~----~----~------~----~------~--~---
