Re: 回复：Re: [google-appengine] Total Stored Data is larger than the actual use.
The thing to note is that doing this won't affect any of the current data. You'll need to either delete the current data and re-upload with those properties set to unindexed or you need to create your model with the properties set to unindexed and then iterate over the current data and retrieve each entity and then re-put the entity. With the composite indexes, you can delete/vacuum them and re-create them sort of like you can create and drop indexes in a normal relational database, but the individual property indexes can't be manipulated as a whole unit. I hope that makes sense. Also, if you're using Java/JDO you use the following annotation: @Extension(vendorName = datanucleus, key = gae.unindexed, value= true) Other frameworks (Objectify, etc.) use different mechanisms. Some other useful tips for keeping datastore usage to a minimum: 1.) keep your property names short (these names are duplicated for each entity as metadata and also in the indexes, etc.) - I use 3 character names 2.) keep your entity names short (these are duplicated inside the keys, etc.) - I use 2 character names 3.) keep your namespace names short (again duplicated inside of keys) 4.) keep your app-id short (this one is difficult because we all want distinctive names) Why the big deal about keys? Because all these keys get copied for each index entry and for single property indexes you don't get just one index, you get two by default (an ascending and a descending index). So, if you've got 20 indexed properties on an entity, you've got 41 indexes (1 for the primary key and 40 for the properties). All that key overhead adds up quick as you've discovered. As an example, I have 391 MB of data and my datastore usage is 400 MB right now in the stats and dashboard. My case isn't probably typical because I've got more blob usage than most, but I offer it as an example. Hope this helps, Stephen CortexConnect (cortexconnect.appspot.com) http://cortexconnect.appspot.com On Sun, Jul 24, 2011 at 10:38 PM, Robert Kluin robert.kl...@gmail.comwrote: In Python it looks like this: class Example(db.Model): prop = db.StringProperty(indexed=False) The Java syntax is more complex, and I don't recall it from memory. ;) Robert On Mon, Jul 25, 2011 at 00:36, lezizi w...@lezizi.org wrote: I've stared it. By the way, can you show me how to disable indexes? -- You received this message because you are subscribed to the Google Groups Google App Engine group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/DZfwwUK0QrgJ. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Total Stored Data is larger than the actual use.
I recently uploaded a large number of entities, the Datastore Statistics reports the size of all the entities is 59 MBytes. However, the main Dashboard shows that the Total Stored Data is 48% used. Why this situation happens on the Dashboard? My App ID is *chemical-chemdata*, and it is using Master/Slave Replication. lezizi -- You received this message because you are subscribed to the Google Groups Google App Engine group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/-WFXpb7QTxcJ. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Total Stored Data is larger than the actual use.
Hi, There are a number of reasons you might see this: 1) indexes are not included in the datastore stats, 2) there is a delay between loading the data and the datastore stats being updated, 3) tasks and blobstore blobs are included in the storage quota but not datastore stats. Indexes are probably one of the biggest factors. Remember the appid, namespace, kind name, and property name(s) and value(s) are all included in every single index row. Also, you have to explicitly disable indexes on most property types. If you'd like stats on indexes star issue 2740: http://code.google.com/p/googleappengine/issues/detail?id=2740 Robert On Sun, Jul 24, 2011 at 01:55, lezizi w...@lezizi.org wrote: I recently uploaded a large number of entities, the Datastore Statistics reports the size of all the entities is 59 MBytes. However, the main Dashboard shows that the Total Stored Data is 48% used. Why this situation happens on the Dashboard? My App ID is chemical-chemdata, and it is using Master/Slave Replication. lezizi -- You received this message because you are subscribed to the Google Groups Google App Engine group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/-WFXpb7QTxcJ. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
回复：Re: [google-appengine] Total Stored Data is larger than the actual use.
I've stared it. By the way, can you show me how to disable indexes? -- You received this message because you are subscribed to the Google Groups Google App Engine group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/DZfwwUK0QrgJ. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: 回复：Re: [google-appengine] Total Stored Data is larger than the actual use.
In Python it looks like this: class Example(db.Model): prop = db.StringProperty(indexed=False) The Java syntax is more complex, and I don't recall it from memory. ;) Robert On Mon, Jul 25, 2011 at 00:36, lezizi w...@lezizi.org wrote: I've stared it. By the way, can you show me how to disable indexes? -- You received this message because you are subscribed to the Google Groups Google App Engine group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/DZfwwUK0QrgJ. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
