So this technically would work in reducing total size, but I haven't heard of this resulting in significant cost reductions for people. The indexes are stored as protocol buffers, so they're fairly space efficient ( http://code.google.com/apis/protocolbuffers/docs/overview.html). A better strategy would be just to reduce the number of indexes you use - you can do this by running a Mapper job (http://code.google.com/p/appengine-mapreduce/) over all your entities, reading them and resaving many of the properties as unindexed ( http://code.google.com/appengine/docs/python/datastore/queries.html#Introduction_to_Indexes ).
You need indexes for query filters and orders. Based on this entity definition: title: string category: int address: string city: string state: int zip_code: string country: string telephone: string website: string menu: text latitude: double longitude: double source: string created: datetime updated: datetime You are probably NEVER going to need an index on menu, website, telephone or address. Ikai Lan Developer Programs Engineer, Google App Engine Blog: http://googleappengine.blogspot.com Twitter: http://twitter.com/app_engine Reddit: http://www.reddit.com/r/appengine On Tue, May 24, 2011 at 7:38 AM, Barry Hunter <barrybhun...@gmail.com>wrote: > On 24 May 2011 15:06, Richard Baron Penman <richar...@gmail.com> wrote: > > I want to fit the datastore into 1GB because this app is for another > client > > who isn't expecting to need billing enabled. > > Is there documentation about these default indices? > > http://code.google.com/appengine/docs/python/datastore/propertyclass.html > (see "indexed") > > but also hinted at here: > > http://code.google.com/appengine/docs/python/datastore/queries.html#Introduction_to_Indexes > > Lots of detail here: > http://code.google.com/appengine/articles/storage_breakdown.html > > > > > > > > On Tue, May 24, 2011 at 11:52 PM, Barry Hunter <barrybhun...@gmail.com> > > wrote: > >> > >> Remember that every single property (well excepting blob/text) has 2 > >> indexes - automatically, unless you explicity say that a property is > >> to be unindexed. > >> > >> Then again 1.5G is only $5-6 a year, even using the high replication > >> datastore. > >> > >> To reduce the storage use, could set properties to be unindexed. But > >> would have to loop over all the entities and re-put them. Probably end > >> up costing you more than just keeping them. > >> > >> > >> On 24 May 2011 14:17, Plumo <richar...@gmail.com> wrote: > >> > hello, > >> > my app has 396MB of entities but is using 1.48GB of storage. I > >> > understand > >> > this may be because the indices add a lot of overhead. > >> > I have 1 index: category ▲ , state ▲ , title ▲ > >> > and 1 big table: > >> > > >> > title: string > >> > category: int > >> > address: string > >> > city: string > >> > state: int > >> > zip_code: string > >> > country: string > >> > telephone: string > >> > website: string > >> > menu: text > >> > latitude: double > >> > longitude: double > >> > source: string > >> > created: datetime > >> > updated: datetime > >> > > >> > Any advice how to reduce this excessive storage overhead? > >> > Richard > >> > > >> > -- > >> > You received this message because you are subscribed to the Google > >> > Groups > >> > "Google App Engine" group. > >> > To post to this group, send email to > google-appengine@googlegroups.com. > >> > To unsubscribe from this group, send email to > >> > google-appengine+unsubscr...@googlegroups.com. > >> > For more options, visit this group at > >> > http://groups.google.com/group/google-appengine?hl=en. > >> > > >> > >> -- > >> You received this message because you are subscribed to the Google > Groups > >> "Google App Engine" group. > >> To post to this group, send email to google-appengine@googlegroups.com. > >> To unsubscribe from this group, send email to > >> google-appengine+unsubscr...@googlegroups.com. > >> For more options, visit this group at > >> http://groups.google.com/group/google-appengine?hl=en. > >> > > > > -- > > You received this message because you are subscribed to the Google Groups > > "Google App Engine" group. > > To post to this group, send email to google-appengine@googlegroups.com. > > To unsubscribe from this group, send email to > > google-appengine+unsubscr...@googlegroups.com. > > For more options, visit this group at > > http://groups.google.com/group/google-appengine?hl=en. > > > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to google-appengine@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
