Yeah, I really wish there was better visibility into what's using storage space, particular indexes since that impacts performance. Do you use tasks or blobstore for anything? Stored 'tasks bytes' count against the Storage Quota too, and it doesn't reset as rapidly as the task-count does last time I checked.
I harass "my" Googlers about the index stats issue pretty often; maybe if you do too they'll give in and get someone on it. ;) Robert On Wed, May 25, 2011 at 00:10, Brandon Wirtz <drak...@digerat.com> wrote: > I did that. I thought since there was still a discrepancy I'd switch to > non-indexed types, in case it was a bug. > > My use case is "Put something" "Get By Key" "Update something" So I want it > to not have any indexes. > > -Brandon > > > -----Original Message----- > From: google-appengine@googlegroups.com > [mailto:google-appengine@googlegroups.com] On Behalf Of Robert Kluin > Sent: Tuesday, May 24, 2011 7:15 PM > To: google-appengine@googlegroups.com > Subject: Re: [google-appengine] Still confused about Storage reported for > billing and Datastore > > Yeah index stats should really be shown somewhere. This is a very common, > confusing, and opaque issue. If you have not already, the issue to star is > 2740. > http://code.google.com/p/googleappengine/issues/detail?id=2740 > > If you've got a lot of data having to map over it all and re-put each entity > to clear the unneeded indexes can cost a lot of CPU time, possibly a couple > years worth of storage fees. > > Also, you don't have to use blob properties to avoid indexing, you can also > set the property's 'indexed' parameter to False in your model definitions. > (sorry if this was already stated, I only skimmed this > thread.) > > > Robert > > > > > > On Thu, May 19, 2011 at 13:47, Brandon Wirtz <drak...@digerat.com> wrote: >> I think I went out of my way to use non-indexable types, but I'll make >> sure that is the case. I often store things as type blob to prevent > indexing. >> But I may have set some things to the wrong type. >> >> You don't think Vacuuming would help with this? >> >> Considering all the other data shown in Quota Details it would be nice >> if Google exposed this. >> >> Thanks for the help I appreciated it both times. >> >> >> -----Original Message----- >> From: google-appengine@googlegroups.com >> [mailto:google-appengine@googlegroups.com] On Behalf Of Stephen >> Johnson >> Sent: Thursday, May 19, 2011 10:42 AM >> To: google-appengine@googlegroups.com >> Subject: Re: [google-appengine] Still confused about Storage reported >> for billing and Datastore >> >> Hi Brandon, >> Different Steve here. Anyway, when I mentioned in a different thread >> about indexes, you also have to count your property indexes as well. >> For each property that you have indexed (and these wouldn't show up in >> your configured indexes) you have two indexes defined: one ascending >> and one descending. If these properties have large amounts of data, >> for example, string/text data then it is copied to those indexes as >> well as the keys so if you have very long key values then it is in >> those indexes. Also, if you have long property names those are copied >> over and over again with each piece of data. In a different thread I >> posted some general guidelines that help to keep these numbers low. >> Steve >> >> On Thu, May 19, 2011 at 9:02 AM, Brandon Wirtz <drak...@digerat.com> > wrote: >>> Yes, Yes, Yes. >>> >>> $9 was the amount of Mt. Dew Per day, not per month. Currently it's >>> across >>> 10 apps, but in the next month it is across 15k domains. Rounding >>> Errors could pay for another 3 interns at that scale. >>> >>> Also because the number seems to be so close to 2.5:1 I wonder if >>> this is a bug. I don't mind paying if that is the pricing. >>> >>> Really All I am looking for is, is it accurate, and if so where is >>> the other storage being used, and do I have control over it. I >>> reduced my bill about 60% last week through re-writing my code, and >>> adding better caching and memory management. If I can find another >>> 10% that goes >> to my bottom line. >>> So Capitalism is driving my innovation. >>> >>> >>> >>> >>> -----Original Message----- >>> From: google-appengine@googlegroups.com >>> [mailto:google-appengine@googlegroups.com] On Behalf Of Stephen >>> Sent: Thursday, May 19, 2011 5:37 AM >>> To: google-appengine@googlegroups.com >>> Subject: Re: [google-appengine] Still confused about Storage reported >>> for billing and Datastore >>> >>> On Thu, May 19, 2011 at 8:36 AM, Brandon Wirtz <drak...@digerat.com> >> wrote: >>>> >>>> I still have 2.5x as much billed usage for HR Storage as I have >>>> reported by the datastore admin. >>>> ... >>>> It's only 10 cents a day on most my apps, but that's $3 a month. >>>> (per >>>> app) which isn't a big deal yet, but it adds up across lots of apps. >>> >>> That's less than you spend on Mountain Dew each month. >>> >>> Have you tried putting Ad Sense on these pages? >>> >>> This aint no hippy commune. This is capitalism and that's what drives >>> innovation. >>> >>>> I don't mind if I'm really using it... >>> >>> Interesting perspective - "Pay for only for what you use", as Amazon >>> would say. >>> >>> :-) >>> >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "Google App Engine" group. >>> To post to this group, send email to google-appengine@googlegroups.com. >>> To unsubscribe from this group, send email to >>> google-appengine+unsubscr...@googlegroups.com. >>> For more options, visit this group at >>> http://groups.google.com/group/google-appengine?hl=en. >>> >>> >>> -- >>> You received this message because you are subscribed to the Google >>> Groups >> "Google App Engine" group. >>> To post to this group, send email to google-appengine@googlegroups.com. >>> To unsubscribe from this group, send email to >> google-appengine+unsubscr...@googlegroups.com. >>> For more options, visit this group at >> http://groups.google.com/group/google-appengine?hl=en. >>> >>> >> >> -- >> You received this message because you are subscribed to the Google >> Groups "Google App Engine" group. >> To post to this group, send email to google-appengine@googlegroups.com. >> To unsubscribe from this group, send email to >> google-appengine+unsubscr...@googlegroups.com. >> For more options, visit this group at >> http://groups.google.com/group/google-appengine?hl=en. >> >> >> -- >> You received this message because you are subscribed to the Google Groups > "Google App Engine" group. >> To post to this group, send email to google-appengine@googlegroups.com. >> To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com. >> For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. >> >> > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to google-appengine@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to google-appengine@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
