We are also finding that deletion is an expensive prospect. So much so that we find ourselves discussing the cost-tradeoff of simply leaving the dead data around because the storage cost is so much lower than what it would take to delete - even using the techniques mentioned in this post.
It hurts me to think that we're leaving disks spinning with junk on them. Google, any improvements on this front that would lead to less datastore CPU usage for deletions? j On Mar 10, 5:01 am, andreas schmid <a.schmi...@gmail.com> wrote: > at this point i would delete the app and create a new one at no cost! > > On Mar 10, 2011, at 3:25 AM, Robert Kluin wrote: > > > > > I've tried using the datastore admin to delete some very large > > datasets as well. The solution Bemmu is using will be *significantly* > > faster, and from my experience, should be more cost effective. > > > I'm also eager to hear what Bemmu thinks after the delete has ran for a > > while. > > > Robert > > > On Wed, Mar 9, 2011 at 21:20, David Mora <dla.m...@gmail.com> wrote: > >> hmmm, well Wesley's option B was merely because the batch operations i > >> think. One nice feature about the map reduce is the mutation pool which > >> handles the logic of batching an operation while you yield thru iterations. > >> I guess in a big dataset like yours make sense (the model retrieval vs key > >> only). Anyways, interested case - i'll love to see where it ends :) > > >> On 9 March 2011 19:26, Simon Knott <knott.si...@gmail.com> wrote: > > >>> I've been given the impression from these forums that the datastore admin > >>> tool is so expensive for exactly the reason you've stated David - it loads > >>> each entity by key before deletion, whereas deleting purely on keys is > >>> much > >>> cheaper CPU-wise as you don't need to bother with the retrieval of the > >>> entire entity to carry out the delete. > > >>> -- > >>> You received this message because you are subscribed to the Google Groups > >>> "Google App Engine" group. > >>> To post to this group, send email to google-appengine@googlegroups.com. > >>> To unsubscribe from this group, send email to > >>> google-appengine+unsubscr...@googlegroups.com. > >>> For more options, visit this group at > >>>http://groups.google.com/group/google-appengine?hl=en. > > >> -- > >>http://about.me/david.mora > > >> -- > >> You received this message because you are subscribed to the Google Groups > >> "Google App Engine" group. > >> To post to this group, send email to google-appengine@googlegroups.com. > >> To unsubscribe from this group, send email to > >> google-appengine+unsubscr...@googlegroups.com. > >> For more options, visit this group at > >>http://groups.google.com/group/google-appengine?hl=en. > > > -- > > You received this message because you are subscribed to the Google Groups > > "Google App Engine" group. > > To post to this group, send email to google-appengine@googlegroups.com. > > To unsubscribe from this group, send email to > > google-appengine+unsubscr...@googlegroups.com. > > For more options, visit this group > > athttp://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
