Last time I used the Datastore Admin to delete a lot of entities it
was amazingly slow, and used a ton of resources. Perhaps that has
changed in the past couple months though? (Sorry Greg ;)
I've found something simple works really well, something like this in a task:
def nuke_stuff(kind, batch_size=1000):
query = kind.all(keys_only=True)
keys = query.fetch(batch_size)
cursor = query.cursor()
while True:
db.delete(keys)
if len(keys) < batch_size:
break
if cursor:
query.with_cursor(cursor)
keys = query.fetch(batch_size)
cursor = query.cursor()
Just stick it in a task. If there is a lot of data it will time
out, causing it to re-run. If you want to get fancy there are several
nice optimizations, you can use. For instance, my production version
of this inserts 'continuation' deleter tasks upfront and does not use
the while loop. You could also spiff up this version just by using
the newly exposed async methods.
Robert
On Fri, May 13, 2011 at 18:01, Brandon Wirtz <drak...@digerat.com> wrote:
> Yeah, That's great for a one-off, but I actually want the equivalent of a
> memcache flush all that I can Cron.
>
>
> -----Original Message-----
> From: google-appengine@googlegroups.com
> [mailto:google-appengine@googlegroups.com] On Behalf Of Greg Darke (Google)
> Sent: Friday, May 13, 2011 2:22 PM
> To: google-appengine@googlegroups.com
> Subject: Re: [google-appengine] Drop Table equivalent?
>
> The currently recommended way of deleting all entities of a particular kind
> is to use datastore admin
> (http://code.google.com/appengine/docs/adminconsole/datastoreadmin.html#Dele
> ting_Entities_in_Bulk).
>
> If the primary version of your code is not in python, you can upload a
> simple application to an alternate version that has datastore admin enabled.
>
> On 13 May 2011 13:12, Brandon Wirtz <drak...@digerat.com> wrote:
>> I know people have asked variations of this question, and the solution
>> has always been to drop entities 1000 at a time,
>>
>>
>>
>> But have an app that the cleanup code had a misspelling so it never
>> did anything except clean things that didn’t exist…
>>
>>
>>
>> Is there a “Drop Table” equivalent? Or do I really query 1000 and
>> delete over and over. Cause it appears that’s going to be like $100
>> which is fine…. If that is the best solution, but if there is a drop
>> table equivalent that is $5 I’d really like that so that I can use it
>> in my code that would normally get charged the $100 over 6 months …
>>
>>
>>
>>
>>
>> --
>> You received this message because you are subscribed to the Google
>> Groups "Google App Engine" group.
>> To post to this group, send email to google-appengine@googlegroups.com.
>> To unsubscribe from this group, send email to
>> google-appengine+unsubscr...@googlegroups.com.
>> For more options, visit this group at
>> http://groups.google.com/group/google-appengine?hl=en.
>>
>
> --
> You received this message because you are subscribed to the Google Groups
> "Google App Engine" group.
> To post to this group, send email to google-appengine@googlegroups.com.
> To unsubscribe from this group, send email to
> google-appengine+unsubscr...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/google-appengine?hl=en.
>
>
> --
> You received this message because you are subscribed to the Google Groups
> "Google App Engine" group.
> To post to this group, send email to google-appengine@googlegroups.com.
> To unsubscribe from this group, send email to
> google-appengine+unsubscr...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/google-appengine?hl=en.
>
>
--
You received this message because you are subscribed to the Google Groups
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.
