If you delete many entities it's faster to use a cursor. An entity is
marked as deleted and will physically be removed in the datastore at a
later time. It takes time to skip the marked entities when executing a
query.
The loop can be modified as such, no need to use count() as it uses
CPU and time also, if the result set is empty there are no entities.
try:
cursor = None
while True:
q = Documents.all(keys_only=True)
if cursor:
q.with_cursor(cursor)
result = q.fetch(200)
if not result:
break
cursor = q.cursor()
db.delete(result)
except Exception, e:
self.response.out.write(repr(e)+'\n')
pass
Op 16 september 2011 18:24 heeft sofia sofiacard...@gmail.com het
volgende geschreven:
I have the same problem.. Datastore quota at 100% and unable to delete an
entity either through admin or using map/reduce. What I did is set up a
script to delete x records at a time. I've managed to decrease data by 45%
in 2 days but I then hit cpu quota so I'm guessing it's gonna take a few
more days until I'm able to delete all the data. What i did was set up this
script, bulkdelete.py:
#!/usr/bin/env python
# -*- coding: UTF-8 -*-
#
#
import time
from google.appengine.ext import webapp
from google.appengine.ext.webapp.util import run_wsgi_app
from google.appengine.ext import db
from lib.model import Documents
class BulkDelete(webapp.RequestHandler):
def get(self):
self.response.headers['Content-Type'] = 'text/plain'
mod = self.request.get('m')
if not mod:
exit
try:
while True:
q = db.GqlQuery(SELECT __key__ FROM Documents ORDER BY date
ASC)
assert q.count()
db.delete(q.fetch(200))
time.sleep(0.5)
except Exception, e:
self.response.out.write(repr(e)+'\n')
pass
# init
application = webapp.WSGIApplication([('/bulkdelete',
BulkDelete)],debug=True)
def main():
run_wsgi_app(application)
if __name__ == '__main__':
main()
and then call it each 5 min through cron.yaml.
--
You received this message because you are subscribed to the Google Groups
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.