As a side note.. once I hit about 250,000 entities deleted it seems the datastore took a nap on me..
So, now I'm waiting for it to finish whatever it is doing underneath before I can continue deleting. Took a nap = db.delete() or .fetch(1) from the model times out. Though, I can .fetch() from my other Models just fine. I figure it is just shuffling the data around and merging it to new tablets. Granted.. the Model has been unavailable (I'm just judging this by seeing if I can do a .fetch(1) using the appengine_console from my local) for 40 minutes.. which is much longer than the brief period of time mentioned for tablet unavailability here: http://code.google.com/appengine/articles/handling_datastore_errors.html Ah.. seems I still can't .fetch from appengine_console, but I can sort of limp along and delete 100 at a time (but it takes about 16 seconds to delete 100 entities now) using the task I have set up to do this. <http://code.google.com/appengine/articles/handling_datastore_errors.html> On Sat, Feb 20, 2010 at 9:54 PM, Eli Jones <eli.jo...@gmail.com> wrote: > I am currently going through the process of deleting 500,000 entities from > my datastore. > > Here are the different stats I have so far > > db.delete() for: > > 100 entities = 2,179 API_CPU > 200 entities = 4,345 API_CPU > 500 entities = 10,845 API_CPU > > So.. it doesn't seem like you get better per entity API_CPU for deleting > more at once. It seems to average about 21 API_CPU per entity deleted. > > There doesn't even really seem to be a general time benefit either. It > seems to average about 1 to 2 seconds per 100 entities deleted. > > On Sat, Feb 20, 2010 at 3:21 AM, kang <areyouloo...@gmail.com> wrote: > >> I'm going to clear the datastore. I use the following code: >> >> old_date = datetime.datetime(2009,10,1) >> old_updates = SomeUpdate.all().filter("updated <",old_date).fetch(20) >> db.delete(old_updates) >> >> it costs me nearly 1982cpu_ms 1945api_cpu_ms every time. Is it normal? >> >> >> -- >> Stay hungry,Stay foolish. >> >> -- >> You received this message because you are subscribed to the Google Groups >> "Google App Engine" group. >> To post to this group, send email to google-appeng...@googlegroups.com. >> To unsubscribe from this group, send email to >> google-appengine+unsubscr...@googlegroups.com<google-appengine%2bunsubscr...@googlegroups.com> >> . >> For more options, visit this group at >> http://groups.google.com/group/google-appengine?hl=en. >> > > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
