Hi Eli, Did you happen to look at the App Engine Console after the datastore was "napping"? A few months ago when clearing a test datastore we hit a similar thing. When we looked at the App Engine Console it said the app was being temporarily throttled.
Was just curious if that is what you encountered too? Robert On Sat, Feb 20, 2010 at 11:49 PM, Eli Jones <eli.jo...@gmail.com> wrote: > As a side note.. once I hit about 250,000 entities deleted it seems the > datastore took a nap on me.. > So, now I'm waiting for it to finish whatever it is doing underneath before > I can continue deleting. > Took a nap = db.delete() or .fetch(1) from the model times out. > Though, I can .fetch() from my other Models just fine. I figure it is just > shuffling the data around and merging it to new tablets. > Granted.. the Model has been unavailable (I'm just judging this by seeing if > I can do a .fetch(1) using the appengine_console from my local) for 40 > minutes.. which is much longer than the brief period of time mentioned for > tablet unavailability here: > http://code.google.com/appengine/articles/handling_datastore_errors.html > Ah.. seems I still can't .fetch from appengine_console, but I can sort of > limp along and delete 100 at a time (but it takes about 16 seconds to delete > 100 entities now) using the task I have set up to do this. > > On Sat, Feb 20, 2010 at 9:54 PM, Eli Jones <eli.jo...@gmail.com> wrote: >> >> I am currently going through the process of deleting 500,000 entities from >> my datastore. >> Here are the different stats I have so far >> db.delete() for: >> 100 entities = 2,179 API_CPU >> 200 entities = 4,345 API_CPU >> 500 entities = 10,845 API_CPU >> So.. it doesn't seem like you get better per entity API_CPU for deleting >> more at once. It seems to average about 21 API_CPU per entity deleted. >> There doesn't even really seem to be a general time benefit either. It >> seems to average about 1 to 2 seconds per 100 entities deleted. >> On Sat, Feb 20, 2010 at 3:21 AM, kang <areyouloo...@gmail.com> wrote: >>> >>> I'm going to clear the datastore. I use the following code: >>> old_date = datetime.datetime(2009,10,1) >>> old_updates = SomeUpdate.all().filter("updated <",old_date).fetch(20) >>> db.delete(old_updates) >>> it costs me nearly 1982cpu_ms 1945api_cpu_ms every time. Is it normal? >>> >>> -- >>> Stay hungry,Stay foolish. >>> >>> -- >>> You received this message because you are subscribed to the Google Groups >>> "Google App Engine" group. >>> To post to this group, send email to google-appeng...@googlegroups.com. >>> To unsubscribe from this group, send email to >>> google-appengine+unsubscr...@googlegroups.com. >>> For more options, visit this group at >>> http://groups.google.com/group/google-appengine?hl=en. >> > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to google-appeng...@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
