On Thu, Jul 30, 2009 at 7:39 PM, someone1<someo...@gmail.com> wrote:
>
> Hey,
>
> Thanks for the quick reply. Won't the task scheduler still take away
> from the CPU time?
Yes, but it runs when you want it, not every minute.
>
> Also, if I were to utilize task queues, would this be how I should
> approach it?
>
> taskqueue.add(url='/batchDelete', params={'key': key})
>
> and the /batchDelete would be something like:
>
> parentObj = db.get(self.request.get('key'))
> try:
> keys = TrackingResult.all(keys_only=True).ancestor(parentObj).fetch
> (100)
> while keys:
> db.delete(keys)
> keys = TrackingResult.all(keys_only=True).ancestor
> (parentObj).fetch(100)
> except DeadlineExceedError:
> taskqueue.add(url='/batchDelete', params={'key': str(parentObj.key
> ())})
That's more or less the best bet. You can have multiple delete tasks
queued concurrently, but unfortunately doing so is tough, since you
have to select different key ranges to delete in each task (otherwise
you'll have multiple tasks trying to delete the same entities).
-Nick Johnson
>
>
> Or is there also a better way of doing that?
> On Jul 30, 2:12 pm, "Nick Johnson (Google)" <nick.john...@google.com>
> wrote:
>> Hi someone1,
>>
>> On Thu, Jul 30, 2009 at 7:02 PM, someone1<someo...@gmail.com> wrote:
>>
>> > Hey,
>>
>> > My app has about 800MB of data that I need to delete. So I made a
>> > cronjob to run every minute that runs the following code:
>>
>> > while TrackingResult.all().count(1) > 0:
>> > db.delete(db.GqlQuery("SELECT __key__ FROM
>> > TrackingResult").fetch(100))
>>
>> You're executing two queries for every batch of results you delete,
>> here. A more efficient options:
>>
>> keys = TrackingResult.all(keys_only=True).fetch(100)
>> while keys:
>> db.delete(keys)
>> keys = TrackingResult.all(keys_only=True).fetch(100)
>>
>> You might also want to look into using the task queue to do bulk
>> deletions like this.
>>
>>
>>
>> > TrackingResult is my entity model with 800MB of data i want gone. The
>> > problem is, I've had this running for 2 days now and only ~350MB has
>> > been deleted. In fact, after about 200MB of data is deleted, all of my
>> > CPU quota is used up after a few hours and I must wait until the next
>> > reset to finish deleting the data.
>>
>> > I have no idea why this is taking so long or if there is a more
>> > efficient way to do this. I do have about 15 indexes, so maybe that
>> > can be adding to this all? I really just wish there was a way to
>> > "truncate" the table...
>>
>> Deleting the indexes first will make deleting the entities cheaper.
>>
>> -Nick Johnson
>>
>>
>>
>> > Does anybody have any help/advice as to what I may be doing wrong? The
>> > entity is a Expando model and has no more than 10-12 properties on it,
>> > one of which is a reference property (but all references were already
>> > deleted).
>>
>> --
>> Nick Johnson, Developer Programs Engineer, App Engine
> >
>
--
Nick Johnson, Developer Programs Engineer, App Engine
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en
-~----------~----~----~----~------~----~------~--~---
