Sorry for my previous nonsense reply... I need more coffee to wake up :) //wbr Pashka R. <pashka.rezni...@gmail.com>
On Tue, Jul 14, 2009 at 12:12, Nick Johnson (Google)<nick.john...@google.com> wrote: > > On Tue, Jul 14, 2009 at 10:06 AM, Pashka R.<pashka.rezni...@gmail.com> wrote: >> >> Hi, >> >> I guess if you will use bulk delete it would help: >> >> delete_me = [] >> for item in Model.all(keys_only=True): >> delete_me.append(item) >> db.delete(delete_me) > > This is not a good approach. Iterating over a Query fetches results in > batches of 20, which means this will require many more round-trips > than would otherwise be necessary. A better approach is: > > db.delete(Model.all(keys_only=True).fetch(BATCH_SIZE)) > > > -Nick Johnson > >> >> >> //wbr Pashka R. <pashka.rezni...@gmail.com> >> >> >> >> On Mon, Jul 13, 2009 at 20:45, Tony<fatd...@gmail.com> wrote: >>> >>> There is no such option, but something that will speed up your deletes >>> significantly is to do a keys_only query. The query you are currently >>> using fetches all 300 entities into memory (including their >>> properties, etc) - but you only need the key to delete the entity. >>> Try something like "query = Homedata.all(keys_only=True)" and you >>> should be able to delete more entities per batch for less cpu. >>> However, it's still going to be a long and cpu-intensive operation for >>> 25000+ entities. >>> >>> On Jul 12, 12:53 pm, jacqueslep <jacquesle...@gmail.com> wrote: >>>> Hello, >>>> i try to delete all records in a datastore, a simple table with a time >>>> serie of int values >>>> there are more than 25000 entries >>>> >>>> when i do : >>>> >>>> homedata_query = Homedata.all() >>>> homedatas = homedata_query.fetch(300) >>>> db.delete(homedatas) >>>> >>>> if the fetch count is greater then 300 i receive a timeout >>>> so i have to repeat this a huge number of time >>>> >>>> it did take me about 20 min to do that >>>> and it cost 2.38 CPU hours because the CPU sec used /sec was up to 8 >>>> This is not useable, i want to store (and be able to delete) more than >>>> 100k entries, >>>> deleting 300 by 300 would take more than one hour to complete and cost >>>> 8h of cpu >>>> >>>> what is the trick to instant delete, like the -c option in the App >>>> Engine SDK ? >>>> >>>> btw can someone explain how the cpu sec used /sec are computed >>> > >>> >> >> > >> > > > > -- > Nick Johnson, App Engine Developer Programs Engineer > Google Ireland Ltd. :: Registered in Dublin, Ireland, Registration > Number: 368047 > > > > --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~----------~----~----~----~------~----~------~--~---
