[google-appengine] Re: RESOLVED: why is the bulkloader so slow?

2009-09-28 Thread djerdo
On 28 Sep, 10:15, "Nick Johnson (Google)" wrote: > > You could do this much more efficiently by loading the list of keys into a > list (say, from a text file), and deleting them in batches, something like > this: > > f = iter(open("todelete.txt", "r")) > for batch in zip(*([f]*20)): >   db.dele

[google-appengine] Re: RESOLVED: why is the bulkloader so slow?

2009-09-28 Thread Nick Johnson (Google)
On Sun, Sep 27, 2009 at 1:07 PM, djerdo wrote: > > Hi Nick, > > Thanks, that's more straightforward. Somewhat related, I'm creating an > auto-generated 'bulk-delete' script that uses the 'remote_api' > handler. Since i know the key_name for each entity that I'm uploading, > I create a script whic

[google-appengine] Re: RESOLVED: why is the bulkloader so slow?

2009-09-27 Thread djerdo
Hi Nick, Thanks, that's more straightforward. Somewhat related, I'm creating an auto-generated 'bulk-delete' script that uses the 'remote_api' handler. Since i know the key_name for each entity that I'm uploading, I create a script which looks something like (ignoring imports): deleteall_my

[google-appengine] Re: RESOLVED: why is the bulkloader so slow?

2009-09-26 Thread Nick Johnson (Google)
Hi djerdo, You can specify key names by overriding the generate_key method of your loader. See here for more details: http://blog.notdot.net/2009/9/Advanced-Bulk-Loading-part-2-Customization -Nick Johnson On Sat, Sep 26, 2009 at 1:54 PM, djerdo wrote: > > > > On 17 Sep, 23:25, djerdo wrote: >