Hi djerdo,
You can specify key names by overriding the generate_key method of your
loader. See here for more details:
http://blog.notdot.net/2009/9/Advanced-Bulk-Loading-part-2-Customization
-Nick Johnson
On Sat, Sep 26, 2009 at 1:54 PM, djerdo <grflana...@gmail.com> wrote:
>
>
>
> On 17 Sep, 23:25, djerdo <grflana...@gmail.com> wrote:
> > Using the bulkloader with the dev appserver, the script adds rows
> > (Entities) at a progressively slower rate, to the point where it
> > becomes unusable when the csv file is large (20,000 rows). Why? Is
> > this a known issue? Are there any workarounds?
> >
> > Thanks
>
> Ok, I wasn't doing it right. FWIW - to resolve reference property
> fields, I was using code such as:
>
> select __key__ from MyModel where id = :1
>
> id being the unique field in the incoming csv file. This seemed to
> work but when i figured out how to specify a key_name when adding an
> entity, I could replace this with
>
> MyModel.get_by_key_name(id).key()
>
> which sped things up a lot.
>
> How to specify a key name when bulkloading entities? I couldn't find
> any examples of this, so (again FWIW), here's a 'Baseloader' class
> that i used:
>
> from google.appengine.api import datastore
> from google.appengine.ext import bulkload
>
> class BaseLoader(bulkload.Loader):
> #abstract class
>
> def __init__(self):
> bulkload.Loader.__init__(self, self.model, self.mapping)
>
> def HandleEntity(self, entity):
> if not hasattr(self, 'unique_field'):
> return entity
> #entity is a dict subclass
> keyname = entity.pop(self.unique_field)
> named_entity = datastore.Entity(self.model, name=keyname)
> named_entity.update(entity)
> return named_entity
>
> Example:
>
> class ContactLoader(Loader):
> model = 'Contact'
> mapping = [
> ('id', str),
> ('name', str),
> ('phone', str),
> ]
> unique_field = 'id'
>
>
>
>
> >
>
--
Nick Johnson, Developer Programs Engineer, App Engine
Google Ireland Ltd. :: Registered in Dublin, Ireland, Registration Number:
368047
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en
-~----------~----~----~----~------~----~------~--~---
