Ran into a gotcha with my App today. I have implemented a User Model
that has a couple of properties (facebook id and email address) that
need to be unique across all User instances in the datastore. I
implemented my own "get_or_insert" method that I run in a
transaction. The get_or_insert method queries for either or both
properties (fb uid or email) depending on parameters. If no object is
found, a new one is inserted.
My problem is there are cases where I want to do this for dozens of
items in one request. The execution time I am seeing on the queries
to check for existing items is prohibitively long. I am getting time
outs when trying to process (get_or_insert) 100 items. I've
experiemented with not using the transaction just to see if it would
change the overall time. The impact was minimal.
I've experiemented with doing bulk puts (putting a list of Model
instances at once). This was great, but does not help as the queries
are still needed to see if the items exist and to try to enforce
uniqueness.
Any one have any advice on an alternative approach that may work?
I've thought about doing it asynchronously using the task queue, but
that does not work with the user experience and flow of the
application. Right now, I think a new approach is needed that avoids
creating the objects at this point in the application flow. Here's
the code that implements the get_or_insert and is run as a
transaction:
def _run_user_get_or_insert(q_by_email, q_by_fbuid, confirm_user,
new_user):
user = None
updated = False
if q_by_email:
user = q_by_email.get()
if not user:
if q_by_fbuid:
user = q_by_fbuid.get()
if not user:
user = new_user
updated = True
if confirm_user and not user.is_confirmed:
user.is_confirmed = True
updated = True
if updated:
user.put()
return user
Running on app engine, it takes ~16s to create 100 new users.
Thanks for your help.
-Scott
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en
-~----------~----~----~----~------~----~------~--~---
