> They are raised inside a transaction, when a conflict is detected with
> another concurrent transaction. The transaction infrastructure will catch
> and retry these several times, and only raise it in the external code if it
> was unable to execute the transaction after several retries.
Yes, but when are conflicts checked? Specifically, is the error
always raised by the statement in the user function that runs into the
conflict or can it be raised later, say during transaction commit.
I've looked at the SDK's implementation of
RunInTransactionCustomRetries (in google/appengine/api/datastore.py).
The except that catches the CONCURRENT_TRANSACTION exception protects
the commit and not the execution of the user function. That suggests
that the user function is run to completion regardless of conflicts
and that the conflict isn't acted upon until a commit is tried.
However, your description and the documentation suggests the real
implementation detects and acts on conflicts while running the user
function.
Here's a user function which demonstrates the difference. (Yes, I
picked an example that I care about. I'm trying to ensure that
memcache data is "not too stale".)
def txn():
...
a.put()
memcache.set('a', a.field)
return a
If the CONCURRENT_TRANSACTION exception is raised while txn is being
run, specifically during a.put(), the memcache.set won't happen when
db.run_in_transaction(txn) fails. If that exception is raised after
txn has exited and during commit (as the SDK code suggests), the
memcache.set will happen whether or not db.run_in_transaction(txn)
fails.
If my understanding of the SDK code is correct and the real
implementation works the same way, namely that conflicts are detected
after the user function completes, how can I ensure that memcache data
is not too stale? (One way is to have that data expire reasonably
quickly, but that reduces the value of memcache.)
Also, what's the definition of "conflict"? Clearly there's a conflict
between a user function that reads a given data store entity and one
that writes the same entity. However, what about the following?
def txn1(a, b):
# notice - no read for a or b
a.put()
b.put()
return True
Does the conflict detection system detect the conflict between
transactions with txn1 for the same datastore entities? (The intent
of transactions with txn1 is to ensure that a and b are mutually
consistent in the datastore.)
Speaking of "definitions of conflict", suppose that conflicts actually
are detected/handled while the user function is being run, so that txn/
txnw can not leave the datastore and memcache inconsistent for very
long. Are txnw and txnr (below) seen as conflicting given the same
key? (They're not conflicting as far as the datastore is concerned,
but remember - I'm trying to keep memcache consistent as well.)
def txnw(key, new_value):
v = db.get(key)
v.field = new_value
db.put(v)
memcache.set(str(key), v.field)
return True
def txnr(key):
v = db.get(key)
memcache.set(str(key), v.field)
return True
Thanks,
-andy
On Oct 9, 4:45 am, "Nick Johnson (Google)" <nick.john...@google.com>
wrote:
> Hi Andy,
>
> On Tue, Oct 6, 2009 at 8:45 PM, Andy Freeman <ana...@earthlink.net> wrote:
>
> > Short version.
>
> > When, exactly, are apiproxy_errors.ApplicationErrors
> > with .application_error == datastore_pb.Error.CONCURRENT_TRANSACTION
> > raised.
>
> They are raised inside a transaction, when a conflict is detected with
> another concurrent transaction. The transaction infrastructure will catch
> and retry these several times, and only raise it in the external code if it
> was unable to execute the transaction after several retries.
>
> -Nick Johnson
>
>
>
> --
> Nick Johnson, Developer Programs Engineer, App Engine
> Google Ireland Ltd. :: Registered in Dublin, Ireland, Registration Number:
> 368047
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en
-~----------~----~----~----~------~----~------~--~---
