Maybe this clarifies my question. The paragraph at bottom is a description of a transaction from Google. It refers to checking the 'last update time' of an entity group and then checking whether 'it has changed since our initial check'.
While I don't expect this short paragraph to describe all of the complexities of App Engine transactions it does lay out a simplistic implementation which does not cover the 'create if it doesn't exist' transaction given on the same page. I have read a number of other descriptions of App Engine transactions and none of them indicate that this case is covered. My expectation is that it is, in fact, covered and these transactions work. But I do need to be sure. *"When a transaction starts, App Engine uses **optimistic concurrency control <http://en.wikipedia.org/wiki/Optimistic_concurrency_control> by checking the last update time for the entity groups used in the transaction. Upon commiting a transaction for the entity groups, App Engine again checks the last update time for the entity groups used in the transaction. If it has changed since our initial check, an error is returned."* https://cloud.google.com/appengine/docs/go/datastore/transactions Francis On 15 May 2015 at 22:45, Francis Stephens <f...@belua.com> wrote: > I appreciate that the question is difficult, and I'm sure it could be > worded differently. However, the reason I have phrased it as I did was > because I really need to clarify this one case. It's very important to me. > > Perhaps it would be clearer if I try to address your description of an app > engine transaction. > > *"When you start a transaction, each entity group (defined by the root > key, whether or not an entity exists at that key) touched is enlisted in > the transaction."* > > In the case I described the transaction is entered simultaneously by two > instances. > > In this scenario both Get(...) calls returns an ErrNoSuchEntity we know > (or I believe) at that point that the key does not exist anywhere in the > datastore, attached to an entity or not. So I would like to know how the > entity group is enlisted in this case. > > *"When you commit a transaction, if any of those entity groups have been > changed by anyone anywhere, your transaction rolls back"* > > When the two Put(...) calls are executed one of them must fail with an > ErrConcurrentTransaction (or ConcurrentModificationException in Java). So > the question I have is, where the information which would allow for one of > the Put(...) calls to fail? > > I have two reasons for asking for clarification on this point. > > 1: Data races are often very subtle, and I would like to clarify my > understanding in this instance. > 2: In the case given above a low probability data-race where the first > calls might be over-written would be acceptable for most counters. That > would be a very small counting loss and the chance of such a race, if it is > possible at all, would have negligible impact. We don't have the same > luxury, so I want be absolutely sure. > > Thanks for your response. > > Francis > > On 15 May 2015 at 01:34, Jeff Schnitzer <j...@infohazard.org> wrote: > >> I'm honestly having a hard time understanding your questions. >> >> GAE transactions are pretty simple. When you start a transaction, each >> entity group (defined by the root key, whether or not an entity exists at >> that key) touched is enlisted in the transaction. When you commit a >> transaction, if any of those entity groups have been changed by anyone >> anywhere, your transaction rolls back (in Java you get >> ConcurrentModificationException). >> >> Your questions would be easier to understand without bringing up >> implementation details like timestamps. The implementation is not >> particularly relevant to the behavior. >> >> Jeff >> >> >> On Thu, May 14, 2015 at 5:49 AM, Francis Stephens <f...@belua.com> wrote: >> >>> It's unclear to me where I should be posting this question. I've posted >>> it to stack overflow in case that is more appropriate, however the nature >>> of the question goes against the SO guidelines. >>> >>> >>> http://stackoverflow.com/questions/30233093/app-engine-mechanics-of-creating-a-unique-entity-if-it-doesnt-exist >>> >>> >>> On Wednesday, 13 May 2015 13:59:57 UTC+1, Francis Stephens wrote: >>>> >>>> We have an issue where we want to lazily create an entity if it does >>>> not exist. There is some discussion going on about how to do this and I >>>> would like to clarify some things around app engine transactions. I will >>>> limit my query to single entity group transactions. >>>> >>>> I am using Go in my examples, but I hope the code is clear enough for >>>> non-Go programmers. >>>> >>>> My understanding is that a transaction, on a single entity group, will >>>> succeed only if the entity group is not modified externally during the >>>> transaction. The 'entity group timestamp' indicating when an entity group >>>> was changed is stored in the root entity of the entity group. So during a >>>> transaction the current 'entity group timestamp' is read and the >>>> transaction can only succeed if it hasn't changed by the end of the >>>> transaction. >>>> >>>> key := datastore.NewKey(c, "Counter", "mycounter", 0, nil) >>>> count := new(Counter) >>>> err := datastore.RunInTransaction(c, func(c appengine.Context) error { >>>> err := datastore.Get(c, key, count) >>>> if err != nil && err != datastore.ErrNoSuchEntity { >>>> return err >>>> } >>>> count.Count++ >>>> _, err = datastore.Put(c, key, count) >>>> return err >>>> }, nil) >>>> >>>> >>>> In the example above (taken from >>>> https://cloud.google.com/appengine/docs/go/datastore/transactions) >>>> there are two non-error cases, I can see. >>>> >>>> 1: The Get succeeds and the 'entity group timestamp' on the counter can >>>> be used to ensure no other transactions update the counter during this >>>> transaction. >>>> 2: The Get fails with ErrNoSuchEntity and the Put is used to store the >>>> counter for the first time. >>>> >>>> In the second case it is possible that another identical transaction is >>>> running. If both transactions' Get return ErrNoSuchEntity how does the >>>> datastore ensure that only one put succeeds? I would expect there to be no >>>> 'entity group timestamp' in the datastore to test against? >>>> >>>> Does the transaction know that it needs to test for the non-existence >>>> of the counter in order for the Put and the entire transaction to succeed? >>>> >>>> Is there a chance in this case for two transactions to succeed and for >>>> one Put to overwrite the other? >>>> >>>> If there is documentation, or videos etc, around the mechanism that >>>> controls this I would love to read it. >>>> >>>> Thanks in advance. >>>> >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "Google App Engine" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to google-appengine+unsubscr...@googlegroups.com. >>> To post to this group, send email to google-appengine@googlegroups.com. >>> Visit this group at http://groups.google.com/group/google-appengine. >>> To view this discussion on the web visit >>> https://groups.google.com/d/msgid/google-appengine/39fce7cf-fa5a-464f-821e-0826f8fdb500%40googlegroups.com >>> <https://groups.google.com/d/msgid/google-appengine/39fce7cf-fa5a-464f-821e-0826f8fdb500%40googlegroups.com?utm_medium=email&utm_source=footer> >>> . >>> >>> For more options, visit https://groups.google.com/d/optout. >>> >> >> -- >> You received this message because you are subscribed to a topic in the >> Google Groups "Google App Engine" group. >> To unsubscribe from this topic, visit >> https://groups.google.com/d/topic/google-appengine/NagJ97YExB0/unsubscribe >> . >> To unsubscribe from this group and all its topics, send an email to >> google-appengine+unsubscr...@googlegroups.com. >> To post to this group, send email to google-appengine@googlegroups.com. >> Visit this group at http://groups.google.com/group/google-appengine. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/google-appengine/CADK-0uhGRUYTb-4ehr3C9Got1oe9bFfiOHLUj3qmWCoebcYB5w%40mail.gmail.com >> <https://groups.google.com/d/msgid/google-appengine/CADK-0uhGRUYTb-4ehr3C9Got1oe9bFfiOHLUj3qmWCoebcYB5w%40mail.gmail.com?utm_medium=email&utm_source=footer> >> . >> >> For more options, visit https://groups.google.com/d/optout. >> > > > > -- > Francis Stephens > Software Developer @ Belua > -- Francis Stephens Software Developer @ Belua -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine. To view this discussion on the web visit https://groups.google.com/d/msgid/google-appengine/CAG1f2XCoSfD9f%2BHpnbNL8j6rQ7AD1tmw3AJ7beL%3DviwUqo8xBA%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
