Hi Jeff, I have CME erratically - so I guess you are saying I should take a look at the look for the CME runs, and check whether the same EG's are accessed by other operations? It does seem that there might be a clash with other operations when the CME occurs. I am not sure about the timing though - there is a updateOrder operation which fails with CME at 14:59:46.684 and the send mail fails with CME at 15:01:06.583... When you say "Any change to that EG" it is the entire EG you mean, or only rows with the same id in the EG?
I need to ensure that my transaction does not fail, as it will result in duplicate emails being sent if it does. There are other operations working on the same EG which comes randomly as they are caused by a webhook in another application. What is the best way to ensure that these webhook operations do not cause the send mail operation to get a CME? I can only think of very hacked and ugly solutions - e.g. to have a lock/switch which is on when the send mail operation is running, on only executing webhook operations when the lock/switch is off. Thanks, -Louise Den tirsdag den 28. marts 2017 kl. 22.11.15 UTC+2 skrev Jeff Schnitzer: > > When you load a key in a datastore transaction, that EG is enlisted in the > transaction. Any change to that EG by any other process in the system will > cause your commit to rollback with CME. Even if your transaction is > "read-only”. > > When I said “linear” I mean that if you have a quiet datastore (no other > activity) and you start a transaction and you perform a series of > operations and you commit the transaction, you should not see CME. > > If you see CME erratically, then you have contention in your system. > If you see CME consistently 100% of the time and you aren’t under heavy > load, then you have a bug in your transaction logic. > > Jeff > > On Tue, Mar 28, 2017 at 6:29 AM, Louise Elmose Hedegaard < > louise...@gmail.com <javascript:>> wrote: > >> Hi, >> >> I stumbled upon an instance in the log where the sendNow list is empty - >> but still the operation fails with ConcurrentModificationException. >> How is that possible - it is only the initial read 1) operation which is >> executed on the datastore. >> >> Thanks, >> -Louise >> >> >> Den tirsdag den 28. marts 2017 kl. 08.58.35 UTC+2 skrev Louise Elmose >> Hedegaard: >>> >>> Hi Adam, >>> >>> Why do you mention timeout - ConcurrentModificationException is not >>> related to timeouts is it? >>> >>> Thanks, >>> -Louise >>> >>> Den lørdag den 25. marts 2017 kl. 00.02.05 UTC+1 skrev Adam (Cloud >>> Platform Support): >>>> >>>> A great article that explains this is 'Timeouts due to write contention >>>> <https://cloud.google.com/appengine/articles/handling_datastore_errors#timeouts-due-to-write-contention> >>>> ': >>>> >>>> *Writes to a single entity group are serialized by the App Engine >>>>> datastore, and thus there's a limit on how quickly you can update one >>>>> entity group. In general, this works out to somewhere between 1 and 5 >>>>> updates per second.* >>>> >>>> >>>> Writing to the same entity twice successively is enough to cause a >>>> ConcurrentModificationException. Strategies to avoid this are discussed in >>>> 'Avoiding >>>> datastore contention >>>> <https://cloud.google.com/appengine/articles/scaling/contention>'. >>>> >>>> On Friday, March 24, 2017 at 3:18:28 AM UTC-4, Louise Elmose Hedegaard >>>> wrote: >>>>> >>>>> Hi, >>>>> >>>>> In my app I have the following logic (pseudoCode): >>>>> >>>>> List<NonSentMails> nonSentMails= getNonSentMails();//1) datastore read >>>>> >>>>> List<NonSentMails> sendNow = new ArrayList<NonSentMails>(); >>>>> for each nonSentMail in nonSentMails >>>>> if (sendNow(nonSentMail)){ >>>>> sendNow.add(nonSentMail); >>>>> } >>>>> >>>>> for each nonSentMail in sendNow >>>>> SentStatus sentStatus = sendMailViaRestApi(); >>>>> create maillog in datastore with sent status//2) datastore write - >>>>> new entry >>>>> nonSentMail.setSentStatus(sentStatus); >>>>> >>>>> update(sendNow) //3) datastore write >>>>> >>>>> >>>>> When comitting the above I get: >>>>> java.util.ConcurrentModificationException: too much contention on >>>>> these datastore entities. please try again. >>>>> >>>>> This is a problem, as the operation is not idempotent - when I send a >>>>> mail via a REST API the mail is sent immediately, and I want the status >>>>> of >>>>> my maillog and NonSentMail objects to reflect this. >>>>> >>>>> I do not understand why this error occurs. >>>>> There are only three operations on the datastore: >>>>> 1) The initial read of non sent mails >>>>> 2) The creation of new mail log entries >>>>> 3) The update of the status of non sent mails >>>>> >>>>> 2) should never lead to ConcurrentModificationException as far as I >>>>> understand it. 1) and 3) operates on the same entities, but 1) only >>>>> reads, >>>>> so I cannot see any problem here either. >>>>> Furthermore there is currently only one user of my app, so there >>>>> should be no other threads accessing the data. >>>>> Note that the elements in sendNow might have the same parents, but >>>>> again I do not believe this should be a problem? >>>>> >>>>> Can you please help me understand why the above code can lead to a >>>>> ConcurrentModificationException? >>>>> >>>>> Thanks, >>>>> -Lull >>>>> >>>>> -- >> You received this message because you are subscribed to the Google Groups >> "Google App Engine" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to google-appengi...@googlegroups.com <javascript:>. >> To post to this group, send email to google-a...@googlegroups.com >> <javascript:>. >> Visit this group at https://groups.google.com/group/google-appengine. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/google-appengine/02ac6e5d-e70d-4da8-9f51-2ac68aefc3d7%40googlegroups.com >> >> <https://groups.google.com/d/msgid/google-appengine/02ac6e5d-e70d-4da8-9f51-2ac68aefc3d7%40googlegroups.com?utm_medium=email&utm_source=footer> >> . >> >> For more options, visit https://groups.google.com/d/optout. >> > > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at https://groups.google.com/group/google-appengine. To view this discussion on the web visit https://groups.google.com/d/msgid/google-appengine/ea889f39-59c1-4493-ab7b-f4db8d25b2a8%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
