Thanks again for eveybody's input. It looks like I have solved my problem by falling back on the low-level datastore access when strong consistency matters. Problems like this happen, and the bottom line is GAE is a great product that's done a lot for me. So kudos to all members, former and present, of the GAE team at Google.
FWIW, here are two observations that could be helpful in understanding the behavior of the gae/JDO function GetObjectById: 1) If you set the option "Unapplied job percentage" to 100% in the SDK, that is, you force "eventually consistent" to mean "never consistent," then GetObjectById still acts strongly consistent. This means that the SDK thinks that GetObjectById should act strongly consistent, just like the low level get function. 2) Demonstrating that GetObjectById can return stale data in the production environment is difficult because the behavior is, by its very nature, intermittent. But one can increase the probability of seeing stale data vastly if one performs the write and subsequent read from two different *versions* of the app. As others have observed, the probability already increases when write and read are performed from different instances of the app. But with different *versions *(one of them being the default version), it's been happening pretty much every time in my context. The low-level get, by contrast, has always acted strongly consistent for me, even across versions. Thomas On Tue, Aug 4, 2015 at 12:07 AM, Jeff Schnitzer <j...@infohazard.org> wrote: > Again, JDO is not my area of expertise, but if so, this seems like a > shockingly obvious issue. Can you post a sample of the code you use to > demonstrate the problem? > > Jeff > > On Mon, Aug 3, 2015 at 8:11 PM, Prashant <antsh...@gmail.com> wrote: > >> I am facings exact same issue. getObjectById returns stale data even if >> call is made inside a transaction. >> >> I performed an experiment - I ran two crons in two different modules, >> both trying to increment their execution count in an (common, shared) >> entity. I was shocked to see that even when I transaction, both crons are >> running as if they are performing transaction in thier own copy of >> DataStore. Evertime, each cron will get copy of entity last updated by >> itself (ignoring updates made by other cron) and on top of that transaction >> will also commit without any exception. >> >> -- >> Prashant >> >> On 8:24AM, Tue, 4 Aug 2015 Thomas Becker <walkswiththeb...@gmail.com> >> wrote: >> >>> Thanks, Jeff! And one more thing, it's pretty clear from the doc, but >>> let me make sure anyway. The same is true for ancestor queries, they >>> guarantee strong consistency even when not performed inside a transaction, >>> right? >>> >>> Thomas >>> >>> On Mon, Aug 3, 2015 at 8:48 PM, Jeff Schnitzer <j...@infohazard.org> >>> wrote: >>> >>>> Yes, that is correct. By default, if you perform a low-level get-by-key >>>> operation, it is strongly consistent whether or not you put it in a >>>> transaction. >>>> >>>> Note that you can, if you want, explicitly request an eventually >>>> consistent get-by-key operation in the low-level api. It does return faster >>>> since it does not require a quorum response. However, this is a pretty >>>> obscure feature. >>>> >>>> Jeff >>>> >>>> On Mon, Aug 3, 2015 at 7:30 PM, Thomas Becker < >>>> walkswiththeb...@gmail.com> wrote: >>>> >>>>> Thanks for your detailed and thoughtful answer, Jeff. I was aware of >>>>> the fact that within a transaction, all reads will return the original >>>>> value even if one writes it within the transaction. That was not the >>>>> problem, but thanks for pointing it out anyway, it's easy to forget! >>>>> >>>>> Other than that, after reading everything on the subject of GAE >>>>> datastore consistency under JDO, I have decided to fall back to low-level >>>>> access in those cases where strong consistency is of the essence. At the >>>>> risk of sounding acerbic or snarky, it seems clear to me that nobody knows >>>>> what's going on in JDO when it comes to consistency. >>>>> >>>>> So just to confirm, it is true that the low-level get-by-key >>>>> guarantees strong consistency even *when performed outside of a >>>>> transaction*, correct? >>>>> >>>>> Thanks again for your thoughtful answer. Very refreshing indeed. >>>>> >>>>> Thomas >>>>> >>>>> On Mon, Aug 3, 2015 at 8:13 PM, Jeff Schnitzer <j...@infohazard.org> >>>>> wrote: >>>>> >>>>>> I'm no expert on JDO, but the low-level api does guarantee strong >>>>>> consistency for get-by-key operations (that is, unless you explicitly ask >>>>>> for eventual consistency). One of those SO responses suggested it is >>>>>> possible to have the GAE/JDO plugin log its low-level operations - that >>>>>> should at least point you in the right direction. >>>>>> >>>>>> One other thing that might be related: In the low-level API, within a >>>>>> single transaction, all reads will return the original value even if you >>>>>> write it within the transaction. I don't know what JDO does with this; I >>>>>> would expect that subsequent reads would return a session-cached instance >>>>>> and hide this little quirk. But maybe it's related to what you are >>>>>> seeing? >>>>>> >>>>>> Jeff >>>>>> >>>>>> On Mon, Aug 3, 2015 at 1:52 PM, Thomas Becker < >>>>>> walkswiththeb...@gmail.com> wrote: >>>>>> >>>>>>> Thanks for your quick reply, Ryan. I have found several postings on >>>>>>> the Web that describe the same problem, namely, GetObjectById returning >>>>>>> stale data. Here are two from StackOverflow: >>>>>>> >>>>>>> >>>>>>> http://stackoverflow.com/questions/26377347/inconsistent-fetch-from-google-app-engine-datastore >>>>>>> >>>>>>> >>>>>>> http://stackoverflow.com/questions/22005349/appengine-jdo-with-multiple-server-instances-loses-transaction-updates-on-entiti >>>>>>> >>>>>>> Under the circumstances, I don't think it would be a good use of my >>>>>>> time to make a demo app. It seems clear to me that the issue exists. >>>>>>> Besides, the problem is so intermittent that a demo app is of very >>>>>>> limited >>>>>>> value. I would be perfectly happy with a statement of the kind, >>>>>>> "GetObjectById() does not guarantee strong consistency, that's how it >>>>>>> is, >>>>>>> and here's what you do instead to get strongly consistent data." I just >>>>>>> need to know what's going on. >>>>>>> >>>>>>> On Mon, Aug 3, 2015 at 2:32 PM, Ryan (Cloud Platform Support) < >>>>>>> rbruy...@google.com> wrote: >>>>>>> >>>>>>>> If you can create a sample app that shows this post on it on the >>>>>>>> PIT <https://code.google.com/p/googleappengine/issues/entry> so I >>>>>>>> can investigate this further. >>>>>>>> >>>>>>>> On Saturday, August 1, 2015 at 6:24:55 PM UTC-4, Thomas Becker >>>>>>>> wrote: >>>>>>>>> >>>>>>>>> I am using the app engine datastore with Java JDO. According to >>>>>>>>> the documentation, ancestor queries are always strongly consistent. >>>>>>>>> Recently, however, I've been seeing a lot of stale data from ancestor >>>>>>>>> queries. It seems to happen randomly. A query may return the most >>>>>>>>> recent >>>>>>>>> data, and a moment later, the same query returns stale data. >>>>>>>>> >>>>>>>> -- >>>>>>>> You received this message because you are subscribed to a topic in >>>>>>>> the Google Groups "Google App Engine" group. >>>>>>>> To unsubscribe from this topic, visit >>>>>>>> https://groups.google.com/d/topic/google-appengine/1re1FtqllnQ/unsubscribe >>>>>>>> . >>>>>>>> To unsubscribe from this group and all its topics, send an email to >>>>>>>> google-appengine+unsubscr...@googlegroups.com. >>>>>>>> To post to this group, send email to >>>>>>>> google-appengine@googlegroups.com. >>>>>>>> Visit this group at http://groups.google.com/group/google-appengine >>>>>>>> . >>>>>>>> To view this discussion on the web visit >>>>>>>> https://groups.google.com/d/msgid/google-appengine/9723734e-0b14-49c0-8c31-3407df8454ad%40googlegroups.com >>>>>>>> <https://groups.google.com/d/msgid/google-appengine/9723734e-0b14-49c0-8c31-3407df8454ad%40googlegroups.com?utm_medium=email&utm_source=footer> >>>>>>>> . >>>>>>>> For more options, visit https://groups.google.com/d/optout. >>>>>>>> >>>>>>> >>>>>>> -- >>>>>>> You received this message because you are subscribed to the Google >>>>>>> Groups "Google App Engine" group. >>>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>>> send an email to google-appengine+unsubscr...@googlegroups.com. >>>>>>> To post to this group, send email to >>>>>>> google-appengine@googlegroups.com. >>>>>>> Visit this group at http://groups.google.com/group/google-appengine. >>>>>>> To view this discussion on the web visit >>>>>>> https://groups.google.com/d/msgid/google-appengine/CAPxj-GA3OKiGfjwgYgKmvZ%3DSi7ZXG3VLvts7%2BjWZ7c2CsC693Q%40mail.gmail.com >>>>>>> <https://groups.google.com/d/msgid/google-appengine/CAPxj-GA3OKiGfjwgYgKmvZ%3DSi7ZXG3VLvts7%2BjWZ7c2CsC693Q%40mail.gmail.com?utm_medium=email&utm_source=footer> >>>>>>> . >>>>>>> >>>>>>> For more options, visit https://groups.google.com/d/optout. >>>>>>> >>>>>> >>>>>> -- >>>>>> You received this message because you are subscribed to a topic in >>>>>> the Google Groups "Google App Engine" group. >>>>>> To unsubscribe from this topic, visit >>>>>> https://groups.google.com/d/topic/google-appengine/1re1FtqllnQ/unsubscribe >>>>>> . >>>>>> To unsubscribe from this group and all its topics, send an email to >>>>>> google-appengine+unsubscr...@googlegroups.com. >>>>>> To post to this group, send email to >>>>>> google-appengine@googlegroups.com. >>>>>> Visit this group at http://groups.google.com/group/google-appengine. >>>>>> To view this discussion on the web visit >>>>>> https://groups.google.com/d/msgid/google-appengine/CADK-0uh4Q8kswYMhD5RcxeQJnO3rFshgkfGEbB%2BXzHVXQLbGiw%40mail.gmail.com >>>>>> <https://groups.google.com/d/msgid/google-appengine/CADK-0uh4Q8kswYMhD5RcxeQJnO3rFshgkfGEbB%2BXzHVXQLbGiw%40mail.gmail.com?utm_medium=email&utm_source=footer> >>>>>> . >>>>>> For more options, visit https://groups.google.com/d/optout. >>>>>> >>>>> >>>>> -- >>>>> You received this message because you are subscribed to the Google >>>>> Groups "Google App Engine" group. >>>>> To unsubscribe from this group and stop receiving emails from it, send >>>>> an email to google-appengine+unsubscr...@googlegroups.com. >>>>> To post to this group, send email to google-appengine@googlegroups.com >>>>> . >>>>> Visit this group at http://groups.google.com/group/google-appengine. >>>>> To view this discussion on the web visit >>>>> https://groups.google.com/d/msgid/google-appengine/CAPxj-GApGFORBdvJd88QvitX_bHLGvAtiQ943WBd0V3Ro9onZw%40mail.gmail.com >>>>> <https://groups.google.com/d/msgid/google-appengine/CAPxj-GApGFORBdvJd88QvitX_bHLGvAtiQ943WBd0V3Ro9onZw%40mail.gmail.com?utm_medium=email&utm_source=footer> >>>>> . >>>>> >>>>> For more options, visit https://groups.google.com/d/optout. >>>>> >>>> >>>> -- >>>> You received this message because you are subscribed to a topic in the >>>> Google Groups "Google App Engine" group. >>>> To unsubscribe from this topic, visit >>>> https://groups.google.com/d/topic/google-appengine/1re1FtqllnQ/unsubscribe >>>> . >>>> To unsubscribe from this group and all its topics, send an email to >>>> google-appengine+unsubscr...@googlegroups.com. >>>> To post to this group, send email to google-appengine@googlegroups.com. >>>> Visit this group at http://groups.google.com/group/google-appengine. >>>> >>> To view this discussion on the web visit >>>> https://groups.google.com/d/msgid/google-appengine/CADK-0ujMaP_xRg6Pjht%3DsCR%3Dc-3hGEJvhXGtOT0h1vcMKhQBqQ%40mail.gmail.com >>>> <https://groups.google.com/d/msgid/google-appengine/CADK-0ujMaP_xRg6Pjht%3DsCR%3Dc-3hGEJvhXGtOT0h1vcMKhQBqQ%40mail.gmail.com?utm_medium=email&utm_source=footer> >>>> . >>>> >>>> For more options, visit https://groups.google.com/d/optout. >>>> >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "Google App Engine" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to google-appengine+unsubscr...@googlegroups.com. >>> To post to this group, send email to google-appengine@googlegroups.com. >>> Visit this group at http://groups.google.com/group/google-appengine. >>> To view this discussion on the web visit >>> https://groups.google.com/d/msgid/google-appengine/CAPxj-GBwjD%2BGO9LDk5ObgTQo7w5%2B7wnN32%3D664CrVg%2BPxD_6nw%40mail.gmail.com >>> <https://groups.google.com/d/msgid/google-appengine/CAPxj-GBwjD%2BGO9LDk5ObgTQo7w5%2B7wnN32%3D664CrVg%2BPxD_6nw%40mail.gmail.com?utm_medium=email&utm_source=footer> >>> . >>> For more options, visit https://groups.google.com/d/optout. >>> >> -- >> You received this message because you are subscribed to the Google Groups >> "Google App Engine" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to google-appengine+unsubscr...@googlegroups.com. >> To post to this group, send email to google-appengine@googlegroups.com. >> Visit this group at http://groups.google.com/group/google-appengine. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/google-appengine/CAPpOf-t%3Dzp2uRUnkN9446_QoSXN6XCKdzNAe1g4GwULN8gg93A%40mail.gmail.com >> <https://groups.google.com/d/msgid/google-appengine/CAPpOf-t%3Dzp2uRUnkN9446_QoSXN6XCKdzNAe1g4GwULN8gg93A%40mail.gmail.com?utm_medium=email&utm_source=footer> >> . >> >> For more options, visit https://groups.google.com/d/optout. >> > > -- > You received this message because you are subscribed to a topic in the > Google Groups "Google App Engine" group. > To unsubscribe from this topic, visit > https://groups.google.com/d/topic/google-appengine/1re1FtqllnQ/unsubscribe > . > To unsubscribe from this group and all its topics, send an email to > google-appengine+unsubscr...@googlegroups.com. > To post to this group, send email to google-appengine@googlegroups.com. > Visit this group at http://groups.google.com/group/google-appengine. > To view this discussion on the web visit > https://groups.google.com/d/msgid/google-appengine/CADK-0uhSRuiH6a3Wn%3DAYYm2DF5GjvQnT1FCVAnjBkga6MF6pew%40mail.gmail.com > <https://groups.google.com/d/msgid/google-appengine/CADK-0uhSRuiH6a3Wn%3DAYYm2DF5GjvQnT1FCVAnjBkga6MF6pew%40mail.gmail.com?utm_medium=email&utm_source=footer> > . > > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine. To view this discussion on the web visit https://groups.google.com/d/msgid/google-appengine/CAPxj-GBWY%2BQXoN%2BS3sX4YbO%2BuhZjd8svpUMnjwzTahzOFd-%3Dyw%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
