Re: [google-appengine] Re: Stale Data from App Engine Datastore
This sounds to me not like an issue with eventual consistency, but an issue with session caching. The thing about different *versions* is that it guarantees requests to go to different instances. I don't remember much about how JDO works, but are you sure you are opening/closing sessions correctly? Jeff On Tue, Aug 4, 2015 at 7:56 PM, Thomas Becker walkswiththeb...@gmail.com wrote: Thanks again for eveybody's input. It looks like I have solved my problem by falling back on the low-level datastore access when strong consistency matters. Problems like this happen, and the bottom line is GAE is a great product that's done a lot for me. So kudos to all members, former and present, of the GAE team at Google. FWIW, here are two observations that could be helpful in understanding the behavior of the gae/JDO function GetObjectById: 1) If you set the option Unapplied job percentage to 100% in the SDK, that is, you force eventually consistent to mean never consistent, then GetObjectById still acts strongly consistent. This means that the SDK thinks that GetObjectById should act strongly consistent, just like the low level get function. 2) Demonstrating that GetObjectById can return stale data in the production environment is difficult because the behavior is, by its very nature, intermittent. But one can increase the probability of seeing stale data vastly if one performs the write and subsequent read from two different *versions* of the app. As others have observed, the probability already increases when write and read are performed from different instances of the app. But with different *versions *(one of them being the default version), it's been happening pretty much every time in my context. The low-level get, by contrast, has always acted strongly consistent for me, even across versions. Thomas On Tue, Aug 4, 2015 at 12:07 AM, Jeff Schnitzer j...@infohazard.org wrote: Again, JDO is not my area of expertise, but if so, this seems like a shockingly obvious issue. Can you post a sample of the code you use to demonstrate the problem? Jeff On Mon, Aug 3, 2015 at 8:11 PM, Prashant antsh...@gmail.com wrote: I am facings exact same issue. getObjectById returns stale data even if call is made inside a transaction. I performed an experiment - I ran two crons in two different modules, both trying to increment their execution count in an (common, shared) entity. I was shocked to see that even when I transaction, both crons are running as if they are performing transaction in thier own copy of DataStore. Evertime, each cron will get copy of entity last updated by itself (ignoring updates made by other cron) and on top of that transaction will also commit without any exception. -- Prashant On 8:24AM, Tue, 4 Aug 2015 Thomas Becker walkswiththeb...@gmail.com wrote: Thanks, Jeff! And one more thing, it's pretty clear from the doc, but let me make sure anyway. The same is true for ancestor queries, they guarantee strong consistency even when not performed inside a transaction, right? Thomas On Mon, Aug 3, 2015 at 8:48 PM, Jeff Schnitzer j...@infohazard.org wrote: Yes, that is correct. By default, if you perform a low-level get-by-key operation, it is strongly consistent whether or not you put it in a transaction. Note that you can, if you want, explicitly request an eventually consistent get-by-key operation in the low-level api. It does return faster since it does not require a quorum response. However, this is a pretty obscure feature. Jeff On Mon, Aug 3, 2015 at 7:30 PM, Thomas Becker walkswiththeb...@gmail.com wrote: Thanks for your detailed and thoughtful answer, Jeff. I was aware of the fact that within a transaction, all reads will return the original value even if one writes it within the transaction. That was not the problem, but thanks for pointing it out anyway, it's easy to forget! Other than that, after reading everything on the subject of GAE datastore consistency under JDO, I have decided to fall back to low-level access in those cases where strong consistency is of the essence. At the risk of sounding acerbic or snarky, it seems clear to me that nobody knows what's going on in JDO when it comes to consistency. So just to confirm, it is true that the low-level get-by-key guarantees strong consistency even *when performed outside of a transaction*, correct? Thanks again for your thoughtful answer. Very refreshing indeed. Thomas On Mon, Aug 3, 2015 at 8:13 PM, Jeff Schnitzer j...@infohazard.org wrote: I'm no expert on JDO, but the low-level api does guarantee strong consistency for get-by-key operations (that is, unless you explicitly ask for eventual consistency). One of those SO responses suggested it is possible to have the GAE/JDO plugin log its low-level operations - that should at least point you in the right direction. One other thing that might be related: In the low-level API, within
Re: [google-appengine] Re: Stale Data from App Engine Datastore
I believe that session caching is specific to the Objectify interface. Since I'm not using that, I don't think there even is a way for me to open/close any sessions. I'm quite sure that there is no such a thing as a JDO session. (There is of course the HTTP session that's provided by the Java servlet environment, but that's got nothing to do with the datastore.) Thomas On Wed, Aug 5, 2015 at 12:44 PM, Jeff Schnitzer j...@infohazard.org wrote: This sounds to me not like an issue with eventual consistency, but an issue with session caching. The thing about different *versions* is that it guarantees requests to go to different instances. I don't remember much about how JDO works, but are you sure you are opening/closing sessions correctly? Jeff On Tue, Aug 4, 2015 at 7:56 PM, Thomas Becker walkswiththeb...@gmail.com wrote: Thanks again for eveybody's input. It looks like I have solved my problem by falling back on the low-level datastore access when strong consistency matters. Problems like this happen, and the bottom line is GAE is a great product that's done a lot for me. So kudos to all members, former and present, of the GAE team at Google. FWIW, here are two observations that could be helpful in understanding the behavior of the gae/JDO function GetObjectById: 1) If you set the option Unapplied job percentage to 100% in the SDK, that is, you force eventually consistent to mean never consistent, then GetObjectById still acts strongly consistent. This means that the SDK thinks that GetObjectById should act strongly consistent, just like the low level get function. 2) Demonstrating that GetObjectById can return stale data in the production environment is difficult because the behavior is, by its very nature, intermittent. But one can increase the probability of seeing stale data vastly if one performs the write and subsequent read from two different *versions* of the app. As others have observed, the probability already increases when write and read are performed from different instances of the app. But with different *versions *(one of them being the default version), it's been happening pretty much every time in my context. The low-level get, by contrast, has always acted strongly consistent for me, even across versions. Thomas On Tue, Aug 4, 2015 at 12:07 AM, Jeff Schnitzer j...@infohazard.org wrote: Again, JDO is not my area of expertise, but if so, this seems like a shockingly obvious issue. Can you post a sample of the code you use to demonstrate the problem? Jeff On Mon, Aug 3, 2015 at 8:11 PM, Prashant antsh...@gmail.com wrote: I am facings exact same issue. getObjectById returns stale data even if call is made inside a transaction. I performed an experiment - I ran two crons in two different modules, both trying to increment their execution count in an (common, shared) entity. I was shocked to see that even when I transaction, both crons are running as if they are performing transaction in thier own copy of DataStore. Evertime, each cron will get copy of entity last updated by itself (ignoring updates made by other cron) and on top of that transaction will also commit without any exception. -- Prashant On 8:24AM, Tue, 4 Aug 2015 Thomas Becker walkswiththeb...@gmail.com wrote: Thanks, Jeff! And one more thing, it's pretty clear from the doc, but let me make sure anyway. The same is true for ancestor queries, they guarantee strong consistency even when not performed inside a transaction, right? Thomas On Mon, Aug 3, 2015 at 8:48 PM, Jeff Schnitzer j...@infohazard.org wrote: Yes, that is correct. By default, if you perform a low-level get-by-key operation, it is strongly consistent whether or not you put it in a transaction. Note that you can, if you want, explicitly request an eventually consistent get-by-key operation in the low-level api. It does return faster since it does not require a quorum response. However, this is a pretty obscure feature. Jeff On Mon, Aug 3, 2015 at 7:30 PM, Thomas Becker walkswiththeb...@gmail.com wrote: Thanks for your detailed and thoughtful answer, Jeff. I was aware of the fact that within a transaction, all reads will return the original value even if one writes it within the transaction. That was not the problem, but thanks for pointing it out anyway, it's easy to forget! Other than that, after reading everything on the subject of GAE datastore consistency under JDO, I have decided to fall back to low-level access in those cases where strong consistency is of the essence. At the risk of sounding acerbic or snarky, it seems clear to me that nobody knows what's going on in JDO when it comes to consistency. So just to confirm, it is true that the low-level get-by-key guarantees strong consistency even *when performed outside of a transaction*, correct? Thanks again for your thoughtful answer. Very refreshing indeed. Thomas On Mon, Aug 3, 2015 at
[google-appengine] Re: Stale Data from App Engine Datastore
Check out this link https://cloud.google.com/appengine/docs/java/datastore/jdo/creatinggettinganddeletingdata#Updating_an_Object specifically: Changes are persisted when the PersistenceManager is closed I ran some test to confirm and not closing resulted in eventual while closing resulted in strong consistency. Give it a try and let me know if that helps. On Saturday, August 1, 2015 at 6:24:55 PM UTC-4, Thomas Becker wrote: I am using the app engine datastore with Java JDO. According to the documentation, ancestor queries are always strongly consistent. Recently, however, I've been seeing a lot of stale data from ancestor queries. It seems to happen randomly. A query may return the most recent data, and a moment later, the same query returns stale data. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine. To view this discussion on the web visit https://groups.google.com/d/msgid/google-appengine/96de2818-896a-4877-aaa0-d496cc81dc06%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [google-appengine] Re: Stale Data from App Engine Datastore
JDO absolutely does have a session. You start a session by opening a persistencemanager and end by closing it. When you close the session, JDO does its dirty change detection and writes out any alterations. Also, repeated loads from an open session will return the same object. JPA works the same way. Objectify's session is very similar to this (minus the dirty change detection). If you aren't closing your PM sessions... this would explain the problem. Jeff On Wed, Aug 5, 2015 at 11:59 AM, Thomas Becker walkswiththeb...@gmail.com wrote: I believe that session caching is specific to the Objectify interface. Since I'm not using that, I don't think there even is a way for me to open/close any sessions. I'm quite sure that there is no such a thing as a JDO session. (There is of course the HTTP session that's provided by the Java servlet environment, but that's got nothing to do with the datastore.) Thomas On Wed, Aug 5, 2015 at 12:44 PM, Jeff Schnitzer j...@infohazard.org wrote: This sounds to me not like an issue with eventual consistency, but an issue with session caching. The thing about different *versions* is that it guarantees requests to go to different instances. I don't remember much about how JDO works, but are you sure you are opening/closing sessions correctly? Jeff On Tue, Aug 4, 2015 at 7:56 PM, Thomas Becker walkswiththeb...@gmail.com wrote: Thanks again for eveybody's input. It looks like I have solved my problem by falling back on the low-level datastore access when strong consistency matters. Problems like this happen, and the bottom line is GAE is a great product that's done a lot for me. So kudos to all members, former and present, of the GAE team at Google. FWIW, here are two observations that could be helpful in understanding the behavior of the gae/JDO function GetObjectById: 1) If you set the option Unapplied job percentage to 100% in the SDK, that is, you force eventually consistent to mean never consistent, then GetObjectById still acts strongly consistent. This means that the SDK thinks that GetObjectById should act strongly consistent, just like the low level get function. 2) Demonstrating that GetObjectById can return stale data in the production environment is difficult because the behavior is, by its very nature, intermittent. But one can increase the probability of seeing stale data vastly if one performs the write and subsequent read from two different *versions* of the app. As others have observed, the probability already increases when write and read are performed from different instances of the app. But with different *versions *(one of them being the default version), it's been happening pretty much every time in my context. The low-level get, by contrast, has always acted strongly consistent for me, even across versions. Thomas On Tue, Aug 4, 2015 at 12:07 AM, Jeff Schnitzer j...@infohazard.org wrote: Again, JDO is not my area of expertise, but if so, this seems like a shockingly obvious issue. Can you post a sample of the code you use to demonstrate the problem? Jeff On Mon, Aug 3, 2015 at 8:11 PM, Prashant antsh...@gmail.com wrote: I am facings exact same issue. getObjectById returns stale data even if call is made inside a transaction. I performed an experiment - I ran two crons in two different modules, both trying to increment their execution count in an (common, shared) entity. I was shocked to see that even when I transaction, both crons are running as if they are performing transaction in thier own copy of DataStore. Evertime, each cron will get copy of entity last updated by itself (ignoring updates made by other cron) and on top of that transaction will also commit without any exception. -- Prashant On 8:24AM, Tue, 4 Aug 2015 Thomas Becker walkswiththeb...@gmail.com wrote: Thanks, Jeff! And one more thing, it's pretty clear from the doc, but let me make sure anyway. The same is true for ancestor queries, they guarantee strong consistency even when not performed inside a transaction, right? Thomas On Mon, Aug 3, 2015 at 8:48 PM, Jeff Schnitzer j...@infohazard.org wrote: Yes, that is correct. By default, if you perform a low-level get-by-key operation, it is strongly consistent whether or not you put it in a transaction. Note that you can, if you want, explicitly request an eventually consistent get-by-key operation in the low-level api. It does return faster since it does not require a quorum response. However, this is a pretty obscure feature. Jeff On Mon, Aug 3, 2015 at 7:30 PM, Thomas Becker walkswiththeb...@gmail.com wrote: Thanks for your detailed and thoughtful answer, Jeff. I was aware of the fact that within a transaction, all reads will return the original value even if one writes it within the transaction. That was not the problem, but thanks for pointing it out anyway, it's easy to forget! Other than that, after reading
Re: [google-appengine] Re: Stale Data from App Engine Datastore
Oh, yeah, that's right, there is the closing of the persistence manager.
All my JDO calls go through one function that has all the boilerplate, and
it ends with
} finally {
pm.close();
}
}
So we can safely assume that this is not the problem.
On Wed, Aug 5, 2015 at 1:36 PM, Jeff Schnitzer j...@infohazard.org wrote:
JDO absolutely does have a session. You start a session by opening a
persistencemanager and end by closing it. When you close the session, JDO
does its dirty change detection and writes out any alterations. Also,
repeated loads from an open session will return the same object. JPA works
the same way.
Objectify's session is very similar to this (minus the dirty change
detection).
If you aren't closing your PM sessions... this would explain the problem.
Jeff
On Wed, Aug 5, 2015 at 11:59 AM, Thomas Becker walkswiththeb...@gmail.com
wrote:
I believe that session caching is specific to the Objectify interface.
Since I'm not using that, I don't think there even is a way for me to
open/close any sessions. I'm quite sure that there is no such a thing as a
JDO session. (There is of course the HTTP session that's provided by the
Java servlet environment, but that's got nothing to do with the datastore.)
Thomas
On Wed, Aug 5, 2015 at 12:44 PM, Jeff Schnitzer j...@infohazard.org
wrote:
This sounds to me not like an issue with eventual consistency, but an
issue with session caching. The thing about different *versions* is
that it guarantees requests to go to different instances.
I don't remember much about how JDO works, but are you sure you are
opening/closing sessions correctly?
Jeff
On Tue, Aug 4, 2015 at 7:56 PM, Thomas Becker
walkswiththeb...@gmail.com wrote:
Thanks again for eveybody's input. It looks like I have solved my
problem by falling back on the low-level datastore access when strong
consistency matters. Problems like this happen, and the bottom line is GAE
is a great product that's done a lot for me. So kudos to all members,
former and present, of the GAE team at Google.
FWIW, here are two observations that could be helpful in understanding
the behavior of the gae/JDO function GetObjectById:
1) If you set the option Unapplied job percentage to 100% in the SDK,
that is, you force eventually consistent to mean never consistent, then
GetObjectById still acts strongly consistent. This means that the SDK
thinks that GetObjectById should act strongly consistent, just like the low
level get function.
2) Demonstrating that GetObjectById can return stale data in the
production environment is difficult because the behavior is, by its very
nature, intermittent. But one can increase the probability of seeing stale
data vastly if one performs the write and subsequent read from two
different *versions* of the app. As others have observed, the
probability already increases when write and read are performed from
different instances of the app. But with different *versions *(one of
them being the default version), it's been happening pretty much every time
in my context. The low-level get, by contrast, has always acted strongly
consistent for me, even across versions.
Thomas
On Tue, Aug 4, 2015 at 12:07 AM, Jeff Schnitzer j...@infohazard.org
wrote:
Again, JDO is not my area of expertise, but if so, this seems like a
shockingly obvious issue. Can you post a sample of the code you use to
demonstrate the problem?
Jeff
On Mon, Aug 3, 2015 at 8:11 PM, Prashant antsh...@gmail.com wrote:
I am facings exact same issue. getObjectById returns stale data even
if call is made inside a transaction.
I performed an experiment - I ran two crons in two different modules,
both trying to increment their execution count in an (common, shared)
entity. I was shocked to see that even when I transaction, both crons
are
running as if they are performing transaction in thier own copy of
DataStore. Evertime, each cron will get copy of entity last updated by
itself (ignoring updates made by other cron) and on top of that
transaction
will also commit without any exception.
--
Prashant
On 8:24AM, Tue, 4 Aug 2015 Thomas Becker walkswiththeb...@gmail.com
wrote:
Thanks, Jeff! And one more thing, it's pretty clear from the doc,
but let me make sure anyway. The same is true for ancestor queries, they
guarantee strong consistency even when not performed inside a
transaction,
right?
Thomas
On Mon, Aug 3, 2015 at 8:48 PM, Jeff Schnitzer j...@infohazard.org
wrote:
Yes, that is correct. By default, if you perform a low-level
get-by-key operation, it is strongly consistent whether or not you put
it
in a transaction.
Note that you can, if you want, explicitly request an eventually
consistent get-by-key operation in the low-level api. It does return
faster
since it does not require a quorum response. However, this is a pretty
obscure feature.
Jeff
On Mon, Aug 3, 2015 at
Re: [google-appengine] Re: Stale Data from App Engine Datastore
Thanks for the suggestion, Ryan. I checked my code, and all my JDO calls go
through one function that has all the boilerplate. The last three four
lines of that function are,
} finally {
pm.close();
}
}
so we can safely assume that the persistence manger always gets closed.
But let me emphasize again, my original post on this was outright wrong:
it's *not *ancestor queries that are giving me the problem, it is
specifically the JDO function GetObjectById. That begs the question, of
course, if GetObjectById is even *supposed* to give strong consistency.
Re-reading the documentation very carefully, I found that the chapter on
the low-level datastore access states that getting entities by id via get()
gives strong consistency. But the chapter on JDO does *not* make a similar
statement about GetObjectById! On the other hand, when you set the option
Unapplied job percentage to 100% in the development environment, the
function GetObjectById will still act strongly consistent. So perhaps that
is the only problem here, that the SDK erroneously treats GetObjectById as
strongly consistent, when really it is not.
Thomas
On Wed, Aug 5, 2015 at 1:23 PM, Ryan (Cloud Platform Support)
rbruy...@google.com wrote:
Check out this link
https://cloud.google.com/appengine/docs/java/datastore/jdo/creatinggettinganddeletingdata#Updating_an_Object
specifically:
Changes are persisted when the PersistenceManager is closed
I ran some test to confirm and not closing resulted in eventual while
closing resulted in strong consistency. Give it a try and let me know if
that helps.
On Saturday, August 1, 2015 at 6:24:55 PM UTC-4, Thomas Becker wrote:
I am using the app engine datastore with Java JDO. According to the
documentation, ancestor queries are always strongly consistent. Recently,
however, I've been seeing a lot of stale data from ancestor queries. It
seems to happen randomly. A query may return the most recent data, and a
moment later, the same query returns stale data.
--
You received this message because you are subscribed to a topic in the
Google Groups Google App Engine group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/google-appengine/1re1FtqllnQ/unsubscribe
.
To unsubscribe from this group and all its topics, send an email to
google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at http://groups.google.com/group/google-appengine.
To view this discussion on the web visit
https://groups.google.com/d/msgid/google-appengine/96de2818-896a-4877-aaa0-d496cc81dc06%40googlegroups.com
https://groups.google.com/d/msgid/google-appengine/96de2818-896a-4877-aaa0-d496cc81dc06%40googlegroups.com?utm_medium=emailutm_source=footer
.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups
Google App Engine group.
To unsubscribe from this group and stop receiving emails from it, send an email
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at http://groups.google.com/group/google-appengine.
To view this discussion on the web visit
https://groups.google.com/d/msgid/google-appengine/CAPxj-GBL-NZU0c8Bo2zLtaEUryAWVXfmUVeEk%3DNicecR9kAx_A%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
Re: [google-appengine] Re: Stale Data from App Engine Datastore
I have less direct experience here since my apps tend to be get-by-key-heavy, but the docs are very explicit about ancestor queries being strongly consistent outside of a transaction by default and I believe them. Of course, you can override this (although I would not recommend it unless you know what you are doing). Jeff On Mon, Aug 3, 2015 at 7:53 PM, Thomas Becker walkswiththeb...@gmail.com wrote: Thanks, Jeff! And one more thing, it's pretty clear from the doc, but let me make sure anyway. The same is true for ancestor queries, they guarantee strong consistency even when not performed inside a transaction, right? Thomas On Mon, Aug 3, 2015 at 8:48 PM, Jeff Schnitzer j...@infohazard.org wrote: Yes, that is correct. By default, if you perform a low-level get-by-key operation, it is strongly consistent whether or not you put it in a transaction. Note that you can, if you want, explicitly request an eventually consistent get-by-key operation in the low-level api. It does return faster since it does not require a quorum response. However, this is a pretty obscure feature. Jeff On Mon, Aug 3, 2015 at 7:30 PM, Thomas Becker walkswiththeb...@gmail.com wrote: Thanks for your detailed and thoughtful answer, Jeff. I was aware of the fact that within a transaction, all reads will return the original value even if one writes it within the transaction. That was not the problem, but thanks for pointing it out anyway, it's easy to forget! Other than that, after reading everything on the subject of GAE datastore consistency under JDO, I have decided to fall back to low-level access in those cases where strong consistency is of the essence. At the risk of sounding acerbic or snarky, it seems clear to me that nobody knows what's going on in JDO when it comes to consistency. So just to confirm, it is true that the low-level get-by-key guarantees strong consistency even *when performed outside of a transaction*, correct? Thanks again for your thoughtful answer. Very refreshing indeed. Thomas On Mon, Aug 3, 2015 at 8:13 PM, Jeff Schnitzer j...@infohazard.org wrote: I'm no expert on JDO, but the low-level api does guarantee strong consistency for get-by-key operations (that is, unless you explicitly ask for eventual consistency). One of those SO responses suggested it is possible to have the GAE/JDO plugin log its low-level operations - that should at least point you in the right direction. One other thing that might be related: In the low-level API, within a single transaction, all reads will return the original value even if you write it within the transaction. I don't know what JDO does with this; I would expect that subsequent reads would return a session-cached instance and hide this little quirk. But maybe it's related to what you are seeing? Jeff On Mon, Aug 3, 2015 at 1:52 PM, Thomas Becker walkswiththeb...@gmail.com wrote: Thanks for your quick reply, Ryan. I have found several postings on the Web that describe the same problem, namely, GetObjectById returning stale data. Here are two from StackOverflow: http://stackoverflow.com/questions/26377347/inconsistent-fetch-from-google-app-engine-datastore http://stackoverflow.com/questions/22005349/appengine-jdo-with-multiple-server-instances-loses-transaction-updates-on-entiti Under the circumstances, I don't think it would be a good use of my time to make a demo app. It seems clear to me that the issue exists. Besides, the problem is so intermittent that a demo app is of very limited value. I would be perfectly happy with a statement of the kind, GetObjectById() does not guarantee strong consistency, that's how it is, and here's what you do instead to get strongly consistent data. I just need to know what's going on. On Mon, Aug 3, 2015 at 2:32 PM, Ryan (Cloud Platform Support) rbruy...@google.com wrote: If you can create a sample app that shows this post on it on the PIT https://code.google.com/p/googleappengine/issues/entry so I can investigate this further. On Saturday, August 1, 2015 at 6:24:55 PM UTC-4, Thomas Becker wrote: I am using the app engine datastore with Java JDO. According to the documentation, ancestor queries are always strongly consistent. Recently, however, I've been seeing a lot of stale data from ancestor queries. It seems to happen randomly. A query may return the most recent data, and a moment later, the same query returns stale data. -- You received this message because you are subscribed to a topic in the Google Groups Google App Engine group. To unsubscribe from this topic, visit https://groups.google.com/d/topic/google-appengine/1re1FtqllnQ/unsubscribe . To unsubscribe from this group and all its topics, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine. To view this discussion
Re: [google-appengine] Re: Stale Data from App Engine Datastore
Again, JDO is not my area of expertise, but if so, this seems like a shockingly obvious issue. Can you post a sample of the code you use to demonstrate the problem? Jeff On Mon, Aug 3, 2015 at 8:11 PM, Prashant antsh...@gmail.com wrote: I am facings exact same issue. getObjectById returns stale data even if call is made inside a transaction. I performed an experiment - I ran two crons in two different modules, both trying to increment their execution count in an (common, shared) entity. I was shocked to see that even when I transaction, both crons are running as if they are performing transaction in thier own copy of DataStore. Evertime, each cron will get copy of entity last updated by itself (ignoring updates made by other cron) and on top of that transaction will also commit without any exception. -- Prashant On 8:24AM, Tue, 4 Aug 2015 Thomas Becker walkswiththeb...@gmail.com wrote: Thanks, Jeff! And one more thing, it's pretty clear from the doc, but let me make sure anyway. The same is true for ancestor queries, they guarantee strong consistency even when not performed inside a transaction, right? Thomas On Mon, Aug 3, 2015 at 8:48 PM, Jeff Schnitzer j...@infohazard.org wrote: Yes, that is correct. By default, if you perform a low-level get-by-key operation, it is strongly consistent whether or not you put it in a transaction. Note that you can, if you want, explicitly request an eventually consistent get-by-key operation in the low-level api. It does return faster since it does not require a quorum response. However, this is a pretty obscure feature. Jeff On Mon, Aug 3, 2015 at 7:30 PM, Thomas Becker walkswiththeb...@gmail.com wrote: Thanks for your detailed and thoughtful answer, Jeff. I was aware of the fact that within a transaction, all reads will return the original value even if one writes it within the transaction. That was not the problem, but thanks for pointing it out anyway, it's easy to forget! Other than that, after reading everything on the subject of GAE datastore consistency under JDO, I have decided to fall back to low-level access in those cases where strong consistency is of the essence. At the risk of sounding acerbic or snarky, it seems clear to me that nobody knows what's going on in JDO when it comes to consistency. So just to confirm, it is true that the low-level get-by-key guarantees strong consistency even *when performed outside of a transaction*, correct? Thanks again for your thoughtful answer. Very refreshing indeed. Thomas On Mon, Aug 3, 2015 at 8:13 PM, Jeff Schnitzer j...@infohazard.org wrote: I'm no expert on JDO, but the low-level api does guarantee strong consistency for get-by-key operations (that is, unless you explicitly ask for eventual consistency). One of those SO responses suggested it is possible to have the GAE/JDO plugin log its low-level operations - that should at least point you in the right direction. One other thing that might be related: In the low-level API, within a single transaction, all reads will return the original value even if you write it within the transaction. I don't know what JDO does with this; I would expect that subsequent reads would return a session-cached instance and hide this little quirk. But maybe it's related to what you are seeing? Jeff On Mon, Aug 3, 2015 at 1:52 PM, Thomas Becker walkswiththeb...@gmail.com wrote: Thanks for your quick reply, Ryan. I have found several postings on the Web that describe the same problem, namely, GetObjectById returning stale data. Here are two from StackOverflow: http://stackoverflow.com/questions/26377347/inconsistent-fetch-from-google-app-engine-datastore http://stackoverflow.com/questions/22005349/appengine-jdo-with-multiple-server-instances-loses-transaction-updates-on-entiti Under the circumstances, I don't think it would be a good use of my time to make a demo app. It seems clear to me that the issue exists. Besides, the problem is so intermittent that a demo app is of very limited value. I would be perfectly happy with a statement of the kind, GetObjectById() does not guarantee strong consistency, that's how it is, and here's what you do instead to get strongly consistent data. I just need to know what's going on. On Mon, Aug 3, 2015 at 2:32 PM, Ryan (Cloud Platform Support) rbruy...@google.com wrote: If you can create a sample app that shows this post on it on the PIT https://code.google.com/p/googleappengine/issues/entry so I can investigate this further. On Saturday, August 1, 2015 at 6:24:55 PM UTC-4, Thomas Becker wrote: I am using the app engine datastore with Java JDO. According to the documentation, ancestor queries are always strongly consistent. Recently, however, I've been seeing a lot of stale data from ancestor queries. It seems to happen randomly. A query may return the most recent data, and a moment later, the same query returns stale data. --
Re: [google-appengine] Re: Stale Data from App Engine Datastore
Thanks again for eveybody's input. It looks like I have solved my problem by falling back on the low-level datastore access when strong consistency matters. Problems like this happen, and the bottom line is GAE is a great product that's done a lot for me. So kudos to all members, former and present, of the GAE team at Google. FWIW, here are two observations that could be helpful in understanding the behavior of the gae/JDO function GetObjectById: 1) If you set the option Unapplied job percentage to 100% in the SDK, that is, you force eventually consistent to mean never consistent, then GetObjectById still acts strongly consistent. This means that the SDK thinks that GetObjectById should act strongly consistent, just like the low level get function. 2) Demonstrating that GetObjectById can return stale data in the production environment is difficult because the behavior is, by its very nature, intermittent. But one can increase the probability of seeing stale data vastly if one performs the write and subsequent read from two different *versions* of the app. As others have observed, the probability already increases when write and read are performed from different instances of the app. But with different *versions *(one of them being the default version), it's been happening pretty much every time in my context. The low-level get, by contrast, has always acted strongly consistent for me, even across versions. Thomas On Tue, Aug 4, 2015 at 12:07 AM, Jeff Schnitzer j...@infohazard.org wrote: Again, JDO is not my area of expertise, but if so, this seems like a shockingly obvious issue. Can you post a sample of the code you use to demonstrate the problem? Jeff On Mon, Aug 3, 2015 at 8:11 PM, Prashant antsh...@gmail.com wrote: I am facings exact same issue. getObjectById returns stale data even if call is made inside a transaction. I performed an experiment - I ran two crons in two different modules, both trying to increment their execution count in an (common, shared) entity. I was shocked to see that even when I transaction, both crons are running as if they are performing transaction in thier own copy of DataStore. Evertime, each cron will get copy of entity last updated by itself (ignoring updates made by other cron) and on top of that transaction will also commit without any exception. -- Prashant On 8:24AM, Tue, 4 Aug 2015 Thomas Becker walkswiththeb...@gmail.com wrote: Thanks, Jeff! And one more thing, it's pretty clear from the doc, but let me make sure anyway. The same is true for ancestor queries, they guarantee strong consistency even when not performed inside a transaction, right? Thomas On Mon, Aug 3, 2015 at 8:48 PM, Jeff Schnitzer j...@infohazard.org wrote: Yes, that is correct. By default, if you perform a low-level get-by-key operation, it is strongly consistent whether or not you put it in a transaction. Note that you can, if you want, explicitly request an eventually consistent get-by-key operation in the low-level api. It does return faster since it does not require a quorum response. However, this is a pretty obscure feature. Jeff On Mon, Aug 3, 2015 at 7:30 PM, Thomas Becker walkswiththeb...@gmail.com wrote: Thanks for your detailed and thoughtful answer, Jeff. I was aware of the fact that within a transaction, all reads will return the original value even if one writes it within the transaction. That was not the problem, but thanks for pointing it out anyway, it's easy to forget! Other than that, after reading everything on the subject of GAE datastore consistency under JDO, I have decided to fall back to low-level access in those cases where strong consistency is of the essence. At the risk of sounding acerbic or snarky, it seems clear to me that nobody knows what's going on in JDO when it comes to consistency. So just to confirm, it is true that the low-level get-by-key guarantees strong consistency even *when performed outside of a transaction*, correct? Thanks again for your thoughtful answer. Very refreshing indeed. Thomas On Mon, Aug 3, 2015 at 8:13 PM, Jeff Schnitzer j...@infohazard.org wrote: I'm no expert on JDO, but the low-level api does guarantee strong consistency for get-by-key operations (that is, unless you explicitly ask for eventual consistency). One of those SO responses suggested it is possible to have the GAE/JDO plugin log its low-level operations - that should at least point you in the right direction. One other thing that might be related: In the low-level API, within a single transaction, all reads will return the original value even if you write it within the transaction. I don't know what JDO does with this; I would expect that subsequent reads would return a session-cached instance and hide this little quirk. But maybe it's related to what you are seeing? Jeff On Mon, Aug 3, 2015 at 1:52 PM, Thomas Becker walkswiththeb...@gmail.com wrote: Thanks for your quick
Re: [google-appengine] Re: Stale Data from App Engine Datastore
I'm no expert on JDO, but the low-level api does guarantee strong consistency for get-by-key operations (that is, unless you explicitly ask for eventual consistency). One of those SO responses suggested it is possible to have the GAE/JDO plugin log its low-level operations - that should at least point you in the right direction. One other thing that might be related: In the low-level API, within a single transaction, all reads will return the original value even if you write it within the transaction. I don't know what JDO does with this; I would expect that subsequent reads would return a session-cached instance and hide this little quirk. But maybe it's related to what you are seeing? Jeff On Mon, Aug 3, 2015 at 1:52 PM, Thomas Becker walkswiththeb...@gmail.com wrote: Thanks for your quick reply, Ryan. I have found several postings on the Web that describe the same problem, namely, GetObjectById returning stale data. Here are two from StackOverflow: http://stackoverflow.com/questions/26377347/inconsistent-fetch-from-google-app-engine-datastore http://stackoverflow.com/questions/22005349/appengine-jdo-with-multiple-server-instances-loses-transaction-updates-on-entiti Under the circumstances, I don't think it would be a good use of my time to make a demo app. It seems clear to me that the issue exists. Besides, the problem is so intermittent that a demo app is of very limited value. I would be perfectly happy with a statement of the kind, GetObjectById() does not guarantee strong consistency, that's how it is, and here's what you do instead to get strongly consistent data. I just need to know what's going on. On Mon, Aug 3, 2015 at 2:32 PM, Ryan (Cloud Platform Support) rbruy...@google.com wrote: If you can create a sample app that shows this post on it on the PIT https://code.google.com/p/googleappengine/issues/entry so I can investigate this further. On Saturday, August 1, 2015 at 6:24:55 PM UTC-4, Thomas Becker wrote: I am using the app engine datastore with Java JDO. According to the documentation, ancestor queries are always strongly consistent. Recently, however, I've been seeing a lot of stale data from ancestor queries. It seems to happen randomly. A query may return the most recent data, and a moment later, the same query returns stale data. -- You received this message because you are subscribed to a topic in the Google Groups Google App Engine group. To unsubscribe from this topic, visit https://groups.google.com/d/topic/google-appengine/1re1FtqllnQ/unsubscribe . To unsubscribe from this group and all its topics, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine. To view this discussion on the web visit https://groups.google.com/d/msgid/google-appengine/9723734e-0b14-49c0-8c31-3407df8454ad%40googlegroups.com https://groups.google.com/d/msgid/google-appengine/9723734e-0b14-49c0-8c31-3407df8454ad%40googlegroups.com?utm_medium=emailutm_source=footer . For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine. To view this discussion on the web visit https://groups.google.com/d/msgid/google-appengine/CAPxj-GA3OKiGfjwgYgKmvZ%3DSi7ZXG3VLvts7%2BjWZ7c2CsC693Q%40mail.gmail.com https://groups.google.com/d/msgid/google-appengine/CAPxj-GA3OKiGfjwgYgKmvZ%3DSi7ZXG3VLvts7%2BjWZ7c2CsC693Q%40mail.gmail.com?utm_medium=emailutm_source=footer . For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine. To view this discussion on the web visit https://groups.google.com/d/msgid/google-appengine/CADK-0uh4Q8kswYMhD5RcxeQJnO3rFshgkfGEbB%2BXzHVXQLbGiw%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
Re: [google-appengine] Re: Stale Data from App Engine Datastore
I am facings exact same issue. getObjectById returns stale data even if call is made inside a transaction. I performed an experiment - I ran two crons in two different modules, both trying to increment their execution count in an (common, shared) entity. I was shocked to see that even when I transaction, both crons are running as if they are performing transaction in thier own copy of DataStore. Evertime, each cron will get copy of entity last updated by itself (ignoring updates made by other cron) and on top of that transaction will also commit without any exception. -- Prashant On 8:24AM, Tue, 4 Aug 2015 Thomas Becker walkswiththeb...@gmail.com wrote: Thanks, Jeff! And one more thing, it's pretty clear from the doc, but let me make sure anyway. The same is true for ancestor queries, they guarantee strong consistency even when not performed inside a transaction, right? Thomas On Mon, Aug 3, 2015 at 8:48 PM, Jeff Schnitzer j...@infohazard.org wrote: Yes, that is correct. By default, if you perform a low-level get-by-key operation, it is strongly consistent whether or not you put it in a transaction. Note that you can, if you want, explicitly request an eventually consistent get-by-key operation in the low-level api. It does return faster since it does not require a quorum response. However, this is a pretty obscure feature. Jeff On Mon, Aug 3, 2015 at 7:30 PM, Thomas Becker walkswiththeb...@gmail.com wrote: Thanks for your detailed and thoughtful answer, Jeff. I was aware of the fact that within a transaction, all reads will return the original value even if one writes it within the transaction. That was not the problem, but thanks for pointing it out anyway, it's easy to forget! Other than that, after reading everything on the subject of GAE datastore consistency under JDO, I have decided to fall back to low-level access in those cases where strong consistency is of the essence. At the risk of sounding acerbic or snarky, it seems clear to me that nobody knows what's going on in JDO when it comes to consistency. So just to confirm, it is true that the low-level get-by-key guarantees strong consistency even *when performed outside of a transaction*, correct? Thanks again for your thoughtful answer. Very refreshing indeed. Thomas On Mon, Aug 3, 2015 at 8:13 PM, Jeff Schnitzer j...@infohazard.org wrote: I'm no expert on JDO, but the low-level api does guarantee strong consistency for get-by-key operations (that is, unless you explicitly ask for eventual consistency). One of those SO responses suggested it is possible to have the GAE/JDO plugin log its low-level operations - that should at least point you in the right direction. One other thing that might be related: In the low-level API, within a single transaction, all reads will return the original value even if you write it within the transaction. I don't know what JDO does with this; I would expect that subsequent reads would return a session-cached instance and hide this little quirk. But maybe it's related to what you are seeing? Jeff On Mon, Aug 3, 2015 at 1:52 PM, Thomas Becker walkswiththeb...@gmail.com wrote: Thanks for your quick reply, Ryan. I have found several postings on the Web that describe the same problem, namely, GetObjectById returning stale data. Here are two from StackOverflow: http://stackoverflow.com/questions/26377347/inconsistent-fetch-from-google-app-engine-datastore http://stackoverflow.com/questions/22005349/appengine-jdo-with-multiple-server-instances-loses-transaction-updates-on-entiti Under the circumstances, I don't think it would be a good use of my time to make a demo app. It seems clear to me that the issue exists. Besides, the problem is so intermittent that a demo app is of very limited value. I would be perfectly happy with a statement of the kind, GetObjectById() does not guarantee strong consistency, that's how it is, and here's what you do instead to get strongly consistent data. I just need to know what's going on. On Mon, Aug 3, 2015 at 2:32 PM, Ryan (Cloud Platform Support) rbruy...@google.com wrote: If you can create a sample app that shows this post on it on the PIT https://code.google.com/p/googleappengine/issues/entry so I can investigate this further. On Saturday, August 1, 2015 at 6:24:55 PM UTC-4, Thomas Becker wrote: I am using the app engine datastore with Java JDO. According to the documentation, ancestor queries are always strongly consistent. Recently, however, I've been seeing a lot of stale data from ancestor queries. It seems to happen randomly. A query may return the most recent data, and a moment later, the same query returns stale data. -- You received this message because you are subscribed to a topic in the Google Groups Google App Engine group. To unsubscribe from this topic, visit https://groups.google.com/d/topic/google-appengine/1re1FtqllnQ/unsubscribe . To unsubscribe from this
Re: [google-appengine] Re: Stale Data from App Engine Datastore
Thanks, Jeff! And one more thing, it's pretty clear from the doc, but let me make sure anyway. The same is true for ancestor queries, they guarantee strong consistency even when not performed inside a transaction, right? Thomas On Mon, Aug 3, 2015 at 8:48 PM, Jeff Schnitzer j...@infohazard.org wrote: Yes, that is correct. By default, if you perform a low-level get-by-key operation, it is strongly consistent whether or not you put it in a transaction. Note that you can, if you want, explicitly request an eventually consistent get-by-key operation in the low-level api. It does return faster since it does not require a quorum response. However, this is a pretty obscure feature. Jeff On Mon, Aug 3, 2015 at 7:30 PM, Thomas Becker walkswiththeb...@gmail.com wrote: Thanks for your detailed and thoughtful answer, Jeff. I was aware of the fact that within a transaction, all reads will return the original value even if one writes it within the transaction. That was not the problem, but thanks for pointing it out anyway, it's easy to forget! Other than that, after reading everything on the subject of GAE datastore consistency under JDO, I have decided to fall back to low-level access in those cases where strong consistency is of the essence. At the risk of sounding acerbic or snarky, it seems clear to me that nobody knows what's going on in JDO when it comes to consistency. So just to confirm, it is true that the low-level get-by-key guarantees strong consistency even *when performed outside of a transaction*, correct? Thanks again for your thoughtful answer. Very refreshing indeed. Thomas On Mon, Aug 3, 2015 at 8:13 PM, Jeff Schnitzer j...@infohazard.org wrote: I'm no expert on JDO, but the low-level api does guarantee strong consistency for get-by-key operations (that is, unless you explicitly ask for eventual consistency). One of those SO responses suggested it is possible to have the GAE/JDO plugin log its low-level operations - that should at least point you in the right direction. One other thing that might be related: In the low-level API, within a single transaction, all reads will return the original value even if you write it within the transaction. I don't know what JDO does with this; I would expect that subsequent reads would return a session-cached instance and hide this little quirk. But maybe it's related to what you are seeing? Jeff On Mon, Aug 3, 2015 at 1:52 PM, Thomas Becker walkswiththeb...@gmail.com wrote: Thanks for your quick reply, Ryan. I have found several postings on the Web that describe the same problem, namely, GetObjectById returning stale data. Here are two from StackOverflow: http://stackoverflow.com/questions/26377347/inconsistent-fetch-from-google-app-engine-datastore http://stackoverflow.com/questions/22005349/appengine-jdo-with-multiple-server-instances-loses-transaction-updates-on-entiti Under the circumstances, I don't think it would be a good use of my time to make a demo app. It seems clear to me that the issue exists. Besides, the problem is so intermittent that a demo app is of very limited value. I would be perfectly happy with a statement of the kind, GetObjectById() does not guarantee strong consistency, that's how it is, and here's what you do instead to get strongly consistent data. I just need to know what's going on. On Mon, Aug 3, 2015 at 2:32 PM, Ryan (Cloud Platform Support) rbruy...@google.com wrote: If you can create a sample app that shows this post on it on the PIT https://code.google.com/p/googleappengine/issues/entry so I can investigate this further. On Saturday, August 1, 2015 at 6:24:55 PM UTC-4, Thomas Becker wrote: I am using the app engine datastore with Java JDO. According to the documentation, ancestor queries are always strongly consistent. Recently, however, I've been seeing a lot of stale data from ancestor queries. It seems to happen randomly. A query may return the most recent data, and a moment later, the same query returns stale data. -- You received this message because you are subscribed to a topic in the Google Groups Google App Engine group. To unsubscribe from this topic, visit https://groups.google.com/d/topic/google-appengine/1re1FtqllnQ/unsubscribe . To unsubscribe from this group and all its topics, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com . Visit this group at http://groups.google.com/group/google-appengine. To view this discussion on the web visit https://groups.google.com/d/msgid/google-appengine/9723734e-0b14-49c0-8c31-3407df8454ad%40googlegroups.com https://groups.google.com/d/msgid/google-appengine/9723734e-0b14-49c0-8c31-3407df8454ad%40googlegroups.com?utm_medium=emailutm_source=footer . For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups
Re: [google-appengine] Re: Stale Data from App Engine Datastore
Yes, that is correct. By default, if you perform a low-level get-by-key operation, it is strongly consistent whether or not you put it in a transaction. Note that you can, if you want, explicitly request an eventually consistent get-by-key operation in the low-level api. It does return faster since it does not require a quorum response. However, this is a pretty obscure feature. Jeff On Mon, Aug 3, 2015 at 7:30 PM, Thomas Becker walkswiththeb...@gmail.com wrote: Thanks for your detailed and thoughtful answer, Jeff. I was aware of the fact that within a transaction, all reads will return the original value even if one writes it within the transaction. That was not the problem, but thanks for pointing it out anyway, it's easy to forget! Other than that, after reading everything on the subject of GAE datastore consistency under JDO, I have decided to fall back to low-level access in those cases where strong consistency is of the essence. At the risk of sounding acerbic or snarky, it seems clear to me that nobody knows what's going on in JDO when it comes to consistency. So just to confirm, it is true that the low-level get-by-key guarantees strong consistency even *when performed outside of a transaction*, correct? Thanks again for your thoughtful answer. Very refreshing indeed. Thomas On Mon, Aug 3, 2015 at 8:13 PM, Jeff Schnitzer j...@infohazard.org wrote: I'm no expert on JDO, but the low-level api does guarantee strong consistency for get-by-key operations (that is, unless you explicitly ask for eventual consistency). One of those SO responses suggested it is possible to have the GAE/JDO plugin log its low-level operations - that should at least point you in the right direction. One other thing that might be related: In the low-level API, within a single transaction, all reads will return the original value even if you write it within the transaction. I don't know what JDO does with this; I would expect that subsequent reads would return a session-cached instance and hide this little quirk. But maybe it's related to what you are seeing? Jeff On Mon, Aug 3, 2015 at 1:52 PM, Thomas Becker walkswiththeb...@gmail.com wrote: Thanks for your quick reply, Ryan. I have found several postings on the Web that describe the same problem, namely, GetObjectById returning stale data. Here are two from StackOverflow: http://stackoverflow.com/questions/26377347/inconsistent-fetch-from-google-app-engine-datastore http://stackoverflow.com/questions/22005349/appengine-jdo-with-multiple-server-instances-loses-transaction-updates-on-entiti Under the circumstances, I don't think it would be a good use of my time to make a demo app. It seems clear to me that the issue exists. Besides, the problem is so intermittent that a demo app is of very limited value. I would be perfectly happy with a statement of the kind, GetObjectById() does not guarantee strong consistency, that's how it is, and here's what you do instead to get strongly consistent data. I just need to know what's going on. On Mon, Aug 3, 2015 at 2:32 PM, Ryan (Cloud Platform Support) rbruy...@google.com wrote: If you can create a sample app that shows this post on it on the PIT https://code.google.com/p/googleappengine/issues/entry so I can investigate this further. On Saturday, August 1, 2015 at 6:24:55 PM UTC-4, Thomas Becker wrote: I am using the app engine datastore with Java JDO. According to the documentation, ancestor queries are always strongly consistent. Recently, however, I've been seeing a lot of stale data from ancestor queries. It seems to happen randomly. A query may return the most recent data, and a moment later, the same query returns stale data. -- You received this message because you are subscribed to a topic in the Google Groups Google App Engine group. To unsubscribe from this topic, visit https://groups.google.com/d/topic/google-appengine/1re1FtqllnQ/unsubscribe . To unsubscribe from this group and all its topics, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine. To view this discussion on the web visit https://groups.google.com/d/msgid/google-appengine/9723734e-0b14-49c0-8c31-3407df8454ad%40googlegroups.com https://groups.google.com/d/msgid/google-appengine/9723734e-0b14-49c0-8c31-3407df8454ad%40googlegroups.com?utm_medium=emailutm_source=footer . For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine. To view this
[google-appengine] Re: Stale Data from App Engine Datastore
I would take a look at this article https://cloud.google.com/appengine/docs/java/datastore/jdo/queries, specifically the second paragraph under 'Datastore Read Policy and Call Deadline'.: ' If the query uses an ancestor filter, you can use transactions https://cloud.google.com/appengine/docs/java/datastore/transactions to ensure a consistent result set.) See the article Transaction Isolation in App Engine https://cloud.google.com/appengine/articles/transaction_isolation for more information on how entities and indexes are updated. ' On Saturday, August 1, 2015 at 6:24:55 PM UTC-4, Thomas Becker wrote: I am using the app engine datastore with Java JDO. According to the documentation, ancestor queries are always strongly consistent. Recently, however, I've been seeing a lot of stale data from ancestor queries. It seems to happen randomly. A query may return the most recent data, and a moment later, the same query returns stale data. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine. To view this discussion on the web visit https://groups.google.com/d/msgid/google-appengine/a672af3b-5de7-472c-9fe0-0d61521e8928%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [google-appengine] Re: Stale Data from App Engine Datastore
Thanks for your quick reply, Ryan. I have found several postings on the Web that describe the same problem, namely, GetObjectById returning stale data. Here are two from StackOverflow: http://stackoverflow.com/questions/26377347/inconsistent-fetch-from-google-app-engine-datastore http://stackoverflow.com/questions/22005349/appengine-jdo-with-multiple-server-instances-loses-transaction-updates-on-entiti Under the circumstances, I don't think it would be a good use of my time to make a demo app. It seems clear to me that the issue exists. Besides, the problem is so intermittent that a demo app is of very limited value. I would be perfectly happy with a statement of the kind, GetObjectById() does not guarantee strong consistency, that's how it is, and here's what you do instead to get strongly consistent data. I just need to know what's going on. On Mon, Aug 3, 2015 at 2:32 PM, Ryan (Cloud Platform Support) rbruy...@google.com wrote: If you can create a sample app that shows this post on it on the PIT https://code.google.com/p/googleappengine/issues/entry so I can investigate this further. On Saturday, August 1, 2015 at 6:24:55 PM UTC-4, Thomas Becker wrote: I am using the app engine datastore with Java JDO. According to the documentation, ancestor queries are always strongly consistent. Recently, however, I've been seeing a lot of stale data from ancestor queries. It seems to happen randomly. A query may return the most recent data, and a moment later, the same query returns stale data. -- You received this message because you are subscribed to a topic in the Google Groups Google App Engine group. To unsubscribe from this topic, visit https://groups.google.com/d/topic/google-appengine/1re1FtqllnQ/unsubscribe . To unsubscribe from this group and all its topics, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine. To view this discussion on the web visit https://groups.google.com/d/msgid/google-appengine/9723734e-0b14-49c0-8c31-3407df8454ad%40googlegroups.com https://groups.google.com/d/msgid/google-appengine/9723734e-0b14-49c0-8c31-3407df8454ad%40googlegroups.com?utm_medium=emailutm_source=footer . For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine. To view this discussion on the web visit https://groups.google.com/d/msgid/google-appengine/CAPxj-GA3OKiGfjwgYgKmvZ%3DSi7ZXG3VLvts7%2BjWZ7c2CsC693Q%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
[google-appengine] Re: Stale Data from App Engine Datastore
If you can create a sample app that shows this post on it on the PIT https://code.google.com/p/googleappengine/issues/entry so I can investigate this further. On Saturday, August 1, 2015 at 6:24:55 PM UTC-4, Thomas Becker wrote: I am using the app engine datastore with Java JDO. According to the documentation, ancestor queries are always strongly consistent. Recently, however, I've been seeing a lot of stale data from ancestor queries. It seems to happen randomly. A query may return the most recent data, and a moment later, the same query returns stale data. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine. To view this discussion on the web visit https://groups.google.com/d/msgid/google-appengine/9723734e-0b14-49c0-8c31-3407df8454ad%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[google-appengine] Re: Stale Data from App Engine Datastore
Thanks for the reply, Ryan. Actually, I need to apologize because the way I described my problem was incorrect. I'm seeing stale data not from queries, but from calls to the JDO function GetObjectById(). The calls are inside a transaction. Since the analogous low-level datastore access function get() is guaranteed to return strongly consistent data, I was under the impression that the same would be true for GetObjectById(). Am I perhaps wrong about this? On Monday, August 3, 2015 at 1:28:14 PM UTC-6, Ryan (Cloud Platform Support) wrote: I would take a look at this article https://cloud.google.com/appengine/docs/java/datastore/jdo/queries, specifically the second paragraph under 'Datastore Read Policy and Call Deadline'.: ' If the query uses an ancestor filter, you can use transactions https://cloud.google.com/appengine/docs/java/datastore/transactions to ensure a consistent result set.) See the article Transaction Isolation in App Engine https://cloud.google.com/appengine/articles/transaction_isolation for more information on how entities and indexes are updated. ' On Saturday, August 1, 2015 at 6:24:55 PM UTC-4, Thomas Becker wrote: I am using the app engine datastore with Java JDO. According to the documentation, ancestor queries are always strongly consistent. Recently, however, I've been seeing a lot of stale data from ancestor queries. It seems to happen randomly. A query may return the most recent data, and a moment later, the same query returns stale data. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine. To view this discussion on the web visit https://groups.google.com/d/msgid/google-appengine/41d50d35-8e07-4f36-9a8f-7e1909f6c242%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
