[google-appengine] Re: entity stuck at the wrong end of the index
On Apr 16, 12:08 pm, ryan wrote: > in both cases, a successful put() on the entity in question will fix > the problem. we're still working on issue 901, and we definitely > expect to have that resolved soon. as for put() and delete() calls > that return errors (like timeouts), we're always working to reduce > those, but they'll unfortunately always be possible, so you'll want to > plan for that. Are you saying that if a put fails due to a timeout (and there appear to be multiple different sorts of timeouts that happen in practice), that the put operation can be half complete? If so, then does google have recommended code for handling timeout exceptions during puts that leave the datastore in a defined state? For example, after a failed put, should I try a delete if a key is available? I'm sure that many people would want to know what the best approach is. Thanks Philip --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: entity stuck at the wrong end of the index
On Apr 16, 12:08 pm, ryan wrote: > the discussion here is pretty on target. thanks, all! > > beyond the points already raised, i'd guess this is caused either by > writes (puts or deletes) that returned errors, and therefore didn't > update all indices, or an index consistency bug that we're currently > working on: > > http://code.google.com/p/googleappengine/issues/detail?id=901 > > in both cases, a successful put() on the entity in question will fix > the problem. we're still working on issue 901, and we definitely > expect to have that resolved soon. as for put() and delete() calls > that return errors (like timeouts), we're always working to reduce > those, but they'll unfortunately always be possible, so you'll want to > plan for that. What is the recommended way to deal with such an error? I'd be okay with the put failing entirely, or eventually updating the indexes automatically once the situation resolves, but partially updating indexes sounds like it can only lead to trouble - particularly if the error condition preventing the put from completing continues for an extended period of time (how long do tablet servers take to be replaced if one fails?) --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: entity stuck at the wrong end of the index
Hi Matthew,
Thanks for these great ideas!
Ben
On Apr 15, 8:30 pm, Matthew Blain wrote:
> Hi Ben,
> One possible way to handle this is to retrieve the top 10 items in
> your 'queue', then loop through them in a transaction until you find
> one which is not recent.
>
> Better yet, you can cache this query--query a bit more, say 100 items,
> save the keys into some simple structure in memcache, and consider
> using memcache.incr to track where you are in those 100 items as you
> test for the ability to pull off the queue--I assume you don't need to
> assign these in exact order to your users.
>
> something like this pseudocode:
>
> get top item from the queue():
> possiblekeys = memcache.get("queuekeys")
> index = memcache.incr("queueindex")
> if index is None or index > len(possiblekeys):
> fallback to current code which does the query of the top 1 item
> else:
> key = possiblekeys[index]
> in transaction:
> check that date is not recent
> update date
> if transaction succeeded:
> return db.get(key)
> else:
> return gettopitemfromqueue()
>
> then some code to fill the queue:
> fillqueue():
> possibleEntities query(500 most recent items)
> possibleKeys = [keys of possibleEntities]
> memcache.set_multi({"queuekeys": possibleKeys, "queueindex": 0})
>
> then you need to fill the queue somehow--either some sort of mechanism
> when the cached queue is almost empty (which has mild locking issues),
> or some external mechanism like cron.
>
> --Matthew
>
> On Apr 15, 1:31 pm, BenNevile wrote:
>
>
>
> > This is happening frequently today. I think it must expose a weakness
> > in the transaction code. Hey, I know that what I'm doing with this
> > code is totally lame -- basically causing many many transaction
> > collisions - but clearly there's a problem here with the transaction
> > code that creates the possibility of corrupting an entity. Not not
> > not not cool.
>
> > I'm working under a harsh time constraint right now (start of hockey
> > playoffs) and my users need this crappy queue I've set up, so I'm not
> > currently at liberty to leave one of these entities in its corrupted
> > state for very long. But I would like to work with Google to get this
> > sorted out.
>
> > Ben
>
> > On Apr 15, 8:04 am, BenNevile wrote:
>
> > > To add another twist, now I have an entity at the top of the stack
> > > that is found using
>
> > > q = FacebookUser.all()
> > > q.filter('authorized = ',True)
> > > q.order('last_nudged_at')
> > > u = q.get()
>
> > > but, if you try to load this entity by key, as you need to do for a
> > > transaction, you get None back. Trying to access that entity's
> > > record in the Data Viewer also threw an exception. I was, however,
> > > able to delete it from the top of that index listing using the Data
> > > Viewer's GQL option.
>
> > > Ben
>
> > > On Apr 14, 3:38 pm, BenNevile wrote:
>
> > > > Hi Jeff,
>
> > > > To clarify, the last_nudged_at property *is* changed, but the entity
> > > > stays at the top of the stack.
>
> > > > Ben
>
> > > > On Apr 14, 12:26 pm, Jeff S wrote:
>
> > > > > Hi Ben,
>
> > > > > If you are experiencing write contention on some of these updates,
> > > > > then that may explain why the last_nudged_at property is not changed
> > > > > and an entity tends to stay at the top of the stack. Do you see
> > > > > datastore timeout errors in these cases (I imagine the exception may
> > > > > be hidden somewhere in the JavaScript <--> server exchange)?
>
> > > > > Thank you,
>
> > > > > Jeff
>
> > > > > On Apr 14, 11:48 am, BenNevile wrote:
>
> > > > > > bump. I have observed this many times over the past few days.
> > > > > > Maybe
> > > > > > has to do with transactions and/or write contention, since there is
> > > > > > a
> > > > > > lot of that in my app.
>
> > > > > > Ben
>
> > > > > > On Apr 1, 8:23 pm, BenNevile wrote:
>
> > > > > > > Hi GAE team and devotees,
>
> > > > > > > I have a entity that looks like
>
> > > > > > > class FacebookUser(db.Model):
> > > > > > > expired_fotm = db.BooleanProperty()
> > > > > > > last_nudged_at = db.DateTimeProperty()
>
> > > > > > > and an index that looks like
>
> > > > > > > - kind: FacebookUser
> > > > > > > properties:
> > > > > > > - name: expired_fotm
> > > > > > > - name: last_nudged_at
>
> > > > > > > Unbeknownst to the users of my application, some javascript is
> > > > > > > constantly polling a worker handler that finds the first entity
> > > > > > > in the
> > > > > > > index, operates on it, then sets "last_nudged_at" to the current
> > > > > > > time. There are more than a million of these entities so it takes
> > > > > > > some time to go through them all.
>
> > > > > > > This afternoon using the Data Viewer and the GQL query "SELECT *
> > > > > > > FROM
> > > > > > > FacebookUser where expired_fotm=True order by last_nudged_at" I
> > > > > > > observed several times that an entity would remain "stu
[google-appengine] Re: entity stuck at the wrong end of the index
Hi Ryan, So I guess what you're saying here is that a put() is not atomic with respect to all indices. I will keep that in mind for sure, but it's unfortunate - adds a lot of code if you have to check all the time to actually see if an index is correct! Ben On Apr 16, 9:08 am, ryan wrote: > the discussion here is pretty on target. thanks, all! > > beyond the points already raised, i'd guess this is caused either by > writes (puts or deletes) that returned errors, and therefore didn't > update all indices, or an index consistency bug that we're currently > working on: > > http://code.google.com/p/googleappengine/issues/detail?id=901 > > in both cases, a successful put() on the entity in question will fix > the problem. we're still working on issue 901, and we definitely > expect to have that resolved soon. as for put() and delete() calls > that return errors (like timeouts), we're always working to reduce > those, but they'll unfortunately always be possible, so you'll want to > plan for that. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: entity stuck at the wrong end of the index
the discussion here is pretty on target. thanks, all! beyond the points already raised, i'd guess this is caused either by writes (puts or deletes) that returned errors, and therefore didn't update all indices, or an index consistency bug that we're currently working on: http://code.google.com/p/googleappengine/issues/detail?id=901 in both cases, a successful put() on the entity in question will fix the problem. we're still working on issue 901, and we definitely expect to have that resolved soon. as for put() and delete() calls that return errors (like timeouts), we're always working to reduce those, but they'll unfortunately always be possible, so you'll want to plan for that. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: entity stuck at the wrong end of the index
Hi Ben,
One possible way to handle this is to retrieve the top 10 items in
your 'queue', then loop through them in a transaction until you find
one which is not recent.
Better yet, you can cache this query--query a bit more, say 100 items,
save the keys into some simple structure in memcache, and consider
using memcache.incr to track where you are in those 100 items as you
test for the ability to pull off the queue--I assume you don't need to
assign these in exact order to your users.
something like this pseudocode:
get top item from the queue():
possiblekeys = memcache.get("queuekeys")
index = memcache.incr("queueindex")
if index is None or index > len(possiblekeys):
fallback to current code which does the query of the top 1 item
else:
key = possiblekeys[index]
in transaction:
check that date is not recent
update date
if transaction succeeded:
return db.get(key)
else:
return gettopitemfromqueue()
then some code to fill the queue:
fillqueue():
possibleEntities query(500 most recent items)
possibleKeys = [keys of possibleEntities]
memcache.set_multi({"queuekeys": possibleKeys, "queueindex": 0})
then you need to fill the queue somehow--either some sort of mechanism
when the cached queue is almost empty (which has mild locking issues),
or some external mechanism like cron.
--Matthew
On Apr 15, 1:31 pm, Ben Nevile wrote:
> This is happening frequently today. I think it must expose a weakness
> in the transaction code. Hey, I know that what I'm doing with this
> code is totally lame -- basically causing many many transaction
> collisions - but clearly there's a problem here with the transaction
> code that creates the possibility of corrupting an entity. Not not
> not not cool.
>
> I'm working under a harsh time constraint right now (start of hockey
> playoffs) and my users need this crappy queue I've set up, so I'm not
> currently at liberty to leave one of these entities in its corrupted
> state for very long. But I would like to work with Google to get this
> sorted out.
>
> Ben
>
> On Apr 15, 8:04 am, Ben Nevile wrote:
>
> > To add another twist, now I have an entity at the top of the stack
> > that is found using
>
> > q = FacebookUser.all()
> > q.filter('authorized = ',True)
> > q.order('last_nudged_at')
> > u = q.get()
>
> > but, if you try to load this entity by key, as you need to do for a
> > transaction, you get None back. Trying to access that entity's
> > record in the Data Viewer also threw an exception. I was, however,
> > able to delete it from the top of that index listing using the Data
> > Viewer's GQL option.
>
> > Ben
>
> > On Apr 14, 3:38 pm, BenNevile wrote:
>
> > > Hi Jeff,
>
> > > To clarify, the last_nudged_at property *is* changed, but the entity
> > > stays at the top of the stack.
>
> > > Ben
>
> > > On Apr 14, 12:26 pm, Jeff S wrote:
>
> > > > Hi Ben,
>
> > > > If you are experiencing write contention on some of these updates,
> > > > then that may explain why the last_nudged_at property is not changed
> > > > and an entity tends to stay at the top of the stack. Do you see
> > > > datastore timeout errors in these cases (I imagine the exception may
> > > > be hidden somewhere in the JavaScript <--> server exchange)?
>
> > > > Thank you,
>
> > > > Jeff
>
> > > > On Apr 14, 11:48 am, BenNevile wrote:
>
> > > > > bump. I have observed this many times over the past few days. Maybe
> > > > > has to do with transactions and/or write contention, since there is a
> > > > > lot of that in my app.
>
> > > > > Ben
>
> > > > > On Apr 1, 8:23 pm, BenNevile wrote:
>
> > > > > > Hi GAE team and devotees,
>
> > > > > > I have a entity that looks like
>
> > > > > > class FacebookUser(db.Model):
> > > > > > expired_fotm = db.BooleanProperty()
> > > > > > last_nudged_at =db.DateTimeProperty()
>
> > > > > > and an index that looks like
>
> > > > > > - kind: FacebookUser
> > > > > > properties:
> > > > > > - name: expired_fotm
> > > > > > - name: last_nudged_at
>
> > > > > > Unbeknownst to the users of my application, some javascript is
> > > > > > constantly polling a worker handler that finds the first entity in
> > > > > > the
> > > > > > index, operates on it, then sets "last_nudged_at" to the current
> > > > > > time. There are more than a million of these entities so it takes
> > > > > > some time to go through them all.
>
> > > > > > This afternoon using the Data Viewer and the GQL query "SELECT *
> > > > > > FROM
> > > > > > FacebookUser where expired_fotm=True order by last_nudged_at" I
> > > > > > observed several times that an entity would remain "stuck" in the
> > > > > > first position in the index after it was processed. Usually it
> > > > > > would
> > > > > > remain there only for 10-30 seconds. However this evening one
> > > > > > entity
> > > > > > stayed in an incorrect index position for several hours. I put() it
> > > > > > again and it properly indexed
[google-appengine] Re: entity stuck at the wrong end of the index
This is happening frequently today. I think it must expose a weakness
in the transaction code. Hey, I know that what I'm doing with this
code is totally lame -- basically causing many many transaction
collisions - but clearly there's a problem here with the transaction
code that creates the possibility of corrupting an entity. Not not
not not cool.
I'm working under a harsh time constraint right now (start of hockey
playoffs) and my users need this crappy queue I've set up, so I'm not
currently at liberty to leave one of these entities in its corrupted
state for very long. But I would like to work with Google to get this
sorted out.
Ben
On Apr 15, 8:04 am, Ben Nevile wrote:
> To add another twist, now I have an entity at the top of the stack
> that is found using
>
> q = FacebookUser.all()
> q.filter('authorized = ',True)
> q.order('last_nudged_at')
> u = q.get()
>
> but, if you try to load this entity by key, as you need to do for a
> transaction, you get None back. Trying to access that entity's
> record in the Data Viewer also threw an exception. I was, however,
> able to delete it from the top of that index listing using the Data
> Viewer's GQL option.
>
> Ben
>
> On Apr 14, 3:38 pm, BenNevile wrote:
>
>
>
> > Hi Jeff,
>
> > To clarify, the last_nudged_at property *is* changed, but the entity
> > stays at the top of the stack.
>
> > Ben
>
> > On Apr 14, 12:26 pm, Jeff S wrote:
>
> > > Hi Ben,
>
> > > If you are experiencing write contention on some of these updates,
> > > then that may explain why the last_nudged_at property is not changed
> > > and an entity tends to stay at the top of the stack. Do you see
> > > datastore timeout errors in these cases (I imagine the exception may
> > > be hidden somewhere in the JavaScript <--> server exchange)?
>
> > > Thank you,
>
> > > Jeff
>
> > > On Apr 14, 11:48 am, BenNevile wrote:
>
> > > > bump. I have observed this many times over the past few days. Maybe
> > > > has to do with transactions and/or write contention, since there is a
> > > > lot of that in my app.
>
> > > > Ben
>
> > > > On Apr 1, 8:23 pm, BenNevile wrote:
>
> > > > > Hi GAE team and devotees,
>
> > > > > I have a entity that looks like
>
> > > > > class FacebookUser(db.Model):
> > > > > expired_fotm = db.BooleanProperty()
> > > > > last_nudged_at = db.DateTimeProperty()
>
> > > > > and an index that looks like
>
> > > > > - kind: FacebookUser
> > > > > properties:
> > > > > - name: expired_fotm
> > > > > - name: last_nudged_at
>
> > > > > Unbeknownst to the users of my application, some javascript is
> > > > > constantly polling a worker handler that finds the first entity in the
> > > > > index, operates on it, then sets "last_nudged_at" to the current
> > > > > time. There are more than a million of these entities so it takes
> > > > > some time to go through them all.
>
> > > > > This afternoon using the Data Viewer and the GQL query "SELECT * FROM
> > > > > FacebookUser where expired_fotm=True order by last_nudged_at" I
> > > > > observed several times that an entity would remain "stuck" in the
> > > > > first position in the index after it was processed. Usually it would
> > > > > remain there only for 10-30 seconds. However this evening one entity
> > > > > stayed in an incorrect index position for several hours. I put() it
> > > > > again and it properly indexed itself.
>
> > > > > Has anyone ever observed anything like this? Let me know if I'm not
> > > > > making myself clear.
>
> > > > > Ben <3 GAE
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---
[google-appengine] Re: entity stuck at the wrong end of the index
To add another twist, now I have an entity at the top of the stack
that is found using
q = FacebookUser.all()
q.filter('authorized = ',True)
q.order('last_nudged_at')
u = q.get()
but, if you try to load this entity by key, as you need to do for a
transaction, you get None back. Trying to access that entity's
record in the Data Viewer also threw an exception. I was, however,
able to delete it from the top of that index listing using the Data
Viewer's GQL option.
Ben
On Apr 14, 3:38 pm, Ben Nevile wrote:
> Hi Jeff,
>
> To clarify, the last_nudged_at property *is* changed, but the entity
> stays at the top of the stack.
>
> Ben
>
> On Apr 14, 12:26 pm, Jeff S wrote:
>
>
>
> > Hi Ben,
>
> > If you are experiencing write contention on some of these updates,
> > then that may explain why the last_nudged_at property is not changed
> > and an entity tends to stay at the top of the stack. Do you see
> > datastore timeout errors in these cases (I imagine the exception may
> > be hidden somewhere in the JavaScript <--> server exchange)?
>
> > Thank you,
>
> > Jeff
>
> > On Apr 14, 11:48 am, Ben Nevile wrote:
>
> > > bump. I have observed this many times over the past few days. Maybe
> > > has to do with transactions and/or write contention, since there is a
> > > lot of that in my app.
>
> > > Ben
>
> > > On Apr 1, 8:23 pm, Ben Nevile wrote:
>
> > > > Hi GAE team and devotees,
>
> > > > I have a entity that looks like
>
> > > > class FacebookUser(db.Model):
> > > > expired_fotm = db.BooleanProperty()
> > > > last_nudged_at = db.DateTimeProperty()
>
> > > > and an index that looks like
>
> > > > - kind: FacebookUser
> > > > properties:
> > > > - name: expired_fotm
> > > > - name: last_nudged_at
>
> > > > Unbeknownst to the users of my application, some javascript is
> > > > constantly polling a worker handler that finds the first entity in the
> > > > index, operates on it, then sets "last_nudged_at" to the current
> > > > time. There are more than a million of these entities so it takes
> > > > some time to go through them all.
>
> > > > This afternoon using the Data Viewer and the GQL query "SELECT * FROM
> > > > FacebookUser where expired_fotm=True order by last_nudged_at" I
> > > > observed several times that an entity would remain "stuck" in the
> > > > first position in the index after it was processed. Usually it would
> > > > remain there only for 10-30 seconds. However this evening one entity
> > > > stayed in an incorrect index position for several hours. I put() it
> > > > again and it properly indexed itself.
>
> > > > Has anyone ever observed anything like this? Let me know if I'm not
> > > > making myself clear.
>
> > > > Ben <3 GAE
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---
[google-appengine] Re: entity stuck at the wrong end of the index
Hi Jeff, To clarify, the last_nudged_at property *is* changed, but the entity stays at the top of the stack. Ben On Apr 14, 12:26 pm, Jeff S wrote: > Hi Ben, > > If you are experiencing write contention on some of these updates, > then that may explain why the last_nudged_at property is not changed > and an entity tends to stay at the top of the stack. Do you see > datastore timeout errors in these cases (I imagine the exception may > be hidden somewhere in the JavaScript <--> server exchange)? > > Thank you, > > Jeff > > On Apr 14, 11:48 am, Ben Nevile wrote: > > > > > bump. I have observed this many times over the past few days. Maybe > > has to do with transactions and/or write contention, since there is a > > lot of that in my app. > > > Ben > > > On Apr 1, 8:23 pm, Ben Nevile wrote: > > > > Hi GAE team and devotees, > > > > I have a entity that looks like > > > > class FacebookUser(db.Model): > > > expired_fotm = db.BooleanProperty() > > > last_nudged_at = db.DateTimeProperty() > > > > and an index that looks like > > > > - kind: FacebookUser > > > properties: > > > - name: expired_fotm > > > - name: last_nudged_at > > > > Unbeknownst to the users of my application, some javascript is > > > constantly polling a worker handler that finds the first entity in the > > > index, operates on it, then sets "last_nudged_at" to the current > > > time. There are more than a million of these entities so it takes > > > some time to go through them all. > > > > This afternoon using the Data Viewer and the GQL query "SELECT * FROM > > > FacebookUser where expired_fotm=True order by last_nudged_at" I > > > observed several times that an entity would remain "stuck" in the > > > first position in the index after it was processed. Usually it would > > > remain there only for 10-30 seconds. However this evening one entity > > > stayed in an incorrect index position for several hours. I put() it > > > again and it properly indexed itself. > > > > Has anyone ever observed anything like this? Let me know if I'm not > > > making myself clear. > > > > Ben <3 GAE --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: entity stuck at the wrong end of the index
Hi Ben, If you are experiencing write contention on some of these updates, then that may explain why the last_nudged_at property is not changed and an entity tends to stay at the top of the stack. Do you see datastore timeout errors in these cases (I imagine the exception may be hidden somewhere in the JavaScript <--> server exchange)? Thank you, Jeff On Apr 14, 11:48 am, Ben Nevile wrote: > bump. I have observed this many times over the past few days. Maybe > has to do with transactions and/or write contention, since there is a > lot of that in my app. > > Ben > > On Apr 1, 8:23 pm, Ben Nevile wrote: > > > Hi GAE team and devotees, > > > I have a entity that looks like > > > class FacebookUser(db.Model): > > expired_fotm = db.BooleanProperty() > > last_nudged_at = db.DateTimeProperty() > > > and an index that looks like > > > - kind: FacebookUser > > properties: > > - name: expired_fotm > > - name: last_nudged_at > > > Unbeknownst to the users of my application, some javascript is > > constantly polling a worker handler that finds the first entity in the > > index, operates on it, then sets "last_nudged_at" to the current > > time. There are more than a million of these entities so it takes > > some time to go through them all. > > > This afternoon using the Data Viewer and the GQL query "SELECT * FROM > > FacebookUser where expired_fotm=True order by last_nudged_at" I > > observed several times that an entity would remain "stuck" in the > > first position in the index after it was processed. Usually it would > > remain there only for 10-30 seconds. However this evening one entity > > stayed in an incorrect index position for several hours. I put() it > > again and it properly indexed itself. > > > Has anyone ever observed anything like this? Let me know if I'm not > > making myself clear. > > > Ben <3 GAE --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: entity stuck at the wrong end of the index
bump. I have observed this many times over the past few days. Maybe has to do with transactions and/or write contention, since there is a lot of that in my app. Ben On Apr 1, 8:23 pm, Ben Nevile wrote: > Hi GAE team and devotees, > > I have a entity that looks like > > class FacebookUser(db.Model): > expired_fotm = db.BooleanProperty() > last_nudged_at = db.DateTimeProperty() > > and an index that looks like > > - kind: FacebookUser > properties: > - name: expired_fotm > - name: last_nudged_at > > Unbeknownst to the users of my application, some javascript is > constantly polling a worker handler that finds the first entity in the > index, operates on it, then sets "last_nudged_at" to the current > time. There are more than a million of these entities so it takes > some time to go through them all. > > This afternoon using the Data Viewer and the GQL query "SELECT * FROM > FacebookUser where expired_fotm=True order by last_nudged_at" I > observed several times that an entity would remain "stuck" in the > first position in the index after it was processed. Usually it would > remain there only for 10-30 seconds. However this evening one entity > stayed in an incorrect index position for several hours. I put() it > again and it properly indexed itself. > > Has anyone ever observed anything like this? Let me know if I'm not > making myself clear. > > Ben <3 GAE --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
