It may be related to you mixing usage of their filter method from the page you first linked to (the apps[-1] method)... and GQLQueries.
Try the method from this page: http://code.google.com/appengine/docs/python/datastore/queriesandindexes.html They are casting the last key grabbed to a string before using it to grab the next GQL batch.. like so: if len(entities) == 21: new_last_key_str = str(entities[19].key()) Or, use the filtering method they use on the remote api page you link to. Mixing them together may be what is causing your problem. On Sat, Dec 12, 2009 at 6:18 PM, bsb <b...@pearcomp.com> wrote: > I dug a bit deeper, as nobody seems to have a clue either. I printed > all keynames that I retrieved from the datastore in the order they > were retrieved. I repeated this 3 times, each time calling the exact > same code. What turns out to happen is that occasionally, the GqlQuery > returns the keys in the wrong order! Suddenly, I get the LAST key in > the table and all following keys in DESCENDING order. When the > GqlQuery randomly flips back to normal, it then goes in ASCENDING > order but from almost at the end of the table. > > The peculiar thing is that this seems to happen at a particular > position in the table each time. The start of the key reversing seems > random, but the last 500 are always the same. I'm wondering if the > index is broken and that causes the problem, cause I seriously can't > think of any other explanation. > > Just to say, I have a custom descending index on the key_name for this > class, if that makes any difference. > > Can someone from Google please look into this? > > Thanks, > Ben > > On Dec 11, 2:04 pm, bsb <b...@pearcomp.com> wrote: > > I doubt that, I did get some explixcit timeouts from the fetch, which > > went away when I reduced the limit from 500 to 300. So, I think if it > > was a timeout, I'd get an exception thrown, no? > > > > Best, > > ben > > > > On Dec 10, 2:48 pm, Eli Jones <eli.jo...@gmail.com> wrote: > > > > > > > > > Maybe one of the fetches times out and it just silently fails.. And > > > prints out its current count? > > > > > On 12/10/09, bsb <b...@pearcomp.com> wrote: > > > > > > No ideas? Noone? I just don't understand why not all keys are > > > > retrieved every time. It doesn't make any sense... > > > > > > Thanks in advance, > > > > Ben > > > > > > On Dec 8, 6:47 pm, bsb <b...@pearcomp.com> wrote: > > > >> I am trying to cycle through all elements of a table via the remote > > > >> API, using a modification of the code snippet > > > >> fromhttp://code.google.com/appengine/articles/remote_api.html: > > > > > >> #---------------------------------------------------- SNIP > > > >> ---------------------------------------------------- > > > >> query = db.GqlQuery("SELECT __key__ FROM Model_mobileapp ORDER BY > > > >> __key__") > > > >> apps = query.fetch(500) > > > >> total = 0 > > > >> while apps: > > > >> total = total + len(apps) > > > >> query2 = db.GqlQuery("SELECT __key__ FROM Model_mobileapp WHERE > > > >> __key__ > :1 ORDER BY __key__", apps[-1]) > > > >> apps = query2.fetch(500) > > > > > >> print "%d" % total > > > >> #---------------------------------------------------- SNAP > > > >> ---------------------------------------------------- > > > > > >> However, I'm getting unpredictable behaviour. Sometimes I get the > > > >> correct number (104512), sometimes I get 15999, 30999 and all sorts > of > > > >> other values. > > > > > >> Does anyone have any clue what this could be caused by? Is there > some > > > >> sort of magic caching going on for the fetch() command, that return > > > >> some keys which are not in the correct sorting order? Could the > > > >> (automatic) ascending key index be broken? Any ideas? > > > > > >> Thanks, > > > >> Ben > > > > > > -- > > > > > > You received this message because you are subscribed to the Google > Groups > > > > "Google App Engine" group. > > > > To post to this group, send email to > google-appeng...@googlegroups.com. > > > > To unsubscribe from this group, send email to > > > > google-appengine+unsubscr...@googlegroups.com<google-appengine%2bunsubscr...@googlegroups.com> > . > > > > For more options, visit this group at > > > >http://groups.google.com/group/google-appengine?hl=en. > > > > > -- > > > Sent from my mobile device > > -- > > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to google-appeng...@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com<google-appengine%2bunsubscr...@googlegroups.com> > . > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > > > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
