I have an app that once a day does a big data processing task. Every now and then it would throw a datastore timeout error. But now it's throwing them constantly.
I thought maybe my data had tripped over some limit on how much you can read, but I just added some instrumentation and it's only reading less than half of the entities. If I was tripping over an undocumented limit, I'd think it would read all most all of them (since only a few get added each day). Basically, the code is simply: for h in HitModel.all(): (do collect up info about h) and there are about 85K HitModel objects in the database. It's dying after reading 35,000 of them (which takes about a minute). It's on HR data store. Still on Python 2.5. App ID is "kaon-log" The error I'm getting is: 2014-10-07 11:16:46.925 The datastore operation timed out, or the data was temporarily unavailable. Traceback (most recent call last): File "/base/data/home/runtimes/python/python_lib/versions/1/google/appengine/ext/webapp/_webapp25.py", line 714, in __call__ handler.get(*groups) File "/base/data/home/apps/s~kaon-log/33.379217403803985923/main.py", line 648, in get for h in HitModel.all(): File "/base/data/home/runtimes/python/python_lib/versions/1/google/appengine/ext/db/__init__.py", line 2326, in next return self.__model_class.from_entity(self.__iterator.next()) File "/base/data/home/runtimes/python/python_lib/versions/1/google/appengine/datastore/datastore_query.py", line 3091, in next next_batch = self.__batcher.next_batch(Batcher.AT_LEAST_OFFSET) File "/base/data/home/runtimes/python/python_lib/versions/1/google/appengine/datastore/datastore_query.py", line 2977, in next_batch batch = self.__next_batch.get_result() File "/base/data/home/runtimes/python/python_lib/versions/1/google/appengine/api/apiproxy_stub_map.py", line 612, in get_result return self.__get_result_hook(self) File "/base/data/home/runtimes/python/python_lib/versions/1/google/appengine/datastore/datastore_query.py", line 2710, in __query_result_hook self._batch_shared.conn.check_rpc_success(rpc) File "/base/data/home/runtimes/python/python_lib/versions/1/google/appengine/datastore/datastore_rpc.py", line 1333, in check_rpc_success raise _ToDatastoreError(err) Timeout: The datastore operation timed out, or the data was temporarily unavailable. Any ideas? (Breaking this up into multiple tasks would be really hard.) -Joshua -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine. For more options, visit https://groups.google.com/d/optout.