I think there must be a problem with RPCs from backends. Given my continued attempts to debug this problem and Vinny's suggestion, I setup another small test case, this time writing to the datastore. I observe the exact same behaviour: a large percentage of writes to datastore hang (seemingly) indefinitely. I was writing 750KBs of data. Test code:
import webapp2 from google.appengine.ext import db from google.appengine.api.logservice import logservice import logging class TestEntity(db.Model): data = db.TextProperty() class StartHandler(webapp2.RequestHandler): def debugMessage(self, msg): logging.debug(msg) logservice.flush() def get(self): data = "f" * 1024 * 750 entity = TestEntity(data=data) self.debugMessage("Entity created") entity.put() self.debugMessage("Entity saved") app = webapp2.WSGIApplication([ ('/_ah/start', StartHandler), ], debug=True) I ran this with 10 instances, 3 times. The number of successful writes was 7,3 and 7 for each time I tested. The unsuccessful writes hang after "Entity created" appears in the log but before "Entity saved" is logged (and these entities do not appear in the datastore). Additional info: - The max time I waited for this was ~15 minutes; I observed hanging of ~1 hour when I was writing/debugging "real" code though. - I have tried with both db and ndb. Same behaviour - Things look ok running on frontends. 60/60 successful writes (using ndb in this case, didn't test with the older API). Unless I'm missing something, this seems to be a real problem with backend instances. Thanks for any suggestions, Ben -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine. For more options, visit https://groups.google.com/groups/opt_out.