Hi Ben, Thanks for the detailed post. What does the application log for those hanged request look like? Do they all only have DeadlineExceededException? What's the stacktrace?
I don't see anything you are doing wrong. Just make sure you are using a relatively recent release of the library. On Tuesday, September 10, 2013 6:14:58 AM UTC-4, Ben Smithers wrote: > > Hi, > > I've been having some problems writing data to google cloud storage using > the python client library ( > https://developers.google.com/appengine/docs/python/googlecloudstorageclient/ > ). > > Frequently, the write to cloudstorage will 'hang' indefinitely. The call > to open a new file is successful, but the write (1MB in size in this test > case) never returns and the file never appears in the bucket. For example, > I launch 10 instances of a (backend) module, each of which attempts to > write a file. Typically, somewhere between 4 and 9 of these will succeed, > with the others hanging after opening. This is the code I am running: > > class StartHandler(webapp2.RequestHandler): > > GCS_BUCKET="/" > > def debugMessage(self, msg): > logging.debug(msg) > logservice.flush() > > def get(self): > suffix = str(backends.get_instance()) > filename=self.GCS_BUCKET + "/testwrite" + suffix + ".txt" > gcs_file = cloudstorage.open(filename, 'w', content_type= > 'text/plain' ) > self.debugMessage("opened file") > gcs_file.write("f" * 1024 * 1024 * 1 + '\n') > self.debugMessage("data written") > gcs_file.close() > self.debugMessage("file closed") > > > I have also attached a tarred example of the full application in case it > is relevant (to run, you should only need to modify the application name in > the two .yaml files and the bucket name in storagetest.py). A few > additional things: > > 1.) I wondered if this was a problem with simultaneous writes so I had > each instance sleep for 30 seconds * its instance number; I observe the > same behaviour. > 2.) I have seen this behaviour on frontend instances, but far far more > rarely. I modified the above to run in response to a user request - once > out of 60 times the write hung after opening (a Deadline Exceeded Exception > was then thrown). > 3.) I have experimented with the RetryParams (though according to the > documentation, the defaults should be sufficient) but to no avail. I also > find it hard to believe this is the issue - I would assume I would be > getting a TimeoutError. > > Has anyone else observed this behaviour? Does anyone have any suggestions > for what I am doing wrong? Or a different approach to try? > > Very grateful for any help, > Ben > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine. For more options, visit https://groups.google.com/groups/opt_out.
