On Wed, Sep 11, 2013 at 7:04 AM, Ben Smithers <smithers....@googlemail.com> wrote:
> I ran this three times (10 instances each time) - in all cases the file is > written with the single character successfully, but a number of the larger > writes then hang. > > So what this looks to me is that GCS and the library are working fine, but the backend (and more importantly, the urfetch connection to the backend) are having difficulty piping down the data into GCS servers. So to speak of this problem in a hypothetical sense, at some point there is an "inflection point" - there is a certain size of data writing that the urlfetch is having difficulty with. You were able to write a single character without a problem, but 1 MB of characters was too much for it. So I were you, I'd try to find that inflection point and then save to Cloud Storage in chunks less than that. For example, try to write to Cloud Storage in chunks of 250 KB, if that works, try writing in chunks of 400 KB, and so on. Of course this is only a bandaid solution - something is clearly wrong with the underlying urlfetch implementation. But if I had to make a recommendation, I'd say skip the whole backend writing thing altogether. It's a waste of money to load up backends for the purposes of writing (unless you're Bill Gates or similar!). You said frontend instances seem to be much better in terms of writing - try using task queues to spawn off frontend requests which are solely dedicated to writing to Cloud Storage. If you need a temporary place to store data, write it to the datastore or Cloud SQL until the tasks can get to the GCS writing. ----------------- -Vinny P Technology & Media Advisor Chicago, IL App Engine Code Samples: http://www.learntogoogleit.com -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine. For more options, visit https://groups.google.com/groups/opt_out.
