Could these be related to datastore latency spikes? Those have been known to occur regularly on master/slave datastore, and we are encouraging developers to move to high replication. You can sign up for migration tool access here:
https://docs.google.com/a/google.com/spreadsheet/viewform?authkey=CLXR0LMN&formkey=dERMcDZuMnlycHoyZDd4Vy1PNXlhWlE6MQ#gid=0 -- Ikai Lan Developer Programs Engineer, Google App Engine plus.ikailan.com | twitter.com/ikai On Tue, Sep 27, 2011 at 4:39 AM, Bernd F <bernd.fi...@gmail.com> wrote: > Tonight it happened again, but after ~1 hour the app recovered on its > own. There must be something wrong with app engine. > > > > On 20 Sep., 00:08, Bernd Final <bernd.fi...@gmail.com> wrote: > > I've had this issue for 5 or 6 times already. My app is running for a > > long time and I've had no problems so far, except this one: > > > > Out of nowhere, 20-30 seconds latency and only serving HTTP 500 errors > > for 99,9% of all requests. In most Servlets (yeah I'm using Java if > > that matters) I'm accessing the memcache, sometimes the datastore > > (Master/Slave). As you can imagine the app is totally unusable when > > this happens. When it occured the first time I didn't to any special, > > just waited for a couple of hours until the app recovered on it's own. > > > > Dashboard Milliseconds/Request Chart: > > Yesterday:http://bit.ly/pSF66E > > Last 30 Days:http://bit.ly/nYdZ2S > > > > When it happend again and again I tried to figure out what's going on, > > luckily I had some kind of Admin-Servlet deployed which allowed me to > > clear the entire memcache. Tried it, and voila, everything back to > > normal. > > > > - The memcache size was up to 1.3-1.5 MB in total (I did a rough > > estimation with the ByteArrayOutputStream trick). 12 Objects are > > stored in the memcache. Each Object is POJO which implements > > Serializable and contains some Strings, Integers and ArrayLists. > > - My first idea was to minimize the memory usage, because I though I > > was hitting the 1MB limit. At this time I didn't know the limit is for > > each object not for the whole cache, anyway I was able to reduce the > > size by 30-40% so the size is now about 800kb - 1MB > > - I'm running a cron job which cleans out garbage of the memcache. > > - This is how I access the memcache on every request: > > > CacheManager.getInstance().getCacheFactory().createCache(Collections.emptyMap()); > > - I'm using put and get Methods for storing / retrieving, > > - The key length is about 160 bytes > > > > Unfortunately I was not able to solve this issue yet, I would be very > > happy for ANY input. > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to google-appengine@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
