Re: [google-appengine] Hanging on last Backend (or Module) memcache writes

2013-08-25 Thread Julian Kent
Ah, it seems you aren't affiliated with Google. I'll see about filing a bug report or something myself then. Anyway, thanks for the help =) Julian On Sunday, 25 August 2013 13:33:47 UTC+2, Julian Kent wrote: > > Thanks Vinny > > I've put a delay of 5 seconds in, and reduced the number of items

Re: [google-appengine] Hanging on last Backend (or Module) memcache writes

2013-08-25 Thread Julian Kent
Thanks Vinny I've put a delay of 5 seconds in, and reduced the number of items in each set_multi_async from 32 to 20. However, this is still a new issue, since I didn't have any problems like this until 1.8.x, even without any delay and dumping as fast as I could. Could you notify the dev guy

Re: [google-appengine] Hanging on last Backend (or Module) memcache writes

2013-08-24 Thread Vinny P
On Sat, Aug 24, 2013 at 9:50 AM, Julian Kent wrote: > In the appstats, further down I'm seeing memcache.Set() entries with times > such as "real=8831941ms", which is clearly where it is hanging. Personally, > I don't consider 8000 seconds, that is over 2 hours, reasonable latency in > a single me

Re: [google-appengine] Hanging on last Backend (or Module) memcache writes

2013-08-24 Thread Julian Kent
OK, I managed to catch one. I saved the appstats page, as well as the logs, here: https://dl.dropboxusercontent.com/u/4066735/LiteRumble_error/LiteRumble_error.zip If you want to step through code the latest is available here: https://bitbucket.org/jkflying/literumble/src/3542f1e636fe8675cf6e5b701

Re: [google-appengine] Hanging on last Backend (or Module) memcache writes

2013-08-23 Thread Vinny P
On Fri, Aug 23, 2013 at 1:36 AM, Julian Kent wrote: > Will do. Right now it is tough to catch these though, they only happen > ~20% of the time. It always seems to happen while I'm asleep :-/ Is there > some way of getting appstats to save to db so I can see it later? > 20% occurrence rate is m

Re: [google-appengine] Hanging on last Backend (or Module) memcache writes

2013-08-22 Thread Julian Kent
On Thursday, 8 August 2013 00:23:30 UTC+2, Vinny P wrote: > What does the variable *botsdict* represent? Based on the naming and > function calls, I assume that it's a dictionary, but what type of objects > is it containing? Is it possible that the contained objects are doing some > kind of p

Re: [google-appengine] Hanging on last Backend (or Module) memcache writes

2013-08-07 Thread Vinny P
On Tue, Aug 6, 2013 at 3:29 AM, Julian Kent wrote: > Here is the relevant code between those two logging points: > > client = memcache.Client() > if len(botsdict) > 0: > splitlist = dict_split(botsdict,32) > What does the variable *botsdict* re

[google-appengine] Hanging on last Backend (or Module) memcache writes

2013-08-06 Thread Julian Kent
I have a batch processing Backend (B4) which does a bunch of unpickle/pickle and Numpy/array stuff. Recently I noticed that I was getting much higher backend charges, which would hit quota almost every time, so I migrated to Modules (also B4), thinking that might solve it. However, I still see t