On Thursday, 8 August 2013 00:23:30 UTC+2, Vinny P wrote:
> What does the variable *botsdict* represent? Based on the naming and > function calls, I assume that it's a dictionary, but what type of objects > is it containing? Is it possible that the contained objects are doing some > kind of processing on their own? > It is a dictionary of zipped, pickled objects, with the key being the memcache key I'm using. The split_dict just breaks the dictionary into a list of dictionaries each of which are small enough to use with a single memcache.set_multi(), which has a 30MB limit. > > Install appstats ( > https://developers.google.com/appengine/docs/python/tools/appstats ) into > your application, you'll be able to see how the memcache calls execute and > at what time. From there we can narrow down the list of problems. > Will do. Right now it is tough to catch these though, they only happen ~20% of the time. It always seems to happen while I'm asleep :-/ Is there some way of getting appstats to save to db so I can see it later? How much memory in memcache are you using? If you're only using ~200 MB, it > might be easier to simply store it within the backend's memory and > not memcache (if you're hungry for memory you can use the B4_1G backend > size, which gives you 1GB of memory and the same processor size as a > regular B4). > This is just a batch processing job for some ranking mechanisms that requires all my scores to be loaded into memory at once, most of my work is done in frontends. Once the batch job is done I dump all the changes I've made to memcache to overwrite any values the frontends might be using. The queue for frontend processing is paused while I'm doing this so I'm not concerned with locking etc. Thanks for the help, and sorry I was slow to respond. Somehow my email updates were turned off! -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine. For more options, visit https://groups.google.com/groups/opt_out.
