Hi all, Thank You for the response.
@Incure, No I was not able to find out why/where the memory leak was happening. I was unable to configure Apptrace for my application, it's missing some dependencies, did not get a chance to figure out what! I am guessing the issues might have been caused due to the 'string' list properties in my model (although the number of items in the string list were low) I have had to refactor the page to get the data it needed differently, but this solution is not scalable, I am still searching for a solution.. @Robert, I am using Python. It truly is hard to find memory leaks in Python, I tried using Apptrace but was not able to get it to work. Functionally there is nothing much that my code does, it fetches all items of a particular Model instance (around 500 records) serlializes it into JSON and sends it back, the rest is done on the client. http://code.google.com/p/apptrace/wiki/UsingApptrace Thanks, Arun. On Feb 3, 11:18 am, Robert Kluin <robert.kl...@gmail.com> wrote: > Hi Arun, > You don't say if you're using Python or Java. For Python debugging > memory usage is hard at best; I'm not sure about Java. > > In Python at least, every entity you load actually stores multiple > copies of the data and it is decoded / encoded from protobufs many > times. So the end result is that your memory usage will be higher > than what you expect. Possibly by quite a bit depending on your data. > > It is also possible that you've got other issues in your code > leading to memory leaks. That's harder to make general comments about > though. > > Robert > > On Tue, Dec 27, 2011 at 11:58, Arun Shanker Prasad > > > > > > > > <arunshankerpra...@gmail.com> wrote: > > Hi All, > > > Hope you had a wonderful holiday :) > > > I am experiencing an issue for the past couple of weeks, am getting > > soft memory exceeded errors and the instance is being terminated.. > > This only started occuring recently. > > > The pages experiencing this error are mostly reporting kind of pages, > > i.e pages that deal with a number of records that are fetched based on > > filters selected by the user and use django.core.paginator.Paginator > > to paginate over the result. I have a maximum of 700 records that are > > fetched for these reports, according to datastore statistics in the > > admin console the average size of the entities being fetched are 4KB. > > > Average Size of Entity: > > 4 KBytes > > > Some of the properties are reference properties and I do use prefetch > > logic to load these references as well, but most of them are also > > small entities. > > > By my caclulations 158MB should be enough to handle a list of ~40448 > > records! I am not able to figure out why the request is using this > > much memory. > > > I have tried eliminating most of the usual suspects (large string > > concatination and so on), even tried deleting the large list after > > they are processed and calling garbage collect, nothing seems to work. > > Next step is to try and setup Apptrace for app engine. > > > Can anyone out there help me with this? Anyone else have any similar > > issues? How did you manage to sort it out? > > > C 2011-12-27 10:03:54.089 > > Exceeded soft private memory limit with 157.648 MB after servicing 4 > > requests total > > W 2011-12-27 10:03:54.090 > > While handling this request, the process that handled this request was > > found to be using too much memory and was terminated. This is likely > > to cause a new process to be used for the next request to your > > application. If you see this message frequently, you may have a memory > > leak in your application. > > > -- > > You received this message because you are subscribed to the Google Groups > > "Google App Engine" group. > > To post to this group, send email to google-appengine@googlegroups.com. > > To unsubscribe from this group, send email to > > google-appengine+unsubscr...@googlegroups.com. > > For more options, visit this group > > athttp://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
