Ah ok. So it sounds like if I drop "print(gc.garbage)" into my code at the
right points, I will be able to see where the leak is occurring? I did that and
then load tested it (not via anything tricky, I'm just using "ab") via the dev
server running locally, but I'm not seeing anything, it's always [] even for
n=1000 or 10000. (100k is running right now but we were seeing this problem
below roughly 10k)
Which all makes me think it could be a problem on heroku's end. I'm going to
try emailing them and testing this in production and then devising ways to test
each component we're using (mainly pymongo and BigML), but any other thoughts
on how I could track this sucker down? I'd love to have complete confidence
that it's not leaking running locally at least.
Thanks!
Dan
On Nov 23, 2012, at 12:30 PM, akaariai wrote:
>
>
> On 22 marras, 08:27, Dan Ancona <afightingfa...@gmail.com> wrote:
>> python 2.7.3, django 1.4.2
>>
>> Our app (a fairly simple tastypie API that runs some machine learning foo on
>> mongodb records) is deployed to heroku and is leaking memory at a pretty
>> good clip. Hitting heroku memory errors after about an hour or two, under
>> significant load.
>>
>> Wandered through some various profiling options and eventually started to
>> get suspicious of tastypie. So I tried ripping that out. And then everything
>> else, until I finally got down to a django view that has nothing but this in
>> it:
>>
>> def stack(request):
>> gc.set_debug(gc.DEBUG_LEAK)
>> print "Uncollectable garbage", gc.garbage
>> return HttpResponse('hi')
>
> If you set gc.DEBUG_LEAK then the GC will not free anything, instead
> it will append garbage to gc.garbage. So, getting everything into
> gc.garbage is expected in that case.
>
> You will want to remove the gc.set_debug() call.
>
> Here is an example without the call:
>
> import gc
> from django.http import HttpResponse
>
> # Objects which have __del__ defined and are part of reference cycle
> will be uncollectable.
> class A(object):
> def __del__(self):
> pass
> class B(object):
> pass
>
> def stack(request):
> a = A()
> b = B()
> # Lets create a cycle
> a.b = b
> b.a = a
> print(gc.garbage)
> return HttpResponse('hi')
>
>
> Load the view a couple of times and you will see garbage accumulating.
> It will contain only the A objects - they are uncollectable because of
> the __del__ method.
>
> - Anssi
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django users" group.
> To post to this group, send email to django-users@googlegroups.com.
> To unsubscribe from this group, send email to
> django-users+unsubscr...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/django-users?hl=en.
>
--
You received this message because you are subscribed to the Google Groups
"Django users" group.
To post to this group, send email to django-users@googlegroups.com.
To unsubscribe from this group, send email to
django-users+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/django-users?hl=en.
