Hi,
On Wed, Sep 12, 2012 at 02:49:13PM +0200, Arndt Droullier wrote:
> I am trying to get to the bottom of some leaking pyramid application.
>
> Now, what I found out is the following simple example where the
> request never get cleaned up:
>
> -------------------------------------------------------------------------------------------------------------------
> from pyramid.config import Configurator
>
> from pyramid.httpexceptions import HTTPFound
> from pyramid.response import Response
> import weakref
>
>
> class Class1(object):
> def __init__(self):
> self.alot="------------------ ".join([str(a) for a in
> range(0,10000)])
>
> def __del__(self):
> print "del object"
>
>
> class View(object):
> def __init__(self, context, request):
> self.request = request
> self.context = context
>
> def __del__(self):
> print "del view"
>
> def test1(self):
> url = self.request.url
> return Response(body='hello world!', content_type='text/plain')
>
>
> class ViewWeakref(object):
> def __init__(self, context, request):
> self._request = weakref.ref(request)
> self.context = context
>
> @property
> def request(self):
> return self._request()
>
> def __del__(self):
> print "del view"
>
> def test1(self):
> url = self.request.url
> return Response(body='hello world!', content_type='text/plain')
>
>
> class Root(object):
> def __getitem__(self, name):
> return Class1()
>
>
> def getRoot(request):
> return Root()
>
>
> def main(global_config, **settings):
> """ This function returns a Pyramid WSGI application.
> """
> config = Configurator(root_factory=getRoot, settings=settings)
> config.add_view(view=View, name="test1", attr="test1")
> return config.make_wsgi_app()
>
> -------------------------------------------------------------------------------------------------------------------
>
> The application traverses the root object ("Root"), generates a
> context ("Class1") and calls a view ("View.test1") for the context.
> So the url to call this was for example "http://127.0.0.1:6543/class/test1";
>
> What happens is __del__ functions are never called and and the
> application consumes more and more memory.How significant is the memory growth? Some actual numbers would help. Does calling gc.collect() help? Does removing the __del__ methods and then calling gc.collect() help? Do you see a constant increase in len(gc.get_objects())? It could be just memory fragmentation. > I think the important part is that the view is a "class" and it stores > the request as class attribute. And in fact with a few modifications > to store the request as a weakref attribute (class ViewWeakref) the > application works allright and everything gets cleaned up. I see no class attributes in your code, only instance attributes. The fact that using a weakref helps is a hint that there's normally a reference cycle, which means reference counting isn't going to free the objects normally, and you have to wait for the next scheduled garbage collection to happen (or call gc.collect() explicitly) for objects to be freed. Note that the amount of memory used by the Python process is unlikely to go down, due to memory fragmentation. Also note that Python cannot free objects participating in a reference cycle if they have __del__ methods on objects. > Now, I don't think this happens every time a view class is involved. > My original application for example stores the request as view class > atrribute (without using weakref) and works allright. But I suppose > there are a few rare cases depending on how views are invoked by > pyramid making it easy to run into leaking applications. I wrote http://pypi.python.org/pypi/objgraph when I was debugging a memory leak in an app's test suite. It may help you narrow down the leak, if there is one. Marius Gedminas -- To express oneself In seventeen syllables Is very diffic -- John Cooper Clark.
signature.asc
Description: Digital signature
