On Wed, Jun 3, 2009 at 1:22 AM, proteus...@gmail.com
wrote:
>
> We've got an application that is running behind apache, mod_wsgi and
> front ending a postgres 8.3 db. We needed to upgrade our Satchmo to
> the latest version which requires Django 1.1. After the upgrade the
> system starts eventually running out of memory until it thrashes and
> locks up the machine under a fairly light load, just a few reqs per
> second. I can understand the system using more memory but it doesn't
> seem to release it when the load stops.
>
> This graph: http://www.flickr.com/photos/proteusguy/3589788760 shows
> the results of both systems which, besides the Django & Satchmo
> versions are identical. You can see that when the test completes on
> the old system the process returns a great amount of memory but the
> new one never releases the memory. In production this will eventually
> grow to consume all RAM. This particular test is of a set of 7
> different requests simulating what a typical user might do.
>
> This graph: http://www.flickr.com/photos/proteusguy/3589788752 shows a
> similar test except it only contains a single request out of the seven
> from the prior test. We were hoping to identify that something in one
> of our requests was the cause of the problem but similar exclusive
> test runs with all 7 requests from the combined test from above show
> similar results - fairly stable memory usage.
>
> This leads me to believe that the issue is systemic to django, most
> likely in the db handling as most of the requests don't touch the
> Satchmo system. If this was a C++ app I'd be able to figure this out
> pretty quick but I'm not sure how to attack the problem in a dynamic
> language environment like python. So this leaves me with two
> questions:
>
> 1. Is anyone else seeing this kind of memory extra memory usage and an
> inability to return memory once requests end?
I'm not seeing anything like this on my deployed sites, running a
fairly recent (post-beta-1) checkout of Django.
> 2. What suggestions does anyone have about how I can determine what
> aspect of the system is using this extra memory and holding onto it?
The best advice I can point you at is Ned Batchelder's blog, when he
diagnosed a memory leak he found when moving from v0.96 to v1.0:
http://nedbatchelder.com/blog/200809/a_server_memory_leak.html
As well as describing a general approach, Ned mentions a few tools,
and the comments mention a few more.
However, I would say that my experience has been that Python code is
very rarely the cause of memory leaks such as these. You have to try
pretty hard to leak in a garbage collected language. You can do it,
but the causes are generally obvious - usually things like code that
has lots of intensive looping causing circular references in data
structures, or global data structures that aren't cleaned up. In
Django, for example, a common source of a "memory leak" is the global
query log that is maintained when DEBUG=True on the server. However,
this isn't really a "leak" in the classical sense - the server is just
keeping hold of everything it was told to store. If you tell Django
not to store every query, then it wont, and you won't "leak" memory.
A more likely cause is any C extension module that you are interfacing
with. Any Django stack includes at least one C module - the database
adaptor. Even if you haven't changed the database adaptor, this is
still an area worth investigating - it's possible that in the
transition from Django 1.0 to 1.1, the database adaptor is being
exercised in a slightly different way, and as a result is exposing a
leak that wasn't previously a problem.
If you can find the source of the memory leak, and it does turn out to
be a problem with something that Django has introduced since 1.0,
fixing it will be a high priority for us. Unfortunately, I can't think
of anything we have done that is an immediate or obvious candidate.
While there have been lots of changes to the DB handling for v1.1, I
can't think of anything that would be a candidate for leaking memory.
Any assistance you can provide (even if it's just at the level of
isolating a test case that leaks reliably) will be gratefully
accepted.
Yours,
Russ Magee %-)
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
"Django users" group.
To post to this group, send email to django-users@googlegroups.com
To unsubscribe from this group, send email to
django-users+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/django-users?hl=en
-~--~~~~--~~--~--~---
