Possible memory leak moving to Django 1.1 from 1.0.2 in mixed use test.
We've got an application that is running behind apache, mod_wsgi and front ending a postgres 8.3 db. We needed to upgrade our Satchmo to the latest version which requires Django 1.1. After the upgrade the system starts eventually running out of memory until it thrashes and locks up the machine under a fairly light load, just a few reqs per second. I can understand the system using more memory but it doesn't seem to release it when the load stops. This graph: http://www.flickr.com/photos/proteusguy/3589788760 shows the results of both systems which, besides the Django & Satchmo versions are identical. You can see that when the test completes on the old system the process returns a great amount of memory but the new one never releases the memory. In production this will eventually grow to consume all RAM. This particular test is of a set of 7 different requests simulating what a typical user might do. This graph: http://www.flickr.com/photos/proteusguy/3589788752 shows a similar test except it only contains a single request out of the seven from the prior test. We were hoping to identify that something in one of our requests was the cause of the problem but similar exclusive test runs with all 7 requests from the combined test from above show similar results - fairly stable memory usage. This leads me to believe that the issue is systemic to django, most likely in the db handling as most of the requests don't touch the Satchmo system. If this was a C++ app I'd be able to figure this out pretty quick but I'm not sure how to attack the problem in a dynamic language environment like python. So this leaves me with two questions: 1. Is anyone else seeing this kind of memory extra memory usage and an inability to return memory once requests end? 2. What suggestions does anyone have about how I can determine what aspect of the system is using this extra memory and holding onto it? thanx very much, -- Ben Scherrey --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django users" group. To post to this group, send email to django-users@googlegroups.com To unsubscribe from this group, send email to django-users+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/django-users?hl=en -~--~~~~--~~--~--~---
Multiple Databases Revisited (Interim)
I've been following the discussion about supporting multiple databases within one project (primarily on a per-app or per-model basis) with interest and look forward to the feature getting solidified and implemented into the trunk one day. That doesn't appear to be a near term event, however, so I was curious what people are doing to address database scalability issues on a practical basis currently. For example, the Django Auth and Sessions models could absolutely be deployed onto their own database servers as we do no foreign key joins to any of those and they have a lot of traffic that I'd like to not be on my other app db servers. Certain models in my apps warrant similar divisions and would significantly ease my capacity improvement efforts if I could just point them to their own db servers. Specifically we're running Postgres (and loving it). We know how to use Slony but it's not real-time enough for most of our requirements. I'm curious what people in our situation are doing to relieve the pressure on their database servers. Appreciate any insights. -- Ben Scherrey --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django users" group. To post to this group, send email to django-users@googlegroups.com To unsubscribe from this group, send email to django-users+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/django-users?hl=en -~--~~~~--~~--~--~---
Re: Authentication/access check when serving static media thru Apache/nginx
The way to do this is by utilizing nginx as a reverse proxy for your dynamic django (presumably apache) server and a peer media server (presumably another nginx setup). Nginx has a great feature (as does lighttpd) where you can have your page request come into django and perform all the authN/authZ checks needed for your static content but redirect the request to the static media server once approved. Let's say you have a user profile with pictures and want the user to have find grained privacy control on his pictures. 1. request for user profile comes into nginx proxy. 2. nginx proxy fwds request to django server which determines which pics are appropriate to view (authorized) of that profile for the logged in user. 3. the django server, rather than returning static links to the media server, populates a field X-Accel-Redirect that points to the static content on the media server. 4. nginx proxy sees the X-Accel-Redirect contents and resubmits the request to the media server and returns its content instead of the content from the django server. 5. user gets only the static content that the django server authorized yet the load for the transfer is moved to the media server. Good luck, -- Ben On Feb 16, 4:44 pm, MrMuffin wrote: > I`m using django to develop something similar to flickr, a site for > photo-sharing. Photos can be public, need authentication or be > personal and not available for anyone but the owner. Serving static > data using django is not optimal, but how can I control access like > this when serving static media using apache or nginx? > > Thanks in advance. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django users" group. To post to this group, send email to django-users@googlegroups.com To unsubscribe from this group, send email to django-users+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/django-users?hl=en -~--~~~~--~~--~--~---
