On Oct 25, 7:19 am, Michel Thadeu Sabchuk <[EMAIL PROTECTED]> wrote: > Hi Graham! > > > What percentage of requests are actually yielding static pages? > > People often too quickly run to lighttpd thinking it will somehow > > magically solve all their problems, but if the bulk of the requests > > are actually dynamic and served by Django, and it is Django and the > > database which are the bottleneck, it will make very little if any > > difference. > > Ok, > > > Now to try and get to what the real issue may be, expand on what you > > mean by 'server stuck when I had too much hits'. > > I couldn´t figure out exactly my problem. The key pages of my site is > cached through locmen, the index page is cron generated. I have too > much media files, my site is a games portal and I have videos, images > and stuff served on the same server,
Okay, when I say static files am thus also referring to media files. In this sort of situation where your static files also consist of a lot of large videos, sound files or images, then moving them to another server and using lighttpd or nginx is definitely a thing to consider. As mentioned by one of the referenced articles, even if you don't have another physical server then running it on the same machine may also help. If a different physical machine or you have multiple IPs on the one machine, then preferable to serve the media files from a different hostname. That way the media web server can still run on port 80, in the case of the same machine by configuring the respective web servers to only listen on the one IP address and not all interfaces. If you only have the one machine, don't have a separate IP and don't want to have a non port 80 public web server, then you'll need to make everything appear on the one hostname/port. Although you could make lighttpd or nginx the first point of call and proxy through to the Apache for the Django application, you may be better off considering a lightweight proxy in front of both web servers. One option for that which I have heard good things about from people I know who use it is 'Pound'. This is available from: http://www.apsis.ch/pound/ The good things they have said about it include it being fast and easy to configure. The ease in being able to add, remove or move backends would also be extremely useful if you get a second box as it may make it easier to transparently and with minimum amount of fuss move stuff between machines as you work out the best architecture. That said, I haven't personally used it, but still worth a look and obviously you would want to test well any architecture you come up with. > a Dual Intel(R) Xeon(TM) CPU > 3.20GHz with 4GB of RAM space. I have the following installation: > > * Apache2.2 > * Python-2.4.2 > * Django-0.96pre served through mod_python > * MySQLdb-1.2.2 > * MySQL-5.0.18 > > At some key pages I use django´s locmen cache. My home page is > somewhat slow because it has too many interchains of news sections > (too much filtering), this way, I generate the home page every 30 > minutes and serve it as a static page. I start this approach this > week, in the past time I let the home page to be cached by locmen too. > > My RAM usage frequently reaches 3GB or more, Do you know which process in particular though are using up all the memory? If it is the Apache child process in which mod_python and your Django application is running are you taking measures like setting MaxRequestsPerThread directive to some non zero value so that the processes are recycled on a regular basis and not just growing and growing in size over time. Disabling keep alive or at least reducing the keep alive timeout can also be help in as much as holding down active memory and allowing more concurrent requests from different users without Apache having to launch new child processes. > A friend told me that I can make python library shared. If you track that one back far enough you will find me at the other end as I was the one who started that recommendation. I got sick of people complaining about mod_python using up so much memory when it was mostly to do with the Python installation not using a shared library and what happens when statically compiled object gets used in dynamically loadable modules. :-) > I read the > Jacob´s post [1] about performance tips and the Chase Davis [2] > recomendations. Yes, both contain useful suggestions. You need to look through some of the comments to understand why though for some things. Although some comments also don't give the full picture, for example, the GIL is not necessarily as big a problem when hosting applications embedded with Apache child processes as some would have you believe. See my prior blog entries for more about GIL and multi core issues: http://blog.dscpl.com.au/2007/09/parallel-python-discussion-and-modwsgi.html http://blog.dscpl.com.au/2007/07/web-hosting-landscape-and-modwsgi.html > I get another box to make server scaling, it is a > Pentium 4 CPU 3.4GHz with 2GB of RAM space. What is the best > alternative to this server? Should I use it as a database server or > should I use it as a media files server, or maybe I should use it for > apache and the another server for database... How to split things up across boxes is always a hard decision. You really need to do some testing which mirrors the requirements of your application as far as database and media file serving loads to get a good idea. Without that it may be more of an educated guess as to which may benefit more. > I had problems like the about 2 months ago and I updated the python > mysql library (my last one had a bug with garbage collection), and the > server goes ok for these 2 months, it start to show me problems again > when I had an increment of users. I could not tell how many was this > increment, but on monday I launch a new online game (it is served on a > partner) and I could measure the incrementation of users on site by > the forum and other interactive sections. The server stopped to run > again today just after I change a header file used by this webgame > (the webgame is hosted by another server, but the header is on my > server), but I don´t know if it is a pertinent information :P Still not sure though what you think is causing the problems. Yes, you have high memory usage but what is using the memory? Is that leading onto swapping? If you can indicate whether it is Apache itself, the Django application, the database etc, will help as far as giving further suggestions. BTW, as far as using mod_python, you might want to also experiment with mod_wsgi. For your sort of high performance web site, you would most likely use it in embedded mode and so not much different to mod_python in that respect, but mod_wsgi does have a smaller memory footprint and lower per request overhead. Graham > I am very thankfull for all your help, > Best regards. > > [1]http://www.jacobian.org/writing/2005/dec/12/django-performance-tips/ > [2]http://www.car-chase.net/2007/apr/30/django-deployment-lessons-learned/ > > -- > Michel Thadeu Sabchuk > MisterApe --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Django users" group. To post to this group, send email to django-users@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-users?hl=en -~----------~----~----~----~------~----~------~--~---
