[web2py] Re: How much memory does web2py need on Unix

2010-02-20 Thread John Heenan
Great! I have modified the web2py slice at http://www.web2pyslices.com/main/slices/take_slice/57 and added in a complete minimalist lighttpd configuration file for use with web2py based on tests I have just completed with a new VPS. I have also showed how to enable SSL (provided you have

[web2py] Re: How much memory does web2py need on Unix

2010-02-20 Thread John Heenan
Done. Please see section 'Complete Minimalist Lighttpd Configuration File' at http://www.web2pyslices.com/main/slices/take_slice/57 I have also added in a section 'Using Lighttpd on port 80 for Secure Web2py admin Access' since it is important to emphasise there should only be one instance of

Re: [web2py] Re: How much memory does web2py need on Unix

2010-02-20 Thread Thadeus Burgess
As another added note.I can further reduce my memory footprint by using wsgi 3.X. 3.X has new directives, one called LazyInitialization, which does not initialize the python interpreter when apache starts, only when a request to the wsgi process is made. Along with this is a directive that tells

[web2py] Re: How much memory does web2py need on Unix

2010-02-20 Thread John Heenan
Round and round we go with more diversionary smokescreens. I have provided a quick 'bang you are there' approach for setting up an accepted and proven web server, Lightttpd, that tests show beats the pants off Apache. Apache fans cry foul and won't accept test results. They blame users for poor

Re: [web2py] Re: How much memory does web2py need on Unix

2010-02-20 Thread Thadeus Burgess
You need to change your tone of voice John. If anyone has been have crying foul, and won't accept test results. It is you. I have shown you a valid apache configuration that is comparable to lighttpd, for the sake of this thread it is important to show that apache can too have a low memory

[web2py] Re: How much memory does web2py need on Unix

2010-02-20 Thread John Heenan
This has gone way off topic and personal, however I suppose I should try and do something to repair the ill feeling. I acknowledge Thadeus is a valuable and talented contributor. I don't have the time or inclination to address anything that does not specifically address relevant topics and doing

[web2py] Re: How much memory does web2py need on Unix

2010-02-19 Thread raven
John, Thank you for the suggestions. When I use the SERVER command, I get can't bind to port errors The other issues you mention all seem OK. r...@x:/etc/lighttpd# /etc/init.d/lighttpd restart Syntax OK * Stopping web server lighttpd [ OK ] Syntax OK

[web2py] Re: How much memory does web2py need on Unix

2010-02-19 Thread raven
I avoided the 'address already in use' errors by using port 8000 Still getting 404 - File not found I found this in the lighttpd error log, which looks like it might have something to do with my problem 2010-02-19 12:44:40: (server.c.931) WARNING: unknown config-key: url.rewrite-once (ignored)

[web2py] Re: How much memory does web2py need on Unix

2010-02-19 Thread raven
Found it !!! I needed to include the mod_rewrite module. Everything seems to be working now On Feb 19, 9:22 am, raven ravenspo...@yahoo.com wrote: John, Thank you for the suggestions. When I use the SERVER command, I get can't bind to port errors

[web2py] Re: How much memory does web2py need on Unix

2010-02-19 Thread raven
Now that I have web2py working with lighttpd, instead of its own web server, my application runs much, much faster and requires just over half as much memory ( 39M for web2py, python and lighttpd instead of 63M ) leaving me plenty of room in my 64M system This is great! Without lighttpd top -

Re: [web2py] Re: How much memory does web2py need on Unix

2010-02-19 Thread Thadeus Burgess
You would have a similar memory footprint running apache/mod_wsgi. Mind you I am also running Ruby (mod_passenger) as well as mod_wsgi, so the memory usage is a tiny bit higher because of the extra ruby processes. free -m total used free sharedbuffers cached

[web2py] Re: How much memory does web2py need on Unix

2010-02-19 Thread John Heenan
I do not accept the data below as complete for comparative purposes since we do not know what the memory use by Apache proper itself is. The figures we are using to add up memory use for web2py are resident memory (RSS) for web server for web2py + python for web2py + database for web2py RSS for

Re: [web2py] Re: How much memory does web2py need on Unix

2010-02-19 Thread Thadeus Burgess
Happy ? ps aux | grep apache ps aux | grep wsgi ps aux | grep postgres ps aux | grep python USER PID %CPU %MEMVSZ RSS TTY STAT START TIME COMMAND root 3878 0.6 1.4 145136 7528 ?Ss 22:49 0:00 /usr/sbin/apache2 -k start www-data 3950 0.0 1.2 145288 6360

Re: [web2py] Re: How much memory does web2py need on Unix

2010-02-19 Thread Thadeus Burgess
Apache: 6360 web2py: 21244 Postgres: 26693 Total of 27MB, during peak hours of this site. PostgreSQL is negligible since you can pick your database. -Thadeus On Fri, Feb 19, 2010 at 10:55 PM, Thadeus Burgess thade...@thadeusb.com wrote: ww-data  3950  0.0  1.2 145288  6360 ?        S    

[web2py] Re: How much memory does web2py need on Unix

2010-02-19 Thread mdipierro
please do. I could use this. On Feb 19, 11:35 pm, John Heenan johnmhee...@gmail.com wrote: Thanks. From below we get the following resident memory use (RSS) for Apache with mod_wgsi and web2py (using Postgre database) 13,888 Apache (=7528+6360) 5,612 Some process associated with Apache for

Re: [web2py] Re: How much memory does web2py need on Unix

2010-02-19 Thread Thadeus Burgess
If the only discussion here is tight vps, then postgresql results should not even come into matter in your bottom line. Taking out 20MB for the postgres process, and add 5 mb for a sqlite in the python/web2py process, thats an extra 15 mb you can play around with. Also, configuring apache to run

[web2py] Re: How much memory does web2py need on Unix

2010-02-18 Thread raven
Well, I gave this a try, and ended up with 404 errors. fcgihandler.py suggests adding to lighttpd.conf server.error-handler-404 = '/test.fcgi' This does not help, nor does server.error-handler-404 = /error-handler.fcgi ( There no files on my system named *.fcgi, so I do not see how this could

[web2py] Re: How much memory does web2py need on Unix

2010-02-18 Thread John Heenan
Hi Raven The name fcgihandler.py is just the name of a web2py start up file passed to Python. The name fcgihandler.py should not or need not appear anywhere in the lightttpd.conf file, since there is no need to give lighttpd the responsibility to start up web2py as an independent process. What

Re: [web2py] Re: How much memory does web2py need on Unix

2010-02-18 Thread Alex Fanjul
I was reviewing this threadand wow! very nice explanation and presentation of playing video streaming via web2py. Many thanks John, your thoughts looks like very interesting. Alex El 14/02/2010 3:12, John Heenan escribió: Thanks for your comments also. Yes .flv and other video files will

[web2py] Re: How much memory does web2py need on Unix

2010-02-18 Thread raven
John, Here is what I have placed in the file /etc/lighttpd/lighttpd.conf $ fastcgi.server = ( /handler_web2py.fcgi = ( handler_web2py = ( #name for logs check-local = disable, socket = /tmp/fcgi.sock, ) ) ) $HTTP[host] == 100.101.102.103 {

[web2py] Re: How much memory does web2py need on Unix

2010-02-18 Thread John Heenan
Raven, 1) Use $SERVER[socket] == 100.101.102.103 { instead of $HTTP[host] == 100.101.102.103 { 2) Did you restart lighttpd with /etc/init.d/lighttpd restart and where there any problems when restarting? 3) Is mod_fastcgi included with server.modules in lighttpd.conf? 4) Is web2py actually

[web2py] Re: How much memory does web2py need on Unix

2010-02-18 Thread John Heenan
Raven $SERVER[socket] == 0.0.0.0:80 { will allow all IP interfaces. John Heenan On Feb 19, 11:42 am, John Heenan johnmhee...@gmail.com wrote: Raven, 1) Use $SERVER[socket]  == 100.101.102.103 { instead of $HTTP[host] == 100.101.102.103 { 2) Did you restart lighttpd with

[web2py] Re: How much memory does web2py need on Unix

2010-02-13 Thread raven
Thank you for the lighttpd howto. It looks very helpful. Hopefully, I will be able to clear some time next week to give it a go. I noticed that, at the end, you spend time on URL remapping. To benefit from this either routes.out of routes.py needs to be used or internal techniques need to be

[web2py] Re: How much memory does web2py need on Unix

2010-02-13 Thread mdipierro
Very well done. Did you ever try streaming files with lighttpd? For example serving via download a large video file, larger than ram? (kpax comes with an example of streaming). Does it work well? Massimo On Feb 13, 1:42 am, John Heenan johnmhee...@gmail.com wrote: As requested I have placed a

[web2py] Re: How much memory does web2py need on Unix

2010-02-13 Thread John Heenan
Thanks for you comments. Yes you are right. I have separated out URL mapping to another section. This dramatically simplifies the configuration. I have also added in a few bits and pieces, such as on running web2py as user www-data instead of user root. Slice is at

[web2py] Re: How much memory does web2py need on Unix

2010-02-13 Thread John Heenan
Thanks for your comments also. Yes .flv and other video files will play with Lightttpd using normal media players, for example, using JW Player in kpax/static/ mediaplayer.swf . It is even possible to use players while the actual video file is hosted on another server to where the main site and

[web2py] Re: How much memory does web2py need on Unix

2010-02-12 Thread Graham Dumpleton
On Feb 12, 6:16 pm, John Heenan johnmhee...@gmail.com wrote: For lower memory footprint on a tight VPS I do not believe any configuration of Apache with web2py (using mod_wgsi or otherwise) will beat a good lighttpd configuration with a FastCGI UNIX socket interface to web2py. I have helped

[web2py] Re: How much memory does web2py need on Unix

2010-02-12 Thread John Heenan
I should stop feeding the troll. It is obvious Graham has no relevant benchmarks to prove his case, has no intention of providing them. None of the links provided make a relevant case. The bizarre 'Nginx + Apache/mod_wsgi' reference to refer to using Nginx for static content and Apache with

[web2py] Re: How much memory does web2py need on Unix

2010-02-12 Thread mdipierro
Let's cool down. You are both very much welcome on this list and you have both shown ability to provide excellent contributions to this list. Graham wrote mod_wsgi for apache and knows it inside out. No question about that. mod_wsgi is the best way to deply web2py on apache. Period. John has

[web2py] Re: How much memory does web2py need on Unix

2010-02-12 Thread raven
John has provided excellent benchmarks that can help us improve and can help us make better usage of the memory. He certainly helped me to get up and running! Something I am interested in following up on. Great. It would be really great if there was an out of the box command line switch to

[web2py] Re: How much memory does web2py need on Unix

2010-02-12 Thread mdipierro
We do not have to close it as long as there is a change of tone. In web2py.py comment this line: import gluon.import_all and you should be good to go with less memory. Massimo On Feb 12, 9:02 am, raven ravenspo...@yahoo.com wrote: John has provided excellent benchmarks that can help us

[web2py] Re: How much memory does web2py need on Unix

2010-02-12 Thread John Heenan
The title of the thread is How much memory does web2py need on Unix. I provided a concrete answer. The question is not answered by stating mod_wgsi is the 'best' way to deploy web2py with Apache. I provided real data about using lighthttpd and a UNIX socket. There is every reason to believe

[web2py] Re: How much memory does web2py need on Unix

2010-02-12 Thread mdipierro
I think this is an excellent point. My data demonstrated a 40MB memory solution is feasible. I am absolutely amazed and disappointed I cannot get such simple points gracefully acknowledged by a self styled 'expert' and by the moderator. The reason I suggested opening a new thread was to

[web2py] Re: How much memory does web2py need on Unix

2010-02-12 Thread John Heenan
As requested I have placed a howto on web2pysplices at http://www.web2pyslices.com/main/slices/take_slice/57 I have written a considerable preface to clarify issues and to indicate why we should pay attention to issues concerned with keeping memory use low. John Heenan On Feb 13, 2:41 am,

[web2py] Re: How much memory does web2py need on Unix

2010-02-11 Thread KMax
Look like OpenVZ container. I play with this a lot. First of all memory size, not always limits your need. 64M - to low. Hardly to run static serve apache. (moderate load). VZ container also has limits for (sample from my server) 100: kmemsize23782914102256 2147483647 2147483647

[web2py] Re: How much memory does web2py need on Unix

2010-02-11 Thread raven
It seems that everyone is running with Apache and gobs of memory available. They cannot really get their heads around running web2py out of the box in 64K. So let me explain why this is important. When I sell a desktop application to someone, all they have to pay for is my coding work. They

Re: [web2py] Re: How much memory does web2py need on Unix

2010-02-11 Thread tiago almeida
You mean 64MB, not 64K, I hope. On Thu, Feb 11, 2010 at 12:30 PM, raven ravenspo...@yahoo.com wrote: It seems that everyone is running with Apache and gobs of memory available. They cannot really get their heads around running web2py out of the box in 64K. So let me explain why this is

[web2py] Re: How much memory does web2py need on Unix

2010-02-11 Thread raven
On Feb 11, 7:54 am, tiago almeida tiago.b.alme...@gmail.com wrote: You mean 64MB, not 64K, I hope. Oops! Yes, I do mean 64M :-) -- You received this message because you are subscribed to the Google Groups web2py-users group. To post to this group, send email to web...@googlegroups.com. To

Re: [web2py] Re: How much memory does web2py need on Unix

2010-02-11 Thread Timothy Farrell
You might try commenting out the lines in import_all.py if you're running the source version. That way it doesn't load every module. -tim On 2/10/2010 9:06 PM, raven wrote: Here are the details of memory usage when web2py is NOT running top - 21:58:38 up 11:57, 2 users, load average:

[web2py] Re: How much memory does web2py need on Unix

2010-02-11 Thread raven
This sounds interesting. Instead of hacking around in import_all.py, I wonder if someone who know what's what, could provide a stripped down version of this file - or even an option to run with a stripped down version that only loads the bare minimum? On Feb 11, 8:49 am, Timothy Farrell

Re: [web2py] Re: How much memory does web2py need on Unix

2010-02-11 Thread Andres Vargas - zodman
im us this script for check the memory usage of my aplications ... checkit... http://gist.github.com/252585 On Thu, Feb 11, 2010 at 8:34 AM, raven ravenspo...@yahoo.com wrote: This sounds interesting. Instead of hacking around in import_all.py, I wonder if someone who know what's what,

[web2py] Re: How much memory does web2py need on Unix

2010-02-11 Thread Kevin Bowling
You probably can't even fit the process scheduler of a modern OS in 64k. Unless you want to code your apps on bare metal (no OS) assembly language, there is no way you'll get any higher level language in 64k. Even then, toss out the notions of a database, transactions, concurrency, and

[web2py] Re: How much memory does web2py need on Unix

2010-02-11 Thread mdipierro
you just comment import import_all from main web2py should still work but will choke later if an app requests a module that does not fit in memory. Anyhow, modules do not take that much space. On Feb 11, 8:34 am, raven ravenspo...@yahoo.com wrote: This sounds interesting. Instead of hacking

Re: [web2py] Re: How much memory does web2py need on Unix

2010-02-11 Thread Thadeus Burgess
I don't see how you are running any system on 64MB (unless it is DOS or some really really old linux version... maybe gentoo... maybe. ) You won't be able to run web2py on this system. Either upgrade the system or get a shared hosting service. -Thadeus On Thu, Feb 11, 2010 at 12:18 PM,

[web2py] Re: How much memory does web2py need on Unix

2010-02-11 Thread raven
I commented out the line import gluon.import_all from web2py.py. This seemed to help a lot! I now have free memory, just a little, and the % memory used was halved. My little application runs fine, so far. top - 13:33:31 up 1 day, 3:32, 3 users, load average: 0.00, 0.00, 0.00 Tasks: 14

[web2py] Re: How much memory does web2py need on Unix

2010-02-11 Thread villas
Oops! Yes, I do mean 64M :-) 64Mb ...holy smoke!! My hoster wouldn't even sell me less than 256Mb. I was doubtful there would be much financial point to this quest for minimal memory, but a quick google later, I came up with this... http://www.ramhost.us/ only $2.99 per month, and I pay

[web2py] Re: How much memory does web2py need on Unix

2010-02-11 Thread John Heenan
How about web2py in a VPS using less than 40MB RAM? You can reduce web2py memory usage by using a newer generation web server with web2py instead of the internal web server with web2py. Apache gets trashed in tests by newer generation web servers such as lightttpd and nginx. Apache also uses far

[web2py] Re: How much memory does web2py need on Unix

2010-02-11 Thread John Heenan
I only use lightttpd for static pages and to remap URLs. Plus a UNIX socket between lighttpd and web2py to service web2py requests through FastCGI. John Heenan On Feb 12, 8:59 am, John Heenan johnmhee...@gmail.com wrote: How about web2py in a VPS using less than 40MB RAM? You can reduce

[web2py] Re: How much memory does web2py need on Unix

2010-02-11 Thread John Heenan
Yes the instructions are confusing. Following are sections from my /etc/lighttpd/lighttpd.conf file. I have edited sections to remove irrelevant detail. The lighttpd remaps help to avoid ugly URLs. I don't use a routes.py file but the zgus app does include some helpers to avoid ugly URLs (this

[web2py] Re: How much memory does web2py need on Unix

2010-02-11 Thread raven
Thanks for the details on modifying the lighttpd configuration file. Actually, I am pretty confident about doing this, having modified it to connect to fossil. However, I am confused by fcgihandler.py It says: This file has to be in the PYTHONPATH What does this mean? The file has to be

[web2py] Re: How much memory does web2py need on Unix

2010-02-11 Thread John Heenan
My fcgihandler.py equivalent file is in the the same directory as web2py.py My init.d startup script changes to this directory before running fcgihanfdler.py with Python. I don't make any separate changes to PYTHONPATH. The web2py FastCGI server needs to know where the UNIX socket is. This is

[web2py] Re: How much memory does web2py need on Unix

2010-02-11 Thread Graham Dumpleton
On Feb 12, 9:59 am, John Heenan johnmhee...@gmail.com wrote: How about web2py in a VPS using less than 40MB RAM? You can reduce web2py memory usage by using a newer generation web server with web2py instead of the internal web server with web2py. Not really. Apache gets trashed in tests

[web2py] Re: How much memory does web2py need on Unix

2010-02-11 Thread John Heenan
Hello Graham, whoever you are. You sound highly confused, clueless aboout how to present objective data and a fairly typical bombastic nerd of the type that clogs up and plagues forums. Get a life John Heenan On Feb 12, 11:32 am, Graham Dumpleton graham.dumple...@gmail.com wrote: On Feb 12,

[web2py] Re: How much memory does web2py need on Unix

2010-02-11 Thread Graham Dumpleton
On Feb 12, 1:04 pm, John Heenan johnmhee...@gmail.com wrote: Hello Graham, whoever you are. You sound highly confused, clueless aboout how to present objective data and a fairly typical bombastic nerd of the type that clogs up and plagues forums. Get a life I think you will find that I

[web2py] Re: How much memory does web2py need on Unix

2010-02-11 Thread John Heenan
Where is the perspective? 1) Even with an 'ideal configuration that does not use MPM pre- forking, Apache still uses threads to service each request (for static files). This is still more inefficient than lightttpd and nginx that use an event model. 2) No one is going to take anyone seriously

[web2py] Re: How much memory does web2py need on Unix

2010-02-11 Thread John Heenan
From the confused manner Graham is conducting himself, he appears to think web2py runs as a process of a web server with threads that belong to the web server. This is not correct. web2py always runs as an independent process, unless web2py uses its internal web server. An external web server

[web2py] Re: How much memory does web2py need on Unix

2010-02-11 Thread Graham Dumpleton
On Feb 12, 1:44 pm, John Heenan johnmhee...@gmail.com wrote: Where is the perspective? 1) Even with an 'ideal configuration that does not use MPM pre- forking, Apache still uses threads to service each request (for static files). This is still more inefficient than lightttpd and nginx that

[web2py] Re: How much memory does web2py need on Unix

2010-02-11 Thread Graham Dumpleton
On Feb 12, 2:00 pm, John Heenan johnmhee...@gmail.com wrote: From the confused manner Graham is conducting himself, he appears to think web2py runs as a process of a web server with threads that belong to the web server. This is not correct. web2py always runs as an independent process,

[web2py] Re: How much memory does web2py need on Unix

2010-02-11 Thread John Heenan
For lower memory footprint on a tight VPS I do not believe any configuration of Apache with web2py (using mod_wgsi or otherwise) will beat a good lighttpd configuration with a FastCGI UNIX socket interface to web2py. I also believe trying to argue otherwise without credible benchmarks is liable

[web2py] Re: How much memory does web2py need on Unix

2010-02-10 Thread raven
Thank you for your answer. 1. The memory needed by the OS does not count, because I am using a virtual private server. 2. I am using web2py 'out of the box' with its own server, not apache. On Feb 10, 2:36 pm, Thadeus Burgess thade...@thadeusb.com wrote: For one, you have two requirements,

[web2py] Re: How much memory does web2py need on Unix

2010-02-10 Thread mdipierro
Are you sure. Usually VPS includes the OS. On Feb 10, 2:23 pm, raven ravenspo...@yahoo.com wrote: Thank you for your answer. 1.  The memory needed by the OS does not count, because I am using a virtual private server. 2.  I am using web2py 'out of the box' with its own server, not apache.

[web2py] Re: How much memory does web2py need on Unix

2010-02-10 Thread raven
Here are the details of memory usage when web2py is NOT running top - 21:58:38 up 11:57, 2 users, load average: 0.00, 0.00, 0.00 Tasks: 10 total, 1 running, 9 sleeping, 0 stopped, 0 zombie Cpu(s): 0.0%us, 0.0%sy, 0.0%ni,100.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st Mem: 65536k