Great! I want to understand it too! Your breakdown helps me think about how to look into this. I will do some more analysis, but for now: - I'm not using forms - I had migrate=True and the app was not compiled earlier. now it's compiled and migrate=False and I have a bigger machine, things are running ok, but I changed quite a few variables. Would migrate=True create locking issues? The problems seemed similar to locking issues. - My machine's ram slowly filled up from ~700mb/1500mb to 1400mb/ 1500mb over 5 hours. But I haven't looked to see which process grew yet, I will investigate.
On Jul 21, 1:13 am, mdipierro <mdipie...@cs.depaul.edu> wrote: > I still want to understand why you are having this problem. I see the > following possibility: > > 1) There is a threshold in requests/seconds that depends on memory > available and it is different for different frameworks. web2py does > more than Flask (which is a microframework by definitions) and this > threshold may be lower. > If this is the case the problem should go away with more ram. > > 2) There is a bug in web2py or one of its modules that causes a memory > leak (did you experience a memory leak?) or a locking issues (for > example a process crashes before a locked resource is released). > > I remember you mentioning have this problem exclusively with actions > using SQLFORM under heavy load. Is that correct? This could help us > narrow down the problem. > > Massimo > > On Jul 20, 7:17 pm, Thadeus Burgess <thade...@thadeusb.com> wrote: > > > > > The solution: I switched to Flask. > > > And the problems dissipated completely, without modifying any configuration > > of the web server. > > > I would not, and will not use web2py for any application that is mission > > critical. For personal sites, or quick projects that I know won't receive > > that much attention, web2py is fine. For quickly prototyping something, > > web2py excels. For stability, reliability, and scalability, use Flask or > > Django. > > > The DAL is great though, nothing quite like it, thats why I am working on a > > Flask-DAL extension, and I am working to re-write parts of the DAL and strip > > out the web2py cohesion (such as SQLFORM validators). > > > Using the DAL inside of flask works fine, and I do not run into these > > errors. This means that the DAL is not the cause of these errors, but web2py > > core code. Most likely in dealing with pages that have forms is where these > > errors arise. Web2py core is messy, and it ignores the wsgi specification > > for the most part. I am sure that these errors arise from the fact that > > web2py uses execfile in many places over and over again, which is a > > discouraged practice among the python community, and you see why now. > > > -- > > Thadeus > > > On Tue, Jul 20, 2010 at 4:17 PM, Michael Toomim <too...@gmail.com> wrote: > > > Thank you for the clarification. > > > > My wsgi.conf has default values, so I have not set maximum-requests. > > > Perhaps there are settings there I should look into? > > > > I still have free memory, so perhaps there is not a memory leak > > > issue. I'm also not sure how one would get memory leaks in web2py, > > > since isn't the environment wiped clean with each request? > > > > This looks similar to the issue here: > > > >http://groups.google.com/group/web2py/browse_thread/thread/49a7ecabf4... > > > Was there any resolution? > > > > I use logging by having the following file in models/0_log.py: > > > > import logging > > > def get_log(): > > > try: > > > f = open(logging.handlers[0].baseFilename, 'r') > > > c = f.readlines() > > > f.close() > > > return {'log':TABLE(*[TR(str(item)) for item in c])} > > > except: > > > return () > > > > # This model file defines some magic to implement app_wide_log. > > > def _init_log(): # Does not work on GAE > > > import os,logging,logging.handlers > > > logger = logging.getLogger(request.application) > > > logger.setLevel(logging.DEBUG) > > > handler = logging.handlers.RotatingFileHandler( > > > os.path.join( # so that it can be served as http:// > > > .../yourapp/static/applog.txt > > > request.folder,'static','applog.txt'),'a',1024*1024,1) > > > handler.setLevel(logging.DEBUG) > > > handler.setFormatter(logging.Formatter("%(asctime)s %(levelname)s % > > > (filename)s:%(lineno)d %(funcName)s(): %(message)s")) > > > logger.addHandler(handler) > > > return logger > > > > logging = > > > cache.ram('app_wide_log',lambda:_init_log(),time_expire=None) > > > > On Jul 20, 2:03 am, mdipierro <mdipie...@cs.depaul.edu> wrote: > > > > Thanks for the clarification. > > > > > @Michael, do you use the logging module? How? > > > > > On Jul 20, 4:00 am, Graham Dumpleton <graham.dumple...@gmail.com> > > > > wrote: > > > > > > On Jul 20, 5:17 pm, mdipierro <mdipie...@cs.depaul.edu> wrote: > > > > > > > The problem with IOError, I can understand. As Graham says, if the > > > > > > client closes the connection before the server responds or if the > > > > > > server timesout the socket is closed and apache logs the IOError. > > > > > > That isn't what I said. If you see that message when using daemon > > > > > mode, the Apache server process that is proxying to the daemon process > > > > > is crashing. This is different to the HTTP client closing the > > > > > connection. You would only see that message if HTTP client closed > > > > > connection if using embedded mode. > > > > > > I know they are using daemon mode as that is the only situation where > > > > > they could also see the message about premature end of script headers. > > > > > > > What I really do not understand is why some requests are handled by > > > > > > multiple threads. web2py is agnostic to this (unless you use Rocket > > > > > > which you do not). web2py only provides a wsgi application which is > > > > > > executed - per thread - by the web server. It is the web server (in > > > > > > your case apache) that spans the thread, maps requests to threads, > > > > > > calls the web2py wsgi application for each of them. > > > > > > > If this is happening it is a problem with apache or with mod_wsgi. > > > > > > More likely the problem is that they are registering the logging > > > > > module from multiple places and that is why logging is displayed more > > > > > than once. They should log the thread ID as well as that would confirm > > > > > whether actually from the same thread where logging module handler has > > > > > been registered multiple times. > > > > > > Multiple registrations of logging handler could occur if it isn't done > > > > > in a thread safe why, ie., so as to avoid multiple threads doing it at > > > > > the same time. > > > > > > Graham > > > > > > > Can > > > > > > you tell us more about the version of ubuntu, apache and mod_wsgi > > > that > > > > > > you are using? Any additional information will be very useful. > > > > > > > Massimo > > > > > > > On Jul 19, 9:01 pm, Michael Toomim <too...@gmail.com> wrote: > > > > > > > > I'm getting errors like these in my apache error logs: > > > > > > > > [Mon Jul 19 18:55:20 2010] [error] [client 65.35.93.74] Premature > > > end > > > > > > > of script headers: wsgihandler.py, referer: > > >http://yuno.us/init/hits/hit?assignmentId=1A7KADKCHTB1IJS3Z5CR16OZM4V... > > > > > > > [Mon Jul 19 18:55:20 2010] [error] [client 143.166.226.43] > > > Premature > > > > > > > end of script headers: wsgihandler.py, referer: > > >http://yuno.us/init/hits/hit?assignmentId=1A9FV5YBGVV54NALMIRILFKHPT1... > > > > > > > [Mon Jul 19 18:55:50 2010] [error] [client 117.204.99.178] > > > > > > > mod_wsgi > > > > > > > (pid=7730): Exception occurred processing WSGI script > > > '/home/toomim/ > > > > > > > projects/utility/web2py/wsgihandler.py'. > > > > > > > [Mon Jul 19 18:55:50 2010] [error] [client 117.201.42.84] mod_wsgi > > > > > > > (pid=7730): Exception occurred processing WSGI script > > > '/home/toomim/ > > > > > > > projects/utility/web2py/wsgihandler.py'. > > > > > > > [Mon Jul 19 18:55:50 2010] [error] [client 117.201.42.84] mod_wsgi > > > > > > > (pid=7730): Exception occurred processing WSGI script > > > '/home/toomim/ > > > > > > > projects/utility/web2py/wsgihandler.py'. > > > > > > > [Mon Jul 19 18:55:50 2010] [error] [client 117.201.42.84] IOError: > > > > > > > failed to write data > > > > > > > [Mon Jul 19 18:55:50 2010] [error] [client 117.201.42.84] mod_wsgi > > > > > > > (pid=7730): Exception occurred processing WSGI script > > > '/home/toomim/ > > > > > > > projects/utility/web2py/wsgihandler.py'. > > > > > > > [Mon Jul 19 18:55:50 2010] [error] [client 117.201.42.84] IOError: > > > > > > > failed to write data > > > > > > > [Mon Jul 19 18:55:50 2010] [error] [client 117.201.42.84] mod_wsgi > > > > > > > (pid=7730): Exception occurred processing WSGI script > > > '/home/toomim/ > > > > > > > projects/utility/web2py/wsgihandler.py'. > > > > > > > [Mon Jul 19 18:55:50 2010] [error] [client 117.201.42.84] IOError: > > > > > > > failed to write data > > > > > > > [Mon Jul 19 18:55:50 2010] [error] [client 117.201.42.84] mod_wsgi > > > > > > > (pid=7730): Exception occurred processing WSGI script > > > '/home/toomim/ > > > > > > > projects/utility/web2py/wsgihandler.py'. > > > > > > > [Mon Jul 19 18:55:50 2010] [error] [client 117.201.42.84] IOError: > > > > > > > failed to write data > > > > > > > > My web app gets about 7 requests per second. At first, things work > > > > > > > fine. Then after a while it seems like every request gets handled > > > by > > > > > > > MULTIPLE threads, because my logging.debug() statements print > > > multiple > > > > > > > copies of each message and it seems my database gets multiple > > > entries. > > > > > > > And I get these errors in the apache logs (with LogLevel debug). > > > > > > > > Any idea what to do? Where to look? I'm on ubuntu.
