Cool. It is a bit fiddly having to drop down to that level to debug such an issue. I would much preferred to be able to point you at the new stuff I am working on as part of my paid job as it would likely have made it a lot simpler to identify the problem in this case. All in good time I guess. :-)
Graham On 17 August 2011 11:22, ellonweb <[email protected]> wrote: > Thanks Graham, this showed me it was a problem inside the WSGI app as > opposed to outside so I ran over the matplotlib docs again and > discovered I had missed out an extra configuration; adding this in > seems to have fixed the problem. Apologies for wasting your time and > thanks again for your earlier help! (Generating these graphs is *slow* > on my little laptop, but lovely and fast when Apache is able to serve > the cached one! (: ) > > On Aug 16, 2:05 am, Graham Dumpleton <[email protected]> > wrote: >> Right now the only thing I can think of to suggest is to log for that >> request when it starts and ends. >> >> You can do that by using a variation of examples shown in: >> >> http://code.google.com/p/modwsgi/wiki/RegisteringCleanupCode >> http://code.google.com/p/modwsgi/wiki/DebuggingTechniques#Tracking_Re... >> >> In other words, lets work out if getting stuck in the application or >> is caused by an external issue. >> >> You might also have a look at: >> >> http://code.google.com/p/modwsgi/wiki/DebuggingTechniques#Extracting_... >> >> as a way of forcing process to dump out what it is doing when it >> appears to be locked up. >> >> Graham >> >> On 15 August 2011 23:04, ellonweb <[email protected]> wrote: >> >> >> >> >> >> >> >> > I'm using matplotlib for the graphs. Adding that setting in my >> > httpd.conf didn't appear to change anything. >> > I've noticed this morning that if I'm entering the URLs for the image >> > in my browser it seems to work fine (20 graphs without locking up), >> > it's only seems to be a problem when the image is requested as part of >> > a page (which is generated by the same WSGI app). But the page has >> > finished loading (necessarily, given the nature of response objects), >> > so I can't see how this could cause a problem. >> >> > On Aug 15, 4:24 am, Graham Dumpleton <[email protected]> >> > wrote: >> >> What are using to generate the images, a C extension module for Python? >> >> >> Could be that you are encountering thread dead locks if code running >> >> in Python sub interpreter. >> >> >> If the only Python web application on host, try setting: >> >> >> WSGIApplicationGroup %{GLOBAL} >> >> >> Graham >> >> >> On 15 August 2011 12:54, ellonweb <[email protected]> wrote: >> >> >> > Hi Graham, >> >> > Sorry to bring this back up; in my move from working prototype to >> >> > actual implementation I have hit an error that is causing the Apache >> >> > process to lock up and max out my CPU with no errors in the error log. >> >> >> > The code works fine in the Django dev server, correctly generating PNG >> >> > graphs and writing them to the filesystem for caching. Without the >> >> > config you helped me with, Apache is able to do the same. With the >> >> > extra config added in, Apache is able to correctly show the graphs >> >> > already stored on the filesystem. However, with this extra config, >> >> > when a graph is requested that isn't already generated, Apache locks >> >> > up. After I kill the process the file is still written to the >> >> > filesystem and loads correctly in my browser. It appears to work >> >> > occasionally but I can't narrow it down, the same graphs will cause >> >> > Apache to lock up sometimes and not others (I'm deleting the cached >> >> > graphs and then refreshing). >> >> >> > I'm not really sure how to start debugging this, your help would be >> >> > much appreciated. >> >> >> > Versions: Apache 2.2.17, mod_wsgi 3.3 (compiled for Py2.6.2), Py 2.6.6 >> >> > Commit of my code, which I can appreciate you probably don't want to >> >> > spend time trawling through but the commit is pretty much limited to >> >> > just my implementations of the things we discussed previously: >> >> >https://github.com/ellonweb/merlin/commit/8aa87683c1766e6d4f173255162... >> >> >> > Please let me know how I should proceed or what further information is >> >> > needed, >> >> > Thanks in advance, >> >> >> > On Jul 19, 5:13 am, Graham Dumpleton <[email protected]> >> >> > wrote: >> >> >> Sorry for taking so long to reply, work is sucking up my free time as >> >> >> well at the moment. >> >> >> >> The issue you are seeing may well relate to this bug in Django. >> >> >> >> https://code.djangoproject.com/ticket/12464#comment:14 >> >> >> >> Unfortunately a few days ago they marked it as closed as they can't >> >> >> see that Django is doing the wrong thing. It isn't helped though by >> >> >> fact that it isn't a simple thing to fix. >> >> >> >> What you may be able to do is use a special WSGI middleware wrapper >> >> >> that adjusts the WSGI environ dictionary values as they get passed >> >> >> through for the error document redirect, perhaps moving REDIRECT_URL >> >> >> out of the way so that Django doesn't muck things up by trying to use >> >> >> it, or otherwise modifying SCRIPT_NAME/PATH_INFO to values that make >> >> >> things do what is necessary. >> >> >> >> Graham >> >> >> >> On 16 July 2011 03:16, ellonweb <[email protected]> wrote: >> >> >> >> > Thanks, this works very well; for reference, I'm able to get the >> >> >> > original request url using REDIRECT_URL (which is accessible in >> >> >> > django's request.META dict) >> >> >> >> > I still have one problem though in detecting the 404 url in the WSGI >> >> >> > app (I'm using the same WSGI for multiple things, same environment >> >> >> > etc), as it seems to vary in a peculiar manner: >> >> >> > With "ErrorDocument 404 /handle_404", if my original request is / >> >> >> > graphs/123, the reported path is /handle_404. /graphs/1234 gives // >> >> >> > handler_404, /graphs/12345 gives /g/handler_404, ... /graphs/ >> >> >> > 1234567890 gives /graphs/handler_404, and with more characters I >> >> >> > start >> >> >> > to get the 1234.. coming through as well! It seems that >> >> >> > "/handler_404" >> >> >> > just replaces the last 11 characters of the request. I realize that >> >> >> > this is probably getting a bit out of the mod_wsgi territory, but any >> >> >> > ideas how to solve this? Or suggestions to detect the 404 url >> >> >> > differently? I know an obvious solution is to use a seperate wsgi app >> >> >> > but it's useful to keep the code/environment combined. (Also, maybe >> >> >> > this is actually just a bug in django's handling?) >> >> >> >> > Many thanks, >> >> >> >> > On Jul 15, 2:21 am, Graham Dumpleton <[email protected]> >> >> >> > wrote: >> >> >> >> Have you tried something like: >> >> >> >> >> Alias /images/ /some/path/images/ >> >> >> >> >> <Directory /some/path/images> >> >> >> >> ErrorDocument 404 /handle_404 >> >> >> >> </Directory> >> >> >> >> >> WSGIScriptAlias /handle_404 /some/path/scripts/handle_404.wsgi >> >> >> >> >> You will need to pick through the WSGI request environment to see >> >> >> >> whether the target file for the original request is included. I >> >> >> >> can't >> >> >> >> remember exactly if it is. >> >> >> >> >> The WSGI script can then generate the file to temporary location and >> >> >> >> then when done move it into correct place and return it for that >> >> >> >> request as response from the WSGI script application. >> >> >> >> >> Graham >> >> >> >> >> On 13 July 2011 14:58, ellonweb <[email protected]> wrote: >> >> >> >> >> > Hi, >> >> >> >> >> > I've got a WSGIScriptAlias set up in Apache, and I'm adding some >> >> >> >> > new >> >> >> >> > functionality at /graphs/ that generate some PNG graphs for my >> >> >> >> > website. I want to set up a caching system for these graphs, >> >> >> >> > however, >> >> >> >> > rather than having to go all the way through WSGI/Python etc. to >> >> >> >> > read >> >> >> >> > the previously generated images, is there a way to configure >> >> >> >> > Apache to >> >> >> >> > look for a file matching the url in some location (for example I >> >> >> >> > use >> >> >> >> > some static images using a normal Alias), and if the files don't >> >> >> >> > exist, have Apache fall back to the WSGI which will generate and >> >> >> >> > output the PNG (and then save it to the file system for future >> >> >> >> > requests that can be served directly by Apache)? Perhaps using the >> >> >> >> > WSGI as a 404 handler? >> >> >> >> >> > Thanks in advance, >> >> >> >> >> > -- >> >> >> >> > You received this message because you are subscribed to the >> >> >> >> > Google Groups "modwsgi" group. >> >> >> >> > To post to this group, send email to [email protected]. >> >> >> >> > To unsubscribe from this group, send email to >> >> >> >> > [email protected]. >> >> >> >> > For more options, visit this group >> >> >> >> > athttp://groups.google.com/group/modwsgi?hl=en. >> >> >> >> > -- >> >> >> > You received this message because you are subscribed to the Google >> >> >> > Groups "modwsgi" group. >> >> >> > To post to this group, send email to [email protected]. >> >> >> > To unsubscribe from this group, send email to >> >> >> > [email protected]. >> >> >> > For more options, visit this group >> >> >> > athttp://groups.google.com/group/modwsgi?hl=en. >> >> >> > -- >> >> > You received this message because you are subscribed to the Google >> >> > Groups "modwsgi" group. >> >> > To post to this group, send email to [email protected]. >> >> > To unsubscribe from this group, send email to >> >> > [email protected]. >> >> > For more options, visit this group >> >> > athttp://groups.google.com/group/modwsgi?hl=en. >> >> > -- >> > You received this message because you are subscribed to the Google Groups >> > "modwsgi" group. >> > To post to this group, send email to [email protected]. >> > To unsubscribe from this group, send email to >> > [email protected]. >> > For more options, visit this group >> > athttp://groups.google.com/group/modwsgi?hl=en. > > -- > You received this message because you are subscribed to the Google Groups > "modwsgi" group. > To post to this group, send email to [email protected]. > To unsubscribe from this group, send email to > [email protected]. > For more options, visit this group at > http://groups.google.com/group/modwsgi?hl=en. > > -- You received this message because you are subscribed to the Google Groups "modwsgi" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/modwsgi?hl=en.
