2008/8/22 Nimrod A. Abing <[EMAIL PROTECTED]>: >>>> Important changes are file descriptor leaks on a graceful restart in >>>> both versions and for version 2.2 possible truncation of data when >>>> using wsgi.file_wrapper. An upgrade would be recommended especially >>>> due to the latter if using wsgi.file_wrapper in a way or on system >>>> affected by the problem. >>> >>> Thanks for the fix for wsgi.file_wrapper. We had been experiencing >>> problems lately with users uploading large files. All tracebacks seem >>> to point to something in wsgi.file_wrapper. Unfortunately it is one of >>> those "heisenbugs". Some random thing/file upload triggers it, it >>> disappears when you try to debug it and it's impossible to recreate. >> >> You mean downloading don't you? At least I take uploading to mean >> putting a file on server, not the other way around, so wouldn't expect >> wsgi.file_wrapper to be involved. I would also have expected you to be >> using Apache 2.2 on UNIX where for static file downloads wouldn't be >> an issue. > > Uploading usually. But I have reason to suspect that it's happening > because users abort the upload.
Which can cause 'IOError: client connection closed' exception to be raised when attempting to read input. > But we also get weird tracebacks from > time to time, these involved URL's where nothing is POSTed. We are > currently using Apache 2.0. We use lighttpd for static content. Am curious to see what these might be and whether they might be originating in mod_wsgi. >> Maybe you want to better explain the problem you are seeing in case it >> is something completely different. One concern would be that if it is >> a large file upload that takes more than 5 seconds, that if you are >> using daemon mode and recycling process based on number of requests, >> that upload is being interrupted when process is recycled, as by >> default will only allow at most 5 seconds for process to restart >> before killing it. In other words, it is not a graceful restart where >> process persists until all active requests complete. > > We are using daemon mode. It's interesting that you brought that up. > It had never occurred to me that the daemon could timeout in the > middle of a POST request, in any case I would not expect it to timeout > in the middle of *any* request. Like I said above, it happens more > often on uploads. I suspect that these users are using dial-up, a lot > of the IPs are from Cox address block. What are the defaults for the > *-timeout options if you do not supply them? The timeouts only come into play when you use 'maximum-requests' option to WSGIDaemonProcess. It is defined by 'shutdown-timeout' and if not defined defaults to 5 seconds. What occurs is that when maximum requests is reached, the process starts an orderly shutdown sequence of not accepting new requests, allowing running requests to complete and then triggering atexit registered callbacks and destroying Python interpreter instances. If however the active requests do not complete within that shutdown timeout period, then the process is killed off without the cleanup occurring. Thus, a long running request can prevent restart of a daemon process occurring immediately and can also prevent proper cleanup occurring. If using 'maximum-requests' it is also a good idea to be running multiple daemon processes, because if you are only running one, a long running request can effectively stall the application for that period, with users seeing a delay. By using multiple daemon processes, when one process is restarting because maximum requests reached, other daemon processes can be handling the requests. You could still get all daemon processes restarting at same time, but chances reduce the more you have. Now, to answer the question as to why the request isn't allowed to complete, it is because I believe that is one of the issues which causes some fastcgi implementations to leave processes around and not kill them off properly. Thus mod_wsgi takes the approach of always ensuring that nothing can stall things completely or leave phantom processes around. >> Thus, long running file uploads perhaps better handled in embedded >> mode, or delegate the URLs for file upload to special daemon process >> that doesn't recycle process so as to avoid problem with it being >> interrupted. > > That would require us to have a subdomain just for this purpose, I'm > predicting this will cause the Django authentication system to fail > somehow. But I will look into it and see if it's feasible. No. You start on right track in later messages, but will answer here. <VirtualHost *> WSGIDaemonProcess default processes=5 threads=10 maximum-requests=5000 WSGIDaemonProcess uploads processes=1 threads=10 WSGIProcessGroup default WSGIScriptAlias / /some/path/app.wsgi <Locaton /my/url/for/uploads> WSGIProcessGroup uploads </Location> ... </VirtualHost> In other words, using <Location> directive with WSGIProcessGroup within it, you can distribute URLs across multiple daemon process groups with different attributes such as number of processes, maximum requests etc. The WSGIProcessGroup inside of <Location> directive will override that defined at scope of <VirtualHost> but only for matched URL prefix. Graham --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "modwsgi" group. To post to this group, send email to [email protected] To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/modwsgi?hl=en -~----------~----~----~----~------~----~------~--~---
