On 26/06/07, Robert Brewer <[EMAIL PROTECTED]> wrote: > I'd like to continue talking about standardization on site-wide process > state and services. > > As described more fully on my blog [1], I'm proposing we create a new > spec for a simple publish-subscribe Bus to manage site-wide state > transitions (start, stop, etc.) and connect application components with > site-wide services (daemonize, autoreload, site logging, etc).
I'm a bit too busy to fully absorb all that is being said on this topic, but have a few comments or observations to make based on my experiences. If they relate to anything, it is possibly just to expand on some of the issues around why a bus might be a reasonable requirement to have so hopefully it is of use in that respect. First comment is about WSGI applications somehow themselves using SIGTERM etc as triggers for things they want to do. For Apache at least, allowing any part of a hosted Python application to register its own signal handlers is a big no no. This is because Apache itself uses a whole range of signals to manage such tasks as shutting down sub processes or signaling worker and/or listener threads within a process that its time to wakeup or shutdown. If a WSGI application starts registering signal handlers it can as a result stop Apache from even being able to process requests. In mod_wsgi I have had to specifically take steps to prevent applications breaking things in this way by replacing signal.signal() on creation of an interpreter. Instead I log a warning that the signal registration has been ignored and otherwise do nothing. This was simply the safest thing to do. Thus I believe a clear statement should be made that UNIX signals are off limits to WSGI applications or components. I have no problem with the underlying server upon which the WSGI adapter sits using them for its own purposes, however, knowledge of signals should stay in that layer. Now it may be the case that a higher level application signal may be generated as a result of a SIGTERM in a specific server implementation, for example shutdown, but the name or type of signal does have to be distinct, ie., "shutdown" and have no linkage to the UNIX signal. In Apache for example, SIGTERM isn't used in child processes to indicate process shutdown, that is instead signalled via what is called the pipe of death. The trigger for process shutdown in this case is actually the destruction of the child process memory pool. I know that Robert's diagram shows the translation of UNIX signal at a lower level to higher level concept, but then elsewhere there was mention of SIGUSR signals. Apache even uses these in some configurations so a hosted user application wouldn't even be able to use them. Anyway, just wanted to make it absolutely clear that I don't believe a hosted WSGI application and associated framework has any business taking direct interest in low level UNIX signals. Next issue was SystemExit exception, raised explicitly or via sys.exit(), and the ability of a WSGI application to itself shutdown the server. Again, shutting down the server using SystemExit is something I see as being off limits to a WSGI application. In mod_wsgi I explicitly ignore SystemExit raised by a WSGI application when handling a request, more or less treating it like any other unexpected exception. This is necessary because within Apache one can't tolerate one application making a decision to shutdown a process because of the impacts it may have on other application code, possibly non Python code, running in the same process. For a Python based web server, also believe that SystemExit in a request handler should be ignored. Obviously though the underlying server itself is free to use its for its own purpose. I actually don't know what other WSGI adapters do about SystemExit, so would be interested to learn what others do. One thing I have sometimes thought about though is whether there is any valid case where a WSGI application should itself be allowed to request the application be shutdown. I think I'd probably have to say no for the general case. But then you have the case of trying to reload an application when the code changes. In this case it may be reasonable because no in process module reloading scheme is ever going to be good enough. The closest one can get doing it in process is to destroy the whole sub interpreter. This though is only a practical thing for something like mod_wsgi and even then doesn't always work because of C extension modules, eg PyProtocols, that don't take kindly to sub interpreters being destroyed and the module then being reloaded into a new sub interpreter. Thus, purely to make administration easier for the WSGI application to request an application reload, perhaps triggered by user through a web page, one might want to signal process shutdown. Even then though it isn't that simple as that, as there has to be a compatible supervisor system in place which will restart the process. In mod_wsgi if using it in embedded mode whereby application runs in Apache child processes, an application requesting a reload like this is not viable as not possible to tell the Apache parent process to restart everything. Even when using daemon mode of mod_wsgi where application runs in separate processes, only the simple case where there is one daemon process for that application is it trivial to initiate a reload. When there are multiple processes in the group, need to somehow signal the other processes in the group to also reload. So that is my brain dump. I guess I'll see where you guys take the discussion. At the moment I am guarded as to whether some of the ideas of how this bus could be used would in practice be transferable. Certainly with Apache it may be quite challenging because of the multiprocess nature of the server and the fact that child processes don't really have any way of controlling the supervisor (parent Apache process). Graham _______________________________________________ Web-SIG mailing list Web-SIG@python.org Web SIG: http://www.python.org/sigs/web-sig Unsubscribe: http://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com