On Dec 21, 2005, at 1:26 PM, Jonathan wrote:
Twisted does all network I/O multiplexed in one thread by way of something like select, poll, epoll, kqueue, WaitForMultipleEvents, or whatever other model is implemented by that reactor. You can use as many other threads as you want for doing whatever else you want, but if you will get callbacks on the reactor thread and you must register callbacks on the reactor thread. This model has been shown to scale extremely well (in any language). Python's GIL gets released by most code that does a lot of work or waits on a resource in C. This means that when you're doing select(), a database call, etc. other threads get a chance to run. Python bytecode runs in separate OS threads, but the GIL causes them to run round-robin. This does two things: it makes it relatively easy to write threaded programs in Python, and it saves on memory as you only need a threadstate data structure per thread, not a whole interpreter. Interpreter-per-processor is of course the most scalable model in either language, because then you can run bytecode in parallel. It's most portable and probably easiest to do this in separate processes rather than threads, especially considering that the multiprocess model can be made to scale across machines too. I don't think any of this is really relevant to mod_perl though, it's bound to the way Apache works. -bob |
- Re: Solving mod_perl scalability issues for lots of slow conn... Bob Ippolito