So, predictably, now that we've gotten mod_proxy_fcgi to the point where it's actualy able to run real applications I'm starting to question some basic assumptions we made when we started out along this course.
The general idea was that we want to be able to get content from some fastcgi processes. That seems pretty similar to what mod_proxy_http does with other http servers, and mod_proxy_ajp with java app servers, and heck, since we're probably going to have lots of back end fastcgi processes it sure is cool that we've got that mod_proxy_balancer stuff to handle that part of the equation. It sure seems like a good idea, doesn't it? And at first glance it is, I mean it basically works, I can set up a balancer group with a bunch of back end fastcgi processes that I started up with the new fcgistarter program, and it'll pretty much do what we want. But there are some issues looming on the horizon. First of all, mod_proxy_balancer really assumes that you can make multiple connections to back end fastcgi processes at once. This may be true for some things that speak fastcgi (python programs that use flup to do it sure look like they'd work for that sort of thing, but I haven't really tried it yet), but in general the vast majority of fastcgi programs are single threaded, non-asynchronous, and are designed to process exactly one connection at a time. This is sort of a problem because mod_proxy_balancer doesn't actually have any mechanism for coordinating between the various httpd processes about who is using what backend process. Second, mod_proxy_balancer doesn't (seem to) have any real mechanism for adding back end processes on the fly, which is something that would be really nice to be able to do. I'd eventually love to be able to tell mod_proxy_fcgi that it should start up N back end processes at startup, and create up to M more if needed at any given time. Processes should be able to be killed off if they become nonresponsive (or probably after processing a certain number of requests), and they should NOT be bound up to a single httpd worker process. This all means that some kind of mechanism for coordinating access to and creation of back end processes needs to be created, and as it moves on it starts to feel less and less like this sort of functionality is generically useful to other back end fastcgi processes. Maybe I'm wrong about that though. Oh, and in order to do any of the really cool stuff we'll also have to rework the way mod_proxy handles arguments that are given to ProxyPass statements, so that they can be passed down to something other than either mod_proxy or mod_proxy_balancer. And even after we do that, we'll still be stuck in this situation where you end up with like a bazillion options on the end of each fastcgi ProxyPass, when really we'd want them to be per-balancer or per-directory or something like that. It just feels kinda clunky. Finally, I have to say that I'm starting to wonder what we're actually getting out of using the proxy framework for this. I mean all it's doing is creating some sockets for us, all the other stuff I just talked about pretty much needs to be implemented itself, and it's questionable whether any of it would be useful for something other than the fastcgi code. So is there some reason I'm missing that justifies staying within the proxy framework, cause I'm really tempted to just create a handler module that reuses most of the mod_proxy_fcgi code, since it sure feels like it'd be easier to write this stuff if I didn't have to shoehorn it into mod_proxy. -garrett
