Greetings, Programs! Here are a couple-three things I think would make a big difference for using Rivet to build and deploy big websites.
1. Named separate virtual interpreters. We use separate virtual interpreters for development (for each developer we diddle auto_path to give them source-controlled private copies of all of our packages) and it's really badass. The problem comes in the all-or-nothing approach of SVI. We set up a virtual host for each developer, for both port 80 (well really 8080 + varnish on 80) and 443. This results in 18 interpreters in each httpd process on our development machine, making for very large httpd processes and slow startup time after a graceful. If we could name the virtual interpreters, we could cut the number of interpreters from 18 to 9 in this case. 2. Virtual interpreter restarts without a graceful Right now if one of our developers changes a package, private to them, they still have to do an apachectl graceful to pick up the change. This restarts all of the httpd processes and reinitializes all of the interpreters. Our interpreter initialization is intense. Each FlightAware httpd process loads 468 packages. I'd like to be able to cause only one vhost's Tcl interpreters to be reloaded by a Tcl_DeleteInterp / Tcl_CreateInterp / Rivet initialization process. Instead of a graceful, you'd be able to specify something like a trigger file for each vhost. Every time a vhost (with separate virtual interpreters) serves a page, it gets the mtime of the trigger file. If the mtime of the trigger file has changed since the last time the interpreter served a page, Rivet deletes the virtual host's interpreter, creates and initializes a new one, and then handles the page. [I tried to write this but kind of lost control of it and was not successful.] This way, developers could totally reload all their libraries without any httpd processes being stopped or started. Also this will lower overall overhead because a lot of times a httpd process won't have ever handled a page in its lifetime for many to most of the virtual interpreters. An additional improvement would be to create the ability to not even initialize a vhost's separate virtual interpreter until the first time it is needed. 3. Apache children inheriting a preloaded interpreter from the parent Apache process This is unrelated to separate virtual interpreters -- this is for the production website. When we graceful a production webserver we take it out of the load balancer first. When we start 200 httpd processes and each one loads 468 packages, it's not pretty. There is a lot of lock contention in FreeBSD while these processes all bang on the same package directories, something we don't totally understand. (This is much faster with ZFS, once tuned, btw.) It takes minutes to settle down before our stuff can put the server back into the webserver. What would be incredibly cool would be to be able to load up the 468 packages in the parent httpd process one time and then have each child process use the same Tcl interpreter already loaded with the packages. This would be a way more than 200-fold improvement in Apache startup time for us (because it would eliminate all the contention.) [I don't even know if this is possible.] I offer this because it's like a possible direction to take Rivet development. I can help, definitely, and provide production load :-), if anyone is interested in taking any of these ideas to further design, code, etc, please let me know. Karl
