Re: [mod_fcgid proposal] defining processing options for particular commands
Ricardo Cantu wrote: On Friday 02 October 2009 11:10:25 am Barry Scott wrote: Jeff Trawick wrote: On Fri, Oct 2, 2009 at 5:15 AM, Barry Scott barry.sc...@onelan.co.uk mailto:barry.sc...@onelan.co.uk wrote: Jeff Trawick wrote: (instead of based on uri or vhost) FCGIDCommand /path/to/command IdleTimeout n MaxProcessLifetime n MinProcesses n MaxProcesses n MaxRequestsPerProcess n InitialEnv var[=val] ... class (the names of these options follow my proposal for the names of existing directives ;) ) When a command is to be started by mod_fcgid, any options specified for the command on this directive override those defined for the uri, vhost, global, or the defaults. When a wrapper is used, it is that wrapper which must be specified on this directive. This directive is not required unless one or more options must be customized for a command. Initially this would be allowed only in global sections. InitialEnv can be repeated. Regarding *class*: Something is needed to disable or alter existing management of applications based on their class. Currently a class is limited to the processes started by the same command within the same vhost (except when ServerName isn't specified) with the same identity. One possibility is to provide an option to ignore the vhost name when managing the class (IgnoreVHost or ClassIsGlobal). Another possibility is to set the name of the class to be used in lieu of the virtual host (ClassName foo), which could be used to the same effect but might be more useful in the future when the process manager can see per-server configs (for existing directives as well as FCGIDCommand). None of this would affect the identity checks. (Processes with different uid/gid would never be considered to be in the same class.) This seems to offer all the features of mod_fastcgi process configuration and then go usefully beyond what mod_fastcgi does. Thanks for looking. Does anyone else care to comment? Is it possible to also ask for the fcgi process to be started before any request arrive? Sure. I guess there could be some InitialProcesses n option on this directive. (If this appears to be forgotten, open a bug at https://issues.apache.org/bugzilla/ and set the severity to enhancement. Product = Apache httpd-2, component = mod_fcgid.) BTW, do you need to pre-spawn just on general principle (don't want any initial delay), or is the on-demand spawning not aggressive enough, such that it takes too long to create an adequate number of application processes? We have a setup that can be CPU time and memory limited. Using Static servers allows the start up overhead to be suffer once at boot time. Our fast CGI servers are python processes that run very fast but can be slow to start, a few seconds, which is bad for response times. So do you want a fixed number of these python processes to be pre-spawned and for the pm to stay out of the way? (never start any more or terminate any that were pre-spawned) Fixed number pre-spawned, never terminated. If any die then restart them. Barry
Re: [mod_fcgid proposal] defining processing options for particular commands
On Fri, Oct 2, 2009 at 10:47 AM, Ricardo Cantu rica...@smartcsc.com wrote: mod_fastcgi's FastCgiServer directive is similar in some respects to the one I propose in this thread, but it has a key difference: It implies that at least one instance/process will be maintained at all times, irrespective of load. This can already be done with FCGIDDefaultMinClassProcessCount. For better or for worse, MinClassProcessCount doesn't result in a pool of processes being started. It means that a process that has exceeded its idle time or process lifetime won't be terminated unless there are more than MinClassProcessCount instances. (maybe it means other stuff too) AFAICT, mod_fcgid only creates processes on-demand at present. The primary motivation for my new directive is to specify options that sometimes need to be associated with the application itself and not with the context of a particular request that the application can handle. This is perfect. Regardless, an option like InitialProcesses could be specified to pre-spawn processes. Another approach, instead of specifying something like InitialProcesses with the other options, is to call this options directive FCGIDCommandOptions, and have another directive that specifies that the command is in fact a static application (to borrow the terminology from mod_fastcgi). Whether an application is started on demand or maintained perpetually (static application), any options specified on FCGIDCommandOptions would override settings from the vhost or defaults. I like the FCGIDCommandOptions idea. There are really two concepts here that need to be addressed. FCGIDCmdOptions is soon to be committed. Initially it will provide only a different way to associate existing types of configuration with a command. One: Would you like your process pre-spawned (and how many) then controlled by the pm based on load and various other directives? Two; Would you like your process pre-spawned and left alone? (Expect for requests that need to be aborted due to hanging up the server) Hopefully this can be implemented with one directive that specifies that a certain number of copies of a command should be started, and existing settings (and maybe a new setting or two) can control which of these two flavors you get?
Re: [mod_fcgid proposal] defining processing options for particular commands
Jeff Trawick wrote: (instead of based on uri or vhost) FCGIDCommand /path/to/command IdleTimeout n MaxProcessLifetime n MinProcesses n MaxProcesses n MaxRequestsPerProcess n InitialEnv var[=val] ... class (the names of these options follow my proposal for the names of existing directives ;) ) When a command is to be started by mod_fcgid, any options specified for the command on this directive override those defined for the uri, vhost, global, or the defaults. When a wrapper is used, it is that wrapper which must be specified on this directive. This directive is not required unless one or more options must be customized for a command. Initially this would be allowed only in global sections. InitialEnv can be repeated. Regarding *class*: Something is needed to disable or alter existing management of applications based on their class. Currently a class is limited to the processes started by the same command within the same vhost (except when ServerName isn't specified) with the same identity. One possibility is to provide an option to ignore the vhost name when managing the class (IgnoreVHost or ClassIsGlobal). Another possibility is to set the name of the class to be used in lieu of the virtual host (ClassName foo), which could be used to the same effect but might be more useful in the future when the process manager can see per-server configs (for existing directives as well as FCGIDCommand). None of this would affect the identity checks. (Processes with different uid/gid would never be considered to be in the same class.) This seems to offer all the features of mod_fastcgi process configuration and then go usefully beyond what mod_fastcgi does. Is it possible to also ask for the fcgi process to be started before any request arrive? Barry
Re: [mod_fcgid proposal] defining processing options for particular commands
On Fri, Oct 2, 2009 at 5:15 AM, Barry Scott barry.sc...@onelan.co.ukwrote: Jeff Trawick wrote: (instead of based on uri or vhost) FCGIDCommand /path/to/command IdleTimeout n MaxProcessLifetime n MinProcesses n MaxProcesses n MaxRequestsPerProcess n InitialEnv var[=val] ... class (the names of these options follow my proposal for the names of existing directives ;) ) When a command is to be started by mod_fcgid, any options specified for the command on this directive override those defined for the uri, vhost, global, or the defaults. When a wrapper is used, it is that wrapper which must be specified on this directive. This directive is not required unless one or more options must be customized for a command. Initially this would be allowed only in global sections. InitialEnv can be repeated. Regarding *class*: Something is needed to disable or alter existing management of applications based on their class. Currently a class is limited to the processes started by the same command within the same vhost (except when ServerName isn't specified) with the same identity. One possibility is to provide an option to ignore the vhost name when managing the class (IgnoreVHost or ClassIsGlobal). Another possibility is to set the name of the class to be used in lieu of the virtual host (ClassName foo), which could be used to the same effect but might be more useful in the future when the process manager can see per-server configs (for existing directives as well as FCGIDCommand). None of this would affect the identity checks. (Processes with different uid/gid would never be considered to be in the same class.) This seems to offer all the features of mod_fastcgi process configuration and then go usefully beyond what mod_fastcgi does. Thanks for looking. Does anyone else care to comment? Is it possible to also ask for the fcgi process to be started before any request arrive? Sure. I guess there could be some InitialProcesses n option on this directive. (If this appears to be forgotten, open a bug at https://issues.apache.org/bugzilla/ and set the severity to enhancement. Product = Apache httpd-2, component = mod_fcgid.) BTW, do you need to pre-spawn just on general principle (don't want any initial delay), or is the on-demand spawning not aggressive enough, such that it takes too long to create an adequate number of application processes?
Re: [mod_fcgid proposal] defining processing options for particular commands
On Friday 02 October 2009 5:35:03 am Jeff Trawick wrote: On Fri, Oct 2, 2009 at 5:15 AM, Barry Scott barry.sc...@onelan.co.ukwrote: Jeff Trawick wrote: (instead of based on uri or vhost) FCGIDCommand /path/to/command IdleTimeout n MaxProcessLifetime n MinProcesses n MaxProcesses n MaxRequestsPerProcess n InitialEnv var[=val] ... class (the names of these options follow my proposal for the names of existing directives ;) ) When a command is to be started by mod_fcgid, any options specified for the command on this directive override those defined for the uri, vhost, global, or the defaults. When a wrapper is used, it is that wrapper which must be specified on this directive. This directive is not required unless one or more options must be customized for a command. Initially this would be allowed only in global sections. InitialEnv can be repeated. Regarding *class*: Something is needed to disable or alter existing management of applications based on their class. Currently a class is limited to the processes started by the same command within the same vhost (except when ServerName isn't specified) with the same identity. One possibility is to provide an option to ignore the vhost name when managing the class (IgnoreVHost or ClassIsGlobal). Another possibility is to set the name of the class to be used in lieu of the virtual host (ClassName foo), which could be used to the same effect but might be more useful in the future when the process manager can see per-server configs (for existing directives as well as FCGIDCommand). None of this would affect the identity checks. (Processes with different uid/gid would never be considered to be in the same class.) Or command name ;) This seems to offer all the features of mod_fastcgi process configuration and then go usefully beyond what mod_fastcgi does. Thanks for looking. Does anyone else care to comment? Yes, the changes you propose definitely fixes what seems to me currently a very odd setup. Is it possible to also ask for the fcgi process to be started before any request arrive? Sure. I guess there could be some InitialProcesses n option on this directive. (If this appears to be forgotten, open a bug at https://issues.apache.org/bugzilla/ and set the severity to enhancement. Product = Apache httpd-2, component = mod_fcgid.) BTW, do you need to pre-spawn just on general principle (don't want any initial delay), or is the on-demand spawning not aggressive enough, such that it takes too long to create an adequate number of application processes? From what I understand the point behind FastCgiExternalServer (process to be started before any request arrive?) is to prevent the process manager from managing that process. Only FCGIDIPCCommTimeout and FCGIDBusyTimeout would apply to InitialProcesses n. Correct me if I'm wrong, don't think it had to do with aggressiveness or delay of spawning.
Re: [mod_fcgid proposal] defining processing options for particular commands
On Fri, Oct 2, 2009 at 9:01 AM, Ricardo Cantu rica...@smartcsc.com wrote: On Friday 02 October 2009 5:35:03 am Jeff Trawick wrote: On Fri, Oct 2, 2009 at 5:15 AM, Barry Scott barry.sc...@onelan.co.uk wrote: Is it possible to also ask for the fcgi process to be started before any request arrive? Sure. I guess there could be some InitialProcesses n option on this directive. (If this appears to be forgotten, open a bug at https://issues.apache.org/bugzilla/ and set the severity to enhancement. Product = Apache httpd-2, component = mod_fcgid.) BTW, do you need to pre-spawn just on general principle (don't want any initial delay), or is the on-demand spawning not aggressive enough, such that it takes too long to create an adequate number of application processes? From what I understand the point behind FastCgiExternalServer (process to be started before any request arrive?) is to prevent the process manager from managing that process. Only FCGIDIPCCommTimeout and FCGIDBusyTimeout would apply to InitialProcesses n. Correct me if I'm wrong, don't think it had to do with aggressiveness or delay of spawning. My understanding is that mod_fastcgi's FastCgiExternalServer is for processes that mod_fastcgi doesn't manage at all. It will route requests to them but it can't start or stop them. I think that matches the only support in httpd trunk's mod_proxy_fcgi. mod_fastcgi's FastCgiServer directive is similar in some respects to the one I propose in this thread, but it has a key difference: It implies that at least one instance/process will be maintained at all times, irrespective of load. The primary motivation for my new directive is to specify options that sometimes need to be associated with the application itself and not with the context of a particular request that the application can handle. Regardless, an option like InitialProcesses could be specified to pre-spawn processes. Another approach, instead of specifying something like InitialProcesses with the other options, is to call this options directive FCGIDCommandOptions, and have another directive that specifies that the command is in fact a static application (to borrow the terminology from mod_fastcgi). Whether an application is started on demand or maintained perpetually (static application), any options specified on FCGIDCommandOptions would override settings from the vhost or defaults.
Re: [mod_fcgid proposal] defining processing options for particular commands
On Friday 02 October 2009 8:14:14 am Jeff Trawick wrote: On Fri, Oct 2, 2009 at 9:01 AM, Ricardo Cantu rica...@smartcsc.com wrote: On Friday 02 October 2009 5:35:03 am Jeff Trawick wrote: On Fri, Oct 2, 2009 at 5:15 AM, Barry Scott barry.sc...@onelan.co.uk wrote: Is it possible to also ask for the fcgi process to be started before any request arrive? Sure. I guess there could be some InitialProcesses n option on this directive. (If this appears to be forgotten, open a bug at https://issues.apache.org/bugzilla/ and set the severity to enhancement. Product = Apache httpd-2, component = mod_fcgid.) BTW, do you need to pre-spawn just on general principle (don't want any initial delay), or is the on-demand spawning not aggressive enough, such that it takes too long to create an adequate number of application processes? From what I understand the point behind FastCgiExternalServer (process to be started before any request arrive?) is to prevent the process manager from managing that process. Only FCGIDIPCCommTimeout and FCGIDBusyTimeout would apply to InitialProcesses n. Correct me if I'm wrong, don't think it had to do with aggressiveness or delay of spawning. My understanding is that mod_fastcgi's FastCgiExternalServer is for processes that mod_fastcgi doesn't manage at all. It will route requests to them but it can't start or stop them. I think that matches the only support in httpd trunk's mod_proxy_fcgi. Your right it doesn't start or stop the process but does manage request handling. mod_fastcgi's FastCgiExternalServer accepts the following options: -appConnTimeout and -idle-timeout. Rather than terminating the process on FCGIDIPCCommTimeout and FCGIDBusyTimeout we would abort the request and log it. mod_fastcgi's FastCgiServer directive is similar in some respects to the one I propose in this thread, but it has a key difference: It implies that at least one instance/process will be maintained at all times, irrespective of load. This can already be done with FCGIDDefaultMinClassProcessCount. The primary motivation for my new directive is to specify options that sometimes need to be associated with the application itself and not with the context of a particular request that the application can handle. This is perfect. Regardless, an option like InitialProcesses could be specified to pre-spawn processes. Another approach, instead of specifying something like InitialProcesses with the other options, is to call this options directive FCGIDCommandOptions, and have another directive that specifies that the command is in fact a static application (to borrow the terminology from mod_fastcgi). Whether an application is started on demand or maintained perpetually (static application), any options specified on FCGIDCommandOptions would override settings from the vhost or defaults. I like the FCGIDCommandOptions idea. There are really two concepts here that need to be addressed. One: Would you like your process pre-spawned (and how many) then controlled by the pm based on load and various other directives? Two; Would you like your process pre-spawned and left alone? (Expect for requests that need to be aborted due to hanging up the server)
Re: [mod_fcgid proposal] defining processing options for particular commands
Jeff Trawick wrote: On Fri, Oct 2, 2009 at 5:15 AM, Barry Scott barry.sc...@onelan.co.uk mailto:barry.sc...@onelan.co.uk wrote: Jeff Trawick wrote: (instead of based on uri or vhost) FCGIDCommand /path/to/command IdleTimeout n MaxProcessLifetime n MinProcesses n MaxProcesses n MaxRequestsPerProcess n InitialEnv var[=val] ... class (the names of these options follow my proposal for the names of existing directives ;) ) When a command is to be started by mod_fcgid, any options specified for the command on this directive override those defined for the uri, vhost, global, or the defaults. When a wrapper is used, it is that wrapper which must be specified on this directive. This directive is not required unless one or more options must be customized for a command. Initially this would be allowed only in global sections. InitialEnv can be repeated. Regarding *class*: Something is needed to disable or alter existing management of applications based on their class. Currently a class is limited to the processes started by the same command within the same vhost (except when ServerName isn't specified) with the same identity. One possibility is to provide an option to ignore the vhost name when managing the class (IgnoreVHost or ClassIsGlobal). Another possibility is to set the name of the class to be used in lieu of the virtual host (ClassName foo), which could be used to the same effect but might be more useful in the future when the process manager can see per-server configs (for existing directives as well as FCGIDCommand). None of this would affect the identity checks. (Processes with different uid/gid would never be considered to be in the same class.) This seems to offer all the features of mod_fastcgi process configuration and then go usefully beyond what mod_fastcgi does. Thanks for looking. Does anyone else care to comment? Is it possible to also ask for the fcgi process to be started before any request arrive? Sure. I guess there could be some InitialProcesses n option on this directive. (If this appears to be forgotten, open a bug at https://issues.apache.org/bugzilla/ and set the severity to enhancement. Product = Apache httpd-2, component = mod_fcgid.) BTW, do you need to pre-spawn just on general principle (don't want any initial delay), or is the on-demand spawning not aggressive enough, such that it takes too long to create an adequate number of application processes? We have a setup that can be CPU time and memory limited. Using Static servers allows the start up overhead to be suffer once at boot time. Our fast CGI servers are python processes that run very fast but can be slow to start, a few seconds, which is bad for response times. We have also had data collection going on in the fast CGI process, but we are moving away from that for a number of reasons. Barry
Re: [mod_fcgid proposal] defining processing options for particular commands
On Friday 02 October 2009 11:10:25 am Barry Scott wrote: Jeff Trawick wrote: On Fri, Oct 2, 2009 at 5:15 AM, Barry Scott barry.sc...@onelan.co.uk mailto:barry.sc...@onelan.co.uk wrote: Jeff Trawick wrote: (instead of based on uri or vhost) FCGIDCommand /path/to/command IdleTimeout n MaxProcessLifetime n MinProcesses n MaxProcesses n MaxRequestsPerProcess n InitialEnv var[=val] ... class (the names of these options follow my proposal for the names of existing directives ;) ) When a command is to be started by mod_fcgid, any options specified for the command on this directive override those defined for the uri, vhost, global, or the defaults. When a wrapper is used, it is that wrapper which must be specified on this directive. This directive is not required unless one or more options must be customized for a command. Initially this would be allowed only in global sections. InitialEnv can be repeated. Regarding *class*: Something is needed to disable or alter existing management of applications based on their class. Currently a class is limited to the processes started by the same command within the same vhost (except when ServerName isn't specified) with the same identity. One possibility is to provide an option to ignore the vhost name when managing the class (IgnoreVHost or ClassIsGlobal). Another possibility is to set the name of the class to be used in lieu of the virtual host (ClassName foo), which could be used to the same effect but might be more useful in the future when the process manager can see per-server configs (for existing directives as well as FCGIDCommand). None of this would affect the identity checks. (Processes with different uid/gid would never be considered to be in the same class.) This seems to offer all the features of mod_fastcgi process configuration and then go usefully beyond what mod_fastcgi does. Thanks for looking. Does anyone else care to comment? Is it possible to also ask for the fcgi process to be started before any request arrive? Sure. I guess there could be some InitialProcesses n option on this directive. (If this appears to be forgotten, open a bug at https://issues.apache.org/bugzilla/ and set the severity to enhancement. Product = Apache httpd-2, component = mod_fcgid.) BTW, do you need to pre-spawn just on general principle (don't want any initial delay), or is the on-demand spawning not aggressive enough, such that it takes too long to create an adequate number of application processes? We have a setup that can be CPU time and memory limited. Using Static servers allows the start up overhead to be suffer once at boot time. Our fast CGI servers are python processes that run very fast but can be slow to start, a few seconds, which is bad for response times. So do you want a fixed number of these python processes to be pre-spawned and for the pm to stay out of the way? (never start any more or terminate any that were pre-spawned) We have also had data collection going on in the fast CGI process, but we are moving away from that for a number of reasons. Barry -- Computer Services Ricardo Cantu Vice President Home office 3506 Buchanan St Suite C Wichita Falls, TX 76308 (940) 696-3010 El Paso branch 1644 Ronnie Reif Dr. El Paso, TX 79936 (915) 219-7119
[mod_fcgid proposal] defining processing options for particular commands
(instead of based on uri or vhost) FCGIDCommand /path/to/command IdleTimeout n MaxProcessLifetime n MinProcesses n MaxProcesses n MaxRequestsPerProcess n InitialEnv var[=val] ... class (the names of these options follow my proposal for the names of existing directives ;) ) When a command is to be started by mod_fcgid, any options specified for the command on this directive override those defined for the uri, vhost, global, or the defaults. When a wrapper is used, it is that wrapper which must be specified on this directive. This directive is not required unless one or more options must be customized for a command. Initially this would be allowed only in global sections. InitialEnv can be repeated. Regarding *class*: Something is needed to disable or alter existing management of applications based on their class. Currently a class is limited to the processes started by the same command within the same vhost (except when ServerName isn't specified) with the same identity. One possibility is to provide an option to ignore the vhost name when managing the class (IgnoreVHost or ClassIsGlobal). Another possibility is to set the name of the class to be used in lieu of the virtual host (ClassName foo), which could be used to the same effect but might be more useful in the future when the process manager can see per-server configs (for existing directives as well as FCGIDCommand). None of this would affect the identity checks. (Processes with different uid/gid would never be considered to be in the same class.)
