Re: [mod_fcgid] how about spin lock on share memory
pqf wrote: Hi, all I am Ryan Pan, who wrote the first version of mod_fcgid. While I uesd mod_fastcgi(not mod_fcgid), one issue that bother me is: while a fastcgi process(created by mod_fastcgi's process manager process)in a dead loop, no one is respond to kick it out. So from time to time, some fastcgi processes would eat up the system cpu resource. So while I wrote mod_fcgid, I create a block of share memory to store the fastcgi process pipe path and pid, then httpd can search this share memory to get the an idle fastcgi process, and once the communication timout(which usually mean the fastcgi process deadly running), httpd will kick out this corrupt process. However there is a new problem now, every time httpd search this share memory, it will have to get a global mutex, which is a combination of process lock an thread lock, is it(a mutex lock for search a free node in a node list) too heavy? Maybe spin lock on share memory is a good idea in this case? But spin lock is system dependented, and apr library doesn't have this interface. I thought about this idea since I wrote mod_fcgid, but I am not sure whether it is a good idea, so any advice from you guys will be highly appreciated. I'd suggest use a mutex until the point that a problem is seen in practice. I doubt that the mutex will show up in any profiling of a request using mod_fcgid. Barry
Re: [mod_fcgid] how about spin lock on share memory
2009/10/13 pqf p...@mailtech.cn: Hi, all I am Ryan Pan, who wrote the first version of mod_fcgid. While I uesd mod_fastcgi(not mod_fcgid), one issue that bother me is: while a fastcgi process(created by mod_fastcgi's process manager process)in a dead loop, no one is respond to kick it out. So from time to time, some fastcgi processes would eat up the system cpu resource. So while I wrote mod_fcgid, I create a block of share memory to store the fastcgi process pipe path and pid, then httpd can search this share memory to get the an idle fastcgi process, and once the communication timout(which usually mean the fastcgi process deadly running), httpd will kick out this corrupt process. However there is a new problem now, every time httpd search this share memory, it will have to get a global mutex, which is a combination of process lock an thread lock, is it(a mutex lock for search a free node in a node list) too heavy? Maybe spin lock on share memory is a good idea in this case? But spin lock is system dependented, and apr library doesn't have this interface. I thought about this idea since I wrote mod_fcgid, but I am not sure whether it is a good idea, so any advice from you guys will be highly appreciated. The thought of a pure spin lock for interruptible code makes me nervous (what if you lose the CPU holding the lock; other threads/processes that need it waste their timeslice until you finish; thread priorities could help, but not across processes), but I don't have the right experience in this area. The easy answer: Crank up the load until the current implementation causes a real performance problem, then experiment with spin locks at the same load ;) Stray thoughts: maybe increasing the granularity of the lock could help (multiple busy lists with the inode used as a hash to get to the proper busy list) different mutexes for different lists (but the usage of some lists may be so simple that it isn't worth dropping the first mutex then getting another one) maybe use compare-swap or compare-double-swap as appropriate for some lists (e.g., handler thread adds to error list using atomic operation, not holding mutex; PM removes entire error list using atomic operation without getting global mutex, then processes one by one)
Re: [mod_fcgid] how about spin lock on share memory
Jeff Trawick wrote: maybe increasing the granularity of the lock could help (multiple busy lists with the inode used as a hash to get to the proper busy list) I happen to have a module, unrelated to mod_fcgid, which manages a fairly large shared-memory cache across a number of user sessions. Over the years I ended up with an implementation that divides it into 32 subcaches and gives each one a global lock; sessions are spread across the subcaches randomly. It may not be elegant but it works well under a pretty high load. Because this runs on Linux and we're just using the APR defaults, the process locks are SysV semaphores. Maybe sometime in the far future, once glibc 2.10 appears on our systems, we might try APR_LOCK_PROC_PTHREAD and see if we can't get the shiny new pthread_mutexattr_setrobust() support into APR. (As a side note, anyone know if this has come up yet in the APR world, maybe for a recent Fedora or something? I'm not sure what distros might have glibc 2.10 in them at all.) Anyhoo, the other thing I've pondered over the years but never tried would be to hack APR's SysV semaphore lock code to permit multiple semaphores in a set -- right now it's just a semget(..., 1, ...) but it would clean up our ipcs output a lot if we could pack those 32 semaphores into one set instead of having just one each in its own set, and especially so if we ever needed more than 32. Well, none of that is related to what I'm supposed to be doing this week (i.e., breaking out my old fcgid patches), nor httpd, really, but it's maybe mildly interesting to someone, or not, as the case may be. :-/ OK, back to work. Chris. -- GPG Key ID: 366A375B GPG Key Fingerprint: 485E 5041 17E1 E2BB C263 E4DE C8E3 FA36 366A 375B
Re: [mod_fcgid] how about spin lock on share memory
Chris Darroch wrote: Because this runs on Linux and we're just using the APR defaults, the process locks are SysV semaphores. Maybe sometime in the far future, once glibc 2.10 appears on our systems, we might try APR_LOCK_PROC_PTHREAD and see if we can't get the shiny new pthread_mutexattr_setrobust() support into APR. (As a side note, anyone know if this has come up yet in the APR world, maybe for a recent Fedora or something? I'm not sure what distros might have glibc 2.10 in them at all.) Ah ... I see that pthread_mutexattr_setrobust_np() and friends are actually in glibc as of about 2.5 or so; I went looking for the POSIX 2008 names and managed to confuse myself. And for that matter, I still can't tell if glibc 2.10 does support the new names (without the _np) or not. Oh well. Chris. -- GPG Key ID: 366A375B GPG Key Fingerprint: 485E 5041 17E1 E2BB C263 E4DE C8E3 FA36 366A 375B
[mod_fcgid] how about spin lock on share memory
Hi, all I am Ryan Pan, who wrote the first version of mod_fcgid. While I uesd mod_fastcgi(not mod_fcgid), one issue that bother me is: while a fastcgi process(created by mod_fastcgi's process manager process)in a dead loop, no one is respond to kick it out. So from time to time, some fastcgi processes would eat up the system cpu resource. So while I wrote mod_fcgid, I create a block of share memory to store the fastcgi process pipe path and pid, then httpd can search this share memory to get the an idle fastcgi process, and once the communication timout(which usually mean the fastcgi process deadly running), httpd will kick out this corrupt process. However there is a new problem now, every time httpd search this share memory, it will have to get a global mutex, which is a combination of process lock an thread lock, is it(a mutex lock for search a free node in a node list) too heavy? Maybe spin lock on share memory is a good idea in this case? But spin lock is system dependented, and apr library doesn't have this interface. I thought about this idea since I wrote mod_fcgid, but I am not sure whether it is a good idea, so any advice from you guys will be highly appreciated. Thanks