Re: [mod_fcgid] how about spin lock on share memory
pqf wrote: > Hi, all > I am Ryan Pan, who wrote the first version of mod_fcgid. > While I uesd mod_fastcgi(not mod_fcgid), one issue that bother me is: while > a fastcgi process(created by mod_fastcgi's process manager process)in a dead > loop, no one is respond to kick it out. So from time to time, some fastcgi > processes would eat up the system cpu resource. > So while I wrote mod_fcgid, I create a block of share memory to store the > fastcgi process pipe path and pid, then httpd can search this share memory to > get the an idle fastcgi process, and once the communication timout(which > usually mean the fastcgi process deadly running), httpd will kick out this > "corrupt" process. > However there is a new problem now, every time httpd search this share > memory, it will have to get a global mutex, which is a combination of process > lock an thread lock, is it(a mutex lock for search a free node in a node > list) too heavy? Maybe spin lock on share memory is a good idea in this case? > But spin lock is system dependented, and apr library doesn't have this > interface. > I thought about this idea since I wrote mod_fcgid, but I am not sure > whether it is a good idea, so any advice from you guys will be highly > appreciated. > I'd suggest use a mutex until the point that a problem is seen in practice. I doubt that the mutex will show up in any profiling of a request using mod_fcgid. Barry
Re: [mod_fcgid] how about spin lock on share memory
2009/10/13 pqf : > Hi, all > I am Ryan Pan, who wrote the first version of mod_fcgid. > While I uesd mod_fastcgi(not mod_fcgid), one issue that bother me is: while > a fastcgi process(created by mod_fastcgi's process manager process)in a dead > loop, no one is respond to kick it out. So from time to time, some fastcgi > processes would eat up the system cpu resource. > So while I wrote mod_fcgid, I create a block of share memory to store the > fastcgi process pipe path and pid, then httpd can search this share memory to > get the an idle fastcgi process, and once the communication timout(which > usually mean the fastcgi process deadly running), httpd will kick out this > "corrupt" process. > However there is a new problem now, every time httpd search this share > memory, it will have to get a global mutex, which is a combination of process > lock an thread lock, is it(a mutex lock for search a free node in a node > list) too heavy? Maybe spin lock on share memory is a good idea in this case? > But spin lock is system dependented, and apr library doesn't have this > interface. > I thought about this idea since I wrote mod_fcgid, but I am not sure whether > it is a good idea, so any advice from you guys will be highly appreciated. The thought of a pure spin lock for interruptible code makes me nervous (what if you lose the CPU holding the lock; other threads/processes that need it waste their timeslice until you finish; thread priorities could help, but not across processes), but I don't have the right experience in this area. The easy answer: Crank up the load until the current implementation causes a real performance problem, then experiment with spin locks at the same load ;) Stray thoughts: maybe increasing the granularity of the lock could help (multiple busy lists with the inode used as a hash to get to the proper busy list) different mutexes for different lists (but the usage of some lists may be so simple that it isn't worth dropping the first mutex then getting another one) maybe use compare-swap or compare-double-swap as appropriate for some lists (e.g., handler thread adds to error list using atomic operation, not holding mutex; PM removes entire error list using atomic operation without getting global mutex, then processes one by one)
Dav Locks corrupting; need some troubleshooting tools
Hi all I'm on apache 2.0.63, and seeing nonsensical Dav Locks (symptoms are that files show as being locked by users that couldn't possibly have locked them). I *think* I've tracked it down to a bug in mod_dav_fs; it appears that if files are deleted from disk (by a mechanism other than Dav), any locks that were taken out on that file 'jump' so that they apply to a different file altogether. Are there any tools that let me inspect the content of DavLockDB? And who's a good person to talk to if I need to pick over the code?
Re: [mod_fcgid] how about spin lock on share memory
Jeff Trawick wrote: maybe increasing the granularity of the lock could help (multiple busy lists with the inode used as a hash to get to the proper busy list) I happen to have a module, unrelated to mod_fcgid, which manages a fairly large shared-memory cache across a number of user sessions. Over the years I ended up with an implementation that divides it into 32 subcaches and gives each one a global lock; sessions are spread across the subcaches randomly. It may not be elegant but it works well under a pretty high load. Because this runs on Linux and we're just using the APR defaults, the process locks are SysV semaphores. Maybe sometime in the far future, once glibc 2.10 appears on our systems, we might try APR_LOCK_PROC_PTHREAD and see if we can't get the shiny new pthread_mutexattr_setrobust() support into APR. (As a side note, anyone know if this has come up yet in the APR world, maybe for a recent Fedora or something? I'm not sure what distros might have glibc 2.10 in them at all.) Anyhoo, the other thing I've pondered over the years but never tried would be to hack APR's SysV semaphore lock code to permit multiple semaphores in a set -- right now it's just a semget(..., 1, ...) but it would clean up our ipcs output a lot if we could pack those 32 semaphores into one set instead of having just one each in its own set, and especially so if we ever needed more than 32. Well, none of that is related to what I'm supposed to be doing this week (i.e., breaking out my old fcgid patches), nor httpd, really, but it's maybe mildly interesting to someone, or not, as the case may be. :-/ OK, back to work. Chris. -- GPG Key ID: 366A375B GPG Key Fingerprint: 485E 5041 17E1 E2BB C263 E4DE C8E3 FA36 366A 375B
Re: [mod_fcgid] how about spin lock on share memory
Chris Darroch wrote: Because this runs on Linux and we're just using the APR defaults, the process locks are SysV semaphores. Maybe sometime in the far future, once glibc 2.10 appears on our systems, we might try APR_LOCK_PROC_PTHREAD and see if we can't get the shiny new pthread_mutexattr_setrobust() support into APR. (As a side note, anyone know if this has come up yet in the APR world, maybe for a recent Fedora or something? I'm not sure what distros might have glibc 2.10 in them at all.) Ah ... I see that pthread_mutexattr_setrobust_np() and friends are actually in glibc as of about 2.5 or so; I went looking for the POSIX 2008 names and managed to confuse myself. And for that matter, I still can't tell if glibc 2.10 does support the new names (without the _np) or not. Oh well. Chris. -- GPG Key ID: 366A375B GPG Key Fingerprint: 485E 5041 17E1 E2BB C263 E4DE C8E3 FA36 366A 375B
Fix for CVE-2008-2364 for httpd 2.0.64-dev
Is there any chance the fix for security vulnerability CVE-2008-2364 ( http://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2008-2364) will be back ported to Apache 2.0.x? The reason I ask is because this particular security vulnerability is missing from the Apache 2.0 list of security vulnerabilities (http://httpd.apache.org/security/vulnerabilities_20.html ). Someone did however point me to following location ( http://www.apache.org/dist/httpd/patches/apply_to_2.0.63/) for an Apache 2.0.x patch.
Re: Cannot compile mod_disk_cache.c (rev. 821993)
Hi! I have applied patches to the 2.2.14 sources. I've copied mod_cache.so and mod_disk_cache.so to the apache modules directory. The problem is NOT solved. As soon as I add CacheQuickHandler Off to the cache configuration, the servers stops caching (new directories aren't appearing in cache folder). Please advise. Thank you. Graham Leggett писал(а) в своём письме Fri, 09 Oct 2009 15:36:56 +0300: Alexander Alfimov wrote: I assume I can apply patches to this code: http://svn.apache.org/repos/asf/httpd/httpd/branches/2.2.x/modules/cache/ Apply the patches to the base of httpd v2.2.14, like this: cd httpd-2.2.14 cat .patch | patch -p0 Regards, Graham -- -- Best Regards, Alexander Alfimov
Re: Cannot compile mod_disk_cache.c (rev. 821993)
I forgot to add that I have a ProxyPass directive in the same config. I.e. I'm trying to build caching proxy with mod_rewrite on the "front". It looks like mod_proxy is run before the mod_cache. Please advise. Thank you. Alexander Alfimov писал(а) в своём письме Thu, 15 Oct 2009 01:47:30 +0300: Hi! I have applied patches to the 2.2.14 sources. I've copied mod_cache.so and mod_disk_cache.so to the apache modules directory. The problem is NOT solved. As soon as I add CacheQuickHandler Off to the cache configuration, the servers stops caching (new directories aren't appearing in cache folder). Please advise. Thank you. Graham Leggett писал(а) в своём письме Fri, 09 Oct 2009 15:36:56 +0300: Alexander Alfimov wrote: I assume I can apply patches to this code: http://svn.apache.org/repos/asf/httpd/httpd/branches/2.2.x/modules/cache/ Apply the patches to the base of httpd v2.2.14, like this: cd httpd-2.2.14 cat .patch | patch -p0 Regards, Graham -- -- Best Regards, Alexander Alfimov