Re: [PATCH] event driven read
On Tue, 2002-10-15 at 10:31, Bill Stoddard wrote: > Something I've been hacking on (in the pejorative sense of the word 'hack'. > Look at the patch and you will see what I mean :-). This should apply and > serve pages on Linux, though the event_loop is clearly broken as it does not > timeout keep-alive connections and will hang on the apr_poll() (and hang the > server) if a client leaves a keep-alive connection active but does not send > anything on it. Scoreboard is broken, code structure is poor, yadda yadda. I > plan to reimplement some of this more cleanly but no idea when I'll get > around to it. Key points: > > 1. routines to read requests must be able to handl getting APR_EAGAIN (or > APR_EWOULDBLOCK): > 2. ap_process_http_connection reimplemented to be state driven > 3. event loop in worker_mpm to wait for pending i/o Revisiting this topic from a looong time ago... Did you ever get a chance to do any further work on this design? IMHO, for 2.1/2.2 we should at least incorporate the changes to allow data to be pushed rather than pulled through the input filter chain. That will set the foundation for an event-driven MPM, even if we're still using a blocking, thread-per-connection worker design in the first 2.2 release. Brian
Re: Request body validation before content handler phase
On Fri, 12 Sep 2003, Kris Verbeeck wrote: > I'm writing an Apache 2.0 module that has to perform some > request body validation before actually deciding which > content handler to use for response generation. Are these your own content handlers? If so, could your module do its dispatching internally, or make them into output filters, or any such thing? > 3) something else? The more generic approach would be to run the selected handler in a 1.3-style subrequest. BTW, please don't post a 5.9K sig to a mailinglist. BTW2, this might be better-suited to apache-modules or usenet than dev. -- Nick Kew
RE: Hooking child termination
> From: William A. Rowe, Jr. > > > >Is there a possibility to add some sort of a hook to a > parent that will > >execute when the child process dies? I would like to have a config > >option that will (something like Windows service manager has for the > >services itself) behave differently on first and on the consecutive > >failures, either launching sendmail, or shutting down the > httpd itself, > >or whatever. > > > > Within the mpm - they way it is determined changes from mpm > to mpm. You ask about "the child process" - there may be one, > or many. You will need to decide if this should trigger on > each individual child process that "goes away" - perhaps with > a useful nugget like that childs' apr_proc_t. > As I see the problem, if the child process exited prematurely then eventually all of them will at some point. > But I suggest you take a look at the otherchild logic first. > Perhaps we goofed? Maybe in 2.1 the very best thing would be > to treat all worker processes as "just another otherchild"? > That seems like it would become alot more extensible. True. IMO all worker processes should be treated as just one. If the process (or one of them) dies due to a fatal error, it is just restarted, meaning that the entire httpd eventually enters in the launch/exit loop, consuming 100% of cpu. I was thinking of something like: ServerOnFatalError 1 restart ServerOnFatalError 2 /do_something ServerOnFatalError 3 /do_something ... ServerOnFatalError n shutdown First argument is the number of times the worker process has exited with the fatal error, and the second is the action. There should also be some sort of the time window when the counter is reset to 0, like: ServerFatalErrorTimeout 60 > Again - that logic all belongs in the mpm itself. > Then the logic has to eventually be implemented in each mpm separately? MT.
[PATCH] A modular mod_log_config for Apache 2
Hi, I wasn't too happy with the current mod_log_config design, which turned out to be a bit unflexible in the writer section, so I modified it in order to be able to happily log into a database. The disadvantages of the existing set_writer way of doing things are: a) The log writer doesn't get all the details of the items to log (namely if they exist or not, i.e. is it 0,"-","NULL" or NULL?), you have to stick with whatever string mod_log_config assembles. b) First everything gets converted into a string only to be converted back to bits and bytes by the database (e.g. MySQL will accept raw data in upcoming releases). Guess this might be a waste of cycles, even if compared to my new design ;) c) You cannot use several log writers at the same time (e.g. log one host into a file and another one into a database). Or at least you'll have to add a bunch of new configuration commands. My approach is simply to move the data -> string conversion from the various log item functions to the log writer itself (and there possibly to decide to dump it completely). Instead of returning strings the log item functions fill out a data block with a pointer to the data (or NULL if the item does not exist) and a type identifier (basically number, string, datetime). The log writer then does whatever it thinks is correct with this data chain. Log writer selection is done within the configuration files by giving Custom-/TransferLog a URL-like descriptor. For example, to log into "log/virtual_only" using the file writer, TransferLog file:log/virtual_only can be used. This is easily extendable, MySQL can e.g. be accessed using CustomLog mysql:[EMAIL PROTECTED]/database "insert ..." Of course, LogFormat is usable, too. So, whatever writer you use, the important stuff can stick to the place where it has always been. Build-in are a file:- (default) and a pipe:-writer. Other writers can be added using a function, this works almost like the old set_writer calls. See mod_log_mysql for an example. The whole thing is 100% backwards compatible (which might make the code look a bit messy :). mod_log_config still accepts data from old-style external log item functions as well as old-style file/pipe paths in Custom-/TransferLog. The set_writer- functions still work, too. That said, I'm a bit unsure if the implementation is correct and bug-free, although it seems to work on my own server - after all, I'm new to this. I'd be happy about any helpful comments on design issues and other thoughts and suggestions. I don't know if it's ok to post the hole 60k here (diff or c doesn't matter), so I put everything up at http://bitbrook.de/privat/st/mod_log_mysql/ , hope nobody minds. Thanks, soenk.e
Re: Hooking child termination
At 01:38 AM 9/11/2003, Mladen Turk wrote: >Hi all, > >Is there a possibility to add some sort of a hook to a parent that will >execute when the child process dies? I would like to have a config >option that will (something like Windows service manager has for the >services itself) behave differently on first and on the consecutive >failures, either launching sendmail, or shutting down the httpd itself, >or whatever. > >What would be the right place to add such a feature; core or mpm? Within the mpm - they way it is determined changes from mpm to mpm. You ask about "the child process" - there may be one, or many. You will need to decide if this should trigger on each individual child process that "goes away" - perhaps with a useful nugget like that childs' apr_proc_t. But I suggest you take a look at the otherchild logic first. Perhaps we goofed? Maybe in 2.1 the very best thing would be to treat all worker processes as "just another otherchild"? That seems like it would become alot more extensible. Again - that logic all belongs in the mpm itself. Bill
Re: new in Apache::Test: t/SMOKE -bug_mode
Rafael Garcia-Suarez wrote: Stas Bekman wrote: It all started when I have noticed a few core files after running t/SMOKE on modperl-2.0's top-level test suite. I couldn't figure out which test caused them. So I have decided to rewrite the smoker to provide me that information. In the future we might autoextract the backtrace as well. Neat. (autoextracting the backtrace automatically is easily done with gdb -x where available.) I didn't know about this option. It doesn't seem to exist on gdb/linux (I have the very latest one 5.3-22mdk). There is also a very old Devel::CoreStack, which doesn't quite work if the gdb takes more than one TTY screen to get to the trace. I couldn't figure out how to circumvent the 'next page' prompt. Instead I wrote (Debug::FaultAutoBT) http://search.cpan.org/search?query=Debug%3A%3AFaultAutoBT&mode=module, which traps segfaults and gets the backtrace without needed a core file. However it requires more work as it doesn't always work. It's modeled after the KDE bug-report utility, which pops a bug report window when some KDE-app crashes and shows you the backtrace immediately. This should make bug reports much more useful. However it comes at a price of running much slower than the normal 'make test' would do, until Test::Harness::Straps won't provide the callbacks to be run after each test, Just in case you don't know this already, the latest Test::Harness contains an experimental -- and undocumented -- form of callbacks. > If you could post something to perl-qa to explain in a few sentences what you need, I'd be happy to help crafting something. And to reuse it for perl :) That's exactly why I have mentioned Test::Harness::Straps, I have submitted a request for the callback feature, and that's when Andy Lester has mentioned the experimental feature in Test::Harness::Straps. All needed is the ability to install a hook to be run after each test is finished (and purhaps before it). Then one can add various useful procedures to happen between each test (e.g. core file finder). __ Stas BekmanJAm_pH --> Just Another mod_perl Hacker http://stason.org/ mod_perl Guide ---> http://perl.apache.org mailto:[EMAIL PROTECTED] http://use.perl.org http://apacheweek.com http://modperlbook.org http://apache.org http://ticketmaster.com
Re: mod_ssl and slow first connection
At 02:46 AM 9/12/2003, Apache-Mailing wrote: >Hi, > >i did test with openssl 0.9.6 and openssl 0.9.7 and it's the same >behaviour, but i found what was wrong. > >If IE or Netscape are accepting both sslv2 and sslv3, and the mod_ssl is >setup with SSLProtocol ALL, it take long time before the popup show up. > >If you force to SSLv3 or SSLv2 with mod_ssl, the popup show up >immediatly. It sounds like the negotiation code in OpenSSL is looking for a block of bytes longer with a longer initiation list than the client provides for an SSLv3 or SSLv2 handshake. This means, since they weren't stacked in the best order, that OpenSSL can only finish handshaking after the client times out to Apache. >I think i had a problem with the couple cipher/protocol too. If you want to help the OpenSSL team fix this, you might use a very limited list of one additional protocol plus SSLv2 or SSLv3. Then see which combination of two ciphers causes everything to choke waiting for the timeout. Bill >Le jeu 11/09/2003 à 18:51, William A. Rowe, Jr. a écrit : >> At 05:20 AM 9/11/2003, Apache-Mailing wrote: >> >Hi, >> > >> >I am actually doing bench on ssl and apache 2.0.47 >> >I can notice that the first connection is slow and the popup about the >> >server certificate take time (~40sec) to show up. >> >> Matthieu, which version of OpenSSL are you building with? >> >> Bill
Re: new in Apache::Test: t/SMOKE -bug_mode
Stas Bekman wrote: > It all started when I have noticed a few core files after running t/SMOKE on > modperl-2.0's top-level test suite. I couldn't figure out which test caused > them. So I have decided to rewrite the smoker to provide me that information. > In the future we might autoextract the backtrace as well. Neat. (autoextracting the backtrace automatically is easily done with gdb -x where available.) > This should make bug reports much more useful. However it comes at a price of > running much slower than the normal 'make test' would do, until > Test::Harness::Straps won't provide the callbacks to be run after each test, Just in case you don't know this already, the latest Test::Harness contains an experimental -- and undocumented -- form of callbacks. If you could post something to perl-qa to explain in a few sentences what you need, I'd be happy to help crafting something. And to reuse it for perl :)
[PATCH] A modular mod_log_config for Apache 2
Hi, I wasn't too happy with the current mod_log_config design, which turned out to be a bit unflexible in the writer section, so I modified it in order to be able to happily log into a database. The disadvantages of the existing set_writer way of doing things are: a) The log writer doesn't get all the details of the items to log (namely if they exist or not, i.e. is it 0,"-","NULL" or NULL?), you have to stick with whatever string mod_log_config assembles. b) First everything gets converted into a string only to be converted back to bits and bytes by the database (e.g. MySQL will accept raw data in upcoming releases). Guess this might be a waste of cycles, even if compared to my new design ;) c) You cannot use several log writers at the same time (e.g. log one host into a file and another one into a database). Or at least you'll have to add a bunch of new configuration commands. My approach is simply to move the data -> string conversion from the various log item functions to the log writer itself (and there possibly to decide to dump it completely). Instead of returning strings the log item functions fill out a data block with a pointer to the data (or NULL if the item does not exist) and a type identifier (basically number, string, datetime). The log writer then does whatever it thinks is correct with this data chain. Log writer selection is done within the configuration files by giving Custom-/TransferLog a URL-like descriptor. For example, to log into "log/virtual_only" using the file writer, TransferLog file:log/virtual_only can be used. This is easily extendable, MySQL can e.g. be accessed using CustomLog mysql:[EMAIL PROTECTED]/database "insert ..." Of course, LogFormat is usable, too. So, whatever writer you use, the important stuff can stick to the place where it has always been. Build-in are a file:- (default) and a pipe:-writer. Other writers can be added using a function, this works almost like the old set_writer calls. See mod_log_mysql for an example. The whole thing is 100% backwards compatible (which might make the code look a bit messy :). mod_log_config still accepts data from old-style external log item functions as well as old-style file/pipe paths in Custom-/TransferLog. The set_writer- functions still work, too. That said, I'm a bit unsure if the implementation is correct and bug-free, although it seems to work on my own server - after all, I'm new to this. I'd be happy about any helpful comments on design issues and other thoughts and suggestions. I don't know if it's ok to post the hole 60k here (diff or c doesn't matter), so I put everything up at http://bitbrook.de/privat/st/mod_log_mysql/ , hope nobody minds. Just a click away, after all :) Thanks, soenk.e
Re: [PATCH] Bug #18756: fix many problems about ldap cache using shared memory
Matthieu Estrade wrote: 1) i moved all this global declaration inside the module config, then i made all rmm_alloc and rmm_free use these module config data. 2) i moved the shm init in post_config hook, then did some code to merge all these rmm and shm addr in all vhost module config 3) i created a directive LDAPSharedCacheFile receiving a filename, to create the file with apr_root_relative which will be used to create the shm. +1 from me - I'll commit to 2.1 in the next few days assuming there are no objections? Regards, Graham -- - [EMAIL PROTECTED] "There's a moon over Bourbon Street tonight..."
[PATCH] Bug #18756: fix many problems about ldap cache using shared memory
Hi,
This is my second and new patch for ldap cache using shared memory.
I am using ldap cache with linux redhat and worker mpm.
The problems i found:
1) All shm cache information (shm addr, rmm addr, lock) are declared as
global.
--> in worker mpm, many strange problems happen.
2) shm initialisation is in child init hook
--> it try to create an shm file per child
3) ldap cache using shm is using a default filename to create shm, in
/tmp
--> if 2 different httpd are used, only one can use ldap cache using
shm.
What i did:
1) i moved all this global declaration inside the module config, then i
made all rmm_alloc and rmm_free use these module config data.
2) i moved the shm init in post_config hook, then did some code to merge
all these rmm and shm addr in all vhost module config
3) i created a directive LDAPSharedCacheFile receiving a filename, to
create the file with apr_root_relative which will be used to create the
shm.
I tested this patch on linux rh9 with 3 vhost using ldap auth, it work
well. I also tested the ldap-status and it work well too.
Feedback are welcome !
regards,
Matthieu
? patch_ldap_cache.diff
Index: include/util_ldap.h
===
RCS file: /home/cvspublic/httpd-2.0/include/util_ldap.h,v
retrieving revision 1.11
diff -u -r1.11 util_ldap.h
--- include/util_ldap.h 14 Feb 2003 16:04:00 - 1.11
+++ include/util_ldap.h 12 Sep 2003 10:08:11 -
@@ -75,6 +75,10 @@
#include "http_protocol.h"
#include "http_request.h"
+#if APR_HAS_SHARED_MEMORY
+#include "apr_rmm.h"
+#include "apr_shm.h"
+#endif
/* Create a set of LDAP_DECLARE(type), LDLDAP_DECLARE(type) and
* LDAP_DECLARE_DATA with appropriate export and import tags for the platform
@@ -97,7 +101,6 @@
#define LDAP_DECLARE_DATA __declspec(dllimport)
#endif
-
/*
* LDAP Connections
*/
@@ -138,9 +141,11 @@
apr_pool_t *pool; /* pool from which this state is allocated */
#if APR_HAS_THREADS
apr_thread_mutex_t *mutex; /* mutex lock for the connection list */
+apr_thread_rwlock_t *util_ldap_cache_lock;
#endif
apr_size_t cache_bytes; /* Size (in bytes) of shared memory cache */
+char *cache_file; /* filename for shm */
long search_cache_ttl; /* TTL for search cache */
long search_cache_size; /* Size (in entries) of search cache */
long compare_cache_ttl; /* TTL for compare cache */
@@ -150,6 +155,15 @@
char *cert_auth_file;
int cert_file_type;
int ssl_support;
+
+#if APR_HAS_SHARED_MEMORY
+apr_shm_t *cache_shm;
+apr_rmm_t *cache_rmm;
+#endif
+
+/* cache ald */
+void *util_ldap_cache;
+
} util_ldap_state_t;
@@ -286,21 +300,21 @@
* @param reqsize The size of the shared memory segement to request. A size
*of zero requests the max size possible from
*apr_shmem_init()
- * @deffunc void util_ldap_cache_init(apr_pool_t *p)
+ * @deffunc void util_ldap_cache_init(apr_pool_t *p, util_ldap_state_t *st)
* @return The status code returned is the status code of the
* apr_smmem_init() call. Regardless of the status, the cache
* will be set up at least for in-process or in-thread operation.
*/
-apr_status_t util_ldap_cache_init(apr_pool_t *pool, apr_size_t reqsize);
+apr_status_t util_ldap_cache_init(apr_pool_t *pool, util_ldap_state_t *st);
/**
* Display formatted stats for cache
* @param The pool to allocate the returned string from
* @tip This function returns a string allocated from the provided pool that describes
* various stats about the cache.
- * @deffunc char *util_ald_cache_display(apr_pool_t *pool)
+ * @deffunc char *util_ald_cache_display(apr_pool_t *pool, util_ldap_state_t *st)
*/
-char *util_ald_cache_display(apr_pool_t *pool);
+char *util_ald_cache_display(apr_pool_t *pool, util_ldap_state_t *st);
/* from apr_ldap_cache_mgr.c */
@@ -310,9 +324,9 @@
* @param The pool to allocate the returned string from
* @tip This function returns a string allocated from the provided pool that describes
* various stats about the cache.
- * @deffunc char *util_ald_cache_display(apr_pool_t *pool)
+ * @deffunc char *util_ald_cache_display(apr_pool_t *pool, util_ldap_state_t *st)
*/
-char *util_ald_cache_display(apr_pool_t *pool);
+char *util_ald_cache_display(apr_pool_t *pool, util_ldap_state_t *st);
#endif /* APU_HAS_LDAP */
#endif /* UTIL_LDAP_H */
Index: modules/experimental/util_ldap.c
===
RCS file: /home/cvspublic/httpd-2.0/modules/experimental/util_ldap.c,v
retrieving revision 1.14
diff -u -r1.14 util_ldap.c
--- modules/experimental/util_ldap.c4 Apr 2003 13:47:13 - 1.14
+++ modules/experimental/util_ldap.c12 Sep 2003 10:08:16 -
@@ -141,6 +141,7 @@
*/
int util_ldap_handler(request_rec *r)
{
+util_ldap_state_t *st = (util_ldap_state_t
*)ap_
Request body validation before content handler phase
Hello, I'm writing an Apache 2.0 module that has to perform some request body validation before actually deciding which content handler to use for response generation. I'm wondering what the best way is to implement this. 1) input filter is a no go, because at the moment the body data passes through the filter, the content handler has already been picked (and is actually pulling that data through the input filters). Right? 2) use the fixup stage, pull all data through the input filters, spool the data while processing (in memory or on disk), remove all input filters from the request structure, insert a new filter that will eventually pass the spooled data to the content handler 3) something else? -- ir. Kris Verbeeck Software Engineer Ubizen - Ubicenter - Philipssite 5 - 3001 Leuven - Belgium T: +32 16 28 70 64 F: +32 16 28 70 77 Ubizen - We Secure e-business - www.ubizen.com smime.p7s Description: S/MIME Cryptographic Signature
Re: mod_ssl and slow first connection
Hi, i did test with openssl 0.9.6 and openssl 0.9.7 and it's the same behaviour, but i found what was wrong. If IE or Netscape are accepting both sslv2 and sslv3, and the mod_ssl is setup with SSLProtocol ALL, it take long time before the popup show up. If you force to SSLv3 or SSLv2 with mod_ssl, the popup show up immediatly. I think i had a problem with the couple cipher/protocol too. Matthieu Le jeu 11/09/2003 à 18:51, William A. Rowe, Jr. a écrit : > At 05:20 AM 9/11/2003, Apache-Mailing wrote: > >Hi, > > > >I am actually doing bench on ssl and apache 2.0.47 > >I can notice that the first connection is slow and the popup about the > >server certificate take time (~40sec) to show up. > > Matthieu, which version of OpenSSL are you building with? > > Bill
