[STATUS] (flood) Wed Oct 23 23:46:05 EDT 2002
flood STATUS: -*-text-*- Last modified at [$Date: 2002/09/06 10:24:42 $] Release: 1.0: Released July 23, 2002 milestone-03: Tagged January 16, 2002 ASF-transfer: Released July 17, 2001 milestone-02: Tagged August 13, 2001 milestone-01: Tagged July 11, 2001 (tag lost during transfer) RELEASE SHOWSTOPPERS: * Everything needs to work perfectly Other bugs that need fixing: * I get a SIGBUS on Darwin with our examples/round-robin-ssl.xml config, on the second URL. I'm using OpenSSL 0.9.6c 21 dec 2001. * iPlanet sends Content-length - there is a hack in there now to recognize it. However, all HTTP headers need to be normalized before checking their values. This isn't easy to do. Grr. * OpenSSL 0.9.6 Segfaults under high load. Upgrade to OpenSSL 0.9.6b. Aaron says: I just found a big bug that might have been causing this all along (we weren't closing ssl sockets). How can I reproduce the problem you were seeing to verify if this was the fix? * SEGVs when /tmp/.rnd doesn't exist are bad. Make it configurable and at least bomb with a good error message. (See Doug's patch.) Status: This is fixed, no? * If APR has disabled threads, flood should as well. We might want to have an enable/disable parameter that does this also, providing an error if threads are desired but not available. * flood needs to clear pools more often. With a long running test it can chew up memory very quickly. We should just bite the bullet and create/destroy/clear pools for each level of our model: farm, farmer, profile, url/request-cycle, etc. * APR needs to have a unified interface for ephemeral port exhaustion, but aparently Solaris and Linux return different errors at the moment. Fix this in APR then take advantage of it in flood. * The examples/analyze-relative scripts fail when there are less than 5 unique URLs. Other features that need writing: * More analysis and graphing scripts are needed * Write robust tool (using tethereal perhaps) to take network dumps and convert them to flood's XML format. Status: Justin volunteers. Aaron had a script somewhere that is a start. * Get chunked encoding support working. Status: Justin volunteers. He got sidetracked by the httpd implementation of input filtering and never finished this. This is a stopgap until apr-serf is completed. * Maybe we should make randfile and capath runtime directives that come out of the XML, instead of autoconf parameters. * We are using apr_os_thread_current() and getpid() in some places when what we really want is a GUID. The GUID will be used to correlate raw output data with each farmer. We may wish to print a unique ID for each of farm, farmer, profile, and url to help in postprocessing. * We are using strtol() in some places and strtoll() in others. Pick one (Aaron says strtol(), but he's not sure). * Validation of responses (known C-L, specific strings in response) Status: Justin volunteers * HTTP error codes (ie. teach it about 302s) Justin says: Yeah, this won't be with round_robin as implemented. Need a linked list-based profile where we can insert new URLs into the sequence. * Farmer (Single-thread, multiple profiles) Status: Aaron says: If you have threads, then any Farmer can be run as part of any Farm. If you don't have threads, you can currently only run one Farmer named Joe right now (this will be changed so that if you don't have threads, flood will attempt to run all Farmers in serial under one process). * Collective (Single-host, multiple farms) This is a number of Farms that have been fork()ed into child processes. * Megaconglomerate (Multiple hosts each running a collective) This is a number of Collectives running on a number of hosts, invoked via RSH/SSH or maybe even some proprietary mechanism. * Other types of urllists a) Random / Random-weighted b) Sequenced (useful with cookie propogation) c) Round-robin d) Chaining of the above strategies Status: Round-robin is complete. * Other types of reports Status: Aaron says: simple reports are functional. Justin added a new type that simply prints the approx. timestamp when the test was run, and the result as OK/FAIL; it is called easy reports (see flood_easy_reports.h). Furthermore, simple_reports and easy_reports both print out the current requesting URI line. Documentation that needs writing: * Documentation?
[STATUS] (perl-framework) Wed Oct 23 23:46:07 EDT 2002
httpd-test/perl-framework STATUS: -*-text-*- Last modified at [$Date: 2002/03/09 05:22:48 $] Stuff to do: * finish the t/TEST exit code issue (ORed with 0x2C if framework failed) * change existing tests that frob the DocumentRoot (e.g., t/modules/access.t) to *not* do that; instead, have Makefile.PL prepare appropriate subdirectory configs for them. Why? So t/TEST can be used to test a remote server. * problems with -d perl mode, doesn't work as documented Message-ID: [EMAIL PROTECTED] Date: Sat, 20 Oct 2001 12:58:33 +0800 Subject: Re: perldb Tests to be written: * t/apache - simulations of network failures (incomplete POST bodies, chunked and unchunked; missing POST bodies; slooow client connexions, such as taking 1 minute to send 1KiB; ...) * t/modules/autoindex - something seems possibly broken with inheritance on 2.0 * t/ssl - SSLPassPhraseDialog exec: - SSLRandomSeed exec:
RE: [STATUS] (httpd-2.0) Wed Oct 23 23:45:19 EDT 2002
-Original Message- From: Rodent of Unusual Size [mailto:Ken.Coar;Golux.Com] Sent: Thursday, October 24, 2002 06:45 To: Apache HTTP server developers Subject: [STATUS] (httpd-2.0) Wed Oct 23 23:45:19 EDT 2002 APACHE 2.0 STATUS: -*-text-*- Last modified at [$Date: 2002/10/22 17:01:44 $] Hi there! I would be happy to see Bugzilla #7617 be applied to the Apache 2.0, as I just have submitted a patch for 2.0.40 with regard to that annoying bug. Or if my patch is insufficient, at least someone with the proper knowledge should open a PR for Apache 2.0 about that behaviour (I do not feel like the right person to do so). -- [EMAIL PROTECTED] GSM +358-40-767 8282 Oy Arrak Software Ab http://www.arrak.fi
Re: [madhusudan_mathihalli@hp.com: [PATCH] Try to use OPENSSL_free instead of free]
Based on Nadav Har'El's e-mail on the mod_ssl community (http://marc.theaimsgroup.com/?l=apache-modsslm=103540998016916w=2), here's a patch for 2.0's mod_ssl. ... +modssl_free(cp); ... +#define modssl_free free I'm not sure this patch addresses my concern. The idea was that OPENSSL_free() should be used, not free(), on memory originally allocated by the OpenSSL library. This is because you cannot (or should not) assume how OPENSSL_malloc() is implemented, and whether or not it's simply malloc(). So you probably meant to do #define modssl_free OPENSSL_free But I have another question about this patch - why did you choose the name modssl_free? These frees are of memory allocated by the OpenSSL library, not by mod_ssl, so the name modssl_free is misleading. What's wrong with using OPENSSL_free for memory allocated and returned by OpenSSL routines? -- Nadav Har'El| Thursday, Oct 24 2002, 18 Heshvan 5763 [EMAIL PROTECTED] |- Phone: +972-53-245868, ICQ 13349191 |He who has more is not happier than he http://nadav.harel.org.il |who wants less.
Re: [madhusudan_mathihalli@hp.com: [PATCH] Try to use OPENSSL_free instead of free]
On Thu, Oct 24, 2002, Nadav Har'El wrote about Re: [[EMAIL PROTECTED]: [PATCH] Try to use OPENSSL_free instead of free]: I'm not sure this patch addresses my concern. The idea was that OPENSSL_free() should be used, not free(), on memory originally allocated by the OpenSSL I should have looked at the Apache 2 code more carefully before complaining about the patch... I didn't realize that Apache 2 was trying to be compatible with another SSL stack, not just OpenSSL, and that modssl_free was defined differently in the two cases. So your patch seems fine. Sorry. -- Nadav Har'El| Thursday, Oct 24 2002, 18 Heshvan 5763 [EMAIL PROTECTED] |- Phone: +972-53-245868, ICQ 13349191 |Two wrongs may not may a right, but three http://nadav.harel.org.il |rights make a left.
Re: [STATUS] (httpd-2.0) Wed Oct 23 23:45:19 EDT 2002
On Fri, 2002-10-25 at 03:31, David Burry wrote: Is it possible to get some of the fixes to mod_logio committed? Wouldn't everyone agree that the current logging of the outgoing bytes is incorrect behavior? Currently it logs the full file size (plus headers) even if it gets cut off in the middle, instead of the actual number of bytes sent. I've seen several patches to fix this but very little comment on it... I've seen lots of comments that it can't be done without major rearchitecting, but Bojan seems to have done it without that by breaking pipelining, am I correct? Actually, the last patch I sent contains one snag I'm still working on. It breaks the core's connection configuration structure, which gets attached to c-conn_config. However, I think I can get around that by using an optional function. As the matter of fact, I'm working on it right now. I also wish that %b would be fixed in a similar manner but I haven't seen any patches for that (or comments about it). Wouldn't everyone agree that it too should log actual bytes sent not just the full file size every time? Apache 2.0 should do everything that 1.3 did, so this logging issue really should be considered a bug, right? I agree with you on this one as well. But at this point I'm unsure how to fix this one. Since I depend on correct outgoing byte count logging to see how many people sucessfully download files, I can live with broken pipelining for now in 2.0, currently I've had to roll back to Apache 1.3 and put in 3 times as many machines (12 instead of 4) I'd really like to return those 8 borrowed machines someday and be able to upgrade to 2.0... but can't do that in the current broken log state. Glad to hear Apache 2.0 makes a huge performance difference. Not so glad to hear you had to resort to going back to 1.3. The only thing I can promise is a patch using optional function (this should guarantee compatibility of core between 43 and 44 and no MMN bump) during the day (Sydney time). It's up to the committers to review and, if they like, commit. Bojan PS. By the number of messages on the list I'm guessing committers must be rather busy on their real jobs these days. Unfortunately there is no way of speeding things up, given this is volunteer effort. Unless, of course, you decide to bribe some of them ;-)
byterange filter/redirect bug?
check this out: [gregamesdaedalus gregames]$ grep 194.65.14.76 .*binaries.* 416 /logs/www/weblog | wc -l 69763 I asked root to block this IP for a while, because we are getting a couple of these every second. I suspect there's an httpd bug here as well as a looping client. We shouldn't be returning 416 Invalid Range for something that's redirected, but it sure looks like we are. I wonder what the byterange filter does when it sees a redirect response, and there are input Range: headers? Greg [gregamesdaedalus gregames]$ grep 194.65.14.76 .*binaries.* 416 /logs/www/weblog | tail www.apache.org 194.65.14.76 - - [24/Oct/2002:13:05:32 -0700] GET /dist/httpd/binaries/win32/apache_1.3.27-win32-x86-no_src.exe HTTP/1.1 416 387 http://www.apache.org/dist/httpd/binaries/win32/; Mozilla/4.0 (compatible; MSIE 5.00; Windows 98) www.apache.org 194.65.14.76 - - [24/Oct/2002:13:05:32 -0700] GET /dist/httpd/binaries/win32/apache_1.3.27-win32-x86-no_src.exe HTTP/1.1 416 387 http://www.apache.org/dist/httpd/binaries/win32/; Mozilla/4.0 (compatible; MSIE 5.00; Windows 98) www.apache.org 194.65.14.76 - - [24/Oct/2002:13:05:33 -0700] GET /dist/httpd/binaries/win32/apache_2.0.43-win32-x86-no_ssl.exe HTTP/1.1 416 387 http://www.apache.org/dist/httpd/binaries/win32/; Mozilla/4.0 (compatible; MSIE 5.00; Windows 98) www.apache.org 194.65.14.76 - - [24/Oct/2002:13:05:33 -0700] GET /dist/httpd/binaries/win32/apache_2.0.43-win32-x86-no_ssl.exe HTTP/1.1 416 387 http://www.apache.org/dist/httpd/binaries/win32/; Mozilla/4.0 (compatible; MSIE 5.00; Windows 98)
Re: [STATUS] (httpd-2.0) Wed Oct 23 23:45:19 EDT 2002
Is it possible to get some of the fixes to mod_logio committed? Wouldn't everyone agree that the current logging of the outgoing bytes is incorrect behavior? Currently it logs the full file size (plus headers) even if it gets cut off in the middle, instead of the actual number of bytes sent. I've seen several patches to fix this but very little comment on it... I've seen lots of comments that it can't be done without major rearchitecting, but Bojan seems to have done it without that by breaking pipelining, am I correct? I also wish that %b would be fixed in a similar manner but I haven't seen any patches for that (or comments about it). Wouldn't everyone agree that it too should log actual bytes sent not just the full file size every time? Apache 2.0 should do everything that 1.3 did, so this logging issue really should be considered a bug, right? Since I depend on correct outgoing byte count logging to see how many people sucessfully download files, I can live with broken pipelining for now in 2.0, currently I've had to roll back to Apache 1.3 and put in 3 times as many machines (12 instead of 4) I'd really like to return those 8 borrowed machines someday and be able to upgrade to 2.0... but can't do that in the current broken log state. Dave
Re: [STATUS] (httpd-2.0) Wed Oct 23 23:45:19 EDT 2002
- Original Message - From: Bojan Smojver [EMAIL PROTECTED] On Fri, 2002-10-25 at 03:31, David Burry wrote: Is it possible to get some of the fixes to mod_logio committed? Wouldn't everyone agree that the current logging of the outgoing bytes is incorrect behavior? Currently it logs the full file size (plus headers) even if it gets cut off in the middle, instead of the actual number of bytes sent. I've seen several patches to fix this but very little comment on it... I've seen lots of comments that it can't be done without major rearchitecting, but Bojan seems to have done it without that by breaking pipelining, am I correct? Actually, the last patch I sent contains one snag I'm still working on. It breaks the core's connection configuration structure, which gets attached to c-conn_config. However, I think I can get around that by using an optional function. As the matter of fact, I'm working on it right now. I see.. ok, I'll keep waiting patiently... Since I depend on correct outgoing byte count logging to see how many people sucessfully download files, I can live with broken pipelining for now in 2.0, currently I've had to roll back to Apache 1.3 and put in 3 times as many machines (12 instead of 4) I'd really like to return those 8 borrowed machines someday and be able to upgrade to 2.0... but can't do that in the current broken log state. Glad to hear Apache 2.0 makes a huge performance difference. Not so glad to hear you had to resort to going back to 1.3. The only thing I can promise is a patch using optional function (this should guarantee compatibility of core between 43 and 44 and no MMN bump) during the day (Sydney time). It's up to the committers to review and, if they like, commit. The memory savings is quite significant, and I'll admit that the 8 extra machines are smaller than the original 4 so it's not exactly 3 times better cpu-wise... the memory caching is where the largest savings comes with disk IO, we have a very high traffic site--usually 3 terabytes transferred per day, last few days have been more like 5 terabytes due to a new release. PS. By the number of messages on the list I'm guessing committers must be rather busy on their real jobs these days. Unfortunately there is no way of speeding things up, given this is volunteer effort. Unless, of course, you decide to bribe some of them ;-) Not exactly the same thing as bribing a commit, but this could get similar results: My manager's manager is actually not opposed to hiring a contractor to fix this... anyone want a temporary job? I don't know if this is the right place to say such things, tell me if there's a better place. When you've got millions of dollars worth of sales depending on an open source project, throwing a little at the open source project isn't such a big deal... I'd gladly do it myself with my company's blessing (on the clock, not volunteer) but I'm not a very experienced C programmer yet, more of a Perl hacker and applications architect so far. This little paragraph had better not get me too flooded with resumes or flames or I'm going to feel dumb, whatever you do don't spam this list with personal replies! ;o) Dave
[PATCH]: Counting I/O with FLUSH and OFN (no MMN bump)
This should really work now and it should not cause major dramas
compatibility wise. Let me know what you think.
Bojan
diff -ruN httpd-2.0-vanilla/include/http_core.h httpd-2.0/include/http_core.h
--- httpd-2.0-vanilla/include/http_core.h Tue Oct 15 03:42:45 2002
+++ httpd-2.0/include/http_core.h Fri Oct 25 08:04:28 2002
-61,6 +61,7
#include apr.h
#include apr_hash.h
+#include apr_optional.h
#include util_filter.h
#if APR_HAVE_STRUCT_RLIMIT
-626,6 +627,16
/* -- */
+/* --
+ *
+ * I/O logging with mod_logio
+ */
+
+APR_DECLARE_OPTIONAL_FN(void, ap_logio_add_bytes_out,
+(conn_rec *c, apr_off_t bytes));
+
+/* -- */
+
#ifdef __cplusplus
}
#endif
diff -ruN httpd-2.0-vanilla/modules/loggers/mod_logio.c httpd-2.0/modules/loggers/mod_logio.c
--- httpd-2.0-vanilla/modules/loggers/mod_logio.c Sat Sep 28 14:18:35 2002
+++ httpd-2.0/modules/loggers/mod_logio.c Fri Oct 25 08:04:14 2002
-79,6 +79,7
#include ap_config.h
#include mod_log_config.h
#include httpd.h
+#include http_core.h
#include http_config.h
#include http_protocol.h
-96,6 +97,16
} logio_config_t;
/*
+ * Optional function for the core to add to bytes_out
+ */
+
+static void ap_logio_add_bytes_out(conn_rec *c, apr_off_t bytes){
+logio_config_t *cf = ap_get_module_config(c-conn_config, logio_module);
+
+cf-bytes_out+=bytes;
+}
+
+/*
* Format items...
*/
-133,24 +144,6
* Logging of input and output filters...
*/
-static apr_status_t logio_out_filter(ap_filter_t *f,
- apr_bucket_brigade *bb) {
-apr_off_t length;
-logio_config_t *cf = ap_get_module_config(f-c-conn_config, logio_module);
-
-if (!cf) { /* Create config */
-cf = apr_pcalloc(f-c-pool, sizeof(*cf));
-ap_set_module_config(f-c-conn_config, logio_module, cf);
-}
-
-apr_brigade_length (bb, 0, length);
-
-if (length 0)
-cf-bytes_out += length;
-
-return ap_pass_brigade(f-next, bb);
-}
-
static apr_status_t logio_in_filter(ap_filter_t *f,
apr_bucket_brigade *bb,
ap_input_mode_t mode,
-162,11 +155,6
status = ap_get_brigade(f-next, bb, mode, block, readbytes);
-if (!cf) { /* Create config */
-cf = apr_pcalloc(f-c-pool, sizeof(*cf));
-ap_set_module_config(f-c-conn_config, logio_module, cf);
-}
-
apr_brigade_length (bb, 0, length);
if (length 0)
-175,11 +163,30
return status;
}
+static apr_status_t logio_out_filter(ap_filter_t *f,
+ apr_bucket_brigade *bb) {
+apr_bucket *b = APR_BRIGADE_LAST(bb);
+
+/* End of data, make sure we flush */
+if (APR_BUCKET_IS_EOS(b)) {
+APR_BRIGADE_INSERT_TAIL(bb,
+apr_bucket_flush_create(f-c-bucket_alloc));
+APR_BUCKET_REMOVE(b);
+apr_bucket_destroy(b);
+}
+
+return ap_pass_brigade(f-next, bb);
+}
+
/*
* The hooks...
*/
static int logio_pre_conn(conn_rec *c) {
+logio_config_t *cf = apr_pcalloc(c-pool, sizeof(*cf));
+
+ap_set_module_config(c-conn_config, logio_module, cf);
+
ap_add_input_filter(logio_filter_name, NULL, NULL, c);
ap_add_output_filter(logio_filter_name, NULL, NULL, c);
-212,6 +219,8
AP_FTYPE_NETWORK - 1);
ap_register_output_filter(logio_filter_name, logio_out_filter, NULL,
AP_FTYPE_NETWORK - 1);
+
+APR_REGISTER_OPTIONAL_FN(ap_logio_add_bytes_out);
}
module AP_MODULE_DECLARE_DATA logio_module =
diff -ruN httpd-2.0-vanilla/server/core.c httpd-2.0/server/core.c
--- httpd-2.0-vanilla/server/core.c Tue Oct 15 06:08:15 2002
+++ httpd-2.0/server/core.c Fri Oct 25 08:08:22 2002
-2737,6 +2737,7
apr_off_t file_offset,
apr_size_t file_bytes_left,
apr_size_t total_bytes_left,
+apr_size_t *bytes_sent,
apr_int32_t flags)
{
apr_status_t rv;
-2748,11 +2749,15
== APR_SUCCESS)
timeout 0); /* socket must be in timeout mode */
+/* Reset the bytes_sent field */
+*bytes_sent = 0;
+
do {
apr_size_t tmplen = file_bytes_left;
rv = apr_sendfile(c-client_socket, fd, hdtr, file_offset, tmplen,
flags);
+*bytes_sent += tmplen;
total_bytes_left -= tmplen;
if (!total_bytes_left || rv != APR_SUCCESS) {
return rv;/* normal case error exit */
-3647,6 +3652,11
*/
#define MAX_IOVEC_TO_WRITE 16
+/* Optional function coming from
Re: [PATCH]: Counting I/O with FLUSH and OFN (no MMN bump)
Just two extra spaces so it's looks nicer...
Bojan
On Fri, 2002-10-25 at 08:11, Bojan Smojver wrote:
This should really work now and it should not cause major dramas
compatibility wise. Let me know what you think.
diff -ruN httpd-2.0-vanilla/include/http_core.h httpd-2.0/include/http_core.h
--- httpd-2.0-vanilla/include/http_core.h Tue Oct 15 03:42:45 2002
+++ httpd-2.0/include/http_core.h Fri Oct 25 08:04:28 2002
-61,6 +61,7
#include apr.h
#include apr_hash.h
+#include apr_optional.h
#include util_filter.h
#if APR_HAVE_STRUCT_RLIMIT
-626,6 +627,16
/* -- */
+/* --
+ *
+ * I/O logging with mod_logio
+ */
+
+APR_DECLARE_OPTIONAL_FN(void, ap_logio_add_bytes_out,
+(conn_rec *c, apr_off_t bytes));
+
+/* -- */
+
#ifdef __cplusplus
}
#endif
diff -ruN httpd-2.0-vanilla/modules/loggers/mod_logio.c httpd-2.0/modules/loggers/mod_logio.c
--- httpd-2.0-vanilla/modules/loggers/mod_logio.c Sat Sep 28 14:18:35 2002
+++ httpd-2.0/modules/loggers/mod_logio.c Fri Oct 25 08:12:59 2002
-79,6 +79,7
#include ap_config.h
#include mod_log_config.h
#include httpd.h
+#include http_core.h
#include http_config.h
#include http_protocol.h
-96,6 +97,16
} logio_config_t;
/*
+ * Optional function for the core to add to bytes_out
+ */
+
+static void ap_logio_add_bytes_out(conn_rec *c, apr_off_t bytes){
+logio_config_t *cf = ap_get_module_config(c-conn_config, logio_module);
+
+cf-bytes_out += bytes;
+}
+
+/*
* Format items...
*/
-133,24 +144,6
* Logging of input and output filters...
*/
-static apr_status_t logio_out_filter(ap_filter_t *f,
- apr_bucket_brigade *bb) {
-apr_off_t length;
-logio_config_t *cf = ap_get_module_config(f-c-conn_config, logio_module);
-
-if (!cf) { /* Create config */
-cf = apr_pcalloc(f-c-pool, sizeof(*cf));
-ap_set_module_config(f-c-conn_config, logio_module, cf);
-}
-
-apr_brigade_length (bb, 0, length);
-
-if (length 0)
-cf-bytes_out += length;
-
-return ap_pass_brigade(f-next, bb);
-}
-
static apr_status_t logio_in_filter(ap_filter_t *f,
apr_bucket_brigade *bb,
ap_input_mode_t mode,
-162,11 +155,6
status = ap_get_brigade(f-next, bb, mode, block, readbytes);
-if (!cf) { /* Create config */
-cf = apr_pcalloc(f-c-pool, sizeof(*cf));
-ap_set_module_config(f-c-conn_config, logio_module, cf);
-}
-
apr_brigade_length (bb, 0, length);
if (length 0)
-175,11 +163,30
return status;
}
+static apr_status_t logio_out_filter(ap_filter_t *f,
+ apr_bucket_brigade *bb) {
+apr_bucket *b = APR_BRIGADE_LAST(bb);
+
+/* End of data, make sure we flush */
+if (APR_BUCKET_IS_EOS(b)) {
+APR_BRIGADE_INSERT_TAIL(bb,
+apr_bucket_flush_create(f-c-bucket_alloc));
+APR_BUCKET_REMOVE(b);
+apr_bucket_destroy(b);
+}
+
+return ap_pass_brigade(f-next, bb);
+}
+
/*
* The hooks...
*/
static int logio_pre_conn(conn_rec *c) {
+logio_config_t *cf = apr_pcalloc(c-pool, sizeof(*cf));
+
+ap_set_module_config(c-conn_config, logio_module, cf);
+
ap_add_input_filter(logio_filter_name, NULL, NULL, c);
ap_add_output_filter(logio_filter_name, NULL, NULL, c);
-212,6 +219,8
AP_FTYPE_NETWORK - 1);
ap_register_output_filter(logio_filter_name, logio_out_filter, NULL,
AP_FTYPE_NETWORK - 1);
+
+APR_REGISTER_OPTIONAL_FN(ap_logio_add_bytes_out);
}
module AP_MODULE_DECLARE_DATA logio_module =
diff -ruN httpd-2.0-vanilla/server/core.c httpd-2.0/server/core.c
--- httpd-2.0-vanilla/server/core.c Tue Oct 15 06:08:15 2002
+++ httpd-2.0/server/core.c Fri Oct 25 08:08:22 2002
-2737,6 +2737,7
apr_off_t file_offset,
apr_size_t file_bytes_left,
apr_size_t total_bytes_left,
+apr_size_t *bytes_sent,
apr_int32_t flags)
{
apr_status_t rv;
-2748,11 +2749,15
== APR_SUCCESS)
timeout 0); /* socket must be in timeout mode */
+/* Reset the bytes_sent field */
+*bytes_sent = 0;
+
do {
apr_size_t tmplen = file_bytes_left;
rv = apr_sendfile(c-client_socket, fd, hdtr, file_offset, tmplen,
flags);
+*bytes_sent += tmplen;
total_bytes_left -= tmplen;
if (!total_bytes_left || rv != APR_SUCCESS) {
return rv;/* normal case error exit
Re: [PATCH]: Counting I/O with FLUSH and OFN (no MMN bump)
Bojan Smojver wrote: This should really work now and it should not cause major dramas compatibility wise. Let me know what you think. Thanks, I'll take a look at this tonight unless anyone else gets to it first. Brian
Re: [PATCH]: Counting I/O with FLUSH and OFN (no MMN bump)
Very cool. Thanks. Bojan On Fri, 2002-10-25 at 08:28, Brian Pane wrote: Bojan Smojver wrote: This should really work now and it should not cause major dramas compatibility wise. Let me know what you think. Thanks, I'll take a look at this tonight unless anyone else gets to it first.
Re: [STATUS] (httpd-2.0) Wed Oct 23 23:45:19 EDT 2002
On Fri, 2002-10-25 at 07:42, David Burry wrote: I see.. ok, I'll keep waiting patiently... The patch for 2.0.43 is here: ftp://ftp.rexursive.com/pub/apache/counting_io_flush-2.0.43.patch You need to apply mod_logio patch for 2.0.43 first. Bojan
RE: [PATCH] Try to use OPENSSL_free instead of free
Can somebody _please_ take some time to review and tell me if the patch is
okay.
Just a observation :
I do understand that everybody is busy doing their own set of things, but
then - do you really solicit patches from non-committers ?.
I've been a silent observer since the last couple of months, and as-per my
observation, everytime a non-committer sends a patch, it takes a couple of
cycles before somebody even responds back to the patch (let alone commit
it). When somebody posts a patch, I'm sure he/she definitely looks forward
for some sort of input from you guys (the core developers/committers, best
of the breed etc) - atleast something like its all crap / you may want to
do this also / this is all good / something else. Am I wrong here ?..
Does anybody not want a response when they post a patch ?.
Talking specifically of my patches, the majority of the commits to the
mod_ssl source was done by Cliff / Justin / DougM. I do realize that they're
busy with other things also - but then, I've not seen many others doing any
development on the SSL front. So, whom do I approach to solicit any feedback
?.
-Madhu
-Original Message-
From: MATHIHALLI,MADHUSUDAN (HP-Cupertino,ex1)
[mailto:madhusudan_mathihalli;hp.com]
Sent: Wednesday, October 23, 2002 4:52 PM
To: '[EMAIL PROTECTED]'
Subject: [PATCH] Try to use OPENSSL_free instead of free
Based on Nadav Har'El's e-mail on the mod_ssl community
(http://marc.theaimsgroup.com/?l=apache-modsslm=103540998016916w=2),
here's a patch for 2.0's mod_ssl.
-Madhu
Index: ssl_engine_kernel.c
===
RCS file: /home/cvspublic/httpd-2.0/modules/ssl/ssl_engine_kernel.c,v
retrieving revision 1.78
diff -u -r1.78 ssl_engine_kernel.c
--- ssl_engine_kernel.c 14 Oct 2002 04:15:58 - 1.78
+++ ssl_engine_kernel.c 23 Oct 2002 23:46:38 -
@@ -968,7 +968,7 @@
X509_NAME *name =
X509_get_subject_name(sslconn-client_cert);
char *cp = X509_NAME_oneline(name, NULL, 0);
sslconn-client_dn = apr_pstrdup(r-connection-pool, cp);
-free(cp);
+modssl_free(cp);
}
clientdn = (char *)sslconn-client_dn;
@@ -1299,11 +1299,11 @@
iname ? iname : -unknown-);
if (sname) {
-free(sname);
+modssl_free(sname);
}
if (iname) {
-free(iname);
+modssl_free(iname);
}
}
@@ -1555,7 +1555,7 @@
Certificate with serial
%ld (0x%lX)
revoked per CRL from issuer %s,
serial, serial, cp);
-free(cp);
+modssl_free(cp);
}
X509_STORE_CTX_set_error(ctx,
X509_V_ERR_CERT_REVOKED);
@@ -1593,6 +1593,7 @@
ap_log_error(APLOG_MARK, APLOG_DEBUG, 0, s,
SSLPROXY_CERT_CB_LOG_FMT %s, sending %s,
sc-vhost_id, msg, dn ? dn : -uknown-);
+modssl_free(dn);
}
/*
Index: ssl_engine_vars.c
===
RCS file: /home/cvspublic/httpd-2.0/modules/ssl/ssl_engine_vars.c,v
retrieving revision 1.20
diff -u -r1.20 ssl_engine_vars.c
--- ssl_engine_vars.c 28 May 2002 21:47:31 - 1.20
+++ ssl_engine_vars.c 23 Oct 2002 23:51:25 -
@@ -334,7 +334,7 @@
xsname = X509_get_subject_name(xs);
cp = X509_NAME_oneline(xsname, NULL, 0);
result = apr_pstrdup(p, cp);
-free(cp);
+modssl_free(cp);
resdup = FALSE;
}
else if (strlen(var) 5 strcEQn(var, S_DN_, 5)) {
@@ -346,7 +346,7 @@
xsname = X509_get_issuer_name(xs);
cp = X509_NAME_oneline(xsname, NULL, 0);
result = apr_pstrdup(p, cp);
-free(cp);
+modssl_free(cp);
resdup = FALSE;
}
else if (strlen(var) 5 strcEQn(var, I_DN_, 5)) {
Index: ssl_toolkit_compat.h
===
RCS file: /home/cvspublic/httpd-2.0/modules/ssl/ssl_toolkit_compat.h,v
retrieving revision 1.25
diff -u -r1.25 ssl_toolkit_compat.h
--- ssl_toolkit_compat.h21 Aug 2002 19:12:46 - 1.25
+++ ssl_toolkit_compat.h23 Oct 2002 23:46:38 -
@@ -105,6 +105,8 @@
#define modssl_set_cipher_list SSL_set_cipher_list
+#define modssl_free OPENSSL_free
+
#define EVP_PKEY_reference_inc(pkey) \
CRYPTO_add(((pkey)-references), +1, CRYPTO_LOCK_X509_PKEY)
@@ -147,6 +149,8 @@
#define modssl_set_cipher_list(ssl, l) \
SSL_set_cipher_list(ssl, (char *)l)
+
+#define modssl_free free
#ifndef PEM_F_DEF_CALLBACK
#define PEM_F_DEF_CALLBACK PEM_F_DEF_CB
Re: [PATCH] Try to use OPENSSL_free instead of free
MATHIHALLI,MADHUSUDAN (HP-Cupertino,ex1) [EMAIL PROTECTED] writes: Can somebody _please_ take some time to review and tell me if the patch is okay. Just a observation : I do understand that everybody is busy doing their own set of things, but then - do you really solicit patches from non-committers ?. yes, of course we solicit patches from non-committers... if a patch isn't responded to quickly you'll have to nag... write up a CHANGES entry for the patch and I'll commit the darn thing -- Jeff Trawick | [EMAIL PROTECTED] Born in Roswell... married an alien...
Re: Enabling RAND redirection on crypto accelerator using OpenSSL ENGINE
Frederic DONNAT [EMAIL PROTECTED] writes: A few month ago i submit a patch for redirecting RAND on crypto accelerator for mod-ssl and apache-1.3.x. A few weeks ago, i see a cvs commit about this on mod-ssl mailing list. But i see that apache-2.0.x have not been updated. maintainers of mod_ssl for Apache 1.3 apparently have to time for Apache 2.0 mod_ssl I post a message for this in mod-ssl dev mailing list, but maybe should i post it somewhere else! yes, if you have a concern about Apache 2.0 mod_ssl please post here, but note that more skills are on mod-ssl dev mailing list So, in fact the patch is for ssl_engine_init.c file in directory ./modules/ssl. Just modify functions calls: - ssl_engine_init () - ssl_init_SSLlibrary () ssl_engine_init() (line 300) should be call earlier, before than ssl_init_SSLlibrary() (line 270). In fact you have to initialyze OpenSSL ENGINE before initialzing the library, due to fact that OpenSSL default function pointer must be set to ENGINE function pointer before library initialisation otherwise you can not modify default settings. Geoff Thorpe comment: ssl_init_SSLLibrary() must be seeding the PRNG, and thus initialising the set-on-first-use pointer in openssl to a default RAND_METHOD. Cliff Woolley comment: Well, I can't do anything about 1.3's mod_ssl, but if somebody can verify for me that the following fixes Apache 2.0's mod_ssl, I'll commit it. apparently nobody verified for Cliff that it fixed the problem with Apache 2.0 can you verify it? can you post a patch with the change? Thanks, -- Jeff Trawick | [EMAIL PROTECTED] Born in Roswell... married an alien...
Re: [STATUS] (httpd-2.0) Wed Oct 23 23:45:19 EDT 2002
Excellent! I will perform some tests with that when I get a chance! You managed to get it working without breaking pipelining even? That's awesome! Not meaning to belittle Bojan's hard work, but for my purposes mod_logio values are not as good as %b would be if %b worked properly... what I ideally need is the byte sent count without the headers... using Bojan's module I can get approximate results but they will be a hair off because they include headers... My main purpose is to detect if and when several meg files have been downloaded all the way vs. if they were cut off in the middle, including if a given user uses some byte-ranging download manager that lets you pause and restart... We also use it for chargebacks to the various departments for bandwidth usage (in this case mod_logio would of course be more accurate than %b though)... We actually had to fudge some of our statistics (duplicated nearby days' data with similar overall throughputs) due to us not catching the problem with Apache 2.0 soon enough... Dave At 09:15 AM 10/25/2002 +1000, Bojan Smojver wrote: On Fri, 2002-10-25 at 07:42, David Burry wrote: I see.. ok, I'll keep waiting patiently... The patch for 2.0.43 is here: ftp://ftp.rexursive.com/pub/apache/counting_io_flush-2.0.43.patch You need to apply mod_logio patch for 2.0.43 first. Bojan
RE: [PATCH] Try to use OPENSSL_free instead of free
Okay here, it comes [complete patch]
Thanks
-Madhu
Index: CHANGES
===
RCS file: /home/cvspublic/httpd-2.0/CHANGES,v
retrieving revision 1.959
diff -u -r1.959 CHANGES
--- CHANGES 24 Oct 2002 15:47:31 - 1.959
+++ CHANGES 25 Oct 2002 00:37:54 -
@@ -1,5 +1,11 @@
Changes with Apache 2.0.44
+ *) mod_ssl uses free() inappropriately in several places, to free
+ memory which has been previously allocated inside OpenSSL.
+ Such memory should be freed with OPENSSL_free(), not with free().
+ [Nadav Har'El [EMAIL PROTECTED],
+ Madhusudan Mathihalli [EMAIL PROTECTED]].
+
Index: ssl_engine_kernel.c
===
RCS file: /home/cvspublic/httpd-2.0/modules/ssl/ssl_engine_kernel.c,v
retrieving revision 1.78
diff -u -r1.78 ssl_engine_kernel.c
--- ssl_engine_kernel.c 14 Oct 2002 04:15:58 - 1.78
+++ ssl_engine_kernel.c 23 Oct 2002 23:46:38 -
@@ -968,7 +968,7 @@
X509_NAME *name = X509_get_subject_name(sslconn-client_cert);
char *cp = X509_NAME_oneline(name, NULL, 0);
sslconn-client_dn = apr_pstrdup(r-connection-pool, cp);
-free(cp);
+modssl_free(cp);
}
clientdn = (char *)sslconn-client_dn;
@@ -1299,11 +1299,11 @@
iname ? iname : -unknown-);
if (sname) {
-free(sname);
+modssl_free(sname);
}
if (iname) {
-free(iname);
+modssl_free(iname);
}
}
@@ -1555,7 +1555,7 @@
Certificate with serial %ld (0x%lX)
revoked per CRL from issuer %s,
serial, serial, cp);
-free(cp);
+modssl_free(cp);
}
X509_STORE_CTX_set_error(ctx, X509_V_ERR_CERT_REVOKED);
@@ -1593,6 +1593,7 @@
ap_log_error(APLOG_MARK, APLOG_DEBUG, 0, s,
SSLPROXY_CERT_CB_LOG_FMT %s, sending %s,
sc-vhost_id, msg, dn ? dn : -uknown-);
+modssl_free(dn);
}
/*
Index: ssl_engine_vars.c
===
RCS file: /home/cvspublic/httpd-2.0/modules/ssl/ssl_engine_vars.c,v
retrieving revision 1.20
diff -u -r1.20 ssl_engine_vars.c
--- ssl_engine_vars.c 28 May 2002 21:47:31 - 1.20
+++ ssl_engine_vars.c 23 Oct 2002 23:51:25 -
@@ -334,7 +334,7 @@
xsname = X509_get_subject_name(xs);
cp = X509_NAME_oneline(xsname, NULL, 0);
result = apr_pstrdup(p, cp);
-free(cp);
+modssl_free(cp);
resdup = FALSE;
}
else if (strlen(var) 5 strcEQn(var, S_DN_, 5)) {
@@ -346,7 +346,7 @@
xsname = X509_get_issuer_name(xs);
cp = X509_NAME_oneline(xsname, NULL, 0);
result = apr_pstrdup(p, cp);
-free(cp);
+modssl_free(cp);
resdup = FALSE;
}
else if (strlen(var) 5 strcEQn(var, I_DN_, 5)) {
Index: ssl_toolkit_compat.h
===
RCS file: /home/cvspublic/httpd-2.0/modules/ssl/ssl_toolkit_compat.h,v
retrieving revision 1.25
diff -u -r1.25 ssl_toolkit_compat.h
--- ssl_toolkit_compat.h21 Aug 2002 19:12:46 - 1.25
+++ ssl_toolkit_compat.h23 Oct 2002 23:46:38 -
@@ -105,6 +105,8 @@
#define modssl_set_cipher_list SSL_set_cipher_list
+#define modssl_free OPENSSL_free
+
#define EVP_PKEY_reference_inc(pkey) \
CRYPTO_add(((pkey)-references), +1, CRYPTO_LOCK_X509_PKEY)
@@ -147,6 +149,8 @@
#define modssl_set_cipher_list(ssl, l) \
SSL_set_cipher_list(ssl, (char *)l)
+
+#define modssl_free free
#ifndef PEM_F_DEF_CALLBACK
#define PEM_F_DEF_CALLBACK PEM_F_DEF_CB
[PATCH]Question - regarding modssl_PEM_read_bio_X509
Hi Jeff,
Since you're reviewing the other mod_ssl patch, can you pl. review
the following patch also ?..
Thanks
-Madhu
-Original Message-
From: MATHIHALLI,MADHUSUDAN (HP-Cupertino,ex1)
[mailto:madhusudan_mathihalli;hp.com]
Sent: Tuesday, October 22, 2002 11:05 AM
To: '[EMAIL PROTECTED]'
Subject: [PATCH]Question - regarding modssl_PEM_read_bio_X509
I thought modssl_PEM_read_bio_X509 should cover the following cases for
OpenSSL API :
#if (SSL_LIBRARY_VERSION 0x00904000)
#define modssl_PEM_read_bio_X509 SOME WAY
#else
#define modssl_PEM_read_bio_X509 OTHER WAY
#endif
The following patch does something similar, and also changes one other place
in ssl_util_ssl.c where PEM_read_bio_X509 was still being used.
-Madhu
Index: ssl_toolkit_compat.h
===
RCS file: /home/cvspublic/httpd-2.0/modules/ssl/ssl_toolkit_compat.h,v
retrieving revision 1.25
diff -u -r1.25 ssl_toolkit_compat.h
--- ssl_toolkit_compat.h21 Aug 2002 19:12:46 - 1.25
+++ ssl_toolkit_compat.h22 Oct 2002 18:01:44 -
@@ -97,7 +97,11 @@
#define modssl_X509_verify_cert X509_verify_cert
-#define modssl_PEM_read_bio_X509 PEM_read_bio_X509
+#if (SSL_LIBRARY_VERSION 0x00904000)
+#define modssl_PEM_read_bio_X509(b, x, cb, arg) PEM_read_bio_X509(b, x, cb)
+#else
+#define modssl_PEM_read_bio_X509(b, x, cb, arg) PEM_read_bio_X509(b, x, cb,
arg)
+#endif
Index: ssl_util_ssl.c
===
RCS file: /home/cvspublic/httpd-2.0/modules/ssl/ssl_util_ssl.c,v
retrieving revision 1.21
diff -u -r1.21 ssl_util_ssl.c
--- ssl_util_ssl.c 15 Sep 2002 00:00:48 - 1.21
+++ ssl_util_ssl.c 22 Oct 2002 17:59:00 -
@@ -519,11 +519,7 @@
}
/* create new extra chain by loading the certs */
n = 0;
-#if SSL_LIBRARY_VERSION 0x00904000
-while ((x509 = PEM_read_bio_X509(bio, NULL, cb)) != NULL) {
-#else
-while ((x509 = PEM_read_bio_X509(bio, NULL, cb, NULL)) != NULL) {
-#endif
+while ((x509 = modssl_PEM_read_bio_X509(bio, NULL, cb, NULL)) != NULL)
{
if (!SSL_CTX_add_extra_chain_cert(ctx, x509)) {
X509_free(x509);
BIO_free(bio);
RE: [PATCH] Try to use OPENSSL_free instead of free
At 06:17 PM 10/24/2002, MATHIHALLI,MADHUSUDAN (HP-Cupertino,ex1) wrote: Can somebody _please_ take some time to review and tell me if the patch is okay. I will sign up for this... Just a observation : I do understand that everybody is busy doing their own set of things, but then - do you really solicit patches from non-committers ?. I've been a silent observer since the last couple of months, and as-per my observation, everytime a non-committer sends a patch, it takes a couple of cycles before somebody even responds back to the patch (let alone commit it). When somebody posts a patch, I'm sure he/she definitely looks forward for some sort of input from you guys (the core developers/committers, best of the breed etc) - atleast something like its all crap / you may want to do this also / this is all good / something else. Am I wrong here ?.. No. Unfortunately, most of us have day jobs that preclude us from really addressing these things quickly enough. Plus, one needs to get into the 'ssl mode' or 'ldap mode' or whatnot, and usually that means blocking out enough time to look at all the relevant bugs and patches to a given module, thinking it all through, and getting the fixes committed. But please make no mistake, Apache HTTP Server would not survive on the codecraft of it's core contributors!!! It takes folks out there looking for mistakes and possible solutions. Sometimes, the solution is dead to right, and sometimes it's in the right spirit but needs some work. Sure, there are oddball patches that don't belong (the code shouldn't even be changed, the coder misunderstood the intent or we did a lousy job documenting the existing behavior) but those are few and far between. Does anybody not want a response when they post a patch ?. Talking specifically of my patches, the majority of the commits to the mod_ssl source was done by Cliff / Justin / DougM. I do realize that they're busy with other things also - but then, I've not seen many others doing any development on the SSL front. So, whom do I approach to solicit any feedback I struggled with that for months, being full of Win32 specific patches and only one active and two somewhat passive committers who had any interest. All that said, things do get applied. Sometimes under a bit of irritated contributor feedback. Sorry for the delay, and I will get looking at those patches by tomorrow. Bill
Re: [PATCH] Apache styling for ssl_util_ssl.c
MATHIHALLI,MADHUSUDAN (HP-Cupertino,ex1) [EMAIL PROTECTED] writes: Some pieces of the code in ssl_util_ssl.c were not aligned properly - the following patch makes it more readable. committed, thanks! -- Jeff Trawick | [EMAIL PROTECTED] Born in Roswell... married an alien...
Re: [STATUS] (httpd-2.0) Wed Oct 23 23:45:19 EDT 2002
Quoting David Burry [EMAIL PROTECTED]: Excellent! I will perform some tests with that when I get a chance! You managed to get it working without breaking pipelining even? That's awesome! That's what I *think*, which has been known to deviate from the truth, from time to time. However, I appreciate all input, especially results of the actual tests. Bojan
Re: [STATUS] (httpd-2.0) Wed Oct 23 23:45:19 EDT 2002
On Thu, Oct 24, 2002 at 05:25:46PM -0700, David Burry wrote:
Excellent! I will perform some tests with that when I get a chance! You managed to
get it working without breaking pipelining even? That's awesome!
Not meaning to belittle Bojan's hard work, but for my purposes mod_logio values are
not as good as %b would be if %b worked properly... what I ideally need is the byte
sent count without the headers... using Bojan's module I can get approximate results
but they will be a hair off because they include headers... My main purpose is to
detect if and when several meg files have been downloaded all the way vs. if they
were cut off in the middle, including if a given user uses some byte-ranging download
manager that lets you pause and restart... We also use it for chargebacks to the
various departments for bandwidth usage (in this case mod_logio would of course be
more accurate than %b though)... We actually had to fudge some of our statistics
(duplicated nearby days' data with similar overall throughputs) due to us not
catching the problem with Apache 2.0 soon enough...
Have you looked at the %...X directive in Apache2?
http://httpd.apache.org/docs-2.0/mod/mod_log_config.html
%...X: Connection status when response is completed.
'X' = connection aborted before the response completed.
'+' = connection may be kept alive after the response is sent.
'-' = connection will be closed after the response is sent.
(This directive was %...c in late versions of Apache 1.3, but
this conflicted with the historical ssl %...{var}c syntax.)
I don't know if this works in Apache2, but this is what the docs say.
Discussion of Apache2 design on this list would suggest that it might
suffer from the same limitation that %b currently exhibits.
(In the past, I used the %c in Apache 1.3 with success when tallying
whether or not a download of a .cab installer completed successfully.
And I didn't have to know the size of the file for comparison. Nice!)
-Glenn
At 09:15 AM 10/25/2002 +1000, Bojan Smojver wrote:
On Fri, 2002-10-25 at 07:42, David Burry wrote:
I see.. ok, I'll keep waiting patiently...
The patch for 2.0.43 is here:
ftp://ftp.rexursive.com/pub/apache/counting_io_flush-2.0.43.patch
You need to apply mod_logio patch for 2.0.43 first.
Bojan
Re: [STATUS] (httpd-2.0) Wed Oct 23 23:45:19 EDT 2002
At 08:40 PM 10/24/2002, Bojan Smojver wrote: Quoting David Burry [EMAIL PROTECTED]: Excellent! I will perform some tests with that when I get a chance! You managed to get it working without breaking pipelining even? That's awesome! That's what I *think*, which has been known to deviate from the truth, from time to time. However, I appreciate all input, especially results of the actual tests. Bojon, I recall you had tested a ton of 'little files' pipelined. What might be more interesting is a 100MB download (over a fast pipe) which is entirely 'sendfile'd out. Apache would consider itself done with the request long before it was finished with the connection. Just a thought. Bill
Re: [STATUS] (httpd-2.0) Wed Oct 23 23:45:19 EDT 2002
I recall you had tested a ton of 'little files' pipelined. What might be more interesting is a 100MB download (over a fast pipe) which is entirely 'sendfile'd out. Apache would consider itself done with the request long before it was finished with the connection. I tested with an 80 MB file and got different numbers depending on when I clicked stop in the browser. The testing was done on 127.0.0.1 (733 MHz Celeron box), so guess it qualifies as a fast pipe. Is that what you meant? Or did you mean pipelined large files? However, I do have a question for you (or anyone else that understands Apache 2.0 internal dymamics better then me). Currently, my patch fetches the optional function every time the connection is created (i.e. in core_pre_connection). I reckon this should actually be done in core_post_config, only once. Do you think that's safe (i.e. I'm guessing function pointers should be OK on graceful restarts, forks etc.).? Bojan
Re: [patch] perchild MPM bug fixes (+ open problem)
Johannes Erdfelt [EMAIL PROTECTED] writes: Problem 1: In worker_thread, there is a variable called csd that is used to get the new socket from lr-accept_func(). If that variable is NULL, then the memory for the new socket is allocated in the per-transaction pool. Unfortunately, the code neglected to reset the variable to NULL after servicing a request. The result is that the first request for each thread worked fine, but subsequent request may have the memory overwritten and resulting in an invalid FD. Solution: Set it to NULL before calling lr-accept_func(). Implemented in the patch below. I cleared that storage just before the call to apr_os_sock_put() so that the reason is more obvious. Problem 2: pass_request fills out an iovec with the headers and the body of the request it wants to pass to another process. It unfortunately uses the wrong variable for the length causing it to always be 0. Solution: Use the correct variable name l. len is never set to anything so I removed that and used 0 in the one reference to it. Implemented in the patch below. applied, but I made another fix (?) too... see my note below, and let me know if it is bad :) Problem 3: receive_from_other_child assumes that the iovec is the same on read as it is on write. This isn't true and readmsg() follows readv() semantics. iovec is a scatter/gather list and as a result, the 2 send buffers are merged into one received buffer with the second always being untouched. It also trusted the lengths in iov.iov_len which will be the size of the original buffer, not the size of the data actually received. Solution: Merge the 2 buffer's into 1 and find the null terminators for the 2 strings. Implemented in the patch below. fix applied as-is Index: perchild.c @@ -1635,7 +1645,6 @@ apr_bucket_brigade *bb = apr_brigade_create(r-pool, c-bucket_alloc); apr_bucket_brigade *sockbb; char request_body[HUGE_STRING_LEN] = \0; -apr_off_t len = 0; looks to me like len should be initialized to sizeof(request_body) since on input to apr_brigade_flatten it should have the maximum length of the char array Thanks! -- Jeff Trawick | [EMAIL PROTECTED] Born in Roswell... married an alien...
Re: [STATUS] (httpd-2.0) Wed Oct 23 23:45:19 EDT 2002
At 09:38 PM 10/24/2002 -0400, Glenn wrote: Have you looked at the %...X directive in Apache2? That's an interesting idea I hadn't thought of... it doesn't solve the chargeback issue but it's worth investigating for detecting successful downloads... Dave
Re: [PATCH]: Counting I/O with FLUSH and OFN (no MMN bump)
Brian, Bill,
This is what I meant by my previous comment about core_pre_connection v.
core_post_config and the optional function fetching. I have tested this and it
worked even after a graceful restart and when I killed all child processes
manually. I'm using prefork on Linux.
This patch should be faster (one less hash table lookup per connection) as it
fetches the optional function only once in core_post_config.
Bojan
Quoting Brian Pane [EMAIL PROTECTED]:
Bojan Smojver wrote:
This should really work now and it should not cause major dramas
compatibility wise. Let me know what you think.
Thanks, I'll take a look at this tonight unless anyone else
gets to it first.
diff -ruN httpd-2.0-vanilla/include/http_core.h httpd-2.0/include/http_core.h
--- httpd-2.0-vanilla/include/http_core.h Tue Oct 15 08:12:57 2002
+++ httpd-2.0/include/http_core.h Fri Oct 25 12:24:02 2002
@@ -61,6 +61,7 @@
#include apr.h
#include apr_hash.h
+#include apr_optional.h
#include util_filter.h
#if APR_HAVE_STRUCT_RLIMIT
@@ -626,6 +627,16 @@
/* -- */
+/* --
+ *
+ * I/O logging with mod_logio
+ */
+
+APR_DECLARE_OPTIONAL_FN(void, ap_logio_add_bytes_out,
+(conn_rec *c, apr_off_t bytes));
+
+/* -- */
+
#ifdef __cplusplus
}
#endif
diff -ruN httpd-2.0-vanilla/modules/loggers/mod_logio.c
httpd-2.0/modules/loggers/mod_logio.c
--- httpd-2.0-vanilla/modules/loggers/mod_logio.c Sat Sep 28 14:18:35 2002
+++ httpd-2.0/modules/loggers/mod_logio.c Fri Oct 25 12:24:03 2002
@@ -79,6 +79,7 @@
#include ap_config.h
#include mod_log_config.h
#include httpd.h
+#include http_core.h
#include http_config.h
#include http_protocol.h
@@ -96,6 +97,16 @@
} logio_config_t;
/*
+ * Optional function for the core to add to bytes_out
+ */
+
+static void ap_logio_add_bytes_out(conn_rec *c, apr_off_t bytes){
+logio_config_t *cf = ap_get_module_config(c-conn_config, logio_module);
+
+cf-bytes_out += bytes;
+}
+
+/*
* Format items...
*/
@@ -133,24 +144,6 @@
* Logging of input and output filters...
*/
-static apr_status_t logio_out_filter(ap_filter_t *f,
- apr_bucket_brigade *bb) {
-apr_off_t length;
-logio_config_t *cf = ap_get_module_config(f-c-conn_config, logio_module);
-
-if (!cf) { /* Create config */
-cf = apr_pcalloc(f-c-pool, sizeof(*cf));
-ap_set_module_config(f-c-conn_config, logio_module, cf);
-}
-
-apr_brigade_length (bb, 0, length);
-
-if (length 0)
-cf-bytes_out += length;
-
-return ap_pass_brigade(f-next, bb);
-}
-
static apr_status_t logio_in_filter(ap_filter_t *f,
apr_bucket_brigade *bb,
ap_input_mode_t mode,
@@ -162,11 +155,6 @@
status = ap_get_brigade(f-next, bb, mode, block, readbytes);
-if (!cf) { /* Create config */
-cf = apr_pcalloc(f-c-pool, sizeof(*cf));
-ap_set_module_config(f-c-conn_config, logio_module, cf);
-}
-
apr_brigade_length (bb, 0, length);
if (length 0)
@@ -175,11 +163,30 @@
return status;
}
+static apr_status_t logio_out_filter(ap_filter_t *f,
+ apr_bucket_brigade *bb) {
+apr_bucket *b = APR_BRIGADE_LAST(bb);
+
+/* End of data, make sure we flush */
+if (APR_BUCKET_IS_EOS(b)) {
+APR_BRIGADE_INSERT_TAIL(bb,
+apr_bucket_flush_create(f-c-bucket_alloc));
+APR_BUCKET_REMOVE(b);
+apr_bucket_destroy(b);
+}
+
+return ap_pass_brigade(f-next, bb);
+}
+
/*
* The hooks...
*/
static int logio_pre_conn(conn_rec *c) {
+logio_config_t *cf = apr_pcalloc(c-pool, sizeof(*cf));
+
+ap_set_module_config(c-conn_config, logio_module, cf);
+
ap_add_input_filter(logio_filter_name, NULL, NULL, c);
ap_add_output_filter(logio_filter_name, NULL, NULL, c);
@@ -212,6 +219,8 @@
AP_FTYPE_NETWORK - 1);
ap_register_output_filter(logio_filter_name, logio_out_filter, NULL,
AP_FTYPE_NETWORK - 1);
+
+APR_REGISTER_OPTIONAL_FN(ap_logio_add_bytes_out);
}
module AP_MODULE_DECLARE_DATA logio_module =
diff -ruN httpd-2.0-vanilla/server/core.c httpd-2.0/server/core.c
--- httpd-2.0-vanilla/server/core.c Tue Oct 15 08:12:59 2002
+++ httpd-2.0/server/core.c Fri Oct 25 12:40:29 2002
@@ -2737,6 +2737,7 @@
apr_off_t file_offset,
apr_size_t file_bytes_left,
apr_size_t total_bytes_left,
+apr_size_t *bytes_sent,
apr_int32_t flags)
{
apr_status_t rv;
@@
Re: [patch] perchild MPM bug fixes (+ open problem)
On Thu, Oct 24, 2002, Jeff Trawick [EMAIL PROTECTED] wrote: Johannes Erdfelt [EMAIL PROTECTED] writes: Problem 2: pass_request fills out an iovec with the headers and the body of the request it wants to pass to another process. It unfortunately uses the wrong variable for the length causing it to always be 0. Solution: Use the correct variable name l. len is never set to anything so I removed that and used 0 in the one reference to it. Implemented in the patch below. applied, but I made another fix (?) too... see my note below, and let me know if it is bad :) Index: perchild.c @@ -1635,7 +1645,6 @@ apr_bucket_brigade *bb = apr_brigade_create(r-pool, c-bucket_alloc); apr_bucket_brigade *sockbb; char request_body[HUGE_STRING_LEN] = \0; -apr_off_t len = 0; looks to me like len should be initialized to sizeof(request_body) since on input to apr_brigade_flatten it should have the maximum length of the char array I'll have to defer to you here. I'm not that familiar with the brigade implementation to know what the appropriate fix for this is. However, it most certainly is better than the current behaviour so I'm not against it. If it's wrong, I'll find out pretty soon :) Thanks for applying these patches! JE
Re: [STATUS] (httpd-2.0) Wed Oct 23 23:45:19 EDT 2002
At 08:45 PM 10/24/2002 -0500, William A. Rowe, Jr. wrote:
At 08:40 PM 10/24/2002, Bojan Smojver wrote:
Quoting David Burry [EMAIL PROTECTED]:
Excellent! I will perform some tests with that when I get a chance! You
managed to get it working without breaking pipelining even? That's awesome!
That's what I *think*, which has been known to deviate from the truth, from time
to time. However, I appreciate all input, especially results of the actual tests.
I recall you had tested a ton of 'little files' pipelined.
What might be more interesting is a 100MB download (over a fast pipe)
which is entirely 'sendfile'd out. Apache would consider itself done with
the request long before it was finished with the connection.
In case someone else wants to independently verify it...
The exact test I was doing was with a 70+ meg .tar.gz file both over a 100mbit
ethernet and a 1.5mbit DSL, starting and canceling it multiple times in Windoze
Internet Explorer 5 or 6 (which appears to effectively use byte range requests for
subsequent tries, by the way) and monitoring what was logged each time. This test
isn't super precise on the byte count (I did not bother to go comb my IE cache) but it
sure is obvious when it consistently logs the whole file size and I only received a
small fraction according to the IE progress bar... Also looking at the byte range
requests with %{Range}i makes it obvious how much IE received previously on each
subsequent try (IE appears to only request the part of the file it hasn't cached yet).
I was thinking of writing a script that did this in a more automated fashion...
perhaps contributing a test to the apache test suite when I figure that thing out...
;o)
Dave
SOS! HELP- APACHE PROCESS PROBLEM
Hi, We have developed a BioInformatics Web based Application using CGIC , Linux and Apache as the webserver. We are using MySQL database for data handling. We have run into a problem wherein if the browser which has sent the request to Apache is closed in between a process, the process continues to run instead of being terminated. This causes the queuing of unwanted processes (both for Apache and MySQL). Can you suggest some solutions to this problem? Is there any way to programatically trap the browser generated( e.g when the browser is closed or stop button is pressed) events in Apache or CGIC? Looking forward to an early reply. regards Chandragupt TCG Software Services (P) Ltd. India Cell Nos. 09818022792 09810255383 09810230704 On Fri, 25 Oct 2002 David Burry wrote : At 09:38 PM 10/24/2002 -0400, Glenn wrote: Have you looked at the %...X directive in Apache2? That's an interesting idea I hadn't thought of... it doesn't solve the chargeback issue but it's worth investigating for detecting successful downloads... Dave __ Give your Company an email address like ravi ravi-exports.com. Sign up for Rediffmail Pro today! Know more. http://www.rediffmailpro.com/signup/
Re: [STATUS] (httpd-2.0) Wed Oct 23 23:45:19 EDT 2002
Quoting Bojan Smojver [EMAIL PROTECTED]: Just realised that the URL is wrong :-( So, here is the correct one: ftp://ftp.rexursive.com/pub/apache/counting_io_flush-2.0.43.patch.gz Bojan On Fri, 2002-10-25 at 07:42, David Burry wrote: I see.. ok, I'll keep waiting patiently... The patch for 2.0.43 is here: ftp://ftp.rexursive.com/pub/apache/counting_io_flush-2.0.43.patch You need to apply mod_logio patch for 2.0.43 first. Bojan
mod_logio patch testing Re: [STATUS] (httpd-2.0) Wed Oct 2323:45:19 EDT 2002
On Thu, 2002-10-24 at 19:45, David Burry wrote:
At 08:45 PM 10/24/2002 -0500, William A. Rowe, Jr. wrote:
At 08:40 PM 10/24/2002, Bojan Smojver wrote:
Quoting David Burry [EMAIL PROTECTED]:
Excellent! I will perform some tests with that when I get a chance! You
managed to get it working without breaking pipelining even? That's awesome!
That's what I *think*, which has been known to deviate from the truth, from time
to time. However, I appreciate all input, especially results of the actual tests.
I recall you had tested a ton of 'little files' pipelined.
What might be more interesting is a 100MB download (over a fast pipe)
which is entirely 'sendfile'd out. Apache would consider itself done with
the request long before it was finished with the connection.
In case someone else wants to independently verify it...
The exact test I was doing was with a 70+ meg .tar.gz file both over a 100mbit
ethernet and a 1.5mbit DSL, starting and canceling it multiple times in Windoze
Internet Explorer 5 or 6 (which appears to effectively use byte range requests for
subsequent tries, by the way) and monitoring what was logged each time. This test
isn't super precise on the byte count (I did not bother to go comb my IE cache) but
it sure is obvious when it consistently logs the whole file size and I only received
a small fraction according to the IE progress bar... Also looking at the byte range
requests with %{Range}i makes it obvious how much IE received previously on each
subsequent try (IE appears to only request the part of the file it hasn't cached yet).
I was thinking of writing a script that did this in a more automated fashion...
perhaps contributing a test to the apache test suite when I figure that thing out...
;o)
I just tried a similar test, using Bojan's latest patch. Using
IE 5.5 on Windows 2k, I repeatedly requested a 32MB file from Apache
w/mod_logio (on Linux with sendfile) and stopped the transfer before
the browser had received the whole response. Each time, the bytes-sent
value in the httpd access log was smaller than the file size.
I got similar results when using ab over the loopback as a test
client.
The only test case in which mod_logio didn't report as small a
byte count as expected was when I used telnet as the client and
stopped it (with ctrl-C) while the server was still sending the
response. A syscall trace on the server showed that, after I
interrupted telnet, the httpd did the remainder of its sendfile
calls (each one of which managed to send the next 48KB of the
requestd file; that appears to be a kernel limit) very, very
quickly. I think that's just because telnet did a half-close
and continued reading and discarding data until the server
closed its end of the connection before actually exiting.
Based on the test results, the patch does appear to be an
improvement over the current code. I'm planning to commit
it tonight.
Brian
