Re: [PATCH]: Counting I/O with FLUSH and OFN (no MMN bump)
On Thu, 2002-10-24 at 19:48, Bojan Smojver wrote: > Brian, Bill, > > This is what I meant by my previous comment about core_pre_connection v. > core_post_config and the optional function fetching. I have tested this and it > worked even after a graceful restart and when I killed all child processes > manually. I'm using prefork on Linux. > > This patch should be faster (one less hash table lookup per connection) as it > fetches the optional function only once in core_post_config. Tested and committed. Thanks! Brian
mod_logio patch testing Re: [STATUS] (httpd-2.0) Wed Oct 2323:45:19 EDT 2002
On Thu, 2002-10-24 at 19:45, David Burry wrote: > At 08:45 PM 10/24/2002 -0500, William A. Rowe, Jr. wrote: > >At 08:40 PM 10/24/2002, Bojan Smojver wrote: > >>Quoting David Burry <[EMAIL PROTECTED]>: > >> > >>> Excellent! I will perform some tests with that when I get a chance! You > >>> managed to get it working without breaking pipelining even? That's awesome! > >> > >>That's what I *think*, which has been known to deviate from the truth, from time > >>to time. However, I appreciate all input, especially results of the actual tests. > > > > I recall you had tested a ton of 'little files' pipelined. > > > > What might be more interesting is a 100MB download (over a fast pipe) > >which is entirely 'sendfile'd out. Apache would consider itself done with > >the request long before it was finished with the connection. > > > In case someone else wants to independently verify it... > > The exact test I was doing was with a 70+ meg .tar.gz file both over a 100mbit >ethernet and a 1.5mbit DSL, starting and canceling it multiple times in Windoze >Internet Explorer 5 or 6 (which appears to effectively use byte range requests for >subsequent tries, by the way) and monitoring what was logged each time. This test >isn't super precise on the byte count (I did not bother to go comb my IE cache) but >it sure is obvious when it consistently logs the whole file size and I only received >a small fraction according to the IE progress bar... Also looking at the byte range >requests with %{Range}i makes it obvious how much IE received previously on each >subsequent try (IE appears to only request the part of the file it hasn't cached yet). > > I was thinking of writing a script that did this in a more automated fashion... >perhaps contributing a test to the apache test suite when I figure that thing out... >;o) I just tried a similar test, using Bojan's latest patch. Using IE 5.5 on Windows 2k, I repeatedly requested a 32MB file from Apache w/mod_logio (on Linux with sendfile) and stopped the transfer before the browser had received the whole response. Each time, the bytes-sent value in the httpd access log was smaller than the file size. I got similar results when using ab over the loopback as a test client. The only test case in which mod_logio didn't report as small a byte count as expected was when I used telnet as the client and stopped it (with ctrl-C) while the server was still sending the response. A syscall trace on the server showed that, after I interrupted telnet, the httpd did the remainder of its sendfile calls (each one of which managed to send the next 48KB of the requestd file; that appears to be a kernel limit) very, very quickly. I think that's just because telnet did a half-close and continued reading and discarding data until the server closed its end of the connection before actually exiting. Based on the test results, the patch does appear to be an improvement over the current code. I'm planning to commit it tonight. Brian
Re: [STATUS] (httpd-2.0) Wed Oct 23 23:45:19 EDT 2002
Quoting Bojan Smojver <[EMAIL PROTECTED]>: Just realised that the URL is wrong :-( So, here is the correct one: ftp://ftp.rexursive.com/pub/apache/counting_io_flush-2.0.43.patch.gz Bojan > On Fri, 2002-10-25 at 07:42, David Burry wrote: > > > I see.. ok, I'll keep waiting patiently... > > The patch for 2.0.43 is here: > > ftp://ftp.rexursive.com/pub/apache/counting_io_flush-2.0.43.patch > > You need to apply mod_logio patch for 2.0.43 first. > > Bojan > >
SOS! HELP- APACHE PROCESS PROBLEM
Hi, We have developed a BioInformatics Web based Application using CGIC , Linux and Apache as the webserver. We are using MySQL database for data handling. We have run into a problem wherein if the browser which has sent the request to Apache is closed in between a process, the process continues to run instead of being terminated. This causes the queuing of unwanted processes (both for Apache and MySQL). Can you suggest some solutions to this problem? Is there any way to programatically trap the browser generated( e.g when the browser is closed or stop button is pressed) events in Apache or CGIC? Looking forward to an early reply. regards Chandragupt TCG Software Services (P) Ltd. India Cell Nos. 09818022792 09810255383 09810230704 On Fri, 25 Oct 2002 David Burry wrote : At 09:38 PM 10/24/2002 -0400, Glenn wrote: >Have you looked at the %...X directive in Apache2? That's an interesting idea I hadn't thought of... it doesn't solve the chargeback issue but it's worth investigating for detecting successful downloads... Dave __ Give your Company an email address like ravi @ ravi-exports.com. Sign up for Rediffmail Pro today! Know more. http://www.rediffmailpro.com/signup/
Re: [STATUS] (httpd-2.0) Wed Oct 23 23:45:19 EDT 2002
At 08:45 PM 10/24/2002 -0500, William A. Rowe, Jr. wrote: >At 08:40 PM 10/24/2002, Bojan Smojver wrote: >>Quoting David Burry <[EMAIL PROTECTED]>: >> >>> Excellent! I will perform some tests with that when I get a chance! You >>> managed to get it working without breaking pipelining even? That's awesome! >> >>That's what I *think*, which has been known to deviate from the truth, from time >>to time. However, I appreciate all input, especially results of the actual tests. > > I recall you had tested a ton of 'little files' pipelined. > > What might be more interesting is a 100MB download (over a fast pipe) >which is entirely 'sendfile'd out. Apache would consider itself done with >the request long before it was finished with the connection. In case someone else wants to independently verify it... The exact test I was doing was with a 70+ meg .tar.gz file both over a 100mbit ethernet and a 1.5mbit DSL, starting and canceling it multiple times in Windoze Internet Explorer 5 or 6 (which appears to effectively use byte range requests for subsequent tries, by the way) and monitoring what was logged each time. This test isn't super precise on the byte count (I did not bother to go comb my IE cache) but it sure is obvious when it consistently logs the whole file size and I only received a small fraction according to the IE progress bar... Also looking at the byte range requests with %{Range}i makes it obvious how much IE received previously on each subsequent try (IE appears to only request the part of the file it hasn't cached yet). I was thinking of writing a script that did this in a more automated fashion... perhaps contributing a test to the apache test suite when I figure that thing out... ;o) Dave
Re: [patch] perchild MPM bug fixes (+ open problem)
On Thu, Oct 24, 2002, Jeff Trawick <[EMAIL PROTECTED]> wrote: > Johannes Erdfelt <[EMAIL PROTECTED]> writes: > > Problem 2: > > pass_request fills out an iovec with the headers and the body of the > > request it wants to pass to another process. It unfortunately uses the > > wrong variable for the length causing it to always be 0. > > > > Solution: > > Use the correct variable name "l". len is never set to anything so I removed > > that and used 0 in the one reference to it. Implemented in the patch below. > > applied, but I made another fix (?) too... see my note below, and let > me know if it is bad :) > > > Index: perchild.c > > @@ -1635,7 +1645,6 @@ > > apr_bucket_brigade *bb = apr_brigade_create(r->pool, c->bucket_alloc); > > apr_bucket_brigade *sockbb; > > char request_body[HUGE_STRING_LEN] = "\0"; > > -apr_off_t len = 0; > > looks to me like len should be initialized to sizeof(request_body) > since on input to apr_brigade_flatten it should have the maximum > length of the char array I'll have to defer to you here. I'm not that familiar with the brigade implementation to know what the appropriate fix for this is. However, it most certainly is better than the current behaviour so I'm not against it. If it's wrong, I'll find out pretty soon :) Thanks for applying these patches! JE
Re: [PATCH]: Counting I/O with FLUSH and OFN (no MMN bump)
Brian, Bill, This is what I meant by my previous comment about core_pre_connection v. core_post_config and the optional function fetching. I have tested this and it worked even after a graceful restart and when I killed all child processes manually. I'm using prefork on Linux. This patch should be faster (one less hash table lookup per connection) as it fetches the optional function only once in core_post_config. Bojan Quoting Brian Pane <[EMAIL PROTECTED]>: > Bojan Smojver wrote: > > >This should really work now and it should not cause major dramas > >compatibility wise. Let me know what you think. > > > > > > Thanks, I'll take a look at this tonight unless anyone else > gets to it first. diff -ruN httpd-2.0-vanilla/include/http_core.h httpd-2.0/include/http_core.h --- httpd-2.0-vanilla/include/http_core.h Tue Oct 15 08:12:57 2002 +++ httpd-2.0/include/http_core.h Fri Oct 25 12:24:02 2002 @@ -61,6 +61,7 @@ #include "apr.h" #include "apr_hash.h" +#include "apr_optional.h" #include "util_filter.h" #if APR_HAVE_STRUCT_RLIMIT @@ -626,6 +627,16 @@ /* -- */ +/* -- + * + * I/O logging with mod_logio + */ + +APR_DECLARE_OPTIONAL_FN(void, ap_logio_add_bytes_out, +(conn_rec *c, apr_off_t bytes)); + +/* -- */ + #ifdef __cplusplus } #endif diff -ruN httpd-2.0-vanilla/modules/loggers/mod_logio.c httpd-2.0/modules/loggers/mod_logio.c --- httpd-2.0-vanilla/modules/loggers/mod_logio.c Sat Sep 28 14:18:35 2002 +++ httpd-2.0/modules/loggers/mod_logio.c Fri Oct 25 12:24:03 2002 @@ -79,6 +79,7 @@ #include "ap_config.h" #include "mod_log_config.h" #include "httpd.h" +#include "http_core.h" #include "http_config.h" #include "http_protocol.h" @@ -96,6 +97,16 @@ } logio_config_t; /* + * Optional function for the core to add to bytes_out + */ + +static void ap_logio_add_bytes_out(conn_rec *c, apr_off_t bytes){ +logio_config_t *cf = ap_get_module_config(c->conn_config, &logio_module); + +cf->bytes_out += bytes; +} + +/* * Format items... */ @@ -133,24 +144,6 @@ * Logging of input and output filters... */ -static apr_status_t logio_out_filter(ap_filter_t *f, - apr_bucket_brigade *bb) { -apr_off_t length; -logio_config_t *cf = ap_get_module_config(f->c->conn_config, &logio_module); - -if (!cf) { /* Create config */ -cf = apr_pcalloc(f->c->pool, sizeof(*cf)); -ap_set_module_config(f->c->conn_config, &logio_module, cf); -} - -apr_brigade_length (bb, 0, &length); - -if (length > 0) -cf->bytes_out += length; - -return ap_pass_brigade(f->next, bb); -} - static apr_status_t logio_in_filter(ap_filter_t *f, apr_bucket_brigade *bb, ap_input_mode_t mode, @@ -162,11 +155,6 @@ status = ap_get_brigade(f->next, bb, mode, block, readbytes); -if (!cf) { /* Create config */ -cf = apr_pcalloc(f->c->pool, sizeof(*cf)); -ap_set_module_config(f->c->conn_config, &logio_module, cf); -} - apr_brigade_length (bb, 0, &length); if (length > 0) @@ -175,11 +163,30 @@ return status; } +static apr_status_t logio_out_filter(ap_filter_t *f, + apr_bucket_brigade *bb) { +apr_bucket *b = APR_BRIGADE_LAST(bb); + +/* End of data, make sure we flush */ +if (APR_BUCKET_IS_EOS(b)) { +APR_BRIGADE_INSERT_TAIL(bb, +apr_bucket_flush_create(f->c->bucket_alloc)); +APR_BUCKET_REMOVE(b); +apr_bucket_destroy(b); +} + +return ap_pass_brigade(f->next, bb); +} + /* * The hooks... */ static int logio_pre_conn(conn_rec *c) { +logio_config_t *cf = apr_pcalloc(c->pool, sizeof(*cf)); + +ap_set_module_config(c->conn_config, &logio_module, cf); + ap_add_input_filter(logio_filter_name, NULL, NULL, c); ap_add_output_filter(logio_filter_name, NULL, NULL, c); @@ -212,6 +219,8 @@ AP_FTYPE_NETWORK - 1); ap_register_output_filter(logio_filter_name, logio_out_filter, NULL, AP_FTYPE_NETWORK - 1); + +APR_REGISTER_OPTIONAL_FN(ap_logio_add_bytes_out); } module AP_MODULE_DECLARE_DATA logio_module = diff -ruN httpd-2.0-vanilla/server/core.c httpd-2.0/server/core.c --- httpd-2.0-vanilla/server/core.c Tue Oct 15 08:12:59 2002 +++ httpd-2.0/server/core.c Fri Oct 25 12:40:29 2002 @@ -2737,6 +2737,7 @@ apr_off_t file_offset, apr_size_t file_bytes_left, apr_size_t total_bytes_left, +apr_size_t *bytes_sent,
Re: [STATUS] (httpd-2.0) Wed Oct 23 23:45:19 EDT 2002
At 09:38 PM 10/24/2002 -0400, Glenn wrote: >Have you looked at the %...X directive in Apache2? That's an interesting idea I hadn't thought of... it doesn't solve the chargeback issue but it's worth investigating for detecting successful downloads... Dave
Re: [patch] perchild MPM bug fixes (+ open problem)
Johannes Erdfelt <[EMAIL PROTECTED]> writes: > Problem 1: > In worker_thread, there is a variable called csd that is used to get > the new socket from lr->accept_func(). If that variable is NULL, then > the memory for the new socket is allocated in the per-transaction pool. > Unfortunately, the code neglected to reset the variable to NULL after > servicing a request. The result is that the first request for each > thread worked fine, but subsequent request may have the memory > overwritten and resulting in an invalid FD. > > Solution: > Set it to NULL before calling lr->accept_func(). Implemented in the patch > below. I cleared that storage just before the call to apr_os_sock_put() so that the reason is more obvious. > Problem 2: > pass_request fills out an iovec with the headers and the body of the > request it wants to pass to another process. It unfortunately uses the > wrong variable for the length causing it to always be 0. > > Solution: > Use the correct variable name "l". len is never set to anything so I removed > that and used 0 in the one reference to it. Implemented in the patch below. applied, but I made another fix (?) too... see my note below, and let me know if it is bad :) > Problem 3: > receive_from_other_child assumes that the iovec is the same on read as > it is on write. This isn't true and readmsg() follows readv() semantics. > iovec is a scatter/gather list and as a result, the 2 send buffers are > merged into one received buffer with the second always being untouched. > It also trusted the lengths in iov.iov_len which will be the size of the > original buffer, not the size of the data actually received. > > Solution: > Merge the 2 buffer's into 1 and find the null terminators for the 2 strings. > Implemented in the patch below. fix applied as-is > Index: perchild.c > @@ -1635,7 +1645,6 @@ > apr_bucket_brigade *bb = apr_brigade_create(r->pool, c->bucket_alloc); > apr_bucket_brigade *sockbb; > char request_body[HUGE_STRING_LEN] = "\0"; > -apr_off_t len = 0; looks to me like len should be initialized to sizeof(request_body) since on input to apr_brigade_flatten it should have the maximum length of the char array Thanks! -- Jeff Trawick | [EMAIL PROTECTED] Born in Roswell... married an alien...
Re: [STATUS] (httpd-2.0) Wed Oct 23 23:45:19 EDT 2002
> I recall you had tested a ton of 'little files' pipelined. > > What might be more interesting is a 100MB download (over a fast pipe) > which is entirely 'sendfile'd out. Apache would consider itself done with > the request long before it was finished with the connection. I tested with an 80 MB file and got different numbers depending on when I clicked stop in the browser. The testing was done on 127.0.0.1 (733 MHz Celeron box), so guess it qualifies as a fast pipe. Is that what you meant? Or did you mean pipelined large files? However, I do have a question for you (or anyone else that understands Apache 2.0 internal dymamics better then me). Currently, my patch fetches the optional function every time the connection is created (i.e. in core_pre_connection). I reckon this should actually be done in core_post_config, only once. Do you think that's safe (i.e. I'm guessing function pointers should be OK on graceful restarts, forks etc.).? Bojan
Re: [STATUS] (httpd-2.0) Wed Oct 23 23:45:19 EDT 2002
At 08:40 PM 10/24/2002, Bojan Smojver wrote: >Quoting David Burry <[EMAIL PROTECTED]>: > >> Excellent! I will perform some tests with that when I get a chance! You >> managed to get it working without breaking pipelining even? That's awesome! > >That's what I *think*, which has been known to deviate from the truth, from time >to time. However, I appreciate all input, especially results of the actual tests. Bojon, I recall you had tested a ton of 'little files' pipelined. What might be more interesting is a 100MB download (over a fast pipe) which is entirely 'sendfile'd out. Apache would consider itself done with the request long before it was finished with the connection. Just a thought. Bill
Re: [STATUS] (httpd-2.0) Wed Oct 23 23:45:19 EDT 2002
On Thu, Oct 24, 2002 at 05:25:46PM -0700, David Burry wrote: > Excellent! I will perform some tests with that when I get a chance! You managed to >get it working without breaking pipelining even? That's awesome! > > Not meaning to belittle Bojan's hard work, but for my purposes mod_logio values are >not as good as %b would be if %b worked properly... what I ideally need is the byte >sent count without the headers... using Bojan's module I can get approximate results >but they will be a hair off because they include headers... My main purpose is to >detect if and when several meg files have been downloaded all the way vs. if they >were cut off in the middle, including if a given user uses some byte-ranging download >manager that lets you pause and restart... We also use it for chargebacks to the >various departments for bandwidth usage (in this case mod_logio would of course be >more accurate than %b though)... We actually had to fudge some of our statistics >(duplicated nearby days' data with similar overall throughputs) due to us not >catching the problem with Apache 2.0 soon enough... Have you looked at the %...X directive in Apache2? http://httpd.apache.org/docs-2.0/mod/mod_log_config.html %...X: Connection status when response is completed. 'X' = connection aborted before the response completed. '+' = connection may be kept alive after the response is sent. '-' = connection will be closed after the response is sent. (This directive was %...c in late versions of Apache 1.3, but this conflicted with the historical ssl %...{var}c syntax.) I don't know if this works in Apache2, but this is what the docs say. Discussion of Apache2 design on this list would suggest that it might suffer from the same limitation that %b currently exhibits. (In the past, I used the %c in Apache 1.3 with success when tallying whether or not a download of a .cab installer completed successfully. And I didn't have to know the size of the file for comparison. Nice!) -Glenn > At 09:15 AM 10/25/2002 +1000, Bojan Smojver wrote: > >On Fri, 2002-10-25 at 07:42, David Burry wrote: > > > >> I see.. ok, I'll keep waiting patiently... > > > >The patch for 2.0.43 is here: > > > >ftp://ftp.rexursive.com/pub/apache/counting_io_flush-2.0.43.patch > > > >You need to apply mod_logio patch for 2.0.43 first. > > > >Bojan
Re: [STATUS] (httpd-2.0) Wed Oct 23 23:45:19 EDT 2002
Quoting David Burry <[EMAIL PROTECTED]>: > Excellent! I will perform some tests with that when I get a chance! You > managed to get it working without breaking pipelining even? That's awesome! That's what I *think*, which has been known to deviate from the truth, from time to time. However, I appreciate all input, especially results of the actual tests. Bojan
Re: [PATCH] Apache styling for ssl_util_ssl.c
"MATHIHALLI,MADHUSUDAN (HP-Cupertino,ex1)" <[EMAIL PROTECTED]> writes: > Some pieces of the code in ssl_util_ssl.c were not aligned properly - the > following patch makes it more readable. committed, thanks! -- Jeff Trawick | [EMAIL PROTECTED] Born in Roswell... married an alien...
RE: [PATCH] Try to use OPENSSL_free instead of free
At 06:17 PM 10/24/2002, MATHIHALLI,MADHUSUDAN (HP-Cupertino,ex1) wrote: >Can somebody _please_ take some time to review and tell me if the patch is >okay. I will sign up for this... >Just a observation : > >I do understand that everybody is busy doing their own set of things, but >then - do you really solicit patches from non-committers ?. > >I've been a silent observer since the last couple of months, and as-per my >observation, everytime a non-committer sends a patch, it takes a couple of >cycles before somebody even responds back to the patch (let alone commit >it). When somebody posts a patch, I'm sure he/she definitely looks forward >for some sort of input from you guys (the core developers/committers, best >of the breed etc) - atleast something like "its all crap" / "you may want to >do this also" / "this is all good" / something else. Am I wrong here ?.. No. Unfortunately, most of us have day jobs that preclude us from really addressing these things quickly enough. Plus, one needs to get into the 'ssl mode' or 'ldap mode' or whatnot, and usually that means blocking out enough time to look at all the relevant bugs and patches to a given module, thinking it all through, and getting the fixes committed. But please make no mistake, Apache HTTP Server would not survive on the codecraft of it's core contributors!!! It takes folks out there looking for mistakes and possible solutions. Sometimes, the solution is dead to right, and sometimes it's in the right spirit but needs some work. Sure, there are oddball patches that don't belong (the code shouldn't even be changed, the coder misunderstood the intent or we did a lousy job documenting the existing behavior) but those are few and far between. >Does anybody not want a response when they post a patch ?. >Talking specifically of my patches, the majority of the commits to the >mod_ssl source was done by Cliff / Justin / DougM. I do realize that they're >busy with other things also - but then, I've not seen many others doing any >development on the SSL front. So, whom do I approach to solicit any feedback I struggled with that for months, being full of Win32 specific patches and only one active and two somewhat passive committers who had any interest. All that said, things do get applied. Sometimes under a bit of irritated contributor feedback. Sorry for the delay, and I will get looking at those patches by tomorrow. Bill
[PATCH]Question - regarding modssl_PEM_read_bio_X509
Hi Jeff, Since you're reviewing the other mod_ssl patch, can you pl. review the following patch also ?.. Thanks -Madhu -Original Message- From: MATHIHALLI,MADHUSUDAN (HP-Cupertino,ex1) [mailto:madhusudan_mathihalli@;hp.com] Sent: Tuesday, October 22, 2002 11:05 AM To: '[EMAIL PROTECTED]' Subject: [PATCH]Question - regarding modssl_PEM_read_bio_X509 I thought modssl_PEM_read_bio_X509 should cover the following cases for OpenSSL API : #if (SSL_LIBRARY_VERSION < 0x00904000) #define modssl_PEM_read_bio_X509 SOME WAY #else #define modssl_PEM_read_bio_X509 OTHER WAY #endif The following patch does something similar, and also changes one other place in ssl_util_ssl.c where PEM_read_bio_X509 was still being used. -Madhu Index: ssl_toolkit_compat.h === RCS file: /home/cvspublic/httpd-2.0/modules/ssl/ssl_toolkit_compat.h,v retrieving revision 1.25 diff -u -r1.25 ssl_toolkit_compat.h --- ssl_toolkit_compat.h21 Aug 2002 19:12:46 - 1.25 +++ ssl_toolkit_compat.h22 Oct 2002 18:01:44 - @@ -97,7 +97,11 @@ #define modssl_X509_verify_cert X509_verify_cert -#define modssl_PEM_read_bio_X509 PEM_read_bio_X509 +#if (SSL_LIBRARY_VERSION < 0x00904000) +#define modssl_PEM_read_bio_X509(b, x, cb, arg) PEM_read_bio_X509(b, x, cb) +#else +#define modssl_PEM_read_bio_X509(b, x, cb, arg) PEM_read_bio_X509(b, x, cb, arg) +#endif Index: ssl_util_ssl.c === RCS file: /home/cvspublic/httpd-2.0/modules/ssl/ssl_util_ssl.c,v retrieving revision 1.21 diff -u -r1.21 ssl_util_ssl.c --- ssl_util_ssl.c 15 Sep 2002 00:00:48 - 1.21 +++ ssl_util_ssl.c 22 Oct 2002 17:59:00 - @@ -519,11 +519,7 @@ } /* create new extra chain by loading the certs */ n = 0; -#if SSL_LIBRARY_VERSION < 0x00904000 -while ((x509 = PEM_read_bio_X509(bio, NULL, cb)) != NULL) { -#else -while ((x509 = PEM_read_bio_X509(bio, NULL, cb, NULL)) != NULL) { -#endif +while ((x509 = modssl_PEM_read_bio_X509(bio, NULL, cb, NULL)) != NULL) { if (!SSL_CTX_add_extra_chain_cert(ctx, x509)) { X509_free(x509); BIO_free(bio);
RE: [PATCH] Try to use OPENSSL_free instead of free
Okay here, it comes [complete patch] Thanks -Madhu Index: CHANGES === RCS file: /home/cvspublic/httpd-2.0/CHANGES,v retrieving revision 1.959 diff -u -r1.959 CHANGES --- CHANGES 24 Oct 2002 15:47:31 - 1.959 +++ CHANGES 25 Oct 2002 00:37:54 - @@ -1,5 +1,11 @@ Changes with Apache 2.0.44 + *) mod_ssl uses free() inappropriately in several places, to free + memory which has been previously allocated inside OpenSSL. + Such memory should be freed with OPENSSL_free(), not with free(). + [Nadav Har'El <[EMAIL PROTECTED]>, + Madhusudan Mathihalli <[EMAIL PROTECTED]>]. + Index: ssl_engine_kernel.c === RCS file: /home/cvspublic/httpd-2.0/modules/ssl/ssl_engine_kernel.c,v retrieving revision 1.78 diff -u -r1.78 ssl_engine_kernel.c --- ssl_engine_kernel.c 14 Oct 2002 04:15:58 - 1.78 +++ ssl_engine_kernel.c 23 Oct 2002 23:46:38 - @@ -968,7 +968,7 @@ X509_NAME *name = X509_get_subject_name(sslconn->client_cert); char *cp = X509_NAME_oneline(name, NULL, 0); sslconn->client_dn = apr_pstrdup(r->connection->pool, cp); -free(cp); +modssl_free(cp); } clientdn = (char *)sslconn->client_dn; @@ -1299,11 +1299,11 @@ iname ? iname : "-unknown-"); if (sname) { -free(sname); +modssl_free(sname); } if (iname) { -free(iname); +modssl_free(iname); } } @@ -1555,7 +1555,7 @@ "Certificate with serial %ld (0x%lX) " "revoked per CRL from issuer %s", serial, serial, cp); -free(cp); +modssl_free(cp); } X509_STORE_CTX_set_error(ctx, X509_V_ERR_CERT_REVOKED); @@ -1593,6 +1593,7 @@ ap_log_error(APLOG_MARK, APLOG_DEBUG, 0, s, SSLPROXY_CERT_CB_LOG_FMT "%s, sending %s", sc->vhost_id, msg, dn ? dn : "-uknown-"); +modssl_free(dn); } /* Index: ssl_engine_vars.c === RCS file: /home/cvspublic/httpd-2.0/modules/ssl/ssl_engine_vars.c,v retrieving revision 1.20 diff -u -r1.20 ssl_engine_vars.c --- ssl_engine_vars.c 28 May 2002 21:47:31 - 1.20 +++ ssl_engine_vars.c 23 Oct 2002 23:51:25 - @@ -334,7 +334,7 @@ xsname = X509_get_subject_name(xs); cp = X509_NAME_oneline(xsname, NULL, 0); result = apr_pstrdup(p, cp); -free(cp); +modssl_free(cp); resdup = FALSE; } else if (strlen(var) > 5 && strcEQn(var, "S_DN_", 5)) { @@ -346,7 +346,7 @@ xsname = X509_get_issuer_name(xs); cp = X509_NAME_oneline(xsname, NULL, 0); result = apr_pstrdup(p, cp); -free(cp); +modssl_free(cp); resdup = FALSE; } else if (strlen(var) > 5 && strcEQn(var, "I_DN_", 5)) { Index: ssl_toolkit_compat.h === RCS file: /home/cvspublic/httpd-2.0/modules/ssl/ssl_toolkit_compat.h,v retrieving revision 1.25 diff -u -r1.25 ssl_toolkit_compat.h --- ssl_toolkit_compat.h21 Aug 2002 19:12:46 - 1.25 +++ ssl_toolkit_compat.h23 Oct 2002 23:46:38 - @@ -105,6 +105,8 @@ #define modssl_set_cipher_list SSL_set_cipher_list +#define modssl_free OPENSSL_free + #define EVP_PKEY_reference_inc(pkey) \ CRYPTO_add(&((pkey)->references), +1, CRYPTO_LOCK_X509_PKEY) @@ -147,6 +149,8 @@ #define modssl_set_cipher_list(ssl, l) \ SSL_set_cipher_list(ssl, (char *)l) + +#define modssl_free free #ifndef PEM_F_DEF_CALLBACK #define PEM_F_DEF_CALLBACK PEM_F_DEF_CB
Re: [STATUS] (httpd-2.0) Wed Oct 23 23:45:19 EDT 2002
Excellent! I will perform some tests with that when I get a chance! You managed to get it working without breaking pipelining even? That's awesome! Not meaning to belittle Bojan's hard work, but for my purposes mod_logio values are not as good as %b would be if %b worked properly... what I ideally need is the byte sent count without the headers... using Bojan's module I can get approximate results but they will be a hair off because they include headers... My main purpose is to detect if and when several meg files have been downloaded all the way vs. if they were cut off in the middle, including if a given user uses some byte-ranging download manager that lets you pause and restart... We also use it for chargebacks to the various departments for bandwidth usage (in this case mod_logio would of course be more accurate than %b though)... We actually had to fudge some of our statistics (duplicated nearby days' data with similar overall throughputs) due to us not catching the problem with Apache 2.0 soon enough... Dave At 09:15 AM 10/25/2002 +1000, Bojan Smojver wrote: >On Fri, 2002-10-25 at 07:42, David Burry wrote: > >> I see.. ok, I'll keep waiting patiently... > >The patch for 2.0.43 is here: > >ftp://ftp.rexursive.com/pub/apache/counting_io_flush-2.0.43.patch > >You need to apply mod_logio patch for 2.0.43 first. > >Bojan
Re: Enabling RAND redirection on crypto accelerator using OpenSSL ENGINE
"Frederic DONNAT" <[EMAIL PROTECTED]> writes: > A few month ago i submit a patch for redirecting RAND on crypto accelerator for >mod-ssl and apache-1.3.x. > > A few weeks ago, i see a cvs commit about this on mod-ssl mailing list. > But i see that apache-2.0.x have not been updated. maintainers of mod_ssl for Apache 1.3 apparently have to time for Apache 2.0 mod_ssl > I post a message for this in mod-ssl dev mailing list, but maybe should i post it >somewhere else! yes, if you have a concern about Apache 2.0 mod_ssl please post here, but note that more skills are on mod-ssl dev mailing list > So, in fact the patch is for ssl_engine_init.c file in directory ./modules/ssl. > Just modify functions calls: > - ssl_engine_init () > - ssl_init_SSLlibrary () > > "ssl_engine_init()" (line 300) should be call earlier, before than >"ssl_init_SSLlibrary()" (line 270). > > In fact you have to initialyze OpenSSL ENGINE before initialzing the library, due to >fact that OpenSSL default function pointer must be set to ENGINE function pointer >before library initialisation otherwise you can not modify default settings. > > Geoff Thorpe comment: > "ssl_init_SSLLibrary() must be seeding the PRNG, and thus initialising the >set-on-first-use pointer in openssl to a default RAND_METHOD." > > Cliff Woolley comment: > Well, I can't do anything about 1.3's mod_ssl, but if somebody can verify > for me that the following fixes Apache 2.0's mod_ssl, I'll commit it. apparently nobody verified for Cliff that it fixed the problem with Apache 2.0 can you verify it? can you post a patch with the change? Thanks, -- Jeff Trawick | [EMAIL PROTECTED] Born in Roswell... married an alien...
Re: [PATCH] Try to use OPENSSL_free instead of free
"MATHIHALLI,MADHUSUDAN (HP-Cupertino,ex1)" <[EMAIL PROTECTED]> writes: > Can somebody _please_ take some time to review and tell me if the patch is > okay. > > Just a observation : > > I do understand that everybody is busy doing their own set of things, but > then - do you really solicit patches from non-committers ?. yes, of course we solicit patches from non-committers... if a patch isn't responded to quickly you'll have to nag... write up a CHANGES entry for the patch and I'll commit the darn thing -- Jeff Trawick | [EMAIL PROTECTED] Born in Roswell... married an alien...
RE: [PATCH] Try to use OPENSSL_free instead of free
Can somebody _please_ take some time to review and tell me if the patch is okay. Just a observation : I do understand that everybody is busy doing their own set of things, but then - do you really solicit patches from non-committers ?. I've been a silent observer since the last couple of months, and as-per my observation, everytime a non-committer sends a patch, it takes a couple of cycles before somebody even responds back to the patch (let alone commit it). When somebody posts a patch, I'm sure he/she definitely looks forward for some sort of input from you guys (the core developers/committers, best of the breed etc) - atleast something like "its all crap" / "you may want to do this also" / "this is all good" / something else. Am I wrong here ?.. Does anybody not want a response when they post a patch ?. Talking specifically of my patches, the majority of the commits to the mod_ssl source was done by Cliff / Justin / DougM. I do realize that they're busy with other things also - but then, I've not seen many others doing any development on the SSL front. So, whom do I approach to solicit any feedback ?. -Madhu > -Original Message- > From: MATHIHALLI,MADHUSUDAN (HP-Cupertino,ex1) > [mailto:madhusudan_mathihalli@;hp.com] > Sent: Wednesday, October 23, 2002 4:52 PM > To: '[EMAIL PROTECTED]' > Subject: [PATCH] Try to use OPENSSL_free instead of free > > > Based on Nadav Har'El's e-mail on the mod_ssl community > (http://marc.theaimsgroup.com/?l=apache-modssl&m=103540998016916&w=2), > here's a patch for 2.0's mod_ssl. > > > -Madhu > > Index: ssl_engine_kernel.c > === > RCS file: /home/cvspublic/httpd-2.0/modules/ssl/ssl_engine_kernel.c,v > retrieving revision 1.78 > diff -u -r1.78 ssl_engine_kernel.c > --- ssl_engine_kernel.c 14 Oct 2002 04:15:58 - 1.78 > +++ ssl_engine_kernel.c 23 Oct 2002 23:46:38 - > @@ -968,7 +968,7 @@ > X509_NAME *name = > X509_get_subject_name(sslconn->client_cert); > char *cp = X509_NAME_oneline(name, NULL, 0); > sslconn->client_dn = apr_pstrdup(r->connection->pool, cp); > -free(cp); > +modssl_free(cp); > } > > clientdn = (char *)sslconn->client_dn; > @@ -1299,11 +1299,11 @@ > iname ? iname : "-unknown-"); > > if (sname) { > -free(sname); > +modssl_free(sname); > } > > if (iname) { > -free(iname); > +modssl_free(iname); > } > } > > @@ -1555,7 +1555,7 @@ > "Certificate with serial > %ld (0x%lX) " > "revoked per CRL from issuer %s", > serial, serial, cp); > -free(cp); > +modssl_free(cp); > } > > X509_STORE_CTX_set_error(ctx, > X509_V_ERR_CERT_REVOKED); > @@ -1593,6 +1593,7 @@ > ap_log_error(APLOG_MARK, APLOG_DEBUG, 0, s, > SSLPROXY_CERT_CB_LOG_FMT "%s, sending %s", > sc->vhost_id, msg, dn ? dn : "-uknown-"); > +modssl_free(dn); > } > > /* > > Index: ssl_engine_vars.c > === > RCS file: /home/cvspublic/httpd-2.0/modules/ssl/ssl_engine_vars.c,v > retrieving revision 1.20 > diff -u -r1.20 ssl_engine_vars.c > --- ssl_engine_vars.c 28 May 2002 21:47:31 - 1.20 > +++ ssl_engine_vars.c 23 Oct 2002 23:51:25 - > @@ -334,7 +334,7 @@ > xsname = X509_get_subject_name(xs); > cp = X509_NAME_oneline(xsname, NULL, 0); > result = apr_pstrdup(p, cp); > -free(cp); > +modssl_free(cp); > resdup = FALSE; > } > else if (strlen(var) > 5 && strcEQn(var, "S_DN_", 5)) { > @@ -346,7 +346,7 @@ > xsname = X509_get_issuer_name(xs); > cp = X509_NAME_oneline(xsname, NULL, 0); > result = apr_pstrdup(p, cp); > -free(cp); > +modssl_free(cp); > resdup = FALSE; > } > else if (strlen(var) > 5 && strcEQn(var, "I_DN_", 5)) { > > Index: ssl_toolkit_compat.h > === > RCS file: /home/cvspublic/httpd-2.0/modules/ssl/ssl_toolkit_compat.h,v > retrieving revision 1.25 > diff -u -r1.25 ssl_toolkit_compat.h > --- ssl_toolkit_compat.h21 Aug 2002 19:12:46 - 1.25 > +++ ssl_toolkit_compat.h23 Oct 2002 23:46:38 - > @@ -105,6 +105,8 @@ > > #define modssl_set_cipher_list SSL_set_cipher_list > > +#define modssl_free OPENSSL_free > + > #define EVP_PKEY_reference_inc(pkey) \ > CRYPTO_add(&((pkey)->references), +1, CRYPTO_LOCK_X509_PKEY) > > @@ -147,6 +149,8 @@ > > #define modssl_set_cipher_list(ssl, l) \ > SSL_set_cipher_list(ssl, (char *)l) > + > +#define modssl_free free > > #ifndef PEM_F_DEF_CALLBACK > #define PEM_F_DEF_CALLBACK PEM_F_DEF_CB >
Re: [STATUS] (httpd-2.0) Wed Oct 23 23:45:19 EDT 2002
On Fri, 2002-10-25 at 07:42, David Burry wrote: > I see.. ok, I'll keep waiting patiently... The patch for 2.0.43 is here: ftp://ftp.rexursive.com/pub/apache/counting_io_flush-2.0.43.patch You need to apply mod_logio patch for 2.0.43 first. Bojan
Re: [PATCH]: Counting I/O with FLUSH and OFN (no MMN bump)
Very cool. Thanks. Bojan On Fri, 2002-10-25 at 08:28, Brian Pane wrote: > Bojan Smojver wrote: > > >This should really work now and it should not cause major dramas > >compatibility wise. Let me know what you think. > > > > > > Thanks, I'll take a look at this tonight unless anyone else > gets to it first.
Re: [PATCH]: Counting I/O with FLUSH and OFN (no MMN bump)
Bojan Smojver wrote: This should really work now and it should not cause major dramas compatibility wise. Let me know what you think. Thanks, I'll take a look at this tonight unless anyone else gets to it first. Brian
Re: [PATCH]: Counting I/O with FLUSH and OFN (no MMN bump)
Just two extra spaces so it's looks nicer... Bojan On Fri, 2002-10-25 at 08:11, Bojan Smojver wrote: > This should really work now and it should not cause major dramas > compatibility wise. Let me know what you think. diff -ruN httpd-2.0-vanilla/include/http_core.h httpd-2.0/include/http_core.h --- httpd-2.0-vanilla/include/http_core.h Tue Oct 15 03:42:45 2002 +++ httpd-2.0/include/http_core.h Fri Oct 25 08:04:28 2002 @@ -61,6 +61,7 @@ #include "apr.h" #include "apr_hash.h" +#include "apr_optional.h" #include "util_filter.h" #if APR_HAVE_STRUCT_RLIMIT @@ -626,6 +627,16 @@ /* -- */ +/* -- + * + * I/O logging with mod_logio + */ + +APR_DECLARE_OPTIONAL_FN(void, ap_logio_add_bytes_out, +(conn_rec *c, apr_off_t bytes)); + +/* -- */ + #ifdef __cplusplus } #endif diff -ruN httpd-2.0-vanilla/modules/loggers/mod_logio.c httpd-2.0/modules/loggers/mod_logio.c --- httpd-2.0-vanilla/modules/loggers/mod_logio.c Sat Sep 28 14:18:35 2002 +++ httpd-2.0/modules/loggers/mod_logio.c Fri Oct 25 08:12:59 2002 @@ -79,6 +79,7 @@ #include "ap_config.h" #include "mod_log_config.h" #include "httpd.h" +#include "http_core.h" #include "http_config.h" #include "http_protocol.h" @@ -96,6 +97,16 @@ } logio_config_t; /* + * Optional function for the core to add to bytes_out + */ + +static void ap_logio_add_bytes_out(conn_rec *c, apr_off_t bytes){ +logio_config_t *cf = ap_get_module_config(c->conn_config, &logio_module); + +cf->bytes_out += bytes; +} + +/* * Format items... */ @@ -133,24 +144,6 @@ * Logging of input and output filters... */ -static apr_status_t logio_out_filter(ap_filter_t *f, - apr_bucket_brigade *bb) { -apr_off_t length; -logio_config_t *cf = ap_get_module_config(f->c->conn_config, &logio_module); - -if (!cf) { /* Create config */ -cf = apr_pcalloc(f->c->pool, sizeof(*cf)); -ap_set_module_config(f->c->conn_config, &logio_module, cf); -} - -apr_brigade_length (bb, 0, &length); - -if (length > 0) -cf->bytes_out += length; - -return ap_pass_brigade(f->next, bb); -} - static apr_status_t logio_in_filter(ap_filter_t *f, apr_bucket_brigade *bb, ap_input_mode_t mode, @@ -162,11 +155,6 @@ status = ap_get_brigade(f->next, bb, mode, block, readbytes); -if (!cf) { /* Create config */ -cf = apr_pcalloc(f->c->pool, sizeof(*cf)); -ap_set_module_config(f->c->conn_config, &logio_module, cf); -} - apr_brigade_length (bb, 0, &length); if (length > 0) @@ -175,11 +163,30 @@ return status; } +static apr_status_t logio_out_filter(ap_filter_t *f, + apr_bucket_brigade *bb) { +apr_bucket *b = APR_BRIGADE_LAST(bb); + +/* End of data, make sure we flush */ +if (APR_BUCKET_IS_EOS(b)) { +APR_BRIGADE_INSERT_TAIL(bb, +apr_bucket_flush_create(f->c->bucket_alloc)); +APR_BUCKET_REMOVE(b); +apr_bucket_destroy(b); +} + +return ap_pass_brigade(f->next, bb); +} + /* * The hooks... */ static int logio_pre_conn(conn_rec *c) { +logio_config_t *cf = apr_pcalloc(c->pool, sizeof(*cf)); + +ap_set_module_config(c->conn_config, &logio_module, cf); + ap_add_input_filter(logio_filter_name, NULL, NULL, c); ap_add_output_filter(logio_filter_name, NULL, NULL, c); @@ -212,6 +219,8 @@ AP_FTYPE_NETWORK - 1); ap_register_output_filter(logio_filter_name, logio_out_filter, NULL, AP_FTYPE_NETWORK - 1); + +APR_REGISTER_OPTIONAL_FN(ap_logio_add_bytes_out); } module AP_MODULE_DECLARE_DATA logio_module = diff -ruN httpd-2.0-vanilla/server/core.c httpd-2.0/server/core.c --- httpd-2.0-vanilla/server/core.c Tue Oct 15 06:08:15 2002 +++ httpd-2.0/server/core.c Fri Oct 25 08:08:22 2002 @@ -2737,6 +2737,7 @@ apr_off_t file_offset, apr_size_t file_bytes_left, apr_size_t total_bytes_left, +apr_size_t *bytes_sent, apr_int32_t flags) { apr_status_t rv; @@ -2748,11 +2749,15 @@ == APR_SUCCESS) && timeout > 0); /* socket must be in timeout mode */ +/* Reset the bytes_sent field */ +*bytes_sent = 0; + do { apr_size_t tmplen = file_bytes_left; rv = apr_sendfile(c->client_socket, fd, hdtr, &file_offset, &tmplen, flags); +*bytes_sent += tmplen; total_bytes_left -= tmplen; if (!to
[PATCH]: Counting I/O with FLUSH and OFN (no MMN bump)
This should really work now and it should not cause major dramas compatibility wise. Let me know what you think. Bojan diff -ruN httpd-2.0-vanilla/include/http_core.h httpd-2.0/include/http_core.h --- httpd-2.0-vanilla/include/http_core.h Tue Oct 15 03:42:45 2002 +++ httpd-2.0/include/http_core.h Fri Oct 25 08:04:28 2002 @@ -61,6 +61,7 @@ #include "apr.h" #include "apr_hash.h" +#include "apr_optional.h" #include "util_filter.h" #if APR_HAVE_STRUCT_RLIMIT @@ -626,6 +627,16 @@ /* -- */ +/* -- + * + * I/O logging with mod_logio + */ + +APR_DECLARE_OPTIONAL_FN(void, ap_logio_add_bytes_out, +(conn_rec *c, apr_off_t bytes)); + +/* -- */ + #ifdef __cplusplus } #endif diff -ruN httpd-2.0-vanilla/modules/loggers/mod_logio.c httpd-2.0/modules/loggers/mod_logio.c --- httpd-2.0-vanilla/modules/loggers/mod_logio.c Sat Sep 28 14:18:35 2002 +++ httpd-2.0/modules/loggers/mod_logio.c Fri Oct 25 08:04:14 2002 @@ -79,6 +79,7 @@ #include "ap_config.h" #include "mod_log_config.h" #include "httpd.h" +#include "http_core.h" #include "http_config.h" #include "http_protocol.h" @@ -96,6 +97,16 @@ } logio_config_t; /* + * Optional function for the core to add to bytes_out + */ + +static void ap_logio_add_bytes_out(conn_rec *c, apr_off_t bytes){ +logio_config_t *cf = ap_get_module_config(c->conn_config, &logio_module); + +cf->bytes_out+=bytes; +} + +/* * Format items... */ @@ -133,24 +144,6 @@ * Logging of input and output filters... */ -static apr_status_t logio_out_filter(ap_filter_t *f, - apr_bucket_brigade *bb) { -apr_off_t length; -logio_config_t *cf = ap_get_module_config(f->c->conn_config, &logio_module); - -if (!cf) { /* Create config */ -cf = apr_pcalloc(f->c->pool, sizeof(*cf)); -ap_set_module_config(f->c->conn_config, &logio_module, cf); -} - -apr_brigade_length (bb, 0, &length); - -if (length > 0) -cf->bytes_out += length; - -return ap_pass_brigade(f->next, bb); -} - static apr_status_t logio_in_filter(ap_filter_t *f, apr_bucket_brigade *bb, ap_input_mode_t mode, @@ -162,11 +155,6 @@ status = ap_get_brigade(f->next, bb, mode, block, readbytes); -if (!cf) { /* Create config */ -cf = apr_pcalloc(f->c->pool, sizeof(*cf)); -ap_set_module_config(f->c->conn_config, &logio_module, cf); -} - apr_brigade_length (bb, 0, &length); if (length > 0) @@ -175,11 +163,30 @@ return status; } +static apr_status_t logio_out_filter(ap_filter_t *f, + apr_bucket_brigade *bb) { +apr_bucket *b = APR_BRIGADE_LAST(bb); + +/* End of data, make sure we flush */ +if (APR_BUCKET_IS_EOS(b)) { +APR_BRIGADE_INSERT_TAIL(bb, +apr_bucket_flush_create(f->c->bucket_alloc)); +APR_BUCKET_REMOVE(b); +apr_bucket_destroy(b); +} + +return ap_pass_brigade(f->next, bb); +} + /* * The hooks... */ static int logio_pre_conn(conn_rec *c) { +logio_config_t *cf = apr_pcalloc(c->pool, sizeof(*cf)); + +ap_set_module_config(c->conn_config, &logio_module, cf); + ap_add_input_filter(logio_filter_name, NULL, NULL, c); ap_add_output_filter(logio_filter_name, NULL, NULL, c); @@ -212,6 +219,8 @@ AP_FTYPE_NETWORK - 1); ap_register_output_filter(logio_filter_name, logio_out_filter, NULL, AP_FTYPE_NETWORK - 1); + +APR_REGISTER_OPTIONAL_FN(ap_logio_add_bytes_out); } module AP_MODULE_DECLARE_DATA logio_module = diff -ruN httpd-2.0-vanilla/server/core.c httpd-2.0/server/core.c --- httpd-2.0-vanilla/server/core.c Tue Oct 15 06:08:15 2002 +++ httpd-2.0/server/core.c Fri Oct 25 08:08:22 2002 @@ -2737,6 +2737,7 @@ apr_off_t file_offset, apr_size_t file_bytes_left, apr_size_t total_bytes_left, +apr_size_t *bytes_sent, apr_int32_t flags) { apr_status_t rv; @@ -2748,11 +2749,15 @@ == APR_SUCCESS) && timeout > 0); /* socket must be in timeout mode */ +/* Reset the bytes_sent field */ +*bytes_sent = 0; + do { apr_size_t tmplen = file_bytes_left; rv = apr_sendfile(c->client_socket, fd, hdtr, &file_offset, &tmplen, flags); +*bytes_sent += tmplen; total_bytes_left -= tmplen; if (!total_bytes_left || rv != APR_SUCCESS) { return rv;/* normal case & error exit */ @
Re: [STATUS] (httpd-2.0) Wed Oct 23 23:45:19 EDT 2002
- Original Message - From: "Bojan Smojver" <[EMAIL PROTECTED]> > On Fri, 2002-10-25 at 03:31, David Burry wrote: > > Is it possible to get some of the fixes to mod_logio committed? Wouldn't > > everyone agree that the current logging of the outgoing bytes is incorrect > > behavior? Currently it logs the full file size (plus headers) even if it > > gets cut off in the middle, instead of the actual number of bytes sent. > > I've seen several patches to fix this but very little comment on it... I've > > seen lots of comments that it can't be done without major rearchitecting, > > but Bojan seems to have done it without that by breaking pipelining, am I > > correct? > > Actually, the last patch I sent contains one snag I'm still working on. > It breaks the core's connection configuration structure, which gets > attached to c->conn_config. However, I think I can get around that by > using an optional function. As the matter of fact, I'm working on it > right now. I see.. ok, I'll keep waiting patiently... > > Since I depend on correct outgoing byte count logging to see how many people > > sucessfully download files, I can live with broken pipelining for now in > > 2.0, currently I've had to roll back to Apache 1.3 and put in 3 times as > > many machines (12 instead of 4) I'd really like to return those 8 > > borrowed machines someday and be able to upgrade to 2.0... but can't do that > > in the current broken log state. > > Glad to hear Apache 2.0 makes a huge performance difference. Not so glad > to hear you had to resort to going back to 1.3. The only thing I can > promise is a patch using optional function (this should guarantee > compatibility of core between 43 and 44 and no MMN bump) during the day > (Sydney time). It's up to the committers to review and, if they like, > commit. The memory savings is quite significant, and I'll admit that the 8 extra machines are smaller than the original 4 so it's not exactly 3 times better cpu-wise... the memory caching is where the largest savings comes with disk IO, we have a very high traffic site--usually 3 terabytes transferred per day, last few days have been more like 5 terabytes due to a new release. > PS. By the number of messages on the list I'm guessing committers must > be rather busy on their real jobs these days. Unfortunately there is no > way of speeding things up, given this is volunteer effort. Unless, of > course, you decide to bribe some of them ;-) Not exactly the same thing as bribing a commit, but this could get similar results: My manager's manager is actually not opposed to hiring a contractor to fix this... anyone want a temporary job? I don't know if this is the right place to say such things, tell me if there's a better place. When you've got millions of dollars worth of sales depending on an open source project, throwing a little at the open source project isn't such a big deal... I'd gladly do it myself with my company's blessing (on the clock, not volunteer) but I'm not a very experienced C programmer yet, more of a Perl hacker and applications architect so far. This little paragraph had better not get me too flooded with resumes or flames or I'm going to feel dumb, whatever you do don't spam this list with personal replies! ;o) Dave
Re: [STATUS] (httpd-2.0) Wed Oct 23 23:45:19 EDT 2002
Is it possible to get some of the fixes to mod_logio committed? Wouldn't everyone agree that the current logging of the outgoing bytes is incorrect behavior? Currently it logs the full file size (plus headers) even if it gets cut off in the middle, instead of the actual number of bytes sent. I've seen several patches to fix this but very little comment on it... I've seen lots of comments that it can't be done without major rearchitecting, but Bojan seems to have done it without that by breaking pipelining, am I correct? I also wish that %b would be fixed in a similar manner but I haven't seen any patches for that (or comments about it). Wouldn't everyone agree that it too should log actual bytes sent not just the full file size every time? Apache 2.0 should do everything that 1.3 did, so this logging issue really should be considered a bug, right? Since I depend on correct outgoing byte count logging to see how many people sucessfully download files, I can live with broken pipelining for now in 2.0, currently I've had to roll back to Apache 1.3 and put in 3 times as many machines (12 instead of 4) I'd really like to return those 8 borrowed machines someday and be able to upgrade to 2.0... but can't do that in the current broken log state. Dave
byterange filter/redirect bug?
check this out: [gregames@daedalus gregames]$ grep "194.65.14.76 .*binaries.* 416 " /logs/www/weblog | wc -l 69763 I asked root to block this IP for a while, because we are getting a couple of these every second. I suspect there's an httpd bug here as well as a looping client. We shouldn't be returning 416 "Invalid Range" for something that's redirected, but it sure looks like we are. I wonder what the byterange filter does when it sees a redirect response, and there are input Range: headers? Greg [gregames@daedalus gregames]$ grep "194.65.14.76 .*binaries.* 416 " /logs/www/weblog | tail www.apache.org 194.65.14.76 - - [24/Oct/2002:13:05:32 -0700] "GET /dist/httpd/binaries/win32/apache_1.3.27-win32-x86-no_src.exe HTTP/1.1" 416 387 "http://www.apache.org/dist/httpd/binaries/win32/"; "Mozilla/4.0 (compatible; MSIE 5.00; Windows 98)" www.apache.org 194.65.14.76 - - [24/Oct/2002:13:05:32 -0700] "GET /dist/httpd/binaries/win32/apache_1.3.27-win32-x86-no_src.exe HTTP/1.1" 416 387 "http://www.apache.org/dist/httpd/binaries/win32/"; "Mozilla/4.0 (compatible; MSIE 5.00; Windows 98)" www.apache.org 194.65.14.76 - - [24/Oct/2002:13:05:33 -0700] "GET /dist/httpd/binaries/win32/apache_2.0.43-win32-x86-no_ssl.exe HTTP/1.1" 416 387 "http://www.apache.org/dist/httpd/binaries/win32/"; "Mozilla/4.0 (compatible; MSIE 5.00; Windows 98)" www.apache.org 194.65.14.76 - - [24/Oct/2002:13:05:33 -0700] "GET /dist/httpd/binaries/win32/apache_2.0.43-win32-x86-no_ssl.exe HTTP/1.1" 416 387 "http://www.apache.org/dist/httpd/binaries/win32/"; "Mozilla/4.0 (compatible; MSIE 5.00; Windows 98)"
Re: [STATUS] (httpd-2.0) Wed Oct 23 23:45:19 EDT 2002
On Fri, 2002-10-25 at 03:31, David Burry wrote: > Is it possible to get some of the fixes to mod_logio committed? Wouldn't > everyone agree that the current logging of the outgoing bytes is incorrect > behavior? Currently it logs the full file size (plus headers) even if it > gets cut off in the middle, instead of the actual number of bytes sent. > I've seen several patches to fix this but very little comment on it... I've > seen lots of comments that it can't be done without major rearchitecting, > but Bojan seems to have done it without that by breaking pipelining, am I > correct? Actually, the last patch I sent contains one snag I'm still working on. It breaks the core's connection configuration structure, which gets attached to c->conn_config. However, I think I can get around that by using an optional function. As the matter of fact, I'm working on it right now. > I also wish that %b would be fixed in a similar manner but I haven't seen > any patches for that (or comments about it). Wouldn't everyone agree that > it too should log actual bytes sent not just the full file size every time? > Apache 2.0 should do everything that 1.3 did, so this logging issue really > should be considered a bug, right? I agree with you on this one as well. But at this point I'm unsure how to fix this one. > Since I depend on correct outgoing byte count logging to see how many people > sucessfully download files, I can live with broken pipelining for now in > 2.0, currently I've had to roll back to Apache 1.3 and put in 3 times as > many machines (12 instead of 4) I'd really like to return those 8 > borrowed machines someday and be able to upgrade to 2.0... but can't do that > in the current broken log state. Glad to hear Apache 2.0 makes a huge performance difference. Not so glad to hear you had to resort to going back to 1.3. The only thing I can promise is a patch using optional function (this should guarantee compatibility of core between 43 and 44 and no MMN bump) during the day (Sydney time). It's up to the committers to review and, if they like, commit. Bojan PS. By the number of messages on the list I'm guessing committers must be rather busy on their real jobs these days. Unfortunately there is no way of speeding things up, given this is volunteer effort. Unless, of course, you decide to bribe some of them ;-)
Re: [madhusudan_mathihalli@hp.com: [PATCH] Try to use OPENSSL_free instead of free]
On Thu, Oct 24, 2002, Nadav Har'El wrote about "Re: [[EMAIL PROTECTED]: [PATCH] Try to use OPENSSL_free instead of free]": > I'm not sure this patch addresses my concern. The idea was that OPENSSL_free() > should be used, not free(), on memory originally allocated by the OpenSSL I should have looked at the Apache 2 code more carefully before complaining about the patch... I didn't realize that Apache 2 was trying to be compatible with another SSL stack, not just OpenSSL, and that modssl_free was defined differently in the two cases. So your patch seems fine. Sorry. -- Nadav Har'El| Thursday, Oct 24 2002, 18 Heshvan 5763 [EMAIL PROTECTED] |- Phone: +972-53-245868, ICQ 13349191 |Two wrongs may not may a right, but three http://nadav.harel.org.il |rights make a left.
Re: [madhusudan_mathihalli@hp.com: [PATCH] Try to use OPENSSL_free instead of free]
> Based on Nadav Har'El's e-mail on the mod_ssl community > (http://marc.theaimsgroup.com/?l=apache-modssl&m=103540998016916&w=2), > here's a patch for 2.0's mod_ssl. >... > +modssl_free(cp); >... > +#define modssl_free free I'm not sure this patch addresses my concern. The idea was that OPENSSL_free() should be used, not free(), on memory originally allocated by the OpenSSL library. This is because you cannot (or should not) assume how OPENSSL_malloc() is implemented, and whether or not it's simply malloc(). So you probably meant to do #define modssl_free OPENSSL_free But I have another question about this patch - why did you choose the name "modssl_free"? These frees are of memory allocated by the OpenSSL library, not by mod_ssl, so the name modssl_free is misleading. What's wrong with using OPENSSL_free for memory allocated and returned by OpenSSL routines? -- Nadav Har'El| Thursday, Oct 24 2002, 18 Heshvan 5763 [EMAIL PROTECTED] |- Phone: +972-53-245868, ICQ 13349191 |He who has more is not happier than he http://nadav.harel.org.il |who wants less.