Re: C as config
On Sun, Jun 6, 2010 at 5:03 AM, Graham Leggett minf...@sharp.fm wrote: [...] We've also been playing with Varnish, one of the cooler things it does is have the ability to suck up an entire response into a RAM buffer and releasing the backend before playing out the response to the browser. I've looked, and we can do this pretty trivially in httpd as a simple connection filter. Very useful for expensive backends like php, instead of waiting many hundreds or thousands of milliseconds for the client to eventually close the connection before cleaning up the request pool and release php resources, you dump the response into a connection buffer, and destroy the request pool asap. That also would work well with the event MPM; in addition to freeing up the request pool, you'd be able to free up the request thread. As long as the documentation explained to users that they need to have enough memory to accommodate MaxClient * MaxOutputBufferedPerRequest (where the latter is a hypothetical name for a configurable limit on the amount of buffered output), such a feature would be a net win for lots of sites. -Brian
Re: canned deflate conf in manual -- time to drop the NS4/vary?
On Fri, Jun 4, 2010 at 2:18 AM, Mark Nottingham m...@mnot.net wrote: [...] It's not a bug in the implementations, it's a grey area in 2616 that HTTPbis has since worked to resolve; http://trac.tools.ietf.org/wg/httpbis/trac/ticket/147 By my reading of the attachments in that ticket, servers (including caches) would be required to treat the following as equivalent to each other: Accept-Encoding: gzip, deflate Accept-Encoding: gzip,deflate Accept-Encoding: deflate, gzip Accept-Encoding: deflate,gzip and the following as different from each other: Accept-Encoding: gzip,deflate Accept-Encoding: gzip If so, the RFC 2616 patch would basically codify current good practices in cache configuration (I recall that the Varnish docs, for example, recommend normalizing the Accept-Encoding before using it in a cache key), and as such it would be a step forward. In practice, given a cache that implements these equivalence rules and an origin server that sets a Vary header on Content-Encoding, I'd expect the cache to end up holding up to three copies of each object: 1. compressed, with a cache key of something like URI+gzip,deflate 2. compressed, with a cache key of URI+gzip 3. uncompressed, with a cache key of URI+ That's fewer copies of the object that the cache would end up with if it did a strict text match on the different permutations of gzip,deflate; but it's still a lot of copies. So I have to ask: why not reduce the number of copies to just one, by turning Content-Encoding into a hop-by-hop header and deprecating the use of Vary to indicate an Accept-Encoding-based variation? Cache implementors could then choose their own policies: - Store one copy of the object, compressed, to optimize for memory use - Store compressed and uncompressed copies of the object, to optimize for CPU use Brian
Re: canned deflate conf in manual -- time to drop the NS4/vary?
On Fri, Jun 4, 2010 at 6:10 AM, Plüm, Rüdiger, VF-Group ruediger.pl...@vodafone.com wrote: [...] Isn't that what Transfer-Encoding is designed for? Yes, and in fact if we were talking about a brand new protocol, I'd probably argue in favor of putting the compression specifier in the Transfer-Encoding. I think a change in the semantics of Content-Encoding in HTTP/1.1 might be a way to obtain similar benefits without breaking existing software. -Brian
Re: Fast by default
On Wed, Jun 2, 2010 at 3:29 AM, Jeffrey E Burgoyne burgo...@keenuh.com wrote: 4 - 1 compression ratio is fine unless you are serving lots of rich content, which generally will see no performance gain if not reduced performance. The rich content doesn't need to go through the deflate filter, though, so you needn't (and shouldn't) incur any CPU overhead for it. The approach I've used successfully on large sites in the past is to do on-the-fly compression when: 1. The response content-type is HTML, CSS, or JavaScript (plus plain text and XML, for sites serving up those in nontrivial amounts) 2. and the request header contains an Accept-Encoding indicating that the client can handle gzip 3. and the User-Agent is not one of the ones with known problems handling compressed content (IE =5 and Netscape =4, if I remember correctly) The only thing stopping me from suggesting this as a default is the concern that others have noted regarding downstream caches. (I think compression probably ought to be a Transfer-Encoding rather than a Content-Encoding, but that's a whole other debate entirely.) -Brian
Re: mod_deflate handling of empty initial brigade
On Wed, Jun 2, 2010 at 3:37 AM, Nick Kew n...@webthing.com wrote: On 2 Jun 2010, at 10:49, Joe Orton wrote: Maybe we should have a strict API here rather than a loose one: filters MUST NOT pass an empty brigade down the filter chain. That's not really an API, it's a rule, and pretty-much unenforcable. Wait a minute...why not just change the implementation of ap_pass_brigade so that it returns APR_SUCCESS immediately when passed an empty brigade? That would solve the problem globally. -Brian
mod_deflate handling of empty initial brigade
In a filter module I'm writing, it's possible for the first pass
through the output filter to end up calling:
ap_pass_brigade(f-next, an_empty_brigade)
If mod_deflate's output filter appears later in the output filter
chain, bad things happen:
1. On the first trip through the output filter chain, mod_deflate just
passes the empty brigade through to the next filter:
/* Do nothing if asked to filter nothing. */
if (APR_BRIGADE_EMPTY(bb)) {
return ap_pass_brigade(f-next, bb);
}
2. Control eventually reaches core_output_filter, which sends the
response headers.
3. On a subsequent trip through the output filter chain, my module calls:
ap_pass_brigade(f-next, a non_empty_brigade)
4. Upon seeing the non empty brigade, mod_deflate's output filter does
all the initialization that it would normally do at the start of a
response. If the response is compressible, deflate_out_filter adds
Content-Encoding: gzip to the output headers. The output headers
have been sent already, though, so the Content-Encoding never gets
sent.
5. The client receives a compressed response without a Content-Encoding header.
I can prevent this by not calling ap_pass_brigade in my module until I
have the first non-empty brigade for the rest of the output filter
chain to work with.
I'm curious, though: which module is doing the wrong thing:
- My module, by sending an empty brigade through the rest of the
output filter chain prior to sending the first non-empty brigade?
- Or mod_deflate, by adding fields to the response header after
calling ap_pass_brigade?
Thanks,
-Brian
Re: mod_deflate handling of empty initial brigade
On Tue, Jun 1, 2010 at 10:28 AM, Matthew Steele mdste...@google.com wrote: I went ahead and created a bug entry/patch to make the (trivial) change to mod_deflate to make it conform to the Guide to writing output filters: https://issues.apache.org/bugzilla/show_bug.cgi?id=49369 Brian, does this patch to mod_deflate fix your problem? Yes. (I'm still going to proceed with the same defensive logic in my own filter, though.) Thanks, -Brian
Re: Fast by default
On Tue, Jun 1, 2010 at 9:04 AM, William A. Rowe Jr. wr...@rowe-clan.net wrote: [...] Plus deflate may provide no benefit, and degrade performance, if the CPU utilization is a greater concern than bandwidth utilization. The CPU utilization is an interesting topic for me because I've been working on a related type of transform (minify of dynamic content) and I've gotten into the habit of measuring nanoseconds of processing time per byte of content. With that in mind, I just added in some instrumentation around the zlib calls in mod_deflate and found that the compression takes 30-60 ns/byte for buffers of a few KB. (This is with Apache 2.2.14 and zlib 1.2.3 on a 2.8GHz x64 processor.) To put that number in perspective, if you were to devote the equivalent of one CPU core worth of processing power on a web server host to compression, 30ns/byte means the host would be able to do real-time compression of ~266Mb/s of outbound traffic. Assume that your monthly bandwidth pricing is $10 per 95th percentile peak Mbps. Assume further that by turning on deflate you can reduce outbound bandwidth usage by 75% (i.e., you're getting 4:1 compression). Thus the CPU core that you've devoted completely to deflate processing will save you ~$2000 per month in bandwidth pricing. If the CPU core costs less than $24000 per year (amortized capital cost plus power, cooling, support, data center space, marginal cost of additional sysadmins needed to support each additional server, etc, etc), then you still come out ahead by turning on deflate. A few additional thoughts: 1. Speeding up the deflate implementation would be an unqualified win. Supposedly the recently-released zlib 1.2.5 is faster, but I don't have any data on it. 2. The best practice I've found for implementing compression in a web server is to do the compression in the server closest to the network edge. E.g., if you have a web server fronted by a proxy cache, do the compression dynamically in the proxy cache rather than the web server. That way the cache doesn't have to store multiple variants of each object. Similarly, if you're using a CDN for content that can benefit from gzip, ask your CDN if they can do conditional compression for non-problematic user-agents on-the-fly. -Brian
Re: httpd-trunk with MSIE (async read broken?) and AJP problems was Re: httpd-trunk sucking CPU on ajax
On Feb 27, 2006, at 11:42 AM, Ruediger Pluem wrote: Do we really have the async read code on trunk? Maybe I missed some commits, but I thought that we only have the async write stuff in the trunk. The trunk had the async write implementation plus a refactored version of the synchronous read code (ported from the async-read-dev branch but still using blocking reads). I've just reverted the latter to eliminate one possible source of problems. The async write completion is still in place in the trunk. Brian
Re: httpd-trunk with MSIE (async read broken?) and AJP problems was Re: httpd-trunk sucking CPU on ajax
Justin Erenkrantz wrote: On Sun, Feb 26, 2006 at 11:55:31PM -0800, Sander Temme wrote: Issues.apache.org, which is running trunk or something very close to it to debug the JIRA issues, has some child processes sucking extreme quantities of CPU. For instance: Also, we're getting reports that IE clients on Windows consistently get 408s (server timeout) immediately from httpd. Some users with Safari and Firefox can see this problem occassionally, but MSIE triggers it a good portion of the time. Coupled with the EOR brokenness (infinite loop?), this sounds like the async-read code is severely busted. Either we should fix it, or we need to revert it. =) I recommend reverting it, unless someone else has time to look at it in the near future. I haven't been able to look at the async code in a few weeks due to other schedule commitments, and I expect that will be the case for a few more weeks. I'll revert the code on the trunk this evening (though if anybody wants to do so earlier, no problem). Brian
Re: svn commit: r371484 - /httpd/httpd/branches/async-read-dev/modules/ssl/ssl_engine_io.c
On Jan 30, 2006, at 4:05 AM, Joe Orton wrote: On Mon, Jan 23, 2006 at 08:04:02AM -, Brian Pane wrote: Author: brianp Date: Mon Jan 23 00:04:01 2006 New Revision: 371484 URL: http://svn.apache.org/viewcvs?rev=371484view=rev Log: Return APR_EAGAIN instead of SSL_ERROR_WANT_READ from the mod_ssl filters; the httpd core and other modules' filters don't know what SSL_ERROR_* means. That does not look right. status is an apr_status_t value in this function; if it is ending up as an SSL_ERROR_* error code then something has gone wrong somewhere else. I agree; the fact that status==SSL_ERROR_WANT_READ is probably a bad thing. I think the only reason this hasn't been a problem in the past is that the mod_ssl input filter has always done blocking reads. With the nonblocking reads on the async-read-dev branch, it's now seeing EAGAIN and (somewhere) converting that into SSL_ERROR_WANT_READ. Thinking about it further, this seems to expose a design flaw in my async read approach: it's possible for the mod_ssl code to hit EAGAIN on either a read attempt or a write attempt during the connection establishment, but there's no way in the current filter API for a filter to return different values for EAGAIN and please do a poll on readability for me vs. EAGAIN and please do a poll on writability for me. The connection state logic in the event MPM assumes that an EAGAIN result from an input filter means poll for readability, but in the case of SSL that's not necessarily true. Anybody have ideas on how to solve this problem? Creating different status codes for APR_EAGAIN_READ and APR_EAGAIN_WRITE would allow mod_ssl to return sufficiently detailed information to the caller, but it would break a lot of existing code that only understands plain old APR_EAGAIN. Brian
async reads - request for comments Re: svn commit: r360461 - in /httpd/httpd/trunk: CHANGES include/ap_mmn.h include/httpd.h server/protocol.c
I'll probably have time to work on the changes to ap_core_input_filter
and/or
the async read code this weekend (starting on the async-read-dev branch).
If anyone has a strong argument for or against putting the buffering of
partial lines inside the ap_core_input_filter context, please speak up.
Thanks,
Brian
On Jan 7, 2006, at 11:57 PM, Brian Pane wrote:
On Jan 3, 2006, at 8:07 AM, Justin Erenkrantz wrote:
AFAICT, ap_read_async_request() on the branch can't handle a partial
line correctly.
Noting of course that ap_core_input_filter is 'cute' and masks EAGAIN.
So, you'll never see EAGAIN from this code path! As I said earlier,
the EAGAIN logic in httpd is completely suspect.
Furthermore, as I read it, ap_read_async_request is assuming that it
gets a complete line from getline_nonblocking - which almost certainly
won't be the case. -- justin
I'm currently working on changing ap_core_input_filter so that it
doesn't mask the EAGAIN in AP_MODE_GETLINE mode. There's
a catch, though:
if (mode == AP_MODE_GETLINE) {
/* we are reading a single LF line, e.g. the HTTP headers */
rv = apr_brigade_split_line(b, ctx-b, block, HUGE_STRING_LEN);
if apr_brigade_split_line returns APR_EAGAIN, it will have
consumed the partial line and put it in b. So if core_input_filter
returns rv at this point, the caller will receive the partial line and
EAGAIN. We'll need to do one of two things:
- Buffer the partial line in ap_core_input_filter, by removing the
buckets from b and putting them back at the start of ctx-b.
- Or buffer the partial line in getline_nonblocking (or
read_partial_request).
Anybody have a preference among these options?
Brian
Re: svn commit: r360461 - in /httpd/httpd/trunk: CHANGES include/ap_mmn.h include/httpd.h server/protocol.c
On Jan 3, 2006, at 8:07 AM, Justin Erenkrantz wrote:
AFAICT, ap_read_async_request() on the branch can't handle a partial
line correctly.
Noting of course that ap_core_input_filter is 'cute' and masks EAGAIN.
So, you'll never see EAGAIN from this code path! As I said earlier,
the EAGAIN logic in httpd is completely suspect.
Furthermore, as I read it, ap_read_async_request is assuming that it
gets a complete line from getline_nonblocking - which almost certainly
won't be the case. -- justin
I'm currently working on changing ap_core_input_filter so that it
doesn't mask the EAGAIN in AP_MODE_GETLINE mode. There's
a catch, though:
if (mode == AP_MODE_GETLINE) {
/* we are reading a single LF line, e.g. the HTTP headers */
rv = apr_brigade_split_line(b, ctx-b, block, HUGE_STRING_LEN);
if apr_brigade_split_line returns APR_EAGAIN, it will have
consumed the partial line and put it in b. So if core_input_filter
returns rv at this point, the caller will receive the partial line and
EAGAIN. We'll need to do one of two things:
- Buffer the partial line in ap_core_input_filter, by removing the
buckets from b and putting them back at the start of ctx-b.
- Or buffer the partial line in getline_nonblocking (or
read_partial_request).
Anybody have a preference among these options?
Brian
Re: svn commit: r360461 - in /httpd/httpd/trunk: CHANGES include/ap_mmn.h include/httpd.h server/protocol.c
On Jan 2, 2006, at 3:41 PM, Justin Erenkrantz wrote:
+static apr_status_t process_request_line(request_rec *r, char *line,
+ int skip_first)
+{
+if (!skip_first (r-the_request == NULL)) {
+/* This is the first line of the request */
+if ((line == NULL) || (*line == '\0')) {
+/* We skip empty lines because browsers have to tack
a CRLF
on to the end + * of POSTs to support old CERN
webservers.
+ */
+int max_blank_lines = r-server-limit_req_fields;
+if (max_blank_lines = 0) {
+max_blank_lines = DEFAULT_LIMIT_REQUEST_FIELDS;
+}
+r-num_blank_lines++;
+if (r-num_blank_lines max_blank_lines) {
+return APR_SUCCESS;
+}
+}
+set_the_request(r, line);
+}
This will cause a segfault if line is null and we are at or above
max_blank_lines. Perhaps you meant for an else clause here?
Yes, thanks for catching that.
+else {
+/* We've already read the first line of the request.
This is
either + * a header field or the blank line terminating
the header
+ */
+if ((line == NULL) || (*line == '\0')) {
+if (r-pending_header_line != NULL) {
+apr_status_t rv = set_mime_header(r,
r-pending_header_line); +if (rv != APR_SUCCESS) {
+return rv;
+}
+r-pending_header_line = NULL;
+}
+if (r-status == HTTP_REQUEST_TIME_OUT) {
+apr_table_compress(r-headers_in,
APR_OVERLAP_TABLES_MERGE);
+r-status = HTTP_OK;
Say what? ...looks at rest of file...
Is this because r-status is unset and we're saying that's it's
'okay' to proceed with the request. If so, this *really* needs a
comment to that effect. It makes no sense whatsoever otherwise.
(We should probably remove the hack to set it to
HTTP_REQUEST_TIME_OUT initially as part of these changes.)
Yeah, setting r-status to HTTP_OK is done here solely to make it work
with the existing logic about HTTP_REQUEST_TIME_OUT meaning still
reading the request header.
+1 for of removing the HTTP_REQUEST_TIME_OUT hack. I was trying
to be conservative by preserving that part of the original logic, but
now
that you mention it, we might as well replace it with something less
subtle
in 2.3. This will break any 3rd party modules that depend upon the
HTTP_REQUEST_TIME_OUT convention, but for a major release
like 2.4 or 3.0 that's a defensible choice.
+}
+}
+else {
+if ((*line == ' ') || (*line == '\t')) {
+/* This line is a continuation of the previous
one */
+if (r-pending_header_line == NULL) {
+r-pending_header_line = line;
+r-pending_header_size = 0;
+}
+else {
+apr_size_t pending_len =
strlen(r-pending_header_line);
+apr_size_t fold_len = strlen(line);
This seems really expensive. You shouldn't need to recount
pending_len each time.
Good point; I'll add something to keep track of the length.
+}
+break;
If I understand your direction, it'd bail out here if it ever got
EAGAIN?
Yes indeed. That's what the version on the async-read-dev branch does.
+request_rec *ap_read_request(conn_rec *conn)
+{
+request_rec *r;
+apr_status_t rv;
+
+r = init_request(conn);
+
+rv = read_partial_request(r);
+/* TODO poll if EAGAIN */
+if (r-status != HTTP_OK) {
+ap_send_error_response(r, 0);
+ap_update_child_status(conn-sbh, SERVER_BUSY_LOG, r);
+ap_run_log_transaction(r);
+return NULL;
+}
Obviously, this can't be the case if you want to do real polling.
This would be the wrong place to poll. You have to exit out of
ap_read_request and go back up to an 'event thread' that waits
until there's data to read on any incoming sockets. Then, you'd
have to call ap_read_request again to 'restart' the parser whenever
there is more data to read.
In my opinion, this code isn't even close to being async.
I'm relieved to hear that it's not async, since you're looking at the
blocking
version. :-) See ap_read_async_request() (still on the async-read-
dev branch).
So, I wonder why it was even merged to trunk right now. You'd
have to deal with partial lines and the state the header parser is
in when it gets the next chunk of data - which this code doesn't
even try to do. The current code is just going to bail when it
doesn't have a 'complete' line.
My hunch is that you plan to build up pending_header_line and delay
parsing until you have the line terminators; but that's not going
to work really well with non-blocking sockets as you have to store
the data you just read
Re: svn commit: r360461 - in /httpd/httpd/trunk: CHANGES include/ap_mmn.h include/httpd.h server/protocol.c
On Jan 2, 2006, at 11:52 AM, Roy T. Fielding wrote: It would be feasible to build up the pending request in a structure other than the request_rec, so that ap_read_async_request() can operate on, say, an ap_partial_request_t instead of a request_rec. My preference so far, though, has been to leave the responsibility for knowing how to parse request headers encapsulated within the request_rec and its associated methods. Maybe you should just keep those changes on the async branch for now. The rest of the server cannot be allowed to degrade just because you want to introduce a new MPM. After the async branch is proven to be significantly faster than prefork, then we can evaluate whether or not the additional complications are worth it. Significantly faster than prefork has never been a litmus test for assessing new features, and I'm -1 for adding it now. A reasonable technical metric for validating the async changes would significantly more scalable than the 2.2 Event MPM or memory footprint competitive with IIS/Zeus/phttpd/one's-competitive-benchmark-of-choice. The bit about degrading the rest of the server is a wonderful sound bite, but we need to engineer the httpd based on data, not FUD. Brian
Re: svn commit: r360461 - in /httpd/httpd/trunk: CHANGES include/ap_mmn.h include/httpd.h server/protocol.c
On Jan 2, 2006, at 2:14 PM, Roy T. Fielding wrote: On Jan 2, 2006, at 1:37 PM, Brian Pane wrote: Significantly faster than prefork has never been a litmus test for assessing new features, and I'm -1 for adding it now. A reasonable technical metric for validating the async changes would significantly more scalable than the 2.2 Event MPM or memory footprint competitive with IIS/Zeus/phttpd/one's-competitive-benchmark-of- choice. Those aren't features. They are properties of the resulting system assuming all goes well. The bit about degrading the rest of the server is a wonderful sound bite, but we need to engineer the httpd based on data, not FUD. I said leave it on the async branch until you have data. You moved it to trunk before you've even implemented the async part, which I think is wrong because the way you implemented it damages the performance of prefork and needlessly creates an incompatible MMN You have yet to present any data backing up the assertion that it damages the performance of prefork. (Repeating the claim doesn't count as a proof.) Having looked at the compiler's inlining of the code that's been factored into separate functions, I'm rather skeptical of the claim. The incompatible MMN point is puzzling, to say the least. As the 4th MMN change in the past quarter, this was by no means an unusual event. And on an unreleased development codeline, the impact to production sites and 3rd party module developers is near zero. Brian
Re: Event MPM: Spinning on cleanups?
On Dec 31, 2005, at 3:01 AM, Paul Querna wrote: I haven't been able to reproduce this since reporting it, and I am still running the Event MPM, but I did recently upgrade from FreeBSD 5.4 - 6.0... If its a rare race condition, that might be enough to hide it. The bucket allocator cleanup problem might have been a contributing factor. Here's the patch I checked in to the trunk late yesterday: http://svn.apache.org/viewcvs?rev=360257view=rev The async write logic may have increased the probability of buckets remaining in brigades in the connection pool after the last request has been processed on that connection (e.g., if a client drops the connection before the response is completely sent). That would be enough to trigger the freeing buckets allocated from an allocator that has already been destroyed situation. Brian
Re: svn commit: r360461 - in /httpd/httpd/trunk: CHANGES include/ap_mmn.h include/httpd.h server/protocol.c
On Dec 31, 2005, at 6:50 PM, Roy T. Fielding wrote: The nonblocking yield should happen inside ap_rgetline (or its replacement), not in the calling routine. The thread has nothing to do until that call is finished or it times out. On the contrary, the thread has some very important work to do before that call finishes or times out: it has other connections to process. If the thread waits until the ap_rgetline completes, a server configuration sized for multiple threads per connection will be vulnerable to a trivially implementable DoS. In any case, this code should be independent of the MPM and no MPM is going to do something useful with a partial HTTP request. I say -1 to adding the input buffer variables to the request_rec. Those variables can be local to the input loop. Are you proposing that the buffers literally become local variables? That generally won't work; the input loop isn't necessarily contained within a single function call, and the reading of a single request's input can cross threads. It would be feasible to build up the pending request in a structure other than the request_rec, so that ap_read_async_request() can operate on, say, an ap_partial_request_t instead of a request_rec. My preference so far, though, has been to leave the responsibility for knowing how to parse request headers encapsulated within the request_rec and its associated methods. Brian
Re: Event MPM: Spinning on cleanups?
I haven't been able to find the bug yet. As a next step, I'll try using valgrind on a build with pool debugging enabled. Brian On Dec 4, 2005, at 11:14 PM, Paul Querna wrote: I finally got around to upgrading to trunk w/ the Event MPM on one of my machines. Within a couple hours of starting it, I had a process spinning, and consuming 100% CPU. Backtrace from the spinning thread: (gdb) thread 6 [Switching to thread 6 (Thread 0x2045 (LWP 100189))]#0 apr_allocator_free (allocator=0x2054ab80, node=0x205a2000) at memory/unix/ apr_pools.c:334 334 if (max_free_index != APR_ALLOCATOR_MAX_FREE_UNLIMITED (gdb) where #0 apr_allocator_free (allocator=0x2054ab80, node=0x205a2000) at memory/unix/apr_pools.c:334 #1 0x280eb952 in apr_bucket_free (mem=0x0) at buckets/apr_buckets_alloc.c:182 #2 0x280e9915 in heap_bucket_destroy (data=0x205a4090) at buckets/apr_buckets_heap.c:36 #3 0x280e9a54 in apr_brigade_cleanup (data=0x2059e6b0) at buckets/apr_brigade.c:44 #4 0x280e9a8b in brigade_cleanup (data=0x2059e6b0) at buckets/apr_brigade.c:34 #5 0x282021bd in run_cleanups (cref=0x2059e028) at memory/unix/apr_pools.c:2044 #6 0x28202f39 in apr_pool_clear (pool=0x2059e018) at memory/unix/apr_pools.c:689 #7 0x08081063 in worker_thread (thd=0x81d1660, dummy=0x0) at event.c:682 #8 0x2820b3e4 in dummy_worker (opaque=0x0) at threadproc/unix/ thread.c:138 #9 0x2823720b in pthread_create () from /usr/lib/libpthread.so.2 #10 0x282fa1ef in _ctx_start () from /lib/libc.so.6 OS: FreeBSD 6.0-STABLE APR: Trunk APR-Util: Trunk HTTPD: Trunk Best guess is that we corrupted a bucket brigade by double freeing it, or something of that kind. This is definitely a new behavior since the async-write code was merged into trunk. It is odd that we could of double-free'ed something on the connection pool. Maybe it isn't a double-free issue at all... I'm too tired to debug much of it tonight. Maybe later this week I will dig deeper. -Paul
apr_allocator Re: Event MPM: Spinning on cleanups?
On Dec 30, 2005, at 6:48 PM, Roy T. Fielding wrote:
On Dec 30, 2005, at 5:51 PM, Brian Pane wrote:
I haven't been able to find the bug yet. As a next step, I'll try
using
valgrind on a build with pool debugging enabled.
On entry to allocator_free, if
(node == node-next node-index current_free_index)
is true, then the do { } while ((node = next) != NULL);
will go into an infinite loop. This is because
if (max_free_index != APR_ALLOCATOR_MAX_FREE_UNLIMITED
index current_free_index) {
node-next = freelist;
freelist = node;
}
does not update current_free_index. I don't know if that is the
problem, but it may be the symptom.
After reading and instrumenting the apr_allocator code, I just found
that odd things happen when
max_free_index == APR_ALLOCATOR_MAX_FREE_UNLIMITED.
The doesn't appear to be the cause of the problem in the Event
MPM, but it's worth noting.
APR_ALLOCATOR_MAX_FREE_UNLIMITED is zero, and it's used
as the initial value for allocator-current_free_index. The problem is
that when apr_allocator_free() stores a block in one of its free
lists, it
subtracts that block's index (a value from 1 to 20) from
allocator-current_free_index.
Note that allocator-current_free_index is unsigned. Thus, when
it starts out at 0, subsequent calls to apr_allocator_free() turn it
into
a number slightly smaller than 2^32. On the next call to
apr_allocator_alloc() that recycles a block from one of the free
lists, this bit of defensive code ends up returning the value of
allocator-current_free_index to zero:
allocator-current_free_index += node-index;
if (allocator-current_free_index allocator-
max_free_index)
allocator-current_free_index = allocator-
max_free_index;
We probably should have a check in apr_allocator_free() to
make sure that we're not causing an unsigned int overflow in
allocator-current_free_index, e.g.,
+if (index current_tree_index) {
+ current_tree_index = 0;
+}
+else {
current_free_index -= index;
+}
In cases where max_free_index is not set to
APR_ALLOCATOR_MAX_FREE_UNLIMITED, all the
current_free_index logic appears to work in a sane
manner: the values of current_free_index and
max_free_index are basically base-2 logarithms
of the amount of space currently in the
allocator-free[] lists and of the max amount
of space that may be stored in these lists.
These variables really could use clearer names
and some descriptive comments, however. They're
not really indices: even without the unsigned
int overflow problem, they can have values much
larger than MAX_INDEX. Maybe it's just me, but
the naming made it very tough to figure out the
underlying powers-of-two trick.
I'll commit a patch when I have time, hopefully
in the next couple of days.
Brian
Re: Event MPM: Spinning on cleanups?
On Dec 4, 2005, at 11:14 PM, Paul Querna wrote: I finally got around to upgrading to trunk w/ the Event MPM on one of my machines. Within a couple hours of starting it, I had a process spinning, and consuming 100% CPU. Backtrace from the spinning thread: (gdb) thread 6 [Switching to thread 6 (Thread 0x2045 (LWP 100189))]#0 apr_allocator_free (allocator=0x2054ab80, node=0x205a2000) at memory/unix/ apr_pools.c:334 334 if (max_free_index != APR_ALLOCATOR_MAX_FREE_UNLIMITED (gdb) where #0 apr_allocator_free (allocator=0x2054ab80, node=0x205a2000) at memory/unix/apr_pools.c:334 Paul, if you're able to reproduce this condition, can you post a dump of the free lists in the allocator? If something allocated from the apr_allocator_t is being freed twice, it may be detectable as a loop in one of the allocator-free[] lists. Thanks, Brian
Re: Event MPM: Spinning on cleanups?
William A. Rowe, Jr. wrote: Paul Querna wrote: Best guess is that we corrupted a bucket brigade by double freeing it, or something of that kind. This is definitely a new behavior since the async-write code was merged into trunk. It is odd that we could of double-free'ed something on the connection pool. Maybe it isn't a double-free issue at all... My other thought comes down to the thread locking. Perhaps we are looking at an issue of two threads touching the per-thread non-locking allocator at once? The reason I suspect is that it -seems- your process ended up with a linked block list with a cyclic loop. That might well be the problem, since the thread that frees the brigade isn't necessarily the same thread that allocated it. I'll take a look at the code later today. Brian
Re: Event MPM: Spinning on cleanups?
On Dec 5, 2005, at 10:30 AM, Brian Pane wrote: William A. Rowe, Jr. wrote: Paul Querna wrote: Best guess is that we corrupted a bucket brigade by double freeing it, or something of that kind. This is definitely a new behavior since the async-write code was merged into trunk. It is odd that we could of double-free'ed something on the connection pool. Maybe it isn't a double-free issue at all... My other thought comes down to the thread locking. Perhaps we are looking at an issue of two threads touching the per-thread non-locking allocator at once? The reason I suspect is that it -seems- your process ended up with a linked block list with a cyclic loop. That might well be the problem, since the thread that frees the brigade isn't necessarily the same thread that allocated it. I'll take a look at the code later today. No, on closer inspection, that's not it. The event MPM creates a new allocator whenever it creates a new ptrans pool (line 1000). If the ptrans pool ever gets destroyed, we might have problems, because it's created as a child of pconf, and I don't think pconf is thread-safe. It doesn't appear that the ptrans pool is ever destroyed within the code, though, so pconf thread-safety problems wouldn't explain the error Paul saw. Brian
Request for comments on ap_read_request refactoring Re: svn commit: r349348 - in /httpd/httpd/branches/async-read-dev: include/httpd.h server/protocol.c
[EMAIL PROTECTED] wrote: Author: brianp Date: Sun Nov 27 18:56:47 2005 New Revision: 349348 URL: http://svn.apache.org/viewcvs?rev=349348view=rev Log: More refactoring of ap_read_request() in preparation for nonblocking read support. WARNING: The code for handlng requests that exceed the field-length or number-of-fields limits isn't complete in this commit. Modified: httpd/httpd/branches/async-read-dev/include/httpd.h httpd/httpd/branches/async-read-dev/server/protocol.c I probably won't have time to do my next iteration of work on this until the weekend. In the meantime, if anybody has an opportunity to review the code, I'd appreciate any and all feedback on it. (Note that the new code is only available on the async-read-dev branch.) My remaining plans for the async reads are: - Clean up the error-case handling - Add an ap_read_async_request() that invokes the new request-reading code in a nonblocking manner - Modify ap_process_async_request() to use ap_read_async_request(), meaning that async MPMs will need to support read completion - Add read completion support to the event MPM - Port the changes back into the 2.3 trunk Brian
Re: Cleanup in 'server/mpm/fdqueue.c'
On Nov 26, 2005, at 2:08 AM, Joe Orton wrote: On Fri, Nov 25, 2005 at 08:18:51PM -0800, Brian Pane wrote: I think the usual reason that the apr_palloc+memset pattern is used instead of apr_pcalloc in httpd and APR code is to allow the compiler generate inline code to fill the structure with zeros. apr_pcalloc has been a macro for quite a long time now :) #define apr_pcalloc(p, size) memset(apr_palloc(p, size), 0, size) And according to the commit log, I probably should have remembered that. :-) http://svn.apache.org/viewcvs?rev=63357view=rev Brian
Re: Cleanup in 'server/mpm/fdqueue.c'
On Nov 21, 2005, at 1:34 PM, Christophe Jaillet wrote: Around line 61 in function ap_queue_info_create, in 'server/mpm/ fdqueue.c' the following sequence can cleaned-up : === qi = apr_palloc(pool, sizeof(*qi)); memset(qi, 0, sizeof(*qi)); === by using a 'apr_pcalloc' : === qi = apr_pcalloc(pool, sizeof(*qi)); === I think the usual reason that the apr_palloc+memset pattern is used instead of apr_pcalloc in httpd and APR code is to allow the compiler generate inline code to fill the structure with zeros. gcc, for example, will apply this optimization if the struct is small. If the struct is large, gcc will generate a call to memset. With an apr_palloc call, the compiler doesn't have enough information to apply the optimization. But in the case of this particular code, which only gets called once, we really don't need the optimization. Brian
Deleting the async-dev branch
I've created a new branch off of the httpd trunk: https://svn.apache.org/repos/asf/httpd/httpd/branches/async-read-dev/ The old async-dev branch is rather out of date at this point, and all its nonblocking write code has since been committed to the trunk. Rather than doing a monstrous merge to get async-dev synchronized with the trunk, my plan is to delete async-dev in a week. If anyone is actively working with the async-dev branch and would be adversely affected by an svn delete, please let me know. Thanks, Brian
Request for comments: connection states for nonblocking request reads
Now that we have nonblocking write completion support in the Event MPM, I've begun thinking about how to implement nonblocking, event-driven reads of incoming requests. Here's a diagram of the connection state diagram with some minor changes to support nonblocking reads: http://www.brianp.net/b/proposed-connection-state-model-for- nonblocking-reads Comments welcome... Thanks, Brian
Re: Deleting the async-dev branch
On Nov 20, 2005, at 1:34 PM, Phillip Susi wrote: Actually you have to use a peg revision, not -r. If you try -r to look at the deleted branch, svn tries to find the named directory in the HEAD revision, then look up it's state as it existed in rev x. This will fail since it does not exist in the head revision, so instead you have to do something like: svn list http://repos/svn/[EMAIL PROTECTED] svn list -r x http://repos/svn/deleted fails because deleted does not exist anymore. By the way, what exactly is the async-dev branch and what are the future plans for it? I actually have spent this entire weekend doing research on the current state of support of async IO in linux and modified the dd program to use overlapped async IO when you pass it the iflag=direct option. It turned out that glibc's posix aio library simply performs sync IO in a background thread, so I ended up hacking together a basic aio implementation that directly uses the kernel's aio support. I'm still testing it but it looks like it yielded substantial cpu savings and throughput increases by keeping the drive directly reading into the userspace buffers. DMA transfers sure beat memcpy()s. The async-dev branch was a place for testing changes to the httpd core and Event MPM to support nonblocking network writes. These changes have since been merged into the httpd trunk. Once the entire response for an HTTP request has been generated, the worker thread hands off the remainder of the response transmission to a central pollset. The new async-read-dev branch is something I just created as a sandbox for developing nonblocking network read support. The async write completion stuff that's now in the trunk does nonblocking network writes, and it uses sendfile to do zero-copy transfer from the filesystem cache to the network on platforms that support it, but it doesn't try to do file I/O asynchronously. Thus it's possible for the current code to block on a disk read while trying to do a nonblocking socket write. (The Event MPM compensates for this by allowing multiple threads to do sendfile calls on different sockets concurrently; if one thread happens to block on a physical disk read, it doesn't delay other connections.) Having a means of doing nonblocking filesystem I/O, via either aio or something like mincore checks, might be a nice addition. Brian
Re: Can't compile trunk on Win
On Nov 12, 2005, at 2:21 PM, Vadim Chekan wrote: On 11/12/05, Jeff Trawick [EMAIL PROTECTED] wrote: On 11/12/05, Vadim Chekan [EMAIL PROTECTED] wrote: Hi all, I've tryed to compile httpd/trunk and got an error: core_filters.obj : error LNK2019: unresolved external symbol _apr_wait_for_io_or_timeout referenced in function _send_brigade_blocking I have apr from apr/branches/1.2.x and looks like it doesn't contain apr_wait_for_io_or_timeout for windows, unix only. wasn't supposed to be used Try this?? Index: server/core_filters.c === [...] Thanks, now it compiles! +1 on Unix, too. Brian
Re: Proper declaration for apr_wait_for_io_or_timeout() (was: Re: NWGNUmakefile update for APR1.2?)
On Nov 11, 2005, at 6:21 PM, Jeff Trawick wrote: On 11/11/05, Brad Nicholes [EMAIL PROTECTED] wrote: On 11/11/2005 at 1:40:32 pm, in message [EMAIL PROTECTED], Jeff Trawick [EMAIL PROTECTED] wrote: On 11/11/05, Brad Nicholes [EMAIL PROTECTED] wrote: Reposting to the APR list with a new subject line. Does this need to be taken care of in APR 1.2.x before httpd 2.2 ships? apr_wait_for_io_or_timeout() is private APR function (not API). Is the caller outside of APR? Yes, HTTPD. trunk/server/core_filters.c Simply shocking ;) That code needs to be fixed to call a real API :( I just double-checked the declaration, and apr_wait_for_io_or_timeout() currently is implemented as part of the public API. I don't have any strong objection to making it private, though (e.g., by wrapping the declaration in an #ifdef APR_PRIVATE or moving apr_support.h to apr/include/arch/ unix). Brian
Re: svn commit: r327945 - in /httpd/httpd/trunk: CHANGES modules/http/http_core.c modules/http/http_request.c server/mpm/experimental/event/event.c
On Oct 30, 2005, at 4:17 AM, Joe Orton wrote: On Sun, Oct 30, 2005 at 09:56:45AM +, Joe Orton wrote: On Sat, Oct 29, 2005 at 02:23:51PM -0700, Brian Pane wrote: With fix 329484 committed, I'm now getting a clean test run of the perl_framework tests with --enable-pool-debug=all Thanks, all test runs passed last night here too. Hmm, lots of segfaults from some manual testing though: ap_process_http_connection is using r after r-pool is destroyed: ... ap_process_request(r); if (ap_extended_status) ap_increment_counts(c-sbh, r); ... ap_update_child_status(c-sbh, SERVER_BUSY_KEEPALIVE, r); The cleanest fix I can think of is to run the ap_increment_counts and ap_update_child_status from the EOR bucket destruction function, before r-pool is freed. I'll implement the change later today. Brian
Re: functions registered for hooks - do they run in same process/thread?
On Oct 30, 2005, at 2:09 PM, Peter Djalaliev wrote: Hello, I understand that a module registers its own functions for different hooks in order to introduce its own functionality at different points of a request handling. I also understand that the module can specify if it wants its function to be executed first, in the middle or last, with relation to other modules' functions registered for the same hook. When these functions are actually called, do they execute in the same process and thread (if threads are used) as the process/thread handling the connection? For example, if the worker MPM module is used, a single thread would handle a single connection. Would the mod_ssl code, for example, execute within the thread for that connection? Strictly speaking, there's no guarantee that a request will be processed by one and only one thread. It's possible for a threaded MPM to hand off a request from one thread to another. For example, the version of the Event MPM in 2.3 can run the logger hook for a request in a different thread than the handler hook. (I think all of the MPMs in 2.0 happen to do all of the processing of a request in the same thread, but that's more an implementation detail than a design guarantee.) What currently _is_ guaranteed, even in 2.3, is that at most one thread at a time will process any given connection. And, as a corollary, at most one thread at a time will process any given request. (IMHO, this guarantee is one that we should preserve in 2.4.) Brian
Re: functions registered for hooks - do they run in same process/thread?
On Oct 30, 2005, at 2:46 PM, Nick Kew wrote: On Sunday 30 October 2005 22:40, Brian Pane wrote: Strictly speaking, there's no guarantee that a request will be processed by one and only one thread. It's possible for a threaded MPM to hand off a request from one thread to another. For example, the version of the Event MPM in 2.3 can run the logger hook for a request in a different thread than the handler hook. Really? I thought the async activity was the I/O, not the request processing. There are two aspects of request processing that can happen asynchronously: - anything the modules add to the connection-level output filters - the logger invocation In the future, there probably should be a third: - processing in the handler phase that involves mainly waiting for network I/O (e.g., mod_proxy) That's going to break lots of things. Like, any resource that's used over more than one hook but that can't cross threads. When was that major breakage discussed here? What resources are you thinking off that can't cross threads? I can't find any examples of thread-local storage in the current set of core modules. Brian
Re: svn commit: r327945 - in /httpd/httpd/trunk: CHANGES modules/http/http_core.c modules/http/http_request.c server/mpm/experimental/event/event.c
On Oct 28, 2005, at 1:10 AM, Joe Orton wrote: There are still crashes in free() for an --enable-pool-debug build (prefork); backtraces like the below seem to show that r-pool is getting destroyed way too early by the looks of it. I just figured out why this was happening. The root cause ended up being embarrassingly simple: r-pool was destroyed at the expected spot, following the invocation of the logger, but the very next operation in the core output filter referenced the brigade that had contained the EOR bucket--a brigade that had been allocated out of r-pool. With fix 329484 committed, I'm now getting a clean test run of the perl_framework tests with --enable-pool-debug=all Thanks for catching the error! Brian
Re: svn commit: r327945 - in /httpd/httpd/trunk: CHANGES modules/http/http_core.c modules/http/http_request.c server/mpm/experimental/event/event.c
On Oct 25, 2005, at 2:33 AM, Joe Orton wrote: On Mon, Oct 24, 2005 at 03:33:17AM -, Brian Pane wrote: Author: brianp Date: Sun Oct 23 20:33:14 2005 New Revision: 327945 URL: http://svn.apache.org/viewcvs?rev=327945view=rev Log: Async write completion for Event MPM (backported from async-dev branch to 2.3 trunk) httpd-test runs against the trunk are failing all over the place today, I guess caused by one of these changes... I just committed a fix. The problem was that the new ap_core_output_filter was leaving the socket timeout set to zero. In situations where the output filter runs _before_ the input filter, like the communication to the origin server in mod_proxy, the zero timeout was causing reads in the input filter to fail. Brian
Re: svn commit: r327945 - in /httpd/httpd/trunk: CHANGES modules/http/http_core.c modules/http/http_request.c server/mpm/experimental/event/event.c
I think I know what the problem with proxy is. When reading the
response
header from the origin server, it falls out of ap_rgetline_core() at
this point:
/* Something horribly wrong happened. Someone didn't block! */
if (APR_BRIGADE_EMPTY(bb)) {
return APR_EGENERAL;
}
It looks like the new core output filter is messing up the blocking mode
when it's called during the sending of the request to the origin server.
I'll continue tracking this down tomorrow.
Brian
On Oct 25, 2005, at 7:06 PM, Brian Pane wrote:
On Oct 25, 2005, at 2:33 AM, Joe Orton wrote:
httpd-test runs against the trunk are failing all over the place
today,
I guess caused by one of these changes...
prefork is failing like:
Failed TestStat Wstat Total Fail Failed List of
Failed
-
--
t/modules/proxy.t13 11 84.62% 1-9 12-13
t/modules/proxy_balancer.t11 100.00% 1
t/modules/rewrite.t 225 22.73% 17-18 20-22
t/protocol/echo.t 255 65280 88 100.00% 1-8
t/security/CAN-2005-2700.t21 50.00% 1
... plus almost all of t/ssl/* fail
Sorry about that. I didn't have proxy and ssl compiled in when I
regression-tested with httpd-test. I'll get started on a fix later
this
evening.
Brian
Re: svn commit: r327945 - in /httpd/httpd/trunk: CHANGES modules/http/http_core.c modules/http/http_request.c server/mpm/experimental/event/event.c
On Oct 25, 2005, at 2:33 AM, Joe Orton wrote: httpd-test runs against the trunk are failing all over the place today, I guess caused by one of these changes... prefork is failing like: Failed TestStat Wstat Total Fail Failed List of Failed -- - t/modules/proxy.t13 11 84.62% 1-9 12-13 t/modules/proxy_balancer.t11 100.00% 1 t/modules/rewrite.t 225 22.73% 17-18 20-22 t/protocol/echo.t 255 65280 88 100.00% 1-8 t/security/CAN-2005-2700.t21 50.00% 1 ... plus almost all of t/ssl/* fail Sorry about that. I didn't have proxy and ssl compiled in when I regression-tested with httpd-test. I'll get started on a fix later this evening. Brian
Re: async write completion prototype
On Oct 10, 2005, at 5:15 PM, Greg Ames wrote: - event-ize lingering close. it eats up roughly the same number of worker threads as synchronous writes for SPECweb99. Is this because the lingering close is waiting a while for the client to close the inbound side of the connection? Or is the lingering close finding that the connection is closed as soon as it does the wait for I/O or timeout--meaning that the reason the server spends a lot of time in lingering close is simply that the code for lingering close (poll+read+close plus various setsockopt calls) takes a while? Brian
Re: async write completion prototype
On Oct 18, 2005, at 7:11 AM, Greg Ames wrote: Brian Pane wrote: I think one contributor to the event results is an issue that Paul Querna pointed out on #httpd-dev the other day: apr_pollset_remove runs in O(n) time with n descriptors in the pollset. thanks, I see it. yeah we are going to have to do something about that. I just committed a change to the epoll version that eliminates the O(n) loop--and the mutex operations and a bit of data structure copying. The version of the Event MPM on the async-dev branch takes advantage of this new feature. I'm seeing a ~5% increase in throughput in a simple test setup (http_load on a client machine driving ~200 concurrent connections over 1Gb/s ethernet to Apache running on Linux 2.6). If anybody with a more industrial-strength load testing setup can try the async-dev version of the Event MPM with a few thousand concurrent connections, I'm eager to hear whether this new epoll code yields a useful speedup. Thanks, Brian
Re: async write completion prototype
I think one contributor to the event results is an issue that Paul Querna pointed out on #httpd-dev the other day: apr_pollset_remove runs in O(n) time with n descriptors in the pollset. Brian On Oct 13, 2005, at 11:36 AM, Brian Akins wrote: Greg Ames wrote: this is interesting to me because Brian Atkins recently reported that the event MPM was much slower. http://mail- archives.apache.org/mod_mbox/httpd-dev/200509.mbox/% [EMAIL PROTECTED] No t in my last name :) Basically, I was referring to the overall hits a box could serve per second. with 512 concurrent connections and about an 8k file, 2.1 with worker served about 22k request/second. event served about 14k. It's been a while since I did the test, and I'm too busy for the next few days to re-run them. -- Brian Akins Lead Systems Engineer CNN Internet Technologies
Re: [pre-release] 2.0.55 *candidate* available for testing
Tested successfully on Linux 2.6.13/x86_64 (Fedora Core 4) with both worker and prefork MPMs. I encountered lots of errors in perl-framework's t/TEST with prefork on Darwin 8.2.0/PPC (OS X 10.4.2). I don't yet know whether these are due to httpd-2.0.55 problems or just problems with my Perl installation. Brian On Oct 9, 2005, at 9:42 PM, William A. Rowe, Jr. wrote: The httpd-2.0.55 candidate, including win32 source .zip and installers*, is now available for testing at http://httpd.apache.org/dev/dist/ Please review this candidate, and when responding, indicate the precise operating system that you have tested. Thank you for your assistance! Bill * note that win32 binary installers were uploaded only at this hour, and it will take up to another two hours for the public server to resync. Thanks for your patience.
async write completion prototype
With the batch of commits I did this weekend, the Event MPM in the async-dev Subversion branch now does write completion in a nonblocking manner. Once an entire response has been generated and passed to the output filter chain, the MPM's poller/listener thread watches the connection for writability events. When the connection becomes writable, the poller thread sends it to one of the worker threads, which writes some more output. At this point, the event-handling code is ready for testing and review by other developers. The main changes on the async-dev branch (compared to the 2.3-dev trunk) are: 1. ap_core_output_filter: rewrite to do nonblocking writes whenever possible. 2. core, http module, and mod_logio: removed the generation of flush buckets where possible. 3. request cleanup and logging: the logger phase and subsequent destruction of the request's pool are now triggered by the destruction of an End-Of-Request bucket in the core output filter. 4. event MPM: asynchronous handling of CONN_STATE_WRITE_COMPLETION. There are several more things that need to be fixed in order to make the asynchronous write completion useful in a production release of httpd-2.x: - The main pollset in the Event MPM currently is sized to hold up to one socket descriptor per worker thread. With asynchronous keepalives and write completion, the pollset should accommodate many descriptors per thread. - The logic for starting more child processes, which Event inherited from Worker, is based on assumptions about the number of concurrent connections being equal to the number of threads. These assumptions aren't valid for a multiple-connections-per-thread MPM. - Similarly, there may be some changes needed in the flow control logic that the listener thread uses to decide whether it can do an accept. - The scoreboard probably needs a redesign. - It may be valuable to have a separate thread pool to run handlers that do arbitrarily lengthy processing, such as mod_perl and mod_php. Brian
Re: [pre-release] 2.0.55 *candidate* available for testing
On Oct 10, 2005, at 2:22 AM, Graham Leggett wrote: Brian Pane said: I encountered lots of errors in perl-framework's t/TEST with prefork on Darwin 8.2.0/PPC (OS X 10.4.2). I don't yet know whether these are due to httpd-2.0.55 problems or just problems with my Perl installation. I ran the build/binbuild.sh script, and httpd built clean on my Darwin 8.2.0. I didn't see any tests being run though, is the test suite fired off from the binbuild script? The test suite has to be downloaded and run separately. Basically, - do a make install of the httpd - checkout asf/repos/httpd/test/trunk from SVN - cd to the perl-framework subdirectory - perl Makefile.pl -apxs /path/to/the/httpd/installation/bin/apxs - ./t/TEST Brian
Re: [pre-release] 2.0.55 *candidate* available for testing
On Oct 10, 2005, at 7:32 AM, William A. Rowe, Jr. wrote: Brian Pane wrote: I encountered lots of errors in perl-framework's t/TEST with prefork on Darwin 8.2.0/PPC (OS X 10.4.2). I don't yet know whether these are due to httpd-2.0.55 problems or just problems with my Perl installation. Hmmm... review this bug (not fixed in 0.9.7, afaict); http://issues.apache.org/bugzilla/show_bug.cgi?id=34332 It looks like most of the errors I saw were due to environmental problems. I just cleaned up my Perl installation on OS X, and now there's just one test case that's failing now: test 10 of t/apache/limits.t. I _think_ the fix for 34332 is in apr-0.9.7; the CHANGES file includes: *) Fix issue with poll() followed by net I/O yielding EAGAIN on Mac OS 10.4 (Darwin 8). [Wilfredo Sanchez] I probably won't be able to look at that failed test case in limits.t in detail for a few days. If anybody else with OS X has time to test 2.0.55 before then, I'd be grateful. Thanks, Brian
Re: async write completion prototype
On Oct 10, 2005, at 12:01 AM, Paul Querna wrote: Brian Pane wrote: With the batch of commits I did this weekend, the Event MPM in the async-dev Subversion branch now does write completion in a nonblocking manner. Once an entire response has been generated and passed to the output filter chain, the MPM's poller/listener thread watches the connection for writability events. When the connection becomes writable, the poller thread sends it to one of the worker threads, which writes some more output. If the content has already been generated, why add the overhead of the context switch/sending to another thread? Can't the same event thread do a non-blocking write? Once it finishes writing, then yes, we do require a context-switch to another thread to do logging/cleanup. I am mostly thinking about downloading a 1 gig file with the current pattern against a slow client. A non-blocking write might only do ~64k at a time, and causing 1 gig/64k context switches, which seems less than optimal. If I had to choose, I'd rather do the context switches than devote a thread (and the associated stack space) to the connection until the writes are finished--especially if the server is delivering a thousand 1GB files to slow clients concurrently. However, it's probably possible to have _both_ a high ratio of connections to threads (for scalability) and a low ratio of context switches to megabytes delivered (for efficiency). The Event MPM currently has to do a lot of context switching because it detects events in one thread and processes them in another. If we add async write completion to the Leader/Followers MPM (or incorporate a leader/follower thread model into Event), it should reduce the context switches considerably. ... - The main pollset in the Event MPM currently is sized to hold up to one socket descriptor per worker thread. With asynchronous keepalives and write completion, the pollset should accommodate many descriptors per thread. The pollset is auto-resizable. That number is just the maximum number of events that will ever be returned by a single call to _poll(). This number if perfect for the number of threads, since we can never dispatch to more than the number of threads we have... Ah, thanks. I missed that key point. It makes more sense now. Brian
Re: svn commit: r307339 - in /httpd/httpd/branches/async-dev
On Oct 9, 2005, at 3:25 AM, Nick Kew wrote: On Sunday 09 October 2005 02:37, [EMAIL PROTECTED] wrote: URL: http://svn.apache.org/viewcvs?rev=307339view=rev Log: Redesign of request cleanup: - A new End-Of-Request bucket is pushed through the output filter chain after the last bucket of the response. - This bucket gets destroyed by ap_core_output_filter() after the buckets in front of it have been sent. - The destroy callback of the EOR bucket invokes the access logger and frees the request's pool. How do you see this looking from a filter programmer's POV? I can see a danger of some nasty bugs arising from confusion between this and EOS buckets: - Existing filters test for EOS and will ignore EOR. That's presumably fine with you. - Sooner or later, someone will write a filter that propagates EOR and not EOS. Bang! Is there no way you could use EOS directly? I can think of two easy solutions: - Wrap the EOR declaration in #ifdef CORE_PRIVATE, just like we do for EOC. - Or modify the EOS bucket API to support an optional on_destroy callback. I slightly prefer the first option, but I'm open to suggestions. Thanks, Brian
Re: svn commit: r307339 - in /httpd/httpd/branches/async-dev
On Oct 9, 2005, at 11:41 AM, Nick Kew wrote:
On Sunday 09 October 2005 18:49, Brian Pane wrote:
On Oct 9, 2005, at 3:25 AM, Nick Kew wrote:
On Sunday 09 October 2005 02:37, [EMAIL PROTECTED] wrote:
URL: http://svn.apache.org/viewcvs?rev=307339view=rev
Log:
Redesign of request cleanup:
- A new End-Of-Request bucket is pushed through the output filter
chain after the last bucket of the response.
- This bucket gets destroyed by ap_core_output_filter() after the
buckets in front of it have been sent.
- The destroy callback of the EOR bucket invokes the access
logger
and frees the request's pool.
How do you see this looking from a filter programmer's POV? I
can see
a danger of some nasty bugs arising from confusion between this and
EOS buckets:
- Existing filters test for EOS and will ignore EOR. That's
presumably
fine with you.
- Sooner or later, someone will write a filter that propagates EOR
and not EOS. Bang!
Is there no way you could use EOS directly?
I can think of two easy solutions:
- Wrap the EOR declaration in #ifdef CORE_PRIVATE, just
like we do for EOC.
So what does my filter see? A bucket of type unknown?
How does this fit with filters of the common form:
loop over buckets {
if (is data) {
read the data;
write something to the next filter;
} else if (is EOS) {
write EOS to next filter;
cleanup and return;
}
/* ignore anything else */
}
If there was an EOR bucket on input, it's lost in the above, yesno?
How do you deal with that? If it doesn't break or break on this
logic,
I'm fine with it.
If this example is a request-layer filter, no problem: the EOR
buckets are inserted at the connection layer.
If a connection-layer filter uses this logic (throwing away
metadata buckets it doesn't understand), it's already broken,
since EOC buckets pass through the connection layer
output filters.
Unless I'm missing something, EOR buckets wouldn't
be at greater risk than EOC buckets are today. We'd
just need to document the implied API rule: don't destroy
EOR buckets if you're not ap_core_output_filter. This
is a subtle enough API change that I wouldn't propose
backporting it to 2.0, but we could introduce it into 2.2
or 2.4.
- Or modify the EOS bucket API to support an optional on_destroy
callback.
Smells of side-effects. EOS buckets get created and destroyed by
intermediate filters - as in my above skeleton.
I agree. There's a subset of problems that could be solved
by adding reference counts to the EOS buckets, but refcounts
wouldn't solve the general problem of request-layer filters that
destroy EOS filters and create new ones.
Brian
Re: svn commit: r306495 - in /httpd/httpd/trunk: include/ap_mmn.h include/http_core.h modules/http/http_core.c server/core.c server/core_filters.c server/protocol.c
On Oct 5, 2005, at 7:04 PM, William A. Rowe, Jr. wrote:
Before I consider backporting for 2.1-dev, I'd very much appreciate
if some of the event mpm ap_process_http_async_connection developers
would confirm that the mechanisms behave the way I expect them to.
IIUC, the event pollset will handle the core timeout, and the
keep_alive_timeout on their own, and the common code to read-between
the request line and headers, headers and body follow the normal
course
of processing as in the non-async model.
Yes, that's how things currently work in the event MPM in 2.2 and
the 2.3 trunk. Strictly speaking, the common code handles the core
timeout, and the MPM's pollset only has to handle the keepalive
timeout. When we add async write completion, the MPM can apply
the core write timeout itself.
On the async-dev branch, my ap_core_output_filter() rewrite
currently does a hack in order to achieve nonblocking writes:
if (nonblocking) {
get the socket's timeout (as set by the NET_TIME filter)
clear the timeout
do the write
restore the timeout value
}
else {
do the blocking write, relying on the timeout set in NET_TIME
}
I'll have to modify the code to figure out for itself what the
timeout should be (rather than looking up a value that the
NET_TIME filter has set in the socket). But that's no problem;
the resulting code will be cleaner than the current hack.
Brian
Re: Urp(!) - indeterminant connection SO_NONBLOCK state?
On Sep 30, 2005, at 11:19 AM, William A. Rowe, Jr. wrote: Linux, Win32 and Netware begin their new socket's life in a blocking state, while Solaris and AIX begin their life in a non-blocking state. [...] Do we want to; * Fix mod_echo with this gross little hack? (and leave it to each protocol module author?) * Modify mod_ssl so that APR_BLOCK_READ actually toggles the socket to blocking mode (or keeps reinvoking the core until it gets the bytes that SSL demands)? * Modify the core so that APR_BLOCK_READ actually toggles the socket blocking mode? * Modify the MPM such that the newly accept()ed socket is toggled into blocking mode? * Modify APR such that a newly created socket's state is of some known value? (Right now, it pretends that it knows if the platform has predetermined that the accept()ed socket's state is inherited from the listening socket - and Linux's state proves this assumption was invalid.) Thoughts/comments? I think the 3rd option--modify the core so that APR_BLOCK_READ actually toggles the socket blocking mode--might be the best way to go. This would provide predictable semantics for modules that know they want to do blocking reads, while still allowing the MPM to put newly accepted sockets in nonblocking mode by default if the MPM has some good reason to do so. Brian
RFC: nonblocking rewrite of ap_core_output_filter
I just committed a new version of ap_core_output_filter() to the async-dev branch. The semantics of the filter have changed in a fundamental way: the original version does blocking writes and usually tries to write out all of the available data before writing (except in cases where it buffers data to avoid small writes), whereas this new version does nonblocking writes in most cases. The goal of the rewrite was to set the stage for a clean implementation of asynchronous write completion in the Event and/or Leader/Followers MPMs. The nonblocking behavior, however, appears to be useful in all MPMs. For example, nonblocking writes can enable mod_include to parse the next bunch of output while awaiting an ack from the client. To avoid infinite memory usage, the new filter does blocking writes if it has = 64KB of data buffered up. There are some significant differences from the earlier nonblocking output filter that I posted a few weeks ago as part of the Event MPM async write completion patch: - If a nonblocking write attempt results in EAGAIN, the new filter returns APR_SUCCESS instead of APR_EAGAIN. The old patch broke too much existing code that wasn't prepared for EAGAIN. The new filter can return APR_SUCCESS without having actually written the entire brigade, but that's also true of the original ap_core_output_filter(), with its various setaside cases. - The new filter doesn't try to ignore flush buffers like the earlier patch did. Instead, when it encounters a flush bucket, it does a blocking write of everything up to that point. The earlier patch tried to detect certain patterns of buckets involving flush, EOS, and EOC that could be interpreted as hand this data off to the write completion thread instead of actually writing it out immediately. But that logic was too brittle, as it depended on knowledge of the bucket patterns that the httpd core just happens to produce. In the new design, the core output filter interprets a flush bucket as write this data out before returning. To implement async write completion on top of this, we'll likely have to remove some of the points in the core that generate flush buffers-- but that's a project for another day. There are a few things missing in the new version: - It doesn't concatenate sequences of really small buckets together the way the original does. If you send it a brigade containing 16 single- byte buckets, it will do a writev of 16 bytes. My inclination is to leave this broken in order to keep the code simple, unless someone has a real-world use case that produces such output. - I haven't yet put in mod_logio support. This shouldn't be difficult to add, but I want to do some experiments with a new design: sending an End-Of- Request bucket that calls the request logger when all the buckets in front of it have been written to the network. If this works, the EOR bucket's destroy function might end up being the cleanest place to call the logio hooks. - It doesn't yet do nonblocking reads on socket buckets. Can anyone recommend a good test case that make use of socket buckets? Thanks, Brian
Re: Proposed connection state diagram
On Sep 16, 2005, at 6:19 PM, Greg Ames wrote: I see a tiny difference from the current event MPM implementation. a new connection initially goes into CONN_STATE_READ_REQUEST_LINE. it could easily be done as drawn too...the decision was almost a toss up. but since several OSes have some sort of accept filtering in the kernel which checks for data present along with the connection, and since I rarely see connections in 'R' state, I decided to skip the overhead of an initial poll() for readability. Thanks for catching that. I've updated diagram to reflect the current implementation, http://www.brianp.net/work/opensource/apache/async.html Brian
semi-asynchronous leader/followers MPM Re: How long until 2.2
On Sep 9, 2005, at 4:25 PM, Brian Pane wrote: This weekend, when I have time to do some more httpd work, I'll do a more robust hack of the leader MPM within the async-dev branch so that it's available for other people to test. Just committed... This version is a bit slower than the worker MPM. It does a poll before read on each new connection, which can be optimized away on platforms where accept(2) doesn't return until the new connection has data available for reading. With this latest batch of changes, the leader MPM is converging toward the design of the event MPM. My goal is to be able to compare the performance of a central listener thread vs. a leader/followers thread pool for async MPMs as we add async write completion. Brian
Re: How long until 2.2
Colm MacCarthaigh wrote: On Fri, Sep 09, 2005 at 04:18:34PM -0400, Bill Stoddard wrote: I guess for some definition of much I am not suprised. I would expect the event MPM to consume more CPU than worker under 'low' workloads (low number of concurrent clients) because it's making more system calls to do the same amount of work. Event should start to show benefits as the number of concurrent connections into the server goes up. That's what I would expect anyway; no idea if the event MPM is sufficiently refined to show any benefit at any workload. I definitely see speed improvements using the event MPM, probably due to the lack of an Accept Mutex. I have to get around to hacking the worker MPM to see if that's what it is. As part of my recent async RD, I hacked a copy of the leader/followers MPM to work with a single, central pollset (like event). It wasn't anywhere near production quality--I never got around to implementing tmeouts, for example--but it was faster than worker in non-real-world testing (client and server talking to each other over Gigabit ethernet.) This weekend, when I have time to do some more httpd work, I'll do a more robust hack of the leader MPM within the async-dev branch so that it's available for other people to test. Brian
Re: Proposed connection state diagram
Torsten Foertsch wrote: On Tuesday 06 September 2005 06:10, Brian Pane wrote: http://www.brianp.net/work/opensource/apache/async.html Shouldn't there be a transition from HANDLER to CHECK_REQUEST_LINE_READABLE in case the full response is sent and keep-alive is on? Yes, good catch: the diagram currently is inconsistent. I need to either add the transition from HANDLER to CHECK_REQUEST_LINE_READABLE or remove the transition from HANDLER to LINGER (except for error cases). That leads into a question: other than error cases, should we always transition from HANDLER to WRITE_COMPLETION, even if the handler happens to have written the entire respose? That makes the state diagram simpler, but it might mean some extra overhead in the implementation. Any suggestions? I'll be traveling for the next couple of days, without access to my Mac and Omnigraffle, so I'll next be able to update the diagram for a while. Please keep the feedback coming; I'll incorporate all the edits into a revised diagram this weekend. Thanks, Brian
Proposed connection state diagram
On the subject of asynchronous write completion, I've drawn a connection state model that combines the current state transitions of the event MPM with new states for write completion and the handler phase. Comments welcome... Am I missing any state transitions (particularly for error cases)? Should there be an access logger state? Are there other good use cases to consider for a nonblocking handler feature, besides mod_proxy? http://www.brianp.net/work/opensource/apache/async.html Thanks, Brian
Re: Proposed connection state diagram
On Sep 5, 2005, at 9:17 AM, Paul Querna wrote: Right now, I am trying to understand the implications of a Handler state. It says we only shift states from Handler - Write Completion once we have the 'Full Request Generated'. So does this mean the Handler will write to the client at all? Yes, in this model the Handler can do writes. There are a few use cases where this seems useful: - Regular sending of files: If the handler is allowed to write, we don't have to wait for a context switch and poll before beginning to send the response. - Handlers that produce a lot of non-file output: If someone writes a filter that produces a megabyte of output, consisting of a bunch of heap or pool buckets, it's better to not try to buffer up the whole response before beginning the write. In my insane vision that I have never written down anywhere before, I thought we would just save brigades onto a buffer inside the C-O- F. As soon as *any* data was buffered it would go into the write completion state. I guess it actually makes sense to have a Handler state. When we are in the Handler state, we know more data is coming. Once we are in write completion, as soon as we are done sending data, the request is finished. Yeah, my rationale was that it was important to differentiate handler might still send some more data to this output filter from handler is done sending data to this output filter, so that we could ensure that only one thread at a time is working with the connection. I'd originally though about having C-O-F hand off brigades to an event-handling threadpool, so that the worker threads running handlers wouldn't do any actual writes to the client. But I gave up on that idea because making the bucket and pool allocators thread-safe would involve a lot of overhead. It's worth noting that my event_output_filter prototype ended up being a bit messy because I made the filter responsible for determining when the transition from Handler to Write Completion state had occurred. To do this, it had to apply knowledge about the patterns of buckets that the core happens to generate. This, along with the logic for deciding when to actually do a blocking write upon seeing a flush bucket, could probably be a lot cleaner if we modified the core to change the state from Handler to Write Completion, rather than making the filter try to figure it out. Comments welcome... Am I missing any state transitions (particularly for error cases)? Should there be an access logger state? Are there other good use cases to consider for a nonblocking handler feature, besides mod_proxy? Logging Sucks. On Fast Machines, it wouldn't be a huge hit, and most likely could run inside the event thread. However, we also might not be logging to a file... there are plenty of other modules that log to remote machines too. I guess my view is that we should let any blocking operation be handled by a Worker Thread, and Logging still seems like one that cannot always be distilled into non-blocking operations. That makes sense. And the example of nonblocking writes to a logger on a remote machine makes me think that we really need to support nonblocking socket I/O for sockets in general--not just sockets that happen to be connections from clients. Supporting the general case of a generic socket poller is a little tricky, because of the need to support different timeouts on different sockets in the pollset. E.g., if the pollset contains two descriptors with timeouts 3 and 10 seconds in the future, respectively, and you add a new descriptor with a timeout 5 seconds in the future, this new timeout needs to be added in the middle of the timeout queue--and if there's a poll event on the descriptor that subsequently cancels the timeout, the timeout needs to be removed from whereever it is in the queue. A timing wheel might be an effective solution: http://citeseer.ist.psu.edu/varghese96hashed.html Brian
Re: Proposed connection state diagram
On Sep 5, 2005, at 5:21 AM, Ian Holsman wrote: are you missing the case where the client disconnects mid-request (CONN_STATE_READ_REQUEST_LINE - Connection_deleted) and when the client disconnects mid-response? CONN_STATE_HANDLER/CONN_STATE_WRITE_COMPLETION - Connection_deleted) case where the client connects/disconnects (TCP-port check) Thanks, those all definitely need to be in the state model. I'll update the diagram later today. Brian
[PATCH] event MPM nonblocking write completion
I've prototyped a modification to the event MPM that uses a (mostly)
nonblocking
output filter in place of ap_core_output_filter().
This patch adds a new connection state, CONN_STATE_WRITE_COMPLETION.
The new network-level filter, event_output_filter(), does nonblocking
writes
(except when it encounters a flush bucket that isn't immediately
before or
after an EOS or EOC). If an attempted write yields EAGAIN, the filter
does a setaside of the unwritten data.
If there still is unwritten data buffered in event_output_filter()
upon completion
of a request, the filter puts the connection in write-completion state.
When the connection is in write-completion state, the main
process_socket()
loop in event.c runs a loop to output the remaining data. The basic
logic is:
while (state == CONN_STATE_WRITE_COMPLETION) {
rv = event_output_filter(...);
if (APR_STATUS_IS_EAGAIN(rv)) {
block until writeability on client socket (or timeout);
}
}
The event_output_filter() is responsible for changing the connection
state
to CONN_STATE_LINGER or CONN_STATE_CHECK_REQUEST_LINE_READABLE
(depending on the keepalive settings) when all of the output has been
written.
Note that this patch keeps a worker thread waiting on the connection
until
it exits CONN_STATE_WRITE_COMPLETION. The real benefits of the
nonblocking output filter will come later, when the event MPM code is
enhanced to hand off the connection to the central poller thread
whenever
it needs to wait for writeability. I've purposely avoided making
that change
so far; I want to make sure the nonblocking filter design is sound
before
adding the complexity of truly async write completion.
Known bugs:
- The new filter doesn't support nonblocking reads of pipe or
socket buckets.
(It's functionally correct, I think, but not optimal.)
- It doesn't yet support EnableSendfile off; and it may not even
compile
on platforms that don't have sendfile.
- Because event_output_filter() can return APR_EAGAIN, the few things
that expect it to do a blocking write--like ap_rwrite()--are
broken. This
could be fixed either by:
1. Changing the callers to understand an EAGAIN response, or
2. Changing event_output_filter() to return APR_SUCCESS instead
of APR_EAGAIN if the connection is not in write-completion
state.
Next steps:
I need a volunteer or two to review this patch. Given the fundamental
nature of some of the design decisions involved, I'd like to get a
second
opinion before committing this into 2.3-dev.
Thanks,
Brian
Index: server/mpm/experimental/event/Makefile.in
===
--- server/mpm/experimental/event/Makefile.in (revision 267492)
+++ server/mpm/experimental/event/Makefile.in (working copy)
@@ -1,5 +1,5 @@
LTLIBRARY_NAME= libevent.la
-LTLIBRARY_SOURCES = event.c fdqueue.c pod.c
+LTLIBRARY_SOURCES = event.c fdqueue.c pod.c event_filters.c
include $(top_srcdir)/build/ltlib.mk
Index: server/mpm/experimental/event/event.c
===
--- server/mpm/experimental/event/event.c (revision 267492)
+++ server/mpm/experimental/event/event.c (working copy)
@@ -50,6 +50,7 @@
#include apr_file_io.h
#include apr_thread_proc.h
#include apr_signal.h
+#include apr_support.h
#include apr_thread_mutex.h
#include apr_proc_mutex.h
#include apr_poll.h
@@ -637,6 +638,25 @@
cs-state = CONN_STATE_LINGER;
}
}
+
+if (cs-state == CONN_STATE_WRITE_COMPLETION) {
+ap_filter_t *output_filter = c-output_filters;
+while (output_filter-next != NULL) {
+output_filter = output_filter-next;
+}
+do {
+apr_status_t rv;
+rv = output_filter-frec-filter_func.out_func(output_filter,
NULL);
+if (APR_STATUS_IS_EAGAIN(rv)) {
+rv = apr_wait_for_io_or_timeout(NULL, cs-pfd.desc.s, 0);
+if (rv != APR_SUCCESS) {
+ap_log_error(APLOG_MARK, APLOG_CRIT, rv, ap_server_conf,
+ poll for write completion failed);
+break;
+}
+}
+} while (cs-state == CONN_STATE_WRITE_COMPLETION);
+}
if (cs-state == CONN_STATE_LINGER) {
ap_lingering_close(c);
@@ -2189,6 +2209,8 @@
return OK;
}
+extern apr_status_t event_output_filter(ap_filter_t *f, apr_bucket_brigade
*bb);
+
static void event_hooks(apr_pool_t * p)
{
/* The worker open_logs phase must run before the core's, or stderr
@@ -2203,6 +2225,8 @@
* to retrieve it, so register as REALLY_FIRST
*/
ap_hook_pre_config(worker_pre_config, NULL, NULL, APR_HOOK_REALLY_FIRST);
+ap_core_output_filter_handle = ap_register_output_filter(CORE,
event_output_filter,
+
Re: [PATCH] Custom Core-Output-Filter
On Sep 4, 2005, at 1:42 PM, Paul Querna wrote: So, here is a hackish patch to allow easier building of custom core output filters, for places like the Event MPM. It uses the APR optional functions to optionally register a function to replace the C-O-F. Thanks, that's a bit cleaner than the hack that I put in to the event_hooks() callback function. To complicate the issue a bit, I'm wondering if we should replace ap_core_output_filter() with a nonblocking version for _all_ MPMs in 2.3/2.4. The filter could query the MPM properties to decide how to handle flush and EOC buckets: - For MPMs with async write completion support, it would setaside the data and change the connection state to CONN_STATE_WRITE_COMPLETION - For MPMs without async write completion, the filter would do a blocking write of all the data. A case where something like this might be useful (aside from the event MPM) is prefork. If people are running prefork because they need to use non-threadsafe modules, but they're running on platforms with thread support, adding an optional write completion thread could yield better scalability. Brian
Re: [PATCH] Custom Core-Output-Filter
On Sep 4, 2005, at 4:26 PM, Paul Querna wrote: I am thinking we should create a svn branch to flesh out some of these ideas? How does 'branches/async-dev' sound to you? +1. I'd also be comfortable doing work under server/mpm/experimental in the 2.3 trunk and then refactoring it up into the core once it's stable, but an async-dev branch would be simpler. Brian
PATCH: lazy initialization of TCP_NODELAY (workaround for 2.6 TCP_CORK problem)
The attached patch delays the setting of TCP_NODELAY on client
connections
until the first time core_output_filter has to do a writev_it_all()
or emulate_sendfile().
My motivation for this is to work around the TCP_NODELAY/TCP_CORK
problem
in Linux 2.6. However, it also will save a couple of syscalls for
any request that
is handled with sendfile(2).
Note that there was an APR bug that caused TCP_NODELAY to be set on the
listener socket at startup as a side-effect of TCP_DEFER_ACCEPT being
set.
I just committed a fix for this. Without that fix, Linux 2.6 systems
using this httpd
patch will still exhibit the corking problem.
Can a couple of volunteers please test and/or review this patch? I'd
appreciate
a second opinion before I commit, given the subtlety of the NODELAY
and CORK
semantics on various platforms.
Thanks,
Brian
Index: server/core.c
===
--- server/core.c (revision 240169)
+++ server/core.c (working copy)
@@ -3827,25 +3827,6 @@
{
core_net_rec *net = apr_palloc(c-pool, sizeof(*net));
-#ifdef AP_MPM_DISABLE_NAGLE_ACCEPTED_SOCK
-apr_status_t rv;
-
-/* The Nagle algorithm says that we should delay sending partial
- * packets in hopes of getting more data. We don't want to do
- * this; we are not telnet. There are bad interactions between
- * persistent connections and Nagle's algorithm that have very severe
- * performance penalties. (Failing to disable Nagle is not much of a
- * problem with simple HTTP.)
- */
-rv = apr_socket_opt_set(csd, APR_TCP_NODELAY, 1);
-if (rv != APR_SUCCESS rv != APR_ENOTIMPL) {
-/* expected cause is that the client disconnected already,
- * hence the debug level
- */
-ap_log_cerror(APLOG_MARK, APLOG_DEBUG, rv, c,
- apr_socket_opt_set(APR_TCP_NODELAY));
-}
-#endif
net-c = c;
net-in_ctx = NULL;
net-out_ctx = NULL;
Index: server/listen.c
===
--- server/listen.c (revision 240164)
+++ server/listen.c (working copy)
@@ -126,10 +126,6 @@
}
}
-#if APR_TCP_NODELAY_INHERITED
-ap_sock_disable_nagle(s);
-#endif
-
if ((stat = apr_socket_bind(s, server-bind_addr)) != APR_SUCCESS) {
ap_log_perror(APLOG_MARK, APLOG_STARTUP|APLOG_CRIT, stat, p,
make_sock: could not bind to address %pI,
Index: server/core_filters.c
===
--- server/core_filters.c (revision 240164)
+++ server/core_filters.c (working copy)
@@ -887,6 +887,10 @@
else
#endif
{
+#ifdef AP_MPM_DISABLE_NAGLE_ACCEPTED_SOCK
+(void)apr_socket_opt_set(net-client_socket,
+ APR_TCP_NODELAY, 1);
+#endif
rv = emulate_sendfile(net, fd, hdtr, foffset, flen,
bytes_sent);
}
@@ -898,7 +902,9 @@
}
else {
apr_size_t bytes_sent;
-
+#ifdef AP_MPM_DISABLE_NAGLE_ACCEPTED_SOCK
+(void)apr_socket_opt_set(net-client_socket, APR_TCP_NODELAY, 1);
+#endif
rv = writev_it_all(net-client_socket,
vec, nvec,
nbytes, bytes_sent);
Re: PATCH: lazy initialization of TCP_NODELAY (workaround for 2.6 TCP_CORK problem)
On Aug 26, 2005, at 12:55 AM, Joe Orton wrote: On Fri, Aug 26, 2005 at 12:42:19AM -0700, Brian Pane wrote: The attached patch delays the setting of TCP_NODELAY on client connections until the first time core_output_filter has to do a writev_it_all() or emulate_sendfile(). My motivation for this is to work around the TCP_NODELAY/TCP_CORK problem in Linux 2.6. However, it also will save a couple of syscalls for any request that is handled with sendfile(2). This will actually end up being *more* expensive on 2.6 systems in the long run, though, right? I'm not convinced this is a good idea. With APR changed to allow setting TCP_CORK and TCP_NODELAY at the same time with 2.6, it is cheaper to just set TCP_NODELAY once on the listening socket and never have to touch it again. I didn't think it was actually possible for APR to allow TCP_CORK and TCP_NODELAY at the same time. From the tcp(7) manual page on my FC4 installation, TCP_CORK If set, don’t send out partial frames. All queued partial frames are sent when the option is cleared again. This is use- ful for prepending headers before calling sendfile (2), or for throughput optimization. This option cannot be combined with TCP_NODELAY. This option should not be used in code intended to be portable. and TCP_NODELAY If set, disable the Nagle algorithm. This means that segments are always sent as soon as possible, even if there is only a small amount of data. When not set, data is buffered until there is a sufficient amount to send out, thereby avoiding the frequent sending of small packets, which results in poor uti- lization of the network. This option cannot be used at the same time as the option TCP_CORK. Is the manpage out of date? If it's possible to use both TCP_CORK and TCP_NODELAY on the same socket in 2.6.13 or later (assuming that's when the fix for the current NODELAY toggling problem becomes available), then yes, my lazy evaluation approach will be less efficient than just setting TCP_NODELAY on the listener socket--for requests that don't use sendfile. For requests that do use sendfile, I think the logic implemented by my patch will be optimal for both 2.6.1-2.6.12 and 2.6.13+ Brian
Re: PATCH: lazy initialization of TCP_NODELAY (workaround for 2.6 TCP_CORK problem)
On Aug 26, 2005, at 1:59 AM, Joe Orton wrote: On Fri, Aug 26, 2005 at 01:23:15AM -0700, Brian Pane wrote: I didn't think it was actually possible for APR to allow TCP_CORK and TCP_NODELAY at the same time. From the tcp(7) manual page on my FC4 installation, That's out of date yes, see recent thread ;) All 2.6.x kernels do allow setting both TCP_CORK and TCP_NODELAY at the same time. All current 2.6.x kernels have the bug in TCP_CORK handling which means that if TCP_NODELAY was ever enabled on the socket, TCP_CORK will not take effect. The fix for that was in the 2.6.13-rc7 release, for the curious: http://www.kernel.org/git/?p=linux/kernel/git/torvalds/ linux-2.6.git;a=commit;h=89ebd197eb2cd31d6187db344d5117064e19fdde Ah, thanks. With this kernel fix forthcoming, the current httpd implementation (sans my patch) will do the right thing for both sendfile and non- sendfile responses. Brian
TCP_CORK in Linux 2.6: not broken, but it doesn't work with TCP_NODELAY
There was some discussion a few weeks ago about whether TCP_CORK was
broken on Linux 2.6.
I did some tests on Linux 2.6.12 and found that TCP_CORK *does*
appear to
work. However, it doesn't work if TCP_NODELAY is also set on the
socket (as
the tcp(7) manual page warns).
I've attached my test program in case it's useful to others. It's
run as
sendfile_test [--cork] [--nodelay] [--nonblock] filename
listener_port_number.
It listens endlessly for connections on the specified port. When it
gets a connection,
it write(2)s a short HTTP response header and then calls sendfile to
transmit the
entire contents of the file. The optional arguments tell it whether
to set TCP_CORK,
TCP_NODELAY, and/or O_NONBLOCK on the accepted connection; the default
is not to set any of these.
Looking at packet traces, the pattern I see is:
* TCP_NODELAY not set, TCP_CORK not set: the HTTP response header
arrives in a packet all by itself, followed by the start of the
file in the next packet.
* TCP_NODELAY not set, TCP_CORK set: the first packet of the response
contains
the HTTP response header and the first part of the file. The
packet is filled with
data up to the MSS, unless the total header+file content is
smaller than the MSS.
* TCP_NODELAY set, TCP_CORK set: the HTTP response header
arrives in a packet all by itself, followed by the start of the
file in the next packet.
* TCP_NODELAY set, TCP_CORK not set: the HTTP response header
arrives in a packet all by itself, followed by the start of the
file in the next packet.
I tested with both blocking or nonblocking sockets, and both modes
yielded
these same results.
httpd-2.x always sets TCP_NODELAY, so it makes sense that people are
observing it sending the response header in a separate packet under
Linux
2.6. (I'm guessing that TCP_CORK works even in the presence of
TCP_NODELAY under 2.4, but I don't have a 2.4 system to test.)
Rather than automatically setting TCP_NODELAY in core_pre_connection(),
perhaps we should set it conditionally in core_output_filter(), where
we have
enough information to tell whether it's needed.
Brian
#include stdio.h
#include stdlib.h
#include unistd.h
#include sys/errno.h
#include sys/fcntl.h
#include sys/poll.h
#include sys/socket.h
#include sys/stat.h
#include sys/types.h
#include netinet/in.h
#include netinet/tcp.h
static void usage();
static int write_all(int connection, const char *buf, size_t size);
static int sendfile_all(int connection, int fd, off_t file_size);
#define LISTENER_PORT 8080
static const char response_header[] = HTTP/1.0 200 OK\r\n\r\n;
int
main(int argc, char **argv)
{
int port_num;
struct sockaddr_in addr;
int listener;
int fd;
struct stat file_info;
int enable_tcp_nodelay = 0;
int enable_tcp_cork = 0;
int enable_nonblocking = 0;
int i;
if (argc 3) {
usage();
}
for (i = 1; i argc - 2; i++) {
if (!strcmp(argv[i], --nodelay)) {
enable_tcp_nodelay = 1;
}
else if (!strcmp(argv[i], --cork)) {
enable_tcp_cork = 1;
}
else if (!strcmp(argv[i], --nonblock)) {
enable_nonblocking = 1;
}
else {
usage();
}
}
fd = open(argv[i], O_RDONLY);
if (fd == -1) {
perror(open);
exit(1);
}
if (fstat(fd, file_info) == -1) {
perror(fstat);
exit(1);
}
port_num = atoi(argv[i + 1]);
listener = socket(AF_INET, SOCK_STREAM, PF_UNSPEC);
if (listener == -1) {
perror(socket);
exit(1);
}
addr.sin_family = AF_INET;
addr.sin_port = htons(port_num);
addr.sin_addr.s_addr = htonl(INADDR_ANY);
if (bind(listener, (struct sockaddr *)addr, sizeof(addr)) == -1) {
perror(bind);
exit(1);
}
if (listen(listener, 1024) == -1) {
perror(listen);
exit(1);
}
for (;;) {
off_t offset = 0;
off_t bytes_remaining = file_info.st_size;
struct sockaddr_in new_addr;
socklen_t addr_length = sizeof(new_addr);
int connection = accept(listener, (struct sockaddr *)new_addr,
addr_length);
if (connection == -1) {
perror(accept);
break;
}
if (enable_nonblocking) {
int flags = fcntl(connection, F_GETFL, 0);
if (flags == -1) {
perror(fcntl(F_GETFL));
break;
}
flags |= O_NONBLOCK;
if (fcntl(connection, F_SETFL, flags) == -1) {
perror(fcntl(F_SETFL));
break;
}
}
if (enable_tcp_nodelay) {
int flag = 1;
if (setsockopt(connection, IPPROTO_TCP, TCP_NODELAY, flag,
sizeof(flag)) == -1) {
perror(setsockopt(TCP_NODELAY));
}
}
if (enable_tcp_cork) {
int flag = 1;
if (setsockopt(connection, IPPROTO_TCP, TCP_CORK, flag,
sizeof(flag)) == -1) {
perror(setsockopt(TCP_CORK));
}
}
if (write_all(connection, response_header, sizeof(response_header) - 1)
== -1) {
perror(write);
}
else if
event MPM: async write completion?
Looking at the pattern of calls to ap_core_output_filter() in the event MPM, it occurred to me that it may be straightforward to hand off the writing of the request to an async completion thread in a lot of useful real-world cases. In function check_pipeline_flush() in http_request.c, a flush bucket is passed to the output filter stack if there's no pipelined request ready to be read (or if keep-alives aren't enabled for the connection). When this occurs, two things are true: * One or more complete responses--and no partial responses--have been sent to the output filter stack for this connection. * If these responses have not been sent to the client, they will now be sent in their entirety before the httpd attempts to do any further reads from this connection. Instead of sending a flush bucket to the output filter stack in this scenario, though, perhaps we could send a new metadata bucket that denotes end of output until everything currently in the pipeline has been written. Upon receiving this bucket, core_output_filter would register the connection with a pollset to watch for writability. A write completion thread would poll in an endless loop, writing data from each set-aside brigade whenever its corresponding connection became writable. Upon sending the last bucket of a set-aside brigade to the network, the completion thread would put the connection in either CONN_STATE_LINGER or CONN_STATE_CHECK_REQUEST_LINE_READABLE state (depending on whether keepalives were enabled) and re-register the connection with the event MPM's main pollset to wait for readability or timeout. The pollset used to wait for writability could be the same pollset that's currently used to watch for readability and timeouts. Similarly, the write completion thread could be combined with the current listener thread. I'm eager to hear some feedback on this idea: * Will it work? Or am I overlooking some design flaw? * Will it help? I.e., will it make a noticeable improvement in the connections-to-threads ratio? * What about the access logger? If the writing of responses were completed asynchronously, a request could be logged before its response was sent--or, worse yet, before the sending of its response failed due to, say, the client closing the connection early. One solution that comes to mind is to have the write completion thread invoke the access logger--perhaps triggered by a metadata bucket in the output brigade. -Brian
Re: Pondering strings in Apache 3.x
On Jul 19, 2005, at 12:55 PM, William A. Rowe, Jr. wrote: Greg and a few others voiced interest in moving from null-term strings to counted strings for a future version of Apache. This was too broad a scope change to make it into 2.0, of course, and was dropped on the floor for the time being. I'm wondering today; what metadata interests us in an ap_string_t prefix header? I have a hunch that a short, 65536, is enough to map most data we want to handle in one chunk; brigades are better for handling large sets of data. Of course we could push that to an int, or size_t, but there would be a small memory penalty. It might be overcome by cpu-specific optimized int or size_t handling behavior, since the assembly code wouldn't need to truncate short values. Perhaps, both bytes allocated/used, in order to play optimized games with string allocation. Perhaps, a refcount? (This doesn't play well with pool allocations, obviously.) But the byte count clearly isn't enough. I'm thinking of; encoding; is this data URI escaped or un-escaped? tainted; is it raw? or has it been untainted with context-specific validity checks? charset; is this native? (e.g. EBCDIC). utf-8? opaque or otherwise a specific set? What else interests us within an 'ap_string_t' header, that would help eliminate bugs within httpd? A random trailing short following the string, in a 'string debug' mode, to detect buffer overflows? Something similar to detect underflows? Open to all ideas. This may be a bit more radical than you were hoping for, but... I like the idea of using a reference-counted, non-null-terminated string type for 3.x. More generally, it would be great to have overflow detection on all arrays. And although I like the performance benefits of the pool memory allocators, I remember how tricky it was to debug some of the pool and bucket lifetime problems that we encountered during the development of 2.0 (especially in filters). All things considered, I don't think I'd mind the overhead of a garbage collection thread. Thus I can't help but wonder: Would 3.0 be a good time to consider trying a Java-based httpd? Brian
Re: [VOTE] mod_ftp for HTTP Server Project
On Jul 7, 2005, at 11:26 AM, Jim Jagielski wrote: I therefore Call A Vote on whether we should support mod_ftp for inclusion into the Incubator and if we should accept mod_ftp upon graduation from the Incubator. +1 Brian
Re: new performance profiling data for httpd-2.1
On Jul 3, 2005, at 11:27 PM, Paul Querna wrote: Brian Pane wrote: I spent some time today profiling the latest httpd-2.1 trunk on OS X, Which version of APR? The shared library names make it sound like APR 1.0.0, not APR/APR-Util trunk? I was using the latest trunk of APR APR-Util from svn. Brian
Re: PCRE in 2.1/2.2
Brian Pane wrote: Richard Jones wrote: On Thu, Nov 25, 2004 at 04:00:46PM -0800, Brian Pane wrote: I volunteer to upgrade the copy of PCRE in httpd-2.1 to the latest version. I'll have a bit of time to work on this over the next few days. Any chance of integrating the fix in bug 27550 at the same time? http://issues.apache.org/bugzilla/show_bug.cgi?id=27550 Rich. I'll take a look at the patch in 27550 some time this weekend if nobody else gets to it first. Okay, I checked out the patch attached to that bug. Supporting the use of an external copy of PCRE makes sense. The patch also changes various core PCRE function names to ap_*. That definitely helps prevent collisions with libc or libpthread versions of the regex functions on some platforms, but changing the PCRE function names may be controversial... would any other committers care to chime in with their thoughts on this? Meanwhile, now that we have the latest release of PCRE in httpd-2.1, one option would help somewhat would be to backport it into httpd-2.0. Brian
Re: PCRE in 2.1/2.2
Justin Erenkrantz wrote: --On Sunday, November 28, 2004 3:36 PM -0800 Brian Pane [EMAIL PROTECTED] wrote: Meanwhile, now that we have the latest release of PCRE in httpd-2.1, one option would help somewhat would be to backport it into httpd-2.0. I doubt we'd be able to maintain binary compatibility if we did that. So, I'd strongly urge that we don't touch PCRE in 2.0.x. -- justin From a binary compatibility perspective, what's the specific PCRE API change you're concerned about? Thanks, Brian
Re: PCRE in 2.1/2.2
Joe Orton wrote: On Fri, Nov 26, 2004 at 11:55:35PM -0800, Brian Pane wrote: Yep. I just did that, and it worked as expected. I ended up with a clean merge: the only files that needed merge conflict resolution were those that had been modified in the srclib/pcre copy: Looks good, thanks Brian. Do you think it's worth keeping all ~1Mb of the doc/ directory in srclib/pcre? I think it might as well be removed, they won't get installed and developers can refer to the vendor/pcre copies if necessary. svn merge will happily ignore the removed files when merging to future versions so there's no issue there. +1, I'll remove docs/* from srclib/pcre and add a README pointing to vendor/pcre. Brian
Re: PCRE in 2.1/2.2
Brad Nicholes wrote: While trying to get NetWare to build with the new PCRE update, I noticed that there is a duplicate of pcreposix.h in both include/ and srclib/pcre/ directories. Currently they don't match which is causing the NetWare build to break. I can sync them up, but the real question is should this file be duplicated in include/ or moved from srclib/pcre/ to include/ ? It's probably easiest to maintain the file in include/ only. I'll try the svn move in my working copy and make sure it works before committing. This may make it a bit more complicated to merge in future PCRE releases, but such merges happen very infrequently. Brian
Re: PCRE in 2.1/2.2
Richard Jones wrote: On Thu, Nov 25, 2004 at 04:00:46PM -0800, Brian Pane wrote: I volunteer to upgrade the copy of PCRE in httpd-2.1 to the latest version. I'll have a bit of time to work on this over the next few days. Any chance of integrating the fix in bug 27550 at the same time? http://issues.apache.org/bugzilla/show_bug.cgi?id=27550 Rich. I'll take a look at the patch in 27550 some time this weekend if nobody else gets to it first. Brian
Re: 2.1 build broken ?
Ian Holsman wrote: hi. I'm building from a clean slate (new box) and am having issues building it. It seems not to be including the stuff in listen.c its OS/X 10.3.6 with november's gcc patch.. no fink on it either. I just did a build from a clean SVN checkout on 10.3.6, and it worked. I was using prefork rather than worker, though, in case that's of any help in isolating the problem. Brian
FW: svn commit: r106690 - in httpd/httpd/trunk: . srclib/pcre srclib/pcre/doc srclib/pcre/doc/html srclib/pcre/testdata
This commit message bounced because it exceeded the mailer daemon's size limit. Here's the list of changed files. You can see the details of the diffs via the link below. Brian Author: brianp Date: Fri Nov 26 23:28:04 2004 New Revision: 106690 URL: http://svn.apache.org/viewcvs?view=revrev=106690 Log: Upgraded the copy of PCRE within srclib/pcre to version 5.0 Added: httpd/httpd/trunk/srclib/pcre/doc/html/ - copied from r106688, httpd/httpd/vendor/pcre/5.0/doc/html/ httpd/httpd/trunk/srclib/pcre/doc/html/index.html - copied unchanged from r106688, httpd/httpd/vendor/pcre/5.0/doc/html/index.html httpd/httpd/trunk/srclib/pcre/doc/html/pcre.html - copied unchanged from r106688, httpd/httpd/vendor/pcre/5.0/doc/html/pcre.html httpd/httpd/trunk/srclib/pcre/doc/html/pcre_compile.html - copied unchanged from r106688, httpd/httpd/vendor/pcre/5.0/doc/html/pcre_compile.html httpd/httpd/trunk/srclib/pcre/doc/html/pcre_config.html - copied unchanged from r106688, httpd/httpd/vendor/pcre/5.0/doc/html/pcre_config.html httpd/httpd/trunk/srclib/pcre/doc/html/pcre_copy_named_substring.html - copied unchanged from r106688, httpd/httpd/vendor/pcre/5.0/doc/html/pcre_copy_named_substring.html httpd/httpd/trunk/srclib/pcre/doc/html/pcre_copy_substring.html - copied unchanged from r106688, httpd/httpd/vendor/pcre/5.0/doc/html/pcre_copy_substring.html httpd/httpd/trunk/srclib/pcre/doc/html/pcre_exec.html - copied unchanged from r106688, httpd/httpd/vendor/pcre/5.0/doc/html/pcre_exec.html httpd/httpd/trunk/srclib/pcre/doc/html/pcre_free_substring.html - copied unchanged from r106688, httpd/httpd/vendor/pcre/5.0/doc/html/pcre_free_substring.html httpd/httpd/trunk/srclib/pcre/doc/html/pcre_free_substring_list.html - copied unchanged from r106688, httpd/httpd/vendor/pcre/5.0/doc/html/pcre_free_substring_list.html httpd/httpd/trunk/srclib/pcre/doc/html/pcre_fullinfo.html - copied unchanged from r106688, httpd/httpd/vendor/pcre/5.0/doc/html/pcre_fullinfo.html httpd/httpd/trunk/srclib/pcre/doc/html/pcre_get_named_substring.html - copied unchanged from r106688, httpd/httpd/vendor/pcre/5.0/doc/html/pcre_get_named_substring.html httpd/httpd/trunk/srclib/pcre/doc/html/pcre_get_stringnumber.html - copied unchanged from r106688, httpd/httpd/vendor/pcre/5.0/doc/html/pcre_get_stringnumber.html httpd/httpd/trunk/srclib/pcre/doc/html/pcre_get_substring.html - copied unchanged from r106688, httpd/httpd/vendor/pcre/5.0/doc/html/pcre_get_substring.html httpd/httpd/trunk/srclib/pcre/doc/html/pcre_get_substring_list.html - copied unchanged from r106688, httpd/httpd/vendor/pcre/5.0/doc/html/pcre_get_substring_list.html httpd/httpd/trunk/srclib/pcre/doc/html/pcre_info.html - copied unchanged from r106688, httpd/httpd/vendor/pcre/5.0/doc/html/pcre_info.html httpd/httpd/trunk/srclib/pcre/doc/html/pcre_maketables.html - copied unchanged from r106688, httpd/httpd/vendor/pcre/5.0/doc/html/pcre_maketables.html httpd/httpd/trunk/srclib/pcre/doc/html/pcre_study.html - copied unchanged from r106688, httpd/httpd/vendor/pcre/5.0/doc/html/pcre_study.html httpd/httpd/trunk/srclib/pcre/doc/html/pcre_version.html - copied unchanged from r106688, httpd/httpd/vendor/pcre/5.0/doc/html/pcre_version.html httpd/httpd/trunk/srclib/pcre/doc/html/pcreapi.html - copied unchanged from r106688, httpd/httpd/vendor/pcre/5.0/doc/html/pcreapi.html httpd/httpd/trunk/srclib/pcre/doc/html/pcrebuild.html - copied unchanged from r106688, httpd/httpd/vendor/pcre/5.0/doc/html/pcrebuild.html httpd/httpd/trunk/srclib/pcre/doc/html/pcrecallout.html - copied unchanged from r106688, httpd/httpd/vendor/pcre/5.0/doc/html/pcrecallout.html httpd/httpd/trunk/srclib/pcre/doc/html/pcrecompat.html - copied unchanged from r106688, httpd/httpd/vendor/pcre/5.0/doc/html/pcrecompat.html httpd/httpd/trunk/srclib/pcre/doc/html/pcregrep.html - copied unchanged from r106688, httpd/httpd/vendor/pcre/5.0/doc/html/pcregrep.html httpd/httpd/trunk/srclib/pcre/doc/html/pcrepartial.html - copied unchanged from r106688, httpd/httpd/vendor/pcre/5.0/doc/html/pcrepartial.html httpd/httpd/trunk/srclib/pcre/doc/html/pcrepattern.html - copied unchanged from r106688, httpd/httpd/vendor/pcre/5.0/doc/html/pcrepattern.html httpd/httpd/trunk/srclib/pcre/doc/html/pcreperform.html - copied unchanged from r106688, httpd/httpd/vendor/pcre/5.0/doc/html/pcreperform.html httpd/httpd/trunk/srclib/pcre/doc/html/pcreposix.html - copied unchanged from r106688, httpd/httpd/vendor/pcre/5.0/doc/html/pcreposix.html httpd/httpd/trunk/srclib/pcre/doc/html/pcreprecompile.html - copied unchanged from r106688, httpd/httpd/vendor/pcre/5.0/doc/html/pcreprecompile.html httpd/httpd/trunk/srclib/pcre/doc/html/pcresample.html - copied unchanged from r106688, httpd/httpd/vendor/pcre/5.0/doc/html/pcresample.html httpd/httpd/trunk/srclib/pcre/doc/html/pcretest.html - copied
Re: PCRE in 2.1/2.2
Joe Orton wrote: I also spent some time recently working out how to apply that process to srclib/pcre and it seemed fairly straight-forward. It seems like the process works OK even if the step to copy the original vendor pcre/current to srclib/pcre is skipped, so the current srclib/pcre can be used to merge into. The steps I came up with would be just: import pcre-3.9 to vendor/pcre/current cp vendor/pcre/current vendor/pcre/pcre-3.9 unpack pcre-5.0 into vendor/pcre/current, svn add everything then commit verify that vendor/pcre/current exactly matches contents of pcre-5.0.tar.gz cp vendor/pcre/current vendor/pcre/pcre-5.0 and then the merge between pcre-3.9 and current can happen. You concur? Yep. I just did that, and it worked as expected. I ended up with a clean merge: the only files that needed merge conflict resolution were those that had been modified in the srclib/pcre copy: pcreposix.c (conflicts due to my old SMALL_NMATCH change, as expected; I replaced this whole file with the 5.0 version) Makefile.in configure.in For the latter two files, I hand-merged just the essential changes from 5.0, leaving out the code that would require aclocal. Brian
PATCH: call aclocal for PCRE in buildconf
I found that the configure.in in recent releases of PCRE uses some
autoconf macros that won't work unless aclocal is called first to
produce a proper aclocal.m4: AC_PROG_LIBTOOL and AC_LIBTOOL_WIN32_DLL
This buildconf patch fixes it for me (and works with both the new PCRE
5.0 and the old version that's currently in the httpd-2.1 tree).
Before I commit it, though, does anyone know of a system where it's not
safe to assume aclocal exists?
Thanks,
Brian
Index: buildconf
===
--- buildconf (revision 106561)
+++ buildconf (working copy)
@@ -85,6 +85,7 @@
touch .deps
rm -f aclocal.m4
rm -f generated_lists
+rm -f srclib/pcre/aclocal.m4
# Remove autoconf 2.5x cache directories
rm -rf autom4te*.cache srclib/pcre/autom4te*.cache
@@ -137,7 +138,7 @@
fi
echo rebuilding $pcre_configure
-(cd srclib/pcre ${AUTOCONF:-autoconf})
+(cd srclib/pcre ${ACLOCAL:-aclocal} ${AUTOCONF:-autoconf})
echo rebuilding $config_h_in
rm -f $config_h_in
Re: PCRE in 2.1/2.2
I volunteer to upgrade the copy of PCRE in httpd-2.1 to the latest version. I'll have a bit of time to work on this over the next few days. One question, though... I'd like to follow the vendor-branch approach outlined in the Subversion book, http://svnbook.red-bean.com/en/1.0/ch07s04.html. In this approach, a clean copy of the 3rd party code is maintained on a branch under some unique path, so that it's easy to generate a diff between versions M and N of the 3rd party code that can then be merged into our code base. Do we have a standard yet for where vendor branches should be located within the httpd Subversion repository? E.g., should the clean PCRE releases be stored under svn.apache.org/repos/asf/httpd/vendor/pcre? or svn.apache.org/repos/vendor/pcre? Or something else entirely? Thanks, Brian
Re: PCRE in 2.1/2.2
Paul Querna wrote: Is there any reason to keep libpcre in the httpd tree? I believe we should use the PCRE installed on the system. If its not on the system, the user can install it. There are several bugs from 3rd party modules that also try to use pcre, and get conflicting symbols because of the version embedded within apache 2.0. No comments in a week, and Ill start taking it out. -Paul Querna About two years ago, I made some performance optimizations to the copy of pcre in the httpd-2.0 tree. I submitted the diffs to the pcre maintainer, but I don't know if they've made it into a subsequent pcre release. The changes, as I recall, were to remove some mallocs/frees in the regexec function. They may or may not still be needed with newer versions of pcre, if the pcre code has changed substantially since then. http://marc.theaimsgroup.com/?t=10165900551 Brian
Re: Event MPM
Paul Querna wrote: Brian Akins wrote: Paul Querna wrote: [...] Have you tried it with higher number of clients -- i.e,. -c 1024? Nope. I was already maxing out my 100mbit LAN at 25 clients. I don't have a good testing area for static content request benchmarking. I am thinking of trying to find an old pentium I with PCI and putting a GigE card in it just for benchmarking. How about modifying ab to add a delay of a second or two between successive read(2) calls on the same connection (and limiting the max read size to a small value, to make sure each response requires multiple reads)? The throughput numbers wouldn't be very impressive, of course, but you'd be able to see how much memory and how many threads the Event MPM uses in handling a real-world number of concurrent connections, compared to Worker. The current patch does not spawn new threads on demand. You need to set ThreadsPerChild 1200 to test that many clients. This is on my short list of things to change before committing the Event MPM to CVS. This part seems surprising. What's the relationship between clients and threads in the Event MPM--does it use a thread per connection? I haven't had a chance to check out the Event MPM yet, but I'm planning to download it and study the code later this week. For what it's worth, I can think of at least one killer app for the Event MPM beyond keep-alive handling: as an efficient (low memory thread use per connection) connection multiplexer in front of an inefficient (high memory thread use per connection) appserver. -Brian
More musings about asynchronous MPMs Re: Event MPM
Paul Querna wrote: Paul Querna wrote: A thread per-connection that is currently being processed. Note that this is not the traditional 'event' model that people write huge papers about and thttpd raves about, but rather a hybrid that uses a Worker Thread todo the processing, and a single 'event' thread to handle places where we are waiting for IO. (Currently accept() and Keep Alive Requests). Perhaps it needs a different name? (eworker?) A future direction to investigate would be to make all of the initial Header parsing be done Async, and then switch to a Worker thread to preform all the post_read hooks, handlers, and filters. I believe this could be done without breaking many 3rd party modules. (SSL could be a bigger challenge here...) Some academics have played with this model of a event thread + worker threads. Best I can find is the Java 'SEDA: An Architecture for Highly Concurrent Server Applications'. [1] Yeah, SEDA's model of processing stages--basically a succession of thread pools through which a request passes, each with a queue in front--looks like a promising way of mixing event-based and non-event-based processing. Another approach I've looked at is to use Schmidt's Reactor design pattern, in which a central listener thread dispatches each received event to one of a pool of worker threads. The worker is then allowed to do arbitrarily complex processing (provided you have enough worker threads) before deciding to tell the listener thread that it wants to wait for another event. I did some prototyping of a Leader/Followers variant of this: have a pool of worker threads that take turns grabbing the next event from a queue of received events. If there are no events left in the queue, the next worker thread does a select/poll/etc (and any other idle workers block on a condition variable). This seemed to work reasonably well in a toy test app (it was *almost* capable of implementing HTTP/0.9, but I never had time to try 1.0 or 1.1; oh, and it depended on the java.nio classes to do the select efficiently, so I didn't even try to commit it to server/mpm/experimental. :-) On a more general topic...and at the risk of igniting a distracting debate...does anybody else out there have an interest in doing an async httpd in Java? There are a lot of reasons *not* to do so, mostly related to all the existing httpd-2.0 modules that wouldn't work. The things that seem appealing about trying a Java-based httpd, though, are: - The pool memory model at the core of httpd-1.x and -2.x isn't well suited to MPM designs where multiple threads need to handle the same connection--possibly at the same time, for example when a handler that's generating content needs to push output buckets to an I/O completion thread to avoid having to block the (probably heavyweight) handler thread or make it event-based. Garbage collection on a per-object basis would be a lot easier. - Modern Java implementations seem to be doing smart things from a scalability perspective, like using kqueue/epoll/etc. - And (minor issue) it's a lot easier to refactor things in Java than in C, and I expect that building a good async MPM that handles dynamic content and proxying effectively will require a lot of iterations of design trial and error. Brian
Re: Some benchs results : WAS: Invitation to HTTPD commiters in tomcat-dev
Henri Gomez wrote: I made some benchs on my Linux Fedora Core 2 on a P4 2.8ghz / 1Gb RAM : Apache 2 alone 1202 req/s TC/Coyote 883 req/s One thing I noticed when looking at Tomcat 5.0.x is that its default, static-file-delivering servlet does a stat(2) of each path prefix leading up to the file. A standard installation of Apache 2.x, with FollowSymlinks enabled, doesn't do these stat calls, for obvious performance reasons. Is the stat'ing of all the directories leading up to the requested file in Tomcat intentional (it *is* valuable in some environments for security purposes), or is it just a side-effect of the implementation? Brian
Re: Some Oprofile results
Brian Akins wrote: I'm writing an optimized caching module. I've been using 2.0.50. Here are the top 50 from oprofile ( http://oprofile.sourceforge.net/ ): Linux cnnsquid2 2.6.7-cnn.1smp #1 SMP Wed Jun 16 13:41:14 EDT 2004 x86_64 x86_64 x86_64 GNU/Linux Using leader mpm. I patched apr_atomics so that atomics work on x84_64 The serving is serving ~27k requests per second: Are there optimizations to apr_palloc in 2.1, this seems to be a good place to optimize. There's not much left to optimize in apr_palloc itself. In the common case, the execution path consists of a single conditional check and a bit of pointer addition. But reducing the number of calls to apr_palloc could yield an improvement. Do you happen to have a way to enable call-graph profiling on your test system, to find out what the top call chains are for the top 50 functions? (My understanding is that recent releases of oprofile support this with an optional patch.) The large amount of time spent in memcpy could also be a good candidate for optimization. Brian
Re: 2.2 Roadmap?
Nick Kew wrote: On Sun, 27 Jun 2004, Paul Querna wrote: The 2.0 branch was made over 18 months ago, it is time to make another stable branch. I believe many people want the AAA changes, and it brings even more features to encourage people to upgrade from 1.3. There's another consideration that could be relevant here: people who never touch an N.0 software release. Bump it to 2.3? :-) With the release of 2.0, we moved to a model of even numbers are production releases, odd numbers are development releases. So 2.2.1 will be the right choice for people who don't want to run N.0 releases. :-) This is only a list from my initial thoughts, please comment and make suggestions. I will take the resulting thread and rewrite the ROADMAP file. Smart filtering. We need much better dynamic configuration of the filter chain, with processing depending on the headers. Think an AddOutputFilterByType that isn't a hackish afterthought, and extend that to work with more than just Content-Type. It also fixes the awkwardness currently involved in ordering a nontrivial filter chain. I've got this working with some minor hacks to 2.0. Need time to generalise/abstract it into a proposal. I like that idea. And it reminds me of another thing related to filtering that I think would be valuable (essential, IMHO) for 2.2: redesigning the input filters so that data is pushed through them, instead of letting each input filter pull data from its predecessor. If we could do this in 2.2.0, it would then be possible to implement an async, multiple-connections-per-thread MPM in some future 2.2.x release without breaking the filter API. Alas, this has been on my to-do list for over a year, and I havent' yet had time to work on it. Another aspect of filters that's been problematic in the past has been ensuring that memory in pool buckets gets freed at the right time. This will become even more complicated if future MPMs have multiple threads handling the same connection. (The more I think about this issue, the more I find myself thinking that we should implement buckets as java.nio.ByteBuffer objects and let a JVM worry about reference counting so that we can just worry about web serving.) Brian
Re: x86_64 atomics and linux
On Jun 2, 2004, at 12:40 PM, Brian Akins wrote: AFAIK, the linux x86 atomic stuff can be used unchanged on Linux x86_64. This is based on my digging in the kernel source. All the functions apr uses are identical. Should I submit a patch? Sure, sounds like a good thing to add. Thanks, Brian
Re: [PATCH] RFC: Allow modification/deletion of specific headers in apr_table_t
On Apr 21, 2004, at 9:11 AM, Sumeet Singh wrote: Hi, In my use-case I am dealing with multiple headers with the same key (e.g. Cookie), and need to modify or remove a specific header based on its key and value (e.g. remove a certain cookie header while leaving the rest in). There is no api in apr_tables that would allow me to remove a given header instance. apr_table_unset will remove all cookies. And I can't have my code remove a header from the apr_array_table_t array because that will render the internal hash index incorrect. Secondly, eventhough I can modify the value of a specific header by iterating over the apr_array_header_t, that would be inefficient because I wouldn't be able to use the internal index_first and index_last values. Therefore I have written three functions (patch files attached) and am hoping that the powers-that-be will be willing to review and roll them into the APR codeline. 1) apr_table_set_one (apr_table_t *t, const char *key, const char *oldVal, const char *newVal) replaces value of header key and value oldVal with value newVal. If oldVal is null, then the first occurance of the header is replaces (this is an optimization for situations where we know that only one header exists and don't care about its current value). If the header is not found, then it behaves like apr_table_add. I finally got a chance to read through the patch. The code makes sense, but I have a question: will apr_table_set_one and apr_table_unset_one provide all the functionality you need? Using Cookie and Set-Cookie headers as an example, it seems like an exact match on the oldVal wouldn't be sufficient. If your response headers include something like this: Content-Type; text/html Set-Cookie: userID=1; max-age=86400; domain=.example.com Cache-Control: private Set-Cookie: sessionID=54321; max-age=1800; domain=.example.com and you want to change or remove the userID cookie, it's a lot easier to do a prefix match on userID= than on the entire value of that table entry. (If that table entry was set by some other module, we may not know what the full value of that entry is.) In the general case, an application might need to: 1. iterate through all the elements in a table that match a given key, 2. selectively modify the value of one or more of the matching elements, or delete one or more of the matching elements, 3. and optionally stop iterating based on some application-defined criteria. This sounds a lot like apr_table_do, except that apr_table_do doesn't allow modification of the table elements. If there were a variant of apr_table_do that allowed modification of the value (but not the key) of visited elements, and the deletion of visited elements, would that work for your application? Brian
Re: [PATCH] RFC: Allow modification/deletion of specific headers in apr_table_t
On Apr 21, 2004, at 9:11 AM, Sumeet Singh wrote: Hi, In my use-case I am dealing with multiple headers with the same key (e.g. Cookie), and need to modify or remove a specific header based on its key and value (e.g. remove a certain cookie header while leaving the rest in). There is no api in apr_tables that would allow me to remove a given header instance. apr_table_unset will remove all cookies. And I can't have my code remove a header from the apr_array_table_t array because that will render the internal hash index incorrect. Secondly, eventhough I can modify the value of a specific header by iterating over the apr_array_header_t, that would be inefficient because I wouldn't be able to use the internal index_first and index_last values. Therefore I have written three functions (patch files attached) and am hoping that the powers-that-be will be willing to review and roll them into the APR codeline. 1) apr_table_set_one (apr_table_t *t, const char *key, const char *oldVal, const char *newVal) replaces value of header key and value oldVal with value newVal. If oldVal is null, then the first occurance of the header is replaces (this is an optimization for situations where we know that only one header exists and don't care about its current value). If the header is not found, then it behaves like apr_table_add. 2) apr_table_setn_one(apr_table_t *t, const char *key, const char *oldVal, const char *newVal) Same as apr_table_set_one exept that it doesn't make a copy of key and newVal. 3) apr_table_unset_one(apr_table_t *t, const char *key, const char *oldVal, const char *newVal) Unsets header key with value oldVal. If oldVal is null, then the first instance of the header (only) is unset (this is an optimization for situations where we know that only one header exists and don't care about its current value). +1 for the concept. I'm willing to review and commit the code if nobody else has any objections in the next couple of days. -Brian
Re: functions in SSI
On Mar 12, 2004, at 11:01 AM, Andre Breiler wrote: Hi, On Fri, 12 Mar 2004, Brian Pane wrote: That definitely sounds useful. I think you can get the same effect, though, by using the existing 2.0/2.1 mod_include hook to add new directives... something like this: !--#set_random var=blah min=$min max=$max -- That's not quite as syntactically elegant, but it has the advantage of not requiring any new parsing code. What do you think? Yes that was one of my other ideas as well :) The reason I asked for the below first was that it makes it a little more easier for the html author and that other modules which might use ap_ssi_parse_string won't get the benefit of functions. In addition I was thinking about a function eval feeding the content of a variable into ap_ssi_parse_string again. The last minor thought was that I wouldn't have to replicate most of the #set code again. The third idea (which we use atm.) is: !--#set var=blah function=random value=$min value=$max . I really don't like that one as it means that I have to modify #set without getting a benefit. What speaks for your idea is that here wouldn't be any code changes to existing code needed and that it prevent dump users even getting near recursions. Let me know what you think if I should go for the more flexible (imho) or the safer way. I like the !--#set_random_var ... approach best because it doesn't require new parsing code, and mod_include's parser is very complex already. On the other hand, adding function support within ap_ssi_parse_string would only require very localized changes, so it would be low-risk. I think either approach would be reasonably safe. Brian
Re: functions in SSI
That definitely sounds useful. I think you can get the same effect, though, by using the existing 2.0/2.1 mod_include hook to add new directives... something like this: !--#set_random var=blah min=$min max=$max -- That's not quite as syntactically elegant, but it has the advantage of not requiring any new parsing code. What do you think? Brian On Mar 11, 2004, at 6:46 AM, Andre Breiler wrote: Hi, I wonder what your opinion is on having functions in addition to variables in SSIs. An example would be --#set var=blah value=random($min,$max) -- . My idea is to make ap_ssi_parse_string aware of functions and call a function which does the SSI function parsing/calling. In addition I'd export a function so that other modules can register SSI functions as well. Andre -- Andre' Breiler | Tel: +44 (0) 1628 40 BBC Internet Services | URL: http://support.bbc.co.uk Maiden House, Vanwell Road | Maidenhead, SL6 4UB | Mail me if possible. And use a Subject line.
Re: filtering huge request bodies (like 650MB files)
On Dec 10, 2003, at 5:15 PM, Cliff Woolley wrote: On Wed, 10 Dec 2003, Stas Bekman wrote: But doesn't unsetting the C-L header cause the C-L filter to automatically attempt to generate a new C-L value? Not unless the C-L filter sees the entire response in the first brigade passed through it. It used to buffer the entire response in order to compute the C-L, but I changed that last year. Brian
Re: Regarding worker MPM and queue_push/pop
On Dec 4, 2003, at 9:18 AM, MATHIHALLI,MADHUSUDAN (HP-Cupertino,ex1) wrote: -Original Message- From: Cliff Woolley [mailto:[EMAIL PROTECTED] [SNIP] On Wed, 3 Dec 2003, MATHIHALLI,MADHUSUDAN (HP-Cupertino,ex1) wrote: instead of having the worker threads compete for the incoming connections (using ap_queue_pop .. and hence mutex_lock), assign the connection to the next free thread on a round-robin basis - if I'm not wrong, zeus does something similar. Isn't that what the leader-follower MPM does? Ah.. I didn't see the leader-follower stuff (I'm probably too focussed on worker MPM). Thanks for pointing it out ! Has there been any analysis done on the leader-follower MPM already ? I did some comparisons between leader-follower and worker a long time ago. At that time, worker was slightly faster than leader-follower on multiprocessor Solaris servers, but the code has changed since then, and the performance characteristics of both are likely to vary among operating systems. If you have the time to test it, it wouldn't hurt to compare leader/follower to worker in your current benchmark environment. Brian
Re: Finding da fun.
Ben Hyde wrote: In this discussion people have been making two kinds of lists. The list of hypothetical reasons why activity seems to be petering out. The list of possible cures for those hypothetical ills. We need a third list. A list of fun hacks that 2.0 enables. It would be good to spend more effort on what the compelling value in the 2.0 family is and making it more accessible for people. Doc that lowers the barrier to entry. Success stories that create a sense of safety. Fundamentally, the filter mechanism in 2.0 makes it easy to mix and match different ways of obtaining content with different ways of delivering content. Need to run mod_include on the output of your servlet engine? In 2.0, you can do so by editing a config file. With most other webservers, 1.3 included, you have to rewrite the appserver connector. Need to compress or encrpyt the output of *any* handler? In 2.0, you can do so by loading the right filter and editing a config file. Need to use special logic for reading files from a document root due to NFS race conditions? In 2.0, just add a new handler and you can seamlessly stream the output through whatever stack of output filters you're already using. As for what's *wrong* with 2.0 (and 2.1), at the top of my list is scalability. We probably need to move to a multiple-connection-per-thread architecture for lower memory usage and more predictable response times under high loads. Historically, this has been one of the most-discussed changes proposed for 2.1; the only limitation is finding time to work on it. :-) Brian
Re: [PATCH] event driven read
On Tue, 2002-10-15 at 10:31, Bill Stoddard wrote: Something I've been hacking on (in the pejorative sense of the word 'hack'. Look at the patch and you will see what I mean :-). This should apply and serve pages on Linux, though the event_loop is clearly broken as it does not timeout keep-alive connections and will hang on the apr_poll() (and hang the server) if a client leaves a keep-alive connection active but does not send anything on it. Scoreboard is broken, code structure is poor, yadda yadda. I plan to reimplement some of this more cleanly but no idea when I'll get around to it. Key points: 1. routines to read requests must be able to handl getting APR_EAGAIN (or APR_EWOULDBLOCK): 2. ap_process_http_connection reimplemented to be state driven 3. event loop in worker_mpm to wait for pending i/o Revisiting this topic from a looong time ago... Did you ever get a chance to do any further work on this design? IMHO, for 2.1/2.2 we should at least incorporate the changes to allow data to be pushed rather than pulled through the input filter chain. That will set the foundation for an event-driven MPM, even if we're still using a blocking, thread-per-connection worker design in the first 2.2 release. Brian
Re: mod_include: YABU (yet another bug uncovered)
This appears to be due to the code at line ~2961 of mod_include.c that tries to delete buckets from *bb until it hits the bucket where the next tag starts. In this test case, the bucket where the next tag starts isn't in *bb at all; it's in ctx-ssi_tag_brigade. I'm evaluating a couple of potential fixes; if I work out something that looks correct, I'll post a patch. Brian On Thu, 2003-08-07 at 13:09, Ron Park wrote: Hi Again, Well, we've come across another mod_include/cross brigade bug for which we currently don't yet have a patch (should have one soon but thought I'd give others a crack at it). :D Here are two simple snippets, one which works, one which causes a core dump even with the latest patch of mod_include: !--#if expr=$FALSE --True!^P--#set var=v value=t --Set!--#else --False!--#set var=v value=t --Set!--#endif --Done !--#if expr=$FALSE --T!^P--#set var=v value=t --Set!--#else --False!--#set var=v value=t --Set!--#endif --Done (The Cntl-P is to cause mod_bucketeer to fire off: need the following setting in your conf file to get this to work: AddOutputFilter BUCKETEER;INCLUDES .html Put each snippet in a file in your docroot, say, true.html and t.html and give them a try. I'm pretty sure it's related to the code around line 491 of mod_include.c (at least that's the part that works differently for each one) but I haven't figured out how it's wrong. Thanks, Ron
Re: Little question on apache internals
On Wed, 2003-06-18 at 14:22, Sebastian Abt wrote: Hi all, I'm currently writing an Apache module (Apache 1.3.27) for virtual masshosting based on a SQL backend (currently MySQL later on PostgreSQL). Therefore I'm searching for information about the differences between requests (request_rec) and servers (server_rec). Can anybody tell me the lifetime of a request or a server record? As far as I understand the used terminology, a request record handles all requests made by a user when navigating to a specific URL. A request_rec lasts for the lifetime of a single HTTP request. So if navigating to a URL involves retrieving an HTML page and four images, there will be five different request_recs--even if a single keepalive connection is used. Brian
Re: calling ap_get_brigade with nonempty brigade
On Sat, 2003-06-14 at 22:47, Joe Schaefer wrote: Brian Pane [EMAIL PROTECTED] writes: Committed, thanks. Brian On Sun, 2003-06-08 at 22:46, Joe Schaefer wrote: [...] diff -u -r1.76 util_filter.h --- include/util_filter.h 3 Feb 2003 17:52:54 - 1.76 +++ include/util_filter.h 9 Jun 2003 05:41:54 - @@ -1,4 +1,4 @@ -/* +//* * The Apache Software License, Version 1.1 * * Copyright (c) 2000-2003 The Apache Software Foundation. All rights Hopefully not this part- I don't know how that additional slash got there. Don't worry, I caught that one :-) Brian
Re: Worker hungs ..
It depends on your operating system. If you're running Solaris, try /usr/proc/bin/pstack pid-of-the-hung process to get a stack trace of every thread. On Linux, you may be able to get some info with gdb (how well this works depends on kernel, libc, and gdb versions). For other OSs, hopefully others can join in with their debugging advice. If you can get a good stack trace, please post it to this list. Thanks, Brian On Sat, 2003-06-14 at 23:27, Pablo Yaggi wrote: Hi, I'm running apache2 with the worker module, I start the server and after some time it is up and running, incoming request hungs forever, I mean if I contact the server with a browser it stablish some kind of connection and it hung on waiting for replay. When it happens I can stop the server in a safe way I have to killall -9 http2. The extra thing I have in my conf is a module I made that changes the userid and groupid based on tables, the module hooks the translate_hook and the it switches from root to a less privileged id, after that of course I can´t be root again. Could my module be the problem ? How can I realize where is worker hunging ? Pablo
Re: calling ap_get_brigade with nonempty brigade
Committed, thanks. Brian On Sun, 2003-06-08 at 22:46, Joe Schaefer wrote: Justin Erenkrantz [EMAIL PROTECTED] writes: [...] Doc patches welcomed to make it more explicit. -- justin Index: include/util_filter.h === RCS file: /home/cvspublic/httpd-2.0/include/util_filter.h,v retrieving revision 1.76 diff -u -r1.76 util_filter.h --- include/util_filter.h 3 Feb 2003 17:52:54 - 1.76 +++ include/util_filter.h 9 Jun 2003 05:41:54 - @@ -1,4 +1,4 @@ -/* +//* * The Apache Software License, Version 1.1 * * Copyright (c) 2000-2003 The Apache Software Foundation. All rights @@ -298,7 +298,8 @@ * filter doesn't read from the network, then ::AP_NOBODY_READ is returned. * The bucket brigade will be empty when there is nothing left to get. * @param filter The next filter in the chain - * @param bucket The current bucket brigade + * @param bucket The current bucket brigade. The original brigade passed + * to ap_get_brigade() must be empty. * @param mode The way in which the data should be read * @param block How the operations should be performed * ::APR_BLOCK_READ, ::APR_NONBLOCK_READ
Re: Apache 2.0.45 -alpha tarball candidates for testing
On Mon, 2003-03-31 at 00:38, Justin Erenkrantz wrote: --On Monday, March 31, 2003 1:56 AM -0600 William A. Rowe, Jr. [EMAIL PROTECTED] wrote: Please test this 2.0.45-alpha candidate for general release within the next 24 hours. Once the dev@ list receives enough positive feedback, we will release the files to general availability, moving them to their Builds, passes httpd-test (modulo my recent commit to httpd-test), and serves pages on Solaris 9 (with .44 modules). +1. -- justin Tested successfully with worker MPM on Linux. +1 Brian
