[patch] adding request structure access to mod_header
We were dancing around trying to get RequestHeader (mod_header) to add
the authenticated user to a proxy request. The approaches we found
involved tricking other modules into calling ap_add_common_vars to
apr_table_addn(r-subprocess_env, REMOTE_USER, r-user);
I decided to simplify by adding %{}r directives à la
RequestHeader set x-webobjects-remote-user %{user}r
which interrogate the request structure directly. I re-used
the request structure names 'cause, well, why not?
Please try to reply to e...@w3.org. eric...@gmail.com is a
temporary hack as ezmlm seems to subscribe the address in the
Return-Path: instead of the From: (grr!).
--
-ericP
--- modules/metadata/mod_headers.c.orig 2010-07-17 15:57:08.0 +
+++ modules/metadata/mod_headers.c 2010-07-17 15:58:36.0 +
@@ -215,6 +215,33 @@
}
}
+#define mod_header_REQUEST(VAR) \
+if (!strcmp(a, #VAR)) \
+return r-VAR ? r-VAR : (null)
+#define mod_header_SERVER(VAR) \
+if (!strcmp(a, #VAR)) \
+return r-server r-server-VAR ? r-server-VAR : (null)
+static const char *header_request_attribute_var(request_rec *r, char *a)
+{
+mod_header_REQUEST(the_request);
+mod_header_REQUEST(protocol);
+mod_header_REQUEST(hostname);
+mod_header_REQUEST(method);
+mod_header_REQUEST(user);
+mod_header_REQUEST(unparsed_uri);
+mod_header_REQUEST(uri);
+mod_header_REQUEST(filename);
+mod_header_REQUEST(canonical_filename);
+mod_header_REQUEST(path_info);
+mod_header_SERVER(defn_name);
+mod_header_SERVER(server_admin);
+mod_header_SERVER(server_hostname);
+ap_log_rerror(APLOG_MARK, APLOG_ERR, 0, r, unknown RequestHeader variable %%{%s}r, a);
+return (null);
+}
+#undef mod_header_SERVER
+#undef mod_header_REQUEST
+
/*
* Config routines
*/
@@ -842,6 +869,7 @@
register_format_tag_handler(t, (const void *)header_request_time);
register_format_tag_handler(e, (const void *)header_request_env_var);
register_format_tag_handler(s, (const void *)header_request_ssl_var);
+register_format_tag_handler(r, (const void *)header_request_attribute_var);
return OK;
}
Re: Apache 2.0 mod_cache
On Wed, Jul 23, 2003 at 03:52:43PM -0700, James B Robinson wrote: Yo, Eric Imagine my surprise while perusing the changelogs for Apache to run across: *) mod_disk_cache works much better. This module should still be considered experimental. [Eric Prud'hommeaux] I've just started trying to get either squid or apache to do mem caching on a half dozen 4G linux boxes and I'm being a bit frustrated by Apache's lack of reporting what is up in the cache. You know much about mod_cache, mod_mem_cache or know of people who do? I was working on some packages that interact with the Vary header and found that disk_cache wasn't paying correct attention to it anyways. I hacked (at) them a bit and left them in a state where they could do relatively simple caching operations. There should be no false hits but there are opportunities for false misses (entities that could have been served from cache but, do to proxy naivete, weren't). The miss scenario is as follows: -CACHE MISS BUG SCENARIO- C1: GET path1 HTTP/1.1 Accept: text/html;q=0.5,application/soap+xml;q=1.0 Accept-Language: fr;q=0.8, en;q=0.7 Accept-Charset: iso-8859-5, unicode-1-1;q=0.8 Accept-Encoding: gzip;q=1.0, identity; q=0.5, *;q=0 Foo: bar proxy passes request to document server (or upstream proxy) and gets back S1: 200 OK Vary: Accept,Accept-Language Expires: Wed, 31 Dec 2003 16:00:00 GMT A: b ...data... and dutifully records the entity along with all of the headers in C1 that were listed in S1.Vary. It stores them in a spot on the disk computed by the hash of path1. hash(path1): Accept: text/html;q=0.5,application/soap+xml;q=1.0 Accept-Language: fr;q=0.8, en;q=0.7 ...S1 data... A subsequent request comes in for path1. Lucky case: has the same headers and the proxy can match them against what it was written at hash(path1). It may have different charset and encoding as they were not listed in the Vary header. Another request comes in with a different Accept-Language header: C2: GET path1 HTTP/1.1 Accept: text/html;q=0.5,application/soap+xml;q=1.0 Accept-Language: esparanto,iso-latin-pig A false hit would be if the the proxy said why I've got one of those and gave back the cached entity. I think I made sure that won't happen. But, I believe the proxy will replace the previously cached entity by what comes back from S2 (upstream response to C2). hash(path1): Accept: text/html;q=0.5,application/soap+xml;q=1.0 Accept-Language: esparanto,iso-latin-pig ...S2 data... -APPRAISAL OF CURRENT CODE- The false miss comes when another request like C1 comes in. The proxy no longer maintains the response S1 so it gets a cache miss and sends a request upstream. The resulting inefficiency can be estimated by observing the traffic coming through and seeing if the clients are sending requests that vary by something listed in the responses Vary header, and that they are doing this more quickly than the entity would expire of natural causes. I believe that the current implementation is well worth its weight in CPU time and maintenance. On the maintenance front, I don't beleive mod_disk_cache cleans up after itself, but a simple find of files with an access time older than some interval will give you a nice least- recently-used algorithm. Or you can give it its own filesystem and let it bump its head and clean up whenever you feel like it. The CPU involved in a cache miss is pretty minimal, a couple entries into a module, computing a hash, and a file open failure. The CPU involved in a cache hit would have to be cracking large keys or searching for aliens before it would be comparable with the time to have sent the request to a distant server. So, we have a working system, but I think it could be improved easily: -PROPOSED FIX- The inefficiency comes from storing the cache entry at a hash calculated only by the request path. If the varied headers were added to that hash, we would have a place to store all the variations of the entity. But, we'd have to be clairvoyant to know which of the request headers that came in would be needed to calculate the hash. To find this, I believe hash(path) needs to contain a list of the Vary headers for the server response(s). ie, after the two request above, the proxy would have -PROPOSAL P1- hash(path1): Accept Accept-Language hash(path1 . Accept: ... . A-L: fr;q=0.8, en;q=0.7): Accept: text/html;q=0.5,application/soap+xml;q=1.0 Accept-Language: fr;q=0.8, en;q=0.7 ...S1 data... hash(path1 . Accept: ... . A-L: esparanto,iso-latin-pig): Accept: text/html;q=0.5,application/soap+xml;q=1.0 Accept-Language: esparanto,iso-latin-pig ...S2 data... -PROPOSAL P1a- Alternatively, hash(path1) could compute a directory name. The Vary list could be stored in hash(path1)/vary and the other documents could be stored in entries like hash(path1)/hash(Accept: ... . A-L: esparanto,iso-latin-pig). It should be easy to ensure
.foo files in the same dir must have the same content-type and charset
In general, AddType/AddCharset directives provide general rules for
file extensions. mod_negotiation.c:handle_map_file provides a way to
do negotiation based on rules supplied in file.var rather than the
AddType/AddCharset rules for *.html.
for the purposes of content negotiation.
The problem is that these Content-Type/Language parameters are used for the negotiation
static int handle_map_file(request_rec *r)
{
negotiation_state *neg = parse_accept_headers(r);
var_rec *best;
int res;
char *udir;
// read .htaccess:
//AddType text/html;charset=iso-8859-1 html htm
// override with foo.var:
//URI: foo.html
//Content-Type: application/xhtml+xml; charset=UTF-8; qs=0.99
//Content-Language: en
if ((res = read_type_map(neg, r))) {
return res;
}
// find the characteristics of the best choice:
res = do_negotiation(r, neg, best, 0);
if (res != 0) return res;
if (r-path_info *r-path_info) {
r-uri[ap_find_path_info(r-uri, r-path_info)] = '\0';
}
udir = ap_make_dirstr_parent(r-pool, r-uri);
udir = ap_escape_uri(r-pool, udir);
// throw away everything except the file_name of the best choice:
ap_internal_redirect(ap_pstrcat(r-pool, udir, best-file_name,
r-path_info, NULL), r);
return OK;
}
It seems like making headers from best and stuffing them into r's
headers_out would be a good way to solve this. ap_internal_redirect
copies the new headers from the r (via internal_internal_redirect) and
i THINK that headers won't be replaced if they're already set.
gdb'd apache 1.3.27
looked at 2.something-or-other
--
-eric
office: +1.617.258.5741 NE43-344, MIT, Cambridge, MA 02144 USA
cell: +1.857.222.5741
([EMAIL PROTECTED])
Feel free to forward this message to any list for any purpose other than
email address distribution.
Re: How can I use a data structure shared by all the httpd process ?
On Wed, Aug 21, 2002 at 08:06:41AM -0700, Ian Holsman wrote: Sébastien Bonnegent wrote: Hi, My purpose is sharing a data structure. For example with an array: - httpd creation, my_array is empty - there are 8 servers launched - server1 add an entry named (X) - server5 add an entry - server3 read an entry - server7 remove the (X) entry After this, my_array contains one line which is the server5 entry. you will need to create a shared memory area to put your structure in, and have some kind of locking mechanism to ensure you don't have any race conditions. have a look at apr-utils/misc/apr_rmm.c apr_queue.c for an idea on what it required. If you are playing with shmem on a linux box, you will probably need to manually free segments from time to time. # ipcs -m keyshmid owner perms bytes nattch status 0x 12345 www-data 60092164 0 dest will give you the list of share memory segments in use. The nattch field gives you the ref count. iprm shm 12345 will free up that memory segment so it can be re-attached. For the moment, I use a apr_array_header_t * my_array and I put it in httpd.h. My problem is : each server process access to a copy of the array, so they could not share data.. Example: server1 array contains 0 element server2 array contains 3 elements server3 But, I want : an unique array Example: server1 \ server2 -\ server3 --\ server4 -- contains X elements ... Please, could someone give me a solution ? sinad -- -eric ([EMAIL PROTECTED]) Feel free to forward this message to any list for any purpose other than email address distribution.
patches for cache_pqueue.h break cache_cache.c
This possibly follows an earlier message:
[EMAIL PROTECTED]
2002/08/14 00:07:44 commit of cache_pqueue.h changed typedefs for
callbacks from function declarations to function pointers. Following
this, cache_cache(_make_money_fast).c failed to compile. I could fix
either, but being strongly in favor of typedefs of function
definitions, I patched cache_pqueue.{h,c}.
Argument for function definitions typedefs:
You can use the typedef to declare your function.
someLib.h:
typedef int (adder)(int origValue);
void registerAddr(adder* addMe);
myCode.c
adder MyAdder;
int main () {registerAddr(MyAddr);}
int MyAdder (int origValue) {return origValue+7;}
If adder were a function pointer
typedef int (adder)(int origValue);
I wouldn't be able to use it in forward declarations/sanity checks like
adder MyAdder;
This mattered to me as I was testing some disk caching proxy patches.
These are included, but not tested. I should get to that in the next
couple of days.
--
-eric
([EMAIL PROTECTED])
Feel free to forward this message to any list for any purpose other than
email address distribution.
Index: httpd-2.0/modules/experimental/cache_pqueue.c
===
RCS file: /home/cvspublic/httpd-2.0/modules/experimental/cache_pqueue.c,v
retrieving revision 1.12
diff -u -r1.12 cache_pqueue.c
--- httpd-2.0/modules/experimental/cache_pqueue.c 14 Aug 2002 01:24:16 -
1.12
+++ httpd-2.0/modules/experimental/cache_pqueue.c 16 Aug 2002 13:19:28 -
@@ -81,9 +81,9 @@
apr_ssize_t size;
apr_ssize_t avail;
apr_ssize_t step;
-cache_pqueue_get_priority pri;
-cache_pqueue_getpos get;
-cache_pqueue_setpos set;
+cache_pqueue_get_priority* pri;
+cache_pqueue_getpos* get;
+cache_pqueue_setpos* set;
void **d;
};
Index: httpd-2.0/modules/experimental/cache_pqueue.h
===
RCS file: /home/cvspublic/httpd-2.0/modules/experimental/cache_pqueue.h,v
retrieving revision 1.4
diff -u -r1.4 cache_pqueue.h
--- httpd-2.0/modules/experimental/cache_pqueue.h 14 Aug 2002 00:07:44 -
1.4
+++ httpd-2.0/modules/experimental/cache_pqueue.h 16 Aug 2002 13:19:28 -
@@ -78,21 +78,21 @@
* @param a the element
* @return the score (the lower the score the longer it is kept int the queue)
*/
-typedef long (*cache_pqueue_set_priority)(long queue_clock, void *a);
-typedef long (*cache_pqueue_get_priority)(void *a);
+typedef long (cache_pqueue_set_priority)(long queue_clock, void *a);
+typedef long (cache_pqueue_get_priority)(void *a);
/** callback function to get a position of a element */
-typedef apr_ssize_t (*cache_pqueue_getpos)(void *a);
+typedef apr_ssize_t (cache_pqueue_getpos)(void *a);
/**
* callback function to set a position of a element
* @param a the element
* @param pos the position to set it to
*/
-typedef void (*cache_pqueue_setpos)(void *a, apr_ssize_t pos);
+typedef void (cache_pqueue_setpos)(void *a, apr_ssize_t pos);
/** debug callback function to print a entry */
-typedef void (*cache_pqueue_print_entry)(FILE *out, void *a);
+typedef void (cache_pqueue_print_entry)(FILE *out, void *a);
/**
* initialize the queue
Index: httpd-2.0/modules/experimental/cache_storage.c
===
RCS file: /home/cvspublic/httpd-2.0/modules/experimental/cache_storage.c,v
retrieving revision 1.25
diff -u -r1.25 cache_storage.c
--- httpd-2.0/modules/experimental/cache_storage.c 23 Jun 2002 06:10:00 -
1.25
+++ httpd-2.0/modules/experimental/cache_storage.c 1 Aug 2002 06:38:22 -
@@ -154,6 +154,16 @@
return 1;
}
+static apr_status_t _failCache (request_rec *r, cache_request_rec *cache) {
+/* headers do not match, so Vary failed */
+ap_log_error(APLOG_MARK, APLOG_INFO, APR_SUCCESS, r-server,
+cache_select_url(): Vary header mismatch - Cached document cannot be
+used. \n);
+apr_table_clear(r-headers_out);
+r-status_line = NULL;
+cache-handle = NULL;
+return DECLINED;
+}
+
/*
* select a specific URL entity in the cache
*
@@ -209,7 +219,7 @@
*
* RFC2616 13.6 and 14.44 describe the Vary mechanism.
*/
-vary = apr_pstrdup(r-pool, apr_table_get(r-headers_out, Vary));
+vary = apr_pstrdup(r-pool, apr_table_get(r-err_headers_out, Vary));
while (vary *vary) {
char *name = vary;
const char *h1, *h2;
@@ -222,12 +232,14 @@
++vary;
}
-/*
- * is this header in the request and the header in the cached
- * request identical? If not, we give up and do a straight get
- */
-h1 = apr_table_get(r-headers_in, name);
-h2 = apr_table_get(h-req_hdrs, name);
+
Re: patches for cache_pqueue.h break cache_cache.c
On Fri, Aug 16, 2002 at 09:55:16AM -0400, Eric Prud'hommeaux wrote:
This possibly follows an earlier message:
[EMAIL PROTECTED]
2002/08/14 00:07:44 commit of cache_pqueue.h changed typedefs for
callbacks from function declarations to function pointers. Following
this, cache_cache(_make_money_fast).c failed to compile. I could fix
either, but being strongly in favor of typedefs of function
definitions, I patched cache_pqueue.{h,c}.
Argument for function definitions typedefs:
You can use the typedef to declare your function.
someLib.h:
typedef int (adder)(int origValue);
void registerAddr(adder* addMe);
myCode.c
adder MyAdder;
int main () {registerAddr(MyAddr);}
int MyAdder (int origValue) {return origValue+7;}
If adder were a function pointer
typedef int (adder)(int origValue);
oops, make that:
typedef int (*adder)(int origValue);
I wouldn't be able to use it in forward declarations/sanity checks like
adder MyAdder;
This mattered to me as I was testing some disk caching proxy patches.
These are included, but not tested. I should get to that in the next
couple of days.
--
-eric
([EMAIL PROTECTED])
Feel free to forward this message to any list for any purpose other than
email address distribution.
Content-Description: cache_pqueue.{h,c} patches
Index: httpd-2.0/modules/experimental/cache_pqueue.c
===
RCS file: /home/cvspublic/httpd-2.0/modules/experimental/cache_pqueue.c,v
retrieving revision 1.12
diff -u -r1.12 cache_pqueue.c
--- httpd-2.0/modules/experimental/cache_pqueue.c 14 Aug 2002 01:24:16 -
1.12
+++ httpd-2.0/modules/experimental/cache_pqueue.c 16 Aug 2002 13:19:28 -
@@ -81,9 +81,9 @@
apr_ssize_t size;
apr_ssize_t avail;
apr_ssize_t step;
-cache_pqueue_get_priority pri;
-cache_pqueue_getpos get;
-cache_pqueue_setpos set;
+cache_pqueue_get_priority* pri;
+cache_pqueue_getpos* get;
+cache_pqueue_setpos* set;
void **d;
};
Index: httpd-2.0/modules/experimental/cache_pqueue.h
===
RCS file: /home/cvspublic/httpd-2.0/modules/experimental/cache_pqueue.h,v
retrieving revision 1.4
diff -u -r1.4 cache_pqueue.h
--- httpd-2.0/modules/experimental/cache_pqueue.h 14 Aug 2002 00:07:44 -
1.4
+++ httpd-2.0/modules/experimental/cache_pqueue.h 16 Aug 2002 13:19:28 -
@@ -78,21 +78,21 @@
* @param a the element
* @return the score (the lower the score the longer it is kept int the queue)
*/
-typedef long (*cache_pqueue_set_priority)(long queue_clock, void *a);
-typedef long (*cache_pqueue_get_priority)(void *a);
+typedef long (cache_pqueue_set_priority)(long queue_clock, void *a);
+typedef long (cache_pqueue_get_priority)(void *a);
/** callback function to get a position of a element */
-typedef apr_ssize_t (*cache_pqueue_getpos)(void *a);
+typedef apr_ssize_t (cache_pqueue_getpos)(void *a);
/**
* callback function to set a position of a element
* @param a the element
* @param pos the position to set it to
*/
-typedef void (*cache_pqueue_setpos)(void *a, apr_ssize_t pos);
+typedef void (cache_pqueue_setpos)(void *a, apr_ssize_t pos);
/** debug callback function to print a entry */
-typedef void (*cache_pqueue_print_entry)(FILE *out, void *a);
+typedef void (cache_pqueue_print_entry)(FILE *out, void *a);
/**
* initialize the queue
Content-Description: disk_cache patches (not tested)
Index: httpd-2.0/modules/experimental/cache_storage.c
===
RCS file: /home/cvspublic/httpd-2.0/modules/experimental/cache_storage.c,v
retrieving revision 1.25
diff -u -r1.25 cache_storage.c
--- httpd-2.0/modules/experimental/cache_storage.c23 Jun 2002 06:10:00 -
1.25
+++ httpd-2.0/modules/experimental/cache_storage.c1 Aug 2002 06:38:22 -
@@ -154,6 +154,16 @@
return 1;
}
+static apr_status_t _failCache (request_rec *r, cache_request_rec *cache) {
+/* headers do not match, so Vary failed */
+ap_log_error(APLOG_MARK, APLOG_INFO, APR_SUCCESS, r-server,
+ cache_select_url(): Vary header mismatch - Cached document cannot be
used. \n);
+apr_table_clear(r-headers_out);
+r-status_line = NULL;
+cache-handle = NULL;
+return DECLINED;
+}
+
/*
* select a specific URL entity in the cache
*
@@ -209,7 +219,7 @@
*
* RFC2616 13.6 and 14.44 describe the Vary mechanism.
*/
-vary = apr_pstrdup(r-pool, apr_table_get(r-headers_out, Vary));
+vary = apr_pstrdup(r-pool, apr_table_get(r-err_headers_out, Vary));
while (vary *vary) {
char *name = vary;
const char *h1, *h2;
@@ -222,12 +232,14 @@
++vary
disk caching patches now tested
This documents tests and provides a test harnes for patches given in Messge-ID: [EMAIL PROTECTED]. Using my good buddies patchPanel and xterm, I tested the disk caching with regards to observing the Vary header. The attached tarball is intended to be expanded in the directory containing httpd-2.0/. ALL EXCITED TO PLAY: If all goes oddly well, you should be able to make -f diskCache/Makefile and go take a leak. When you come back, you should have three xterms proxy server manual server manual client Anything you type into manual client will go to proxy server and be relayed to a newly running apache proxy server running on port 9003. The proxy server will relay its requests on to whatever is specified in the GET and HOST lines. The sample request sends the requests on to the manual server where you will have to provide the response. You can GET stuff from a real server, but this gives you a bit more control. SPECIFIC TESTS: You enter a forest of xterms, each with a different title. The one labeld connected 3 9004 should have connected 3 to 9004 in the top. Enter a '/' followed by the command cat diskCache/proxyTest-request.http 1 This will send a request to the proxy server which will relay it to the manual server. Now you have to play the role of manual server so you go to that window, enter a '/' and give the command cat diskCache/proxyTest-response.http 2 to give a response back to the proxy and client. The proxy should create the file [EMAIL PROTECTED] which holds the headers for the request and response. You can mark where you are in each patchPanel with /echo virgin cache The proxy server should have seen GET http://localhost:9005/doc1 HTTP/1.1 Host: localhost:9005 Header1: Value1 Header2: Value2 HTTP/1.1 200 OK Date: Fri, 16 Aug 2002 21:12:14 GMT Server: Apache/2.0.41-dev (Unix) Content-Location: doc1.xhtml Vary: Header1,Header2 Last-Modified: Mon, 05 Aug 2002 08:32:14 GMT ETag: 4301b8-102-c6d39f80;fb6d5700 Accept-Ranges: bytes Cache-Control: max-age=60 Expires: Fri, 16 Aug 2003 20:26:20 GMT Content-Type: text/plain Via: 1.1 127.0.0.1:9003 Content-Length: 14 line 1 line 2 Entering the same request in (quickly, before you lose your the proxy server's attention) should result in the same reply, without touching the manual server. Changing one of the vary fields: GET http://localhost:9005/doc1 HTTP/1.1 Host: localhost:9005 Header1: Value1 Header2: Value2b should result in the request going back to the manual server. Thus I declare the vary support a success. BORED NOW: After you get tired of screwing around with this stuff make -f diskCache/Makefile kill will kill off the proxy server and you can /quit in each of the patchPanel windows. TODO: -There are a few code paths with conditional caching that I haven't looked at. -It would be cool to retrieve the Vary headers for foo (by the current mechanism), compute a new hash from the request values of those headers, and look for a cached variant matching that request. This could be done only when the first cache fails the varies tests. Then it would only create work when would have had to fail anyways. -Add HTTP Extensions support to the cache validity calculation. I have patches for this, but I'll wait 'till I've got fewer outstanding patches (2 currently, typedefFuncs-20020816.patch and diskCache-20020816.patch). Despite these todos, I believe it is a good idea to integrate this patches as disk caching doesn't really work now so they at least improve the situation. -- -eric ([EMAIL PROTECTED]) Feel free to forward this message to any list for any purpose other than email address distribution.
Re: disk caching patches now tested
I have to screw at least one thing up per post. Here are the promised attachments... -- -eric ([EMAIL PROTECTED]) Feel free to forward this message to any list for any purpose other than email address distribution. diskCache-test-20020816.tar.gz Description: application/tar-gz
Re: [PATCH] making mod_disk_cache work like mod_mem_cache
:8081 Content-Type: text/plain; qs=0.4; charset=ISO-8859-1 Age: 98403064 Content-Length: 473 # # robots.txt for http://www.w3.org/ = stale cache not used -- simulated with forcing ap_cache_check_freshness to return 0. I've also tested the date parsing stuff by setting up a 10 second expiry and seeing it do the right thing. GET http://mr-pink.w3.org:8081/robots.txt HTTP/1.1 Host: mr-pink.w3.org:8081 Foo: bar HTTP/1.1 200 OK Date: Fri, 02 Aug 2002 06:55:53 GMT Server: Apache/1.3.26 (Unix) PHP/3.0.18 Last-Modified: Thu, 06 Jun 2002 09:44:28 GMT P3P: policyref=http://www.w3.org/2001/05/P3P/p3p.xml; Cache-Control: max-age=21600 Expires: Fri, 02 Aug 2002 12:54:18 GMT ETag: 3cff2efc Accept-Ranges: bytes Via: 1.1 mr-pink.w3.org:8081 Content-Type: text/plain; qs=0.4; charset=ISO-8859-1 Age: 98403064 Content-Length: 473 # # robots.txt for http://www.w3.org/ ... = end of samples Thanks, Cheers, -- -eric Eric Prud'hommeaux wrote: On Thu, Aug 01, 2002 at 08:52:49AM -0400, Bill Stoddard wrote: mod_mem_cache fomr HEAD should work. mod_disk_cache is still broken. Try a config like this: LoadModule cache_module modules/mod_cache.so IfModule mod_cache.c CacheOn On # CacheMaxExpire 2 # CacheDefaultExpire 1 LoadModule mem_cache_module modules/mod_mem_cache.so IfModule mod_mem_cache.c CacheEnable mem / MCacheSize 4096 MCacheMaxObjectCount 100 MCacheMinObjectSize 1 MCacheMaxObjectSize 100 # CacheDefaultExpire 1 /IfModule /IfModule Sweet -- that gave me a template to work with to make mod_disk_cache work. I made disk_cache store the request headers so the Vary: parser would have something to work with. I also through in the rest of the times needed to calculate the expiration. Also attached is the cache_storage.patch described earlier. Perhaps someone up on mem-cache will be able to see if I should push this functionality into mod_disk_cache. -Original Message- From: Eric Prud'hommeaux [mailto:[EMAIL PROTECTED]] Sent: Thursday, August 01, 2002 1:06 AM To: Apache HTTP server developers Subject: apache 2 disk-cache SEGV Has anybody seen --enable-disk-cache or --enable-mem-cache work under apache 2? In my scenario: --enable-maintainer-mode --with-mpm=prefork --enable-rewrite --enable-expires --enable-speling --disable-auth --enable-headers --enable-info --disable-userdir --enable-dav --enable-proxy --enable-proxy-connect --enable-proxy-ftp --enable-proxy-http --enable-file-cache --enable-cache --enable-disk-cache --enable-mem-cache --no-create --no-recursion --prefix=/usr/local/apache-2-clean the cache context's filename is copied to r-filename cache_url_handler (modules/experimental/mod_cache.c:192) cache_read_entity_headers (modules/experimental/mod_cache.c:192) cache_select_url which (modules/experimental/mod_cache.c:287) r-filename = apr_pstrdup(r-pool, info-filename ); before the content_set filter sets it. cache_in_filter (modules/experimental/mod_cache.c:732) info-filename = apr_pstrdup(r-pool, r-filename ); Can someone who has this working set breakpoints at cache_url_handler and cache_in_filter and let me know if they see the same behavior? The two are registered with ap_hook_quick_handler(cache_url_handler, NULL, NULL, APR_HOOK_FIRST); and ap_register_output_filter(CACHE_IN, cache_in_filter, NULL, AP_FTYPE_CONTENT_SET); CACHE_IN is added to the request several places in cache_url_handler (a bit too late) and in cache_conditional_filter, which is also added to the request in cache_url_handler. From this it appears that the info-filename will _always_ be copied to r-filename before it has been initialized. If you are replicating this problem, you will probably need to created the disk cache directory /usr/local/apache-2-clean/proxy (in this case). Otherwise, the cache is never written and so never read and so the code path never arises. Since this is a cache re-use issue, you'll have to GET something twice through the proxy in rapid enough succession that the cached copy doesn't go stale. For now, I just cheesed around it with +++ modules/experimental/mod_cache.c:287 +if (info-filename) r-filename = apr_pstrdup(r-pool, info-filename ); but never having seen it work, I don't know what the intended path is. -- -eric ([EMAIL PROTECTED]) Feel free to forward this message to any list for any purpose other than email address distribution. Index: httpd-2.0/modules/experimental/mod_disk_cache.c === RCS file: /home/cvspublic/httpd-2.0/modules/experimental/mod_disk_cache.c,v retrieving revision 1.36 diff -u -r1.36 mod_disk_cache.c --- httpd
[PATCH] making mod_disk_cache work like mod_mem_cache
On Thu, Aug 01, 2002 at 08:52:49AM -0400, Bill Stoddard wrote:
mod_mem_cache fomr HEAD should work. mod_disk_cache is still broken.
Try a config like this:
LoadModule cache_module modules/mod_cache.so
IfModule mod_cache.c
CacheOn On
# CacheMaxExpire 2
# CacheDefaultExpire 1
LoadModule mem_cache_module modules/mod_mem_cache.so
IfModule mod_mem_cache.c
CacheEnable mem /
MCacheSize 4096
MCacheMaxObjectCount 100
MCacheMinObjectSize 1
MCacheMaxObjectSize 100
# CacheDefaultExpire 1
/IfModule
/IfModule
Sweet -- that gave me a template to work with to make mod_disk_cache
work. I made disk_cache store the request headers so the Vary: parser
would have something to work with. I also through in the rest of the
times needed to calculate the expiration.
Also attached is the cache_storage.patch described earlier. Perhaps
someone up on mem-cache will be able to see if I should push this
functionality into mod_disk_cache.
-Original Message-
From: Eric Prud'hommeaux [mailto:[EMAIL PROTECTED]]
Sent: Thursday, August 01, 2002 1:06 AM
To: Apache HTTP server developers
Subject: apache 2 disk-cache SEGV
Has anybody seen --enable-disk-cache or --enable-mem-cache work under
apache 2? In my scenario:
--enable-maintainer-mode --with-mpm=prefork --enable-rewrite
--enable-expires --enable-speling --disable-auth --enable-headers
--enable-info --disable-userdir --enable-dav --enable-proxy
--enable-proxy-connect --enable-proxy-ftp --enable-proxy-http
--enable-file-cache --enable-cache --enable-disk-cache
--enable-mem-cache --no-create --no-recursion
--prefix=/usr/local/apache-2-clean
the cache context's filename is copied to r-filename
cache_url_handler (modules/experimental/mod_cache.c:192)
cache_read_entity_headers (modules/experimental/mod_cache.c:192)
cache_select_url which (modules/experimental/mod_cache.c:287)
r-filename = apr_pstrdup(r-pool, info-filename );
before the content_set filter sets it.
cache_in_filter (modules/experimental/mod_cache.c:732)
info-filename = apr_pstrdup(r-pool, r-filename );
Can someone who has this working set breakpoints at cache_url_handler
and cache_in_filter and let me know if they see the same behavior?
The two are registered with
ap_hook_quick_handler(cache_url_handler, NULL, NULL, APR_HOOK_FIRST);
and
ap_register_output_filter(CACHE_IN,
cache_in_filter,
NULL,
AP_FTYPE_CONTENT_SET);
CACHE_IN is added to the request several places in cache_url_handler (a
bit too late) and in cache_conditional_filter, which is also added to the
request in cache_url_handler. From this it appears that the info-filename
will _always_ be copied to r-filename before it has been initialized.
If you are replicating this problem, you will probably need to created
the disk cache directory /usr/local/apache-2-clean/proxy (in this case).
Otherwise, the cache is never written and so never read and so the code
path never arises.
Since this is a cache re-use issue, you'll have to GET something twice
through the proxy in rapid enough succession that the cached copy
doesn't go stale.
For now, I just cheesed around it with
+++ modules/experimental/mod_cache.c:287
+if (info-filename)
r-filename = apr_pstrdup(r-pool, info-filename );
but never having seen it work, I don't know what the intended path is.
--
-eric
([EMAIL PROTECTED])
Feel free to forward this message to any list for any purpose other than
email address distribution.
--
-eric
([EMAIL PROTECTED])
Feel free to forward this message to any list for any purpose other than
email address distribution.
Index: httpd-2.0/modules/experimental/mod_disk_cache.c
===
RCS file: /home/cvspublic/httpd-2.0/modules/experimental/mod_disk_cache.c,v
retrieving revision 1.36
diff -u -r1.36 mod_disk_cache.c
--- httpd-2.0/modules/experimental/mod_disk_cache.c 17 Jul 2002 14:52:36 -
1.36
+++ httpd-2.0/modules/experimental/mod_disk_cache.c 1 Aug 2002 21:50:11 -
@@ -237,7 +237,7 @@
if ((temp = strchr(urlbuff[0], '\n')) != NULL) /* trim off new line character */
*temp = '\0'; /* overlay it with the null terminator */
-if (!apr_date_checkmask(urlbuff,
)) {
+if (!apr_date_checkmask(urlbuff,
+ )) {
return APR_EGENERAL;
}
@@ -246,6 +246,10 @@
info-expire = ap_cache_hex2usec(urlbuff + offset);
offset += (sizeof(info-expire)*2) + 1;
dobj-version = ap_cache_hex2usec(urlbuff + offset);
+offset += (sizeof(info-expire)*2) + 1;
+info-request_time = ap_cache_hex2usec(urlbuff + offset);
+offset += (sizeof(info-expire)*2) + 1;
+info-response_time = ap_cache_hex2usec(urlbuff + offset);
/* check that we
apache 2 disk-cache SEGV
Has anybody seen --enable-disk-cache or --enable-mem-cache work under apache 2? In my scenario: --enable-maintainer-mode --with-mpm=prefork --enable-rewrite --enable-expires --enable-speling --disable-auth --enable-headers --enable-info --disable-userdir --enable-dav --enable-proxy --enable-proxy-connect --enable-proxy-ftp --enable-proxy-http --enable-file-cache --enable-cache --enable-disk-cache --enable-mem-cache --no-create --no-recursion --prefix=/usr/local/apache-2-clean the cache context's filename is copied to r-filename cache_url_handler (modules/experimental/mod_cache.c:192) cache_read_entity_headers (modules/experimental/mod_cache.c:192) cache_select_url which (modules/experimental/mod_cache.c:287) r-filename = apr_pstrdup(r-pool, info-filename ); before the content_set filter sets it. cache_in_filter (modules/experimental/mod_cache.c:732) info-filename = apr_pstrdup(r-pool, r-filename ); Can someone who has this working set breakpoints at cache_url_handler and cache_in_filter and let me know if they see the same behavior? The two are registered with ap_hook_quick_handler(cache_url_handler, NULL, NULL, APR_HOOK_FIRST); and ap_register_output_filter(CACHE_IN, cache_in_filter, NULL, AP_FTYPE_CONTENT_SET); CACHE_IN is added to the request several places in cache_url_handler (a bit too late) and in cache_conditional_filter, which is also added to the request in cache_url_handler. From this it appears that the info-filename will _always_ be copied to r-filename before it has been initialized. If you are replicating this problem, you will probably need to created the disk cache directory /usr/local/apache-2-clean/proxy (in this case). Otherwise, the cache is never written and so never read and so the code path never arises. Since this is a cache re-use issue, you'll have to GET something twice through the proxy in rapid enough succession that the cached copy doesn't go stale. For now, I just cheesed around it with +++ modules/experimental/mod_cache.c:287 +if (info-filename) r-filename = apr_pstrdup(r-pool, info-filename ); but never having seen it work, I don't know what the intended path is. -- -eric ([EMAIL PROTECTED]) Feel free to forward this message to any list for any purpose other than email address distribution.
Duplicate headers in apache 2 disk-cache
Resources served by proxy after consulting a stale cache have
duplicates of the non-specially handled headers. This is because
mod_disk_cache's read_headers reads the cached headers into the
err_headers_out and ap_http_header_filter merges them with the
ones read from a proxy call.
-Details-
In server/util_script.c:498 ap_scan_script_header_err_core takes a
table of headers that is has read, in my case, from a proxy cache
file, and merges them onto r-err_headers_out.
apr_table_overlap(r-err_headers_out, merge,
APR_OVERLAP_TABLES_MERGE);/* It doesn't seem that an error has arisen. */
During the process of reading in the headers, it has set several
fields in the request:
content_type, status, status_line, last_modified, cookies
and set some particular headers in the r-headers_out:
Location, Content-Length, Transfer-Encoding
In the case of disk caching, read_headers (mod_disk_cache:465) calls
ap_scan_script_header_err_core (via ap_scan_script_header_err) and
sets the Content-Type from the previously set r-content_type. Only
a few headers get set this way:
(gdb) dump_table r-headers_out
[0] 'Last-Modified'='Fri, 26 Jul 2002 22:55:45 GMT'
[1] 'Content-Type'='multipart/mixed; boundary=3a6c43a796c1d96d'
The rest are merged into err_headers_out:
(gdb) dump_table r-err_headers_out
[0] 'Date'='Sat, 27 Jul 2002 16:17:34 GMT'
[1] 'Server'='Apache/2.0.40-dev (Unix) DAV/2'
[2] 'Cache-Control'='max-age=21600'
[3] 'Expires'='Sat, 27 Jul 2002 22:17:34 GMT'
[4] 'ETag'='201-158f-ac96fe40'
[5] 'Accept-Ranges'='bytes'
[6] '00-Package-Language'='http://www.w3.org/1999/02/22-rdf-syntax-ns#'
[7] 'Vary'='00-Package-Language,Man'
[8] '00-Content-Type'='text/html; charset=iso-8859-1'
[9] '00-Content-Length'='5519'
[10] 'Man'='http://www.w3.org/2001/07/07-MetaPackage/;; ns=00'
[11] 'Via'='1.1 localhost:8003'
Since the query is not served from a cache, the proxy gets new headers
from the document server and stores them in r-headers_out.
ap_http_header_filter (modules/http/http_protocol.c:1522) merges
headers_out and err_headers_out onto a new headers_out:
if (!apr_is_empty_table(r-err_headers_out)) {
r-headers_out = apr_table_overlay(r-pool, r-err_headers_out,
r-headers_out);
}
This happens at
#0 ap_http_header_filter (f=0x81e1020, b=0x81e6000) at http_protocol.c:1519
#1 0x080d3e03 in ap_pass_brigade (next=0x81e1020, bb=0x81e6000) at util_filter.c:540
#2 0x080bdb93 in _httpExt_genExtensionHeaders (f=0x81e1dc0, b=0x81e6000) at
mod_http_ext.c:231
#3 0x080d3e03 in ap_pass_brigade (next=0x81e1dc0, bb=0x81e6000) at util_filter.c:540
#4 0x080d714d in ap_content_length_filter (f=0x81e1008, b=0x81e6000) at
protocol.c:1355
#5 0x080d3e03 in ap_pass_brigade (next=0x81e1008, bb=0x81e6000) at util_filter.c:540
#6 0x080872de in ap_byterange_filter (f=0x81e0ff0, bb=0x81e6000) at
http_protocol.c:2803
#7 0x080d3e03 in ap_pass_brigade (next=0x81e0ff0, bb=0x81e6000) at util_filter.c:540
#8 0x08063fa8 in cache_in_filter (f=0x81d3c50, in=0x81e6000) at mod_cache.c:744
#9 0x080d3e03 in ap_pass_brigade (next=0x81d3c50, bb=0x81e6000) at util_filter.c:540
#10 0x0808185f in ap_proxy_http_process_response (p=0x81c8100, r=0x81e0360,
p_conn=0x81e6050, origin=0x81e6248, backend=0x81c8658,
conf=0x811f230, bb=0x81e6000, server_portstr=0xb6a4 :8003) at
proxy_http.c:926
#11 0x08081ca9 in ap_proxy_http_handler (r=0x81e0360, conf=0x811f230, url=0x81e6140
/2001/10/22-HttpExt-apache/Overview.html,
proxyname=0x0, proxyport=0) at proxy_http.c:1088
#12 0x08077d54 in proxy_run_scheme_handler (r=0x81e0360, conf=0x811f230,
url=0x81d3dfe http://localhost:8003/2001/10/22-HttpExt-apache/Overview.html;,
proxyhost=0x0, proxyport=0) at mod_proxy.c:1121
#13 0x080767f5 in proxy_handler (r=0x81e0360) at mod_proxy.c:463
#14 0x080c4571 in ap_run_handler (r=0x81e0360) at config.c:193
#15 0x080c4dae in ap_invoke_handler (r=0x81e0360) at config.c:400
#16 0x080880bc in ap_process_request (r=0x81e0360) at http_request.c:257
#17 0x0808229a in ap_process_http_connection (c=0x81c8208) at http_core.c:293
#18 0x080d1081 in ap_run_process_connection (c=0x81c8208) at connection.c:85
#19 0x080d146c in ap_process_connection (c=0x81c8208, csd=0x81c8138) at
connection.c:207
#20 0x080c2d37 in child_main (child_num_arg=0) at prefork.c:696
#21 0x080c2e1d in make_child (s=0x811e4d0, slot=0) at prefork.c:736
#22 0x080c2f54 in startup_children (number_to_start=5) at prefork.c:808
#23 0x080c3376 in ap_mpm_run (_pconf=0x8119a30, plog=0x8163b58, s=0x811e4d0) at
prefork.c:1024
#24 0x080ca023 in main (argc=2, argv=0xbab4) at main.c:645
#25 0x4016714f in __libc_start_main () from /lib/libc.so.6
This leaves duplicate headers as the the err_headers_out was set by
reading an expired response from the disk and the headers_out come
from reading the proxied headers_in.
HTTP/1.1 200 OK
Date: Tue, 30 Jul 2002 03:58:24 GMT
Server: Apache/2.0.40-dev (Unix) DAV/2
Cache-Control: max-age=1
sub requests are all GETs
Can anybody explain why ap_set_sub_req_protocol does rnew-method = GET; rnew-method_number = M_GET; instead of rnew-method = r-method; rnew-method_number = r-method_number; ? The consequence is that functions like negotiation sub_req = ap_sub_req_lookup_file(dirent.name, r, NULL); check auth on the wrong method. You can check this by POSTing to foo and having a limit on POST for foo.php3 (as opposed to the whole directory). A quick way to check is to set a breakpoint in ap_set_sub_req_protocol and telnet localhost 80 POST /Overview HTTP/1.0 Content-Length: 5 abcd Any calls to the auth modules will have a method of GET despite the POST action they will eventually execute. All auth modules and the like could check for this: int method = r-main ? r-main-method_number : r-method_number; but it seems better to have the sub request default to the method of the request that inspired it. There may be some modules that may count on the default behavior, like mod_include, but I think they should specifically make the new method be a GET as they are not duplicating the parent request's behaviour. -- -eric ([EMAIL PROTECTED]) Feel free to forward this message to any list for any purpose other than email address distribution.
Re: sub requests are all GETs
On Wed, Sep 05, 2001 at 05:46:15AM -0700, Greg Stein wrote:
Take a look at ap_sub_req_method_uri. That might do the trick for you.
I don't think there is a similar one for files right now.
Thanks. I took a look at ap_sub_req_method_uri and am still whining:
ap_sub_req_method_uri takes a method string argument and returns a sub
req with that method. All functions that could be creating POST, PUT,
etc requests should perhaps use it like this:
rnew = ap_sub_req_method_uri(r-method, newUri, r, somefilter)
but they don't. They all call ap_sub_req_lookup_uri wich hard codes it:
return ap_sub_req_method_uri(GET, new_file, r, next_filter);
I haven't tested, but it seems like a number of the callers of
ap_sub_req_lookup_uri may be subject to the problem I'm addressing
in ap_sub_req_lookup_file.
The problem I'm trying to solve here is to make the ACLs on negotiated
files work out of the box. As it is, if the ACL is only on foo.php3, a
POST to foo will cause, for instance, mod_auth's check_user_access to
check the ACLs for POST on foo and later, after mod_negotiation does
its trick, check the ACLs for GET on foo.php3. If Joe user is allowed
to GET foo.php3 he can sneak around the ACLs by POSTing to foo.
I saw this problem back in 1.3 and figured I'd tackle it if it was
still around in 2.0. I think the solution proposed below will work
but I haven't checked mod_{dav,include,autoindex} to see that they
don't mean to create a sub req that truly is a GET and not whatever
the parent req was.
On Wed, Sep 05, 2001 at 08:17:15AM -0400, Eric Prud'hommeaux wrote:
Can anybody explain why ap_set_sub_req_protocol does
rnew-method = GET;
rnew-method_number = M_GET;
instead of
rnew-method = r-method;
rnew-method_number = r-method_number;
? The consequence is that functions like negotiation
sub_req = ap_sub_req_lookup_file(dirent.name, r, NULL);
check auth on the wrong method. You can check this by POSTing to
foo and having a limit on POST for foo.php3 (as opposed to the
whole directory). A quick way to check is to set a breakpoint in
ap_set_sub_req_protocol and
telnet localhost 80
POST /Overview HTTP/1.0
Content-Length: 5
abcd
Any calls to the auth modules will have a method of GET despite
the POST action they will eventually execute.
All auth modules and the like could check for this:
int method = r-main ? r-main-method_number : r-method_number;
but it seems better to have the sub request default to the method
of the request that inspired it. There may be some modules that
may count on the default behavior, like mod_include, but I think
they should specifically make the new method be a GET as they are
not duplicating the parent request's behaviour.
--
-eric
([EMAIL PROTECTED])
Feel free to forward this message to any list for any purpose other than
email address distribution.
--
Greg Stein, http://www.lyra.org/
--
-eric
([EMAIL PROTECTED])
Feel free to forward this message to any list for any purpose other than
email address distribution.
Re: cvs commit: httpd-2.0/modules/http http_protocol.c
On Sun, Aug 26, 2001 at 05:20:40PM -0700, Greg Stein wrote: On Sun, Aug 26, 2001 at 03:18:41PM -0700, Ryan Bloom wrote: On Sunday 26 August 2001 12:54, Doug MacEachern wrote: On Sun, 26 Aug 2001, Marc Slemko wrote: hang on, is this about keepalives or chunked encoding? both. the check always fails because ap_content_length_filter has set content length before ap_set_keepalive is called. the right fix would probably be to check http/1.1-oneness eariler and remove (or not add) the ap_content_length_filter if r-chunked. Can't do that. The content-length filter always computes the full content length, even if it isn't put in the response. That way, we can log it correctly. I'd prefer to log it as 0 or somesuch, rather than buffer the response just for the sake of logging. Not of a particular mind here, but it *does* seem expensive to buffer just for logging's sake. Thoughts? I'd like to see ap_read_request and ap_set_sub_req_protocol set r-clength to -1 or AP_CONTENT_LENGTH_UNSET and use that to distinguish an empty response from one that hasn't had its content length set. I'm screwing around with some packaging filters which end up computing the content length while they're at it. Unless I remove ap_content_length_filter from the filter chain, it re-walks the brigade. I'm a little leary of removing it by name (CONTENT-LENGTH) as it's not a constant and may change from version to version. I'd like to just leave it alone, set the clength, and count on it to if (f-r-clength != AP_CONTENT_LENGTH_UNSET) return ap_pass_brigade(f-next, b) Reasonable? Crazy? -- -eric ([EMAIL PROTECTED]) Feel free to forward this message to any list for any purpose other than email address distribution.
Re: appending to the content brigade
DER may be remaned or removed This special type ensures that the HTTP header filter ends up in the proper location in the filter chain. AP_FTYPE_TRANSCODE These filters implement transport encodings (e.g., chunking). AP_FTYPE_CONNECTION These filters will alter the content, but in ways that are more strongly associated with the connection. Examples are splitting * an HTTP connection into multiple requests and buffering HTTP * responses across multiple requests. It is important to note that these types of filters are not allowed in a sub-request. A sub-request's output can certainly be filtered by ::AP_FTYPE_CONTENT filters, but all of the "final processing" is determined by the main request. AP_FTYPE_NETWORK These filters don't alter the content. They are responsible for sending/receiving data to/from the client. Calling conventions and contracts Filters may be called repeatedly. Filters store context needed between calls in the ap_filter_t's ctx pointer. Eventually, each filter should call the downstream filters by calling ap_pass_brigade with ap_filter_t's next filter. Example: serving a file In the common paradigm, one or more AP_FTYPE_CONTENT filters are passed a brigade by the content generator. For example, from the default_handler which serves file requests: bb = apr_brigade_create(r->pool); e = apr_bucket_file_create(fd, 0, r->finfo.size); APR_BRIGADE_INSERT_HEAD(bb, e); e = apr_bucket_eos_create(); APR_BRIGADE_INSERT_TAIL(bb, e); return ap_pass_brigade(r->output_filters, bb); The next filters executed are those registered as AP_FTYPE_HTTP_HEADER. The ap_http_header_filter function generates a brigade with the response code and mime headers. This brigade is passed to TRANSCODE and CONNECTION filters: ... b2 = apr_brigade_create(r->pool); basic_http_header(r, b2, protocol); ... ap_pass_brigade(f->next, b2); ... ap_http_header_filter removes itself (why?) from the list of filters and calls the downstream filters again with the content brigade: ... ap_remove_output_filter(f); return ap_pass_brigade(f->next, b); Eric Prud'hommeaux, 24th August 2001 Apache HTTP Server Version 2.0 Title: Apache 2.0 Buckets and Bucket Brigades Apache HTTP Server Version 2.0 Buckets and Bucket Brigades Buckets provide uniform access to a variety of data sources. Bucket brigades are collections of buckets usually use to represent a block of data. For instance, a bucket brigade may contain buckets of headers allocated from a request pool followed by a bucket representing a local file. A filter can add data before or after this file block without having to copy the the data. apr_brigade_to_iovec makes it easy to pass this scatter-write paradigm through the sockets layer all the way to the ethernet and disk controllers. Table of contents Types of buckets Bucket virual functions Using buckets and bucket brigades Bucket type and function names Calling conventions and contracts Common idioms Brigade functions Brigade creation and destruction Adding data to brigades Brigade data consumption Misc Brigade macros Bucket list manipluation macros ... Types of buckets Apache provides a set of standard bucket types allowing data collection from a variety of sources: transient (const char *buf, apr_size_t nbyte) Represents a data allocated off the stack. When the setaside function is called, this data is copied on to the heap. heap (const char *buf, apr_size_t nbyte, int copy, apr_size_t *w) Represents a data allocated from the heap. pool (const char *buf, apr_size_t length, apr_pool_t *pool) Represents a data that was allocated from a pool. IF this bucket is still available when the pool is cleared, the data is copied on to the heap. file (apr_file_t *fd, apr_off_t offset, apr_size_t len) Represents a file on disk. mmap (apr_mmap_t *mm, apr_off_t start, apr_size_t length) This bucket represents an MMAP'ed file. socket (apr_socket_t *thissock) Represents a socket connection to another machine. pipe (apr_file_t *thispipe) Represents a pipe to another program. Marker buckets eos (void) Signifies that there will be no more data, ever. All filters MUST send all data to the next filter when they receive a bucket of this type. flush (void) Signifies that all data should be flushed to the next filter. The flush bucket should be sent with the other bucke
appending to the content brigade
I'm implementing a content filter which wraps the content in multipart
mime. I can generate a separator, my data, separator, original payload
and separator. The problem is that if I just concatonate the brigades,
I end up with an EOS before my final separator. This works (the final
bucket gets sent down the wire) but, if I understand the meaning of
EOS I won't get into heaven this way. I figured I could copy all but
the EOS from the original payload to the new brigade:
-APR_BRIGADE_CONCAT(bsend, bPayload);
+APR_BRIGADE_FOREACH(e, bPayload) {
+ if (!APR_BUCKET_IS_EOS(e))
+ APR_BRIGADE_INSERT_TAIL(bsend, e);
+}
#...
apr_brigade_destroy(bPayload);
e = apr_bucket_pool_create(boundary, boundryLength, r-pool);
APR_BRIGADE_INSERT_TAIL(bsend, e);
e = apr_bucket_eos_create();
APR_BRIGADE_INSERT_TAIL(bsend, e);
This corrupted bsend on the first insert. I hadn't even gotten as far
as destroying bPayload, which I expected to be a bad idea. The
byterange filter uses some function calls to copy pieces of the
brigade. Is this the sanctioned approach?
--
-eric
([EMAIL PROTECTED])
Feel free to forward this message to any list for any purpose other than
email address distribution.
Re: Letting Table store non-char data
On Wed, Aug 22, 2001 at 09:03:21PM -0700, Ian Holsman wrote: On Wed, 2001-08-22 at 20:58, Greg Stein wrote: On Wed, Aug 22, 2001 at 03:59:56PM -0700, Ian Holsman wrote: On Wed, 2001-08-22 at 15:41, Brian Pane wrote: Ian Holsman wrote: On Wed, 2001-08-22 at 14:12, Ryan Bloom wrote: We had binary tables, but we removed them, because that is what hash tables are for. agreed. but there are no hash tables on the request rec structure. Yes there is. apr_pool_userdata_set(..., r-pool) Thanks. never knew about this function. I am switching my code to use pool userdata but wonder if swapping the parms make make the calling convention more consistent with the other pool methods that take the pool as first parm. - apr_status_t apr_pool_userdata_get(void **, char *, apr_pool_t *); + apr_status_t apr_pool_userdata_get(apr_pool_t *, char *, void **); # this ptr attrvalue -- -eric ([EMAIL PROTECTED]) Feel free to forward this message to any list for any purpose other than email address distribution.
