Re: mod_include supporting POST subrequests
On Fri 20 Mar 2009, Graham Leggett wrote: Torsten Foertsch wrote: I need the include virtual directive to be able to issue POST requests. It should pass the request body to the subrequest. So I came up with the attached patch. It allows to write !--#include method=post virtual=... -- or !--#include method=inherit virtual=... -- [...] Something like this has already been added to trunk, take a look at the KEEP_BODY and KEPT_BODY filters in modules/filters/mod_request.c. I did and, frankly, it is not the solution I was looking for. One has to define a max. body size to be kept. The body is kept in RAM which can be a problem unless KeptBodySize is rather small. So I developed my patch further. It defers now the ap_discard_request_body call as much as possible. This gives output filters the chance to read the req body. If the client is expecting a 100 Continue message it is sent just before the first line of output. Is there a chance for the patch to make it into 2.3++? If yes I'll merge it with the KEPT_BODY stuff. Currently my httpd passes the current test framework with a few more patches that are not related to this one (see 2.2.11 mod_include thread) with one exception. Since the request body is read when output is potentially already on the wire a HTTP_REQUEST_ENTITY_TOO_LARGE error cannot be sent to the client if it sends the request body in chunked TE. The only sensible solution that I can think of would be to always send a 413 response if TE is chunked and a LimitRequestBody is active. On Fri 20 Mar 2009, Nick Kew wrote: Erm ... that's ringing alarm bells. The client, not the server, determines HTTP methods. Or are you talking about proxied subrequests here? I see it a bit different. Subrequests for included documents are made on behalf of the HTML programmer who wrote the frame. He decides to pass on the request body and he decides which method to use, IMHO. And yes, the problem comes from subrequests that are proxied to another server. Torsten -- Need professional mod_perl support? Just hire me: torsten.foert...@gmx.net --- modules/filters/mod_include.c.orig 2008-03-17 15:32:47.0 +0100 +++ modules/filters/mod_include.c 2009-03-25 14:49:14.0 +0100 @@ -1656,6 +1656,7 @@ apr_bucket_brigade *bb) { request_rec *r = f-r; +enum {METHOD_GET, METHOD_POST, METHOD_INHERIT} method; if (!ctx-argc) { ap_log_rerror(APLOG_MARK, @@ -1674,6 +1675,8 @@ return APR_SUCCESS; } +method=METHOD_GET; + while (1) { char *tag = NULL; char *tag_val = NULL; @@ -1686,6 +1689,29 @@ break; } +if (tag[0] == 'm' !strcmp(tag, method)) { +if ((tag_val[0] == 'g' || tag_val[0] == 'G') + !strcasecmp(tag_val, get)) { +method=METHOD_GET; +} +else if ((tag_val[0] == 'p' || tag_val[0] == 'P') + !strcasecmp(tag_val, post)) { +method=METHOD_POST; +} +else if ((tag_val[0] == 'i' || tag_val[0] == 'I') + !strcasecmp(tag_val, inherit)) { +method=METHOD_INHERIT; +} +else { +ap_log_rerror(APLOG_MARK, APLOG_ERR, 0, r, unknown value + \%s\ to parameter \method\ of tag + include in %s, tag_val, r-filename); +SSI_CREATE_ERROR_BUCKET(ctx, f, bb); +break; +} +continue; +} + if (strcmp(tag, virtual) strcmp(tag, file)) { ap_log_rerror(APLOG_MARK, APLOG_ERR, 0, r, unknown parameter \%s\ to tag include in %s, tag, r-filename); @@ -1712,7 +1738,15 @@ } } else { -rr = ap_sub_req_lookup_uri(parsed_string, r, f-next); +if (method == METHOD_GET +|| (method == METHOD_INHERIT strcmp(r-method, POST))) { +rr = ap_sub_req_lookup_uri(parsed_string, r, f-next); +} +else { /* POST */ +method = METHOD_POST; +apr_table_setn(r-notes, subreq-pass-request-body, 1); +rr = ap_sub_req_method_uri(POST, parsed_string, r, f-next); +} } if (!error_fmt rr-status != HTTP_OK) { @@ -1734,10 +1768,22 @@ ap_set_module_config(rr-request_config, include_module, r); } +/* XXX: would be good to check for EOS on rr-input_filters + * if method==POST and issue a warning if so. + */ + if (!error_fmt ap_run_sub_req(rr)) { error_fmt = unable to include \%s\ in parsed file %s; } +/* method=POST must be specified *before* *each* + * virtual=... + */ +if (method != METHOD_GET) { +method = METHOD_GET; +
Re: mod_include supporting POST subrequests
Torsten Foertsch wrote: I did and, frankly, it is not the solution I was looking for. One has to define a max. body size to be kept. The body is kept in RAM which can be a problem unless KeptBodySize is rather small. So I developed my patch further. It defers now the ap_discard_request_body call as much as possible. This gives output filters the chance to read the req body. If the client is expecting a 100 Continue message it is sent just before the first line of output. Is there a chance for the patch to make it into 2.3++? If yes I'll merge it with the KEPT_BODY stuff. Having two separate mechanisms to solve the same problem is not ideal. In addition, creating a solution that only works in one place (mod_include), is less ideal still. It should be relatively straightforward to amend the KEEP_BODY and KEPT_BODY filters so that, by default, the first attempt to read the body is passed through, and the second and subsequent attempts to read the body return an empty brigade. This will give you the behaviour you are looking for, and it will work anywhere within the server, not just in mod_include. Regards, Graham -- smime.p7s Description: S/MIME Cryptographic Signature
Re: mod_include supporting POST subrequests
On Wed 01 Apr 2009, Graham Leggett wrote: Is there a chance for the patch to make it into 2.3++? If yes I'll merge it with the KEPT_BODY stuff. Having two separate mechanisms to solve the same problem is not ideal. In addition, creating a solution that only works in one place (mod_include), is less ideal still. It should be relatively straightforward to amend the KEEP_BODY and KEPT_BODY filters so that, by default, the first attempt to read the body is passed through, and the second and subsequent attempts to read the body return an empty brigade. This will give you the behaviour you are looking for, and it will work anywhere within the server, not just in mod_include. That is what I thought to do. If KeptBodySize is 0 the body is passed to the first (sub)request that is reading it. All subsequent subrequests will see an empty stream. If KeptBodySize is 0 the first (sub)request reads the whole body and the KEEP_BODY_FILTER saves as much as is configured. Subsequent subrequests are passed this kept body. I also thought of writing the body to temporary files if configured. So it's possible to preserve larger body without much headache about memory consumption. Plus, I think it would be a nice feature to be able to !--#include method=post body=a=b;c=d virtual=... -- and perhaps also to include an encoding=multipart/form-data, would it not? BTW, the current patch is not only for mod_include. It should work (although not tested) for other filters/handlers as well as long as the main request sets the subreq-pass-request-body note to prevent the header table to be overwritten for the subrequest. This more or less resembles what r-kept_body does in 2.3. Alternatively the caller could overwrite the header table after creating the subreq but before running it and set the original CL and TE headers. But to restate my question, can I take your reply as yes, go ahead, it would be nice to have that feature in apache httpd? Torsten -- Need professional mod_perl support? Just hire me: torsten.foert...@gmx.net
mod_include supporting POST subrequests
Hi, I need the include virtual directive to be able to issue POST requests. It should pass the request body to the subrequest. So I came up with the attached patch. It allows to write !--#include method=post virtual=... -- or !--#include method=inherit virtual=... -- I think the patch is right so far but I wouldn't mind if some more experienced eyes had a look. One problem remains. What if the HTML author uses that method=post thing twice? The first call has consumed the post body. The second will try to read something but the client has already sent all it can. Hence, I'd like to if (!eof_on_input()) { rr-headers_in = r-headers_in; } How can I determine if the request body for the current request has already been consumed? Is there a chance that a patch like this will be accepted for inclusion in apache? Torsten -- Need professional mod_perl support? Just hire me: torsten.foert...@gmx.net
Re: mod_include supporting POST subrequests
Torsten Foertsch wrote: I need the include virtual directive to be able to issue POST requests. It should pass the request body to the subrequest. So I came up with the attached patch. It allows to write !--#include method=post virtual=... -- or !--#include method=inherit virtual=... -- I think the patch is right so far but I wouldn't mind if some more experienced eyes had a look. One problem remains. What if the HTML author uses that method=post thing twice? The first call has consumed the post body. The second will try to read something but the client has already sent all it can. Hence, I'd like to if (!eof_on_input()) { rr-headers_in = r-headers_in; } How can I determine if the request body for the current request has already been consumed? Is there a chance that a patch like this will be accepted for inclusion in apache? Something like this has already been added to trunk, take a look at the KEEP_BODY and KEPT_BODY filters in modules/filters/mod_request.c. To switch it on, define the KeptBodySize directive: http://httpd.apache.org/docs/trunk/mod/mod_request.html#keptbodysize When the body is kept, the request method is maintained across mod_include, so a POST to foo.shtml will stay a POST through !--#include virtual=bar.cgi--. You can nest includes as deep as you like, the body is only set aside once. Regards, Graham -- smime.p7s Description: S/MIME Cryptographic Signature
Re: mod_include supporting POST subrequests
On Fri, 20 Mar 2009 20:17:08 +0100 Torsten Foertsch torsten.foert...@gmx.net wrote: I need the include virtual directive to be able to issue POST requests. It should pass the request body to the subrequest. So I came up with the attached patch. Did you mean to attach something? It allows to write !--#include method=post virtual=... -- or !--#include method=inherit virtual=... -- Erm ... that's ringing alarm bells. The client, not the server, determines HTTP methods. Or are you talking about proxied subrequests here? One problem remains. What if the HTML author uses that method=post thing twice? The first call has consumed the post body. The second will try to read something but the client has already sent all it can. You'd want something like mod_request or mod_form to parse the data and make them available to other modules in a convenient form. -- Nick Kew Application Development with Apache - the Apache Modules Book http://www.apachetutor.org/