Re: mod-cache-requestor plan
On 7/15/05, Colm MacCarthaigh <[EMAIL PROTECTED]> wrote: > On Fri, Jul 15, 2005 at 01:23:29AM -0500, Parin Shah wrote: > > - we need to maintain a counter for url in this case which would > > decide the priority of the url. But mainting this counter should be a > > low overhead operation, I believe. > > Is a counter strictly speaking the right approach? Why not a time of > last access? > > I havn't run a statistical analysis but based on my logs the likelyhood > of a url being accessed is very highly correlated to how recently it has > been accessed before. A truly popular page will always have been > accessed recently, a page that is becoming popular (and therefore very > likely to get future hits) will have been accessed recently and a page > who's popularity is rapidly diminishing will not have been accessed > recently. > Last Access Time is definetaly better solution when compared to counter mechanism. Would like to know other ppl's opinion too. Thanks, Parin.
Re: 2.0.55 Release
On Fri, 15 Jul 2005, Sascha Kersken wrote: > Hi Nick, > > > Are you aware that 2.1.x has builtin SQL support, through the apr_dbd > > framework? There's a MySQL driver for it, although it's not bundled > > with Apache because it's GPL (for MySQL compatibility), and that's > > not compatible with distribution from the ASF. > > I read about it on this list and find it very interesting, but > unfortunately, I did not have the chance to try it out yet - my publisher > would kill me for exceeding the deadline any longer ;-). > Although I think it will not alter the way PHP/MySQL applications (the > book's main topic) work dramatically right now, I still think it's a > fascinating new feature and that some day soon, I should write a feature > article about it for the book's companion web site. So thank you for your > hint. Another week could be interesting. If any sufficiently senior and/or listened-to MySQL people are seen at ApacheCon, we might make some progress on the licensing issue:-) -- Nick Kew
Re: [patch 1.3] The http_protocol.c C-L + T-E patch
Ok, I'm a dork - those apr_'s are ap_'s in 1.3. My bad - sorry.
Still a wise choice, grab it unmauled by email from
http://people.apache.org/~wrowe/httpd-1.3-proto-cl-te.patch
At 03:26 PM 7/15/2005, William A. Rowe, Jr. wrote:
>folks, the same patch Paul/Joe worked out for 2.1, then 2.0,
>should still probably fall on 1.3 even though proxy is not
>affected. Other modules surely could be hit.
>
>Votes/Comments? I think this is it for getting 1.3.34 out.
>
>Index: src/main/http_protocol.c
>===
>--- src/main/http_protocol.c(revision 209835)
>+++ src/main/http_protocol.c(working copy)
>@@ -1214,6 +1214,14 @@
> ap_log_transaction(r);
> return r;
> }
>+if (ap_table_get(r->headers_in, "Transfer-Encoding")
>+&& ap_table_get(r->headers_in, "Content-Length")) {
>+/* 2616 section 4.4, point 3: "if both Transfer-Encoding
>+ * and Content-Length are received, the latter MUST be
>+ * ignored"; so unset it here to prevent any confusion
>+ * later. */
>+ap_table_unset(r->headers_in, "Content-Length");
>+}
> }
> else {
> ap_kill_timeout(r);
[patch 1.3] The http_protocol.c C-L + T-E patch
folks, the same patch Paul/Joe worked out for 2.1, then 2.0,
should still probably fall on 1.3 even though proxy is not
affected. Other modules surely could be hit.
Votes/Comments? I think this is it for getting 1.3.34 out.
Index: src/main/http_protocol.c
===
--- src/main/http_protocol.c(revision 209835)
+++ src/main/http_protocol.c(working copy)
@@ -1214,6 +1214,14 @@
ap_log_transaction(r);
return r;
}
+if (apr_table_get(r->headers_in, "Transfer-Encoding")
+&& apr_table_get(r->headers_in, "Content-Length")) {
+/* 2616 section 4.4, point 3: "if both Transfer-Encoding
+ * and Content-Length are received, the latter MUST be
+ * ignored"; so unset it here to prevent any confusion
+ * later. */
+apr_table_unset(r->headers_in, "Content-Length");
+}
}
else {
ap_kill_timeout(r);
Re: [patch 2.0] http body request/response/trace conformance
At 03:03 PM 7/14/2005, William A. Rowe, Jr. wrote:
>To simplify - Jeff Joe and I reviewed two of the patches, and they
>are committed. Two patches are available for comment;
changelog;
*) Added TraceEnable [on|off|extended] per-server directive to alter
the behavior of the TRACE method. This addresses a flaw in proxy
conformance to RFC 2616 - previously the proxy server would accept
a TRACE request body although the RFC prohibited it. The default
remains 'TraceEnable on'. [William Rowe]
>http://people.apache.org/~wrowe/httpd-2.0-trace.patch
and changelog;
*) SECURITY: CAN-2005-2088
proxy: Correctly handle the Transfer-Encoding and Content-Length
headers. Discard the request Content-Length whenever T-E: chunked
is used, always passing one of either C-L or T-E: chunked whenever
the request includes a request body. Resolves an entire class of
proxy HTTP Request Splitting/Spoofing attacks. [William Rowe]
The newest flavor based on my most recent commits from Roy and Jeff's
feedback is available at;
http://people.apache.org/~wrowe/httpd-2.0-proxy-request-2.patch
and 2.0 STATUS is updated accordingly. Votes/Comments please?
>Although proxy-request.patch will evolve as this discussion
>continues; Jeff caused me to look, again, at the code and
>recognize another edge case already committed to trunk
>(and also in the patch.) proxy-request.patch will ultimately
>mirror what we agree to on trunk.
>
>And FYI, revert r219061 (below) from 2.1 or 2.0 to see the
>continued misbehavior of proxy without the proxy-request.patch.
>
>Bill
>
>--- httpd/httpd/branches/2.0.x/server/protocol.c (original)
>+++ httpd/httpd/branches/2.0.x/server/protocol.c Thu Jul 14 09:51:55 2005
>@@ -885,6 +885,15 @@
> apr_brigade_destroy(tmp_bb);
> return r;
> }
>+
>+if (apr_table_get(r->headers_in, "Transfer-Encoding")
>+&& apr_table_get(r->headers_in, "Content-Length")) {
>+/* 2616 section 4.4, point 3: "if both Transfer-Encoding
>+ * and Content-Length are received, the latter MUST be
>+ * ignored"; so unset it here to prevent any confusion
>+ * later. */
>+apr_table_unset(r->headers_in, "Content-Length");
>+}
> }
> else {
> if (r->header_only) {
Re: 2.0.55 Release
Hi Bill, > As Nick implies, stay tuned to this channel. It's closing in. I'm watching "httpd_tv" all day anyway, so I'm sure I won't miss it ;-). Thanks. Sascha
Re: 2.0.55 Release
Hi Nick, > Are you aware that 2.1.x has builtin SQL support, through the apr_dbd > framework? There's a MySQL driver for it, although it's not bundled > with Apache because it's GPL (for MySQL compatibility), and that's > not compatible with distribution from the ASF. I read about it on this list and find it very interesting, but unfortunately, I did not have the chance to try it out yet - my publisher would kill me for exceeding the deadline any longer ;-). Although I think it will not alter the way PHP/MySQL applications (the book's main topic) work dramatically right now, I still think it's a fascinating new feature and that some day soon, I should write a feature article about it for the book's companion web site. So thank you for your hint. Regards Sascha
Re: [patch 2.0] http body request/response/trace conformance
At 06:14 AM 7/15/2005, Roy T. Fielding wrote:
>-1 This is a violation of HTTP:
>
>+if (apr_table_get(r->subprocess_env, "force-proxy-request-1.0")
>+ || ((r->proto_num < HTTP_VERSION(1,1))
>+ && !apr_table_get(r->subprocess_env,
>"force-proxy-request-1.1"))) {
>
>In all cases except when specifically configured otherwise,
>the proxy must send HTTP/1.1 if it supports responses in HTTP/1.1.
Thanks for the clarification, Roy!
Bill
Re: svn commit: r218978 - in /httpd/httpd/trunk: CHANGES modules/proxy/mod_proxy_http.c
At 06:34 AM 7/15/2005, Roy T. Fielding wrote: >>All of it, except for the preference to RB_STREAM_CHUNKED when, >>perhaps, we could be more sub-optimal, falling back on RB_SPOOL_CL. >> >>Many RB_STREAM_CL choices, before, were equally dangerous, and that >>C-L == length_read test in the stream_reqbody_chunked() was meant >>to exclude future abuse. > >We should be sending C-L if the brigade includes the entire message. >CL is always preferred for requests. The only time we should send >T-E on a proxied request is if we cannot buffer enough of the >received request to know how large it will be. ++1 and committed! The solution was blindingly obvious, try to fetch MAX_MEM_SPOOL bytes, and determine if we hit EOS before proceeding to elect -any- C-L or T-E method :) The logic works out like this once it exceeds MAX_MEM_SPOOL; was it T-E? Unless the user overrides by setting proxy-sendcl or by setting force-proxy-downgrade-1.0, keep using T-E. proxy-sendchunks causes proxy-sendcl to be ignored for this case. was it C-L? Only stream (for unfiltered requests) or spool as C-L, unless the user overrides with proxy-sendchunks (probably to specific applications or servers that they are certain will handle chunked request bodies effectively.) proxy-sendcl causes proxy-sendchunks to be ignored for this case. Question for folks familiar with the input stack; should we loop until MAX_MEM_SPOOL is hit, or is one call to the filter (as my patch has done) enough to get about MAX_MEM_SPOOL bytes? >> * No longer upgrade HTTP/1.0 requests to the origin server as >>HTTP/1.1 unless the force-proxy-request-1.1 envvar is set. >>[William Rowe] > >-1 (veto), since it is a clear violation of > http://www.ietf.org/rfc/rfc2145.txt >and the intent of HTTP versions. Thanks! Reverted. Bill
Re: 2.0.55 Release
At 12:09 PM 7/15/2005, Nick Kew wrote: >On Fri, 15 Jul 2005, Sascha Kersken wrote: > >> I am currently finishing the companion CD-ROM for my Beginners' Guide To >> MySQL (for O'Reilly Germany). So I am wondering whether the announced >> release 2.0.55 will be available within the next few days or whether I >> should just use 2.0.54. Does anybody know an approximate release date >yet? > >(as for your question, this list is where it gets decided, so the >archives are your best answer) As Nick implies, stay tuned to this channel. It's closing in. Bill
rc3 for testing (was Re: Apache Request problems with tildes)
Joe Schaefer <[EMAIL PROTECTED]> writes: > Marc Gràcia <[EMAIL PROTECTED]> writes: > >> The call to: >> >> my @values = $ar->param($parname); >> >> Only returns the params previous to the tilde ended param value (not >> included). > > Thanks; see if this patch helps Better yet: please test rc3 at http://people.apache.org/~joes/libapreq2-2.06-dev-rc3.tar.gz -- Joe Schaefer
Re: 2.0.55 Release
Subject: Re: 2.0.55 Release On Fri, 15 Jul 2005, Sascha Kersken wrote: > Hi, > > I am currently finishing the companion CD-ROM for my Beginners' Guide To > MySQL (for O'Reilly Germany). So I am wondering whether the announced > release 2.0.55 will be available within the next few days or whether I > should just use 2.0.54. Does anybody know an approximate release date yet? Are you aware that 2.1.x has builtin SQL support, through the apr_dbd framework? There's a MySQL driver for it, although it's not bundled with Apache because it's GPL (for MySQL compatibility), and that's not compatible with distribution from the ASF. (as for your question, this list is where it gets decided, so the archives are your best answer) -- Nick Kew
Re: Apache Request problems with tildes
Marc Gràcia <[EMAIL PROTECTED]> writes:
> The call to:
>
> my @values = $ar->param($parname);
>
> Only returns the params previous to the tilde ended param value (not
> included).
Thanks; see if this patch helps
Index: library/util.c
===
--- library/util.c (revision 216190)
+++ library/util.c (working copy)
@@ -351,6 +351,11 @@
*d = c;
}
+else if (s + 1 >= end) {
+*d = c;
+*charset = APREQ_CHARSET_LATIN1;
+}
+
/* utf8 cases */
else if (c < 0xE0) {
--
Joe Schaefer
2.0.55 Release
Hi, I am currently finishing the companion CD-ROM for my Beginners' Guide To MySQL (for O'Reilly Germany). So I am wondering whether the announced release 2.0.55 will be available within the next few days or whether I should just use 2.0.54. Does anybody know an approximate release date yet? Regards Sascha P.S.: Of course, I would test-drive release candidates when available and report building problems.
Re: svn commit: r218978 - in /httpd/httpd/trunk: CHANGES modules/proxy/mod_proxy_http.c
* RFC 2616 says "All HTTP/1.1 applications MUST be able to receive and decode the "chunked" transfer-coding, and MUST ignore chunk-extension extensions they do not understand." I read this as "an HTTP/1.1 server must accept "chunked" or quit reporting it complies with the HTTP/1.1 specification". No, you should read it as an HTTP/1.1 server MUST be able to receive and decode the "chunked" transfer-coding, because that's what I wrote. In other words, it must be able to parse the message. All of it, except for the preference to RB_STREAM_CHUNKED when, perhaps, we could be more sub-optimal, falling back on RB_SPOOL_CL. Many RB_STREAM_CL choices, before, were equally dangerous, and that C-L == length_read test in the stream_reqbody_chunked() was meant to exclude future abuse. We should be sending C-L if the brigade includes the entire message. CL is always preferred for requests. The only time we should send T-E on a proxied request is if we cannot buffer enough of the received request to know how large it will be. * No longer upgrade HTTP/1.0 requests to the origin server as HTTP/1.1 unless the force-proxy-request-1.1 envvar is set. [William Rowe] -1 (veto), since it is a clear violation of http://www.ietf.org/rfc/rfc2145.txt and the intent of HTTP versions. Roy
Re: [patch 2.0] http body request/response/trace conformance
-1 This is a violation of HTTP:
+if (apr_table_get(r->subprocess_env, "force-proxy-request-1.0")
+ || ((r->proto_num < HTTP_VERSION(1,1))
+ && !apr_table_get(r->subprocess_env,
"force-proxy-request-1.1"))) {
In all cases except when specifically configured otherwise,
the proxy must send HTTP/1.1 if it supports responses in HTTP/1.1.
Roy
Re: mod-cache-requestor plan
On Fri, Jul 15, 2005 at 01:23:29AM -0500, Parin Shah wrote: > - we need to maintain a counter for url in this case which would > decide the priority of the url. But mainting this counter should be a > low overhead operation, I believe. Is a counter strictly speaking the right approach? Why not a time of last access? They give slightly different results, but each is more useful than the other in certain situations. Before htcacheclean existed, I used find and simply deleted files in order of oldest atime attribute until I had enough free space. That kind of behaviour was very useful in my situation (although it involved mounting without the noatime mount option, which I dislike for other reasons). I havn't run a statistical analysis but based on my logs the likelyhood of a url being accessed is very highly correlated to how recently it has been accessed before. A truly popular page will always have been accessed recently, a page that is becoming popular (and therefore very likely to get future hits) will have been accessed recently and a page who's popularity is rapidly diminishing will not have been accessed recently. -- Colm MacCárthaighPublic Key: [EMAIL PROTECTED]
