William Rowe Jr. is now V.P., Apache HTTP Server
Since I have been elected onto the ASF board and don't want to be criticizing my own reports, I have resigned as chair of the project. The PMC nominated OtherBill for the post and, after I added a just-in-time resolution to today's board meeting agenda, I am happy to report that it was unanimously approved by the board: WHEREAS, the Board of Directors heretofore appointed Roy T. Fielding to the office of Vice President, Apache HTTP Server, and WHEREAS, the Board of Directors is in receipt of the resignation of Roy T. Fielding from the office of Vice President, Apache HTTP Server, and WHEREAS, the Project Management Committee of the Apache HTTP Server project has chosen by vote to recommend William Rowe Jr. as the successor to the post; NOW, THEREFORE, BE IT RESOLVED, that Roy T. Fielding is relieved and discharged from the duties and responsibilities of the office of Vice President, Apache HTTP Server, and BE IT FURTHER RESOLVED, that William Rowe Jr. be and hereby is appointed to the office of Vice President, Apache HTTP Server, to serve in accordance with and subject to the direction of the Board of Directors and the Bylaws of the Foundation until death, resignation, retirement, removal or disqualification, or until a successor is appointed. Happy coding, Roy
Re: mod_deflate DoS using HEAD
William A. Rowe, Jr. wrote: So +1 to the proposed patch; in fact, +1 on unsetting C-L and treating HEAD to the same processing as 304. +1. Since it's a SHOULD not a MUST, we can be pragmatic with the headers. That's back to Eric's original patch, isn't it? -- Nick Kew
Re: mod_deflate DoS using HEAD
Joe Orton wrote: > On Wed, Jul 15, 2009 at 11:03:24AM +0200, "Plüm, Rüdiger, VF-Group" wrote: >>> I'm confused. Why do this check so late, and why does r->bytes_sent >>> matter? Why does it "screw up the protocol" if the DEFLATE >> All depends on the first brigade that passes mod_deflate. If this brigade >> contains the whole response *and* mod_deflate can compress this response >> in one go meaning calling ap_passbrigade only once to sent the fully >> compressed >> response then the content-length filter can determine the length of the >> content >> and add it to the HEAD response as the same GET request would be delivered >> with a C-L. > > I understand that, but, the whole point of doing the check early, as > Eric's proposed patch did, is to *avoid* doing all that work because it > may be exhorbitantly expensive, even in the case of a single brigade > containing the whole response - that's the issue in hand. > > The GET/304 case already omits the C-L from the response, so, I'm still > struggling to see how doing the same for HEAD somehow breaks HTTP > caching in any way. Ok, rereading 9.4, it discusses "new field values indicate that the cached entity differs from the current entity" - new; not omitted. In the absence of the header, the entity doesn't "change". It is unknown. So a cache author can/should/would[?] treat it as fresh, as you point out with respect to 304. So +1 to the proposed patch; in fact, +1 on unsetting C-L and treating HEAD to the same processing as 304.
Re: mod_deflate DoS using HEAD
On Tue, Jul 14, 2009 at 11:47 AM, "Plüm, Rüdiger, VF-Group" <
ruediger.pl...@vodafone.com> wrote:
>
> All very true. But how about the following patch. It should do no
> harm and should solve the issue in at least some cases (I think
> in most cases):
>
> Index: modules/filters/mod_deflate.c
>
> +if (r->header_only && r->bytes_sent) {
> +ap_remove_output_filter(f);
> +return ap_pass_brigade(f->next, bb);
> +}
>
+1
It doesn't break HTTP or caching, is simple, and any "extra" overhead (if
you want to call it that) is limited to one zlib buffer as you pointed out
later.
Greg
Re: AuthBasicProvider failover and mod_authnz_ldap
On Wed, Jul 15, 2009 at 10:58 AM, Brad Nicholes wrote: > The question here is given this context, should AUTH_GENERAL_ERROR == > AUTH_USER_NOT_FOUND? Given this context, the answer is probably yes. > However are there any cases dealing with authn_alias where the answer should > be no? To care about this path, you have to have multiple providers with some overlap of userids. Tough to say how someone could both want the duplicate IDs but would not want the 2nd provider to have a chance. Change in default to trunk and figure out how to make it configurable w/o using the word Authoritative? -- Eric Covener cove...@gmail.com
Re: Hacking in an SLA for proxied requests in mod_proxy_http
Brian Akins wrote: > Is the proxy-timeout for the entire request to be returned, the first byte, > or just an i/o timeout? To set a 900ms timeout the code does approximately this: apr_interval_time_t new_timeout = apr_time_make(0, 900 * (APR_USEC_PER_SEC/1000)); apr_socket_timeout_set(backend->sock, new_timeout); I don't _really_ know what effect that has but I believe it's first-byte response timeout. I'm hoping that 'rpluem', 'jim', 'niq', etc (people who committed code in the mod) will jump in and give guidance about correctness and side effects of the hack. [Sorry all for the wretched grammar in the first paragraph of original email, ouch!] - Neal
RE: mod_deflate DoS using HEAD
> -Original Message- > From: Joe Orton [mailto:jor...@redhat.com] > Sent: Mittwoch, 15. Juli 2009 15:29 > To: dev@httpd.apache.org > Subject: Re: mod_deflate DoS using HEAD > > On Wed, Jul 15, 2009 at 11:03:24AM +0200, "Plüm, Rüdiger, > VF-Group" wrote: > > > I'm confused. Why do this check so late, and why does > r->bytes_sent > > > matter? Why does it "screw up the protocol" if the DEFLATE > > > > All depends on the first brigade that passes mod_deflate. > If this brigade > > contains the whole response *and* mod_deflate can compress > this response > > in one go meaning calling ap_passbrigade only once to sent > the fully compressed > > response then the content-length filter can determine the > length of the content > > and add it to the HEAD response as the same GET request > would be delivered > > with a C-L. > > I understand that, but, the whole point of doing the check early, as > Eric's proposed patch did, is to *avoid* doing all that work > because it > may be exhorbitantly expensive, even in the case of a single brigade > containing the whole response - that's the issue in hand. It isn't really this expensive, because of the second condition. Each time the internal zlib outgoing buffer with the compressed data within is full we pass this data down the chain. We don't buffer this data in the filter. It is a good filter :-). This means if this passing is not the *only* passing mod_deflate does down the chain we cannot go the C-L road anyway. So most larger responses even in one brigade if not extremely compressable will cause mod_deflate aborting compression in this case. IMHO the rule is that if the C-L of the compressed response is larger than the size of zlibs outgoing buffer we will sent the response in T-E chunked anyway. Thus we can stop compressing for a HEAD request once we got over this limit which should happen fairly early. > > The GET/304 case already omits the C-L from the response, so, > I'm still > struggling to see how doing the same for HEAD somehow breaks HTTP > caching in any way. IMHO it can cause an unnecessary invalidation of a cache entry. This is not nice. Whether this could be called a "break" is another story. Regards Rüdiger
RE: AuthBasicProvider failover and mod_authnz_ldap
> -Original Message- > From: Brad Nicholes > Sent: Mittwoch, 15. Juli 2009 16:58 > To: dev@httpd.apache.org > Subject: Re: AuthBasicProvider failover and mod_authnz_ldap > authn_alias where the answer should be no? The second issue > is what should authnz_ldap do? Authnz_ldap has already been > coded for redundancy if it is configured for it. If there is > a problem in this case, then it is a bug that should be looked at. > I guess this case was less about a second ldap server should the first fail (this can be done with Authnz_ldap as you mention) but about a different auth provider should the Ldap provider fail. Regards Rüdiger
Re: AuthBasicProvider failover and mod_authnz_ldap
>>> On 7/13/2009 at 3:31 PM, in message <1404e5910907131431m42ec4cffwc08caf273b71f...@mail.gmail.com>, Eric Covener wrote: > PR#47521 points out that when mod_authnz_ldap has some fatal LDAP > connectivity error, it doesn't allow other AuthBasicProviders to have > a shot at checking the userid. > > It seems like the normal use case for two providers is when there are > two disjoint user repositories, and we only move on to search the > second when the user of interest isn't found in the first. > > For LDAP, should we treat a failure to even search the database this > same way, allowing it to move onto other providers > (AUTH_USER_NOT_FOUND vs. AUTH_GENERAL_ERROR)? It seems to me that the > LDAP backends often have poor reliability and lots of use cases would > want the 2nd provider for emergencies, at little expense (hypothetical > attacker that took out your LDAP servers, and compromised e.g. > AuthUserFile). > > Thoughts? There are actually two issues to consider in the context of PR#47521. The first issue is what should mod_authn_alias do if it gets an AUTH_GENERAL_ERROR vs AUTH_USER_NOT_FOUND. Apparently, according to the bug, mod_authn_alias just stops which is probably what the intention was when I coded it (years ago in another lifetime ;) . The question here is given this context, should AUTH_GENERAL_ERROR == AUTH_USER_NOT_FOUND? Given this context, the answer is probably yes. However are there any cases dealing with authn_alias where the answer should be no? The second issue is what should authnz_ldap do? Authnz_ldap has already been coded for redundancy if it is configured for it. If there is a problem in this case, then it is a bug that should be looked at. Brad
Re: Hacking in an SLA for proxied requests in mod_proxy_http
I haven't looked at the code, but +1 for the idea. We had a hack that did something somewhat similar, but it was gross and in 2.0 - we never used it in prod. Is the proxy-timeout for the entire request to be returned, the first byte, or just an i/o timeout? -- Brian Akins
Re: mod_deflate DoS using HEAD
On Wed, Jul 15, 2009 at 11:03:24AM +0200, "Plüm, Rüdiger, VF-Group" wrote: > > I'm confused. Why do this check so late, and why does r->bytes_sent > > matter? Why does it "screw up the protocol" if the DEFLATE > > All depends on the first brigade that passes mod_deflate. If this brigade > contains the whole response *and* mod_deflate can compress this response > in one go meaning calling ap_passbrigade only once to sent the fully > compressed > response then the content-length filter can determine the length of the > content > and add it to the HEAD response as the same GET request would be delivered > with a C-L. I understand that, but, the whole point of doing the check early, as Eric's proposed patch did, is to *avoid* doing all that work because it may be exhorbitantly expensive, even in the case of a single brigade containing the whole response - that's the issue in hand. The GET/304 case already omits the C-L from the response, so, I'm still struggling to see how doing the same for HEAD somehow breaks HTTP caching in any way. Regards, Joe
Re: How do I manipulate request_rec Object in worker.c
On Wed, Jul 15, 2009 at 7:39 AM, ricardo13 wrote: > > hi all, > > I modified request_rec simply adding a field in request_rec. > Now, I would like get this field in worker.c > > My doubt about this is because worker.c manipulates only sockets and doesn't > request_rec object. > How do I do this ?? How do you do what? Why do you think you need to know the details of a request before a request has been read? -- Eric Covener cove...@gmail.com
Re: mod_deflate DoS using HEAD
"William A. Rowe, Jr." writes: > Joe Orton wrote: >> >> Does 2616 mandate that a resource must always >> exactly the same set of content-codings across methods and time? >> (AFAICT there is no MUST on that front; it's a SHOULD if anything) > > Read through to the end, it breaks all proxied content; > > 9.4 HEAD > >The HEAD method is identical to GET except that the server MUST NOT >return a message-body in the response. The metainformation contained >in the HTTP headers in response to a HEAD request SHOULD be identical >to the information sent in response to a GET request. This method can >be used for obtaining metainformation about the entity implied by the >request without transferring the entity-body itself. This method is >often used for testing hypertext links for validity, accessibility, >and recent modification. > >The response to a HEAD request MAY be cacheable in the sense that the >information contained in the response MAY be used to update a >previously cached entity from that resource. If the new field values >indicate that the cached entity differs from the current entity (as >would be indicated by a change in Content-Length, Content-MD5, ETag >or Last-Modified), then the cache MUST treat the cache entry as >stale. Doesn't that last sentence just indicate that the cache entry will be invalidated? That would add some possibly unnecessary work fetching the content again the next time it's requested, but I wouldn't think it would break anything. -- Dan Poirier
Re: [warn] worker http://ip_maquina/ already used by another worker
Hi, I've resolvel this error. Thank you Ricardo ricardo13 wrote: > > Hi, > > I have a webcluster and I'm prioritize the requests. > > I modify request_rec adding a field called prior and conditional test ( > IF() ) in mod_rewrite for classify. > > All time that start APACHE show the follow: > # sudo apachectl -k start > [warn] worker http://ip_maquina/ already used by another worker > > What's this ?? > > Thank you. > Ricardo > -- View this message in context: http://www.nabble.com/-warn--worker-http%3A--ip_maquina--already-used-by-another-worker-tp24486952p24495981.html Sent from the Apache HTTP Server - Dev mailing list archive at Nabble.com.
Re: [warn] worker http://ip_maquina/ already used by another worker
Hi,
I studied this error and I undestand that error is created for ProxyPass.
My httpd.conf:
RewriteEngine on
RewriteLog /usr/local/apache2/logs/rewrite_log
RewriteLogLevel 5
RewriteLock /usr/local/apache2/logs/file.lock
RewriteMap prgmap prg:/usr/local/apache2/admControl # the output is
"/server1/" or "/server2/"
RewriteRule ^(.*) ${prgmap:$1} [PT]
ProxyPass /server1/ http://192.168.2.29/index.html
#ProxyPassReverse /server1/ http://192.168.2.29/index.html
ProxyPass /server2/ http://192.168.2.29/index.html
#ProxyPassReverse /server2/ http://192.168.2.29/index.html
Why happen this error ??
Thank you
Ricardo
ricardo13 wrote:
>
> Hi,
>
> I have a webcluster and I'm prioritize the requests.
>
> I modify request_rec adding a field called prior and conditional test (
> IF() ) in mod_rewrite for classify.
>
> All time that start APACHE show the follow:
> # sudo apachectl -k start
> [warn] worker http://ip_maquina/ already used by another worker
>
> What's this ??
>
> Thank you.
> Ricardo
>
--
View this message in context:
http://www.nabble.com/-warn--worker-http%3A--ip_maquina--already-used-by-another-worker-tp24486952p24495754.html
Sent from the Apache HTTP Server - Dev mailing list archive at Nabble.com.
RE: mod_deflate DoS using HEAD
> -Original Message-
> From: Joe Orton [mailto:jor...@redhat.com]
> Sent: Mittwoch, 15. Juli 2009 09:51
> To: dev@httpd.apache.org
> Subject: Re: mod_deflate DoS using HEAD
>
> On Tue, Jul 14, 2009 at 05:47:16PM +0200, "Plüm, Rüdiger,
> VF-Group" wrote:
> >
> >
> > > -Original Message-
> > > From: William A. Rowe, Jr.
> > > Sent: Montag, 13. Juli 2009 23:58
> > > To: dev@httpd.apache.org
> > > Subject: Re: mod_deflate DoS using HEAD
> > >
> > > Nick Kew wrote:
> > > > Eric Covener wrote:
> > > >
> > > >> /* For a 304 response, only change the headers */
> > > >> -if (r->status == HTTP_NOT_MODIFIED) {
> > > >> +if (r->status == HTTP_NOT_MODIFIED ||
> r->header_only) {
> > > >
> > > > Technically speaking, screws up the protocol.
> > > >
> > > > IMHO it would be acceptable provided:
> > > > (a) it's an option for the admin, rather than enforced
> > > > (b) it's documented
> > > > (c) the headers are correct: either Content-Encoding is
> > > > unset (uncompressed response) or Content-Length is
> > > > unset. Probably the former.
> > >
> > > Agreed. It's not a DoS. If the admin wants to conserve CPU
> > > resources, they must either;
> > >
> > > * cache the deflated pages (avoid user-agent header if there
> > >are multiples, which reminds me we need a module to unset the
> > >accept deflate trigger on non-compliant browsers running
> > >very-first in the quick_handler.)
> > >
> > > * create gzip'ed content, navigate the choice of content through
> > >multiviews.
> > >
> > > * do not do server-side deflation (it is expensive).
> > >
> >
> > All very true. But how about the following patch. It should do no
> > harm and should solve the issue in at least some cases (I think
> > in most cases):
>
> I'm confused. Why do this check so late, and why does r->bytes_sent
> matter? Why does it "screw up the protocol" if the DEFLATE
All depends on the first brigade that passes mod_deflate. If this brigade
contains the whole response *and* mod_deflate can compress this response
in one go meaning calling ap_passbrigade only once to sent the fully compressed
response then the content-length filter can determine the length of the content
and add it to the HEAD response as the same GET request would be delivered
with a C-L. If the above is not true the according GET response would be
delivered with T-E chunked anyway (in the HTTP/1.1 case or without anything
but closing the connection after sending the response in the HTTP/1.0 case).
So there is no point in doing further compression. And r->bytes_sent != 0
indicates that we have been already in the content length filter and that it
cannot
measure the length of the response for creating a C-L header as I tried to
explain
in my comment.
Well one might argue that the number of cases where the first brigade
contains the whole response *and* mod_deflate can compress this response
in one go meaning calling ap_passbrigade only once to sent the fully compressed
response is so low that this doesn't justify this approach and that we should
just get out of the way in the case of HEAD requests. Especially as providing
the correct C-L in a HEAD response is only a SHOULD (see below)
> filter does
> nothing for a HEAD request? Because of the concern that a HEAD will
> return a different C-L & C-E to a GET on the same resource
> with the same
> Accept-Encoding header? Does 2616 mandate that a resource
> must always
> exactly the same set of content-codings across methods and time?
> (AFAICT there is no MUST on that front; it's a SHOULD if anything)
Exactly this is my impression and it is backed by 9.4 in RFC2616.
But you are correct it is only a SHOULD.
Regards
Rüdiger
Re: mod_deflate DoS using HEAD
Joe Orton wrote: > > I'm confused. Why do this check so late, and why does r->bytes_sent > matter? Why does it "screw up the protocol" if the DEFLATE filter does > nothing for a HEAD request? Because of the concern that a HEAD will > return a different C-L & C-E to a GET on the same resource with the same > Accept-Encoding header? Does 2616 mandate that a resource must always > exactly the same set of content-codings across methods and time? > (AFAICT there is no MUST on that front; it's a SHOULD if anything) Read through to the end, it breaks all proxied content; 9.4 HEAD The HEAD method is identical to GET except that the server MUST NOT return a message-body in the response. The metainformation contained in the HTTP headers in response to a HEAD request SHOULD be identical to the information sent in response to a GET request. This method can be used for obtaining metainformation about the entity implied by the request without transferring the entity-body itself. This method is often used for testing hypertext links for validity, accessibility, and recent modification. The response to a HEAD request MAY be cacheable in the sense that the information contained in the response MAY be used to update a previously cached entity from that resource. If the new field values indicate that the cached entity differs from the current entity (as would be indicated by a change in Content-Length, Content-MD5, ETag or Last-Modified), then the cache MUST treat the cache entry as stale.
Re: mod_deflate DoS using HEAD
On Tue, Jul 14, 2009 at 05:47:16PM +0200, "Plüm, Rüdiger, VF-Group" wrote:
>
>
> > -Original Message-
> > From: William A. Rowe, Jr.
> > Sent: Montag, 13. Juli 2009 23:58
> > To: dev@httpd.apache.org
> > Subject: Re: mod_deflate DoS using HEAD
> >
> > Nick Kew wrote:
> > > Eric Covener wrote:
> > >
> > >> /* For a 304 response, only change the headers */
> > >> -if (r->status == HTTP_NOT_MODIFIED) {
> > >> +if (r->status == HTTP_NOT_MODIFIED || r->header_only) {
> > >
> > > Technically speaking, screws up the protocol.
> > >
> > > IMHO it would be acceptable provided:
> > > (a) it's an option for the admin, rather than enforced
> > > (b) it's documented
> > > (c) the headers are correct: either Content-Encoding is
> > > unset (uncompressed response) or Content-Length is
> > > unset. Probably the former.
> >
> > Agreed. It's not a DoS. If the admin wants to conserve CPU
> > resources, they must either;
> >
> > * cache the deflated pages (avoid user-agent header if there
> >are multiples, which reminds me we need a module to unset the
> >accept deflate trigger on non-compliant browsers running
> >very-first in the quick_handler.)
> >
> > * create gzip'ed content, navigate the choice of content through
> >multiviews.
> >
> > * do not do server-side deflation (it is expensive).
> >
>
> All very true. But how about the following patch. It should do no
> harm and should solve the issue in at least some cases (I think
> in most cases):
I'm confused. Why do this check so late, and why does r->bytes_sent
matter? Why does it "screw up the protocol" if the DEFLATE filter does
nothing for a HEAD request? Because of the concern that a HEAD will
return a different C-L & C-E to a GET on the same resource with the same
Accept-Encoding header? Does 2616 mandate that a resource must always
exactly the same set of content-codings across methods and time?
(AFAICT there is no MUST on that front; it's a SHOULD if anything)
Regards, Joe
