Re: [PATCH] ProxyAddXHeaders for mod_proxy
Hi,
looks like same patch works with the trunk out of the box.
However, attaching same patch against the trunk branch.
Short instructions to test it:
1. Apply the patch.
2. Compile httpd with mod_proxy and mox_rewrite enabled.
3. Add the following lines to httpd.conf:
ProxyRequests On
RewriteEngine on
RewriteCond %{REQUEST_FILENAME} !^proxy:
RewriteRule ^(.*)$ http://%{HTTP_HOST}$1 [NS,L,P]
4. Test behaviour on remote (proxied) site:
* ProxyAddXHeaders Off -- no X-Forwarded-* headers present;
* ProxyAddXHeaders On -- X-Forwarded-* headers always present.
* ProxyAddXHeaders ReverseOnly -- X-Forwarded-* headers present only
in a reverse proxy request (this is the default and behaviour is same
as without the patch).
On 4/4/07, Jim Jagielski [EMAIL PROTECTED] wrote:
On Feb 28, 2007, at 11:59 AM, Dziugas Baltrunas wrote:
Hi, list,
attaching same patch with small correction to the one submitted
previously (was unnecessary double check for PROXYREQ_REVERSE).
Patch is for httpd-2.2.x branch.
Patches should be against trunk. Once approved
and proven in the trunk environ, then they
may be proposed for backporting to 2.2.x
--
Dziugas Baltrunas
http://dziugas.baltrunas.lt/
httpd_trunk_proxyaddxheaders.patch
Description: Binary data
Re: [PATCH] ProxyAddXHeaders for mod_proxy
Hi, any commens/objections on this patch? On 2/28/07, Dziugas Baltrunas [EMAIL PROTECTED] wrote: Hi, list, attaching same patch with small correction to the one submitted previously (was unnecessary double check for PROXYREQ_REVERSE). Patch is for httpd-2.2.x branch. On 2/28/07, Dziugas Baltrunas [EMAIL PROTECTED] wrote: Hi list, for those of us who are using Apache HTTPD as a internet proxy (a combination of forward and reverse proxy modes) it's important that requests going to the remote servers (which are not known backends but anonymous boxes in this case) would look the same independently whether the original request came in reverse or forward fashion. Currently mod_proxy_http adds X-Forwarded-* headers unconditionally when we have a reverse proxy request and never for forward proxy requests. Attached patch adds a new ProxyAddXHeaders directive with a possible options of On|Off|ReverseOnly, later option leaving the default in order not to break compatibility with current configurations. With this option it is possible to always add X-Forwarded-For, X-Forwarded-Host and X-Forwarded-Server headers for both reverse and forward proxy modes, for reverse proxy only or not to add them at all. Comments and suggestions are welcome. -- Dziugas Baltrunas -- Dziugas Baltrunas -- Dziugas Baltrunas http://dziugas.baltrunas.lt/
Re: [PATCH] ProxyAddXHeaders for mod_proxy
Hi, Nick, if you recall my initial paragraph, I'm here to address the configuration of httpd acting as a generic HTTP proxy which is able to handle both a standard (forward proxy mode) and transparent (reverse proxy mode) proxy requests. This is the same what other proxy servers (like Squid) do. For this to achieve I use 'ProxyRequests On' together with couple of mod_rewrite rules with [P] flag (but not ProxyPass rules since every remote host needs to be proxied). The problem is that with current mod_proxy_http outgoing http request is dependant on how the original request was made to httpd. If original request was in forward fashion, no X-Forwarded-* headers are added to the outgoing request. Otherwise, for reverse proxy requests Privacy Note comes into play and additional X-Forwarded-* headers become visible (i.e. added by httpd) for remote hosts. The intention was not to distinguish the way on how the original request was made and stay consistent (that is, always add some headers or never) when forwarding the request to a remote host. I hope now the problem should be more clear. On 4/4/07, Nick Kew [EMAIL PROTECTED] wrote: On Wed, 4 Apr 2007 12:08:15 +0300 Dziugas Baltrunas [EMAIL PROTECTED] wrote: Hi, any commens/objections on this patch? Hmmm ... not sure I understand the problem you're fixing. Currently mod_proxy_http adds X-Forwarded-* headers unconditionally when we have a reverse proxy request and never for forward proxy requests. I'm guessing this is the crux of your point. In the source code is a comment headed Privacy Note, comprising three paragraphs explaining this behaviour. Perhaps it would be useful if you could address that comment directly. Also useful would be if you could explain what information you want is actually being lost in a forward proxy (if indeed that is your underlying problem)? -- Nick Kew Application Development with Apache - the Apache Modules Book http://www.apachetutor.org/ -- Dziugas Baltrunas http://dziugas.baltrunas.lt/
[PATCH] ProxyAddXHeaders for mod_proxy
Hi list,
for those of us who are using Apache HTTPD as a internet proxy (a
combination of forward and reverse proxy modes) it's important that
requests going to the remote servers (which are not known backends
but anonymous boxes in this case) would look the same independently
whether the original request came in reverse or forward fashion.
Currently mod_proxy_http adds X-Forwarded-* headers unconditionally
when we have a reverse proxy request and never for forward proxy
requests.
Attached patch adds a new ProxyAddXHeaders directive with a possible
options of On|Off|ReverseOnly, later option leaving the default in
order not to break compatibility with current configurations. With
this option it is possible to always add X-Forwarded-For,
X-Forwarded-Host and X-Forwarded-Server headers for both reverse and
forward proxy modes, for reverse proxy only or not to add them at all.
Comments and suggestions are welcome.
--
Dziugas Baltrunas
Index: docs/manual/mod/mod_proxy.html.en
===
--- docs/manual/mod/mod_proxy.html.en (revision 512643)
+++ docs/manual/mod/mod_proxy.html.en (working copy)
@@ -83,6 +83,7 @@
liimg alt= src=../images/down.gif / a
href=#proxyrequestsProxyRequests/a/li
liimg alt= src=../images/down.gif / a
href=#proxytimeoutProxyTimeout/a/li
liimg alt= src=../images/down.gif / a
href=#proxyviaProxyVia/a/li
+liimg alt= src=../images/down.gif / a
href=#proxyaddxheadersProxyAddXHeaders/a/li
/ul
h3Topics/h3
ul id=topics
@@ -1168,11 +1169,33 @@
/ul
/div
+div class=topa href=#page-headerimg alt=top src=../images/up.gif
//a/div
+div class=directive-sectionh2a name=ProxyAddXHeaders
id=ProxyAddXHeadersProxyAddXHeaders/a a name=proxyaddxheaders
id=proxaddxheadersDirective/a/h2
+table class=directive
+trtha
href=directive-dict.html#DescriptionDescription:/a/thtdInformation
provided in codeX-Forwarded-For/code,
+codeX-Forwarded-Host/code and codeX-Forwarded-Server/code
headers./td/tr
+trtha
href=directive-dict.html#SyntaxSyntax:/a/thtdcodeProxyAddXHeaders
On|Off|ReverseOnly/code/td/tr
+trtha
href=directive-dict.html#DefaultDefault:/a/thtdcodeProxyAddXHeaders
ReverseOnly/code/td/tr
+trtha href=directive-dict.html#ContextContext:/a/thtdserver
config, virtual host/td/tr
+trtha
href=directive-dict.html#StatusStatus:/a/thtdExtension/td/tr
+trtha
href=directive-dict.html#ModuleModule:/a/thtdmod_proxy/td/tr
+/table
+pThis directive controls the use of the codeX-Forwarded-For/code,
codeX-Forwarded-Host/code and codeX-Forwarded-Server/code headers./p
+ul
+liIf set to codeReverseOnly/code, which is the default,
codeX-Forwarded-*/code headers are only added when
+we have a reverse proxy request./li
+
+liIf set to codeOn/code, codeX-Forwarded-*/code headers are
added both for reverse and forward proxy requests./li
+
+liIf set to codeOff/code, no codeX-Forwarded-*/code headers are
added for either reverse or forward proxy requests./li
+
+/ul
/div
+/div
div class=bottomlang
pspanAvailable Languages: /spana href=../en/mod/mod_proxy.html
title=Englishnbsp;ennbsp;/a |
a href=../ja/mod/mod_proxy.html hreflang=ja rel=alternate
title=Japanesenbsp;janbsp;/a/p
/divdiv id=footer
p class=apacheCopyright 2006 The Apache Software Foundation.br /Licensed
under the a href=http://www.apache.org/licenses/LICENSE-2.0;Apache License,
Version 2.0/a./p
p class=menua href=../mod/Modules/a | a
href=../mod/directives.htmlDirectives/a | a href=../faq/FAQ/a | a
href=../glossary.htmlGlossary/a | a
href=../sitemap.htmlSitemap/a/p/div
-/body/html
\ No newline at end of file
+/body/html
Index: modules/proxy/mod_proxy_http.c
===
--- modules/proxy/mod_proxy_http.c (revision 512643)
+++ modules/proxy/mod_proxy_http.c (working copy)
@@ -699,7 +699,7 @@
* ProxyVia option for details.
*/
-if (PROXYREQ_REVERSE == r-proxyreq) {
+if (conf-x_fwd_for == x_fwd_on || (conf-x_fwd_for == x_fwd_reverse
PROXYREQ_REVERSE == r-proxyreq)) {
const char *buf;
/* Add X-Forwarded-For: so that the upstream has a chance to
@@ -711,7 +711,9 @@
/* Add X-Forwarded-Host: so that upstream knows what the
* original request hostname was.
*/
-if ((buf = apr_table_get(r-headers_in, Host))) {
+buf = PROXYREQ_REVERSE == (PROXYREQ_REVERSE == r-proxyreq) ?
+ apr_table_get(r-headers_in, Host) :
r-hostname;
+if (buf) {
apr_table_mergen(r-headers_in, X-Forwarded-Host, buf);
}
Index: modules/proxy/mod_proxy.c
===
--- modules/proxy/mod_proxy.c (revision 512643)
+++ modules/proxy/mod_proxy.c (working copy)
@@ -870,6 +870,8 @@
ps-timeout_set = 0;
ps-badopt = bad_error;
ps-badopt_set = 0;
+ps-x_fwd_for = x_fwd_reverse;
+ps-x_fwd_for_set
Re: [PATCH] ProxyAddXHeaders for mod_proxy
Hi, list,
attaching same patch with small correction to the one submitted
previously (was unnecessary double check for PROXYREQ_REVERSE).
Patch is for httpd-2.2.x branch.
On 2/28/07, Dziugas Baltrunas [EMAIL PROTECTED] wrote:
Hi list,
for those of us who are using Apache HTTPD as a internet proxy (a
combination of forward and reverse proxy modes) it's important that
requests going to the remote servers (which are not known backends
but anonymous boxes in this case) would look the same independently
whether the original request came in reverse or forward fashion.
Currently mod_proxy_http adds X-Forwarded-* headers unconditionally
when we have a reverse proxy request and never for forward proxy
requests.
Attached patch adds a new ProxyAddXHeaders directive with a possible
options of On|Off|ReverseOnly, later option leaving the default in
order not to break compatibility with current configurations. With
this option it is possible to always add X-Forwarded-For,
X-Forwarded-Host and X-Forwarded-Server headers for both reverse and
forward proxy modes, for reverse proxy only or not to add them at all.
Comments and suggestions are welcome.
--
Dziugas Baltrunas
--
Dziugas Baltrunas
Index: docs/manual/mod/mod_proxy.html.en
===
--- docs/manual/mod/mod_proxy.html.en (revision 512643)
+++ docs/manual/mod/mod_proxy.html.en (working copy)
@@ -83,6 +83,7 @@
liimg alt= src=../images/down.gif / a
href=#proxyrequestsProxyRequests/a/li
liimg alt= src=../images/down.gif / a
href=#proxytimeoutProxyTimeout/a/li
liimg alt= src=../images/down.gif / a
href=#proxyviaProxyVia/a/li
+liimg alt= src=../images/down.gif / a
href=#proxyaddxheadersProxyAddXHeaders/a/li
/ul
h3Topics/h3
ul id=topics
@@ -1168,11 +1169,33 @@
/ul
/div
+div class=topa href=#page-headerimg alt=top src=../images/up.gif
//a/div
+div class=directive-sectionh2a name=ProxyAddXHeaders
id=ProxyAddXHeadersProxyAddXHeaders/a a name=proxyaddxheaders
id=proxaddxheadersDirective/a/h2
+table class=directive
+trtha
href=directive-dict.html#DescriptionDescription:/a/thtdInformation
provided in codeX-Forwarded-For/code,
+codeX-Forwarded-Host/code and codeX-Forwarded-Server/code
headers./td/tr
+trtha
href=directive-dict.html#SyntaxSyntax:/a/thtdcodeProxyAddXHeaders
On|Off|ReverseOnly/code/td/tr
+trtha
href=directive-dict.html#DefaultDefault:/a/thtdcodeProxyAddXHeaders
ReverseOnly/code/td/tr
+trtha href=directive-dict.html#ContextContext:/a/thtdserver
config, virtual host/td/tr
+trtha
href=directive-dict.html#StatusStatus:/a/thtdExtension/td/tr
+trtha
href=directive-dict.html#ModuleModule:/a/thtdmod_proxy/td/tr
+/table
+pThis directive controls the use of the codeX-Forwarded-For/code,
codeX-Forwarded-Host/code and codeX-Forwarded-Server/code headers./p
+ul
+liIf set to codeReverseOnly/code, which is the default,
codeX-Forwarded-*/code headers are only added when
+we have a reverse proxy request./li
+
+liIf set to codeOn/code, codeX-Forwarded-*/code headers are
added both for reverse and forward proxy requests./li
+
+liIf set to codeOff/code, no codeX-Forwarded-*/code headers are
added for either reverse or forward proxy requests./li
+
+/ul
/div
+/div
div class=bottomlang
pspanAvailable Languages: /spana href=../en/mod/mod_proxy.html
title=Englishnbsp;ennbsp;/a |
a href=../ja/mod/mod_proxy.html hreflang=ja rel=alternate
title=Japanesenbsp;janbsp;/a/p
/divdiv id=footer
p class=apacheCopyright 2006 The Apache Software Foundation.br /Licensed
under the a href=http://www.apache.org/licenses/LICENSE-2.0;Apache License,
Version 2.0/a./p
p class=menua href=../mod/Modules/a | a
href=../mod/directives.htmlDirectives/a | a href=../faq/FAQ/a | a
href=../glossary.htmlGlossary/a | a
href=../sitemap.htmlSitemap/a/p/div
-/body/html
\ No newline at end of file
+/body/html
Index: modules/proxy/mod_proxy_http.c
===
--- modules/proxy/mod_proxy_http.c (revision 512643)
+++ modules/proxy/mod_proxy_http.c (working copy)
@@ -699,7 +699,7 @@
* ProxyVia option for details.
*/
-if (PROXYREQ_REVERSE == r-proxyreq) {
+if (conf-x_fwd_for == x_fwd_on || (conf-x_fwd_for == x_fwd_reverse
PROXYREQ_REVERSE == r-proxyreq)) {
const char *buf;
/* Add X-Forwarded-For: so that the upstream has a chance to
@@ -711,7 +711,8 @@
/* Add X-Forwarded-Host: so that upstream knows what the
* original request hostname was.
*/
-if ((buf = apr_table_get(r-headers_in, Host))) {
+buf = (PROXYREQ_REVERSE == r-proxyreq) ? apr_table_get(r-headers_in,
Host) : r-hostname;
+if (buf) {
apr_table_mergen(r-headers_in, X-Forwarded-Host, buf);
}
Index: modules/proxy/mod_proxy.c
===
--- modules/proxy/mod_proxy.c (revision 512643
Re: Limiting response body length
Hi list,
thanks for the replies. Looks like squid in case Content-Length
response header is missing, does it's limitation in a hard way (snip
from src/client_side.c):
} else if (clientReplyBodyTooLarge(http, http-out.offset - 4096)) {
/* 4096 is a margin for the HTTP headers included in out.offset */
comm_close(fd);
} else {
However this seems to be the only way in case we want to avoid content
buffering. mod_security also relies on Content-Length an if is not
present, output buffering (and I suppose the limitation as well) stops
(snip from apache2/apache2_io.c:output_filter):
case 0 :
/* We do not want to observe this response body
* but we need to remain attached to observe
* when it is completed so that we can run
* the RESPONSE_BODY phase.
*/
msr-of_skipping = 1;
msr-resbody_status = RESBODY_STATUS_NOT_READ;
break;
On 2/13/07, Nick Kew [EMAIL PROTECTED] wrote:
On Mon, 12 Feb 2007 23:35:24 +0100
Henrik Nordstrom [EMAIL PROTECTED] wrote:
mån 2007-02-12 klockan 21:55 + skrev Nick Kew:
Because the chunking filter is equipped to discard the chunk that
takes it over the limit, and substitute end-of-chunking.
If we do that in a new filter, we have to reinvent that wheel.
Not sure substitue end-of-chunking is a reasonable thing here. It's
an abort condition, not an EOF condition. Imho you'd better abort the
flow, that way telling the client that the request failed instead of
silently truncating the response.
How would you abort it other than by truncating it?
Don't forget, the headers are long gone.
If you don't send an end-marker, the client will
sit there waiting for more.
--
Nick Kew
Application Development with Apache - the Apache Modules Book
http://www.apachetutor.org/
--
Dziugas
Re: mod_cache: MISS or HIT
Hi, I was looking for something more reliable so I now realize it will require patching of mod_cache.c by adding some flag (i.e. setting some key r-notes, registering mod_log_config modifier via log_pfn_register or similar). On 2/12/07, Joshua Slive [EMAIL PROTECTED] wrote: On 2/12/07, Dziugas Baltrunas [EMAIL PROTECTED] wrote: I'm trying to figure out the way how to put information in access log (via mod_log_config) whether the request was a cache hit or miss (similar to what squid does - TCP_MISS and TCP_HIT). I think this information is necessary for any proxy server (or acting like it) and it's always wise to know hit/miss ratio. You can log the Age HTTP response header for this. If it is present, you have a cache hit, and if not, a miss. Of course, this probably wouldn't be reliable if your cache was in front of another cache. But it works in most circumstances. Joshua. -- Dziugas
mod_cache: MISS or HIT
Hi list, I'm using httpd 2.2.4 as both reverse and forward proxy having caching enabled with mod_cache. I'm trying to figure out the way how to put information in access log (via mod_log_config) whether the request was a cache hit or miss (similar to what squid does - TCP_MISS and TCP_HIT). I think this information is necessary for any proxy server (or acting like it) and it's always wise to know hit/miss ratio. Since at least officially mod_cache (and it's sub-modules) does not export such information, I thought about registering custom mod_log_config tag (via log_pfn_register). However, looking at the mod_*_cache source and trying to distinguish cache request from an ordinary one didn't seem trivial. Just several observations so far: 1. In case it's HIT, output goes over CACHE_OUT, otherwise it's CACHE_SAVE. Q: is it possible for external module to check (e.g. by traversing r-output_filters) which filter _was_ used? 2. For mod_file_cache there is a possible workaround to rely on r-clength which is set to match-finfo.size (inside mod_file_cache.c) in case we have CACHE_OUT. clength seems to be referenced nowhere else except mod_file_cache.c and http/http_protocol.c where it's set to 0. However, for mod_mem_cache things are different. Thanks. -- Dziugas Baltrunas
Limiting response body length
Hi list, I'm wondering if there any plans to implement implement a response body length limitation inside mod_proxy? For now we have only a global LimitRequestBody and what I'm looking is an analog LimitResponseBody. In case Apache HTTP works as a reverse/forward proxy, it's usual case for a proxy to block downloading files of a certain size (huge MP3 files as an example). To illustrate, squid for this purpose has reply_body_max_size [1] parameter. Looks like it is only Content-Length response header (if any) dependent, but IMHO with Apache filtering mechanism it should be possible to cancel a fetch transaction when output byte counter reaches some boundary. I've also tried to look at exiting third party modules like mod_throttle [2] (deprecated), mod_cband [3] and mod_bw [4], but all of them seem to be tied on bandwidth limitations and not a limits of a single request. Thanks in advance. [1] http://www.visolve.com:81/squid/squid30/accesscontrols.php#reply_body_max_size [2] http://www.snert.com/Software/apache.html [3] http://cband.linux.pl/ [4] http://bwmod.sourceforge.net/, http://svn.apache.org/viewvc/httpd/mod_bw/ -- Dziugas Baltrunas
