Re: [users@httpd] ProxyPass option mapping=servlet hurts mod_rewrite
Hi Hendrik; > > > RewriteRule "^/alpha/gobeta/(.*)$" http://server2.localnet:8080/beta/$1 > > [P] > If the RewriteRule is specified with the option [P] it works correctly. The > request > https://example.ort/alpha/gobeta/anypath is sent to the beta backend. > > > Also, it seems to me that "/alpha/gobeta/.." is not matched by the > > "ProxyPass /beta ...", so it really boils down to the RewriteRule only. > > What do you expect here? > But when I take the option [PT] the behavior changes. Without > "mapping=servlet" the > request is sent to the beta backend. With "mapping=servlet" the request is > replied > by a 404 NOT FOUND. - Is this change of behavior acceptable? I don't think that it's a change in behaviour, what changes is ProxyPass with and without mapping= with regard to the following matches (RewriteRule or ), but existing ProxyPass without mapping= and RewriteRules work like before. The ProxyPass mapping=servlet has to apply some special normalization to the request uri-path (notably stripping path parameters defined by the servlet specification) so that further mapping in httpd works according to the backend application(s) server (e.g. Tomcat), so that authn/authz in and RewriteRules have the same path representation as the backend and can enforce relevant access or forwarding rules. Still the potential path ";parameters" from the original uri-path need to be forwarded to the backend since they will finally be consumed by the applications (which is the point of the servlet specification). So there is no way for a RewriteRule happening *after* this normalization to be able to modify the uri-path (e.g. with a [PT] rule) without losing everything stripped by the normalization, while sending the resulting uri-path to the backend (as I proposed with a [P] rule) would be quite useless actually. I'm afraid that the interaction between a ProxyPass mapping=servlet and the following RewriteRules is then limited to non-rewrite rules or rewrite rules that "break" the proxying (like [F] or [R] or internal redirects to a local resource), which is is what r1898509 addresses, but reconstructing a servlet uri-path based on the original one and random regex substitutions to the path segments is outside the scope of httpd or mod_proxy modules (this is also why there is no ProxyPassMatch mapping=servlet so far), at least I wouldn't engage in such dev but since httpd is a participative project anyone is free to propose a patch ;) > Maybe it is acceptable cause the option [PT] is the wrong choice if I want > the request > to be replied by the proxied beta backend. > But some inexperienced admins will run into trouble. (like me ;-) Admins have the choice to use a ProxyPass with mapping=servlet which does simple/prefix rewrite like in: ProxyPass /alpha/gobeta http://server2.localnet:8080/beta mapping=servlet which should be enough for most use cases (including your simple example) without any RewriteRule (besides eventually to prevent this ProxyPass from applying by using some further [R], [F] or (internal) redirect rules, or yet some policies); Or admins can use a ProxyPass[Match] without mapping=servlet and then RewriteRules/, but then they need to take path parameters into account in the regexes (which is not very practical if not impossible in some cases). Best of both worlds where httpd does application specific normalization and restores it across all possible rewrites is not something someone has proposed to implement so far.. Regards; Yann. - To unsubscribe, e-mail: users-unsubscr...@httpd.apache.org For additional commands, e-mail: users-h...@httpd.apache.org
Re: [users@httpd] ProxyPass option mapping=servlet hurts mod_rewrite
Hi Yann, > RewriteRule "^/alpha/gobeta/(.*)$" http://server2.localnet:8080/beta/$1 [P] If the RewriteRule is specified with the option [P] it works correctly. The request https://example.ort/alpha/gobeta/anypath is sent to the beta backend. > Also, it seems to me that "/alpha/gobeta/.." is not matched by the > "ProxyPass /beta ...", so it really boils down to the RewriteRule only. > What do you expect here? But when I take the option [PT] the behavior changes. Without "mapping=servlet" the request is sent to the beta backend. With "mapping=servlet" the request is replied by a 404 NOT FOUND. - Is this change of behavior acceptable? Maybe it is acceptable cause the option [PT] is the wrong choice if I want the request to be replied by the proxied beta backend. But some inexperienced admins will run into trouble. (like me ;-) Regards, Hendrik
Re: [users@httpd] ProxyPass option mapping=servlet hurts mod_rewrite
On Thu, Mar 3, 2022 at 12:24 PM Yann Ylavic wrote: > > Hi Hendrik; > > > > > after reading your commit comment in https://svn.apache.org/r1898509 I > > realised that one important test case is missing: > > What happens if mod_rewrite manipulates the URL of a target that is proxied > > with the option mapping=servlet? > > > > From my point of view this test case fails. :-( > >ProxyPass /beta http://server2.localnet:8080/beta mapping=servlet > >RewirteRule "^/alpha/gobeta" /beta [PT,L] > > Calling https://example.org/alpha/gobeta/test sends back a 404 instead > > the beta content. > > In this case mod_proxy could not map the requests to the beta backend. > > Given that "ProxyPass ... mapping=servlet" applies before the > RewriteRule, I don't see how you can have a RewriteRule [PT] that > either "cancels" the ProxyPass (like in your RewriteMap examples) > and/or that rewrites the uri but keeps proxying (like in the above > example). > > Isn't: > RewriteRule "^/alpha/gobeta/(.*)$" http://server2.localnet:8080/beta/$1 [P] > (or alike) what you are looking for in the above example? Also, it seems to me that "/alpha/gobeta/.." is not matched by the "ProxyPass /beta ...", so it really boils down to the RewriteRule only. What do you expect here? > > Regards; > Yann. - To unsubscribe, e-mail: users-unsubscr...@httpd.apache.org For additional commands, e-mail: users-h...@httpd.apache.org
Re: [users@httpd] ProxyPass option mapping=servlet hurts mod_rewrite
Hi Hendrik; > > after reading your commit comment in https://svn.apache.org/r1898509 I > realised that one important test case is missing: > What happens if mod_rewrite manipulates the URL of a target that is proxied > with the option mapping=servlet? > > From my point of view this test case fails. :-( >ProxyPass /beta http://server2.localnet:8080/beta mapping=servlet >RewirteRule "^/alpha/gobeta" /beta [PT,L] > Calling https://example.org/alpha/gobeta/test sends back a 404 instead the > beta content. > In this case mod_proxy could not map the requests to the beta backend. Given that "ProxyPass ... mapping=servlet" applies before the RewriteRule, I don't see how you can have a RewriteRule [PT] that either "cancels" the ProxyPass (like in your RewriteMap examples) and/or that rewrites the uri but keeps proxying (like in the above example). Isn't: RewriteRule "^/alpha/gobeta/(.*)$" http://server2.localnet:8080/beta/$1 [P] (or alike) what you are looking for in the above example? Regards; Yann. - To unsubscribe, e-mail: users-unsubscr...@httpd.apache.org For additional commands, e-mail: users-h...@httpd.apache.org
Re: [users@httpd] ProxyPass option mapping=servlet hurts mod_rewrite
Hi Yann, after reading your commit comment in https://svn.apache.org/r1898509 I realised that one important test case is missing: What happens if mod_rewrite manipulates the URL of a target that is proxied with the option mapping=servlet? >From my point of view this test case fails. :-( ProxyPass /beta http://server2.localnet:8080/beta mapping=servlet RewirteRule "^/alpha/gobeta" /beta [PT,L] Calling https://example.org/alpha/gobeta/test sends back a 404 instead the beta content. In this case mod_proxy could not map the requests to the beta backend. more details in attachment: mod_rewrite_vs_proxy_pre_trans-v2_test_log2.txt regards, Hendrik ## test set up Server: httpd-2.4.52 with patch mod_rewrite_vs_proxy_pre_trans-v2.diff Hostname: example.org DocumentRoot: /var/www/example/docs ProxyPass /alpha http://server1.localnet:8080/alpha ProxyPass /beta http://server2.localnet:8080/beta mapping=servlet ProxyPass /gamma http://server3.localnet:8080/gamma RewriteRule "^/alpha/gobeta" /beta/ [PT,L] RewriteRule "^/alpha/gogamma" /gamma/ [PT,L] Require all granted Require all granted Require all granted ## test case "rewrite to other backend" # (OK) calling https://example.com/alpha/gogamma/test sends back the gamma content [proxy:trace2] mod_proxy.c(884): AH03461: attempting to match URI path '/alpha/gogamma/test' against prefix '/beta' for proxying [rewrite:trace2] mod_rewrite.c(480): init rewrite engine with requested uri /alpha/gogamma/test. Original filename = n/a [rewrite:trace3] mod_rewrite.c(480): applying pattern '^/alpha/gobeta' to uri '/alpha/gogamma/test' [rewrite:trace3] mod_rewrite.c(480): applying pattern '^/alpha/gogamma' to uri '/alpha/gogamma/test' [rewrite:trace2] mod_rewrite.c(480): rewrite '/alpha/gogamma/test' -> '/gamma/' [rewrite:trace2] mod_rewrite.c(480): forcing '/gamma/' to get passed through to next API URI-to-filename handler [proxy:trace2] mod_proxy.c(884): AH03461: attempting to match URI path '/gamma/' against prefix '/alpha' for proxying [proxy:trace2] mod_proxy.c(884): AH03461: attempting to match URI path '/gamma/' against prefix '/gamma' for proxying [proxy:trace1] mod_proxy.c(1000): AH03464: URI path '/gamma/' matches proxy handler 'proxy:http://server3.localnet:8080/gamma/' [authz_core:debug] mod_authz_core.c(818): AH01626: authorization result of Require all granted: granted [authz_core:debug] mod_authz_core.c(818): AH01626: authorization result of : granted [proxy:trace2] proxy_util.c(2337): http: found worker http://server3.localnet:8080/gamma for http://server3.localnet:8080/gamma/ # (FAILED) calling https://example.com/alpha/gobeta/test sends back a 404 HTTP_NOT_FOUND [proxy:trace2] mod_proxy.c(884): AH03461: attempting to match URI path '/alpha/gobeta/test' against prefix '/beta' for proxying [rewrite:trace2] mod_rewrite.c(480): init rewrite engine with requested uri /alpha/gobeta/test. Original filename = n/a [rewrite:trace3] mod_rewrite.c(480): applying pattern '^/alpha/gobeta' to uri '/alpha/gobeta/test' [rewrite:trace2] mod_rewrite.c(480): rewrite '/alpha/gobeta/test' -> '/beta/' [rewrite:trace2] mod_rewrite.c(480): forcing '/beta/' to get passed through to next API URI-to-filename handler [proxy:trace2] mod_proxy.c(884): AH03461: attempting to match URI path '/beta/' against prefix '/alpha' for proxying [proxy:trace2] mod_proxy.c(884): AH03461: attempting to match URI path '/beta/' against prefix '/gamma' for proxying [authz_core:debug] mod_authz_core.c(818): AH01626: authorization result of Require all granted: granted [authz_core:debug] mod_authz_core.c(818): AH01626: authorization result of : granted [core:info] AH00128: File does not exist: /var/www/example/docs/beta/ - To unsubscribe, e-mail: users-unsubscr...@httpd.apache.org For additional commands, e-mail: users-h...@httpd.apache.org
Re: [users@httpd] ProxyPass option mapping=servlet hurts mod_rewrite
Hi, > > I have applied your patch to my httpd-2.4.52 and created two test cases. > One with a simple RewriteRule and a second one using a RewriteMap. > Both are working fine. :-) Thanks for testing! Now checked in https://svn.apache.org/r1898509 Regards; Yann. - To unsubscribe, e-mail: users-unsubscr...@httpd.apache.org For additional commands, e-mail: users-h...@httpd.apache.org
Re: [users@httpd] ProxyPass option mapping=servlet hurts mod_rewrite
Hi Yann
Anyway, could you please try the attached patch and see if it works for you?
>
I have applied your patch to my httpd-2.4.52 and created two test cases.
One with a simple RewriteRule and a second one using a RewriteMap.
Both are working fine. :-)
Test logs attached: mod_rewrite_vs_proxy_pre_trans-v2_test_log.txt
Thank you for the very quick response!
Regards,
Hendrik
--
--
Hendrik Harms
mail: hendrik.ha...@gmail.com
Test-Setup:
## Both test cases:
httpd-2.4.52 with patch mod_rewrite_vs_proxy_pre_trans-v2.diff
Hostname: example.org
ProxyPass /alpha http://server1.localnet:8080/alpha
ProxyPass /beta http://server2.localnet:8080/beta mapping=servlet
## First test case: "simple RewriteRule"
RewriteRule "^/alpha" - [F]
RewriteRule "^/beta" - [F]
# log extract
# https://example.org/alpha/anypath/
[proxy:trace2] mod_proxy.c(884): AH03461: attempting to match URI path
'/alpha/anypath/' against prefix '/beta' for proxying
[rewrite:trace2] mod_rewrite.c(480): init rewrite engine with requested
uri /alpha/anypath/. Original filename = n/a
[rewrite:trace3] mod_rewrite.c(480): applying pattern '^/alpha' to uri
'/alpha/anypath/'
[rewrite:trace2] mod_rewrite.c(480): forcing responsecode 403 for
/alpha/anypath/
# log extract
# https://example.org/beta/anypath/
[proxy:trace2] mod_proxy.c(884): AH03461: attempting to match URI path
'/beta/anypath/' against prefix '/beta' for proxying
[proxy:trace1] mod_proxy.c(986): AH10248: Servlet path '/beta/anypath/'
(/beta/anypath/) matches proxy handler
'proxy:http://server2.localnet:8080/beta/anypath/'
[rewrite:trace2] mod_rewrite.c(480): init rewrite engine with requested
uri /beta/anypath/. Original filename =
proxy:http://server2.localnet:8080/beta/anypath/
[rewrite:trace3] mod_rewrite.c(480): applying pattern '^/alpha' to uri
'/beta/anypath/'
[rewrite:trace3] mod_rewrite.c(480): applying pattern '^/beta' to uri
'/beta/anypath/'
[rewrite:trace2] mod_rewrite.c(480): forcing responsecode 403 for
/beta/anypath/
## Second test case using RewriteMap to set a maintanance page covering the
backend.
## We could set this maintanance page without restarting the httpd server by
using
## a RewriteMap :-)
# map-maintain.conf
alpha /cover/503.shtml
beta /cover/503.shtml
# httpd.conf e.g. extra/httpd-default.conf
RewriteMap maintain txt:conf/map-maintain.conf
RewriteRule ^/([^/]+) ${maintain:$1|%{REQUEST_URI}} [NC,PT]
# log extract
# https://example.org/alpha/anypath/
[proxy:trace2] mod_proxy.c(884): AH03461: attempting to match URI path
'/alpha/anypath/' against prefix '/beta' for proxying
[rewrite:trace2] mod_rewrite.c(480): init rewrite engine with requested
uri /alpha/anypath/. Original filename = n/a
[rewrite:trace3] mod_rewrite.c(480): applying pattern '^/([^/]+)' to uri
'/alpha/anypath/'
[rewrite:trace6] mod_rewrite.c(480): cache lookup FAILED, forcing new map
lookup
[rewrite:trace5] mod_rewrite.c(480): map lookup OK: map=maintain[txt]
key=alpha -> val=/cover/503.shtml
[rewrite:trace2] mod_rewrite.c(480): rewrite '/alpha/anypath/' ->
'/cover/503.shtml'
[rewrite:trace2] mod_rewrite.c(480): forcing '/cover/503.shtml' to get
passed through to next API URI-to-filename handler
# log extract
# https://example.org/beta/anypath/
[proxy:trace2] mod_proxy.c(884): AH03461: attempting to match URI path
'/beta/anypath/' against prefix '/beta' for proxying
[proxy:trace1] mod_proxy.c(986): AH10248: Servlet path '/beta/anypath/'
(/beta/anypath/) matches proxy handler
'proxy:http://server2.localnet:8080/beta/anypath/'
[rewrite:trace2] mod_rewrite.c(480): init rewrite engine with requested
uri /beta/anypath/. Original filename =
proxy:http://server2.localnet:8080/beta/anypath/
[rewrite:trace3] mod_rewrite.c(480): applying pattern '^/([^/]+)' to uri
'/beta/anypath/'
[rewrite:trace6] mod_rewrite.c(480): cache lookup FAILED, forcing new map
lookup
[rewrite:trace5] mod_rewrite.c(480): map lookup OK: map=maintain[txt]
key=beta -> val=/cover/503.shtml
[rewrite:trace2] mod_rewrite.c(480): rewrite '/beta/anypath/' ->
'/cover/503.shtml'
[rewrite:trace2] mod_rewrite.c(480): forcing '/cover/503.shtml' to get
passed through to next API URI-to-filename handler
-
To unsubscribe, e-mail: users-unsubscr...@httpd.apache.org
For additional commands, e-mail: users-h...@httpd.apache.org
Re: [users@httpd] ProxyPass option mapping=servlet hurts mod_rewrite
Hi Hendrik,
>
> Is this a bug or do I have to use the "mapping=servlet" option very carefully?
I'd say both..
mod_proxy mapping= acts very early in request processing and kind of
"appropriates" the request URI to mod_proxy, confusing mod_rewrite, so
it's probably a bug because your use case is legitimate and should be
handled.
But you'll also have to be careful because early mapping also means
that the normalization applied to the request URI depends on the
mapping, so the URI as seen by mod_rewrite and/or sections
depends on the mapping. For instance mapping=servlet/decoded will not
%-decode the URI internally (besides the "unreserved" characters as
defined by the RFC), so you could have to use the %-encoded form of
some characters in a RewriteRule to match special URIs (this is not
the case in your exemple configuration, I'm warning just in case..).
Anyway, could you please try the attached patch and see if it works for you?
Regards;
Yann.
Index: modules/mappers/mod_rewrite.c
===
--- modules/mappers/mod_rewrite.c (revision 1898463)
+++ modules/mappers/mod_rewrite.c (working copy)
@@ -4576,7 +4576,7 @@ static int hook_uri2file(request_rec *r)
unsigned int port;
int rulestatus;
void *skipdata;
-const char *oargs;
+char *ofilename, *oargs;
/*
* retrieve the config structures
@@ -4629,7 +4629,10 @@ static int hook_uri2file(request_rec *r)
/*
* remember the original query string for later check, since we don't
* want to apply URL-escaping when no substitution has changed it.
+ * also, we'll restore original r->filename if we decline this
+ * request.
*/
+ofilename = r->filename;
oargs = r->args;
/*
@@ -4672,13 +4675,14 @@ static int hook_uri2file(request_rec *r)
apr_table_setn(r->subprocess_env, ENVVAR_SCRIPT_URI, var);
if (!(saved_rulestatus = apr_table_get(r->notes,"mod_rewrite_rewritten"))) {
-/* if filename was not initially set,
- * we start with the requested URI
+/* if r->filename was not initially set, or if proxy_pre_translate_name()
+ * set its reverse "proxy:" URL, we start with the requested URI
*/
-if (r->filename == NULL) {
+if (r->filename == NULL || r->proxyreq == PROXYREQ_REVERSE) {
r->filename = apr_pstrdup(r->pool, r->uri);
-rewritelog((r, 2, NULL, "init rewrite engine with requested uri %s",
-r->filename));
+rewritelog((r, 2, NULL, "init rewrite engine with requested uri "
+"%s. Original filename = %s",
+r->filename, ofilename ? ofilename : "n/a"));
}
else {
rewritelog((r, 2, NULL, "init rewrite engine with passed filename "
@@ -4702,6 +4706,7 @@ static int hook_uri2file(request_rec *r)
if (rulestatus) {
unsigned skip;
apr_size_t flen;
+int to_proxyreq;
if (ACTION_STATUS == rulestatus) {
int n = r->status;
@@ -4711,7 +4716,19 @@ static int hook_uri2file(request_rec *r)
}
flen = r->filename ? strlen(r->filename) : 0;
-if (flen > 6 && strncmp(r->filename, "proxy:", 6) == 0) {
+to_proxyreq = (flen > 6 && strncmp(r->filename, "proxy:", 6) == 0);
+
+/* If a proxy reverse/pre_trans filename was rewritten to a new uri
+ * this is not a proxy request anymore.
+ */
+if (r->proxyreq == PROXYREQ_REVERSE && !to_proxyreq) {
+if (r->handler && strcmp(r->handler, "proxy-server") == 0) {
+r->handler = NULL;
+}
+r->proxyreq = PROXYREQ_NONE;
+}
+
+if (to_proxyreq) {
/* it should be go on as an internal proxy request */
/* check if the proxy module is enabled, so
@@ -4735,9 +4752,7 @@ static int hook_uri2file(request_rec *r)
r->filename = apr_pstrcat(r->pool, r->filename,
r->path_info, NULL);
}
-if ((r->args != NULL)
-&& ((r->proxyreq == PROXYREQ_PROXY)
-|| (rulestatus == ACTION_NOESCAPE))) {
+if (r->args && (r->proxyreq || rulestatus == ACTION_NOESCAPE)) {
/* see proxy_http:proxy_http_canon() */
r->filename = apr_pstrcat(r->pool, r->filename,
"?", r->args, NULL);
@@ -4878,7 +4893,9 @@ static int hook_uri2file(request_rec *r)
}
}
else {
-rewritelog((r, 1, NULL, "pass through %s", r->filename));
+rewritelog((r, 1, NULL, "pass through %s (%s)",
+r->filename, ofilename));
+r->filename = ofilename;
return DECLINED;
}
}
@@ -5213,7 +5230,8 @@ static int hook_fixup(request_rec *r)
}
}
else {
-rewritelog((r, 1,
[users@httpd] ProxyPass option mapping=servlet hurts mod_rewrite
Hi httpd users, After adding the mapping=servlet option in the ProxyPass configuration, my RewriteRule no longer works as desired. Example: Hostname: example.org ProxyPass /alpha http://server1.localnet:8080/alpha ProxyPass /beta http://server2.localnet:8080/beta mapping=servlet RewriteRule ^/alpha - [F] RewriteRule ^/beta - [F] Calling https://example.org/alpha/anypath/ sends back a 403 Forbidden like desired. Calling https://example.org/beta/anypath/ sends back the beta content. The RewriteRule does not catch the request. I've increased the Logging LogLevel proxy:trace3 LogLevel rewrite:trace3 Calling the alpha url I see this in my error.log: [proxy:trace2] mod_proxy.c(884): AH03461: attempting to match URI path '/alpha/anypath/' against prefix '/beta' for proxying [rewrite:trace2] mod_rewrite.c(480): init rewrite engine with requested uri /alpha/anypath/ [rewrite:trace3] mod_rewrite.c(480): applying pattern '^/alpha' to uri '/alpha/anypath/' [rewrite:trace2] mod_rewrite.c(480): forcing responsecode 403 for /alpha/anypath/ Calling the beta url I see this in my log: [proxy:trace2] mod_proxy.c(884): AH03461: attempting to match URI path '/beta/anypath/' against prefix '/beta' for proxying [proxy:trace1] mod_proxy.c(986): AH10248: Servlet path '/beta/anypath/' (/beta/anypath/) matches proxy handler 'proxy: http://server2.localnet:8080/beta/anypath/' [rewrite:trace2] mod_rewrite.c(480): init rewrite engine with passed filename proxy:http://server2.localnet:8080/beta/anypath/. Original uri = /beta/anypath/ [rewrite:trace3] mod_rewrite.c(480): applying pattern '^/alpha' to uri 'proxy:http://server2.localnet:8080/beta/anypath/' [rewrite:trace3] mod_rewrite.c(480): applying pattern '^/beta' to uri 'proxy:http://server2.localnet:8080/beta/anypath/' [rewrite:trace1] mod_rewrite.c(480): pass through proxy: http://server2.localnet:8080/beta/anypath/ [proxy_http:trace1] mod_proxy_http.c(98): HTTP: canonicalising URL http://server2.localnet:8080/beta/anypath/ [proxy:trace2] proxy_util.c(2337): http: found worker http://server2.localnet:8080/beta for http://server2.localnet:8080/beta/anypath/ Is this a bug or do I have to use the "mapping=servlet" option very carefully? regards, Hendrik
