Re: Improving mod_jk URI forwarding
Rainer Jung wrote: Mladen Turk wrote: Can you write a simple example of the uris that make the problem and why we would need to encode the %. How it is passed now, and how it would be passed with your proposal. Original URI: /myapp/%252e%252e/otherapp/danger JkMount /myapp/* Apache httpd will correctly decode the URI to /myapp/%2e%2e/otherapp/danger mod_jk does map it *correctly* to /myapp and forwards it to Tomcat. It does not IMO, and that's what I'm talking. Inside mod_jk we should decode /myapp/%2e%2e/otherapp/danger to /myapp/../otherapp/danger Do a normalization of the uri that will end up as /otherapp/danger before hitting map_uri_to_worker If there is no JkMount for /otherapp/ it will be denied, if it is, the rewritten uri /otherapp/danger will be send instead /myapp/%2e%2e/otherapp/danger. Of course we can simply send /myapp/%2e%2e/otherapp/danger to tomcat if the match is OK for /otherapp/, and let the tomcat do a normalization once again. In that case we won't need to encode the normalized uri inside mod_jk once more. Regards, Mladen. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Improving mod_jk URI forwarding
Before I answer, let me first ask a question: What's wrong withg my
suggestion? Or even better: use the encoding done with mod_proxy_ajp?
Original URI:
/myapp/%252e%252e/otherapp/danger
JkMount /myapp/*
Apache httpd will correctly decode the URI to
/myapp/%2e%2e/otherapp/danger
mod_jk does map it *correctly* to /myapp and forwards it to Tomcat.
It does not IMO, and that's what I'm talking.
Inside mod_jk we should decode
/myapp/%2e%2e/otherapp/danger to
/myapp/../otherapp/danger
No, If the original URI was /myapp/%252e%252e/otherapp/danger, then it
is not correct to end up with /otherapp/danger as a decoded URL. A
percent sign is a valid character in a ressource path. If one wants to
use it in ressource paths, one needs to encode it ('%25'), and it is not
allowed to decode '%25XX' again after decoding to '%XX' once.
So %252e - %2e and that's it, no further decoding. It is not a '.',
because it is decoded already.
Why do you think, that
/myapp/%252e%252e/otherapp/danger
is equivalent to
/myapp/../otherapp/danger ?
Do a normalization of the uri that will end up as
/otherapp/danger before hitting map_uri_to_worker
If there is no JkMount for /otherapp/ it will be
denied, if it is, the rewritten uri /otherapp/danger
will be send instead /myapp/%2e%2e/otherapp/danger.
Of course we can simply send /myapp/%2e%2e/otherapp/danger
to tomcat if the match is OK for /otherapp/,
and let the tomcat do a normalization once again.
As I said, double decoding (again) is not allowed and is the source of
all evil.
In that case we won't need to encode the normalized
uri inside mod_jk once more.
I'm sure we need to, because double encoding is not allowed, but Tomcat
unfortunately does a second decoding.
Some evidence:
0) RFC 2396, RFC 3986
=
http://www.ietf.org/rfc/rfc2396.txt
Section 2.4.2. When to Escape and Unescape:
Implementers should be careful not to
escape or unescape the same string more than once, since unescaping
an already unescaped string might lead to misinterpreting a percent
data character as another escaped character, or vice versa in the
case of escaping an already escaped string.
and even stricter the younger
http://tools.ietf.org/html/rfc3986
2.4. When to Encode or Decode
Implementations must not
percent-encode or decode the same string more than once, as decoding
an already decoded string might lead to misinterpreting a percent
data octet as the beginning of a percent-encoding, or vice versa in
the case of percent-encoding an already percent-encoded string.
1) IIS Bug
==
http://www.cve.mitre.org/cgi-bin/cvename.cgi?name=CAN-2001-0333
which leads to
http://www.microsoft.com/technet/security/Bulletin/MS01-026.mspx
What's wrong with the decoding operation?
There's nothing wrong with the decoding operation per se. The
vulnerability results because, through an implementation flaw, the
decoding operation is performed a second time, after the security checks
on the request have been completed.
2) mod_security Bug
===
http://www.modsecurity.org/download/CHANGES
BUG Fixed a double URL-decoding bug (Apache first, then us), which
could sometimes lead to a false positive.
3) DAV Bugs
===
http://dav.sourceforge.net/
The following bugs should have gone:
* [ 1519718 ] davfs2 fails to properly decode complex escape sequences
* [ 1522903 ] chokes on directory names containing ' _ % characters
* [ 1539444 ] mounting of webdav drive fails
* [ 1539445 ] unable to access files in mounted webdav drive
* [ 1558525 ] davfs2-1.0.2_p20060820 mount fails
These bugs were related to ... and incorrect double url-decoding of urls.
Regards,
Rainer
-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
Re: Improving mod_jk URI forwarding
On Sun, 2007-05-20 at 18:17 +0200, Rainer Jung wrote:
Before I answer, let me first ask a question: What's wrong withg my
suggestion? Or even better: use the encoding done with mod_proxy_ajp?
For me there is nothing wrong except it adds 2 JKoptions or 3 :-)
We know that Tomcat is going to normalise a url we have already
normalised. Shouldn't we check that a second normalisation (like the
Tomcat one) gives a different url and if yes have a flag to return
forbidden? (Yes that would be a 4th option).
Cheers
Jean-Frederic
Original URI:
/myapp/%252e%252e/otherapp/danger
JkMount /myapp/*
Apache httpd will correctly decode the URI to
/myapp/%2e%2e/otherapp/danger
mod_jk does map it *correctly* to /myapp and forwards it to Tomcat.
It does not IMO, and that's what I'm talking.
Inside mod_jk we should decode
/myapp/%2e%2e/otherapp/danger to
/myapp/../otherapp/danger
No, If the original URI was /myapp/%252e%252e/otherapp/danger, then it
is not correct to end up with /otherapp/danger as a decoded URL. A
percent sign is a valid character in a ressource path. If one wants to
use it in ressource paths, one needs to encode it ('%25'), and it is not
allowed to decode '%25XX' again after decoding to '%XX' once.
So %252e - %2e and that's it, no further decoding. It is not a '.',
because it is decoded already.
Why do you think, that
/myapp/%252e%252e/otherapp/danger
is equivalent to
/myapp/../otherapp/danger ?
Do a normalization of the uri that will end up as
/otherapp/danger before hitting map_uri_to_worker
If there is no JkMount for /otherapp/ it will be
denied, if it is, the rewritten uri /otherapp/danger
will be send instead /myapp/%2e%2e/otherapp/danger.
Of course we can simply send /myapp/%2e%2e/otherapp/danger
to tomcat if the match is OK for /otherapp/,
and let the tomcat do a normalization once again.
As I said, double decoding (again) is not allowed and is the source of
all evil.
In that case we won't need to encode the normalized
uri inside mod_jk once more.
I'm sure we need to, because double encoding is not allowed, but Tomcat
unfortunately does a second decoding.
Some evidence:
0) RFC 2396, RFC 3986
=
http://www.ietf.org/rfc/rfc2396.txt
Section 2.4.2. When to Escape and Unescape:
Implementers should be careful not to
escape or unescape the same string more than once, since unescaping
an already unescaped string might lead to misinterpreting a percent
data character as another escaped character, or vice versa in the
case of escaping an already escaped string.
and even stricter the younger
http://tools.ietf.org/html/rfc3986
2.4. When to Encode or Decode
Implementations must not
percent-encode or decode the same string more than once, as decoding
an already decoded string might lead to misinterpreting a percent
data octet as the beginning of a percent-encoding, or vice versa in
the case of percent-encoding an already percent-encoded string.
1) IIS Bug
==
http://www.cve.mitre.org/cgi-bin/cvename.cgi?name=CAN-2001-0333
which leads to
http://www.microsoft.com/technet/security/Bulletin/MS01-026.mspx
What's wrong with the decoding operation?
There's nothing wrong with the decoding operation per se. The
vulnerability results because, through an implementation flaw, the
decoding operation is performed a second time, after the security checks
on the request have been completed.
2) mod_security Bug
===
http://www.modsecurity.org/download/CHANGES
BUG Fixed a double URL-decoding bug (Apache first, then us), which
could sometimes lead to a false positive.
3) DAV Bugs
===
http://dav.sourceforge.net/
The following bugs should have gone:
* [ 1519718 ] davfs2 fails to properly decode complex escape sequences
* [ 1522903 ] chokes on directory names containing ' _ % characters
* [ 1539444 ] mounting of webdav drive fails
* [ 1539445 ] unable to access files in mounted webdav drive
* [ 1558525 ] davfs2-1.0.2_p20060820 mount fails
These bugs were related to ... and incorrect double url-decoding of urls.
Regards,
Rainer
-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
Re: Improving mod_jk URI forwarding
Jean-Frederic wrote: On Sun, 2007-05-20 at 18:17 +0200, Rainer Jung wrote: Before I answer, let me first ask a question: What's wrong withg my suggestion? Or even better: use the encoding done with mod_proxy_ajp? For me there is nothing wrong except it adds 2 JKoptions or 3 :-) If we think the new way is the correct way, we could have it as the default, letting the old ones there for compatibility with existing configurations as non standard options. The new one would not need an explicit name if it gets to be the standard. We know that Tomcat is going to normalise a url we have already normalised. Shouldn't we check that a second normalisation (like the Tomcat one) gives a different url and if yes have a flag to return forbidden? (Yes that would be a 4th option). Here I try to argue, that encoding '%' before forwarding and decoding by tomcat should lead to the identity operation. I though a little more about it and most likely encoding '+' is also necessary, because besides the '%' decoding, Tomcat will most likely (I have to check) also decode '+' - ' '. At the end I tend to use the same function, that's used in mod_proxy_ajp to reencode before forwarding (although only encoding '%' and '+' would be faster). Rejecting requests with double encoding can already be done by mod_rewrite, because mod_rewrite operates on the decoded URI, so you only need to check for '%' (and '+' if you like). Regards, Rainer - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Improving mod_jk URI forwarding
On Sun, 2007-05-20 at 18:17 +0200, Rainer Jung wrote:
Before I answer, let me first ask a question: What's wrong withg my
suggestion?
Well thinking to it I am not very happy with the change:
+++
-#define JK_OPT_FWDURIDEFAULTJK_OPT_FWDURICOMPATUNPARSED
+#define JK_OPT_FWDURIDEFAULTJK_OPT_FWDURIESCAPEDMINIMAL
+++
Because that is what the SPEC's says.
Cheers
Jean-Frederic
Or even better: use the encoding done with mod_proxy_ajp?
Original URI:
/myapp/%252e%252e/otherapp/danger
JkMount /myapp/*
Apache httpd will correctly decode the URI to
/myapp/%2e%2e/otherapp/danger
mod_jk does map it *correctly* to /myapp and forwards it to Tomcat.
It does not IMO, and that's what I'm talking.
Inside mod_jk we should decode
/myapp/%2e%2e/otherapp/danger to
/myapp/../otherapp/danger
No, If the original URI was /myapp/%252e%252e/otherapp/danger, then it
is not correct to end up with /otherapp/danger as a decoded URL. A
percent sign is a valid character in a ressource path. If one wants to
use it in ressource paths, one needs to encode it ('%25'), and it is not
allowed to decode '%25XX' again after decoding to '%XX' once.
So %252e - %2e and that's it, no further decoding. It is not a '.',
because it is decoded already.
Why do you think, that
/myapp/%252e%252e/otherapp/danger
is equivalent to
/myapp/../otherapp/danger ?
Do a normalization of the uri that will end up as
/otherapp/danger before hitting map_uri_to_worker
If there is no JkMount for /otherapp/ it will be
denied, if it is, the rewritten uri /otherapp/danger
will be send instead /myapp/%2e%2e/otherapp/danger.
Of course we can simply send /myapp/%2e%2e/otherapp/danger
to tomcat if the match is OK for /otherapp/,
and let the tomcat do a normalization once again.
As I said, double decoding (again) is not allowed and is the source of
all evil.
In that case we won't need to encode the normalized
uri inside mod_jk once more.
I'm sure we need to, because double encoding is not allowed, but Tomcat
unfortunately does a second decoding.
Some evidence:
0) RFC 2396, RFC 3986
=
http://www.ietf.org/rfc/rfc2396.txt
Section 2.4.2. When to Escape and Unescape:
Implementers should be careful not to
escape or unescape the same string more than once, since unescaping
an already unescaped string might lead to misinterpreting a percent
data character as another escaped character, or vice versa in the
case of escaping an already escaped string.
and even stricter the younger
http://tools.ietf.org/html/rfc3986
2.4. When to Encode or Decode
Implementations must not
percent-encode or decode the same string more than once, as decoding
an already decoded string might lead to misinterpreting a percent
data octet as the beginning of a percent-encoding, or vice versa in
the case of percent-encoding an already percent-encoded string.
1) IIS Bug
==
http://www.cve.mitre.org/cgi-bin/cvename.cgi?name=CAN-2001-0333
which leads to
http://www.microsoft.com/technet/security/Bulletin/MS01-026.mspx
What's wrong with the decoding operation?
There's nothing wrong with the decoding operation per se. The
vulnerability results because, through an implementation flaw, the
decoding operation is performed a second time, after the security checks
on the request have been completed.
2) mod_security Bug
===
http://www.modsecurity.org/download/CHANGES
BUG Fixed a double URL-decoding bug (Apache first, then us), which
could sometimes lead to a false positive.
3) DAV Bugs
===
http://dav.sourceforge.net/
The following bugs should have gone:
* [ 1519718 ] davfs2 fails to properly decode complex escape sequences
* [ 1522903 ] chokes on directory names containing ' _ % characters
* [ 1539444 ] mounting of webdav drive fails
* [ 1539445 ] unable to access files in mounted webdav drive
* [ 1558525 ] davfs2-1.0.2_p20060820 mount fails
These bugs were related to ... and incorrect double url-decoding of urls.
Regards,
Rainer
-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
Re: Improving mod_jk URI forwarding
Well thinking to it I am not very happy with the change: +++ -#define JK_OPT_FWDURIDEFAULTJK_OPT_FWDURICOMPATUNPARSED +#define JK_OPT_FWDURIDEFAULTJK_OPT_FWDURIESCAPEDMINIMAL +++ Because that is what the SPEC's says. I doubt, that the SPEC is very relevant for the reverse proxy situation of mod_jk. I read the old discussions, and I really got the impression that not much of SPEC compliance was left over. What are we really talking about w.r.t. SPECs? Regards, Rainer - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Improving mod_jk URI forwarding
Jean-Frederic wrote: On Sun, 2007-05-20 at 18:17 +0200, Rainer Jung wrote: Before I answer, let me first ask a question: What's wrong withg my suggestion? Well thinking to it I am not very happy with the change: +++ -#define JK_OPT_FWDURIDEFAULTJK_OPT_FWDURICOMPATUNPARSED +#define JK_OPT_FWDURIDEFAULTJK_OPT_FWDURIESCAPEDMINIMAL +++ Because that is what the SPEC's says. The Servlet SPEC says, that the web container should not decode RequestURI. So if the reverse proxy tries to come close to this requirement, it needs to send the URI as unencoded as it can. Now the reverse proxy should have the ability to modify the URI (in the sense of mod_rewrite). If we accept, that mod_rewrite in httpd 1.3-2.2 is only able to operate on the decoded URI, we have no chance of making this interoperable with forwarding the original undecoded URI. Using the encoding from mod_proxy_ajp we at least try to produce a URI which is as close to the original one, as we can achieve. At least logically the new URI is fine. All in all I think that compat unparsed will result in more user problems as default then the new proposed way. The whole discussion is: 1) At the moment all options have a problem (mod_rewrite, sessions, security) 2) My proposal should be checked if it is secure. If so, it fixes mod_rewrite, sessions and security. This seems to have some value. If you (community) agree on that value, I find it correct to add this option. The answers might depend on - encode only '%' - encode '%' and '+' (easy, performing, but needs some thinking about correctness) - encode like in mod_proxy_ajp (not as performing but more complete) 3) Yes, it will influence how well the container can obey the spec (as two of the three existing options also do). So we could discuss, if the value is huge enough, that we make it the default (and thus need no more explicit option name), or we keep compat unparsed the default and add a new option. Regards, Rainer - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Improving mod_jk URI forwarding
Rainer Jung [EMAIL PROTECTED] wrote in message news:[EMAIL PROTECTED] Jean-Frederic wrote: On Sun, 2007-05-20 at 18:17 +0200, Rainer Jung wrote: Before I answer, let me first ask a question: What's wrong withg my suggestion? Well thinking to it I am not very happy with the change: +++ -#define JK_OPT_FWDURIDEFAULTJK_OPT_FWDURICOMPATUNPARSED +#define JK_OPT_FWDURIDEFAULTJK_OPT_FWDURIESCAPEDMINIMAL +++ Because that is what the SPEC's says. The Servlet SPEC says, that the web container should not decode RequestURI. So if the reverse proxy tries to come close to this requirement, it needs to send the URI as unencoded as it can. Now the reverse proxy should have the ability to modify the URI (in the sense of mod_rewrite). If we accept, that mod_rewrite in httpd 1.3-2.2 is only able to operate on the decoded URI, we have no chance of making this interoperable with forwarding the original undecoded URI. Yes, RFC 2616 permits *any* intermediate proxy to alter the URI in *any* way before passing it on. So to be meaningful, the Servlet spec requirement can only apply to the URI that TC receives from the last proxy in the chain. Using the encoding from mod_proxy_ajp we at least try to produce a URI which is as close to the original one, as we can achieve. At least logically the new URI is fine. All in all I think that compat unparsed will result in more user problems as default then the new proposed way. The whole discussion is: 1) At the moment all options have a problem (mod_rewrite, sessions, security) 2) My proposal should be checked if it is secure. If so, it fixes mod_rewrite, sessions and security. This seems to have some value. If you (community) agree on that value, I find it correct to add this option. The answers might depend on - encode only '%' - encode '%' and '+' (easy, performing, but needs some thinking about correctness) We pass the original query string to TC, so there is no point in worrying about '+' (since it only means ' ' in the qs). - encode like in mod_proxy_ajp (not as performing but more complete) 3) Yes, it will influence how well the container can obey the spec (as two of the three existing options also do). So we could discuss, if the value is huge enough, that we make it the default (and thus need no more explicit option name), or we keep compat unparsed the default and add a new option. Regards, Rainer - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Improving mod_jk URI forwarding
Rainer Jung wrote:
Before I answer, let me first ask a question: What's wrong withg my
suggestion? Or even better: use the encoding done with mod_proxy_ajp?
Because it doesn't solve the real problem.
Original URI:
/myapp/%252e%252e/otherapp/danger
JkMount /myapp/*
Apache httpd will correctly decode the URI to
/myapp/%2e%2e/otherapp/danger
mod_jk does map it *correctly* to /myapp and forwards it to Tomcat.
It does not IMO, and that's what I'm talking.
Inside mod_jk we should decode
/myapp/%2e%2e/otherapp/danger to
/myapp/../otherapp/danger
No, If the original URI was /myapp/%252e%252e/otherapp/danger, then it
is not correct to end up with /otherapp/danger as a decoded URL.
Yes, but it was already rewritten by apache to
/myapp/%2e%2e/otherapp/danger
If we send that to Tomcat because we mapped to /myapp/*
that it's a security breakage.
A percent sign is a valid character in a ressource path. If one wants to
use it in ressource paths, one needs to encode it ('%25'), and it is not
allowed to decode '%25XX' again after decoding to '%XX' once.
So %252e - %2e and that's it, no further decoding. It is not a '.',
because it is decoded already.
Why do you think, that
/myapp/%252e%252e/otherapp/danger
is equivalent to
/myapp/../otherapp/danger ?
Because it is on the Tomcat side and it ends up
as /otherapp/danger.
Now, your suggestion would send an faulty uri for
something that shouldn't be passed by the mod_jk at
the first place, because *we know* how the uri will
be decoded on Tomcat.
Regards,
Mladen.
-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
Re: Improving mod_jk URI forwarding
Bill Barker wrote: Now the reverse proxy should have the ability to modify the URI (in the sense of mod_rewrite). If we accept, that mod_rewrite in httpd 1.3-2.2 is only able to operate on the decoded URI, we have no chance of making this interoperable with forwarding the original undecoded URI. Yes, RFC 2616 permits *any* intermediate proxy to alter the URI in *any* way before passing it on. So to be meaningful, the Servlet spec requirement can only apply to the URI that TC receives from the last proxy in the chain. Exactly my point. We can rewrite the uri in what ever way we wish as long it is RFC compliant uri. If that means temporary normalizing uri and looking for a malicious requests like /foo/../bar/ what's the problem? We know that this particular request will end up as /bar/ on Tomcat side, so I really see no reason why we shouldn't try to detect that and then if we have JkMount /bar/* we can send as original uri either /bar/ or /foo/../bar/ At the end the Tomcat will serve the same resource. Regards, Mladen. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Improving mod_jk URI forwarding
Why do you think, that /myapp/%252e%252e/otherapp/danger is equivalent to /myapp/../otherapp/danger ? Because it is on the Tomcat side and it ends up as /otherapp/danger. No it is not. If you send /myapp/%252e%252e/otherapp/danger to Tomcat, it will map the URL into the /myapp context (which is correct). Now, your suggestion would send an faulty uri for something that shouldn't be passed by the mod_jk at the first place, because *we know* how the uri will be decoded on Tomcat. Unless we change the AJP connector, there are only three possibilities: - either we iterate over URL decoding unless no more '%' signs are in it - or we deny all URLs with '%25' in them - or we reencode the URL As I understand you, the first two are more what you are about. But this will not allow any requests to be processed, which have a percent character as part of their ressource description. But those URLs are valid! Once again: I think that reencoding and afterwards decoding by Tomcat is the identity operation. So we are in the very god situation, that all mod_jk/httpd logic has been done against the same URL, that will be used by Tomcat. I tried to provide some evidence, why a *decoded* URL of the form /myapp/%2e%2e/otherapp/danger is valid, is unequal to /otherapp/danger and belongs to the /myapp context. Also I showed, why double decoding usually is assumed to be bad practise. Assume there s some content scanning in front of Apache and we double decode, then the same problem will be between the component in front and us. We will end up by simply denying everything with '%25' in it and I find that much more inadequate then reencoding the URL. Why do you think it is a faulty uri and we shouldn't pass it along? Simply reencode and erything is fine. Regards, Rainer - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Improving mod_jk URI forwarding
Rainer Jung wrote: Hi, now that we changed the default way how to forward URIs from mod_jk to Tomcat (mod_jk 1.2.23) because of a directory traversal issue, I want to propose a better long term solution. My proposal is that we make our own decoder if the URI is encoded and then do a match and forward that. Hope I don't speak complete rubbish :) Anyhow, I simply don't like adding new features and options. Think we done a lot of them recently. If there is a problem we should hide the problem from the user and do that internally instead adding new config. Regards, Mladen. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Improving mod_jk URI forwarding
Mladen Turk wrote: My proposal is that we make our own decoder if the URI is encoded and then do a match and forward that. As far as I understand you suggestion, this would not help. There's nothing wrong with our decoder (the httpd decoder), what's wrong is, that the decoded URI gets decoded a second time by Tomcat. Double decoding is the fault (there's a nice comment about that in httpd source code). Hope I don't speak complete rubbish :) Anyhow, I simply don't like adding new features and options. Think we done a lot of them recently. If there is a problem we should hide the problem from the user and do that internally instead adding new config. I think the new proposed way of forwarding (the '%' encoding) is the one, which will efficiently hide the problem. The point why we can't just let things stay like they are now is, - one option does not work with mod_rewrite - one option is not safe - one option does not work with URL encoded sessions IDs This is not new, it's like that since a long time. What is new, is that we now take the unsafety serious and end up with either broken sessions or broken mod_rewrite interop. Regards, Rainer - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Improving mod_jk URI forwarding
On Sat, 2007-05-19 at 14:27 +0200, Rainer Jung wrote: Hi, now that we changed the default way how to forward URIs from mod_jk to Tomcat (mod_jk 1.2.23) because of a directory traversal issue, I want to propose a better long term solution. What's the problem? === I think we still miss something... The mapping (map_uri_to_worker) uses r-uri. Should we use s-req_uri or a carefully normalised uri to do the mapping? Fixing s-req_uri doesn't help to fix the mapping (where the problem comes from). Cheers Jean-Frederic - Access control is often defined in terms of URI prefixes, because they are mapped to web applications - URIs can contain character sequences that change the URI prefix (like /../ - URIs can contain encoded characters The general rule to apply access control is to first decode encoded characters, then normalize the path by resolving the special character sequences and then apply access control. In a system with cascaded components we might check access control on a frontend component but finally process the request on a backend component. The backend component might not know about the frontend component and thus again decode and normalize the request URI. Now it becomes important, that the result of this decode and normalize on the backend does not differ from decode and normalize on the frontend, which did the access control check. It is especially important, that decoding is not done twice in a row, because then a malicious user could use double encoded special character sequences. How do we handle it in mod_jk = By default, we now chose to forward the original undecoded and unnormalized URL via mod_jk to Tomcat. Unfortunately this has serious drawbacks, because all internal Apache httpd processing is done on the decoded normalized URL. One such example is mod_rewrite. Combinations of mod_rewrite and mod_jk are typical for more complex sites. mod_rewrite is only able to rewrite the decoded normalized URI, so any rewriting gets obsolete if mod_jk forwards the original request URI. The other two existing options in mod_jk are forwarding the decoded request (which results in the double encode issue) and forwarding an encoded form of the decoded request. The last option sounds interesting, but has a massive drawback. It uses Apache httpd encode function, which also encodes ';' as %3B. But at the moment the AJP connector on the Tomcat side looks for session IDs in the request URI before decoding the URI. It only checks for ;jsessionid= and not for %3bjsessionid= or %3Bjsessionid=. What do I propose = My proposal is to use the encoded URI to forward to Tomcat. This should be safe, because encoding before forwarding and afterwards decoding in Tomcat should be the identity. To keep mod_jk compatible with the existing AJP connector, I suggest to use a variant of ForwardURIEscaped, where we escape all the characters we escape with ForwardURIEscaped, except for the semicolon ';'. This should still be safe, because the percent sign is the decoding indicator and we wil still encode '%' as '%25' to prevent double decoding. Another option would be to only encode '%' when forwarding. For the same reasons this should be save and will produce less overhead. I did a test implementation for the both approaches. Patches are under http://people.apache.org/~rjung/mod_jk-dev/URIForwarding/ (for 1.2.23 and for trunk). The new options are JkOption ForwardURIEscapedPartial (do not escape ';') and JkOption ForwardURIEscapedMinimal (only escape '%'). I only tested the patch against trunk, but the 1.2.23 port I coded is completely straightforward, so it should also work. Could you please think about the problem and test if you still find a way to break the partial or minimal escaping solution? No directory traversal should be possible and both should work well with mod_rewrite and URL encoded session IDs. Maybe you can think of other ways breaking the combination mod_jk/Tomcat, which are not related to double encoding. Since we have 1.2.23 out, there is no hurry. I would welcome comments on the safety of this approach. Regards, Rainer - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Improving mod_jk URI forwarding
Jean-Frederic wrote: On Sat, 2007-05-19 at 14:27 +0200, Rainer Jung wrote: Hi, now that we changed the default way how to forward URIs from mod_jk to Tomcat (mod_jk 1.2.23) because of a directory traversal issue, I want to propose a better long term solution. What's the problem? === I think we still miss something... The mapping (map_uri_to_worker) uses r-uri. Should we use s-req_uri or a carefully normalised uri to do the mapping? Fixing s-req_uri doesn't help to fix the mapping (where the problem comes from). Yes, the mapping uses r-uri, and it has to, because the maping must go against a uudecoded normalized URI. But to be consistent with Tomcats view of the URI, we should send a URI to Tomcat, which gets interpreted in the same way. My proposal is to send r-uri to Tomcat with encoded '%'. That way decoding by Tomcat should result in the same URI r-uri and since this one was already normalized by Apache, Tomcat normalization should not change it any more. So fixing s-req_uri in my opinion does help to fix the mapping (which was correct!), because it prevents Tomcat from changing the URI in a wrong way. Regards, Rainer - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Improving mod_jk URI forwarding
Rainer Jung wrote: Mladen Turk wrote: My proposal is that we make our own decoder if the URI is encoded and then do a match and forward that. As far as I understand you suggestion, this would not help. There's nothing wrong with our decoder (the httpd decoder), what's wrong is, that the decoded URI gets decoded a second time by Tomcat. Double decoding is the fault (there's a nice comment about that in httpd source code). You got me wrong. I suggest we decode the encoded uri, do mapping, remove ;jsessionid=xxx and send that to the Tomcat. This way tomcat won't have double encoding issue. And it's completely legitimate if we comply to the RFC. This would also solve malicious mapping attempts like /app1/../app2 before they even hit tomcat. Regards, Mladen. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Improving mod_jk URI forwarding
On Sat, 2007-05-19 at 18:57 +0200, Rainer Jung wrote: Jean-Frederic wrote: On Sat, 2007-05-19 at 14:27 +0200, Rainer Jung wrote: Hi, now that we changed the default way how to forward URIs from mod_jk to Tomcat (mod_jk 1.2.23) because of a directory traversal issue, I want to propose a better long term solution. What's the problem? === I think we still miss something... The mapping (map_uri_to_worker) uses r-uri. Should we use s-req_uri or a carefully normalised uri to do the mapping? Fixing s-req_uri doesn't help to fix the mapping (where the problem comes from). Yes, the mapping uses r-uri, and it has to, because the maping must go against a uudecoded normalized URI. What about url like /context-a/../context-b/? There could be a problem if the goal is not to map /context-b. Should we normalise /context-a/../context-b/ to /context-b and then do the mapping. Cheers Jean-frederic But to be consistent with Tomcats view of the URI, we should send a URI to Tomcat, which gets interpreted in the same way. My proposal is to send r-uri to Tomcat with encoded '%'. That way decoding by Tomcat should result in the same URI r-uri and since this one was already normalized by Apache, Tomcat normalization should not change it any more. So fixing s-req_uri in my opinion does help to fix the mapping (which was correct!), because it prevents Tomcat from changing the URI in a wrong way. Regards, Rainer - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Improving mod_jk URI forwarding
Jean-Frederic wrote: What about url like /context-a/../context-b/? There could be a problem if the goal is not to map /context-b. Should we normalise /context-a/../context-b/ to /context-b and then do the mapping. Yes. It would require some programming of course, but it'll solve the issues without adding extra config options. Regards, Mladen. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Improving mod_jk URI forwarding
Mladen Turk wrote: You got me wrong. I suggest we decode the encoded uri, do mapping, remove ;jsessionid=xxx and send that to the Tomcat. This way tomcat won't have double encoding issue. And it's completely legitimate if we comply to the RFC. This would also solve malicious mapping attempts like /app1/../app2 before they even hit tomcat. It would not help. Tomcat *does* another decoding in the connector! If the URI contains %252e%252e (example) httpd correctly decodes this as %2e%2e and it is correct to not decode this a second time. httpd and mod_jk do not normalize this away after the first decoding, because it is not equal to ... So if we forward the decoded and normalized URI to Tomcat, the Tomcat connector decodes a second time and then this will be .. and normalized away on the Tomcat side. Furthermore: I don't understand your comment about removing ;jsessionid. We cannot remove it, this will kill the session. But as you know that yourself I must have misunderstood your first sentence. Regards, Rainer - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Improving mod_jk URI forwarding
Jean-Frederic wrote: What about url like /context-a/../context-b/? There could be a problem if the goal is not to map /context-b. Should we normalise /context-a/../context-b/ to /context-b and then do the mapping. If the *original* URI is /context-a/../context-b/ then apache httpd normalizes this (correctly) to /context-b/. If JkMount matches /conext-a we don't forward, if it matches /context-b we forward. I don't get the point here. What kind of problem (encoding, decoding, mod_jk, Tomcat) do you want to produce with sch a URI? By the way: for such a simple URI httpd and Tomcat both do the same normalization (resulting in /context-b). - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Improving mod_jk URI forwarding
Mladen Turk wrote: Jean-Frederic wrote: What about url like /context-a/../context-b/? There could be a problem if the goal is not to map /context-b. Should we normalise /context-a/../context-b/ to /context-b and then do the mapping. Yes. It would require some programming of course, but it'll solve the issues without adding extra config options. ?? This normalization is already done by httpd and used in mod_jk, totally independent of the existing forwarding option. Request without any special characters are not my concern here. For those simple requests of course all existing forwarding options result in the same forwarded URI. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Improving mod_jk URI forwarding
Rainer Jung wrote: Mladen Turk wrote: You got me wrong. I suggest we decode the encoded uri, do mapping, remove ;jsessionid=xxx and send that to the Tomcat. This way tomcat won't have double encoding issue. And it's completely legitimate if we comply to the RFC. This would also solve malicious mapping attempts like /app1/../app2 before they even hit tomcat. It would not help. Tomcat *does* another decoding in the connector! Look, what I'm saying is that we should simplify all the JkOptions ForwardURI* . IMHO they all originate from the fact that uri in the Apache can come from multiple pre-processing stages that modify the original URI. The solution is very simple but it would require that we write the URI decoder. When the uri comes to the mod_jk before doing any mapping or anything else we should decode that uri and then use it, and send that uri to the Tomcat (the one we rewrote). Regards, Mladen. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Improving mod_jk URI forwarding
Look, what I'm saying is that we should simplify all the JkOptions ForwardURI* . IMHO they all originate from the fact that uri in the Apache can come from multiple pre-processing stages that modify the original URI. The solution is very simple but it would require that we write the URI decoder. When the uri comes to the mod_jk before doing any mapping or anything else we should decode that uri and then use it, and send that uri to the Tomcat (the one we rewrote). OK Mladen, I understand that, but I think it's not correct. I read the discussion threads from 2001 which let to those options and they come from the fact, that different Tomcat version handle the forwarded URIs in different ways. I think this part of the discussion is in fact only historic and yes I agree we don't actually need those options any more. But: none of the existing options does the right thing. That's why I suggested another way of handling the forward. I think my sugeggested variant forward r-uri with encoded '%' is the right way of handling it. And no: I don't think we need our own decoder. Apache http decoder is OK (as long as we talk about httpd). It does nothing wrong. For instance it would be very bad to instead decode iteratively until not more '%' are in the URL, that's definitely wrong. Decoding and forwarding the result to Tomcat is exactly our default until 1.2.22 and produces the security problem. There's nothing wrong with the decoder, only with decoding two times, once in mod_jk and a second time in Tomcat. Regards, Rainer - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Improving mod_jk URI forwarding
Rainer Jung wrote: OK Mladen, I understand that, but I think it's not correct. Might be. But: none of the existing options does the right thing. That's why I suggested another way of handling the forward. I think my sugeggested variant forward r-uri with encoded '%' is the right way of handling it. So, what is the actual problem. If there is something that is potentially malicious we should deal with that without any specific additional config. Can you write a simple example of the uris that make the problem and why we would need to encode the %. How it is passed now, and how it would be passed with your proposal. Regards, Mladen. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Improving mod_jk URI forwarding
Mladen Turk wrote: Rainer Jung wrote: OK Mladen, I understand that, but I think it's not correct. Might be. But: none of the existing options does the right thing. That's why I suggested another way of handling the forward. I think my sugeggested variant forward r-uri with encoded '%' is the right way of handling it. So, what is the actual problem. If there is something that is potentially malicious we should deal with that without any specific additional config. Can you write a simple example of the uris that make the problem and why we would need to encode the %. How it is passed now, and how it would be passed with your proposal. Original URI: /myapp/%252e%252e/otherapp/danger JkMount /myapp/* Apache httpd will correctly decode the URI to /myapp/%2e%2e/otherapp/danger mod_jk does map it *correctly* to /myapp and forwards it to Tomcat. How does it forward it? That depends on the chosen forwarding option. Old default: forwarding as /myapp/%2e%2e/otherapp/danger Tomcat's AJP connector now does an automatic decoding of the request, which ends up as /myapp/../otherapp/danger. Then Tomcat normalizes this as /otherapp/danger and we serve content from a context we didn't want to. The other options we have at the moment: New default: forwarding the original request URI /myapp/%252e%252e/otherapp/danger. Tomcat does decoding to /myapp/%2e%2e/otherapp/danger and there is no normalization possible. So Tomcat checks for something called %2e%2e/otherapp/danger in the context /myapp and will most likely find nothing, so throw an error. 3rd option (escaped forwarding): will forward /myapp/%252e%252e/otherapp/danger, so the same result as with our new default (OK). The problem is: the new default ignores rewrites done by mod_rewrite and a lot of people use mod_rewrite together with mod_jk. The third option, that also fixes the problem will break any normal request with a jsessionid: /myapp/valid;jsessionid=12345 gets forwarded as /myapp/valid%3Bjsessionid=12345 and Tomcat's AJP connector does not detect the jsessionid and Tomcat actually looks for a ressource valid%3Bjsessionid=12345 in /myapp. My proposal: forward the httpd internal URI with only encoded '%': So we would forward /myapp/%252e%252e/otherapp/danger but keep compatibility with mod_rewrite and with jsessionid. Of course one could als improve the situation by changing the AJP connector implementation on the Tomcat side and in the long run this might be interesting, but I find my suggestion better, because: - it will be compatible with existing Tomcat AJP connector - it should solve al three problems (security, mod_rewrite, jsessionid) - it is not heavy on performance, we only need to do something if the request after httpd decoding still has a '%' in it. What I would like very much, if anybody would think about URIs maybe breaking this %' encoding construction. I personally think it's safe, because encoding '%' in a decoded URI and afterwards decoding again should behave as the identity on the URI. So Tomcat at the end should work on the same URI, that we have used for our matching. Regards, Rainer - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Improving mod_jk URI forwarding
Maybe that helps: mod_proxy_ajp (httpd 2.2.4) does something very similar. In mod_proxy_ajp.c function proxy_ajp_canon() calls ap_proxy_canonenc(). This function for a reverse proxy request does encode a couple of chars in the already decoded URI before forwarding it. It encodes all chars except: a-zA-Z0-9 (alnum) and $-_.+!*'(),;:@=. So '%' will be encoded before forwardin and ';' not. This behaviour fits to my theory, but none of our options are close to this. We could also borrow the behaviour/code from there. Regards, Rainer - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
