Re: BUG? Tomcat-4.0.3 eats PathInfo slashes, TDK leaves them alone
[Followup to my own post; I did some digging and now understand why my app works under TDK2 but not Tomcat; perhaps this in the archive will help others who encounter this. I expect it will most often bite people working on things like the Commons httpclient, or proxies.] I was using PathInfo to hold URLs my app wants to proxy. I found that multiple slashes, like http://localhost:8080/gov.nasa.hq.sna.intranetbroker.Proxy/http://example.co m got transmogrified to collapse the double-slash: http://localhost:8080/gov.nasa.hq.sna.intranetbroker.Proxy/http://example.co m I tried a workaround to url-encode my target URL, but Tomcat rejected the URI early on, logging the rejection to catalina_log: 2002-03-06 23:53:43 HttpProcessor[58080][4] Invalid request URI: '/sna/servlet/gov.nasa.hq.sna.intranetbroker.Proxy/http%3A%2F%2Fexample.com' catalina/src/share/org/apache/catalina/connector/http/HttpProcessor.java class version 1.29 added a collapse the slashes, transform dot-slash and backslash types of strings. This is responsible for my first problem. Version 1.30 added the change which causes it to reject URL-encoded slashes, along with dots, backslashes, and percents. This is responsible for the second problem. I expect these were both done to prevent URL-based attacks which might cause the server to wander around the filesystem where it's not supposed to be. Unfortunately, it breaks my app even though I'm not accessing the filesystem -- just trying to proxy to other sites. Only in part. It was also a big security hole for mapping and security constraints (you could bypass authentication and access protected areas using '/' encoding). BTW, you have to encode '/' and ':' in your URI, as you did in the second case. The first URL with unencoded special chars is invalid (ie, it may work, but it's not sure). These changes have deeper implications for folks writing HTTP clients and proxies. If the client's desired target URL has any of the characters [/\.%] in them and they've been URL-encoded (perhaps by the previous page on the target server) the entire URI will be rejected. Similarly, if it has unencoded sequences of these (like the double-slash) the target URL will be damaged. I understand, but this won't be fixed in 4.0.x, as it is way too risky. We could consider fixing it in 4.1, since we'll have the benefit of going through a whole beta phase. I'm not sure what the answer is. I don't know what types of problems this normalization is designed to prevent. Perhaps the normalization could be made more conservative, changing only the part up to the ContextPath and ServletPath, and leave PathInfo and QueryString unchanged. Is there anything I could do to help here? I'm not a stud Java coder, but I'd be happy to contribute. Remy -- To unsubscribe: mailto:[EMAIL PROTECTED] For additional commands: mailto:[EMAIL PROTECTED] Troubles with the list: mailto:[EMAIL PROTECTED]
Re: BUG? Tomcat-4.0.3 eats PathInfo slashes, TDK leaves them alone
Remy Maucherat [EMAIL PROTECTED] writes:
BTW, you have to encode '/' and ':' in your URI, as you did in the second
case. The first URL with unencoded special chars is invalid (ie, it may
work, but it's not sure).
But if I encode `/' (as %2F), then HttpProcessor.normalize() will
reject it:
// Prevent encoding '%', '/', '.' and '\', which are special reserved
// characters
if ((normalized.indexOf(%25) = 0)
|| (normalized.indexOf(%2F) = 0)
|| (normalized.indexOf(%2E) = 0)
|| (normalized.indexOf(%5C) = 0)
|| (normalized.indexOf(%2f) = 0)
|| (normalized.indexOf(%2e) = 0)
|| (normalized.indexOf(%5c) = 0)) {
return null;
}
and you're rejecting URIs with this encoded-slash to avoid bypassing
security. So it sounds like there's no real way to pass in a slash.
Sorry if I'm being stupid and not seeing the right way to do this.
I understand, but this won't be fixed in 4.0.x, as it is way too risky.
We could consider fixing it in 4.1, since we'll have the benefit of going
through a whole beta phase.
OK, thanks for the feedback. If I can help, let me know.
--
To unsubscribe: mailto:[EMAIL PROTECTED]
For additional commands: mailto:[EMAIL PROTECTED]
Troubles with the list: mailto:[EMAIL PROTECTED]
Re: BUG? Tomcat-4.0.3 eats PathInfo slashes, TDK leaves them alone
[Followup to my own post; I did some digging and now understand why my app works under TDK2 but not Tomcat; perhaps this in the archive will help others who encounter this. I expect it will most often bite people working on things like the Commons httpclient, or proxies.] I was using PathInfo to hold URLs my app wants to proxy. I found that multiple slashes, like http://localhost:8080/gov.nasa.hq.sna.intranetbroker.Proxy/http://example.com got transmogrified to collapse the double-slash: http://localhost:8080/gov.nasa.hq.sna.intranetbroker.Proxy/http://example.com I tried a workaround to url-encode my target URL, but Tomcat rejected the URI early on, logging the rejection to catalina_log: 2002-03-06 23:53:43 HttpProcessor[58080][4] Invalid request URI: '/sna/servlet/gov.nasa.hq.sna.intranetbroker.Proxy/http%3A%2F%2Fexample.com' catalina/src/share/org/apache/catalina/connector/http/HttpProcessor.java class version 1.29 added a collapse the slashes, transform dot-slash and backslash types of strings. This is responsible for my first problem. Version 1.30 added the change which causes it to reject URL-encoded slashes, along with dots, backslashes, and percents. This is responsible for the second problem. I expect these were both done to prevent URL-based attacks which might cause the server to wander around the filesystem where it's not supposed to be. Unfortunately, it breaks my app even though I'm not accessing the filesystem -- just trying to proxy to other sites. These changes have deeper implications for folks writing HTTP clients and proxies. If the client's desired target URL has any of the characters [/\.%] in them and they've been URL-encoded (perhaps by the previous page on the target server) the entire URI will be rejected. Similarly, if it has unencoded sequences of these (like the double-slash) the target URL will be damaged. I'm not sure what the answer is. I don't know what types of problems this normalization is designed to prevent. Perhaps the normalization could be made more conservative, changing only the part up to the ContextPath and ServletPath, and leave PathInfo and QueryString unchanged. Is there anything I could do to help here? I'm not a stud Java coder, but I'd be happy to contribute. -- To unsubscribe: mailto:[EMAIL PROTECTED] For additional commands: mailto:[EMAIL PROTECTED] Troubles with the list: mailto:[EMAIL PROTECTED]
Re: BUG? Tomcat-4.0.3 eats PathInfo slashes, TDK leaves them alone
Remy Maucherat [EMAIL PROTECTED] writes:
BTW, you have to encode '/' and ':' in your URI, as you did in the
second
case. The first URL with unencoded special chars is invalid (ie, it may
work, but it's not sure).
But if I encode `/' (as %2F), then HttpProcessor.normalize() will
reject it:
// Prevent encoding '%', '/', '.' and '\', which are special
reserved
// characters
if ((normalized.indexOf(%25) = 0)
|| (normalized.indexOf(%2F) = 0)
|| (normalized.indexOf(%2E) = 0)
|| (normalized.indexOf(%5C) = 0)
|| (normalized.indexOf(%2f) = 0)
|| (normalized.indexOf(%2e) = 0)
|| (normalized.indexOf(%5c) = 0)) {
return null;
}
and you're rejecting URIs with this encoded-slash to avoid bypassing
security. So it sounds like there's no real way to pass in a slash.
Sorry if I'm being stupid and not seeing the right way to do this.
I was just saying that the '/' should be encoded to comply with the
standard. Tomcat will indeed reject that URL, I know that.
I don't plan to put that normalization code in the new HTTP connector, so
we'll see how many security problems it create. I think it would be a lot
safer to keep the current HTTP connector frozen, and don't fix that issue
there at all.
Remy
--
To unsubscribe: mailto:[EMAIL PROTECTED]
For additional commands: mailto:[EMAIL PROTECTED]
Troubles with the list: mailto:[EMAIL PROTECTED]
BUG? Tomcat-4.0.3 eats PathInfo slashes, TDK leaves them alone
I'm developing a webapp under TDK which gets a target URL in the
PathInfo. This is working fine in TDK. But when I move the webapp to
standalone Tomcat-4.0.3, I see that Tomcat is gratuitously eating
slashes in PathInfo, so it's compressing stuff like
http://example.com
to
http:/example.com
and breaking my app.
I wrote a dinky test case, attached below, but it basically just takes
the HttpServletRequest and writes out its URL pieces, the most
important being req.getPathInfo().
My test URL is:
http://localhost:8080/sna/servlet/gov.nasa.hq.sna.intranetbroker.SlashTest/oj///simpson?lawyer=cochran
When I run it under TDK, the result is:
Method: GET
Scheme: http
ServerName: localhost
ServerPort: 8080
ContextPath: /sna
ServletPath: /servlet/gov.nasa.hq.sna.intranetbroker.SlashTest
RequestURL:
http://localhost:8080/sna/servlet/gov.nasa.hq.sna.intranetbroker.SlashTest/oj///simpson
RequestURI: /sna/servlet/gov.nasa.hq.sna.intranetbroker.SlashTest/oj///simpson
QueryString: lawyer=cochran
PathInfo: /oj///simpson
When I do the same with Tomcat (on port 58080), I get:
Method: GET
Scheme: http
ServerName: localhost
ServerPort: 58080
ContextPath: /sna
ServletPath: /servlet/gov.nasa.hq.sna.intranetbroker.SlashTest
RequestURL:
http://localhost:58080/sna/servlet/gov.nasa.hq.sna.intranetbroker.SlashTest/oj/simpson
RequestURI: /sna/servlet/gov.nasa.hq.sna.intranetbroker.SlashTest/oj/simpson
QueryString: lawyer=cochranfoo
PathInfo: /oj/simpson
Note that it's modifying what it reports as the incoming URL, in
PathInfo, RequestURL, and RequestURI. That shouldn't be happening,
should it? Something not quite right in Tomcat?
(I don't want to have to URL-encode this, because I've run into
problems before -- my app is a proxy and url-encoded FORM variable
using METHOD=GET get trashed.)
Suggestions? Thanks.
I'm including the test code, in case maybe I'm doing something
stupid. The list manager didn't allow posting as an attachment, so
I'll just inline the body here.
public class SlashTest extends HttpServlet
{
public void doGet(HttpServletRequest req,
HttpServletResponse res)
throws java.io.IOException
{
res.setContentType(text/plain);
PrintWriter out = res.getWriter();
out.println( \n Method:+ req.getMethod()
+ \n Scheme:+ req.getScheme()
+ \n ServerName:+ req.getServerName()
+ \n ServerPort:+ req.getServerPort()
+ \n ContextPath: + req.getContextPath()
+ \n ServletPath: + req.getServletPath()
+ \n RequestURL:+ req.getRequestURL()
+ \n RequestURI:+ req.getRequestURI()
+ \n QueryString: + req.getQueryString()
+ \n PathInfo: + req.getPathInfo()
);
}
}
--
To unsubscribe: mailto:[EMAIL PROTECTED]
For additional commands: mailto:[EMAIL PROTECTED]
Troubles with the list: mailto:[EMAIL PROTECTED]
Re: BUG? Tomcat-4.0.3 eats PathInfo slashes, TDK leaves them alone
Good work. With this description, the developers should be able to quickly
identify the bug and squash it. I recommend you submit this as a problem in
the bugs database.
Chris Shenton wrote:
I'm developing a webapp under TDK which gets a target URL in the
PathInfo. This is working fine in TDK. But when I move the webapp to
standalone Tomcat-4.0.3, I see that Tomcat is gratuitously eating
slashes in PathInfo, so it's compressing stuff like
http://example.com
to
http:/example.com
and breaking my app.
I wrote a dinky test case, attached below, but it basically just takes
the HttpServletRequest and writes out its URL pieces, the most
important being req.getPathInfo().
My test URL is:
http://localhost:8080/sna/servlet/gov.nasa.hq.sna.intranetbroker.SlashTest/oj///simpson?lawyer=cochran
When I run it under TDK, the result is:
Method: GET
Scheme: http
ServerName: localhost
ServerPort: 8080
ContextPath: /sna
ServletPath: /servlet/gov.nasa.hq.sna.intranetbroker.SlashTest
RequestURL:
http://localhost:8080/sna/servlet/gov.nasa.hq.sna.intranetbroker.SlashTest/oj///simpson
RequestURI: /sna/servlet/gov.nasa.hq.sna.intranetbroker.SlashTest/oj///simpson
QueryString: lawyer=cochran
PathInfo: /oj///simpson
When I do the same with Tomcat (on port 58080), I get:
Method: GET
Scheme: http
ServerName: localhost
ServerPort: 58080
ContextPath: /sna
ServletPath: /servlet/gov.nasa.hq.sna.intranetbroker.SlashTest
RequestURL:
http://localhost:58080/sna/servlet/gov.nasa.hq.sna.intranetbroker.SlashTest/oj/simpson
RequestURI: /sna/servlet/gov.nasa.hq.sna.intranetbroker.SlashTest/oj/simpson
QueryString: lawyer=cochranfoo
PathInfo: /oj/simpson
Note that it's modifying what it reports as the incoming URL, in
PathInfo, RequestURL, and RequestURI. That shouldn't be happening,
should it? Something not quite right in Tomcat?
(I don't want to have to URL-encode this, because I've run into
problems before -- my app is a proxy and url-encoded FORM variable
using METHOD=GET get trashed.)
Suggestions? Thanks.
I'm including the test code, in case maybe I'm doing something
stupid. The list manager didn't allow posting as an attachment, so
I'll just inline the body here.
public class SlashTest extends HttpServlet
{
public void doGet(HttpServletRequest req,
HttpServletResponse res)
throws java.io.IOException
{
res.setContentType(text/plain);
PrintWriter out = res.getWriter();
out.println( \n Method:+ req.getMethod()
+ \n Scheme:+ req.getScheme()
+ \n ServerName:+ req.getServerName()
+ \n ServerPort:+ req.getServerPort()
+ \n ContextPath: + req.getContextPath()
+ \n ServletPath: + req.getServletPath()
+ \n RequestURL:+ req.getRequestURL()
+ \n RequestURI:+ req.getRequestURI()
+ \n QueryString: + req.getQueryString()
+ \n PathInfo: + req.getPathInfo()
);
}
}
--
To unsubscribe: mailto:[EMAIL PROTECTED]
For additional commands: mailto:[EMAIL PROTECTED]
Troubles with the list: mailto:[EMAIL PROTECTED]
--
To unsubscribe: mailto:[EMAIL PROTECTED]
For additional commands: mailto:[EMAIL PROTECTED]
Troubles with the list: mailto:[EMAIL PROTECTED]
Re: BUG? Tomcat-4.0.3 eats PathInfo slashes, TDK leaves them alone
Good work. With this description, the developers should be able to
quickly
identify the bug and squash it. I recommend you submit this as a problem
in
the bugs database.
It's already there (bug 4707), but the problem is that this URL
normalization was introduced to counter some URL based attacks used to get
around security constraints.
It is also forbidden to URLEncode the '/' character (again to counter the
same type of attacks).
Due to the high risk, it won't be fixed in the 4.0.x branch (I don't want to
have to make emergency releases too often ;-)).
Remy
Chris Shenton wrote:
I'm developing a webapp under TDK which gets a target URL in the
PathInfo. This is working fine in TDK. But when I move the webapp to
standalone Tomcat-4.0.3, I see that Tomcat is gratuitously eating
slashes in PathInfo, so it's compressing stuff like
http://example.com
to
http:/example.com
and breaking my app.
I wrote a dinky test case, attached below, but it basically just takes
the HttpServletRequest and writes out its URL pieces, the most
important being req.getPathInfo().
My test URL is:
http://localhost:8080/sna/servlet/gov.nasa.hq.sna.intranetbroker.SlashTest/
oj///simpson?lawyer=cochran
When I run it under TDK, the result is:
Method: GET
Scheme: http
ServerName: localhost
ServerPort: 8080
ContextPath: /sna
ServletPath: /servlet/gov.nasa.hq.sna.intranetbroker.SlashTest
RequestURL:
http://localhost:8080/sna/servlet/gov.nasa.hq.sna.intranetbroker.SlashTest/o
j///simpson
RequestURI:
/sna/servlet/gov.nasa.hq.sna.intranetbroker.SlashTest/oj///simpson
QueryString: lawyer=cochran
PathInfo: /oj///simpson
When I do the same with Tomcat (on port 58080), I get:
Method: GET
Scheme: http
ServerName: localhost
ServerPort: 58080
ContextPath: /sna
ServletPath: /servlet/gov.nasa.hq.sna.intranetbroker.SlashTest
RequestURL:
http://localhost:58080/sna/servlet/gov.nasa.hq.sna.intranetbroker.SlashTest/
oj/simpson
RequestURI:
/sna/servlet/gov.nasa.hq.sna.intranetbroker.SlashTest/oj/simpson
QueryString: lawyer=cochranfoo
PathInfo: /oj/simpson
Note that it's modifying what it reports as the incoming URL, in
PathInfo, RequestURL, and RequestURI. That shouldn't be happening,
should it? Something not quite right in Tomcat?
(I don't want to have to URL-encode this, because I've run into
problems before -- my app is a proxy and url-encoded FORM variable
using METHOD=GET get trashed.)
Suggestions? Thanks.
I'm including the test code, in case maybe I'm doing something
stupid. The list manager didn't allow posting as an attachment, so
I'll just inline the body here.
public class SlashTest extends HttpServlet
{
public void doGet(HttpServletRequest req,
HttpServletResponse res)
throws java.io.IOException
{
res.setContentType(text/plain);
PrintWriter out = res.getWriter();
out.println( \n Method:+ req.getMethod()
+ \n Scheme:+ req.getScheme()
+ \n ServerName:+ req.getServerName()
+ \n ServerPort:+ req.getServerPort()
+ \n ContextPath: + req.getContextPath()
+ \n ServletPath: + req.getServletPath()
+ \n RequestURL:+ req.getRequestURL()
+ \n RequestURI:+ req.getRequestURI()
+ \n QueryString: + req.getQueryString()
+ \n PathInfo: + req.getPathInfo()
);
}
}
--
To unsubscribe: mailto:[EMAIL PROTECTED]
For additional commands: mailto:[EMAIL PROTECTED]
Troubles with the list: mailto:[EMAIL PROTECTED]
--
To unsubscribe: mailto:[EMAIL PROTECTED]
For additional commands: mailto:[EMAIL PROTECTED]
Troubles with the list: mailto:[EMAIL PROTECTED]
--
To unsubscribe: mailto:[EMAIL PROTECTED]
For additional commands: mailto:[EMAIL PROTECTED]
Troubles with the list: mailto:[EMAIL PROTECTED]
