Re: Tomcat closes connections on unexpected status codes
Op do 18 apr 2024 om 17:42 schreef Mark Thomas : > On 18/04/2024 15:18, Stefan Ansing wrote: > > Hi Rémy, Mark, > > > > > > > > I just want to make sure that we’re understanding each other. I can see > > that the connection needs to be closed in certain conditions to prevent > > request smuggling attacks. I certainly don’t want to change that > behaviour. > > > > However, I’m facing a scenario where an application is responding to a > > valid request (from HTTP perspective), with a valid response using these > > status codes (more specifically status codes 400 and 500). > > If the request is a valid HTTP request then a 400 status doesn't seem > appropriate to me. > > If the server is correctly handling that request to generate the > response, a 500 status doesn't seem right either. > > > > > I don’t think that in this scenario a request smuggling attack could be > > executed, or am I missing something? > > The main issue is if the original request is invalid HTTP there is no > way to determine where the next HTTP request starts. > > If there is a proxy in the mix then the risks of something going wrong > tend to go up. > > Mark > > - > To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org > For additional commands, e-mail: users-h...@tomcat.apache.org > > Hi Mark, I can see your point of view regarding the use of the status codes for application errors. Unfortunately the HTTP spec doesn't clearly differentiate the use of status codes for protocol or application errors. Which is probably why these status codes are now also commonly used for application errors. Tomcat (and other servlet containers) currently allow applications to set any status code, but with the current behaviour of Tomcat this leads to unexpected side effects for some status codes. This behaviour makes it so that Tomcat might not be fit for our purpose (Spring Boot services to services communications). I think the way to resolve that is to alter the behaviour in Tomcat to differentiate between protocol and application errors when using these status codes (and to make this behaviour potentially configurable). I also think that this change would benefit most users of Tomcat since the behaviour in this scenario is unnecessary. Would the Tomcat developers be willing to do that? Stefan
Re: Tomcat closes connections on unexpected status codes
Op do 18 apr 2024 om 15:41 schreef Rémy Maucherat : > On Thu, Apr 18, 2024 at 1:17 PM Mark Thomas wrote: > > > > On 18/04/2024 09:07, Stefan Ansing wrote: > > > Hi, > > > > > > We've observed some unexpected behaviour in Apache Tomcat (version > 10.1.19) > > > where we see that HTTP/1.1 connections are closed whenever a servlet > > > application returns the following status codes: 400, 408, 411, 414, > 500, > > > 503, 501. This causes client applications to rapidly reconnect and > induce > > > high server-side CPU load due to doing TLS handshakes. > > > > > > The 400 and 500 status codes are commonly used in RESTful > microservices to > > > communicate errors. Reviewing RFC 9112 I couldn't find any requirement > for > > > closing connections on these status codes. > > > > > > After testing with Undertow (version 2.3.12), where we didn't see the > same > > > behaviour, we believe that these status codes do not necessitate a new > > > connection. > > > > The Tomcat developers disagree. Connections are closed after these > > status codes to avoid various forms of request smuggling attacks. > > > > > Checking the Tomcat sources makes me believe that the behaviour is > > > hard-coded[1]. I'm reaching out here to re-evaluate the list of status > > > codes and to discuss the possibilities of making the behaviour > configurable. > > > > Making this list of status codes configurable seems reasonable. The > > default can stay as current and if users want to change it then they > > have to accept the associated security risks. > > If it's insecure, then it would still be a valid CVE even if the > configuration is optional. We don't have an "allowSmuggling" attribute > on the connector to relax header or status line parsing, even though > many would have wanted it in the past ... > > Rémy > > > Mark > > > > > > > > > > A colleague of mine reported a bug for this issue: > > > https://bz.apache.org/bugzilla/show_bug.cgi?id=68901 > > > > > > Kind regards, > > > Stefan Ansing > > > > > > [1]: > > > > https://github.com/apache/tomcat/blame/bc900e0100de9879604b93af4722c272ab3d1a24/java/org/apache/coyote/http11/Http11Processor.java#L604-L617 > > > > > > > - > > To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org > > For additional commands, e-mail: users-h...@tomcat.apache.org > > > > - > To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org > For additional commands, e-mail: users-h...@tomcat.apache.org > > Hi Rémy, Mark, I just want to make sure that we’re understanding each other. I can see that the connection needs to be closed in certain conditions to prevent request smuggling attacks. I certainly don’t want to change that behaviour. However, I’m facing a scenario where an application is responding to a valid request (from HTTP perspective), with a valid response using these status codes (more specifically status codes 400 and 500). I don’t think that in this scenario a request smuggling attack could be executed, or am I missing something? Stefan
Tomcat closes connections on unexpected status codes
Hi, We've observed some unexpected behaviour in Apache Tomcat (version 10.1.19) where we see that HTTP/1.1 connections are closed whenever a servlet application returns the following status codes: 400, 408, 411, 414, 500, 503, 501. This causes client applications to rapidly reconnect and induce high server-side CPU load due to doing TLS handshakes. The 400 and 500 status codes are commonly used in RESTful microservices to communicate errors. Reviewing RFC 9112 I couldn't find any requirement for closing connections on these status codes. After testing with Undertow (version 2.3.12), where we didn't see the same behaviour, we believe that these status codes do not necessitate a new connection. Checking the Tomcat sources makes me believe that the behaviour is hard-coded[1]. I'm reaching out here to re-evaluate the list of status codes and to discuss the possibilities of making the behaviour configurable. A colleague of mine reported a bug for this issue: https://bz.apache.org/bugzilla/show_bug.cgi?id=68901 Kind regards, Stefan Ansing [1]: https://github.com/apache/tomcat/blame/bc900e0100de9879604b93af4722c272ab3d1a24/java/org/apache/coyote/http11/Http11Processor.java#L604-L617
