Re: [EXTERNAL] Re: Request header too large..
On 06/11/2018 00:02, Amit Pande wrote: > Thanks Mark for the update. Seems like there is a potential bug in our client > components which happened to keep on adding to existing header list. > > https://tomcat.apache.org/tomcat-8.5-doc/config/http.html > > From the documentation: > > maxHttpHeaderSize :The maximum size of the request and response HTTP header, > specified in bytes. If not specified, this attribute is set to 8192 (8 KB). > > Although some online searches (e.g. > https://serverfault.com/questions/56691/whats-the-maximum-url-length-in-tomcat) > suggest that this maxHttpHeaderSize applies to request line as well, could > the documentation be made more elaborate? > > Can we mention that this attribute puts the upper bound on combined size of > all request headers and the request line? Sure. Patch / pull request welcome. Mark > > Thanks, > Amit > > On 11/3/18, 4:39 AM, "Mark Thomas" wrote: > > On 03/11/2018 01:35, Amit Pande wrote: > > Thanks Chris. Yes, I will soon send out the patch for review. > > Logically, I can see some hurdles for such a patch that are going to be > difficult to overcome. > > Tomcat will process all of the received data up to the limit. It is on > the subsequent read to get more data that the exception is thrown. > > The issue is how to determine which header is actually the problem as it > may not be the one currently being read. > > Consider the following: > > ReqLine - 1024 bytes > Headers - 5 * 128 bytes > Header - 1 * 6*1024 bytes > Headers - 5 * 128 bytes > > Is the problem the 6k header? The 1k request line? Has the client > duplicated the block of 5 128 byte headers? Or is the request valid and > Tomcat's limit too low? > > There is no way for Tomcat to determine where the root cause lies. The > only reliable way to figure out what went wrong is for a knowledgeable > system admin to look at the complete set of headers. > > Mark > > - > To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org > For additional commands, e-mail: users-h...@tomcat.apache.org > > > > > - > To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org > For additional commands, e-mail: users-h...@tomcat.apache.org > - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: [EXTERNAL] Re: Request header too large..
Thanks Mark for the update. Seems like there is a potential bug in our client components which happened to keep on adding to existing header list. https://tomcat.apache.org/tomcat-8.5-doc/config/http.html From the documentation: maxHttpHeaderSize :The maximum size of the request and response HTTP header, specified in bytes. If not specified, this attribute is set to 8192 (8 KB). Although some online searches (e.g. https://serverfault.com/questions/56691/whats-the-maximum-url-length-in-tomcat) suggest that this maxHttpHeaderSize applies to request line as well, could the documentation be made more elaborate? Can we mention that this attribute puts the upper bound on combined size of all request headers and the request line? Thanks, Amit On 11/3/18, 4:39 AM, "Mark Thomas" wrote: On 03/11/2018 01:35, Amit Pande wrote: > Thanks Chris. Yes, I will soon send out the patch for review. Logically, I can see some hurdles for such a patch that are going to be difficult to overcome. Tomcat will process all of the received data up to the limit. It is on the subsequent read to get more data that the exception is thrown. The issue is how to determine which header is actually the problem as it may not be the one currently being read. Consider the following: ReqLine - 1024 bytes Headers - 5 * 128 bytes Header - 1 * 6*1024 bytes Headers - 5 * 128 bytes Is the problem the 6k header? The 1k request line? Has the client duplicated the block of 5 128 byte headers? Or is the request valid and Tomcat's limit too low? There is no way for Tomcat to determine where the root cause lies. The only reliable way to figure out what went wrong is for a knowledgeable system admin to look at the complete set of headers. Mark - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: [EXTERNAL] Re: Request header too large..
On 03/11/2018 01:35, Amit Pande wrote: > Thanks Chris. Yes, I will soon send out the patch for review. Logically, I can see some hurdles for such a patch that are going to be difficult to overcome. Tomcat will process all of the received data up to the limit. It is on the subsequent read to get more data that the exception is thrown. The issue is how to determine which header is actually the problem as it may not be the one currently being read. Consider the following: ReqLine - 1024 bytes Headers - 5 * 128 bytes Header - 1 * 6*1024 bytes Headers - 5 * 128 bytes Is the problem the 6k header? The 1k request line? Has the client duplicated the block of 5 128 byte headers? Or is the request valid and Tomcat's limit too low? There is no way for Tomcat to determine where the root cause lies. The only reliable way to figure out what went wrong is for a knowledgeable system admin to look at the complete set of headers. Mark - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: [EXTERNAL] Re: Request header too large..
Thanks Chris. Yes, I will soon send out the patch for review. Thanks, Amit On 11/2/18, 5:25 PM, "Christopher Schultz" wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA256 Amit, On 11/2/18 17:16, Amit Pande wrote: > As per current implementation (below snippet is from 8.5.28), if > the request header is too large (by default >8K, the default > maxHttpHeaderSize), is thrown below error message is seen. > > > For request -- 02-Nov-2018 15:15:47.649 INFO [catalina-exec-40] > org.apache.coyote.http11.Http11Processor.service Error parsing HTTP > request header Note: further occurrences of HTTP header parsing > errors will be logged at DEBUG level. > java.lang.IllegalArgumentException: Request header is too large at > org.apache.coyote.http11.Http11InputBuffer.parseHeaders(Http11InputBuf fer.java:575) > > at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:70 3) > at > org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLigh t.java:66) > > at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractPro tocol.java:790) > > For response— > > 02-Nov-2018 15:15:47.867 SEVERE [catalina-exec-40] > org.apache.coyote.http11.Http11Processor.endRequest Error finishing > response org.apache.coyote.http11.HeadersTooLargeException: An > attempt was made to write more data to the response headers than > there was room available in the buffer. Increase maxHttpHeaderSize > on the connector or write less data into the response headers. at > org.apache.coyote.http11.Http11OutputBuffer.checkLengthBeforeWrite(Htt p11OutputBuffer.java:544) > > at org.apache.coyote.http11.Http11OutputBuffer.write(Http11OutputBuffer.jav a:481) > at > org.apache.coyote.http11.Http11OutputBuffer.sendStatus(Http11OutputBuf fer.java:361) > > at org.apache.coyote.http11.Http11Processor.prepareResponse(Http11Processor .java:1339) > at > org.apache.coyote.AbstractProcessor.action(AbstractProcessor.java:260) > > at org.apache.coyote.http11.Http11Processor.endRequest(Http11Processor.java :1523) > > > However, this doesn’t tell us the name of the offending header > which I think is valuable in debugging the issues. > > Do you feel it’s worth the enhancement request? > > Also, I noticed the comments > > “ // (1) Headers plus request line size does not exceed its limit > > > > // (2) There are enough bytes to avoid expanding the buffer when > > > // reading body “ > > It says request line size ..does that mean maxHttpHeaderSize comes > in play for request line (which includes query params) as well? Yes. The request headers are read before selecting which context will service the request. The request line, which not formally a part of the "headers" per the HTTP spec, are considered to be "part of the headers" as far as Tomcat is concerned, because they all share the same buffer. To locate the offending header, look at the end of the request, then re-wind to the previous "\r\nheader:". If you don't find "\r\nheader:", then it was the request line. But it could be request + partial-header-name, so you'd have to be careful. I'm not sure it's worth it to add a debugging enhancement (you probably already know what your huge headers are), but I am by no means the King of Tomcat: feel free to request that enhancement in BZ and see if you can get some support. If you write a patch/PR for it, it will almost certainly be added as long as its not too intrusive :) - -chris -BEGIN PGP SIGNATURE- Comment: Using GnuPG with Thunderbird - https://www.enigmail.net/ iQIzBAEBCAAdFiEEMmKgYcQvxMe7tcJcHPApP6U8pFgFAlvcztQACgkQHPApP6U8 pFhj+Q//QIMUy44I2SDH5kM7UeMk2SCB/5J5hW79RGfIGITFpmEMttBONF8wilg7 U7ldyjDuu2Msn+ynRPVg5lxf9pq3OQJKySv7xFc3AG4C44em4xRrJI8aHGsLxH5N dQyyJY/oP61HzeiiiyDLYxQY3sidC+4gieOEJ2oG5ZsXJwFpvfc0Q1LyiIzN8fXa o2PkH0B2kPsksgXD8WZLPXAFMgAUdmUu0wVzM0lt/kBvlG9LGW/vEd+fVeLrG93s UEs3IrWT2aTWnOVyZl2LAPhL2GynfrDUmsmsc8EF6kljXiX3IWphaH9z1nbCpLFM zDsUoIRpqm1sgXObV52zW8YbnaUzIEAZaI7lIlrOctRvc9ACeCvKkIc+aVylgnmR zpEhFz/gOchl1T6x2+MXLHavch653F8OgYY52/TCqxQheVKSHjc2jJu9UTcb+7z0 EBbfeWvUeEdNd7edGORhKx6tJLotbRhF1Yr2sjxxkY8I31RGHv+ZlNStOuxr5uoI 0c68L2XGbe9gBaLbjopYOfb+Y8y73iX3wfJPKIlvXl8LkupA1wyg0NAwOn+ZDL2E Rl865F3+DaaVaFoSRkpgp7pUrKfh9wi/N2CdCJ13hH6WY32/JD8+Z2X+YuRfMu6+ 6dh+NiVvrQc998/gutkx+OnH+RTC0STJp6RTzYNZbnL5hi5z1Ms= =bjZe -END PGP SIGNATURE- - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional