Re: AJP with HTTPD - Buffer Size on long URLs
Nilesh / Others, I'm facing the same problem too - I was trying to find about where the "ProxyIOBufferSize" directive has to be added to. Is it the "httpd.h" file (for the max value) & then subsequently the "httpd.conf" to set any limit within the max value, if required. The mod_proxy says "mod_proxy.c" is the module file, but I'm not certain on it. Could you please let me know on where you added the directive, also did you recompile Apache after changing the ProxyIOBufferSize ? Thanks, WYG -- View this message in context: http://tomcat.10.n6.nabble.com/AJP-with-HTTPD-Buffer-Size-on-long-URLs-tp2165638p4985037.html Sent from the Tomcat - User mailing list archive at Nabble.com. - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
RE: AJP with HTTPD - Buffer Size on long URLs
I stand corrected. What I do recall is that in the 1999's I was forced to build an HTTP/1.1 server from scratch (in objective-C) and, when faced with the question "at what point in reading the URI should I give up and decide this is not a HTTP request?", I found 4k to be the 'correct' answer. Since RFC2068 was the basis for that server, I was lazy and assumef that that's where it originated. Anyway, when creating arbitrary long URIs, you can be sure that at some point any HTTP server will give up, because it is more or less forced to store the URI in precious RAM. Probably the 4k limit was intended as "the maximum size you can expect a HTTP server to accept", anything beyond that is "at your own peril". The "SHOULD return 414" is easily explained: If it stops reading the URL, it has no knowledge of the client's intended protocol yet, it is not aware of the other headers in the request, and as such the server may not be able to determine whether the client really expects a HTTP response at all. So the safe thing to do is just close the connection and give up. Having said that, there is a very clear distinction between GET and POST requests. The main difference is that POST requests in general have a side-effect, and cannot be expected to return the same result twice. For example, "POST /mything" might return "created a file" the first time, and "file already exists" the second time. M. From: Caldarale, Charles R [mailto:chuck.caldar...@unisys.com] > Sent: maandag 30 november 2009 15:54 > To: Tomcat Users List > Subject: RE: AJP with HTTPD - Buffer Size on long URLs > > > From: Looijmans, Mike [mailto:mike.looijm...@oce.com] > > Subject: RE: AJP with HTTPD - Buffer Size on long URLs > > > > > Looijmans, Mike wrote: > > > > The RFC specs a maximum URL size of 4k. > > > > > > Where precisely did you find that ? > > > > RFC2068 (old HTTP/1.1 spec) > > Citing an obsoleted RFC is a bit odd. Regardless, the actual > wording from section 3.2.1 of 2068 and 2616 (the superseding > document) is: > > "The HTTP protocol does not place any a priori limit on the > length of a URI." > > Followed shortly by: > > "A server SHOULD return 414 (Request-URI Too Long) status if > a URI is longer than the server can handle (see section 10.4.15)." > > (Note the SHOULD, not MUST.) > > There is also a warning note: > > "Note: Servers should be cautious about depending on URI > lengths above 255 bytes, because some older client or proxy > implementations may not properly support these lengths." > > No mention of a 4K limit anywhere that I can find. > > - Chuck This message and attachment(s) are intended solely for use by the addressee and may contain information that is privileged, confidential or otherwise exempt from disclosure under applicable law. If you are not the intended recipient or agent thereof responsible for delivering this message to the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this communication in error, please notify the sender immediately by telephone and with a 'reply' message. Thank you for your co-operation. - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: AJP with HTTPD - Buffer Size on long URLs
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Mike, On 11/30/2009 3:10 AM, Looijmans, Mike wrote: > The RFC specs a maximum URL size of 4k. That should be enough for everybody. ...along with 640k of regular memory. I'll let you read André's and Chuck's harangues about your dubious recollection of the HTTP specification. On a related note, Apache httpd (used by the OP, so definitely relevant, here) has a configuration option for limiting the length of the first-line of the request from a client: http://httpd.apache.org/docs/2.2/mod/core.html#limitrequestline The default limit in 2.0 and 2.2 is 8190 (an seemingly strange number unless you account for the CR LF end-of-line marker required by the specification (in section 2.2, since you asked). Presumably, httpd uses fgets and the default buffer size of 8190 gives them a round-numbered buffer size... though for no particular reason. Since Apache httpd will choke after 8177 characters of URI (8190 - 13 required characters for "GET" and the HTTP version identifier), the OP would be wise to change this setting in httpd.conf. Or switch to POST, which is probably the right answer, here. - -chris -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.10 (MingW32) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAksUItcACgkQ9CaO5/Lv0PB6oQCfdpF6kZpqyrglITbfEisLK4cO MDcAoJE5HOrvzVuQpTOFNGXHT40RiQt/ =PIjv -END PGP SIGNATURE- - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: AJP with HTTPD - Buffer Size on long URLs
Caldarale, Charles R wrote: From: Looijmans, Mike [mailto:mike.looijm...@oce.com] Subject: RE: AJP with HTTPD - Buffer Size on long URLs Looijmans, Mike wrote: The RFC specs a maximum URL size of 4k. Where precisely did you find that ? RFC2068 (old HTTP/1.1 spec) Citing an obsoleted RFC is a bit odd. Regardless, the actual wording from section 3.2.1 of 2068 and 2616 (the superseding document) is: "The HTTP protocol does not place any a priori limit on the length of a URI." Followed shortly by: "A server SHOULD return 414 (Request-URI Too Long) status if a URI is longer than the server can handle (see section 10.4.15)." (Note the SHOULD, not MUST.) There is also a warning note: "Note: Servers should be cautious about depending on URI lengths above 255 bytes, because some older client or proxy implementations may not properly support these lengths." No mention of a 4K limit anywhere that I can find. Right. +1. My point here (toward Mike) was that one should avoid propagating rumors or incorrect information, on a list that is read by unsuspecting users which may then believe that this is the ultimate truth. This being said, the specs do not set a specific limit to a URI length, but it is certain that any server software has a practical one, if only to avoid some types of DoS attacks. So my point to the original poster, was to recommend the use of a POST rather than a GET, if the application is such that it already now exceeds 8K for a URI. In addition, even if one knows how many individual input fields there may be in a form which sends such a URI, and how long each field is in principle, it is much harder to predict how long a URI this will actually generate once URI-escaping has taken place, and each non-ASCII character has been replaced by a triplet of bytes. There is no such arbitrary limit (or if there is, it is MUCH higher) for the body of a POST. In addition, at least for the body of a POST, there is a possibility of indicating the character set of the data, which in fact there is not for data contained in a URI. - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
RE: AJP with HTTPD - Buffer Size on long URLs
> From: Looijmans, Mike [mailto:mike.looijm...@oce.com] > Subject: RE: AJP with HTTPD - Buffer Size on long URLs > > > Looijmans, Mike wrote: > > > The RFC specs a maximum URL size of 4k. > > > > Where precisely did you find that ? > > RFC2068 (old HTTP/1.1 spec) Citing an obsoleted RFC is a bit odd. Regardless, the actual wording from section 3.2.1 of 2068 and 2616 (the superseding document) is: "The HTTP protocol does not place any a priori limit on the length of a URI." Followed shortly by: "A server SHOULD return 414 (Request-URI Too Long) status if a URI is longer than the server can handle (see section 10.4.15)." (Note the SHOULD, not MUST.) There is also a warning note: "Note: Servers should be cautious about depending on URI lengths above 255 bytes, because some older client or proxy implementations may not properly support these lengths." No mention of a 4K limit anywhere that I can find. - Chuck THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY MATERIAL and is thus for use only by the intended recipient. If you received this in error, please contact the sender and delete the e-mail and its attachments from all computers. - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
RE: AJP with HTTPD - Buffer Size on long URLs
> Looijmans, Mike wrote: > > The RFC specs a maximum URL size of 4k. > > Where precisely did you find that ? RFC2068 (old HTTP/1.1 spec) This message and attachment(s) are intended solely for use by the addressee and may contain information that is privileged, confidential or otherwise exempt from disclosure under applicable law. If you are not the intended recipient or agent thereof responsible for delivering this message to the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this communication in error, please notify the sender immediately by telephone and with a 'reply' message. Thank you for your co-operation. - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: AJP with HTTPD - Buffer Size on long URLs
Looijmans, Mike wrote: The RFC specs a maximum URL size of 4k. Where precisely did you find that ? As per my own memory, it is not as clear as that. But in various places, it warns against "too long" URLs, not so much because of the httpd server itself, but because intermediate agents may have lower limits (proxies, firewalls, connectors..) The Java Servlet Spec may also have something to say about this. - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
RE: AJP with HTTPD - Buffer Size on long URLs
The RFC specs a maximum URL size of 4k. That should be enough for everybody. Note that you can mix and match as required: Use the URL portion of your request to identify the target of the request, and put the additional data in the POST body. > -Original Message- > From: André Warnier [mailto:a...@ice-sa.com] > Sent: zaterdag 28 november 2009 13:11 > To: Tomcat Users List > Subject: Re: AJP with HTTPD - Buffer Size on long URLs > > Nilesh Bansal wrote: > > Using ProxyIOBufferSize as 32192 totally worked even though the > > documentation suggests otherwise. I am using httpd 2.2.14 > with Tomcat > > 6.0.16. Thank you for the tip, now I can again use my long urls. > > > This may work for now, but someone should tell you that > sending large amounts of data in a HTTP GET request is not > such a good idea. It will get you in trouble sooner or later, > for various reasons. > You should use a POST request for that kind of thing. > This message and attachment(s) are intended solely for use by the addressee and may contain information that is privileged, confidential or otherwise exempt from disclosure under applicable law. If you are not the intended recipient or agent thereof responsible for delivering this message to the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this communication in error, please notify the sender immediately by telephone and with a 'reply' message. Thank you for your co-operation. - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: AJP with HTTPD - Buffer Size on long URLs
Nilesh Bansal wrote: Using ProxyIOBufferSize as 32192 totally worked even though the documentation suggests otherwise. I am using httpd 2.2.14 with Tomcat 6.0.16. Thank you for the tip, now I can again use my long urls. This may work for now, but someone should tell you that sending large amounts of data in a HTTP GET request is not such a good idea. It will get you in trouble sooner or later, for various reasons. You should use a POST request for that kind of thing. - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: AJP with HTTPD - Buffer Size on long URLs
Using ProxyIOBufferSize as 32192 totally worked even though the documentation suggests otherwise. I am using httpd 2.2.14 with Tomcat 6.0.16. Thank you for the tip, now I can again use my long urls. thanks, Nilesh On Fri, Nov 27, 2009 at 11:32 PM, Caldarale, Charles R < chuck.caldar...@unisys.com> wrote: > > From: Nilesh Bansal [mailto:nileshban...@gmail.com] > > Subject: Re: AJP with HTTPD - Buffer Size on long URLs > > > > I am using mod_proxy_ajp (and not mod_jk) > > There's the ProxyIOBufferSize parameter; the 2.2 doc says the max value is > 8192, but the 2.3 doc says the minimum is 512 with no mention of a maximum. > Perhaps there's a change in trunk that allows the larger values - or you > might just try it and see what happens. > > Or switch to mod_jk... > > - Chuck > > > THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY > MATERIAL and is thus for use only by the intended recipient. If you received > this in error, please contact the sender and delete the e-mail and its > attachments from all computers. > >
RE: AJP with HTTPD - Buffer Size on long URLs
> From: Nilesh Bansal [mailto:nileshban...@gmail.com] > Subject: Re: AJP with HTTPD - Buffer Size on long URLs > > I am using mod_proxy_ajp (and not mod_jk) There's the ProxyIOBufferSize parameter; the 2.2 doc says the max value is 8192, but the 2.3 doc says the minimum is 512 with no mention of a maximum. Perhaps there's a change in trunk that allows the larger values - or you might just try it and see what happens. Or switch to mod_jk... - Chuck THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY MATERIAL and is thus for use only by the intended recipient. If you received this in error, please contact the sender and delete the e-mail and its attachments from all computers.
Re: AJP with HTTPD - Buffer Size on long URLs
I am using mod_proxy_ajp (and not mod_jk) and I was not able to find any suitable way of specifying the worker.properties file with this parameter on max_packet_size with mod_proxy. Any pointers will be most appreciated. I am still not sure where in httpd.conf to set the AJP specific parameters. thanks, Nilesh On Fri, Nov 27, 2009 at 10:44 PM, Caldarale, Charles R < chuck.caldar...@unisys.com> wrote: > > From: Nilesh Bansal [mailto:nileshban...@gmail.com] > > Subject: AJP with HTTPD - Buffer Size on long URLs > > > > Is there a way the buffer size can be increased? > > As usual, it's in the docs: > > http://tomcat.apache.org/connectors-doc/reference/workers.html#Advanced%20Worker%20Directives > > Look at the max_packet_size attribute. Note that a corresponding setting > must be made in the AJP element in Tomcat's conf/server.xml > file. > > - Chuck > > > THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY > MATERIAL and is thus for use only by the intended recipient. If you received > this in error, please contact the sender and delete the e-mail and its > attachments from all computers. > >
RE: AJP with HTTPD - Buffer Size on long URLs
> From: Nilesh Bansal [mailto:nileshban...@gmail.com] > Subject: AJP with HTTPD - Buffer Size on long URLs > > Is there a way the buffer size can be increased? As usual, it's in the docs: http://tomcat.apache.org/connectors-doc/reference/workers.html#Advanced%20Worker%20Directives Look at the max_packet_size attribute. Note that a corresponding setting must be made in the AJP element in Tomcat's conf/server.xml file. - Chuck THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY MATERIAL and is thus for use only by the intended recipient. If you received this in error, please contact the sender and delete the e-mail and its attachments from all computers.
AJP with HTTPD - Buffer Size on long URLs
Hi, I have recently configured the apache httpd to use AJP for connecting to backend Tomcat servers over AJP (ajp://tomcat.server/8009). When I try to query the frontend httpd server with a long url (over 8000 characters), the request fails. The following is printed in error_log of httpd [Fri Nov 27 18:30:13 2009] [error] ajp_msg_append_cvt_string(): BufferOverflowException 4 6117 [Fri Nov 27 18:30:13 2009] [error] ajp_marshal_into_msgb: Error appending the query string [Fri Nov 27 18:30:13 2009] [error] ajp_send_header: ajp_marshal_into_msgb failed [Fri Nov 27 18:30:13 2009] [error] (120001)APR does not understand this error code: proxy: AJP: request failed to (null) (tomcat.server) Is there a way the buffer size can be increased? thanks, Nilesh
