Data corruption with large messages
Vitals: Tomcat version: 4.1.27 Tomcat configuration: Out-of-the-box (except for my app's WEB-INF) OS: solaris 2.9 My servlet skill level: medium-low Problem: When receiving a HTTP 1.0 POST with a 10kbyte payload, my doPost() method writes the message body to a file. The file is the right size, but my data is nulled out after correctly receiving about 2kB. In frustration, I set the Connector bufferSize parameter to "100", to discover that I could now receive about 4kbytes of my 10kbyte message. I then proceeded to write my own Java server to receive the message and write it to a file, and it works just fine. I am at a loss - can anyone suggest what could be causing this problem? Thanks, Dan Stewart - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Tomcat sucks at receiving large messages
When receiving a HTTP 1.0 POST with a 10kbyte payload, my doPost() method writes the message body to a file. The file is the right size, but my data is nulled out (set to 0) after correctly receiving about 2kbytes. In frustration, I set the Connector bufferSize parameter to "100", to discover that I could now receive about 4kbytes of my 10kbyte message. I then proceeded to write my own Java server to receive the message and write it to a file, and it works just fine. I am at a loss - can anyone suggest what could be causing this problem? Vitals: Tomcat version: 4.1.27 Tomcat configuration: Out-of-the-box (except for my app's WEB-INF) OS: solaris 2.9 My servlet skill level: medium-low - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: Tomcat sucks at receiving large messages
Here it is: public class AdapterServlet extends HttpServlet { public void service(HttpServletRequest req, HttpServletResponse res) { BufferedReader reader = req.getReader(); try { char [] charArr = new char[req.getContentLength()]; reader.read(charArr); String str = new String(charArr); try { File f = new File("servlet.out"); PrintWriter out = new PrintWriter(new FileWriter(f)); out.print(str); out.flush(); out.close(); } catch(IOException err { System.err.println(err.toString()); } } catch(IOException err) { System.err.println(err.toString()); } } -Original Message- From: Shapira, Yoav [mailto:[EMAIL PROTECTED] Sent: Tuesday, September 30, 2003 8:34 AM To: Tomcat Users List Subject: RE: Tomcat sucks at receiving large messages Howdy, Perhaps if you share your servlet which writes the message body to a file, we could help you write a better servlet ;) Yoav Shapira Millennium ChemInformatics >-Original Message- >From: Stewart, Daniel J [mailto:[EMAIL PROTECTED] >Sent: Tuesday, September 30, 2003 11:31 AM >To: [EMAIL PROTECTED] >Subject: Tomcat sucks at receiving large messages > > >When receiving a HTTP 1.0 POST with a 10kbyte payload, my doPost() >method writes the message body to a file. The file is the right size, >but my data is nulled out (set to 0) after correctly receiving about >2kbytes. > >In frustration, I set the Connector bufferSize parameter to "100", >to discover that I could now receive about 4kbytes of my 10kbyte >message. I then proceeded to write my own Java server to receive the >message and write it to a file, and it works just fine. > >I am at a loss - can anyone suggest what could be causing this problem? > >Vitals: >Tomcat version: 4.1.27 >Tomcat configuration: Out-of-the-box (except for my app's WEB-INF) >OS: solaris 2.9 >My servlet skill level: medium-low > > > >- >To unsubscribe, e-mail: [EMAIL PROTECTED] >For additional commands, e-mail: [EMAIL PROTECTED] This e-mail, including any attachments, is a confidential business communication, and may contain information that is confidential, proprietary and/or privileged. This e-mail is intended only for the individual(s) to whom it is addressed, and may not be saved, copied, printed, disclosed or used by anyone else. If you are not the(an) intended recipient, please immediately delete this e-mail from your computer system and notify the sender. Thank you. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: Tomcat sucks at receiving large messages
Since I will be occasionally receiving messages in the 10Mbyte range, I can't read in a line at a time - it takes too long. The bug in the code below is because BufferedReader.read() will not necessarily return the whole buffer. So I replace the line reader.read(charArr); With this: int length = req.getContentLength(); char [] charArr = new char[length]; int readResult = 0; int sum = 0; do { sum += readResult; length -= readResult; readResult = reader.read(charArr, sum, length); } while (readResult < length); Thanks for your help. Any other critiques on the use of the standard library are welcome. Dan -Original Message- From: Shapira, Yoav [mailto:[EMAIL PROTECTED] Sent: Tuesday, September 30, 2003 11:30 AM To: Tomcat Users List Subject: RE: Tomcat sucks at receiving large messages Howdy, >public void service(HttpServletRequest req, HttpServletResponse res) { > BufferedReader reader = req.getReader(); > try { >char [] charArr = new char[req.getContentLength()]; >reader.read(charArr); >String str = new String(charArr); > >try { > File f = new File("servlet.out"); > PrintWriter out = new PrintWriter(new FileWriter(f)); > out.print(str); > out.flush(); > out.close(); >} catch(IOException err { System.err.println(err.toString()); } > > } catch(IOException err) { System.err.println(err.toString()); } } What happens if you ditch the req.getContentLength() approach (there are times when it will be -1 anyways), and do something like: BufferedReader reader = req.getReader(); StringBuffer contents = new StringBuffer(); String line = null; while((line = reader.readLine()) != null) { contents.append(line); } System.out.println(contents); (Later we'll worry about the writing -- first make sure you're reading the entire contents). Yoav Shapira This e-mail, including any attachments, is a confidential business communication, and may contain information that is confidential, proprietary and/or privileged. This e-mail is intended only for the individual(s) to whom it is addressed, and may not be saved, copied, printed, disclosed or used by anyone else. If you are not the(an) intended recipient, please immediately delete this e-mail from your computer system and notify the sender. Thank you. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: Tomcat sucks at receiving large messages
This code will be running in a controlled environment, with known clients, where the largest message size is known (~10M). This code takes the entire body and forwards it on to another messaging system, so I have no choice but to deal with the entire message. And I can't read it a byte or line at a time, because it would take too long. Take a look at my other response to this subject to see the code that fixed my problem. I am open to any other suggestions Dan -Original Message- From: Walker Chris [mailto:[EMAIL PROTECTED] Sent: Wednesday, October 01, 2003 1:21 AM To: 'Tomcat Users List' Subject: RE: Tomcat sucks at receiving large messages Hi, I should have thought that as a general principle it's not a good idea to try to store the response in a byte array. I recently worked on a piece of code that did just that (worse, actually, it then copied the array into a String). Sooner or later a really big upload will blow up the application. Reading and writing a byte at a time (with appropriate buffering) requires a bit more ingenuity, especially when you're searching for things like boundary strings in the response, but it's the only way to remove any constraint on upload size. Chris Walker -Original Message- From: Shapira, Yoav [mailto:[EMAIL PROTECTED] Sent: 30 September 2003 19:30 To: Tomcat Users List Subject: RE: Tomcat sucks at receiving large messages Howdy, >public void service(HttpServletRequest req, HttpServletResponse res) { > BufferedReader reader = req.getReader(); > try { >char [] charArr = new char[req.getContentLength()]; >reader.read(charArr); >String str = new String(charArr); > >try { > File f = new File("servlet.out"); > PrintWriter out = new PrintWriter(new FileWriter(f)); > out.print(str); > out.flush(); > out.close(); >} catch(IOException err { System.err.println(err.toString()); } > > } catch(IOException err) { System.err.println(err.toString()); } } What happens if you ditch the req.getContentLength() approach (there are times when it will be -1 anyways), and do something like: BufferedReader reader = req.getReader(); StringBuffer contents = new StringBuffer(); String line = null; while((line = reader.readLine()) != null) { contents.append(line); } System.out.println(contents); (Later we'll worry about the writing -- first make sure you're reading the entire contents). Yoav Shapira This e-mail, including any attachments, is a confidential business communication, and may contain information that is confidential, proprietary and/or privileged. This e-mail is intended only for the individual(s) to whom it is addressed, and may not be saved, copied, printed, disclosed or used by anyone else. If you are not the(an) intended recipient, please immediately delete this e-mail from your computer system and notify the sender. Thank you. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]