Data corruption with large messages
Vitals: Tomcat version: 4.1.27 Tomcat configuration: Out-of-the-box (except for my app's WEB-INF) OS: solaris 2.9 My servlet skill level: medium-low Problem: When receiving a HTTP 1.0 POST with a 10kbyte payload, my doPost() method writes the message body to a file. The file is the right size, but my data is nulled out after correctly receiving about 2kB. In frustration, I set the Connector bufferSize parameter to "100", to discover that I could now receive about 4kbytes of my 10kbyte message. I then proceeded to write my own Java server to receive the message and write it to a file, and it works just fine. I am at a loss - can anyone suggest what could be causing this problem? Thanks, Dan Stewart - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Tomcat sucks at receiving large messages
When receiving a HTTP 1.0 POST with a 10kbyte payload, my doPost() method writes the message body to a file. The file is the right size, but my data is nulled out (set to 0) after correctly receiving about 2kbytes. In frustration, I set the Connector bufferSize parameter to "100", to discover that I could now receive about 4kbytes of my 10kbyte message. I then proceeded to write my own Java server to receive the message and write it to a file, and it works just fine. I am at a loss - can anyone suggest what could be causing this problem? Vitals: Tomcat version: 4.1.27 Tomcat configuration: Out-of-the-box (except for my app's WEB-INF) OS: solaris 2.9 My servlet skill level: medium-low - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: Tomcat sucks at receiving large messages
Here it is:
public class AdapterServlet extends HttpServlet {
public void service(HttpServletRequest req, HttpServletResponse res) {
BufferedReader reader = req.getReader();
try {
char [] charArr = new char[req.getContentLength()];
reader.read(charArr);
String str = new String(charArr);
try {
File f = new File("servlet.out");
PrintWriter out = new PrintWriter(new FileWriter(f));
out.print(str);
out.flush();
out.close();
} catch(IOException err { System.err.println(err.toString()); }
} catch(IOException err) { System.err.println(err.toString()); }
}
-Original Message-
From: Shapira, Yoav [mailto:[EMAIL PROTECTED]
Sent: Tuesday, September 30, 2003 8:34 AM
To: Tomcat Users List
Subject: RE: Tomcat sucks at receiving large messages
Howdy,
Perhaps if you share your servlet which writes the message body to a
file, we could help you write a better servlet ;)
Yoav Shapira
Millennium ChemInformatics
>-Original Message-
>From: Stewart, Daniel J [mailto:[EMAIL PROTECTED]
>Sent: Tuesday, September 30, 2003 11:31 AM
>To: [EMAIL PROTECTED]
>Subject: Tomcat sucks at receiving large messages
>
>
>When receiving a HTTP 1.0 POST with a 10kbyte payload, my doPost()
>method writes the message body to a file. The file is the right size,
>but my data is nulled out (set to 0) after correctly receiving about
>2kbytes.
>
>In frustration, I set the Connector bufferSize parameter to "100",
>to discover that I could now receive about 4kbytes of my 10kbyte
>message. I then proceeded to write my own Java server to receive the
>message and write it to a file, and it works just fine.
>
>I am at a loss - can anyone suggest what could be causing this problem?
>
>Vitals:
>Tomcat version: 4.1.27
>Tomcat configuration: Out-of-the-box (except for my app's WEB-INF)
>OS: solaris 2.9
>My servlet skill level: medium-low
>
>
>
>-
>To unsubscribe, e-mail: [EMAIL PROTECTED]
>For additional commands, e-mail: [EMAIL PROTECTED]
This e-mail, including any attachments, is a confidential business
communication, and may contain information that is confidential,
proprietary and/or privileged. This e-mail is intended only for the
individual(s) to whom it is addressed, and may not be saved, copied,
printed, disclosed or used by anyone else. If you are not the(an)
intended recipient, please immediately delete this e-mail from your
computer system and notify the sender. Thank you.
-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
RE: Tomcat sucks at receiving large messages
Since I will be occasionally receiving messages in the 10Mbyte range, I
can't read in a line at a time - it takes too long.
The bug in the code below is because BufferedReader.read() will not
necessarily return the whole buffer. So I replace the line
reader.read(charArr);
With this:
int length = req.getContentLength();
char [] charArr = new char[length];
int readResult = 0;
int sum = 0;
do {
sum += readResult;
length -= readResult;
readResult = reader.read(charArr, sum, length);
} while (readResult < length);
Thanks for your help. Any other critiques on the use of the standard
library are welcome.
Dan
-Original Message-
From: Shapira, Yoav [mailto:[EMAIL PROTECTED]
Sent: Tuesday, September 30, 2003 11:30 AM
To: Tomcat Users List
Subject: RE: Tomcat sucks at receiving large messages
Howdy,
>public void service(HttpServletRequest req, HttpServletResponse res) {
> BufferedReader reader = req.getReader();
> try {
>char [] charArr = new char[req.getContentLength()];
>reader.read(charArr);
>String str = new String(charArr);
>
>try {
> File f = new File("servlet.out");
> PrintWriter out = new PrintWriter(new FileWriter(f));
> out.print(str);
> out.flush();
> out.close();
>} catch(IOException err { System.err.println(err.toString()); }
>
> } catch(IOException err) { System.err.println(err.toString()); } }
What happens if you ditch the req.getContentLength() approach (there are
times when it will be -1 anyways), and do something like: BufferedReader
reader = req.getReader(); StringBuffer contents = new StringBuffer();
String line = null; while((line = reader.readLine()) != null) {
contents.append(line);
}
System.out.println(contents);
(Later we'll worry about the writing -- first make sure you're reading
the entire contents).
Yoav Shapira
This e-mail, including any attachments, is a confidential business
communication, and may contain information that is confidential,
proprietary and/or privileged. This e-mail is intended only for the
individual(s) to whom it is addressed, and may not be saved, copied,
printed, disclosed or used by anyone else. If you are not the(an)
intended recipient, please immediately delete this e-mail from your
computer system and notify the sender. Thank you.
-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
RE: Tomcat sucks at receiving large messages
This code will be running in a controlled environment, with known
clients, where the largest message size is known (~10M). This code
takes the entire body and forwards it on to another messaging system, so
I have no choice but to deal with the entire message. And I can't read
it a byte or line at a time, because it would take too long.
Take a look at my other response to this subject to see the code that
fixed my problem. I am open to any other suggestions
Dan
-Original Message-
From: Walker Chris [mailto:[EMAIL PROTECTED]
Sent: Wednesday, October 01, 2003 1:21 AM
To: 'Tomcat Users List'
Subject: RE: Tomcat sucks at receiving large messages
Hi,
I should have thought that as a general principle it's not a good idea
to try to store the response in a byte array. I recently worked on a
piece of code that did just that (worse, actually, it then copied the
array into a String). Sooner or later a really big upload will blow up
the application.
Reading and writing a byte at a time (with appropriate buffering)
requires a bit more ingenuity, especially when you're searching for
things like boundary strings in the response, but it's the only way to
remove any constraint on upload size.
Chris Walker
-Original Message-
From: Shapira, Yoav [mailto:[EMAIL PROTECTED]
Sent: 30 September 2003 19:30
To: Tomcat Users List
Subject: RE: Tomcat sucks at receiving large messages
Howdy,
>public void service(HttpServletRequest req, HttpServletResponse res) {
> BufferedReader reader = req.getReader();
> try {
>char [] charArr = new char[req.getContentLength()];
>reader.read(charArr);
>String str = new String(charArr);
>
>try {
> File f = new File("servlet.out");
> PrintWriter out = new PrintWriter(new FileWriter(f));
> out.print(str);
> out.flush();
> out.close();
>} catch(IOException err { System.err.println(err.toString()); }
>
> } catch(IOException err) { System.err.println(err.toString()); } }
What happens if you ditch the req.getContentLength() approach (there are
times when it will be -1 anyways), and do something like: BufferedReader
reader = req.getReader(); StringBuffer contents = new StringBuffer();
String line = null; while((line = reader.readLine()) != null) {
contents.append(line);
}
System.out.println(contents);
(Later we'll worry about the writing -- first make sure you're reading
the entire contents).
Yoav Shapira
This e-mail, including any attachments, is a confidential business
communication, and may contain information that is confidential,
proprietary and/or privileged. This e-mail is intended only for the
individual(s) to whom it is addressed, and may not be saved, copied,
printed, disclosed or used by anyone else. If you are not the(an)
intended recipient, please immediately delete this e-mail from your
computer system and notify the sender. Thank you.
-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
