Re: buffer expand warning in Tomcat (apache-tomcat-8.0.0-RC1-embed)
Hi Mark, a bit more follow-up on accessing the servlet InputStream:
You advised You'd be better off dropping the call to in.ready() and doing
a blocking
read on the socket.If you remove the call to
in.ready(), I'm fairly sure you'll see the warnings disappear.
I just thought I'd let you know, that prediction was absolutely correct.
On Thu, Aug 29, 2013 at 7:09 AM, Vince Stewart stewart.vi...@gmail.comwrote:
Appreciate this a lot Mark.
I'm pretty sure my previous code had a short sleep in each loop but
thankfully, in-coming data rarely exceeds the Servlet InputBuffer size of
8192 so looping is rare.
I have been trying to get to grips with the asynchronous stuff but have
not got it going yet. The concept is very appealing. Thanks for your
contributions.
On Wed, Aug 28, 2013 at 11:00 PM, Mark Thomas ma...@apache.org wrote:
On 28/08/2013 09:41, Mark Thomas wrote:
On 27/08/2013 03:40, Vince Stewart wrote:
hi all,
thought I would add some progress on this topic.
I have changed my method for reading from the HttpServletRequest
object but
the same warning message is thrown for every 8192 bytes read. I no
longer
regard my code to be suspect though am happy to be corrected. The
application operates completely normally except for the warning
message. The
code I am using to read the input is shown below.
public void doPost(HttpServletRequest httpServletRequest.etc
...other code..
char[] cbuf=new char[8192];
int i=0;
int requestLength=httpServletRequest.getContentLength();
BufferedReader in=httpServletRequest.getReader();
StringBuilder sb=new StringBuilder(requestLength);
while(sb.length()requestLength){
if(in.ready()){
i=in.read(cbuf,0,reqLen);
}
sb.append(cbuf,0,i);
}
in.close();
String message=sb.toString();
//.etc
It looks like there is a possible Tomcat bug but the above code is not
the way to read data from the client as it puts the thread into a tight
loop while it is waiting for more data to arrive.
I've just cleaned up the code for Tomcat 8.0.x to fix the bug that your
example highlighted. Thanks for such a clear problem statement that made
it very easy to track down the root cause of the issue. The fix will be
in 8.0.0-RC2 onwards.
That said, my comments about there being a better way to do what you are
doing stand.
Mark
You'd be better off dropping the call to in.ready() and doing a blocking
read on the socket. The elapsed time should be the same (might even be a
little less) and your CPU consumption during the read when the client is
slow sending data will be significantly lower. If you remove the call to
in.ready(), I'm fairly sure you'll see the warnings disappear.
Ideally, you should look at non-blocking IO as supported by Servlet 3.1
but that might be too big a change as it fundamentally changes how data
is read and written.
Mark
-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org
-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org
--
Vince Stewart
--
Vince Stewart
Re: buffer expand warning in Tomcat (apache-tomcat-8.0.0-RC1-embed)
On 27/08/2013 03:40, Vince Stewart wrote:
hi all,
thought I would add some progress on this topic.
I have changed my method for reading from the HttpServletRequest object but
the same warning message is thrown for every 8192 bytes read. I no longer
regard my code to be suspect though am happy to be corrected. The
application operates completely normally except for the warning message. The
code I am using to read the input is shown below.
public void doPost(HttpServletRequest httpServletRequest.etc
...other code..
char[] cbuf=new char[8192];
int i=0;
int requestLength=httpServletRequest.getContentLength();
BufferedReader in=httpServletRequest.getReader();
StringBuilder sb=new StringBuilder(requestLength);
while(sb.length()requestLength){
if(in.ready()){
i=in.read(cbuf,0,reqLen);
}
sb.append(cbuf,0,i);
}
in.close();
String message=sb.toString();
//.etc
It looks like there is a possible Tomcat bug but the above code is not
the way to read data from the client as it puts the thread into a tight
loop while it is waiting for more data to arrive.
You'd be better off dropping the call to in.ready() and doing a blocking
read on the socket. The elapsed time should be the same (might even be a
little less) and your CPU consumption during the read when the client is
slow sending data will be significantly lower. If you remove the call to
in.ready(), I'm fairly sure you'll see the warnings disappear.
Ideally, you should look at non-blocking IO as supported by Servlet 3.1
but that might be too big a change as it fundamentally changes how data
is read and written.
Mark
-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org
Re: buffer expand warning in Tomcat (apache-tomcat-8.0.0-RC1-embed)
On 28/08/2013 09:41, Mark Thomas wrote:
On 27/08/2013 03:40, Vince Stewart wrote:
hi all,
thought I would add some progress on this topic.
I have changed my method for reading from the HttpServletRequest object but
the same warning message is thrown for every 8192 bytes read. I no longer
regard my code to be suspect though am happy to be corrected. The
application operates completely normally except for the warning message. The
code I am using to read the input is shown below.
public void doPost(HttpServletRequest httpServletRequest.etc
...other code..
char[] cbuf=new char[8192];
int i=0;
int requestLength=httpServletRequest.getContentLength();
BufferedReader in=httpServletRequest.getReader();
StringBuilder sb=new StringBuilder(requestLength);
while(sb.length()requestLength){
if(in.ready()){
i=in.read(cbuf,0,reqLen);
}
sb.append(cbuf,0,i);
}
in.close();
String message=sb.toString();
//.etc
It looks like there is a possible Tomcat bug but the above code is not
the way to read data from the client as it puts the thread into a tight
loop while it is waiting for more data to arrive.
I've just cleaned up the code for Tomcat 8.0.x to fix the bug that your
example highlighted. Thanks for such a clear problem statement that made
it very easy to track down the root cause of the issue. The fix will be
in 8.0.0-RC2 onwards.
That said, my comments about there being a better way to do what you are
doing stand.
Mark
You'd be better off dropping the call to in.ready() and doing a blocking
read on the socket. The elapsed time should be the same (might even be a
little less) and your CPU consumption during the read when the client is
slow sending data will be significantly lower. If you remove the call to
in.ready(), I'm fairly sure you'll see the warnings disappear.
Ideally, you should look at non-blocking IO as supported by Servlet 3.1
but that might be too big a change as it fundamentally changes how data
is read and written.
Mark
-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org
-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org
Re: buffer expand warning in Tomcat (apache-tomcat-8.0.0-RC1-embed)
Appreciate this a lot Mark.
I'm pretty sure my previous code had a short sleep in each loop but
thankfully, in-coming data rarely exceeds the Servlet InputBuffer size of
8192 so looping is rare.
I have been trying to get to grips with the asynchronous stuff but have not
got it going yet. The concept is very appealing. Thanks for your
contributions.
On Wed, Aug 28, 2013 at 11:00 PM, Mark Thomas ma...@apache.org wrote:
On 28/08/2013 09:41, Mark Thomas wrote:
On 27/08/2013 03:40, Vince Stewart wrote:
hi all,
thought I would add some progress on this topic.
I have changed my method for reading from the HttpServletRequest object
but
the same warning message is thrown for every 8192 bytes read. I no
longer
regard my code to be suspect though am happy to be corrected. The
application operates completely normally except for the warning
message. The
code I am using to read the input is shown below.
public void doPost(HttpServletRequest httpServletRequest.etc
...other code..
char[] cbuf=new char[8192];
int i=0;
int requestLength=httpServletRequest.getContentLength();
BufferedReader in=httpServletRequest.getReader();
StringBuilder sb=new StringBuilder(requestLength);
while(sb.length()requestLength){
if(in.ready()){
i=in.read(cbuf,0,reqLen);
}
sb.append(cbuf,0,i);
}
in.close();
String message=sb.toString();
//.etc
It looks like there is a possible Tomcat bug but the above code is not
the way to read data from the client as it puts the thread into a tight
loop while it is waiting for more data to arrive.
I've just cleaned up the code for Tomcat 8.0.x to fix the bug that your
example highlighted. Thanks for such a clear problem statement that made
it very easy to track down the root cause of the issue. The fix will be
in 8.0.0-RC2 onwards.
That said, my comments about there being a better way to do what you are
doing stand.
Mark
You'd be better off dropping the call to in.ready() and doing a blocking
read on the socket. The elapsed time should be the same (might even be a
little less) and your CPU consumption during the read when the client is
slow sending data will be significantly lower. If you remove the call to
in.ready(), I'm fairly sure you'll see the warnings disappear.
Ideally, you should look at non-blocking IO as supported by Servlet 3.1
but that might be too big a change as it fundamentally changes how data
is read and written.
Mark
-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org
-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org
--
Vince Stewart
Re: buffer expand warning in Tomcat (apache-tomcat-8.0.0-RC1-embed)
hi all,
thought I would add some progress on this topic.
I have changed my method for reading from the HttpServletRequest object but
the same warning message is thrown for every 8192 bytes read. I no longer
regard my code to be suspect though am happy to be corrected. The
application operates completely normally except for the warning message. The
code I am using to read the input is shown below.
public void doPost(HttpServletRequest httpServletRequest.etc
...other code..
char[] cbuf=new char[8192];
int i=0;
int requestLength=httpServletRequest.getContentLength();
BufferedReader in=httpServletRequest.getReader();
StringBuilder sb=new StringBuilder(requestLength);
while(sb.length()requestLength){
if(in.ready()){
i=in.read(cbuf,0,reqLen);
}
sb.append(cbuf,0,i);
}
in.close();
String message=sb.toString();
//.etc
--
View this message in context:
http://tomcat.10.x6.nabble.com/buffer-expand-warning-in-Tomcat-apache-tomcat-8-0-0-RC1-embed-tp5003745p5003777.html
Sent from the Tomcat - User mailing list archive at Nabble.com.
-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org
buffer expand warning in Tomcat (apache-tomcat-8.0.0-RC1-embed)
greetings all, since commencing with version 8 (embedded) I have encountered no problems except when the HttpServletRequest object receives a message bigger than 16384 k. In that circumstance a warning(below) is repeated X times for messages bigger than 8192+(X*8192). I notice in the InternalNioInputBuffer.expand method, the comment //should not happen precedes the logging of the warning message. I am using a modified BufferedReader class with a buffer size of 8192 and the warning seems to coincide with each read. This same class has operated without a problem for years but I am sure my reader is causing this new problem. So the question is how best to read a large message from the ServletInputStream in tomcat-8 = Aug 26, 2013 10:59:33 AM org.apache.coyote.http11.InternalNioInputBuffer expand WARNING: Expanding buffer size. Old size: 16384, new size: 24576 java.lang.Exception at org.apache.coyote.http11.InternalNioInputBuffer.expand(InternalNioInputBuffer.java:420) at org.apache.coyote.http11.InternalNioInputBuffer.readSocket(InternalNioInputBuffer.java:467) at org.apache.coyote.http11.InternalNioInputBuffer.nbRead(InternalNioInputBuffer.java:187) at org.apache.coyote.http11.InternalNioInputBuffer.available(InternalNioInputBuffer.java:171) at org.apache.coyote.http11.AbstractHttp11Processor.action(AbstractHttp11Processor.java:818) at org.apache.coyote.Request.action(Request.java:373) at org.apache.catalina.connector.InputBuffer.available(InputBuffer.java:243) at org.apache.catalina.connector.CoyoteInputStream.available(CoyoteInputStream.java:137) at sun.nio.cs.StreamDecoder.inReady(StreamDecoder.java:362) at sun.nio.cs.StreamDecoder.implRead(StreamDecoder.java:323) at sun.nio.cs.StreamDecoder.read(StreamDecoder.java:177) at java.io.InputStreamReader.read(InputStreamReader.java:184) at oodbpds.BigBufferedReader.read1(BigBufferedReader.java:136) at oodbpds.BigBufferedReader.read(BigBufferedReader.java:156) at oodbpds.ASIFTI.doPost(ASIFTI.java:90) at javax.servlet.http.HttpServlet.service(HttpServlet.java:647) at javax.servlet.http.HttpServlet.service(HttpServlet.java:728) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:303) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:223) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:107) at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:504) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:155) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:75) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:90) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:494) at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1009) at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:632) at org.apache.coyote.http11.Http11NioProtocol$Http11ConnectionHandler.process(Http11NioProtocol.java:223) at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1592) at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1550) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) at java.lang.Thread.run(Thread.java:722) -- Vince Stewart
