On 14.05.2009 14:34, David kerber wrote: > This post is a follow-on to my "performance with many small requests" > thread from a couple days ago. > > I have been watching my app, and taking thread dumps when things seem to > be running a bit slower than I would like. Right now, the biggest > bottleneck seems to be on the POST data read. Here is the thread dump > of interest (there are several identical ones in my full dump): > > [2009-05-13 15:49:50] [info] "http-1024-Processor11" > [2009-05-13 15:49:50] [info] daemon > [2009-05-13 15:49:50] [info] prio=6 tid=0x26c97858 > [2009-05-13 15:49:50] [info] nid=0xe18 > [2009-05-13 15:49:50] [info] runnable > [2009-05-13 15:49:50] [info] [0x2761f000..0x2761fa64] > [2009-05-13 15:49:50] [info] at > java.net.SocketInputStream.socketRead0(Native Method) > [2009-05-13 15:49:50] [info] at > java.net.SocketInputStream.read(Unknown Source) > [2009-05-13 15:49:50] [info] at > org.apache.coyote.http11.InternalInputBuffer.fill(InternalInputBuffer.java:747) > > [2009-05-13 15:49:50] [info] at > org.apache.coyote.http11.InternalInputBuffer$InputStreamInputBuffer.doRead(InternalInputBuffer.java:777) > > [2009-05-13 15:49:50] [info] at > org.apache.coyote.http11.filters.IdentityInputFilter.doRead(IdentityInputFilter.java:115) > > [2009-05-13 15:49:50] [info] at > org.apache.coyote.http11.InternalInputBuffer.doRead(InternalInputBuffer.java:712) > > [2009-05-13 15:49:50] [info] at > org.apache.coyote.Request.doRead(Request.java:423) > [2009-05-13 15:49:50] [info] at > org.apache.catalina.connector.InputBuffer.realReadBytes(InputBuffer.java:283) > > [2009-05-13 15:49:50] [info] at > org.apache.tomcat.util.buf.ByteChunk.substract(ByteChunk.java:404) > [2009-05-13 15:49:50] [info] at > org.apache.catalina.connector.InputBuffer.read(InputBuffer.java:298) > [2009-05-13 15:49:50] [info] at > org.apache.catalina.connector.CoyoteInputStream.read(CoyoteInputStream.java:192) > > [2009-05-13 15:49:51] [info] at > eddsrv.EddRcvr.processRequest(EddRcvr.java:198) > [2009-05-13 15:49:51] [info] at eddsrv.EddRcvr.doPost(EddRcvr.java:99) > [2009-05-13 15:49:51] [info] at > javax.servlet.http.HttpServlet.service(HttpServlet.java:709) > [2009-05-13 15:49:51] [info] at > javax.servlet.http.HttpServlet.service(HttpServlet.java:802) > [2009-05-13 15:49:51] [info] at > org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:252) > > [2009-05-13 15:49:51] [info] at > org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:173) > > [2009-05-13 15:49:51] [info] at > org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:213) > > [2009-05-13 15:49:51] [info] at > org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:178) > > [2009-05-13 15:49:51] [info] at > org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:126) > > [2009-05-13 15:49:51] [info] at > org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:105) > > [2009-05-13 15:49:51] [info] at > org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:107) > > [2009-05-13 15:49:51] [info] at > org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:148) > [2009-05-13 15:49:51] [info] at > org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:869) > [2009-05-13 15:49:51] [info] at > org.apache.coyote.http11.Http11BaseProtocol$Http11ConnectionHandler.processConnection(Http11BaseProtocol.java:667) > > [2009-05-13 15:49:51] [info] at > org.apache.tomcat.util.net.PoolTcpEndpoint.processSocket(PoolTcpEndpoint.java:527) > > [2009-05-13 15:49:51] [info] at > org.apache.tomcat.util.net.LeaderFollowerWorkerThread.runIt(LeaderFollowerWorkerThread.java:80) > > [2009-05-13 15:49:51] [info] at > org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:684) > > [2009-05-13 15:49:51] [info] at java.lang.Thread.run(Unknown Source) > [2009-05-13 15:49:51] [info] > > > > > The line eddsrv.EddRcvr.processRequest(EddRcvr.java:198) points to my > InputStream.read() line. All these variables are declared locally in > the method doing the work, not at the class level, so there is no > synchronization needed on them. This method is called from the doPost() > method of the servlet class. > > Here are a few code lines for context setting: > > req is my HttpServletRequest > iStream is a ServletInputStream > > > Then I have: > > len = req.getContentLength(); > b = new byte[ len ]; > iStream = req.getInputStream(); > /* this is the line 198 that the above thread dump is waiting for: */ > strLen = iStream.read( b, 0, len ); > iStream.close(); > > > And then I convert b to a string and finish processing. Is there > anything stupid I'm missing that would cause the .read() to be slow? Or > is it just that my OS tcp/ip stack and/or network infrastructure appear > to be slow? I did quite a bit of reading on InputStream performance, it > couldn't find any suggestions to improve on what I've already got (the > biggest one was to read the whole thing at once rather than > byte-by-byte, and I'm already doing that).
socketRead0() most likely is really waiting for data to arrive. So now that you have an indication your performance is restricted by network, go ahead, install Wireshark and sniff a little traffic to verify, that your clients are really slow in sending the data (or invalidate that theory). Caution: you should do a couple of thread dumps and verify, that the code is waiting in socketRead0() in nearly all cases. Of course it's always possible that it is pure coincidence in a single dump. Regards, Rainer --------------------------------------------------------------------- To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
