On 25/11/2012 00:50, Alex Moskvin wrote: > Hi, > > I am using Tomcat 7.0.30 and OpenEJB 4.5 to host high loaded app and at > some point found with VisualVM there is a bottleneck when there is a lot of > concurrent requests (about 400-600 req/sec). Requests and responses are > small (usually not larger than 2kb in JSON format) - almost all CPU time is > occupied by TaskQueue.poll() calls and general response latency becomes > very slow. > I tried APR/Nio connectors and different threading setting and that didn't > help anyhow. Tuning (increasing) amount of threads in thread pools didn't > help as well. > Tests were conducted on Intel Core i7 3450, 8Gb RAM, 100Mb network, RHEL > 6.3 x64. > > How to interpret this relatively low request rate? What I might be doing > wrong? What can be tuned in Tomcat to improve request handling? > Because we expect much larger request rate processing speed on the > mentioned hardware. > > Here is a VisualVM top output: > > Hot spots - method; self time [%]; self time; self time (cpu) > > org.apache.tomcat.util.threads.TaskQueue.poll() 75.508965 57 668 249 ms > (75,5%) 0,000 ms
That indicates idle threads waiting for work to do. Not an issue. > net.spy.memcached.internal.OperationFuture.get() 14.22508 10 864 079 ms > (14,2%) 6 798 ms > net.spy.memcached.internal.BulkGetFuture.internalGet() 5.6411114 4 308 270 > ms (5,6%) 0,000 ms This is more interesting. That looks like the application is waiting for memCached to complete a task. I'd look into this first. > org.apache.tomcat.util.net.NioEndpoint$Poller.run() 0.71728086 547 806 ms > (0,7%) 530 252 ms That is the NIO connector's Poller processing incoming connections. I'm a little surprised to see this appearing rather than application code but given the relatively low % I'd not focus too much time on here just yet. Mark --------------------------------------------------------------------- To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
