Are you on Java 5, 6 or 7? Each release sees some tweaking of the Java multithreading model as well as performance improvements (and bug fixes) in the Sun HotSpot runtime.
You may be tripping over the TCP/IP multithreaded connection manager. You might wish to create each client thread with a separate socket. Also, here is a standard bit of benchmarking advice: include "think time". This means that instead of sending requests constantly, each thread should time out for a few seconds before sending the next request. This simulates a user "stopping and thinking" before clicking the mouse again. This helps simulate the quantity of threads, etc. which are stopped and waiting at each stage of the request pipeline. As it is, you are trying to simulate the throughput behaviour without simulating the horizontal volume. (Benchmarking is much harder than it looks.) On Wed, Sep 23, 2009 at 9:43 AM, Grant Ingersoll <gsing...@apache.org> wrote: > > On Sep 23, 2009, at 12:09 PM, Michael wrote: > >> On Wed, Sep 23, 2009 at 12:05 PM, Yonik Seeley >> <yo...@lucidimagination.com>wrote: >> >>> On Wed, Sep 23, 2009 at 11:47 AM, Michael <solrco...@gmail.com> wrote: >>>> >>>> If this were IO bound, wouldn't I see the same results when sending my 8 >>>> requests to 8 Tomcats? There's only one "disk" (well, RAM) whether I'm >>>> querying 8 processes or 8 threads in 1 process, right? >>> >>> Right - I was thinking IO bound at the Lucene Directory level - which >>> synchronized in the past and led to poor concurrency. Buy your Solr >>> version is recent enough to use the newer unsynchronized method by >>> default (on non-windows) >>> >> >> Ah, OK. So it looks like comparing to Jetty is my only next step. >> Although >> I'm not sure what I'm going to do based on the result of that test -- if >> Jetty behaves differently, then I still don't know why the heck Tomcat is >> behaving badly! :) > > > Have you done any profiling to see where hotspots are? Have you looked at > garbage collection? Do you have any full collections occurring? What > garbage collector are you using? How often are you updating/committing, > etc? > > > -------------------------- > Grant Ingersoll > http://www.lucidimagination.com/ > > Search the Lucene ecosystem (Lucene/Solr/Nutch/Mahout/Tika/Droids) using > Solr/Lucene: > http://www.lucidimagination.com/search > > -- Lance Norskog goks...@gmail.com
