Thank you Grant and Lance for your comments -- I've run into a separate snag which puts this on hold for a bit, but I'll return to finish digging into this and post my results. - Michael On Thu, Sep 24, 2009 at 9:23 PM, Lance Norskog <goks...@gmail.com> wrote:
> Are you on Java 5, 6 or 7? Each release sees some tweaking of the Java > multithreading model as well as performance improvements (and bug > fixes) in the Sun HotSpot runtime. > > You may be tripping over the TCP/IP multithreaded connection manager. > You might wish to create each client thread with a separate socket. > > Also, here is a standard bit of benchmarking advice: include "think > time". This means that instead of sending requests constantly, each > thread should time out for a few seconds before sending the next > request. This simulates a user "stopping and thinking" before clicking > the mouse again. This helps simulate the quantity of threads, etc. > which are stopped and waiting at each stage of the request pipeline. > As it is, you are trying to simulate the throughput behaviour without > simulating the horizontal volume. (Benchmarking is much harder than it > looks.) > > On Wed, Sep 23, 2009 at 9:43 AM, Grant Ingersoll <gsing...@apache.org> > wrote: > > > > On Sep 23, 2009, at 12:09 PM, Michael wrote: > > > >> On Wed, Sep 23, 2009 at 12:05 PM, Yonik Seeley > >> <yo...@lucidimagination.com>wrote: > >> > >>> On Wed, Sep 23, 2009 at 11:47 AM, Michael <solrco...@gmail.com> wrote: > >>>> > >>>> If this were IO bound, wouldn't I see the same results when sending my > 8 > >>>> requests to 8 Tomcats? There's only one "disk" (well, RAM) whether > I'm > >>>> querying 8 processes or 8 threads in 1 process, right? > >>> > >>> Right - I was thinking IO bound at the Lucene Directory level - which > >>> synchronized in the past and led to poor concurrency. Buy your Solr > >>> version is recent enough to use the newer unsynchronized method by > >>> default (on non-windows) > >>> > >> > >> Ah, OK. So it looks like comparing to Jetty is my only next step. > >> Although > >> I'm not sure what I'm going to do based on the result of that test -- if > >> Jetty behaves differently, then I still don't know why the heck Tomcat > is > >> behaving badly! :) > > > > > > Have you done any profiling to see where hotspots are? Have you looked > at > > garbage collection? Do you have any full collections occurring? What > > garbage collector are you using? How often are you updating/committing, > > etc? > > > > > > -------------------------- > > Grant Ingersoll > > http://www.lucidimagination.com/ > > > > Search the Lucene ecosystem (Lucene/Solr/Nutch/Mahout/Tika/Droids) using > > Solr/Lucene: > > http://www.lucidimagination.com/search > > > > > > > > -- > Lance Norskog > goks...@gmail.com >
