You could try http://technet.microsoft.com/en-us/sysinternals/bb896653.aspx Erick
On Jan 18, 2008 4:33 AM, Geert-Jan Brits <[EMAIL PROTECTED]> wrote: > Thanks. And on Windows? > > 2008/1/18, Otis Gospodnetic <[EMAIL PROTECTED]>: > > > > Paging - quite possible. Run vmstat 2' on that server when its slow and > > see if there is any paging (look at the 'si' and 'so' columns). > > > > Otis > > -- > > Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch > > > > ----- Original Message ---- > > From: Geert-Jan Brits <[EMAIL PROTECTED]> > > To: solr-user@lucene.apache.org > > Sent: Thursday, January 17, 2008 6:10:56 AM > > Subject: Re: batch indexing takes more time than shown on SOLR output > --> > > something to do with IO? > > > > No Solr-commit is sent until the end. > > Since client and server on this moment are on the same machine network > > IO > > should be small as well I think. Also as you mentioned the response is > > very > > small so it can't be that either. > > > > As to what IO-activity I was thinking about: I was merely guessing > > here, but > > I thought that maybe the creation of indices for indexed fields were > > not > > accounted for in the supplied number. Which is something I can't > > imagine, > > but still with these numbers my head makes all sorts of strange > > scenario's > > ;-) > > > > After checking some other machine stats while doing a big update I > > think I > > know what's going on (please correct me if it doesn't sound plausible): > > client and server (on the same machine with 2GB RAM) are causing > > excessive > > page swapping (on the same disk, yeah I know, I must get a different > > setup) > > which causes SOLR-server to have difficulties with creating its indices > > on > > disk. I think this is going on since all goes pretty good (no big > > discrepencies) until Ram gets's more or less exhausted. > > > > could this be it? I'm going to test with 2 machines I guess. > > > > Thanks, > > Geert-Jan > > > > 2008/1/17, Chris Hostetter <[EMAIL PROTECTED]>: > > > > > > : INFO: {add=[10485, 10488, 10489, 10490, 10491, 10495, 10497, 10498, > > > ...(42 > > > : more) > > > : ]} 0 875 > > > : > > > : However, when timing this instruction on the client-side (I use > > SOlrJ > > > --> > > > : req.process(server)) I get totally different numbers (in the > > beginning > > > the > > > : client-side measured time is about 2 seconds on average but after > > some > > > time > > > : this time goes up to about 30-40 seconds, altough the > > solr-outputted > > > time > > > : stays between 0.8-1.3 seconds? > > > > > > as Otis mentioned, that time is the raw processing of the request, > > not > > > counting any network IO between the client and the server, or any > > time > > > spent by the "ResponseWriter" formating the response. you can get > > more > > > accurate numbers about exctly how long the server spent doing all of > > these > > > things from the access log of your servlet container (which should be > > > recording the time only after every last byte is written back to the > > > client. > > > > > > that said: there's really no reason for as big a descrepency as you > > are > > > describing particularly on updates where the ResposneWriter has > > almost > > > nothing to do (30-40 seconds per update?!?!?!) > > > > > > I'm not very familiar with SolrJ, but are you by any chance using it > > in a > > > way that sends a commit after every update command? (commits can get > > > successifly longer as your index gets bigger.) > > > > > > : Does this have anything to do with costly IO-activity that is > > accounted > > > for > > > : in the SOLR output? If this is true, what tool do you recommend > > using to > > > : monitor IO-activity? > > > > > > Which IO-activity are you talking about? > > > > > > > > > > > > > > > -Hoss > > > > > > > > > > > > > > >
