Re: Indexing slowdowns

2010-07-08 Thread Mark Miller
On 7/8/10 8:55 PM, Yonik Seeley wrote: > Hmm, did the default number of background merge threads change > sometime recently? I seem to recall so, but I can't find a reference > to it. > > -Yonik > http://www.lucidimagination.com It did change - from 3 to 1-3: maxThreadCount = Math.max(1, Math.m

Re: Indexing slowdowns

2010-07-08 Thread Yonik Seeley
Hmm, did the default number of background merge threads change sometime recently? I seem to recall so, but I can't find a reference to it. -Yonik http://www.lucidimagination.com

Re: Indexing slowdowns

2010-07-08 Thread Robert Muir
On Thu, Jul 8, 2010 at 7:44 PM, Mark Holland wrote: > > Can anyone suggest where I might start looking for answers? I have a > yourkit > snapshot if anyone would care to see it. > > Doesn't sound good. I'd like to see whatever data you can provide (i worry it might be something in analysis) --

Indexing slowdowns

2010-07-08 Thread Mark Holland
Since I began using the 2010-05-18 nightly I'm experiencing indexing slow downs which I didn't with solr-1.4. I'm seeing indexing slow down roughly every 7m records. I'm indexing about 28m in total. These records are batched into csv files of 1m rows, which are loaded with stream.file. Solr happil