RE: NativeFSLockFactory, ConcurrentMergeScheduler: why locks?

2009-08-12 Thread Fuad Efendi
From: Jason Rutherglen [mailto:jason.rutherg...@gmail.com] Sent: August-11-09 8:48 PM To: solr-user@lucene.apache.org Subject: Re: NativeFSLockFactory, ConcurrentMergeScheduler: why locks? > 1 minute of document updates (about 100,000 documents) and then SOLR stops 100,000 docs in a minut

Re: NativeFSLockFactory, ConcurrentMergeScheduler: why locks?

2009-08-11 Thread Jason Rutherglen
roblem...) > > > Thanks, > Fuad > > > > -Original Message- > From: Jason Rutherglen [mailto:jason.rutherg...@gmail.com] > Sent: August-11-09 4:45 PM > To: solr-user@lucene.apache.org > Subject: Re: NativeFSLockFactory, ConcurrentMergeScheduler: why locks?

RE: NativeFSLockFactory, ConcurrentMergeScheduler: why locks?

2009-08-11 Thread Fuad Efendi
lr-user@lucene.apache.org Subject: Re: NativeFSLockFactory, ConcurrentMergeScheduler: why locks? Fuad, The lock indicates to external processes the index is in use, meaning it's not cause ConcurrentMergeScheduler to block. ConcurrentMergeScheduler does merge in it's own thread, howeve

Re: NativeFSLockFactory, ConcurrentMergeScheduler: why locks?

2009-08-11 Thread Jason Rutherglen
Fuad, The lock indicates to external processes the index is in use, meaning it's not cause ConcurrentMergeScheduler to block. ConcurrentMergeScheduler does merge in it's own thread, however if the merges are large then they can spike IO, CPU, and cause the machine to be somewhat unresponsive. Wh

FW: NativeFSLockFactory, ConcurrentMergeScheduler: why locks?

2009-08-11 Thread Fuad Efendi
;t lock isn't it? Why new docs can't be added? Thanks -Original Message- From: Fuad Efendi Sent: August-11-09 1:09 PM To: solr-user@lucene.apache.org Subject: NativeFSLockFactory, ConcurrentMergeScheduler: why locks? 1. I always have files lucene--write.lock and lucen

NativeFSLockFactory, ConcurrentMergeScheduler: why locks?

2009-08-11 Thread Fuad Efendi
1. I always have files lucene--write.lock and lucene--n-write.lock which I believe shouldn't be used with NativeFSLockFactory 2. I use mergeFactor=100 and ramBufferSizeMB=256, few GB indes size. I tried mergeFactor=10 and mergeFactor=1000. It seems ConcurrentMergeSchedul