Hi all,
We're observing search threads slowing down during directory copies performed
during updates to the index. The thread dump shows search threads blocked on a
FSDirectory$FSIndexInput$Descriptor instance:
"Worker Thread - 12" daemon prio=10 tid=0x082b2400 nid=0x4654 waiting for
monitor entry [0x988ed000..0x988edf30]
java.lang.Thread.State: BLOCKED (on object monitor)
at
org.apache.lucene.store.FSDirectory$FSIndexInput.readInternal(FSDirectory.java:542)
- waiting to lock <0xc79c9900> (a
org.apache.lucene.store.FSDirectory$FSIndexInput$Descriptor)
at
org.apache.lucene.store.BufferedIndexInput.refill(BufferedIndexInput.java:152)
at
org.apache.lucene.store.BufferedIndexInput.readByte(BufferedIndexInput.java:38)
at org.apache.lucene.store.IndexInput.readVInt(IndexInput.java:76)
at org.apache.lucene.index.SegmentTermDocs.read(SegmentTermDocs.java:133)
at
org.apache.lucene.index.MultiSegmentReader$MultiTermDocs.read(MultiSegmentReader.java:573)
at org.apache.lucene.search.TermScorer.next(TermScorer.java:106)
at
org.apache.lucene.search.ConjunctionScorer.init(ConjunctionScorer.java:80)
at
org.apache.lucene.search.ConjunctionScorer.next(ConjunctionScorer.java:48)
at org.apache.lucene.search.BooleanScorer2.score(BooleanScorer2.java:319)
at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:146)
at org.apache.lucene.search.Searcher.search(Searcher.java:118)
...
The directory copy we do is a 'cp -pr <src-dir>/* <dest-dir>' command prior to
applying changes (addDocument calls) on the current "available" segment. This
takes >7 mins to copy a directory of size 1.4G. During this time window, the
searches are slow and the above thread stacks are observed.
Could there be any system level limits we're hitting?
Our test environment is:
lucene-2.3.2
4x2.6 GHz, 16G memory
Red Hat 3.4.6-9
Thanks and regards,
- Gagan
