Not closing an IndexReader most probably (to say the least) results in a
mem-leak -> OOM
> But if I close if given that it is share by multiple threads I will
>need to check each time before doing the search if IndexReader is still open
>correct?
You can make use of IndexReader#incRef/#decRef , i.e.
ir.incRef();
try
{
<your search code>
}
finally
{
ir.decRef();
}
...
IFF ir.getRefCount() > 1 THEN you are safe to close the "old" ir.
Maybe SearcherManager
http://blog.mikemccandless.com/2011/09/lucenes-searchermanager-simplifies.html
fits your needs?
-----Ursprüngliche Nachricht-----
Von: Liviu Matei [mailto:liviu...@gmail.com]
Gesendet: Mittwoch, 14. Mai 2014 11:06
An: java-user@lucene.apache.org
Betreff: Issue with Lucene 3.6.1 and MMapDirectory
Hi,
I am encountering the following issue with Lucene 3.6.1 if you could let me
know if I am doing something wrong / there is a mistake I am making it would be
great.
In order to improve the performance of the application that I am working at I
went to the approach of reusing the IndexReader and reopening it every 8 hours
in order to get the latest changes. (IndexReader is declared as a global static
variable). The search method is called from multiple threads in parallel so the
index reader is shared between threads. Now if I don't close the old index
reader I am noticing increases of virtual memory with every new reindex reopen
(this should not be an issue on 64 bit Linux correct - this is the
configuration I am using and the indexes are on a shared mount NTFS file system
).
Also from time to times I noticed JVM crasches with the following stack:
Thread Stack Trace:
at memcpy+160()@0x381aa7b060
-- Java stack --
at java/nio/DirectByteBuffer.get(DirectByteBuffer.java:294)[optimized]
at
org/apache/lucene/store/MMapDirectory$MMapIndexInput.readBytes(MMapDirectory.java:298)[optimized]
at org/apache/lucene/store/DataInput.readBytes(DataInput.java:72)
at
org/apache/lucene/index/CompoundFileReader$CSIndexInput.readInternal(CompoundFileReader.java:275)[optimized]
at
org/apache/lucene/store/BufferedIndexInput.refill(BufferedIndexInput.java:270)[optimized]
at
org/apache/lucene/store/BufferedIndexInput.readByte(BufferedIndexInput.java:40)[inlined]
at
org/apache/lucene/store/DataInput.readVInt(DataInput.java:107)[inlined]
at
org/apache/lucene/store/BufferedIndexInput.readVInt(BufferedIndexInput.java:217)[optimized]
at org/apache/lucene/index/FieldsReader.doc(FieldsReader.java:235)
at
org/apache/lucene/index/SegmentReader.document(SegmentReader.java:492)
at
org/apache/lucene/index/DirectoryReader.document(DirectoryReader.java:568)
at org/apache/lucene/index/MultiReader.document(MultiReader.java:252)
at org/apache/lucene/index/IndexReader.document(IndexReader.java:1138)
at
org/apache/lucene/search/IndexSearcher.doc(IndexSearcher.java:258)[inlined]
Can you please tell me if all this corruption is caused by the fact that I am
not closing the old IndexReader. But if I close if given that it is share by
multiple threads I will need to check each time before doing the search if
IndexReader is still open correct? Let's say in a thread I am reopening the
IndexReader and in another thread I am afterwards reusing the old one in that
case I should do the check correct? Or is there a smarter mechanism in place.
Any help with this would be more than welcome.
Thank you very much and best regards,
Liviu
