[
https://issues.apache.org/jira/browse/PYLUCENE-2?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andi Vajda resolved PYLUCENE-2.
-------------------------------
Resolution: Invalid
Please reopen once you are sure this occurs only with PyLucene.
> Memory leak when searching in real time reader
> ----------------------------------------------
>
> Key: PYLUCENE-2
> URL: https://issues.apache.org/jira/browse/PYLUCENE-2
> Project: PyLucene
> Issue Type: Bug
> Environment: ubuntu 9.10, python 2.6, pylucene 3.0
> Reporter: feng xiaojie
> Priority: Critical
> Original Estimate: 336h
> Remaining Estimate: 336h
>
> Our codes is following:
> We have 31 index dirs in /tmp (there are about 5 million records in our
> indexs), we want to real time search, so we use the writer.getReader() to get
> the real time reader.
> Then we did the search repeatly, finally java 'out of memory' issue will
> happen(about 10 minites).
> initVM(CLASSPATH,initialheap='100m',maxheap='100m')
> keywordQuery = QueryParser(Version.LUCENE_CURRENT,"content",
> StandardAnalyzer(Version.LUCENE_CURRENT)).parse("when AND you")
> writers = []
> for i in range(1,32):
> dir = os.path.join("/tmp",str(i))
>
> luceneDir = SimpleFSDirectory(File(dir))
>
> writer = IndexWriter(luceneDir,
> StandardAnalyzer(Version.LUCENE_CURRENT),
> False,IndexWriter.MaxFieldLength.LIMITED)
> writer.setRAMBufferSizeMB(32.0)
> writer.setUseCompoundFile(True)
> writer.setMergeFactor(10)
> writers.append(writer)
>
> while True:
> searchersList = []
> readers = []
> for writer in writers:
> reader = writer.getReader()
> searcher = IndexSearcher(reader)
> searchersList.append(searcher)
> readers.append(reader)
> multiSearcherInstance = MultiSearcher(searchersList)
> docs =
> multiSearcherInstance.search(keywordQuery,IndexerCons.TOP_DOC_NUMBER).scoreDocs
>
> multiSearcherInstance.close()
> for searcher in searchersList:
> searcher.close()
> for reader in readers:
> reader.close()
> Then we use the normal reader (directly open from the dirs) instead of the
> real time reader, the test is OK, no 'out of memory' issue.
> The bug maybe come from java lucene, i don't sure.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
