[
https://issues.apache.org/jira/browse/LUCENE-842?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Thomas Connolly updated LUCENE-842:
-----------------------------------
Description:
When using the org.apache.lucene.search.ParallelMultiSearcher to search on a
single searcher (reading a single index), continuous runs result in a memory
leak.
Substituting the MultiSearcher does not result in a memory leak. and is the
workaround currently used.
And example of the code used is as follows. Note the close routine was added
for the individual searchers and the MultiSearcher otherwise the was a leak in
MultiSearcher.
private void doSearch(Search search)
{
IndexSearcher[] indexSearchers = null;
MultiSearcher multiSearcher = null;
try
{
indexSearchers = getIndexSearcher();
// aggregate the searches across multiple indexes
multiSearcher = new ParallelMultiSearcher(indexSearchers); //
causes LEAK BAD
//multiSearcher = new MultiSearcher(indexSearchers); // NO leak GOOD
final QueryParser parser = new QueryParser("content", new
ExtendedStandardAnalyser());
final Query query = parser.parse(search.getQuery());
final Hits hits = multiSearcher.search(query,
getFilter(search.getFilters()), getSort(search.getSort()));
// process hits...
}
finally
{
close(indexSearchers);
close(multiSearcher);
}
}
/**
* Close the index searchers.
*
* @param indexSearchers Index Searchers.
*/
private static void close(IndexSearcher[] indexSearchers)
{
if (indexSearchers != null)
{
for (IndexSearcher indexSearcher : indexSearchers)
{
try
{
indexSearcher.close();
}
catch (IOException ioex)
{
LOGGER.warn("Unable to close the index searcher!", ioex);
}
}
}
}
/**
* Close the multi-searcher.
*
* @param aMultiSearcher Index Searchers.
*/
private static void close(MultiSearcher aMultiSearcher)
{
try
{
aMultiSearcher.close();
}
catch (IOException ioex)
{
LOGGER.warn("Unable to close the multi searcher!", ioex);
}
}
was:
When using the org.apache.lucene.search.ParallelMultiSearcher to search on a
single searcher (reading a single index), continuous runs result in a memory
leak.
Substituting the MultiSearcher does not result in a memory leak. and is the
workaround currently used.
And example of the code used is as follows. Note the close routine was added
for the individual searchers and the MultiSearcher otherwise the was a leak in
MultiSearcher.
private void doSearch(Search search)
{
IndexSearcher[] indexSearchers = null;
MultiSearcher multiSearcher = null;
try
{
indexSearchers = getIndexSearcher();
// aggregate the searches across multiple indexes
//multiSearcher = new ParallelMultiSearcher(indexSearchers);
multiSearcher = new MultiSearcher(indexSearchers);
final QueryParser parser = new QueryParser("content", new
ExtendedStandardAnalyser());
final Query query = parser.parse(search.getQuery());
final Hits hits = multiSearcher.search(query,
getFilter(search.getFilters()), getSort(search.getSort()));
// process hits...
}
finally
{
close(indexSearchers);
close(multiSearcher);
}
}
/**
* Close the index searchers.
*
* @param indexSearchers Index Searchers.
*/
private static void close(IndexSearcher[] indexSearchers)
{
if (indexSearchers != null)
{
for (IndexSearcher indexSearcher : indexSearchers)
{
try
{
indexSearcher.close();
}
catch (IOException ioex)
{
LOGGER.warn("Unable to close the index searcher!", ioex);
}
}
}
}
/**
* Close the multi-searcher.
*
* @param aMultiSearcher Index Searchers.
*/
private static void close(MultiSearcher aMultiSearcher)
{
try
{
aMultiSearcher.close();
}
catch (IOException ioex)
{
LOGGER.warn("Unable to close the multi searcher!", ioex);
}
}
Updated the code example to leak.
Doron what do you suggest? Do you want me to attached the search index,
libraries, and a main routine? It is pretty easy to execute this code, I'm
simply showing the simple logic.
The first close, on the searchers, should not be necessary as the MultiSearcher
should close all searchers, but I found it is.
> ParallelMultiSearcher memory leak
> ---------------------------------
>
> Key: LUCENE-842
> URL: https://issues.apache.org/jira/browse/LUCENE-842
> Project: Lucene - Java
> Issue Type: Bug
> Components: Search
> Affects Versions: 2.1
> Environment: Windows XP SP2 and Red Hat EL 4
> Reporter: Thomas Connolly
> Priority: Critical
>
> When using the org.apache.lucene.search.ParallelMultiSearcher to search on a
> single searcher (reading a single index), continuous runs result in a memory
> leak.
> Substituting the MultiSearcher does not result in a memory leak. and is the
> workaround currently used.
> And example of the code used is as follows. Note the close routine was added
> for the individual searchers and the MultiSearcher otherwise the was a leak
> in MultiSearcher.
> private void doSearch(Search search)
> {
> IndexSearcher[] indexSearchers = null;
>
> MultiSearcher multiSearcher = null;
> try
> {
> indexSearchers = getIndexSearcher();
>
> // aggregate the searches across multiple indexes
> multiSearcher = new ParallelMultiSearcher(indexSearchers); //
> causes LEAK BAD
> //multiSearcher = new MultiSearcher(indexSearchers); // NO leak
> GOOD
> final QueryParser parser = new QueryParser("content", new
> ExtendedStandardAnalyser());
> final Query query = parser.parse(search.getQuery());
>
> final Hits hits = multiSearcher.search(query,
> getFilter(search.getFilters()), getSort(search.getSort()));
> // process hits...
> }
> finally
> {
> close(indexSearchers);
> close(multiSearcher);
> }
> }
> /**
> * Close the index searchers.
> *
> * @param indexSearchers Index Searchers.
> */
> private static void close(IndexSearcher[] indexSearchers)
> {
> if (indexSearchers != null)
> {
> for (IndexSearcher indexSearcher : indexSearchers)
> {
> try
> {
> indexSearcher.close();
> }
> catch (IOException ioex)
> {
> LOGGER.warn("Unable to close the index searcher!", ioex);
> }
> }
> }
> }
>
> /**
> * Close the multi-searcher.
> *
> * @param aMultiSearcher Index Searchers.
> */
> private static void close(MultiSearcher aMultiSearcher)
> {
> try
> {
> aMultiSearcher.close();
> }
> catch (IOException ioex)
> {
> LOGGER.warn("Unable to close the multi searcher!", ioex);
> }
> }
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
