Michael McCandless wrote:
Technically you should call directory.close() as well, but missing
that will not lead to too many open files.
How often is that RuntimeException being thrown? EG if a single
document is frequently hitting an exception during analysis, your code
doesn't close the IndexWriter in that situation. It's better to use a
try/finally and close the IndexWriter in the finally clause, to cover
that case.
Are you sure nothing else is using up file descriptors? EG the
createDocument call does not open any files?
Mike
The runtimeException is occurring all the time, Im waiting for some more
information from the user. Since the post I've since added
directory.close() too, I thought this would cause a problem when I call
IndexSearcher with it as a parameter but it seems to still work - the
documentation is not very clear on this point. I see your poibnt about
the try/finally I'll make that chnage.
There are many other parts of the code that use filedescriptors, but the
problem has never occurred before moving to a FSDirectory
thanks paul
heres an example of my search code, is this ok ?
public boolean recNoColumnMatchesSearch(Integer columnId, Integer recNo,
String search)
{
try
{
IndexSearcher is = new IndexSearcher(directory);
//Build a query based on the fields, searchString and
standard analyzer
QueryParser parser = new
QueryParser(String.valueOf(columnId) + INDEXED, analyzer);
Query query = parser.parse(search);
MainWindow.logger.finer("Parsed Search Query Is" +
query.toString() + "of type:" + query.getClass());
//Create a filter,to restrict search to one row
Filter filter = new QueryFilter(new TermQuery(new
Term(ROW_NUMBER, String.valueOf(recNo))));
//run the search
Hits hits = is.search(query, filter);
Iterator i = hits.iterator();
if (i.hasNext())
{
return true;
}
}
catch (ParseException pe)
{
//Problem with syntax rather than throwing exception and
causing everything to stop we just
//log and return false
MainWindow.logger.warning("Search Query invalid:" +
pe.getMessage());
return false;
}
catch (IOException e)
{
MainWindow.logger.warning("DataIndexer.Unable to do perform
reno match search:" + search + ":" + e);
}
return false;
Paul Taylor wrote:
Hi, I have been using a RAMDirectory for indexing without any
problem, but I then moved to a file based directory to reduce memory
usage. this has been working fine on Windows and OSX and my version
of linux (redhat) but is failing on a version of linux (archlinux)
with 'Too many files opened' , but they are only indexing 32
documents , I can index thousands without a problem. It mentions this
error in the Lucene FAQ but I am not dealing directly with the
filesystem myself, this is my code for creating an index is it okay
or is there some kind of close that I am missing
thanks for any help Paul
public synchronized void reindex()
{
MainWindow.logger.info("Reindex start:" + new Date());
TableModel tableModel = table.getModel();
try
{
//Recreate the RAMDirectory uses too much memory
//directory = new RAMDirectory();
directory =
FSDirectory.getDirectory(Platform.getPlatformLicenseFolder()+ "/" +
TAG_BROWSER_INDEX);
IndexWriter writer = new IndexWriter(directory, analyzer,
true);
//Iterate through all rows
for (int row = 0; row < tableModel.getRowCount(); row++)
{
//for each row make a new document
Document document = createDocument(row);
writer.addDocument(document);
}
writer.optimize();
writer.close();
}
catch (Exception e)
{
throw new RuntimeException("Problem indexing Data:" +
e.getMessage());
}
}
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
