Re: Too many open files issue

John Wang wrote: In the Lucene code, I don't see where the reader speicified when creating a field is closed. That holds on to the file. I am looking at DocumentWriter.invertDocument() It is closed in a finally clause on line 170, when the TokenStream is closed. Doug -

Re: Too many open files issue

I have also seen this problem. In the Lucene code, I don't see where the reader speicified when creating a field is closed. That holds on to the file. I am looking at DocumentWriter.invertDocument() Thanks -John On Mon, 22 Nov 2004 16:21:35 -0600, Chris Lamprecht <[EMAIL PROTECTED]> wrote: >

Re: Too many open files issue

A useful resource for increasing the number of file handles on various operating systems is the Volano Report: http://www.volano.com/report/ > I had requested help on an issue we have been facing with the "Too many > open files" Exception garbling the search indexes and crashing the > search on t

Re: Too many open files issue

I'm sorry, I wasn't involved in the original conversation but maybe I can jump in with some info that will help. The number of files depends on the merge factor, number of segments, and number of indexed fields in your index. It also depends on whether you are using "compound files" or not (thi

RE: Too many open files issue

If you are on linux the number of file handles for a session is much lower than that for the whole machine. "ulimit -n" will tell you. There are instructions on the web for changing this setting, it involves the /etc/security/limits.conf and setting the values for "nofile". (bulkadm is my use

RE: Too many Open Files + lucene 1.4.1 + Linux O/s

Original Message- From: Dmitry Serebrennikov [mailto:[EMAIL PROTECTED] Sent: Sunday, October 03, 2004 5:08 AM To: Lucene Users List Subject: Re: Too many Open Files + lucene 1.4.1 + Linux O/s Karthik N S wrote: >Hi Luceners, > > >Apologies. > > >Other day

Re: Too many Open Files + lucene 1.4.1 + Linux O/s

Karthik N S wrote: Hi Luceners, Apologies. Other day was Trying to Search using the "Luceneweb" version with Lucene1-4-1.zip and O/s = Linux, J2SDK version "1.4.2_03-b02" With Roughly around 500 Documents (715116 kb ) Indexed using Lucene1.4-final.jar and writer.setUseCompoundFile(t

RE: too many open files

I suspect it has to do with this change: --- jakarta-lucene/src/java/org/apache/lucene/index/SegmentMerger.java 2004/08/08 13:03:59 1.12 +++ jakarta-lucene/src/java/org/apache/lucene/index/SegmentMerger.java 2004/08/11 17:37:52 1.13 I wouldn't know where to start to reproduce the pro

Re: too many open files

Patrick, For your second problem, are you seeing a behavior similar to the one discussed in the following thread? http://www.mail-archive.com/[EMAIL PROTECTED]/msg08952.html If yes, you can see the solution there. Thanks, George --- Patrick Kates <[EMAIL PROTECTED]> wrote: > I am having tw

Re: Too many open files error occurs when changing 1.3 final to 1.4 rc2

I am not 100% certain now, but I _think_ there were some changes that required that you re-index your data when upgrading to 1.4rc2. I would check the CHANGES file (link on the site, just look at the complete file). Otis --- juan lu <[EMAIL PROTECTED]> wrote: > I had been using 1.3 final for 1 m

RE: Too Many Open Files

Thanks Doug! Reducing the MergeFactor to 10 reduced the number of files in the index dramatically. -Reece -Original Message- From: Doug Cutting [mailto:[EMAIL PROTECTED] Sent: Tuesday, October 07, 2003 8:20 PM To: Lucene Users List Subject: Re: Too Many Open Files Wilton, Reece wrote

Re: Too Many Open Files

to:[EMAIL PROTECTED] Sent: Tuesday, October 07, 2003 8:20 PM To: Lucene Users List Subject: Re: Too Many Open Files Wilton, Reece wrote: > The index directory that Lucene created has 2,322 files in it. When I > try to open it I get the dreaded "Too Many Open Files" problem: >

RE: Too Many Open Files

: Lucene Users List Subject: Re: Too Many Open Files Wilton, Reece wrote: > The index directory that Lucene created has 2,322 files in it. When I > try to open it I get the dreaded "Too Many Open Files" problem: > java.io.FileNotFoundException: C:\Index\_1lvq.f107 (Too many ope

Re: Too Many Open Files

Wilton, Reece wrote: The index directory that Lucene created has 2,322 files in it. When I try to open it I get the dreaded "Too Many Open Files" problem: java.io.FileNotFoundException: C:\Index\_1lvq.f107 (Too many open files) The index has about 50,000 docs in it. It was created with a merg

Re: Too Many Open Files

try to check your codes if you are openning a file, close it after using. --buics "Wilton, Reece" <[EMAIL PROTECTED]> wrote: Hi, The index directory that Lucene created has 2,322 files in it. When I try to open it I get the dreaded "Too Many Open Files" problem: java.io.FileNotFoundException: C

RE: Too many open files?

> -Original Message- > From: Hang Li [mailto:[EMAIL PROTECTED]] > Sent: Tuesday, July 23, 2002 5:22 PM > To: Lucene Users List > Subject: Re: Too many open files? > > > I did close searcher after each done. Maybe I should try the > CachedSearcher someone po

Re: Too many open files?

I cached searcher now. As the results, multiple threads try to use the same searcher. It seems it is much SLOWER than each thread has its own searcher. Are there any synchronized methods/blocks in Lucene causing this performance problem? Scott Ganyo wrote: > Yup. Cache and reuse your Searcher

Re: Too many open files?

just close your Searcher after finishing work - Original Message - From: "Hang Li" <[EMAIL PROTECTED]> To: "Lucene Users List" <[EMAIL PROTECTED]> Sent: Tuesday, July 23, 2002 5:59 PM Subject: Too many open files? > > > > I have seen a lot postings about this topic. Any final thoughts?

Forked files? was: RE: Too many open files?

y, July 23, 2002 10:13 AM > To: 'Lucene Users List' > Subject: RE: Too many open files? > > > Are you closing the searcher after each when done? > > No: Waiting for the garbage collector is not a good idea. > > Yes: It could be a timeout on the OS holding the

Re: Too many open files?

ld be a timeout on the OS holding the files handles. > > Either way, the only real option is to avoid thrashing the searchers... > > Scott > > > -Original Message- > > From: Hang Li [mailto:[EMAIL PROTECTED]] > > Sent: Tuesday, July 23, 2002 10:10 AM > >

RE: Too many open files?

: Hang Li [mailto:[EMAIL PROTECTED]] > Sent: Tuesday, July 23, 2002 10:10 AM > To: Lucene Users List > Subject: Re: Too many open files? > > > Thanks for your quick reponse, I still want to know why we ran out of > file descriptors. > > --Yup. Cache and reuse

Re: Too many open files?

Thanks for your quick reponse, I still want to know why we ran out of file descriptors. --Yup. Cache and reuse your Searcher as much as possible. --Scott > -Original Message- > From: Hang Li [mailto:[EMAIL PROTECTED]] > Sent: Tuesday, July 23, 2002 9:59 AM > To: Lucene Users List > Sub

RE: Too many open files?

Yup. Cache and reuse your Searcher as much as possible. Scott > -Original Message- > From: Hang Li [mailto:[EMAIL PROTECTED]] > Sent: Tuesday, July 23, 2002 9:59 AM > To: Lucene Users List > Subject: Too many open files? > > > > > > I have seen a lot postings about this topic. Any fi

Re: too many open files in system

> how many open files you think can be used at your process?? Not sure. It varies with usage pattern. I will check it out in any case. > cat /proc/sys/fs/file-max cat: /proc/sys/fs/file-max: No such file or directory > echo 5 > /proc/sys/fs/file-max Unfortunately, I cannot use this kind o

Re: too many open files in system

petite_abeille wrote: >> On Tuesday, 9. April 2002 14:08, you wrote: >> >>> root wrote: >>> Doesn't Lucene releases the filehandles?? because I get "too many open files in system" after running lucene a while! >>> >>> >>> Are you closing the readers and writers after you've fi

Re: too many open files in system

> On Tuesday, 9. April 2002 14:08, you wrote: >> root wrote: >>> Doesn't Lucene releases the filehandles?? >>> >>> because I get "too many open files in system" after running lucene a >>> while! >> >> Are you closing the readers and writers after you've finished using >> them? >> >> cheers, >> >>

RE: too many open files in system

- > From: Nader S. Henein [mailto:[EMAIL PROTECTED]] > Sent: Tuesday, April 09, 2002 11:09 AM > To: Lucene Users List; [EMAIL PROTECTED]; [EMAIL PROTECTED] > Subject: RE: too many open files in system > > > that might be the case I'm indexing 200 000 files each one > h

RE: too many open files in system

that might be the case I'm indexing 200 000 files each one has about 30 XML fields each one has a set of attributes .. could that be it ? -Original Message- From: Karl Øie [mailto:[EMAIL PROTECTED]] Sent: Tuesday, April 09, 2002 7:03 PM To: Lucene Users List Subject: Re: too many

Re: too many open files in system

I have worked a little with the cocoon indexer and it indexes each xml-attribute in a Field. I have done some indexing on both plaintext and xml sources and i think the "Too many open files" problem is directly related to number of fields stored in a document in a index. the reason for this is

RE: too many open files in system

nt indexed ? , becasue that would give me an idea of how many > file > handles > I would need > > -Original Message- > From: Otis Gospodnetic [mailto:[EMAIL PROTECTED]] > Sent: Tuesday, April 09, 2002 6:42 PM > To: Lucene Users List; [EMAIL PROTECTED] > Subject:

RE: too many open files in system

want indexed ? , becasue that would give me an idea of how many file handles I would need -Original Message- From: Otis Gospodnetic [mailto:[EMAIL PROTECTED]] Sent: Tuesday, April 09, 2002 6:42 PM To: Lucene Users List; [EMAIL PROTECTED] Subject: RE: too many open files in system This sounds l

Re: too many open files in system

Judging from other messages in this thread it seems that the cause of your problem could be an unoptimized index (somebody said that lots of files need to be opened for searches). Try optimizing your index. Optimizing an index will reduce the number of files comprising your index. Otis --- roo

RE: too many open files in system

then i set my > XSLT > parser to that file.. ? > > I am very curious about this whole thing and any help would be a > super help. > > Regards, > Ian > > -Original Message- > From: Nader S. Henein [mailto:[EMAIL PROTECTED]] > Sent: Tuesday, Apri

RE: too many open files in system

4:50 PM To: Lucene Users List Subject: Re: too many open files in system On Tuesday, 9. April 2002 14:08, you wrote: > root wrote: > > Doesn't Lucene releases the filehandles?? > > > > because I get "too many open files in system" after running lucene a > >

RE: too many open files in system

List Subject: Re: too many open files in system On Tuesday, 9. April 2002 14:08, you wrote: > root wrote: > > Doesn't Lucene releases the filehandles?? > > > > because I get "too many open files in system" after running lucene a > > while! > > Are you

Re: too many open files in system

On Tuesday, 9. April 2002 14:08, you wrote: > root wrote: > > Doesn't Lucene releases the filehandles?? > > > > because I get "too many open files in system" after running lucene a > > while! > > Are you closing the readers and writers after you've finished using them? > > cheers, > > Chris Yes

RE: too many open files in system

it's not a matter of releasing the handles, it needs to keep them open, this tricked me as well I thought it kept the file handles of the source XML files open, but if you look at the code it actually reads the contents of the files from an HTTP request, the file handles are consumed by the files

Re: too many open files in system

root wrote: > > Doesn't Lucene releases the filehandles?? > > because I get "too many open files in system" after running lucene a while! Are you closing the readers and writers after you've finished using them? cheers, Chris -- To unsubscribe, e-mail: For additi

Re: Too many open files exception...

addikar" <[EMAIL PROTECTED]> Sent: Monday, March 18, 2002 10:55 AM Subject: Re: Too many open files exception... : Nikhil: : : I don't have an answer to your question, but I can tell you that you're : not likely to get one unless you supply a sample of code that ge