Re: Homogeneous vs Heterogeneous indexes (was: FileNotFoundException)

On Tuesday, April 30, 2002, at 01:57 AM, Steven J. Owens wrote: > Just be glad you aren't doing this on Solaris with JDK 1.1.6 I know... In fact I'm looking forward to port my stuff to 1.4... As my app is very much IO bond I'm really excited by this nio madness... :-) > Yes and no. Setting ul

Re: Homogeneous vs Heterogeneous indexes (was: FileNotFoundException)

petite, On Mon, Apr 29, 2002 at 07:54:43PM +0200, petite_abeille wrote: > As a final note, several people suggested to increase the number of file > descriptors per process with something like "ulimit"... Just be glad you aren't doing this on Solaris with JDK 1.1.6, where I first ran into

Re: Lucene's scalability

Great, Thanks for the quick response, I am very interested in hearing how lucene handles itself in the 15-20 million doc range. I will be doing some testing this week with lucene and will report my findings as well. I am also testing FAST and AltaVista and I will post some comparison details.

Re: Homogeneous vs Heterogeneous indexes (was: FileNotFoundException)

On Mon, 29 Apr 2002, petite_abeille wrote: > As a final note, several people suggested to increase the number of > file descriptors per process with something like "ulimit"... From what > I learned today, I think it's a *bad* idea to have to change some > system parameters just because your/my ap

RE: Lucene's scalability

I currently have an index of ~ 12 million documents, which are each about that size (but in xml form). When they are transformed for lucene to index, there are upwards of 50 searchable fields. The index is about 10 GB right now. I have not yet had any problems with "pushing the limits" of lucen

Lucene's scalability

Is there a known limit to the number of documents that Lucene can handle efficiently? I'm looking to index around 15 million, 2K docs which contain 7-10 searchable fields. Should I be attempting this with Lucene? Thanks, Joel

Homogeneous vs Heterogeneous indexes (was: FileNotFoundException)

First of, thanks to Jagadesh Nandasamy who directed me to the right direction. It seems, that in my situation, more homogeneous indexes work better than fewer heterogeneous indexes: I have a dozen class that I'm indexing. They vary from two fields to more than a dozen field per document (aka

Re: rc4 and FileNotFoundException: an update

Hello, > > and what was their ulimit and what is the > > ulimit on your OSX machine? > > Just curious. > > I don't know. Does it matter? Of course it does - a low (u)limit is a part of your problem, perhaps. Otis P.S. I don't know how Winblows deals with file descriptors. Try your applicatio

Re: FileNotFoundException: code example

Hello, I'll put my comments inline... --- petite_abeille <[EMAIL PROTECTED]> wrote: > Hello again, > > attached is the source code of the only class interacting directly > with > Lucene in my app. Sorry for not providing a complete test case as > it's > hard for me to come up with something s

Re: rc4 and FileNotFoundException: an update

> Does this mean you tried it on other OSs and it worked? Yes. > Which ones? Win2k SP2 > What JDK did those have jre 1.4.0 > and what was their ulimit and what is the > ulimit on your OSX machine? > Just curious. I don't know. Does it matter? PA -- To unsubscribe, e-mail:

Re: rc4 and FileNotFoundException: an update

--- petite_abeille <[EMAIL PROTECTED]> wrote: > > I don't know what environment you're using Lucene in. > > The problem seems to be specially bad on osx (10.1.4 + JRE 1.3.1 + > latest updates). Does this mean you tried it on other OSs and it worked? Which ones? What JDK did those have and what

Re: FileNotFoundException: code example

> > I would add some logging to the code > > You lost me here... Where should I add some logging? At the top of every method, or next to every lucene call, or wherever anything interesting happens. System.out.println("Calling search()"); or use log4j or whatever you prefer. > > to get more

Re: FileNotFoundException: code example

Hi petite, I will try to be brief... In lucene the number of files created depends on the number of fields the document has so lets take an example you want to index 100 files if each file contains 10 fields document.add(Field.Text(UNIQUE_ID, "1234

Re: too many open files in system

> how many open files you think can be used at your process?? Not sure. It varies with usage pattern. I will check it out in any case. > cat /proc/sys/fs/file-max cat: /proc/sys/fs/file-max: No such file or directory > echo 5 > /proc/sys/fs/file-max Unfortunately, I cannot use this kind o

[Off-List] Too Many Open Files

Heya Folks... Julian (sitting in front of me and looking bad... hi Jules) told me that one of you guys had a problem with Lucene and a "Too Many Files Open" exception... Reading back from the archives, I found this:

Re: FileNotFoundException: code example

> I would add some logging to the code You lost me here... Where should I add some logging? > to get more idea of which Lucene methods are > actually being called, when, in what sequence. I typical sequence looks like that: - search() - deleteIndexWithID() - indexValuesWithID() PA -- To u

Re: too many open files in system

petite_abeille wrote: >> On Tuesday, 9. April 2002 14:08, you wrote: >> >>> root wrote: >>> Doesn't Lucene releases the filehandles?? because I get "too many open files in system" after running lucene a while! >>> >>> >>> Are you closing the readers and writers after you've fi

Re: FileNotFoundException: code example

Hello I can't spot anything but I'm definitely not the world's greatest Lucene expert and maybe someone else will. Perhaps the stack trace will help too. Failing that, and if you still can't come up with a complete self-contained test case to show the problem, I would add some logging to the c

Re: Italian web sites

The first one. Bye Laura > What does it mean? "Italian website" can be: > - site that use italian language > - site owned by an italian organization > - site hosted in a italian geographical site > Every definition has a different solution. > > Date sent:Wed, 24 Ap

Re: too many open files in system

> On Tuesday, 9. April 2002 14:08, you wrote: >> root wrote: >>> Doesn't Lucene releases the filehandles?? >>> >>> because I get "too many open files in system" after running lucene a >>> while! >> >> Are you closing the readers and writers after you've finished using >> them? >> >> cheers, >> >>

Re: rc4 and FileNotFoundException: an update

> I don't know what environment you're using Lucene in. The problem seems to be specially bad on osx (10.1.4 + JRE 1.3.1 + latest updates). PA. -- To unsubscribe, e-mail: For additional commands, e-mail:

Re: rc4 and FileNotFoundException: an update

> I don't know what environment you're using Lucene in. However, we had > this "too > many open files" problem on our Solaris box, and increasing the number > of file > descriptors through the ulimit -n command fixed it. Thanks. That should help. However, I have a little desktop app and it wil

Re: rc4 and FileNotFoundException: an update

Hi Petite, > "SZFinder.findObjectsWithSpecificationInStore: java.io.FileNotFoundException: _2.f14 (Too many open files)" I don't know what environment you're using Lucene in. However, we had this "too many open files" problem on our Solaris box, and increasing the number of file descriptors th