Re: Fastest way to fetch N documents with unique keys within large numbers of indexes..

Kevin, Those results are awsome. Could you please give those of us that were following but not quite understanding everything some pseudo code or some more explaination? Thanks, andrew -Original Message- From: Kevin Burton <[EMAIL PROTECTED]> Sent: Jun 7, 2005 7:18 PM To: java-user@

About Stemmers and Analyzers

I have 2 stop words list (the platform works with two language at same time, and it's very normal that the people use in the search)and I've to create a new class that extends Analyzer true? I'm basing on Gerhard Schwarz's work for GermanAnalyzer. How have I to compile all the trunk/contrib/an

OR query on multiple fields causes low coord

Hi, I have a question with regards to an OR query on multiple fields. It seems that the more fields I'm splitting the documents into, the lower the coord is getting. As a result when I want to query the string S on many fields (a query like F1:(S) F2:(S) ... Fn:(S) ) I'm getting close-to-zero c

Source Code of Zilverline Search Engine version 1.4.0 released

All, I've just released the Source Code of Zilverline version 1.4.0. Please take look at http://www.zilverline.org, and have a swing at it. cheers, Michael Franken - To unsubscribe, e-mail: [EMAIL PROTECTED] For additi

Exception in full text search

Hello, I'm able to create index file for full text search.And i'm sure it has the required entries as i have traced the traversal path through the tables i have specified. And also documents are added to the index file. But when i specify some string to search,it throws an exception like t

Re: OR query on multiple fields causes low coord

On Thursday 09 June 2005 13:14, M. Mokotov wrote: > Hi, > > I have a question with regards to an OR query on multiple fields. > > It seems that the more fields I'm splitting the documents into, the lower > the coord is getting. > As a result when I want to query the string S on many fields (a q

Re: Exception in full text search

hi, this exception does not seems to be lucene related concerning the way to look at index content, you could try luke (http://www.getopt.org/luke/) hope it will help --- sven Le jeudi 9 juin 2005 à 18:06:07, vous écriviez : a> Hello, a> I'm able to create index file for full text sear

Re: Fastest way to fetch N documents with unique keys within large numbers of indexes..

Andrew Boyd wrote: Kevin, Those results are awsome. Could you please give those of us that were following but not quite understanding everything some pseudo code or some more explaination? Ug.. I hate to say this bug ignore these numbers. Turns out that I was hitting a cache ... I thou

Optimizing indexes with mulitiple processors?

Is it possible to get Lucene to do an index optimize on multiple processors? Its a single threaded algorithm currently right? Its a shame since I have a quad machine but I'm only using 1/4th of the capacity. Thats a heck of a performance hit. Kevin -- Use Rojo (RSS/Atom aggregator)! - v

Re: Indexing from multiple applications to a central index.

I think your setup is right for a centralized IndexQueueManager that is subscribed to topics to which your distributed servers push data to index via JMS. That way you get an easy way to add more machines to the cluster, you get persistence of not-yet-indexed data, and you get a queuing mechanism

Re: Optimizing indexes with mulitiple processors?

Optimize is disk I/O bound. So I am not sure what multiple CPUs will buy you. Bill On 6/9/05, Kevin Burton <[EMAIL PROTECTED]> wrote: > Is it possible to get Lucene to do an index optimize on multiple > processors? > > Its a single threaded algorithm currently right? > > Its a shame since I ha

Re: Optimizing indexes with mulitiple processors?

You can segment your indexes into n physical parts (perhaps 4), then index those n parts concurrently. When you query you will use some kind of mulit searcher to span the parts. The one thing you may care about is that if you are going todo a recrawl / update of documents against the existing ind

Lucene 1.4.3 QueryParser cannot parse "great!" ?

Hi, We are using lucene 1.4.3, we indexed a string Company picnic is great! by IndexWriter created with StandardAnalyzer, into a field called "all". I also confirmed StandardAnalyzer processed my input string as tokens: company picnic great as I expected. But when I try to create Query by Qu

Re: Lucene 1.4.3 QueryParser cannot parse "great!" ?

Here is a list of special characters that must be excaped in a query. + - && || ! ( ) { } [ ] ^ " ~ * ? : \ Query q = QueryParser.parse("great\!", "all", new StandardAnalyzer()); On 6/9/05, Zhang, Lisheng <[EMAIL PROTECTED]> wrote: > Hi, > > We are using lucene 1.4.3, we indexed a string > >

RE: Lucene 1.4.3 QueryParser cannot parse "great!" ?

Hi Richard, Thanks very much! That works. Lisheng -Original Message- From: Richard Krenek [mailto:[EMAIL PROTECTED] Sent: Thursday, June 09, 2005 3:18 PM To: java-user@lucene.apache.org Subject: Re: Lucene 1.4.3 QueryParser cannot parse "great!" ? Here is a list of special characters t

Re: Optimizing indexes with mulitiple processors?

I found with a fast RAID controller that I can easily be CPU bound, some of the io is related to latency. You can hide the latency by having overlapping IO (you get that with multiple indexers going on at the same time). I think there possibly could be more horsepower you can get out of the inver

Re: Lucene 1.4.3 QueryParser cannot parse "great!" ?

See QueryParser.escape(), it automatically escapes these special characters for you. On 6/9/05, Zhang, Lisheng <[EMAIL PROTECTED]> wrote: > Hi Richard, > > Thanks very much! That works. > > Lisheng > > -Original Message- > From: Richard Krenek [mailto:[EMAIL PROTECTED] > Sent: Thursday,

RE: Lucene 1.4.3 QueryParser cannot parse "great!" ?

Hi Chris, Thanks very much, that also works well! Lisheng -Original Message- From: Chris Lamprecht [mailto:[EMAIL PROTECTED] Sent: Thursday, June 09, 2005 6:03 PM To: java-user@lucene.apache.org Subject: Re: Lucene 1.4.3 QueryParser cannot parse "great!" ? See QueryParser.escape(), it

RE: Lucene 1.4.3 QueryParser cannot parse "great!" ?

Hi, Thanks for the info, that solved my problem, may I ask one question further: what is the reason why "!" must be escaped? I can see why "+ - * ? \ ~ :" must be escaped since they have some usage in query formulation, is "!" part of the query syntax? Best regards, Lisheng -Original Me

Re: Lucene 1.4.3 QueryParser cannot parse "great!" ?

! can be used before a term for NOT, similar to - (minus). On 6/9/05, Zhang, Lisheng <[EMAIL PROTECTED]> wrote: > Hi, > > Thanks for the info, that solved my problem, may > I ask one question further: what is the reason > why "!" must be escaped? > > I can see why "+ - * ? \ ~ :" must be esca

Re: Optimizing indexes with mulitiple processors?

To follow up. I was surprised to find that from the experiment of indexing 4k documents to local disk (Dell PE with onboard RAID with 256MB cache). I got the following data from my profile: 70 % time was spent in inverting the document 30 % in merge Ok that part isnt surprising. However only ab

Re: Optimizing indexes with mulitiple processors?

Bill Au wrote: Optimize is disk I/O bound. So I am not sure what multiple CPUs will buy you. Now on my system with large indexes... I often have the CPU at 100%... Kevin -- Use Rojo (RSS/Atom aggregator)! - visit http://rojo.com. See irc.freenode.net #rojo if you want to chat. Rojo

Re: Optimizing indexes with mulitiple processors?

Chris Collins wrote: To follow up. I was surprised to find that from the experiment of indexing 4k documents to local disk (Dell PE with onboard RAID with 256MB cache). I got the following data from my profile: 70 % time was spent in inverting the document 30 % in merge Oh.. yeah.. thats i

Re: Optimizing indexes with mulitiple processors?

Well I am currently looking at merging too. In my application merging will occur against a filer (read as higher latency device). I am currently working on how to stage indices on local disk before moving to a filer. Assume I must move to a filer eventually for whatever crazzy reason I need to..