RE: best way to share cookie info (user search history, etc.) between two load balanaced lucene search servers..

2007-11-12 Thread Chhabra, Kapil
Kumar, This seems to be more of a non-lucene problem. Anyhow, have a look at "ha-proxy"; this is closer to what you require. We can discuss more on this off the lucene list, if required. Regards, kapilChhabra -Original Message- From: kumarlimbu [mailto:[EMAIL PROTECTED] Sent: Tuesday

RE: best way to share cookie info (user search history, etc.) between two load balanaced lucene search servers..

2007-11-12 Thread kumarlimbu
Hi Kapil, Thanks for your response. We are looking at replicating complete lucene index on both servers. The main reason for this is the ease of scaling compared to say splitting the index between the 2 servers. Since we will have separate indexes we won't be using MultiSearcher/ParallelMultiSea

Re: Optimizing index takes too long

2007-11-12 Thread Lucene User
what type of documents are indexing regards gaurav On 11/11/07, Barry Forrest <[EMAIL PROTECTED]> wrote: > > Hi, > > Optimizing my index of 1.5 million documents takes days and days. > > I have a collection of 10 million documents that I am trying to index > with Lucene. I've divided the colle

Re: Optimizing index takes too long

2007-11-12 Thread Barry Forrest
On Nov 12, 2007 1:15 PM, J.J. Larrea <[EMAIL PROTECTED]> wrote: > > 2. Since the full document and its longer bibliographic subfields are > being indexed but not stored, my guess is that the large size of the index > segments is due to the inverted index rather than the stored data fields. > But

substring indexing to avoid 'TooManyClauses' exception

2007-11-12 Thread Hardy Ferentschik
Hi, I have a question regarding the way I got around the 'TooManyClauses' exception when using wild card queries (http://wiki.apache.org/lucene-java/LuceneFAQ#head-06fafb5d19e786a50fb3dfb8821a6af9f37aa831). I am using Lucene in conjunction with Hibernate Search (http://www.hibernate.org/

indexing a large number of fields (and nesting them)

2007-11-12 Thread Seneviratne_Yasoja
Hi, I need to index tons of meta-data fields along with every document (around 80 fields, mostly strings of 32 characters, some integers, bools and dates too, a couple of strings are longer like 64 chars or 120). Also it would be nice if there was a way to represent nested fields, and query for t

Re: Create and populate a field when indexing

2007-11-12 Thread Erick Erickson
Sure, that'd work. It would also work to just continue to use f.getPath() whenever you need it. FWIW Erick On Nov 12, 2007 1:43 PM, KR <[EMAIL PROTECTED]> wrote: > > > Erick Erickson wrote: > > > > Sure, just define it in the same scope as you want to refer to it. > > Of course, that tells you

Re: Create and populate a field when indexing

2007-11-12 Thread KR
Erick Erickson wrote: > > Sure, just define it in the same scope as you want to refer to it. > Of course, that tells you nothing ... > > Java variables go out of scope when the last '}' *at the same level* > is passed. For intance: > > { >string s1; >{ > string s2; >} // s2 i

Re: Comparing Two Indexes

2007-11-12 Thread Erick Erickson
To paraphrase "Why do you want to know"? These kinds of questions are so lacking in context that meaningful help is hard to offer. What problem are you trying to solve? Why do you want to compare indexes? Erick On Nov 9, 2007 12:14 PM, Lucene User <[EMAIL PROTECTED]> wrote: > Hi, > > I wanted tw

Re: Create and populate a field when indexing

2007-11-12 Thread Erick Erickson
Sure, just define it in the same scope as you want to refer to it. Of course, that tells you nothing ... Java variables go out of scope when the last '}' *at the same level* is passed. For intance: { string s1; { string s2; } // s2 is out of scope after this line. // s1 is still

Re: fieldcache gives OOM. Deos a LRU-style fieldcache exist?

2007-11-12 Thread Mark Miller
https://issues.apache.org/jira/browse/LUCENE-831 This patch should be a good start for you. Not sure how stale it is at this point though. Britske wrote: First my question: Is there an (experimental / patch-version) lucene-fieldcache available which uses some kind of eviction-strategy (LRU

Re: get original term for synonym

2007-11-12 Thread mark harwood
>>Is there a simple way to make the synonym return the word it points to? Give the highlighter the same analyzer you used to create the index, not the one you use to parse the query. This should ensure the set of words to be highlighted includes all synonyms. Cheers Mark - Original Messag

fieldcache gives OOM. Deos a LRU-style fieldcache exist?

2007-11-12 Thread Britske
First my question: Is there an (experimental / patch-version) lucene-fieldcache available which uses some kind of eviction-strategy (LRU or whatever) so that OOM's would never happen in my case, but instead some sorts would simply get evicted? Now the background for those who are interested:

Re: Optimizing index takes too long

2007-11-12 Thread Eric Louvard
You could have a look at this thread. http://www.gossamer-threads.com/lists/lucene/java-user/29354 regards. Barry Forrest schrieb: > Hi, > > Optimizing my index of 1.5 million documents takes days and days. > > I have a collection of 10 million documents that I am trying to index > with Lucene.

get original term for synonym

2007-11-12 Thread Matthijs Bierman
Hi there, Currently I am trying to get synonyms to work. I have gotten as far as injecting them into the index as Token.type SYNONYM. Lucene then finds the original word and synonym and points to the same document. So far so good. However, I am stuck at highlighting the result. I have highli

Re: Optimizing index takes too long

2007-11-12 Thread Michael McCandless
> I am using the 2.3-dev version only because LUCENE-843 suggested > that this might be a path to faster indexing. I started out using > 2.2 and can easily go back. I am using default MergePolicy and > MergeScheduler. Did you note any indexing or optimize speed differences between 2.2 & 2.3-dev?

RE: best way to share cookie info (user search history, etc.) between two load balanaced lucene search servers..

2007-11-12 Thread Chhabra, Kapil
Ah! There are so many ways to do this as there are so many questions unanswered in your mail. What kind of load balancer are you going to install? Will you be replicating the complete lucene index on both the servers? Do you plan to use the MultiSearcher/ParellelMultiSearcher? Do these servers sha

best way to share cookie info (user search history, etc.) between two load balanaced lucene search servers..

2007-11-12 Thread kumarlimbu
Hi Everyone, We are planning on scaling our current web server by adding a machine with similar specification. Both machine will be running lucene searches. What we plan to do is add a load balancer in front of these servers. Our requirement is to be able to share user info (user search history,

RE: TermDocs.skipTo error

2007-11-12 Thread Mike Streeton
Yonik, Thanks for this , I have checked and it is not at the end of the index, but something funny is happening. It is a multireader each reader within it has 100,000 documents, but when it fails it is trying to access a document e.g. 123,456, in the segment that only has 100,000 Many Thank