Re: Using Lucene for searching tokens, not storing them.

2006-04-16 Thread karl wettin
16 apr 2006 kl. 19.18 skrev karl wettin: For any interested party, I do this because I have a fairly small corpus with very heavy load. I think there is a lot to win by not creating new instances of what not, seeking in the file-centric Directory, parsing pseudo-UTF8, et.c. at query time.

Re: Using Lucene for searching tokens, not storing them.

2006-04-16 Thread Paul Elschot
On Sunday 16 April 2006 19:18, karl wettin wrote: > > 15 apr 2006 kl. 21.32 skrev Paul Elschot: > >> > >> implements TermPositions { > >> public int nextPosition() throws IOException { > > > > This enumerates all positions of the Term in the document > > as returned by the Tokenizer used

Re: Using Lucene for searching tokens, not storing them.

2006-04-16 Thread karl wettin
15 apr 2006 kl. 21.32 skrev Paul Elschot: implements TermPositions { public int nextPosition() throws IOException { This enumerates all positions of the Term in the document as returned by the Tokenizer used by the Analyzer Aha. And I didn't see the TermPositionVector until now. T

Re: Using Lucene for searching tokens, not storing them.

2006-04-15 Thread Paul Elschot
On Saturday 15 April 2006 19:25, karl wettin wrote: > > 14 apr 2006 kl. 18.31 skrev Doug Cutting: > > > karl wettin wrote: > >> I would like to store all in my application rather than using the > >> Lucene persistency mechanism for tokens. I only want the search > >> mechanism. I do not nee

Re: Using Lucene for searching tokens, not storing them.

2006-04-15 Thread karl wettin
14 apr 2006 kl. 18.31 skrev Doug Cutting: karl wettin wrote: I would like to store all in my application rather than using the Lucene persistency mechanism for tokens. I only want the search mechanism. I do not need the IndexReader and IndexWriter as that will be a natural part of my a

Re: Using Lucene for searching tokens, not storing them.

2006-04-14 Thread Doug Cutting
karl wettin wrote: Do I have to worry about passing a null Directory to the default constructor? A null Directory should not cause you problems. Doug - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mai

Re: Using Lucene for searching tokens, not storing them.

2006-04-14 Thread Yonik Seeley
On 4/14/06, karl wettin <[EMAIL PROTECTED]> wrote: > Do I have to worry about passing a null Directory to the default > constructor? That's not an easy road you are trying to take, but it should be doable. There are some final methods you can't override, but just set directoryOwner=false and close

Re: Using Lucene for searching tokens, not storing them.

2006-04-14 Thread karl wettin
14 apr 2006 kl. 18.31 skrev Doug Cutting: karl wettin wrote: I would like to store all in my application rather than using the Lucene persistency mechanism for tokens. I only want the search mechanism. I do not need the IndexReader and IndexWriter as that will be a natural part of my a

Re: Using Lucene for searching tokens, not storing them.

2006-04-14 Thread Doug Cutting
karl wettin wrote: I would like to store all in my application rather than using the Lucene persistency mechanism for tokens. I only want the search mechanism. I do not need the IndexReader and IndexWriter as that will be a natural part of my application. I only want to use the Searchable.

Re: Using Lucene for searching tokens, not storing them.

2006-04-14 Thread karl wettin
14 apr 2006 kl. 18.01 skrev Christophe: On 14 Apr 2006, at 08:55, karl wettin wrote: I don't want to use Lucene for persistence. I do not want to store tokens nor field text in a FSDirectory or in a RAMDirectory. I want to take store the the tokens in my application. If I understand your

Re: Using Lucene for searching tokens, not storing them.

2006-04-14 Thread Chris Lu
Thanks, Christophe. Hi, Kevin, I think your question means you want to store the Analyzed tokens yourself? If so, you can use Analyzer to directly process the text, and save the analyzed results in your application, maybe later use it in some RDBMS? or BerkelyDB? Chris Lu ---

Re: Using Lucene for searching tokens, not storing them.

2006-04-14 Thread karl wettin
14 apr 2006 kl. 17.56 skrev karl wettin: 14 apr 2006 kl. 17.51 skrev Christophe: Are you contemplating having your own index and index format? In that case, it's not clear to me how much leverage you will be getting using Lucene at all. Could you explain in more detail what you are tr

Re: Using Lucene for searching tokens, not storing them.

2006-04-14 Thread Christophe
On 14 Apr 2006, at 08:55, karl wettin wrote: I don't want to use Lucene for persistence. I do not want to store tokens nor field text in a FSDirectory or in a RAMDirectory. I want to take store the the tokens in my application. If I understand your question, I think that the first answer wa

Re: Using Lucene for searching tokens, not storing them.

2006-04-14 Thread karl wettin
14 apr 2006 kl. 17.51 skrev Christophe: Are you contemplating having your own index and index format? In that case, it's not clear to me how much leverage you will be getting using Lucene at all. Could you explain in more detail what you are trying to do? I want to use the parts of Luc

Re: Using Lucene for searching tokens, not storing them.

2006-04-14 Thread karl wettin
14 apr 2006 kl. 17.51 skrev karl wettin: 14 apr 2006 kl. 17.46 skrev Chris Lu: On 4/14/06, karl wettin <[EMAIL PROTECTED]> wrote: I would like to store all in my application rather than using the Lucene persistency mechanism for tokens. I only want the search mechanism. I do not need the In

Re: Using Lucene for searching tokens, not storing them.

2006-04-14 Thread Christophe
On 14 Apr 2006, at 08:51, karl wettin wrote: You missunderstand all my questions. I must admit I was not sure I understood your question, either. In order to search, Lucene needs an index. That index is maintained by the IndexReader and IndexWriter classes. Are you contemplating having

Re: Using Lucene for searching tokens, not storing them.

2006-04-14 Thread karl wettin
14 apr 2006 kl. 17.46 skrev Chris Lu: On 4/14/06, karl wettin <[EMAIL PROTECTED]> wrote: I would like to store all in my application rather than using the Lucene persistency mechanism for tokens. I only want the search mechanism. I do not need the IndexReader and IndexWriter as that will be a

Re: Using Lucene for searching tokens, not storing them.

2006-04-14 Thread Chris Lu
use Store.NO when creating Field Chris Lu --- Full-Text Lucene Search on Any Databases http://www.dbsight.net Faster to Setup than reading marketing materials! On 4/14/06, karl wettin <[EMAIL PROTECTED]> wrote: > I would like to store all in my application rat

Using Lucene for searching tokens, not storing them.

2006-04-14 Thread karl wettin
I would like to store all in my application rather than using the Lucene persistency mechanism for tokens. I only want the search mechanism. I do not need the IndexReader and IndexWriter as that will be a natural part of my application. I only want to use the Searchable. So I looked at exte