Thanks so much for your explaination. But there is one thing that I want to make sure is that in case that i add the same token to the same field, internally is it redundancy?
And in case, that I have many fields. What is the best way to list up the frequency of all the tokens from different fields without knowing in advance what are the tokens that we have. Once again, thank you very much for your reply. Best regards, Sengly On 4/4/07, Erick Erickson <[EMAIL PROTECTED]> wrote:
See below On 4/4/07, Sengly Heng <[EMAIL PROTECTED]> wrote: > > Dear all, > > My problem is a little bit strange. Instead of parsing the content of the > document to the indexer. I am adding one by one. Here is a piece of my > code > : > > Document doc = new Document(); > doc.add(Field.Text("Features", "blue"); > doc.add(Field.Text("Features","beautiful"); > doc.add(Field.Text("Features", "black"); > doc.add(Field.Text("Features","white"); > doc.add(Field.Text("Features", "blue"); > doc.add(Field.Text("Features","blue"); > > I'd like to know whether the internal representation of this is like when > we > add at once the whole content of the document as a long string? There's no internal difference unless you implement an Analyzer that returns a value other than 1 from getPositionIncrementGap(). Why would you do this you ask? Occasionally, it is useful to index data in the same field but NOT allow proximity queries to span separate additions. Imagine indexing a book and, for some reason, you didn't want span queries to cross, say, chapters. You could do something like doc.add("page", <contents of page 1>); doc.add("page", <contents of page 2>); then in your Analyzer, have something in getPositionIncrementGap like if (page is beginning of chapter) { return 1000; } else { return 1; } Now, no Span query with a slop of less than 1,000 would match from the last page in one chapter to the first page of another. FWIW Erick I'd like > also to count the number of "blue" occurence from the document. How to do > this? See the other reply <G>. Thank you very much for your suggestion in advance. > > Regards, > > Sengly >