On Dec 9, 2005, at 3:18 PM, Alan Chandler wrote:
I am slowly making may way through lucene, as witnessed by earlier
threads to
this mailing list.
But I am stuck again, going round in circles with the Javadocs.
I want to display the results of a user entered search where for
each document
I put out a small summary with the searched for words highlighted.
When I wrote the Analyzer for my documents, I produced the
tokenstream to
generate Token objects with the start end end positions of each
term in them
Now, from my Hits object I can find each document I need to output,
but how do
I get back to the Tokens I originally produced.
Are you using Lucene 1.4.3? Or the latest Subversion version?
The Lucene index does not keep all of the information in the Token's
emitted by the analyzer (unless specified to do so, but 1.4.3 didn't
support the fancier features).
So, the fail-safe way is to re-tokenize the original text (perhaps
stored in the Lucene index) and hand that TokenStream to the
Highlighter.
Or you can experiment with the additional Field constructors to
enable the storage of token offsets and the Highlighter can use those
for a little better performance, but it's likely to be unnoticeable
for your application to simply re-tokenize on the fly for only the
fields you're displaying. Storing the token offsets increases the
index size, of course.
Erik
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
