The Highlighter works by comparing the TokenStream of the document with
the Tokens in the query. The TokenStream can be rebuilt from the index
if you use TermVectors with TokenSources or you can get it by
reanalyzing the document. Each Token from the TokenStream is checked
against Tokens in the query, and if there is a match you have a
Highlight. The original text is then reconstructed with the Highlights
from info in the TokenStream about original offsets into the document
for each Token. Also, there is a Fragment system that will break apart
the Highlighted text into score sorted text Fragments.
That is why the original contrib does not work with PhraseQuery's. It
simply matches Tokens from the query with those in the TokenStream.
LUCENE-794 takes the TokenStream and shoves it into a MemoryIndex. Then,
after converting the query to a SpanQuery approximation, getSpans is
called on the index for the query. The Spans provide a bound on what
positions should be Highlighted. Everything else is done exactly like
the original Highlighter (This is a patch that fits into the original
Highlighter framework that was developed, thereby retaining all of its
richness :) ).
Marjan Celikik wrote:
Mark Miller wrote:
Oh yeah...something that you may not have seen is that this has a
dependency on MemoryIndex from contrib. You need that jar as well.
- Mark
Hm, I need the source code. How do I download the files from
https://issues.apache.org/jira/browse/LUCENE-794 (all I see are some
.patch files)?
What I really need is a how the highlighter works in a nutshell... I
am working on a publication and I want to have a reference to Lucene
and its highlighting...
Thanks again.
Marjan.
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
