Can you describe this in a little more detail; I'm not exactly sure what you
mean.
Break your large text documents into multiple Lucene documents. Rather
than dividing them up into entirely discreet chunks of text consider
storing/indexing *overlapping* sections of text with an overlap as big
as the largest "slop" factor you use on Phrase/Span queries so that you
don't cut any potential phrases in half and fail to match e.g.
This non-overlapping indexing scheme will not match a search for "George
Bush"
Doc 1 = ".... outgoing president George "
Doc 2= "Bush stated that ..."
While this overlapping scheme will match...
Doc 1 = ".... outgoing president George "
Doc 2= "president George Bush stated that ..."
This fragmenting approach helps avoid the performance cost of
highlighting very large documents.
The remaining issue is to remove duplicates in your search results when
you match multiple chunks e.g. Lucene Docs #1 and #2 both refer to Input
Doc#1 and will match a search for "president". You will need to store a
field for the "original document number" and remove any duplicates (or
merge them in the display if that is what is required).
Cheers,
Mark
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org
For additional commands, e-mail: java-user-h...@lucene.apache.org
