OK, gotcha. I now see what you mean. StandardAnalyzer uses the
StandardTokenizer, whereas StopAnalyzer uses the LowerCaseTokenizer,
which divides text at non-letters. What you most likely will need to
do is create a Tokenizer that outputs the original token, and outputs
the parts of it
Grant,
Thanks for your reply and the pointer to the custom code sample. I will be
checking into that today. I did delve into the src for the OOTB analyzers
and was aware of what they did. Still, the StandardAnalyzer does not do what
I want. The real difference between my needs and the results of t
Hi Tim,
From the StandardAnalyzer code, the TokenStream looks like:
/** Constructs a [EMAIL PROTECTED] StandardTokenizer} filtered by a [EMAIL
PROTECTED]
StandardFilter}, a [EMAIL PROTECTED] LowerCaseFilter} and a [EMAIL PROTECTED]
StopFilter}. */
public TokenStream tokenStream(String fi
