This is a nice trick, and one I've created as an example in LIA, in
fact. There is one constraint with using CharTokenizer to do this -
there is a max word length of 255. This probably is not much of an
issue in practice, but I wanted to point out that it is not a true
"keyword analyzer" and does not prevent tokenization if the length is
greater than 255.
Erik
On Jun 11, 2004, at 4:24 AM, Eric Jain wrote:
There doesn't seem to be an Analyzer that doesn't do anything included
with Lucene, is there? This would seem useful to prevent tokenization
of certain fields in queries, together with the
PerFieldAnalyzerWrapper. But perhaps there is a better way to
accomplish this?
private static class NullAnalyzer
extends Analyzer
{
public TokenStream tokenStream(String fieldName, Reader reader)
{
return new CharTokenizer(reader)
{
protected boolean isTokenChar(char c)
{
return true;
}
};
}
}
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
