Re: Which tokenizer to use with the NameFinder models?

[Jörn Kottmann]

On 4/10/11 10:52 AM, Olivier Grisel wrote:
Ok then let me be more specific:
Has en-ner-{person,place,organization}.bin been trained with the
output of SimpleTokenizer class or with TokenizerME + en-token.bin?
I didn't tokenize the training data myself, but it has been tokenized
with some kind of character class tokenizer like the Simple Tokenizer.

You should definitely use the Simple Tokenizer for the English Name 
Finder models.

Jörn