Yes, first I tried the sample code but with my model file:
InputStream modelIn = new FileInputStream("en-ner-person.bin");
try {
TokenNameFinderModel model = new TokenNameFinderModel(modelIn);
NameFinderME nameFinder = new NameFinderME(model);
Span nameSpans[] = nameFinder.find(sentence);
logger.debug(nameSpans.size());
// do something with the names
nameFinder.clearAdaptiveData()
}
catch (IOException e) {
e.printStackTrace();
}
finally {
if (modelIn != null) {
try {
modelIn.close();
}
catch (IOException e) {
}
}
}
When I ran it, it would load the model in about 60 seconds but never
identify any entities. I gave the same input to the evaluator tool and it
identified the identities. So then I changed to this code where sentence is
the input:
TokenNameFinderModel personModel = new
TokenNameFinderModelLoader().load(new File(personModelPath));
ObjectStream<String> untokenizedLineStream = new PlainTextByLineStream(new
StringReader(sentence));
NameFinderME nameFinder = new NameFinderME(model);
String line = untokenizedLineStream.read();
String whitespaceTokenizerLine[] =
WhitespaceTokenizer.INSTANCE.tokenize(line);
Span[] spans = nameFinder.find(whitespaceTokenizerLine);
And this worked. I was just interested in why the second code worked and
the first did not. I feel sure I had the path to the model correct in the
first code because it showed a debug line about loading the model and
delayed about 60 seconds. I did not see any errors - just an empty array of
name spans.
Thanks,
Jeff
On Mon, Oct 21, 2013 at 11:48 AM, Jörn Kottmann <[email protected]> wrote:
> On 10/21/2013 05:01 PM, Jeffrey Zemerick wrote:
>
>> I successfully trained a model and ran the evaluator tool. I then tried to
>> load my model using the sample code in the documentation [1] that works
>> with the pre-built models available but no entities were ever identified.
>> I
>> looked through the code of the Evaluator tool and found the model was
>> being
>> loaded differently (via the TokenNameFinderModelLoader class) so I
>> switched
>> to using that code and now my model works. I just wanted to see if there
>> might be any reason why the sample code in the documentation did not work.
>>
>
> The sample code should work. Can you share the code you used?
>
> Thanks,
> Jörn
>
