Hello! Does anyone know, what tokenizer is used when applying NameFinderME for training a custom named entity recognition model? I was searching but I could not find this information.
I have to attach the same tokenizer when using the trained model, but I
don't know which one was used.
Therefore at the moment I just tokenize via:
String[] tokens = sentence.getCoveredText().split("\\s+");
Thank you for feedback!
lg Markus
