On 04/20/2012 08:33 AM, Joan Codina wrote:
So, the processing is corrent but the <SPLIT>'s are missing at for example "Haag." or "Chicago's" And i wonder if there is a missing parameter or I need another dictionary.
Just checked the code, looks like it cannot output the <SPLIT> markers. We should fix that.There is also a nice method inside the cmd line tool (DictionaryDetokenizerTool) which can produce a detokenized string. We should move that one to the DictionaryDetokenizer.
Jörn
