Arraymancer looks fantastic! Also, thanks for the links, I’m sure they might be 
helpful for community members browsing the forum; however, I think the library 
I’m working on is a more general purpose library, akin to OpenFST. It appears 
that BlingFire only supports FSAs (and I am not sure if it supports weighted 
FSAs), and both sentencepiece and BlingFire seem highly focused on 
tokenization. FSA/FST libraries (well, library singular, as the only actively 
maintained one that I am aware of is OpenFST) especially weighted ones are far 
more versatile, and can be directly used for translation, generating word 
inflections, etc... I hope that it will be of use to Nim users that previously 
relied on OpenFST! That library is an enormous C++ codebase that isn’t very 
portable.

Reply via email to