Meknavin, Surapant, Paisarn Charoenpornsawat, and Boonserm Kijsirikul. "Feature-based thai word segmentation." Proceedings of of NLPRS (1997) reported that accuracy of maximal matching (one that I use) is about 80%. BTW this is the best known algorithm that requires only a few rules and word list.
By lttoolbox, we may be able to create a word splitter, which is similar to what Yuen Poovarawan proposed in 1980s. The accuracy is not supposed to be much different from maximal matching. So I will try this approach and report you the result in the next mail. Vee On 4 January 2017 at 17:11, Francis Tyers <[email protected]> wrote: > A 2017-01-04 10:45, Vee Satayamas escrigué: >>> What is the external tool ? How does it work ? What is the licence ? >>> >>> My preference in general would be to reproduce >>> the functionality in Apertium. Word segmentation for languages with >>> writing >>> systems that don't write spaces is not something we do (well?, at >>> all?) >>> yet >>> and it would be great functionality to have. >>> >>> F. >> >> I wrote this one https://github.com/veer66/chamkho. The license is >> LGPL. >> It works based on finding a shortest path from directed acyclic graph >> that is created by word list and some rules e.g. rule for handing >> white spaces. >> > > How well does it work ? Have you evaluated it ? > > If it's a good method we could probably reimplement the algorithm in > lttoolbox. > > Fran > > ------------------------------------------------------------------------------ > Check out the vibrant tech community on one of the world's most > engaging tech sites, SlashDot.org! http://sdm.link/slashdot > _______________________________________________ > Apertium-stuff mailing list > [email protected] > https://lists.sourceforge.net/lists/listinfo/apertium-stuff ------------------------------------------------------------------------------ Check out the vibrant tech community on one of the world's most engaging tech sites, SlashDot.org! http://sdm.link/slashdot _______________________________________________ Apertium-stuff mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/apertium-stuff
