Hi team, I have found there is missing specification of the file mode opening in the make-factor-brown-cluster-mkcls.perl. See the read_cluster_from_mkcls function in scripts/training/wrappers/make-factor-brown-cluster-mkcls.perl file.
The fix is to add the following code bellow line 36: binmode(CLUSTER_FILE, "utf8"); This resolves encoding problems when user wants to process text containing extended characters. Please let me know how can I implement the fix to github repository. Thank you, Tomas
_______________________________________________ Moses-support mailing list Moses-support@mit.edu http://mailman.mit.edu/mailman/listinfo/moses-support
