Ken - should we add encoding on open to all python scripts, rather than set the PYTHONIOENCODING env variable? That's basically what happens with the perl scripts/
What python/Linux version are you using? I don't see it on my version (Python 2.7.3, Ubuntu 12.04) Qin - Thanks. I've added you as admin for moses on github. We may change this if it doesn't suit you. mgiza is a sister project of moses https://github.com/moses-smt/mgiza So everyone who has commit access to moses also has access to mgiza, which is quite a lot! We monitor all commits to mgiza on the same mailing list as moses in case people mess around, eg. http://lists.inf.ed.ac.uk/pipermail/moses-commits/2014-November/001826.html On 14 November 2014 09:42, Gao Qin <pku.gao...@gmail.com> wrote: > Good idea, I am not yet admin of the new repro, Hieu will add me and I cam > make change then. > > --Q > > On Thu, Nov 13, 2014 at 8:54 AM, Kenneth Heafield <m...@kheafield.com> > wrote: > >> Hi, >> >> MGIZA has some Python programs that process raw text: >> https://github.com/moses-smt/mgiza/tree/master/mgizapp/scripts . >> >> Since those scripts were released, Python messed up file encoding >> and >> made the default ascii. Should we just change every open call to have >> encoding = 'utf-8' ? >> >> Kenneth >> > > -- Hieu Hoang Research Associate University of Edinburgh http://www.hoang.co.uk/hieu
_______________________________________________ Moses-support mailing list Moses-support@mit.edu http://mailman.mit.edu/mailman/listinfo/moses-support
