[Moses-support] KenLM with 16GB of texts

2015-03-25 Thread liling tan
Dear Moses dev/users, Has anyone tried to build a language model from 16 GB of texts? What does Last input should have been poison. mean? Does anyone know how to estimate the output size of the language model file given 16GB of texts with 8 grams? How about 5grams, how big will it get? We've

Re: [Moses-support] KenLM with 16GB of texts

2015-03-25 Thread Marcin Junczys-Dowmunt
Hi, you do not have enough space in /tmp, see No space left on device in /tmp/TuM5Ow. The poison-message is just another echo of that. You can use the -T path to more space option to set a path where you have more space. You probably need something around 100-200 GB (16 GB of compressed or