That looks like an IRSTLM-format binary file. You might need the "--text yes" flag for compile-lm.
You can confirm that the file is in the right format by opening it with a text viewer such as "less". The ARPA format is defined here: http://www-speech.sri.com/projects/srilm/manpages/ngram-format.5.html -Jon On Thu, Jul 28, 2011 at 1:15 PM, Lee Ball (Applied Language) < lee.b...@appliedlanguage.com> wrote: > Hello again, > > I ran compile-lm against the model.lm.es.gz and got the output > model.lm.es.blm but when using 8 0 5 in my moses.ini it crashes with this: > > <unk> 0 >> " not blank File: model.es.lm.blm >> Aborted > > > Is this down to bad artefacts in the language model? > > Kind regards, > > Lee Ball > Infrastructure Manager > lee.b...@appliedlanguage.com > Tel: +44 (0)844 854 8945 > > Applied Language Solutions > High quality language solutions delivered on time...with a smile! > > www.appliedlanguage.com > Tel (UK): +44 (0)845 367 7000 > Tel (US): +1 (800) 579-5010 > > Riverside Court, Huddersfield Road, Delph, Oldham, OL3 5FZ. UK > Registered in the UK 5122429 > > Pride in everything we do | Respect everyone like a friend > [image: An Environmentally Friendly Company]Think of the environment; > please don't print this e-mail unless you really need to. > > [image: Fast Track 100 2009][image: Queens Award for Business] > <http://twitter.com/appliedlanguage> > > > > On 28 July 2011 17:18, Kenneth Heafield <mo...@kheafield.com> wrote: > >> FYI we resolved the problem off-list. KenLM does not load IRST's iARPA >> format. You must first run IRST"s compile-lm to generate an ARPA. I >> might add an error message specific to this case. >> >> On 07/27/11 09:27, Lee Ball (Applied Language) wrote: >> > Hi guys, >> > >> > I just tried using KenLM out of interest for the first time with an old >> > set of models that were using IRSTLM. >> > >> > Based on this: http://kheafield.com/code/kenlm/moses/ I changed the >> > settings from: >> > >> > [lmodel-file] >> > 1 0 5 /home/moses/languagemodels/model.es.lm.gz >> > >> > to >> > >> > [lmodel-file] >> > 8 0 5 /home/moses/languagemodels/model.es.lm.gz >> > >> > I also tried >> > >> > [lmodel-file] >> > 9 0 5 /home/moses/languagemodels/model.es.lm.gz >> > >> > I then ran mosesserver with this new moses.ini file and it appeared to >> > output the contents of model.lm.gz and then it stopped with this output: >> > >> > \end\ >> > " not blank File: /home/moses/languagemodels/model.es.lm.gz >> > Aborted >> > >> > >> > >> > IRSTLM was used to create this language model initially. Although the >> > moses.ini was from a slightly older build which doesn't mention KenLM as >> > an option in the language model comments, I assumed it would still work? >> > >> > The output error is the same for both 9 and 8 settings. It works fine >> > with using IRSTLM. >> > >> > Kind regards, >> > >> > Lee Ball >> > Infrastructure Manager >> > lee.b...@appliedlanguage.com <mailto:lee.b...@appliedlanguage.com> >> > Skype ID: lee.ball_appliedlanguage >> > >> > Applied Language Solutions >> > High quality language solutions delivered on time...with a smile! >> > >> > www.appliedlanguage.com <http://www.appliedlanguage.com/> >> > Tel (UK): +44 (0)845 367 7000 >> > Tel (US): +1 (800) 579-5010 >> > >> > Riverside Court, Huddersfield Road, Delph, Oldham, OL3 5FZ. UK >> > Registered in the UK 5122429 >> > >> > Pride in everything we do | Respect everyone like a friend >> > An Environmentally Friendly CompanyThink of the environment; please >> > don't print this e-mail unless you really need to. >> > >> > Fast Track 100 2009Queens Award for >> > Business <http://twitter.com/appliedlanguage> >> > >> > >> > >> > >> > _______________________________________________ >> > Moses-support mailing list >> > Moses-support@mit.edu >> > http://mailman.mit.edu/mailman/listinfo/moses-support >> >> _______________________________________________ >> Moses-support mailing list >> Moses-support@mit.edu >> http://mailman.mit.edu/mailman/listinfo/moses-support >> > > > _______________________________________________ > Moses-support mailing list > Moses-support@mit.edu > http://mailman.mit.edu/mailman/listinfo/moses-support > >
_______________________________________________ Moses-support mailing list Moses-support@mit.edu http://mailman.mit.edu/mailman/listinfo/moses-support
