Try removing this bit of text and just calling the lmplz binary directly. It's not clear to me why that wrapper script still exists.
$moses-script-dir/ems/support/lmplz-wrapper.perl -bin -------- Forwarded Message -------- Subject: Re: [Moses-support] BadDiscountException Date: Tue, 24 Feb 2015 06:16:43 -0800 From: fatma elzahraa Eltaher <fatmaelta...@gmail.com> To: Kenneth Heafield <mo...@kheafield.com> I use kenlm model and when try to add --discount_fallback=1 for setting I get this error Unknown option: discount_fallback. I attached config.toy where must I change to solve this problem ? thank you, Fatma El-Zahraa El -Taher Teaching Assistant at Computer & System department Faculty of Engineering, Azhar University Email : fatmaelta...@gmail.com <mailto:fatmaelta...@gmail.com> mobile: +201141600434 On Tue, Feb 24, 2015 at 5:22 AM, Kenneth Heafield <mo...@kheafield.com <mailto:mo...@kheafield.com>> wrote: The closed-form estimates for Kneser-Ney are not well-defined on toy or class-based data. I recommend using more training data. If this is a class-based model, pass --discount_fallback. Kenneth On 02/24/2015 08:04 AM, fatma elzahraa Eltaher wrote: > Dears, > I get the following error in LM_toy_train.65.STDERR: > Unigram tokens 25188 types 39 > === 2/5 Calculating and sorting adjusted counts === > Chain sizes: 1:468 2:322921696 3:605478272 4:968765120 5:1412782592 > /home/fatma/Desktop/Folder/mosesdecoder/lm/builder/adjust_counts.cc:50 > in void > lm::builder::{anonymous}::StatCollector::CalculateDiscounts(const > lm::builder::DiscountConfig&) threw BadDiscountException because `s.n[j] > == 0'. > Could not calculate Kneser-Ney discounts for 1-grams with adjusted count > 4 because we didn't observe any 1-grams with adjusted count 3; Is this > small or artificial data? > How do I fix it? > > > thank you, > > > > Fatma El-Zahraa El -Taher > > Teaching Assistant at Computer & System department > > Faculty of Engineering, Azhar University > > Email : fatmaelta...@gmail.com <mailto:fatmaelta...@gmail.com> <mailto:fatmaelta...@gmail.com <mailto:fatmaelta...@gmail.com>> > mobile: +201141600434 > > > > _______________________________________________ > Moses-support mailing list > Moses-support@mit.edu <mailto:Moses-support@mit.edu> > http://mailman.mit.edu/mailman/listinfo/moses-support > _______________________________________________ Moses-support mailing list Moses-support@mit.edu <mailto:Moses-support@mit.edu> http://mailman.mit.edu/mailman/listinfo/moses-support
config.toy
Description: Binary data
_______________________________________________ Moses-support mailing list Moses-support@mit.edu http://mailman.mit.edu/mailman/listinfo/moses-support
