Miles Osborne wrote: > you want to also check that ngrams are not getting pruned by > probability (in addition to counts)
Yes, in fact, this: http://www.speech.sri.com/projects/srilm/manpages/ngram-count.1.html seems to suggest that pruning is done based on not changing perplexity very much, rather than raw count or even probability. > this whole business is a bit on the murky side and the only reason > i know about it was when i was writing a disk-based version of > ngram-count a year or so back I'm starting to think it's a lost cause to try to get one LM implementation to act very much like the other. Thanks for the insights, though! - John Burger MITRE _______________________________________________ Moses-support mailing list Moses-support@mit.edu http://mailman.mit.edu/mailman/listinfo/moses-support
