Hi, I think oov-feature=1 just activates the OOV count feature while leaving LM score unchanged. So it would still include p(<unk> | in).
One might try setting the OOV feature weight to -weight_LM * weird_moses_internal_constant * log p(<unk>) in an attempt to cancel out the log p(<unk>) terms. However that won't work either because: 1) It will still charge backoff penalties, b(the)b(house) in the example. 2) The context will be lost each time so it's p(house) not p(house | the). If the <unk>s follow a pattern, such as appearing every other word, one could insert them into the ARPA file though that would waste memory. I don't think there's any way to accomplish exactly what OP asked for without coding (though it wouldn't be that hard once one understands how the LM infrastructure works). Kenneth On 01/14/2016 11:07 PM, Philipp Koehn wrote: > Hi, > > You may get the behavior you want by adding > "oov-feature=1" > to your LM specification line in moses.ini > and also add a second weight with value "0" to the corresponding LM > weight setting. > > This will then only use the scores > p(the|<s>) > p(house|<s>,the,<unk>) ---> backoff to p(house) > p(in|<s>,the,<unk>,house,<unk>) ---> backoff to p(in) > > -phi > > On Thu, Jan 14, 2016 at 8:25 AM, LUONG NGOC Quang > <quangngoclu...@gmail.com <mailto:quangngoclu...@gmail.com>> wrote: > > Dear All, > > I am currently using a SRILM Language Model (LM) in my Moses > decoder. Does anyone know how can I ask the decoder, at the decoding > time, skip all out-of-vocabulary words when computing the LM score > (instead of doing back-off)? > > For instance, with the n-gram: "the <unk> house <unk> in", I would > like the decoder to assign it the probability of the phrase: "the > house in" (existing in the LM). > > Do I need more options/declarations in moses.ini file? > > Any help is very much appreciated, > > Best, > Quang > > > > _______________________________________________ > Moses-support mailing list > Moses-support@mit.edu <mailto:Moses-support@mit.edu> > http://mailman.mit.edu/mailman/listinfo/moses-support > > > > > _______________________________________________ > Moses-support mailing list > Moses-support@mit.edu > http://mailman.mit.edu/mailman/listinfo/moses-support > _______________________________________________ Moses-support mailing list Moses-support@mit.edu http://mailman.mit.edu/mailman/listinfo/moses-support