Hi, Here's a cheap server for fragment scoring.
socat TCP4-LISTEN:2000,fork EXEC:"bin/fragment lm/test.arpa" Then in another terminal socat TCP4-CONNECT:localhost:2000 STDIO <text Now if you want to translate fragments instead then that's what Moses is for, though keep in mind that it will always prepend <s> and append </s> for translation. Kenneth On 05/26/14 02:04, Albert Llorens wrote: > Thanks, Kenneth. > > Yes, I want to score sentence fragments. I want to use Moses for fragment > translation, but only for frequent or probable fragments. I'll try what you > suggest. Any chance the query could be done remotely, using mosesserver or > anything else? > > Kind regards. > > Albert > > > -----Original Message----- > From: moses-support-boun...@mit.edu [mailto:moses-support-boun...@mit.edu] On > Behalf Of Kenneth Heafield > Sent: viernes, 23 de mayo de 2014 17:34 > To: moses-support@mit.edu > Subject: Re: [Moses-support] Get the probability of a given n-gram in a > language model > > Hi, > > You can use bin/query on an ARPA or KenLM file. Then just type > sentences at it (or use a file as stdin). By default it will assume you are > scoring sentences. You can pass -n to not wrap in <s> and </s>. > > It appears that you are asking to score sentence fragments. The > leading words will be scored using unigrams, bigrams, etc. from, say, a > 5-gram model. If you are using Kneser-Ney, these lower-order probabilities > (unigrams through 4-grams) are conditioned on having backed off to them. If > you want accurate scores for sentence fragments, build a model of order 1, > order 2, order 3, etc. then combine them using > > build_binary -r "1.arpa 2.arpa 3.arpa 4.arpa" 5.arpa 5.rest > > You can then use > > bin/fragment 5.rest <fragments > > to attain log10 frequencies. For more on this rant, read > > http://kheafield.com/professional/edinburgh/rest_paper.pdf > > Kenneth > > On 05/23/14 05:13, Albert Llorens wrote: >> Hi, >> >> >> >> Is there a straightforward way I can ask Moses for the probability (or >> the frequency) of a given n-gram in a given language model? If so, can >> I do the query through mosesserver? >> >> >> >> Thanks. >> >> >> >> Kind regards. >> >> >> >> Albert >> >> >> >> >> >> _______________________________________________ >> Moses-support mailing list >> Moses-support@mit.edu >> http://mailman.mit.edu/mailman/listinfo/moses-support >> > _______________________________________________ > Moses-support mailing list > Moses-support@mit.edu > http://mailman.mit.edu/mailman/listinfo/moses-support > _______________________________________________ Moses-support mailing list Moses-support@mit.edu http://mailman.mit.edu/mailman/listinfo/moses-support