Hi,
I believe MITLM can produce an ARPA file that you can then load with
the existing toolkits. MITLM was never designed to do real-time queries.
With regard to the Microsoft web language model, I tried this a while
ago with a different decoder. The problem is that you should expect to
make 2 million queries/sentence. Many of these are duplicated, so I
wrote a local cache with unbounded memory. It was still way too slow.
Asked the Bing people how many queries per second and they said
unlimited. Then I brought their servers down and they didn't recover
until somebody on their side manually fixed it. This happened several
times and they were nice about it. Also, at the time, it was limited to
serving trigrams (the 5-grams were too big, apparently), so I wasn't
getting much in terms of quality on n-best reranking.
Kenneth
On 09/18/12 04:36, krishna nanda wrote:
> Hello Hieu,
>
> Thanks for your reply. Yes, I managed to run moses in cygwin without any
> problems. No changes were required.
>
> I have another question:
> I saw that moses has support for Ken/IRST/SRI/Rand Language models.
> But is there an easy way to consume other language models like the
> Microsoft web language model or MITLM in moses?
>
> Thank you
> Krishna
>
>
> On Fri, Aug 24, 2012 at 10:01 PM, Hieu Hoang <[email protected]
> <mailto:[email protected]>> wrote:
>
> the last 2 number (4 31) are not scores and are ignored. They are
> counts that was used to calculate the probabilities.
>
> there's no option to calculate joint probabilities. i suppose you
> need to calculate p(s) or p(t) which can be done, but may require a
> lot of memory. try it yourself and add it to moses if it works.
>
> so you managed to run moses in cygwin all the way to getting a bleu
> score? was there anything you need to change?
>
> On 23/08/2012 16:57, krishna nanda wrote:
>> Hello Hoang and Barry,
>>
>> Thanks a lot for your reply. I was able to install moses and run
>> it in cygwin. I have some quick questions:
>>
>> I found the format (contents) of the phrase table here:
>> http://www.statmt.org/moses/?n=FactoredTraining.ScorePhrases
>> according to which, there are 5 scores for a phrase pair.
>>
>> But, the phrase table I generated from the news corpus has 7
>> scores for a phrase pair like this:
>> ,au cours de ||| ,as of ||| .25 2.36556e-06 0.0333581 0.00128335
>> 2.718 ||| 4 31
>> I was not clear on the above format.
>>
>> Secondly, is there an option to also generate joint probabilities
>> of phrase pairs?
>>
>> Thank you
>> Krishna
>>
>>
>> On Sun, Aug 19, 2012 at 11:12 PM, Hieu Hoang
>> <[email protected] <mailto:[email protected]>> wrote:
>>
>> running Moses on cygwin should be the same as running it on
>> linux or mac. If you have any problems, please get back to us.
>>
>> the document you pointed to is old now, i've changed the
>> website to reflect that.
>>
>> On 17/08/2012 08:45, krishna nanda wrote:
>>> Hello,
>>>
>>> I am looking to install Moses in Cygwin. However, I found the
>>> document on the website under "windows installation" to be
>>> not up to date:
>>> http://www.statmt.org/moses/?n=Development.GetStarted
>>>
>>> It still uses "regenerate-makefiles.sh",which is not there in
>>> the latest source from github. Is there an updated version?
>>>
>>> Thank you
>>> Krishna
>>>
>>>
>>>
>>>
>>>
>>> _______________________________________________
>>> Moses-support mailing list
>>> [email protected] <mailto:[email protected]>
>>> http://mailman.mit.edu/mailman/listinfo/moses-support
>>
>>
>> _______________________________________________
>> Moses-support mailing list
>> [email protected] <mailto:[email protected]>
>> http://mailman.mit.edu/mailman/listinfo/moses-support
>>
>>
>
>
>
>
> _______________________________________________
> Moses-support mailing list
> [email protected]
> http://mailman.mit.edu/mailman/listinfo/moses-support
_______________________________________________
Moses-support mailing list
[email protected]
http://mailman.mit.edu/mailman/listinfo/moses-support
