Re: [Moses-support] Train recasing model using IRSTLM

2011-11-24 Thread Jehan Pages
Hi all, rather than having to search through email archive, as I guess we are not the only one who won't use SRILM because it is proprietary (or some other reason), I thought the best would be to modify the existing script to be able to switch to IRSTLM when desired. I have just made a pull reques

Re: [Moses-support] Incremental training... how?

2011-11-24 Thread Jehan Pages
Hi, 2011/11/25 Philipp Koehn : > Hi, > > check the FAQ: > http://www.statmt.org/moses/?n=Moses.AdvancedFeatures#ntoc27 I think he is actually speaking of this page you mention. It explains all the data preparation, then at the very end of the section, part "update model", it just says: « Now tha

[Moses-support] Tuning of hierarchical models: main::create_extractor_script() called too early

2011-11-24 Thread Daniel Schaut
Hi all, I've got a quick question regarding the tuning of hierarchical phrase-based models. When calling mert, my terminal outputs an error I can't find in the mailing lists: main::create_extractor_script() called too early to check prototype at ./mert-moses.pl line 666 The script didn't stop a

Re: [Moses-support] Mert and METEOR

2011-11-24 Thread Christian Federmann
Ah yes... the usual suspect ;)   Might be worth implementing though...  let's see.   Cheers,    Christian   Christophe Servan hat am 24. November 2011 um 19:19 geschrieben: > Hi, > Maybe the lack of time ;-) > > Christophe > > > Le 24/11/2011 19:10, Christian Federmann a écrit : > > Hi all, >

Re: [Moses-support] Mert and METEOR

2011-11-24 Thread Christophe Servan
Hi, Maybe the lack of time ;-) Christophe Le 24/11/2011 19:10, Christian Federmann a écrit : > Hi all, > > is there a specific reason/obstacle that METEOR is not available for MERT? > > Cheers, > Christian > > > > > Christophe Servan hat am 24. November 2011 um > 19:02 geschrieb

Re: [Moses-support] Mert and METEOR

2011-11-24 Thread Christian Federmann
Hi all,   is there a specific reason/obstacle that METEOR is not available for MERT?   Cheers,    Christian     Christophe Servan hat am 24. November 2011 um 19:02 geschrieben: > Hi Wilker, > I confirm there is currently no scorer for METEOR in MERT. > Do you plan do implement it ? > > Best reg

Re: [Moses-support] Mert and METEOR

2011-11-24 Thread Christophe Servan
Hi Wilker, I confirm there is currently no scorer for METEOR in MERT. Do you plan do implement it ? Best regards, Christophe -- Christophe SERVAN - PhD - Postdoc Laboratoire Informatique de l'Université du Maine (LIUM) Institut Claude Cha

Re: [Moses-support] Incremental training... how?

2011-11-24 Thread Philipp Koehn
Hi, check the FAQ: http://www.statmt.org/moses/?n=Moses.AdvancedFeatures#ntoc27 -phi 2011/11/21 Kádár Tamás (KTamas) : > Hi > > I'm experimenting with incremental training. The documentation says: > "Updating a running Moses instance is done via XML RPC". And, uh, > that's it. I can't find any o

[Moses-support] Mert and METEOR

2011-11-24 Thread Wilker Aziz
Hi all, I checked on Moses manual and also the mailing list archives and I couldn't quite find the information I need, so I'm trying with you. As I understood, zmert was remove from Moses and MERT is now able to optimize towards different metrics. I know that a version of METEOR adapted for zmert

Re: [Moses-support] Training with n-class model

2011-11-24 Thread Philipp Koehn
Hi, if you get errors such as ERROR: Forbidden zero sentence length 0 ERROR: Forbidden zero sentence length 0 the you have empty lines in your parallel corpus. Please run the corpus cleaning script first. -phi On Tue, Nov 22, 2011 at 3:39 PM, Raja Bensalem wrote: > Hello > I'm translating fr

Re: [Moses-support] Filtering LMs

2011-11-24 Thread Miles Osborne
the problem is that there are many ways to create five grams etc from common words --words you will find in a large parallel set. you will save on space, but it will not be significant. nowadays there really is no need to use SRILM at all. before considering drastic action, look at these other m

Re: [Moses-support] Filtering LMs

2011-11-24 Thread Kenneth Heafield
http://kheafield.com/code/filter.html filters ARPA files. This is a separate project from KenLM. I think most LM toolkits don't check/require that the ARPA is normalized. At least, KenLM, IRSTLM, and SRILM work fine with filtered models. On 11/24/11 13:07, Miles Osborne wrote: > this can be don

Re: [Moses-support] Filtering LMs

2011-11-24 Thread Miles Osborne
this can be done, but it tends to not save much space. also it does not help deal with OOVs, which the language model can still score even though they are not in the parallel set. if you are worried about saving space then you should either look at KenLM or RandLM Miles On 24 November 2011 12:5

[Moses-support] Filtering LMs

2011-11-24 Thread Thomas Schoenemann
Dear all,  I hope that this is not too stupid a question, and that it hasn't been asked recently. In the MOSES EMS, when running experiments the phrase table is automatically reduced to only those phrases that actually occur in the respective dev/test set. Obviously this saves a lot of memory