i'd check to see how unknown words are handled in either the SRILM or in IRSTLM --that may explain the differences
(as for the size of a tuning set, the more the better; right now i'm doing Europarl runs (Fr -- En) and our dev set has 2k sentences.) Miles 2008/8/2 John D. Burger <[EMAIL PROTECTED]> > Hieu Hoang wrote: > > > The 3rd & 4th columns of the phrase table contain alignment > > information > > about the words in the phrase. How it is created changed recently. > > > > However, this information isn't used by the decoder in the main > > trunk so > > shouldn't affect performance. > > OK, so that's a red herring. > > > If the performance of the decoder is lower for the same weights, I > > will be > > very concerned. Can you tell me if this is the case? > > No, that's not what's happening. MERT arrives at different weights > in the two versions - that's where they start to differ. With the > exact same phrase table and reordering model, the two versions start > diverging with the first iteration of MERT - about a point lower, > which carries through to the last iteration, and then also a point > lower evaluating on a held-out devtest. All of the data is exactly > the same. > > Another difference I forgot about is that our new build is with IRST > LM, the old one was with SRILM. Duh, that could very well send MERT > in a different direction. I should have thought of that earlier. We > may have to rebuild with SRI for me to get a better handle on the > differences. > > > Which older version of Moses are you comparing it against? > > It's almost exactly a year old, sadly. What's the easiest way to > tell what version it is? > > Miles asked about the size of the tuning set - it's 812 segments. > That's not that small, is it? > > Thanks for your prompt replies and suggestions. > > - John D. Burger > MITRE > > > _______________________________________________ > Moses-support mailing list > Moses-support@mit.edu > http://mailman.mit.edu/mailman/listinfo/moses-support > > -- The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336.
_______________________________________________ Moses-support mailing list Moses-support@mit.edu http://mailman.mit.edu/mailman/listinfo/moses-support
