Dear Moses community, To make the question clearer:
The question is why does the cumulative score add the brevity penalty before taking the exponent at every order of ngram but the individual score only takes the brevity penalty into account at https://github.com/moses-smt/mosesdecoder/blob/master/scripts/generic/mteval-v13a.pl#L874 Any pointers to the papers describing the cumulative score would be nice =) Thanks in advance again, Nat On Tue, Oct 25, 2016 at 11:58 AM, Nat Gillin <nat.gil...@gmail.com> wrote: > Dear Moses Community, > > When using mteval-13a.pl, we note that the output looks like this: > > length ratio: 1.07303974221267 (1998/1862), penalty (log): 0 > > NIST score = 5.0564 BLEU score = 0.2318 for system "Google" > > > # ------------------------------------------------------------------------ > > > Individual N-gram scoring > > 1-gram 2-gram 3-gram 4-gram 5-gram 6-gram 7-gram > 8-gram 9-gram > > ------ ------ ------ ------ ------ ------ ------ > ------ ------ > > NIST: 4.4488 0.5554 0.0477 0.0045 0.0000 0.0000 0.0000 > 0.0000 0.0000 "Google" > > > BLEU: 0.5415 0.2972 0.1752 0.1025 0.0626 0.0354 0.0193 > 0.0085 0.0017 "Google" > > > # ------------------------------------------------------------------------ > > Cumulative N-gram scoring > > 1-gram 2-gram 3-gram 4-gram 5-gram 6-gram 7-gram > 8-gram 9-gram > > ------ ------ ------ ------ ------ ------ ------ > ------ ------ > > NIST: 4.4488 5.0043 5.0520 5.0564 5.0564 5.0564 5.0564 > 5.0564 5.0564 "Google" > > > BLEU: 0.5415 0.4012 0.3044 0.2318 0.1784 0.1362 0.1031 > 0.0754 0.0493 "Google" > > And at https://github.com/moses-smt/mosesdecoder/blob/master/ > scripts/generic/mteval-v13a.pl#L823, it tries to calculate the cumulative > score by accumulate the individual ngram precisions and at each order of > ngram add to it and do a normalization before calculating the cumulative > score for each order of nrgram. > > The question is why does it add the brevity penalty? (i.e. $len_score) > > Also, is this score discussed in any paper? > > Thanks in advance for the clarifications! > > Regards, > Nat > >
_______________________________________________ Moses-support mailing list Moses-support@mit.edu http://mailman.mit.edu/mailman/listinfo/moses-support
