tuning step is not repeated. decoding use the same moses.ini and the same input but different parameters: moses/mosesdecoder/65c75ff/bin/moses -search-algorithm 1 -cube-pruning-pop-limit 5000 -s 5000 -threads 8 -text-type "test" -v 0 -f wmt18_en-de/evaluation/test.filtered.ini.7 < wmt18_en-de/evaluation/test.input.tc.1 > wmt18_en-de/evaluation/test.output.7
vs. with alignment: moses/mosesdecoder/65c75ff/bin/moses -search-algorithm 1 -cube-pruning-pop-limit 5000 -s 5000 -threads 8 --mark-unknown --unknown-word-prefix UNK_ --print-alignment-info -text-type "test" -v 0 -f wmt18_en-de/evaluation/test.filtered.ini.7 < wmt18_en-de/evaluation/test.input.tc.1 > wmt18_en-de/evaluation/test.output.9 both are followed by the following steps: moses/mosesdecoder/scripts/ems/support/remove-segmentation-markup.perl < wmt18_en-de/evaluation/test.output.7 > wmt18_en-de/evaluation/test.cleaned.7 moses/mosesdecoder/scripts/recaser/detruecase.perl < wmt18_en-de/evaluation/test.cleaned.7 > wmt18_en-de/evaluation/test.truecased.7 and equivalently with: moses/mosesdecoder/scripts/ems/support/remove-segmentation-markup.perl < wmt18_en-de/evaluation/test.output.9 > wmt18_en-de/evaluation/test.cleaned.9 moses/mosesdecoder/scripts/recaser/detruecase.perl < wmt18_en-de/evaluation/test.cleaned.9 > wmt18_en-de/evaluation/test.truecased.9 scoring step use test.truecased.7 and test.truecased.9. Ergun On Fri, Aug 24, 2018 at 5:15 PM Ergun Bicici <bic...@gmail.com> wrote: > > only the evaluation decoding steps are repeated that are steps 10, 9, and > 7 in the following steps in EMS output: > 48 TRAINING:consolidate -> re-using (1) > 47 TRAINING:prepare-data -> re-using (1) > 46 TRAINING:run-giza -> re-using (1) > 45 TRAINING:run-giza-inverse -> re-using (1) > 44 TRAINING:symmetrize-giza -> re-using (1) > 43 TRAINING:build-lex-trans -> re-using (1) > 40 TRAINING:build-osm -> re-using (1) > 39 TRAINING:extract-phrases -> re-using (1) > 38 TRAINING:build-reordering -> re-using (1) > 37 TRAINING:build-ttable -> re-using (1) > 34 TRAINING:create-config -> re-using (1) > 28 TUNING:truecase-input -> re-using (1) > 24 TUNING:truecase-reference -> re-using (1) > 21 TUNING:filter -> re-using (1) > 20 TUNING:apply-filter -> re-using (1) > 19 TUNING:tune -> re-using (1) > 18 TUNING:apply-weights -> re-using (1) > 15 EVALUATION:test:truecase-input -> re-using (1) > 12 EVALUATION:test:filter -> re-using (1) > 11 EVALUATION:test:apply-filter -> re-using (1) > > > > *10 EVALUATION:test:decode -> run 9 EVALUATION:test:remove-markup -> > run 7 EVALUATION:test:detruecase-output -> run *3 > EVALUATION:test:multi-bleu-c -> run > 2 EVALUATION:test:analysis-coverage -> re-using (1) > 1 EVALUATION:test:analysis-precision -> run > > > On Fri, Aug 24, 2018 at 4:39 PM Hieu Hoang <hieuho...@gmail.com> wrote: > >> are you rerunning tuning for each case? Or are you using exactly the same >> moses.ini file for the with and with alignment experiments? >> >> Hieu Hoang >> http://statmt.org/hieu >> >> >> On Fri, 24 Aug 2018 at 14:34, Ergun Bicici <bic...@gmail.com> wrote: >> >>> >>> Dear Moses maintainers, >>> >>> I discovered that the translations obtained differ when alignment flags >>> (--mark-unknown >>> --unknown-word-prefix UNK --print-alignment-inf) are used. Comparison >>> table is attached (en-ru and ru-en are being recomputed). We expect them to >>> be the same since alignment flags only print additional information and >>> they are not supposed to alter decoding. In both, the same EMS system was >>> re-run with the alignment information flags or not. >>> >>> - Average of the absolute difference is 0.0094 BLEU (about 1 BLEU >>> points). >>> - Average of the difference is 0.0051 BLEU (about 0.5 BLEU points, >>> results are better with alignment flags). >>> >>>  >>> >>> /opt/Programs/SMT/moses/mosesdecoder/bin/moses --version >>> >>> Moses code version (git tag or commit hash): >>> mmt-mvp-v0.12.1-2775-g65c75ff07-dirty >>> Libraries used: >>> Boost version 1.62.0 >>> >>> git status >>> On branch RELEASE-4.0 >>> Your branch is up to date with 'origin/RELEASE-4.0'. >>> >>> >>> Note: Using alignment information to recase tokens was tried in [1] for >>> en-fi and en-tr to claim positive results. We tried this method in all >>> translation directions we considered as as can be seen in the align row, >>> this only improves the performance for tr-en and en-tr and for tr-en Moses >>> provides better translations without the alignment flags. >>> [1]The JHU Machine Translation Systems for WMT 2016 >>> Shuoyang Ding, Kevin Duh, Huda Khayrallah, Philipp Koehn and Matt Post >>> http://www.statmt.org/wmt16/pdf/W16-2310.pdf >>> >>> >>> Best Regards, >>> Ergun >>> >>> Ergun Biçici >>> http://bicici.github.com/ <http://ergunbicici.blogspot.com/> >>> >>> _______________________________________________ >>> Moses-support mailing list >>> Moses-support@mit.edu >>> http://mailman.mit.edu/mailman/listinfo/moses-support >>> >> > > -- > > Regards, > Ergun > > > -- Regards, Ergun
_______________________________________________ Moses-support mailing list Moses-support@mit.edu http://mailman.mit.edu/mailman/listinfo/moses-support
