grep your model files. I'm pretty sure there's a | character in there somewhere
On 27/09/2010 15:02, Souhir Gahbiche wrote: > I'm sure that there's no bar characters in the data. > > SG > > 2010/9/27, Miles Osborne<mi...@inf.ed.ac.uk>: >> looking at your output: >>> [ERROR] Malformed input at >> Expected input to have words composed of 1 factor(s) (form FAC1|FAC2|...) >> but instead received input with 0 factor(s). >> sh: line 1: 5114 Aborted >> make sure you have no bar (|) characters in the data >> >> Miles >> >> >> On 27 September 2010 14:45, Souhir Gahbiche<s.gahbi...@gmail.com> wrote: >>> Hi all, >>> >>> I'm trying to tune my system, but the tuning stops at the first >>> iteration. Here is my mert.log file: >>> >>> After default: -l mem_free=0.5G -hard >>> Using SCRIPTS_ROOTDIR: >>> /vol/mt2/tools/nadi/moses-scripts/scripts-20090923-1833/ >>> SYNC distortionchecking weight-count for ttable-file >>> checking weight-count for lmodel-file >>> checking weight-count for distortion-file >>> Executing: mkdir -p /working/tuningdev2009/mert >>> Executing: >>> /vol/mt2/tools/nadi/moses-scripts/scripts-20090923-1833//training/filter-model-given-input.pl >>> ./filtered /tmp/souhir >>> /model/mosesdev2009.ini >>> /working/tuningdev2009/ar.project-syndicate.2009-07.v1.de >>> v.bw.mada.tok >>> filtering the phrase tables... Fri Aug 27 16:43:10 CEST 2010 >>> The filtered model was ready in /working/tuningdev2009/mert/filtered, >>> not doing a >>> nything. >>> run 1 start at Fri Aug 27 16:43:10 CEST 2010 >>> Parsing --decoder-flags: |-v 0| >>> Saving new config to: ./run1.moses.ini >>> Saved: ./run1.moses.ini >>> Normalizing lambdas: 0 1 1 1 1 1 1 1 1 0.3 0.2 0.3 0.2 0 >>> DECODER_CFG = -w %.6f -lm %.6f -d %.6f %.6f %.6f %.6f %.6f %.6f %.6f >>> -tm %.6f %.6f %.6f %.6f %.6f >>> values = 0 0.111111111111111 0.111111111111111 0.111111111111111 >>> 0.111111111111111 0.111111111111111 0.111111111111111 0.1 >>> 11111111111111 0.111111111111111 0.0333333333333333 0.0222222222222222 >>> 0.0333333333333333 0.0222222222222222 0 >>> Executing: /vol/mt2/tools/nadi/moses/moses-cmd/src/moses -v 0 -config >>> filtered/moses.ini -inputtype 0 -w 0.000000 -lm 0.111111 >>> -d 0.111111 0.111111 0.111111 0.111111 0.111111 0.111111 0.111111 -tm >>> 0.033333 0.022222 0.033333 0.022222 0.000000 -n-best-li >>> st run1.best100.out 100 -i >>> /working/tuningdev2009/ar.project-syndicate.2009-07.v1 >>> .dev.bw.mada.tok> run1.out >>> (1) run decoder to produce n-best lists >>> params = -v 0 >>> decoder_config = -w 0.000000 -lm 0.111111 -d 0.111111 0.111111 >>> 0.111111 0.111111 0.111111 0.111111 0.111111 -tm 0.033333 0.0222 >>> 22 0.033333 0.022222 0.000000 >>> Loading lexical distortion models... >>> have 1 models >>> Creating lexical reordering... >>> weights: 0.111 0.111 0.111 0.111 0.111 0.111 >>> Loading table into memory...done. >>> Created lexical orientation reordering >>> [ERROR] Malformed input at >>> Expected input to have words composed of 1 factor(s) (form FAC1|FAC2|...) >>> but instead received input with 0 factor(s). >>> sh: line 1: 5114 Aborted >>> /vol/mt2/tools/nadi/moses/moses-cmd/src/moses -v 0 -config >>> filtered/moses.ini -inputt >>> ype 0 -w 0.000000 -lm 0.111111 -d 0.111111 0.111111 0.111111 0.111111 >>> 0.111111 0.111111 0.111111 -tm 0.033333 0.022222 0.033333 >>> 0.022222 0.000000 -n-best-list run1.best100.out 100 -i >>> /working/tuningdev2009/ar >>> .project-syndicate.2009-07.v1.dev.bw.mada.tok> run1.out >>> Exit code: 134 >>> The decoder died. CONFIG WAS -w 0.000000 -lm 0.111111 -d 0.111111 >>> 0.111111 0.111111 0.111111 0.111111 0.111111 0.111111 -tm 0.0 >>> 33333 0.022222 0.033333 0.022222 0.000000 >>> >>> The file run1.out is empty. I tried many times, but every time it >>> stopps at the same level. >>> I looked for the moses.ini. It works perfectly when I use two phrase >>> tables. >>> >>> Here's my moses.ini used : >>> >>> ######################### >>> ### MOSES CONFIG FILE ### >>> ######################### >>> >>> # input factors >>> [input-factors] >>> 0 >>> >>> # mapping steps >>> [mapping] >>> 0 T 0 >>> >>> # translation tables: source-factors, target-factors, number of scores, >>> file >>> [ttable-file] >>> 0 0 5 /working/model/phrase >>> >>> # no generation models, no generation-file section >>> >>> # language models: type(srilm/irstlm), factors, order, file >>> [lmodel-file] >>> 0 0 4 /working/lmm/newsLM+news-train08.fr.4gki.arpa.gz >>> >>> # limit on how many phrase translations e for each phrase f are loaded >>> # 0 = all elements loaded >>> [ttable-limit] >>> 20 >>> >>> # distortion (reordering) files >>> [distortion-file] >>> 0-0 msd-bidirectional-fe 6 /working/model/reordering-table.gz >>> >>> # distortion (reordering) weight >>> [weight-d] >>> 0.3 >>> 0.3 >>> 0.3 >>> 0.3 >>> 0.3 >>> 0.3 >>> 0.3 >>> >>> # language model weights >>> [weight-l] >>> 0.5000 >>> >>> # translation model weights >>> [weight-t] >>> 0.2 >>> 0.2 >>> 0.2 >>> 0.2 >>> 0.2 >>> >>> # no generation models, no weight-generation section >>> >>> # word penalty >>> [weight-w] >>> -1 >>> >>> [distortion-limit] >>> 6 >>> >>> Any ideas? >>> Thanks >>> SG >>> _______________________________________________ >>> Moses-support mailing list >>> Moses-support@mit.edu >>> http://mailman.mit.edu/mailman/listinfo/moses-support >>> >> >> >> -- >> The University of Edinburgh is a charitable body, registered in >> Scotland, with registration number SC005336. >> > _______________________________________________ > Moses-support mailing list > Moses-support@mit.edu > http://mailman.mit.edu/mailman/listinfo/moses-support > > _______________________________________________ Moses-support mailing list Moses-support@mit.edu http://mailman.mit.edu/mailman/listinfo/moses-support
