Hi Guchun -text-type is connected with some experimental domain adaptation features. -t is an abbreviation for -report-segmentation
cheers - Barry On 10/11/12 09:38, Guchun Zhang wrote: > Hi Barry, > > Just a quick question that what -t and -text-type in the decoder are for? > > Many thanks, > > Guchun > > On 10 November 2012 09:26, Guchun Zhang <gzh...@alphacrc.com > <mailto:gzh...@alphacrc.com>> wrote: > > Hi Barry, > > Both problems are gone now on SGE. Excellent. > > Cheers, > > Guchun > > > On 9 November 2012 13:41, Guchun Zhang <gzh...@alphacrc.com > <mailto:gzh...@alphacrc.com>> wrote: > > Hi Barry, > > I am doing it now. :-) > > Cheers, > > Guchun > > > On 9 November 2012 13:35, Barry Haddow > <bhad...@staffmail.ed.ac.uk > <mailto:bhad...@staffmail.ed.ac.uk>> wrote: > > Hi Guchun > > The -text-type problem should be fixed by a commit I made > yesterday. I haven't tested SGE operation though, so can > you try again with the latest EMS? > > cheers - Barry > > > On 09/11/12 13:26, Guchun Zhang wrote: > > Thanks, Barry. > > The following is the output file of one of the qsub jobs: > > Linux ubuntu11 3.2.0-24-generic #39-Ubuntu SMP Mon May > 21 16:52:17 UTC 2012 x86_64 x86_64 x86_64 GNU/Linux > Defined parameters (per moses.ini or switch): > config: /home/guchun/Data/mt-files/ > tests/exp-new-training/model/ > EN_US-FR_FR//evaluation/model. filtered.ini.1 > cube-pruning-pop-limit: 5000 > distortion-file: 0-0 > wbe-msd-bidirectional-fe-allff 6 > /home/guchun/Data/mt-files/ > tests/exp-new-training/model/ > EN_US-FR_FR//evaluation/model. > filtered.1/reordering-table.1. > 0-0.wbe-msd-bidirectional-fe > distortion-limit: 6 > input-factors: 0 > input-file: model.input.tc.1.split16701-aa > inputtype: 0 > lmodel-file: 0 0 7 /home/guchun/Data/mt-files/ > tests/exp-new-training/model/ > EN_US-FR_FR//lm/model.lm.1 0 1 7 > /home/guchun/Data/mt-files/ > tests/exp-new-training/model/ > EN_US-FR_FR//lm/model=pos.lm.1 > mapping: 0 T 0 > search-algorithm: 1 > stack: 5000 > ttable-file: 1 0 0,1 5 > /home/guchun/Data/mt-files/ > tests/exp-new-training/model/ > EN_US-FR_FR//evaluation/model. > filtered.1/phrase-table.0-0,1. 1.1 > ttable-limit: 20 > weight-d: 0.063647 0.0630623 0.0827805 > 0.0571843 0.0856819 0.0255637 0.0459574 > weight-l: 0.139309 0.0196229 > weight-t: 0.136273 0.0197609 0.021245 > 0.0725207 -0.0529247 > weight-w: -0.114467 > *ERROR:illegal switch: -t-text-type* > > exit status 1 > exit status 0 > > I checked -t wasn't present in the config file but > appeared in the command. A typo in mert-moses.pl > <http://mert-moses.pl> <http://mert-moses.pl>? > > > I will pull the change you made and try it out in a > moment. > > Cheers, > > Guchun > On 9 November 2012 12:15, Barry Haddow > <bhad...@staffmail.ed.ac.uk > <mailto:bhad...@staffmail.ed.ac.uk> > <mailto:bhad...@staffmail.ed. ac.uk > <mailto:bhad...@staffmail.ed.ac.uk>>> wrote: > > Hi Guchun > > I've pushed a change that may fix the second > error. Can you let me > know, and if it doesn't fix it, can you send me > your ini file to > help with testing. > > For the first error, could you post one of the > decoder log files > to see why the decoder is crashing, > > cheers - Barry > > > On 09/11/12 11:15, Guchun Zhang wrote: > > Hi there, > > I hope you can help me out with two new > problems after > installing a new Moses commit. I have spent > some time on them, > but to no avail. > > The first occurred during decode. The command is > > ##### > /home/guchun/Work/moses/moses- > scripts/generic/moses- > parallel.pl <http://parallel.pl> > <http://moses-parallel.pl> > <http://moses-parallel.pl> -decoder > /home/guchun/Work/moses/ > mosesdecoder/bin/moses -config > /home/guchun/Data/mt-files/ > tests/exp-new-training/model/ > EN_US-FR_FR//evaluation/model. > filtered.ini.1 -input-file > /home/guchun/Data/mt-files/ > tests/exp-new-training/model/ > EN_US-FR_FR//evaluation/model. > > input.tc.1 *--jobs 32* -decoder-parameters > "-search-algorithm > 1 -cube-pruning-pop-limit 5000 -s 5000 -t" > > /home/guchun/Data/mt-files/ > tests/exp-new-training/model/ > EN_US-FR_FR//evaluation/model. output.2 > -text-type "test" > > > ##### > > The models are built using the EMS factored > example. The error > message is > > ----- > Split (-aa) were not entirely translated > outputN=0 inputN=36 > outputfile=model.input.tc.1. split17159-aa.trans > inputfile=model.input.tc.1. split17159-aa > .... > everything crashed, not trying to resubmit jobs > Got interrupt or something failed. > kill_all_and_quit > qdel 17791 > Executing: qdel 17791 > Exit code: 1 > .... > Translation was not performed correctly > or some of the submitted jobs died. > qdel function was called for all submitted jobs > ----- > > Using bin/moses -thread 32 .... works fine. > > The second error is during the first iteration > of tuning with > a corpus having 4 factors. The command is > > ##### > /home/guchun/Work/moses/moses- > scripts/training/mert-moses.pl <http://mert-moses.pl> > <http://mert-moses.pl> <http://mert-moses.pl> > /home/guchun/Data/mt-files/ > tests/exp-new-training/model/ > EN_US-FR_FR/rb//tuning/input. tc.1 > /home/guchun/Data/mt-files/ > tests/exp-new-training/model/ > EN_US-FR_FR/rb//tuning/ > reference.tc.1 /home/guchun/Work/moses/ > mosesdecoder/bin/moses > /home/guchun/Data/mt-files/ > tests/exp-new-training/model/ > EN_US-FR_FR/rb//tuning/moses. filtered.ini.2 > --nbest 100 > --working-dir /home/guchun/Data/mt-files/ > tests/exp-new-training/model/ > EN_US-FR_FR/rb//tuning/tmp.2 > --decoder-flags "" --rootdir > /home/guchun/Work/moses/moses- > scripts -mertdir /home/guchun/Work/moses/ > mosesdecoder/bin > > --no-filter-phrase-table *--jobs 32* > > ##### > > The error message is > > ----- > The decoder returns the scores in this order: > d d d d d d d lm > lm w tm tm tm tm tm tm tm tm tm tm tm tm tm tm > tm g g g g > Mismatched lambdas. Decoder returned d d d d d > d d lm lm w tm > tm tm tm tm tm tm tm tm tm tm tm tm tm tm g g > g g, we expected > d d d d d d d lm lm w tm tm tm tm tm g g tm tm > tm tm tm g g tm > tm tm tm tm at /home/guchun/Work/moses/moses- > scripts/training/mert-moses.pl > <http://mert-moses.pl> <http://mert-moses.pl> > <http://mert-moses.pl> line 1155. > > ----- > > I never had these problems with a previous > commit back in > March. Any hints will be warmly welcomed. > > Many thanks, > > Guchun > > > ______________________________ _________________ > Moses-support mailing list > Moses-support@mit.edu <mailto:Moses-support@mit.edu> > <mailto:Moses-support@mit.edu > <mailto:Moses-support@mit.edu>> > http://mailman.mit.edu/ mailman/listinfo/moses-support > > <http://mailman.mit.edu/ > mailman/listinfo/moses-support > <http://mailman.mit.edu/mailman/listinfo/moses-support> > > > > > -- The University of Edinburgh is a charitable > body, registered in > Scotland, with registration number SC005336. > > > > > > -- > The University of Edinburgh is a charitable body, > registered in > Scotland, with registration number SC005336. > > > > > -- The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336. _______________________________________________ Moses-support mailing list Moses-support@mit.edu http://mailman.mit.edu/mailman/listinfo/moses-support
