hi there
the input sentence can't contain <> characters.
If you had tokenized the input with the moses script
/scripts/tokenizer/tokenizer.perl
then those characters would have been converted to
If you used your own tokenizer, be sure to
/scripts/tokenizer/escape-special-chars.perl
before feeding it to the moses decoder.
And of course, you should process your training data the same way you
process your test and tuning data
On 12/10/2019 2:13 AM, Claudia Matos Veliz wrote:
> Hello everyone
>
> I’m getting a very estrange error while translating using Moses. I’m
> doing 10-CV experiments and in one of the folds I get a Segmentation
> Fault error. For the rest everything finished without problem. I hope
> you can help me with that. Bellow you can see the log. I get the error
> in the first line of the file I want to translate. I’ve tried removing
> the line and I got the same error.
>
> $ /opt/mosesdecoder-4.0-prebuilt/moses/bin/moses -config
> ../model/moses.ini -input-file token.all.fold2.test.clean.ori >
> token.all.fold2.test.norm
> Defined parameters (per moses.ini or switch):
> config: ../model/moses.ini
> distortion-limit: 6
> feature: UnknownWordPenalty WordPenalty PhrasePenalty
> PhraseDictionaryOnDisk name=TranslationModel0 num-features=4
> path=/home/claudia/SMT_ext_data/data/mixed/amica/1folds/2/model/phrase-table.table
>
> input-factor=0 output-factor=0 LexicalReordering
> name=LexicalReordering0 num-features=6
> type=wbe-msd-bidirectional-fe-allff input-factor=0 output-factor=0
> path=/home/claudia/SMT_ext_data/data/mixed/amica/1folds/2/model/reordering-table.wbe-msd-bidirectional-fe.gz
>
> Distortion KENLM name=LM0 factor=0
> path=/home/claudia/SMT_ext_data/data/lm/lm/sms.cgn.token.lm order=5
> input-factors: 0
> input-file: token.all.fold2.test.clean.ori
> mapping: 0 T 0
> weight: UnknownWordPenalty0= 1 WordPenalty0= -1 PhrasePenalty0= 0.2
> TranslationModel0= 0.2 0.2 0.2 0.2 LexicalReordering0= 0.3 0.3 0.3 0.3
> 0.3 0.3 Distortion0= 0.3 LM0= 0.5
> line=UnknownWordPenalty
> FeatureFunction: UnknownWordPenalty0 start: 0 end: 0
> line=WordPenalty
> FeatureFunction: WordPenalty0 start: 1 end: 1
> line=PhrasePenalty
> FeatureFunction: PhrasePenalty0 start: 2 end: 2
> line=PhraseDictionaryOnDisk name=TranslationModel0 num-features=4
> path=/home/claudia/SMT_ext_data/data/mixed/amica/1folds/2/model/phrase-table.table
>
> input-factor=0 output-factor=0
> FeatureFunction: TranslationModel0 start: 3 end: 6
> line=LexicalReordering name=LexicalReordering0 num-features=6
> type=wbe-msd-bidirectional-fe-allff input-factor=0 output-factor=0
> path=/home/claudia/SMT_ext_data/data/mixed/amica/1folds/2/model/reordering-table.wbe-msd-bidirectional-fe.gz
> Initializing Lexical Reordering Feature..
> FeatureFunction: LexicalReordering0 start: 7 end: 12
> line=Distortion
> FeatureFunction: Distortion0 start: 13 end: 13
> line=KENLM name=LM0 factor=0
> path=/home/claudia/SMT_ext_data/data/lm/lm/sms.cgn.token.lm order=5
> Loading the LM will be faster if you build a binary file.
> Reading /home/claudia/SMT_ext_data/data/lm/lm/sms.cgn.token.lm
> 5---10---15---20---25---30---35---40---45---50---55---60---65---70---75---80---85---90---95--100
>
> FeatureFunction: LM0 start: 14 end: 14
> Loading UnknownWordPenalty0
> Loading WordPenalty0
> Loading PhrasePenalty0
> Loading LexicalReordering0
> Loading table into memory...done.
> Loading Distortion0
> Loading LM0
> Loading TranslationModel0
> Created input-output object : [2.910] seconds
> Translating: ik vind die zo mooii . ik hou van je
> Segmentation fault (core dumped)
>
>
>
>
>
>
>
> ___
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
--
Hieu Hoang
http://statmt.org/hieu
___
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support
