| Hi, I was adding a new feature function in Moses (https://github.com/KonceptGeek/mosesdecoder/blob/master/moses/FF/CoarseBiLM.cpp). It works fine when I test it for 1-2 sentences, but when I’m trying to tune my parameters, I’m getting segmentation faults or sometimes it is bad_alloc. Following was one of the commands that was executed during the tuning process which caused the Segmentation Fault or bad_alloc: moses -threads 40 -v 0 -config filtered/moses.ini -weight-overwrite 'CoarseLM100= 0.075758 LM0= 0.075758 CoarseBiLMNotClustered= 0.075758 WordPenalty0= -0.151515 PhrasePenalty0= 0.030303 CoarseBiLMClustered= 0.075758 TranslationModel0= 0.030303 0.030303 0.030303 0.030303 Distortion0= 0.045455 CoarseLM1600= 0.075758 LexicalReordering0= 0.045455 0.045455 0.045455 0.045455 0.045455 0.045455' -n-best-list run1.best100.out 100 distinct -input-file tune.word.lc.cn The log is enclosed in this email. Any pointers would be very useful. Thanks, Jasneet |
Loading the LM will be faster if you build a binary file. Reading /cs/natlang-projects/jasneet/Data/BilingEmbeddings/mt/lm/english.giga5.lm.gz ----5---10---15---20---25---30---35---40---45---50---55---60---65---70---75---80---85---90---95--100 *The ARPA file is missing <unk>. Substituting log10 probability -100.000. *************************************************************************************************** Loading table into memory...done. Key: tgtWordToClusterId LoadManyToOneMap Value: /cs/natlang-projects/jasneet/Data/BilingEmbeddings/outputs-clusters/mkcls/cn-en-roland-generic-big/clusters/clusters.400.en Key: srcWordToClusterId LoadManyToOneMap Value: /cs/natlang-projects/jasneet/Data/BilingEmbeddings/outputs-clusters/mkcls/cn-en-roland-generic-big/clusters/clusters.400.cn Key: bitokenToBitokenId LoadManyToOneMap Value: /cs/natlang-projects/jasneet/Data/BilingEmbeddings/outputs-clusters/mkcls/cn-en-roland-generic-big/corpus/corpus.bitoken.400.class.cnen.tags Key: bitokenIdToClusterId LoadManyToOneMap Value: /cs/natlang-projects/jasneet/Data/BilingEmbeddings/outputs-clusters/mkcls/cn-en-roland-generic-big/clusters/clusters.400.bitoken Key: lm Key: ngrams Key: tgtWordToClusterId LoadManyToOneMap Value: /cs/natlang-projects/jasneet/Data/BilingEmbeddings/outputs-clusters/mkcls/cn-en-roland-generic-big/clusters/clusters.400.en Key: srcWordToClusterId LoadManyToOneMap Value: /cs/natlang-projects/jasneet/Data/BilingEmbeddings/outputs-clusters/mkcls/cn-en-roland-generic-big/clusters/clusters.400.cn Key: bitokenToBitokenId LoadManyToOneMap Value: /cs/natlang-projects/jasneet/Data/BilingEmbeddings/outputs-clusters/mkcls/cn-en-roland-generic-big/corpus/corpus.bitoken.400.class.cnen.tags Key: lm Key: ngrams Key: tgtWordToClusterId LoadManyToOneMap Value: /cs/natlang-projects/jasneet/Data/BilingEmbeddings/outputs-clusters/mkcls/cn-en-roland-generic-big/clusters/clusters.100.en Key: lm Key: ngrams Key: tgtWordToClusterId LoadManyToOneMap Value: /cs/natlang-projects/jasneet/Data/BilingEmbeddings/outputs-clusters/mkcls/cn-en-roland-generic-big/clusters/clusters.1600.en Key: lm Key: ngrams terminate called after throwing an instance of 'std::bad_alloc' what(): std::bad_alloc
_______________________________________________ Moses-support mailing list Moses-support@mit.edu http://mailman.mit.edu/mailman/listinfo/moses-support
