Hey Mike, Is it possible for you to make the phrase table available /home/mike/stelae-projects/de-en/phrasemodel/tmp.15419/pt.txt.gz publically so we can try to reproduce the problem?
Cheers, Nick On Tue, May 2, 2017 at 6:06 AM, Mike Ladwig <mdlad...@gmail.com> wrote: > Got an exception creating a PT2 with yesterday's master: > > Binarize phrase and reordering model in probing table format: > /home/mike/stelae5/mosesdecoder/scripts/generic/binarize4moses2.perl > --phrase-table=/home/mike/stelae-projects/de-en/phrasemodel/model/phrase-table.gz > --lex-ro=/home/mike/stelae-projects/de-en/phrasemodel/model/reordering-table.wbe-msd-bidirectional-fe.gz > --output-dir=/home/mike/stelae-projects/de-en/phrasemodel/PT2 > --num-lex-scores=6 > Executing: gzip -dc > /home/mike/stelae-projects/de-en/phrasemodel/model/phrase-table.gz | > /home/mike/stelae5/mosesdecoder/scripts/generic/../../contrib/sigtest-filter/filter-pt > -n 0 | gzip -c > > /home/mike/stelae-projects/de-en/phrasemodel/tmp.15419/pt.gz > sh: > /home/mike/stelae5/mosesdecoder/scripts/generic/../../contrib/sigtest-filter/filter-pt: > No such file or directory > Executing: > /home/mike/stelae5/mosesdecoder/scripts/generic/../../bin/processLexicalTableMin > -in > /home/mike/stelae-projects/de-en/phrasemodel/model/reordering-table.wbe-msd-bidirectional-fe.gz > -out /home/mike/stelae-projects/de-en/phrasemodel/tmp.15419/lex-ro -T . > -threads all > Used options: > Text reordering table will be read from: > /home/mike/stelae-projects/de-en/phrasemodel/model/reordering-table.wbe-msd-bidirectional-fe.gz > Output reordering table will be written to: > /home/mike/stelae-projects/de-en/phrasemodel/tmp.15419/lex-ro.minlexr > Step size for source landmark phrases: 2^10=1024 > Phrase fingerprint size: 16 bits / P(fp)=1.52588e-05 > Single Huffman code set for score components: no > Using score quantization: no > Running with 24 threads > > Pass 1/2: Creating phrase index + Counting scores > ..................................................[5000000] > ..................................................[10000000] > ..................................................[15000000] > ..................................................[20000000] > ..................................................[25000000] > ..................................................[30000000] > ..................................................[35000000] > ..................................................[40000000] > ..................................................[45000000] > ..................................................[50000000] > ..................................................[55000000] > ..................................................[60000000] > ..................................................[65000000] > ..................................................[70000000] > ..................................................[75000000] > .................... > > Intermezzo: Calculating Huffman code sets > Creating Huffman codes for 32003 scores > Creating Huffman codes for 16732 scores > Creating Huffman codes for 31335 scores > Creating Huffman codes for 32076 scores > Creating Huffman codes for 15096 scores > Creating Huffman codes for 31659 scores > > Pass 2/2: Compressing scores > ..................................................[5000000] > ..................................................[10000000] > ..................................................[15000000] > ..................................................[20000000] > ..................................................[25000000] > ..................................................[30000000] > ..................................................[35000000] > ..................................................[40000000] > ..................................................[45000000] > ..................................................[50000000] > ..................................................[55000000] > ..................................................[60000000] > ..................................................[65000000] > ..................................................[70000000] > ..................................................[75000000] > .................... > > Saving to > /home/mike/stelae-projects/de-en/phrasemodel/tmp.15419/lex-ro.minlexr > Done > Executing: > /home/mike/stelae5/mosesdecoder/scripts/generic/../../bin/addLexROtoPT > /home/mike/stelae-projects/de-en/phrasemodel/tmp.15419/pt.gz > /home/mike/stelae-projects/de-en/phrasemodel/tmp.15419/lex-ro.minlexr | > gzip -c > > /home/mike/stelae-projects/de-en/phrasemodel/tmp.15419/pt.withLexRO.gz > Executing: ln -s pt.withLexRO.gz > /home/mike/stelae-projects/de-en/phrasemodel/tmp.15419/pt.txt.gz > Executing: > /home/mike/stelae5/mosesdecoder/scripts/generic/../../bin/CreateProbingPT2 > --num-scores 4 --log-prob --input-pt > /home/mike/stelae-projects/de-en/phrasemodel/tmp.15419/pt.txt.gz > --output-dir /home/mike/stelae-projects/de-en/phrasemodel/PT2 > --num-lex-scores 6 > Starting... > Reading phrase table finished, writing remaining files to disk. > terminate called after throwing an instance of 'util::ProbingSizeException' > what(): ./util/probing_hash_table.hh:150 in > util::ProbingHashTable<EntryT, HashT, EqualT, ModT>::Entry* > util::ProbingHashTable<EntryT, HashT, EqualT, ModT>::Insert(const T&) [with > T = probingpt::Entry; EntryT = probingpt::Entry; HashT = boost::hash<long > unsigned int>; EqualT = std::equal_to<long unsigned int>; ModT = > util::DivMod; util::ProbingHashTable<EntryT, HashT, EqualT, > ModT>::MutableIterator = probingpt::Entry*; util::ProbingHashTable<EntryT, > HashT, EqualT, ModT>::Entry = probingpt::Entry] threw ProbingSizeException > because `++entries_ >= buckets_'. > Hash table with 1 buckets is full. > > _______________________________________________ > Moses-support mailing list > Moses-support@mit.edu > http://mailman.mit.edu/mailman/listinfo/moses-support > _______________________________________________ Moses-support mailing list Moses-support@mit.edu http://mailman.mit.edu/mailman/listinfo/moses-support
