How much RAM is used is dependent on the phrase-table & reordering model size, not the test set size.
Since it blows up at during loading of the phrase-table, I would think it ran out of ram. I would run the script again and use top to monitor carefully how much RAM is being used just before it crashes. The only other thing I can think of is there's some gunk in the phrase-table, but that's unlikely if you are using the downloaded phrase-table as the data has been cleaned with the usual scripts. * Looking for MT/NLP opportunities * Hieu Hoang http://moses-smt.org/ On 17 May 2017 at 12:25, Per Starbäck <starb...@stp.lingfil.uu.se> wrote: > Hieu Hoang writes: > > > thanks. It may not be the phrase-based/hiero mixup as I had suggested. > > > > Maybe the system ran out of RAM. > > I don't think so. It doesn't grow much at all when running, and this is > a rather small test just done to show me the problem. > > I think now I should have included not only the backtrace from moses but > also the end of its output before crashing: > ====================================================================== > [...] > Initializing LexicalReordering.. > line=Distortion > FeatureFunction: Distortion0 start: 13 end: 13 > line=KENLM lazyken=0 name=LM0 factor=0 > path=.../lm/news-commentrary-v8.de-en.blm.en > order=3 > FeatureFunction: LM0 start: 14 end: 14 > Loading UnknownWordPenalty0 > Loading WordPenalty0 > Loading PhrasePenalty0 > Loading LexicalReordering0 > Loading table into memory...done. > Loading Distortion0 > Loading LM0 > Loading TranslationModel0 > Start loading text phrase table. Moses format : [6.706] seconds > Reading .../mert-work/filtered/phrase-table.0-0.1.1.gz > ----5---10---15---20---25---30---35---40---45---50---55--- > 60---65---70---75---80---85---90---95--100 > ====================================================================== > > > Before you run tuning, you should also binarize the phrase-table and > > lexicalised reordering model, ... > > She followed the tutorial at > http://www.statmt.org/moses/?n=Moses.Baseline (but with much less > training data) when preparing the non-working example to show me. > If doing that results in a segfault there must be something wrong > somewhere. Isn't the backtrace useful? >
_______________________________________________ Moses-support mailing list Moses-support@mit.edu http://mailman.mit.edu/mailman/listinfo/moses-support
