You may want to try changing the persistent cache options in Moses. By default Moses caches translation tables used for each sentence (to reduce disk IO and thus speed decoding), but if you have lots of translation/generation steps and big and/or complicated translation tables I suppose this could use up memory quickly.
http://www.statmt.org/moses/?n=Moses.DecoderParameters On Wed, Oct 14, 2009 at 11:23 AM, Mark Fishel <[email protected]> wrote: > Dear Support, > > when translating with a simple factored translation model, moses dies > without memory after having translated at most 20 sentences. In the > process the memory usage of the decoder keeps growing (although > sentences should be treated independently, which indicates a memory > leak in moses). The phrase and reordering tables are all binarized. > The precise error message is: > > terminate called after throwing an instance of 'std::bad_alloc' > what(): St9bad_alloc > > The model is trained with the following command-line: > > nohup nice train-factored-phrase-model.perl --parallel -root-dir ./ > --corpus train --f et --e en --lm > 0:3:/home/harts/smt/results/hsmt/exp_pos7g_ee/lmWord3g. > en.gz --lm 1:7:/home/harts/smt/results/hsmt/exp_pos7g_ee/lmPos7g.en.gz > -reordering distance,msd-bidirectional-fe --reordering-factors 0-0 > --translation-fa > ctors 0-0 --generation-factors 0-1 --decoding-steps t0,g0 &> log.tm.txt & > > The funny part is that a factor-less baseline translates 3000 > sentences with no problems. > > Is it a memory leak? Why is there a difference between factor-less and > factored translation? Will be thankful for any help or advice, > > Best regards, > > Mark Fishel > Dept. of Computer Science > University of Tartu > _______________________________________________ > Moses-support mailing list > [email protected] > http://mailman.mit.edu/mailman/listinfo/moses-support > _______________________________________________ Moses-support mailing list [email protected] http://mailman.mit.edu/mailman/listinfo/moses-support
