Your report of 100% physical usage, growing swap usage and low CPU load is normal when working with limited RAM machines. With only 4 Gb Ram and the new (larger) EuroParl v6 corpus, you could train for 3 or 4 days depending on how you setup your swap partition. Even then, it's possible you will run out of RAM before it's finished. Upgrading to 8 Gb ram is a move in the right direction.
Once it's finished training, you'll want to use the binarized the tables and language model, which MMM's train-1.11 script creates. Tom On Mon, 18 Apr 2011 14:52:10 +0100, Philipp Koehn <pko...@inf.ed.ac.uk> wrote: > Hi, > > I am not familiar with the MMM setup, but one of the causes > of memory use may be the translation table. You should use > the on-disk translation table. > > -phi > > On Mon, Apr 18, 2011 at 2:47 PM, David Wilkinson > <davidzw...@hotmail.com> wrote: >> I have set up an Ubuntu 10.04 system with the moses-for-mere-mortals >> scripts. The default corpus trained in about 6-7 hours on my system >> (Athlon >> x3 3.2Ghz, 4Gb Ram). I am now trying to train the system with the >> Europarl >> German-English parallel corpus (about 45m words in each language), >> again >> using the default moses-for-mere-mortals settings. The system has >> been >> running for 24 hrs and is currently using all the physical memory >> and about >> 1.2Gb of swap. None of the cores are being used more than 10%, so >> like this >> it will take a very long time to finish. If I double the ram to 8gb, >> will >> this be sufficient? >> Many Thanks >> David >> _______________________________________________ >> Moses-support mailing list >> Moses-support@mit.edu >> http://mailman.mit.edu/mailman/listinfo/moses-support >> >> > _______________________________________________ > Moses-support mailing list > Moses-support@mit.edu > http://mailman.mit.edu/mailman/listinfo/moses-support _______________________________________________ Moses-support mailing list Moses-support@mit.edu http://mailman.mit.edu/mailman/listinfo/moses-support
