Try Compact Table? http://www.statmt.org/moses/?n=Advanced.RuleTables#ntoc3
2017年4月5日 下午7:33，"RR" <rmi...@gmail.com>寫道： > Hi, > > I am currently working on a system to explore MOSES decoding parameter > space - running MOSES in decoding mode with one parameter set, getting a > BLEU score and rerunning with different parameters. > > More than 50% of my time is being spent repeatedly "loading" the phrase > table, language model and reordering model. For example, > > $ ./example.fr | .../moses -f ./run11.moses.ini -threads 31 > > ./trans_baseline > > gives: "Start loading text phrase table. Moses format : [37.296] seconds." > > 37 seconds just to load a 590MB phrase table, even if it is on RAMDISK. > htop indicates that for those 37 seconds, a single core is 100% utilized. > > My understanding is that some kind of optimized data structure (a hash > table of some kind?) is being created in those 37 seconds, which is then > lost and recreated when I re-run MOSES with a different value of a decoding > parameter. > > If I want to eliminate this loading time, what is the best way forward? Is > there a phrase table format that avoids a significant loading time (i.e. a > 500MB phrase table loads in < 1 sec)? Should I try to find a way to run a > single MOSES server but with different decoding parameters? I have a system > with 130GB RAM and I am using a small phrase table that has been filtered > for my decoding set. > > Thanks in advance, > > Roko > > _______________________________________________ > Moses-support mailing list > Moses-support@mit.edu > http://mailman.mit.edu/mailman/listinfo/moses-support > >
_______________________________________________ Moses-support mailing list Moses-support@mit.edu http://mailman.mit.edu/mailman/listinfo/moses-support
