Miles Osborne pisze: > filtering etc might give you a speed-up (eg a constant one --less > stuff to load) but if filtering is safe w.r.t to the source data, then > you shouldn't see much here. > > (pruning the table should make it faster since there will be fewer > options to consider, but this is not safe)
Actually, this is contrary to what Johnson et al. say in their paper, and my subjective (not measured) experience was definitely in their favor. As long as you have really clean data, you don't want to lose any of it, but if alignments are lousy, translations ambiguous etc., you want to cut it off, and Jan wants to do that (see his post). I was even filtering more and got better results by heuristically discarding unprobable phrases from the phrase table (based on Fran's idea he had about discarding unprobable alignments). Again, this is subjective, anecdotal, etc., but before that I was getting complete garbage. Note: my pair was English-Polish and Polish English. > i guess you might also see fewer page faults and the like with a > smaller model and that will help matters. btw, quantising and binarising language models helps as well Marcin > but in general, the beam size is the most direct way to make it faster. > Miles > > 2009/5/4 Francis Tyers <fty...@prompsit.com>: >> El lun, 04-05-2009 a las 14:08 +0100, Miles Osborne escribió: >>> actually, i think Jan wants a speedup, not a space saving. >> Does filtering the phrase table before translation not decrease the >> total time to make a translation (including the time taken to load the >> phrase table etc.)? That was my experience, and it appears to be >> something that he hasn't done, but perhaps my set up is unusual... >> >> Fran >> >>> your best bet is to reduce the size of the beam: >>> >>> http://www.statmt.org/moses/?n=Moses.Tutorial#ntoc6 >>> >>> Miles >>> 2009/5/4 Francis Tyers <fty...@prompsit.com>: >>>> El lun, 04-05-2009 a las 14:54 +0200, Jan Helak escribió: >>>>> Hello everyone :) >>>>> >>>>> I try to build two-way translator for polish and english languages as a >>>>> project on one of my subjects. By now, I created a one-way translator >>>>> (polish->english) as a beta version, but severals problems have came: >>>>> >>>>> (1) A translator must work in two-ways. How to achieve this? >>>> Make another directory and train two models. >>>> >>>>> (2) Time of traslating for phrases is two long ( 4 min. for one >>>>> sentence). How to accelerate this (decresing a quality of translation >>>>> is acceptable). >>>> You can try filtering the phrase table before translating (see PART V - >>>> Filtering Test Data), or using a binarised phrase table (see Memory-Map >>>> LM and Phrase Table). >>>> >>>> http://ufallab2.ms.mff.cuni.cz/~bojar/teaching/NPFL087/export/HEAD/lectures/02-phrase-based-Moses-installation-tutorial.html >>>> >>>> Regards, >>>> >>>> Fran >>>> >>>> _______________________________________________ >>>> Moses-support mailing list >>>> Moses-support@mit.edu >>>> http://mailman.mit.edu/mailman/listinfo/moses-support >>>> >>> >>> >> > > > _______________________________________________ Moses-support mailing list Moses-support@mit.edu http://mailman.mit.edu/mailman/listinfo/moses-support
