OK, confirmed. The alignment points do not vanish with "-encoding None".
But this still seems to be a bug, the more efficient compression should not lose that many alignment points, this should only happen very rarely if at all. W dniu 16.11.2012 16:27, Marcin Junczys-Dowmunt pisze: > Hi, > this might actually be a consequence of the compression method I use. > You can also check and recompact the phrase table with the following option > > -encoding None > > It will be larger by some 20-30%, but it should not be missing alignment > points. If it still does then this is an implementation mistake I have > to fix. > > I will check your data now. > > W dniu 16.11.2012 16:22, Rico Sennrich pisze: >> On Fri, 2012-11-16 at 15:08 +0000, Hieu Hoang wrote: >>> can you put the sample data somewhere public like dropbox or googgle >>> drive. >>> >>> i'll take a look at it as well >>> >> the table is on >> https://docs.google.com/open?id=0B5nFI-wykGgneDFVNFFfNVZFYzQ >> >> the command I used to build the compact table: >> >> ~/mosesdecoder/bin/processPhraseTableMin -in sample_data.gz -out >> sample_data -alignment-info -rankscore 0 -nscores 3 >> >> >> To reproduce the problem, add a check to assert that alignment.size() > >> 0 in PhraseTableCreator::EncodeAlignment() . >> >> _______________________________________________ >> Moses-support mailing list >> Moses-support@mit.edu >> http://mailman.mit.edu/mailman/listinfo/moses-support > _______________________________________________ > Moses-support mailing list > Moses-support@mit.edu > http://mailman.mit.edu/mailman/listinfo/moses-support _______________________________________________ Moses-support mailing list Moses-support@mit.edu http://mailman.mit.edu/mailman/listinfo/moses-support
