[Moses-support] compact phrase table: missing alignments

2012-11-16 Thread Rico Sennrich
Hi again, my last problems with the compact tables was a configuration problem, but I think this one isn't. I'm working on stuff that relies on word alignment during decoding, but for some phrase pairs, targetPhrase-GetAlignTerm() is missing alignment pairs in compact models (everything is dandy

Re: [Moses-support] compact phrase table: missing alignments

2012-11-16 Thread Rico Sennrich
On Fri, 2012-11-16 at 15:08 +, Hieu Hoang wrote: can you put the sample data somewhere public like dropbox or googgle drive. i'll take a look at it as well the table is on https://docs.google.com/open?id=0B5nFI-wykGgneDFVNFFfNVZFYzQ the command I used to build the compact table:

Re: [Moses-support] compact phrase table: missing alignments

2012-11-16 Thread Marcin Junczys-Dowmunt
Hi, this might actually be a consequence of the compression method I use. You can also check and recompact the phrase table with the following option -encoding None It will be larger by some 20-30%, but it should not be missing alignment points. If it still does then this is an implementation

Re: [Moses-support] compact phrase table: missing alignments

2012-11-16 Thread Marcin Junczys-Dowmunt
OK, confirmed. The alignment points do not vanish with -encoding None. But this still seems to be a bug, the more efficient compression should not lose that many alignment points, this should only happen very rarely if at all. W dniu 16.11.2012 16:27, Marcin Junczys-Dowmunt pisze: Hi, this

Re: [Moses-support] compact phrase table: missing alignments

2012-11-16 Thread Marcin Junczys-Dowmunt
OK. Fixed. Hieu messed up the alignment :P But in his defense, I was looking directly at it and did not see it for some time :) W dniu 16.11.2012 16:39, Marcin Junczys-Dowmunt pisze: OK, confirmed. The alignment points do not vanish with -encoding None. But this still seems to be a bug, the