Re: [Moses-support] pruning phrase tables that have more than one factor

2013-07-26 Thread Česlav Przywara
Hi Andrew, you should have no problems using filter-pt to prune phrase tabels with multiple factors. The program prints quite a lot of status information during its run, maybe try to check the log? Or post it to the list for us to investigate :-) Cheers, Ceslav on 26.7.2013 19:41 Andrew

Re: [Moses-support] pruning phrase tables that have more than one factor

2013-07-24 Thread Česlav Przywara
Hi Andrew, filter-pt prunes phrase tables with multiple factors just fine. Cheers, Ceslav on 23.7.2013 22:44 Andrew Vine said the following: Hi, I would like to prune some phrase tables following the method described here.. http://www.statmt.org/moses/?n=Moses.AdvancedFeatures#ntoc19 Could

Re: [Moses-support] Question

2013-04-17 Thread Česlav Przywara
Hi Benyamin, to set enviroment variable IRSTLM, you have to call export IRSLTLM=/home/benyamin/irstlm before any other command that requires this variable to be set. Try to run this sequence instead of your single command: export IRSLTLM=/home/benyamin/irstlm irstlm/bin/add-start-end.sh

[Moses-support] Kneser-Ney smoothing bug

2013-03-06 Thread Česlav Przywara
Hello, I'm reusing code for smoothing from the consolidate tool in one of my projects and I've noticed that there might be a small bug in Kneser-Ney discounting implementation: if (kneserNeyFlag) { ... if (countEF 2) D = kneserNey_D1; if (countEF 3) D = kneserNey_D2; ... } Shouldn't

Re: [Moses-support] relent-filter

2013-03-04 Thread Česlav Przywara
willing to share the results of your experimentation with the relent-filter. On 01/03/2013 13:30, Česlav Przywara wrote: Hi all, I'd like to ask, if anyone have ever tried out the tool for pruning of phrase table based on Relative Entropy (contrib/relent-filter)? It is also mentioned

Re: [Moses-support] relent-filter

2013-03-04 Thread Česlav Przywara
to use the RELEASE-0.91, or whatever you version you want On 4 March 2013 09:11, Česlav Przywara ces...@przywara.cz mailto:ces...@przywara.cz wrote: Hi Hieu, If you'll make relent-filter work, I'll be happy to share my results. Cheers, Ceslav on 3.3.2013 12:50 Hieu Hoang

[Moses-support] relent-filter

2013-03-01 Thread Česlav Przywara
Hi all, I'd like to ask, if anyone have ever tried out the tool for pruning of phrase table based on Relative Entropy (contrib/relent-filter)? It is also mentioned on Advanced Features page: http://www.statmt.org/moses/?n=Moses.AdvancedFeatures#ntoc23 I wanted to give a try, but didn't manage

Re: [Moses-support] Phrase table pruning still supported?

2012-07-22 Thread Česlav Przywara
Marcin, did you manage to make sigfilter working? Cheers, Ceslav on 25/06/12 19:37 Marcin Junczys-Dowmunt said the following: Thanks, I will take a look. W dniu 25.06.2012 16:26, Kenneth Heafield pisze: I uploaded my Frankenstein version: https://github.com/kpu/SALM . Compile with bjam,

[Moses-support] Minor inconsistency in lexical scores evaluation

2012-07-12 Thread Česlav Przywara
Hi All, I've discovered that in some cases, when there are multiple alignments with the most frequent count occuring for a certain phrase pair, the scorer pick a distinct alignments as the best ones when producing direct and inverse halves of the phrase table. As a result, for such phrase

Re: [Moses-support] new PhraseExtractor -- segmentation faul; the old one run smooth

2012-05-30 Thread Česlav Przywara
Hi Hieu, if I understand the new parallelization scheme of phrase table *scoring* correctly, then: a) --parallel switches on concurrent processing of direct and inverse phrase table halves b) --cores N switches on (further) parallelization of each phrase table half processing (splitting it

Re: [Moses-support] Error encountered in make World (srilm installation)

2011-09-09 Thread Česlav Przywara
Taylor, are you running 64-bit Ubuntu? Cause I've recently compiled SRILM on my Ubuntu and probably came across the very same problem. (I don't remember the filename, but stubs.h sounds familiar to me.) Anyway, I think your problem might be caused by SRILM being compiled in 32-bit mode

Re: [Moses-support] Phrase table scoring runs too long

2011-07-16 Thread Česlav Przywara
:35, Česlav Przywara wrote: Hello, I'm building a translation model using phrase-extract toolset and I have troubles with phrase scorer. Scoring of extract file consumed almost 43 cpu hours so far and is still running, but there has been no increment of the output file for last 16 hours. May

[Moses-support] Phrase table scoring runs too long

2011-07-15 Thread Česlav Przywara
Hello, I'm building a translation model using phrase-extract toolset and I have troubles with phrase scorer. Scoring of extract file consumed almost 43 cpu hours so far and is still running, but there has been no increment of the output file for last 16 hours. May there be some inconsistency

Re: [Moses-support] Missing header dependency

2011-03-09 Thread Česlav Przywara
Hi, I came across the same problem (using gcc 4.4.5) when compiling sigtest-filter. #include cstring in sigtest-filter/filter-pt.cpp solved the issue, but it would be nice to have it fixed in repository. Regards, Ceslav on 08/03/11 16:46 Hieu Hoang said the following: ah sorry, added