Re: [Moses-support] mert segfault with multi-thread

2012-06-25 Thread Marcin Junczys-Dowmunt
Hi Tom, Yes, checked a few minutes ago with the latest changes. Error still occurring. I have prepared a tarball with the necessary files (*.dat *.opt merge.init) and put it here: http://junczys.ayz.pl/mertsegfault.tgz The error occurs with the following command: mert -d 14 --scconfig case:true

Re: [Moses-support] Questions with (1) LM not found and (2) decoder dying, both during tuning

2012-06-25 Thread Philipp Koehn
Hi, the lm switch for train-model.perl consists of a 4-tuple for each language model of the form: < factor, order, filename, type > If you do not use factored translation models, then factor is 0. If you use IRSTLM, then type is 0, if you use SRILM it is 1, for KenLM it is 8. In your example > l

Re: [Moses-support] combining a module supporting reordering

2012-06-25 Thread Philipp Koehn
Hi, there is a whole line of research that does rule-based reordering based on syntactic analysis before passing on reordered input to the statistical machine translation pipeline. Take a look at this paper and the follow-up work: http://scholar.google.com/scholar?cluster=7772004091347723084&hl=e

Re: [Moses-support] mert segfault with multi-thread

2012-06-25 Thread Tom Hoar
Marcin, There have been some very recent changes to mert. Are you using an update after the changes below? Tom Original Message Subject: Re: [Moses-support] mert failure Date: Mon, 25 Jun 2012 09:52:12 -0400 From: Hieu Hoang To:moses-support@mit.edu t

Re: [Moses-support] Different between Moses Server and daemon.pl

2012-06-25 Thread Philipp Koehn
Hi, daemon.pl is a Perl wrapper around Moses to allow it to stay persistent in memory. It is part of the web-based Moses. This is what you would use when you are using the web-based Moses package: http://www.statmt.org/moses/?n=Moses.WebTranslation The server mode of Moses is doing essentially th

Re: [Moses-support] changes to the training scripts

2012-06-25 Thread Philipp Koehn
Hi, thanks for pointing this out - I changed it. -phi On Sun, Jun 24, 2012 at 9:08 PM, Tom Hoar wrote: >  Other recent changes to the scripts have enhanced tokenizer.perl to >  handle non-printing ASCII and other control characters. A few weeks ago, >  the section #escape special chars (~ line

[Moses-support] Questions with (1) LM not found and (2) decoder dying, both during tuning

2012-06-25 Thread Daniel.T.Seita
Hi, I am trying to tune a dataset according to how the manual does it, but I am having problems. First, I got this error: http://comments.gmane.org/gmane.comp.nlp.moses.user/6239 I tried to modify the ini file to set it to be 1 (because I'm using IRSTLM and I have successfully tuned the ba

Re: [Moses-support] mert segfault with multi-thread

2012-06-25 Thread Marcin Junczys-Dowmunt
Yes, I do. W dniu 25.06.2012 20:11, Christophe Servan pisze: > Hi Marcin, > Do you use the script mert-moses-multi.pl when you use the option > "-sctype MERGE" ? > > Cheers, > > Christophe ___ Moses-support mailing list Moses-support@mit.edu http://mailm

Re: [Moses-support] mert segfault with multi-thread

2012-06-25 Thread Christophe Servan
Hi Marcin, Do you use the script mert-moses-multi.pl when you use the option "-sctype MERGE" ? Cheers, Christophe Le 25/06/2012 19:40, Marcin Junczys-Dowmunt a écrit : > Hi, > the latest mert segfaults for more than one thread, but works nicely > with one thread. Also it seems that the option

[Moses-support] mert segfault with multi-thread

2012-06-25 Thread Marcin Junczys-Dowmunt
Hi, the latest mert segfaults for more than one thread, but works nicely with one thread. Also it seems that the option "-sctype MERGE" is somehow connected with this effect. Without this option there is no segfault with any number of threads. Is this reproducible for anyone? Best, Marcin _

Re: [Moses-support] Phrase table pruning still supported?

2012-06-25 Thread Marcin Junczys-Dowmunt
Thanks, I will take a look. W dniu 25.06.2012 16:26, Kenneth Heafield pisze: > I uploaded my Frankenstein version: https://github.com/kpu/SALM . > Compile with bjam, now uses Boost for hash tables. Tested with gcc > 4.6.1. But the Moses-side build needs to be modified to link against > libsalm.s

[Moses-support] Different between Moses Server and daemon.pl

2012-06-25 Thread Fong Po Po
Dear all: What different between Moses Server and daemon.pl? Both of them can make Moses do translation for remote users. Can you help me? If I develop web-based system of Moses, do you suggest me to use Moses Server or daemon.pl? I know that daemon.pl can be used on Moses (Traditional M

Re: [Moses-support] Phrase table pruning still supported?

2012-06-25 Thread Kenneth Heafield
I uploaded my Frankenstein version: https://github.com/kpu/SALM . Compile with bjam, now uses Boost for hash tables. Tested with gcc 4.6.1. But the Moses-side build needs to be modified to link against libsalm.so instead of directly referencing object files. Kenneth On 06/25/2012 10:48 AM, M

Re: [Moses-support] mert failure

2012-06-25 Thread Hieu Hoang
thanks. gcc version-specific error. Fixed https://github.com/moses-smt/mosesdecoder/commit/8498b17a41e403020897b491fa63a2c2845ab088 looking at the other issues you mentioned. Will respond later On 25/06/2012 05:01, Tom Hoar wrote: Hieu, I just updated my source and mert fails to build. M

Re: [Moses-support] Phrase table pruning still supported?

2012-06-25 Thread Barry Haddow
Hi As far as I know, the filter code only needs samt for suffix arrays. But since Moses also includes a suffix array implementation, it should be possible to remove the samt dependency and make building the filter code much easier... cheers - Barry On Monday 25 June 2012 14:16:23 Kenneth Heafi

Re: [Moses-support] Phrase table pruning still supported?

2012-06-25 Thread Marcin Junczys-Dowmunt
Ah, I just see that SALM is not compiling anyway, at least not with a recent gcc version, some headers need to be added. SALM as such seems abandoned? Pruning is such a nice feature. Could the Moses suffix array implementation be used instead in the future? 25/6/2012, "Marcin Junczys-Dowmunt" na

Re: [Moses-support] Phrase table pruning still supported?

2012-06-25 Thread Marcin Junczys-Dowmunt
Thanks you! looked for "filter-pt", but not for "filter-pt.cpp" 25/6/2012, "Kenneth Heafield" napisał/a: >Hi, > > Looks like it was moved to contrib/sigtest-filter . Has its own >Makefile (not called by the main build system) and depends on SALM. If >anybody wants an exercise in writing

Re: [Moses-support] Phrase table pruning still supported?

2012-06-25 Thread Kenneth Heafield
Hi, Looks like it was moved to contrib/sigtest-filter . Has its own Makefile (not called by the main build system) and depends on SALM. If anybody wants an exercise in writing Jam, this is something to try. Kenneth On 06/25/2012 10:04 AM, Marcin Junczys-Dowmunt wrote: > > Hi all, > i

[Moses-support] Phrase table pruning still supported?

2012-06-25 Thread Marcin Junczys-Dowmunt
Hi all, is filter-pt for phrase table pruning still part of Moses? It is mentioned on the web page, but I do not see it anywhere after building. Best, Marcin ___ Moses-support mailing list Moses-support@mit.edu http://mailman.mit.edu/mailman/listinfo/m

Re: [Moses-support] changes to the training scripts

2012-06-25 Thread Tom Hoar
Hieu, The bjam command line currently supports various independent paths: --prefix=/path/to/prefix sets the install prefix [default is source root]. --bindir=/path/to/prefix/bin sets the bin directory [PREFIX/bin] --libdir=/path/to/prefix/lib sets the lib directory [PREFIX/lib] --includedi

[Moses-support] mert failure

2012-06-25 Thread Tom Hoar
Hieu, I just updated my source and mert fails to build. My tarball backup of the previous github revision on 21 June compiles fine. You checked-in some changes to github a couple days ago regarding mert. Here's my command line: /home/tahoar/domy-2.5/src/mosesdecoder/bjam -j2 -a --debug-