Re: [Moses-support] Unique k-best lists in moses?

2015-03-11 Thread Philip Williams
Hi Chris, if you add the argument "distinct" to the -n-best-list option then that should have the same effect, albeit far less efficiently (internally, Moses simply generates a longer list and then filters it). I'm guessing you already knew about that option though. If that's too slow for what y

Re: [Moses-support] Moses segmentation fault under multi-thread

2015-03-11 Thread Barry Haddow
Hi Liangyou This is a bit strange. The stacktrace suggests corruption in the feature vector, but it would be surprising that we haven't seen it before. Can you reproduce the problem when running under valgrind? The memory could be getting corrupted by a completely different source, which we'll

[Moses-support] Unique k-best lists in moses?

2015-03-11 Thread Chris Dyer
Hi all, can anyone tell me if moses can be configured to generate "unique" k-best lists using the Huang et al. (2006) dynamic programming algorithm (Section 5.2 of http://www.cis.upenn.edu/~lhuang3/amta06-sdtedl.pdf)? Thanks-- Chris ___ Moses-support mail

Re: [Moses-support] In-memory loading of compact phrases

2015-03-11 Thread Jesús González Rubio
2015-03-11 19:21 GMT+00:00 Marcin Junczys-Dowmunt : > Maybe someone will correct me, but if I am not wrong, the gziped version > already calculates the future score while loading (i.e. the phrase is being > scored by the language model). The compact phrase table cannot do this > during loading an

Re: [Moses-support] In-memory loading of compact phrases

2015-03-11 Thread Marcin Junczys-Dowmunt
Maybe someone will correct me, but if I am not wrong, the gziped version already calculates the future score while loading (i.e. the phrase is being scored by the language model). The compact phrase table cannot do this during loading and doing this on-line. This will be the reason for the slow

Re: [Moses-support] In-memory loading of compact phrases

2015-03-11 Thread Marcin Junczys-Dowmunt
Hi, Try measuring the differences again after a full system reboot (fresh reboot before each mesurement) or after purging OS read/write caches. Your phrase tables are most likely cached, which means they are in fact in memory. Best, Marcin W dniu 11.03.2015 o 19:31, Jesús González Rubio pisze

[Moses-support] In-memory loading of compact phrases

2015-03-11 Thread Jesús González Rubio
Hi, I'm obtaining some unintuitive timing results when using compact phrase tables. The average translation time per sentence is much higher for them in comparison to using gzip'ed phrase tables. Particularly important is the difference in time required to collect the options. This table summarize

Re: [Moses-support] Moses segmentation fault under multi-thread

2015-03-11 Thread Liangyou Li
Thanks Matthias. This problem doesn't always happen. Even on the same data, sometimes it can run without crash, sometimes it doesn't. But, by now, the error only happens when the decoder outputs features. (-n-best-list or -output-search-graph-hypergraph) Cheers Liangyou On Wed, Mar 11, 2015 at

Re: [Moses-support] Moses segmentation fault under multi-thread

2015-03-11 Thread Matthias Huck
Hi, I've recently been using these sparse feature functions without any issues in multi-threaded chart-based decoding. There might be a problem with thread safety, but I currently can't tell why you got the segmentation fault. You should investigate this in more detail. Cheers, Matthias On Wed

Re: [Moses-support] Moses V3 release as tarball?

2015-03-11 Thread Lane Schwartz
Thanks! On Tue, Mar 10, 2015 at 5:15 PM, Hieu Hoang wrote: > done >http://www.statmt.org/moses/RELEASE-3.0/binaries/*/*.tgz > > Hieu Hoang > Research Associate (until March 2015) > University of Edinburgh > http://www.hoang.co.uk/hieu > > On 10 March 2015 at 21:48, Lane Schwartz wrote: >> >>

[Moses-support] Recasing on EMS

2015-03-11 Thread burak aydın
Hi, What should be changed in 'config.toy' to open recasing instead of truecasing while using EMS? Regards, Burak ___ Moses-support mailing list Moses-support@mit.edu http://mailman.mit.edu/mailman/listinfo/moses-support

[Moses-support] Moses segmentation fault under multi-thread

2015-03-11 Thread Liangyou Li
When I run moses with sparse features in multi-threads ( parameter -threads all), I got segmentation fault. The three sparse features are: *SourceWordDeletionFeature factor=0* *TargetWordInsertionFeature factor=0* *WordTranslationFeature input-factor=0 output-factor=0* The exact comma

Re: [Moses-support] Error while compiling Moses v3.0 on Cygwin

2015-03-11 Thread Muhammad Danial Raza
Thank you Hieu, the second point did the trick for me. On Tue, Mar 10, 2015 at 8:43 PM, Hieu Hoang wrote: > there seems to be a problem with the unit tests, similar to >https://www.mail-archive.com/moses-support%40mit.edu/msg11603.html > Either >1. Ignore it, if the executables compiled