Re: [Moses-support] moses-chart binary missing?

2014-11-26 Thread Eric Baucom
that works, thanks! On Wed, Nov 26, 2014 at 4:20 PM, Hieu Hoang wrote: > There should be a softlink which points the file moses. Moses and > Moses_chart are being merged. Everything in the tutorial should still work > On 26 Nov 2014 21:16, "Eric Baucom" wrote: > >> I am interested in experiment

[Moses-support] Moses for Mere Mortals: Update

2014-11-26 Thread João Luis Rosas
Hi, There was recently a message about problems compiling Moses for Mere Mortals in Ubuntu 14.04 (http://thread.gmane.org/gmane.comp.nlp.moses.user/11390/focus=11406). We therefore have decided to review the whole package, added some slight changes to it (in the end, a good deal of them), made it

Re: [Moses-support] Problem at a training stage - linguistic model not correctly trained

2014-11-26 Thread João Luis A. d. C. Rosas
Radian Yazynin writes: > > > Dear Support staff members, > I will appreciate if you help me with one issue. > These are my first steps... > I am trying out a Moses-For-Mere-Mortals package (Moses + IRSTLM + RandLM +  MGIZA) > in Ubuntu 14.04.1 64bit. > Compilation resulted in one error only: M

Re: [Moses-support] METEOR: difference between ranking task and other tasks

2014-11-26 Thread Marcin Junczys-Dowmunt
Thanks, that's a very useful answer. I figured something similar, but I was curious how come these huge differences between the methods are never reported anywhere. Even in your paper they are just a few percent. Also, could it be that the default METEOR setting is slighlty overfitting to the

Re: [Moses-support] METEOR: difference between ranking task and other tasks

2014-11-26 Thread Michael Denkowski
Hi Marcin, Meteor scores can vary widely across tasks due to the training data and goal. The default ranking task tries to replicate WMT rankings, so the absolute scores are not as important as the relative scores between systems. The adequacy task tries to fit Meteor scores to numeric adequacy

Re: [Moses-support] moses-chart binary missing?

2014-11-26 Thread Hieu Hoang
There should be a softlink which points the file moses. Moses and Moses_chart are being merged. Everything in the tutorial should still work On 26 Nov 2014 21:16, "Eric Baucom" wrote: > I am interested in experimenting with tree-to-tree translations, so I > recently installed Moses according to t

[Moses-support] moses-chart binary missing?

2014-11-26 Thread Eric Baucom
I am interested in experimenting with tree-to-tree translations, so I recently installed Moses according to the guidelines here: http://www.statmt.org/moses/?n=Development.GetStarted . The installation completed successfully, and I am able to successfully translate using the sample models as descr

Re: [Moses-support] Unknown single words that are part of phrases

2014-11-26 Thread Matthias Huck
Hi, Supposedly your phrase table does not contain an entry "Gitarre ||| guitar" because this word pair is always unaligned in your training data. You could try to improve your word alignment quality. Alternatively, you could implement a procedure in the manner of the "forced single word heuristic

[Moses-support] CFP: NAACL HLT 2015 final call for papers

2014-11-26 Thread Saif Mohammad
[Apologies for the cross postings.] NAACL HLT 2015 FINAL CALL FOR PAPERS === ### The 2015 Conference of the North American Chapter of the Association for Computational Linguistics - Human Language Technologies (NAACL HLT 2015) ### ### May 31 to June 5, 2015. ### #

Re: [Moses-support] Delvin et al 2014

2014-11-26 Thread Alexandra Birch
OK, Here is 1-4: 1. You would normally train bilingual lm on the same corpus as the SMT model, but it is not required 2. Yes, but there are also other ways to make training faster which you might want to explore 3. Yes it is important that the bilingual lm corpus matches the format that will be p

Re: [Moses-support] Unknown single words that are part of phrases

2014-11-26 Thread Raj Dabre
Hello, If I am not wrong this is most likely due to the grow (-diag) method applied to the word aligned data (both directions) before phrase extraction. Furthermore. one word translations should exist (but not always) search for them. Regards. On Thu, Nov 27, 2014 at 12:53 AM, Vera Aleks

Re: [Moses-support] Delvin et al 2014

2014-11-26 Thread Nikolay Bogoychev
Hey, I can only answer 5-7 5. The alignment file is the one that's usually called aligned.1.grow-diag-final-and and contains lines such as: 0-0 1-1 2-2 3-3 0-0 1-1 2-2 3-3 4-4 6. Yes. Basically prune vocab value of 16000 would take the 16000 most common words in the corpus and discard the rest (

[Moses-support] Unknown single words that are part of phrases

2014-11-26 Thread Vera Aleksic, Linguatec GmbH
Hi, I have observed many times that some words do not exist as single word translations in the phrase table, although they exist in the training corpus and in multiword phrases. An example: German-English translation for "Gitarre" is unknown, i.e. there is no single word entry for "Gitarre"

Re: [Moses-support] Delvin et al 2014

2014-11-26 Thread Tom Hoar
Thanks again. It's very useful feedback. We're now preparing to move from v1.0 to 3.x. We skipped Moses 2.x. So, I'm not familiar with the new moses.ini syntax. Here are some more questions to help us get started playing with the extract_training.py options: 1. I'm assuming corpus.e and corp

[Moses-support] METEOR: difference between ranking task and other tasks

2014-11-26 Thread Marcin Junczys-Dowmunt
Hi, A question concerning METEOR, maybe someone has some experience. I am seeing huge differences between values for English with the defauly task "ranking" and any other of the tasks (e.g. "adq"). up to 30-40 points. Is this normal? In the literature I only ever see marginal differences of ma

Re: [Moses-support] Delvin et al 2014

2014-11-26 Thread Nikolay Bogoychev
Hey, Tom 1) It's independent. You just add -with-oxlm and -with-nplm to the stack 2) Yes, they are both thread safe, you can run the decoder with however many threads you wish. 3) It doesn't create a separate binary. The compilation flag adds a new feature inside moses that is called BilingualNPLM

Re: [Moses-support] Delvin et al 2014

2014-11-26 Thread Tom Hoar
Thanks Nikolay! This is a great start. I have a few clarification questions. 1) does this replace or run independently of traditional language models like KenLM? I.e. when compiling, we can use -with-kenlm, -with-irstlm, -with-randlm and -with-srilm together. Are -with-oxlm and -with-nplm add

Re: [Moses-support] Delvin et al 2014

2014-11-26 Thread Nikolay Bogoychev
Fix formatting... Hey, BilingualLM is implemented and as of last week resides within moses master: https://github.com/moses-smt/mosesdecoder/blob/master/moses/LM/BilingualLM.cpp To compile it you need a NeuralNetwork backend for it. Currently there are two supported: Oxlm and Nplm. Adding a new

Re: [Moses-support] Delvin et al 2014

2014-11-26 Thread Nikolay Bogoychev
Hey, BilingualLM is implemented and as of last week resides within moses master: https://github.com/moses-smt/mosesdecoder/blob/master/moses/LM/BilingualLM.cpp To compile it you need a NeuralNetwork backend for it. Currently there are two supported: Oxlm and Nplm. Adding a new backend is relative

Re: [Moses-support] how to test whether tcmalloc is used?

2014-11-26 Thread Barry Haddow
How about: nm -C moses | grep tcmalloc On 26/11/14 11:34, Hieu Hoang wrote: > Best to do what Rico says, but > > If the tcmalloc library is dynamically linked to Moses, running ldd > will show it is linked into moses: > #ldd bin/moses > . > libtcmalloc_minimal.so.4 => /usr/lib/libt

[Moses-support] Delvin et al 2014

2014-11-26 Thread Tom Hoar
Hieu, Sorry I missed you in Vancouver. I just reviewed your slide deck from the MosesCore TAUS Round Table in Vancouver (taus-moses-industry-roundtable-2014-changes-in-moses-hieu-hoang-university-of-edinburgh). In particular, I'm interested in the "Bilingual Language Models" that "replicat

Re: [Moses-support] how to test whether tcmalloc is used?

2014-11-26 Thread Hieu Hoang
Best to do what Rico says, but If the tcmalloc library is dynamically linked to Moses, running ldd will show it is linked into moses: #ldd bin/moses . libtcmalloc_minimal.so.4 => /usr/lib/libtcmalloc_minimal.so.4 (0x7ff49f5a2000) ... You can force it to statically link by delet

Re: [Moses-support] Format of binarized phrase tables

2014-11-26 Thread Raj Dabre
Hello Marcin, Yes please. It would save me lots of time. Thanks. Regards. On Wed, Nov 26, 2014 at 6:50 PM, Marcin Junczys-Dowmunt wrote: > Hi, > > I have a JNI interface to my compact phrase table somewhere, I guess I can > put that in contrib within a day or two if there is interest. > > bes

Re: [Moses-support] how to test whether tcmalloc is used?

2014-11-26 Thread Rico Sennrich
Li Xiang writes: > > I compile Moses with tcmalloc. How can I test whether tcmalloc is used and evaluate the performance ? > there's probably many ways, but here's three: at compile time, you will see the following message if tcmalloc is not enabled: "Tip: install tcmalloc for faster threadi

Re: [Moses-support] Format of binarized phrase tables

2014-11-26 Thread Marcin Junczys-Dowmunt
Hi, I have a JNI interface to my compact phrase table somewhere, I guess I can put that in contrib within a day or two if there is interest. best, Marcin W dniu 2014-11-26 10:45, Barry Haddow napisał(a): > Hi Raj > > The format of these tables is not described anywhere. You'd have to

Re: [Moses-support] Format of binarized phrase tables

2014-11-26 Thread Barry Haddow
Hi Raj The format of these tables is not described anywhere. You'd have to read the code in moses/TranslationModel/PhraseDictionaryTree.cpp, and then try to convert it it Java. A better plan would be to use JNI to call the C++ code -- a similar approach has been followed in the python interfac