Re: [Moses-support] CreateProbingPT2 exception

2017-05-02 Thread Nikolay Bogoychev
Hey Mike, Is it possible for you to make the phrase table available /home/mike/stelae-projects/de-en/phrasemodel/tmp.15419/pt.txt.gz publically so we can try to reproduce the problem? Cheers, Nick On Tue, May 2, 2017 at 6:06 AM, Mike Ladwig wrote: > Got an exception creating a PT2 with yesterd

Re: [Moses-support] Rebuilding moses binary only

2017-03-30 Thread Nikolay Bogoychev
I've been asking this same question since late 2013..? On Thu, Mar 30, 2017 at 10:30 PM, Marcin Junczys-Dowmunt wrote: > Hi list, > > is there a way to tell bjam to only rebuild the moses binary and not the > 84 unrelated targets that just happen to be rebuilt out of solidarity? > > Thanks, > > M

Re: [Moses-support] BilingualNPLM: A target phrase with no alignments detected!

2016-03-19 Thread Nikolay Bogoychev
Hey Jeremy, The error you should get should be: "A target phrase with no alignments detected! " << targetPhrase << "Check if there is something wrong with your phrase table."); which should also include the targetPhrase in question. My guess is that you use PhraseDictionaryCompact as your phrase

Re: [Moses-support] Blingual neural lm, log-likelihood: -nan

2015-09-21 Thread Nikolay Bogoychev
Hey Jian, I have encountered this problem with nplm myself and couldn't really find a solution that works every time. Basically what happens is that there is a token that occurs very frequently on the same position and it's weights become huge and eventually not a number which propagates to the r

Re: [Moses-support] nplm ngram total order in ems

2015-08-01 Thread Nikolay Bogoychev
Hey John, This is correct. So imagine the situation of order 5 and source window 4: is aligned to and your source window is 4: 4 tokens before and after s0, which results in a 14gram in total. Cheers, Nick On Sat, Aug 1, 2015 at 4:30 PM, John Joseph Morgan < johnjosephmor...@gmail.com> wrot

Re: [Moses-support] Parallelizer multi core

2015-07-31 Thread Nikolay Bogoychev
Hey, I have opposed this change in the past for two reasons: Using more than 4 threads doesn't help unless the user is using PhraseDictionaryCompact. See this issue https://github.com/moses-smt/mosesdecoder/issues/39 in fact on most machines you rarely want to run moses on all available threads.

Re: [Moses-support] nplm / Bilingual LM

2015-07-03 Thread Nikolay Bogoychev
Hey Marwa, I can't reproduce the problem. Using latest moses git and nplm from https://github.com/rsennrich/nplm it compiles just file and I get both BilingualNPLM and NeuralLM FFs I can suggest that you do a ./bjam clean and try recompiling again. Cheers, Nick On Fri, Jul 3, 2015 at 12:19 PM,

Re: [Moses-support] ProbingPT creation and factor support

2015-04-22 Thread Nikolay Bogoychev
Hey, Probingpt supports reading from gzipped files but I don't think it supports factors. I didn't code for factors specifically anyways. I am not sure if the factor support has to be built into the phrase table or is independent of it and is part of moses. Cheers, Nick On 22 Apr 2015 6:03 pm, "

Re: [Moses-support] bilingual LM (nan nan nan)

2015-04-21 Thread Nikolay Bogoychev
Hey Marwa, We have been having this problem with NPLM and we have found no "real solution". There were couple of threads on the mailing list with this problem so far. Basically the solution that we use is to lower the learning rate (from 1 to .5. If .5 doesn't work to .25 and so on) and increase th

Re: [Moses-support] ProbingPT tests not building

2015-04-01 Thread Nikolay Bogoychev
Done. Thanks for spotting it. On Wed, Apr 1, 2015 at 4:49 PM, Jeroen Vermeulen < j...@precisiontranslationtools.com> wrote: > On 01/04/15 22:29, Nikolay Bogoychev wrote: > > > Those tests are indeed obsolete, I used them to test some behavior when > > I was building pr

Re: [Moses-support] ProbingPT tests not building

2015-04-01 Thread Nikolay Bogoychev
Hey Jeroen, Those tests are indeed obsolete, I used them to test some behavior when I was building probingPT but the function in question became part of the HuffmanDecoder class as getTargetWordFromID You don't need to build or worry about those tests (there isn't a Jamfile in that directory) I st

Re: [Moses-support] Unicode Issues when Using Compact Phrase Table, Binaries vs. Own Build

2015-03-30 Thread Nikolay Bogoychev
Hey Венци, Did you by any chance binarize your phrase tables from a raw text format or from gunzip (or any other supported compressed text formats)? I recently run into similar issues with my phrase table (ProbingPT) if the input phrase table had not been compressed during binary creation. I wasn

Re: [Moses-support] Forbidden link to binaries

2015-03-19 Thread Nikolay Bogoychev
Hey Per, The link seems to be outdated, as it points to RELEASE-1.0. You can find the current ones here: http://www.statmt.org/moses/RELEASE-3.0/binaries/ Cheers, Nick On Thu, Mar 19, 2015 at 2:22 PM, Per Tunedal wrote: > Hi, > I just read the page http://www.statmt.org/moses/?n=Moses.Release

Re: [Moses-support] where is "premultiply" member of class "neuralLM" ?

2015-02-08 Thread Nikolay Bogoychev
Hey, I replied to your previous query but I got mail delivery failiure notification, so I am trying again: In order to use NPLM with moses you should use this fork of NPLM: https://github.com/rsennrich/nplm Cheers, Nick On Sun, Feb 8, 2015 at 9:38 AM, Jianri Li wrote: > Hi, all > > I resend

Re: [Moses-support] Error while compile with nplm

2015-02-07 Thread Nikolay Bogoychev
Hey, In order to use NPLM with moses you should use this fork of NPLM: https://github.com/rsennrich/nplm Cheers, Nick On Sat, Feb 7, 2015 at 6:22 PM, Jianri Li wrote: > Hi, moses users > I was trying to compile with nplm, but I got some errors like this: > -- > moses/LM

Re: [Moses-support] nplm Building LM

2015-01-12 Thread Nikolay Bogoychev
Hey, Refer to the moses documentation on how to use NPLM LM during decoding: http://www.statmt.org/moses/?n=FactoredTraining.BuildingLanguageModel#ntoc31 In particular you need to add this: NeuralLM factor= order= path=filename To your moses.ini where filename is model.NUMBER. The 10 files mod

Re: [Moses-support] how to compile with nplm library

2014-12-29 Thread Nikolay Bogoychev
Hey, First you need to checkout and compile this fork of nplm: https://github.com/rsennrich/nplm Then you need to compile moses with nplm switch: ./bjam --with-nplm=path/to/nplm Then you can see how to use it here http://www.statmt.org/moses/?n=FactoredTraining.BuildingLanguageModel#ntoc31 On 30

Re: [Moses-support] Delvin et al 2014

2014-11-26 Thread Nikolay Bogoychev
the size of the vocabulary you reference in #4 below (i.e. 16K, 500K, >etc)? >7. Re --source-context & --target-context, are these the BilingualLM >equivalents to a typical LM's order or ngrams for each? >8. Re --tagged-corpus, is this for POS factored corpora? > &

Re: [Moses-support] Delvin et al 2014

2014-11-26 Thread Nikolay Bogoychev
A files, larger or smaller? Also, are they binarized with mmap reads or > do they have to load into RAM? > > Thanks, > Tom > > > > > > On 11/26/2014 08:04 PM, Nikolay Bogoychev wrote: > > Fix formatting... > > Hey, > > BilingualLM is implemented and as of

Re: [Moses-support] Delvin et al 2014

2014-11-26 Thread Nikolay Bogoychev
y used to prepare the neural network language model. target_ngrams doesn't include the predicted word (so target_ngrams = 4, would mean 1 word predicted and 4 target context word) The total of the model would target_ngrams + source_ngrams + 1) I will write a proper documentation in the follo

Re: [Moses-support] Delvin et al 2014

2014-11-26 Thread Nikolay Bogoychev
Hey, BilingualLM is implemented and as of last week resides within moses master: https://github.com/moses-smt/mosesdecoder/blob/master/moses/LM/BilingualLM.cpp To compile it you need a NeuralNetwork backend for it. Currently there are two supported: Oxlm and Nplm. Adding a new backend is relative

Re: [Moses-support] Moses profiling

2014-09-15 Thread Nikolay Bogoychev
If you want to use google-perftools for profiling http://code.google.com/p/gperftools/wiki/GooglePerformanceTools compile moses with: ./bjam --full-tcmalloc link=shared On Sat, Sep 13, 2014 at 9:54 PM, Arturo Argueta wrote: > Is there any way to enable profiling on moses? I've heard that one > m